6

I've been having this issue for quite some time, where websites viewed through webkit-based browsers inconsistently load images. By inconsistent, I mean that on one trial run, an image, or several images will load successfully, only to have others that will not. On another trial run of that very same website, the images that didn't load previously will suddenly load --only to have the ones that previously did load, suddenly not load. This behavior is so non-linear that I am having extreme difficulties finding out the source of the issue. I notice that this problem is replicable with browsers such as jumanji, dwb, and vimperator. I believe the common factor amongst all of these browsers is that they use webkit. Repeatedly reloading the webpage will sometimes produce a result where all resources are loaded correctly.

Here is a screenshot of described behavior (from the webkit-based luakit): enter image description here

As you can see these are two failed images, that illustrate common behavior here. I cannot replicate this issue with browsers such as firefox or chrome (which I believe use gecko and blink respectively). If I right-click on the image/element and open it in a new window, I am able to view the image with no problems. I am running Arch Linux kernel 3.12.9-1-ck. Any help/insight as to what might be happening would be much appreciated. Thank you.

UPDATE: Every broken image, when inspected as an element by debugging console in luakit outputs something of this general form:

GET [web address here] Cannot resolve hostname [domain here]

UPDATE 2: I tried installing luakit on a virtualbox installation of kali-linux that I have on my system (debian-based) via apt-get install luakit, and interesting result ... No symptoms of unresolved hostnames/broken images/failed resources. Browsing is also comparatively faster in this virtual environment.

Solution:

Following the suggestion proposed by @harrymc (using Google public DNS) has completely destroyed all symptoms of poor page loading. According to @harrymc, it is due to faulty/slow DNS, and/or poor DNS caching strategies. More specifically, what caused this problem was a poor DNS, and what appears to be a rather hasty time-out protocol built into the webkit engine. These two factors are a recipe for disaster.

A More Open-Ended Thought-Arc:

One other conclusion is the inefficiency of Webkit browsers in that they issue multiple DNS queries for the same website, rather than remembering the first query. Another conclusion is that the ISP's DNS server apparently sometimes cannot handle multiple parallel requests (since the browser probably handles multiple images in parallel via threads), perhaps because they now have more clients but not enough DNS servers. --harrymc

eazar001
  • 33
  • 9
  • I'm running Arch Linux with the same kernel, but both `luakit` and `uzbl` (both Webkit-based) work without issue. Are you running an ad-blocking script that could cause that? – Anko - inactive in protest Jan 30 '14 at 09:21
  • @Anko, I have not installed an adblocker script for `luakit`, nor for `dwb`. Keep in mind: the same images that won't load as in the picture above will show up later if I reload (after an indefinite amount of reloads). – eazar001 Jan 30 '14 at 09:25
  • My impression of the matter was basically that, if it was a firewall-related or ad-blocker type of situation, then I would see the same images/scripts/resources being blocked over and over again. However, this problem is so nebulous to me, that I would not be surprised by the proposal of any culprit. Side-effects can be triggered by the most unlikely catalysts sometimes .... – eazar001 Jan 30 '14 at 09:53
  • I'm also mystified. The error implies a networking issue, but other browsers' working correctly rules that out. Just to check, you could [run some manual DNS queries](https://library.linode.com/linux-tools/common-commands/dig) for the problematic hostnames using `dig` from the `dnsutils` package. – Anko - inactive in protest Jan 30 '14 at 11:30
  • The most common cause for images that do exist to not load is exceeded wait-time. I don't use any of your browsers, but you could search for a wait-time configuration parameter, if one exists. The difference with Firefox or Chrome might be that their wait-parameter is larger. – harrymc Jan 30 '14 at 14:12
  • @Anko, I'll definitely give it a shot when time frees up, thanks for the advice, I'll apprise you of the results when i can. – eazar001 Jan 30 '14 at 15:29
  • @harrymc, TBH, I actually had this thought as well ... I noticed that load times seemed longer than non-webkit based browsers. Unfortunately, I do not think there is a parameter I can pass to any of these browser executables, although I will look for webkit based alternatives to see if any do offer the option. – eazar001 Jan 30 '14 at 15:33
  • 1
    @harrymc, the most disturbing thing about that thought however is: I have an 18 Mbps connection throughput, and I get great average case performance on healthy websites. So why is webkit seemingly under-performing to the point where I am getting these "hypothesized" time-outs? This is, assuming your suggestion is true. The implication is maddening. – eazar001 Jan 30 '14 at 15:39

1 Answers1

3

From Webkit timeout kills long running tasks :

We have just been forced to refactor/recode a significant portion of one of our AIR based RIA's due to an arbitrary decision made by the Webkit team to restrict all XML HTTP requests via a hard coded, hidden timeout of 60 seconds. This decision not only affects AIR but also affects Safari and other browsers based upon Webkit.

Although this doesn't necessarily pertain to your problem, it does point to the existence of a hardcoded timeout in Webkit.

If your problem is related to timeouts in Webkit being too short, the question is then why are you experiencing long waits for images, given that you have a fast connection.

As a first test I suggest changing your DNS server to Google Public DNS or OpenDNS, and see if this makes a difference. If it does, than the problem is with your ISP being too slow on DNS or on using its own cache.

Another reference at disabling HTTP keepalive by User-Agent :

A long-standing bug in Safari causes file uploads to hang when keepalive connections are improperly reused.

https://bugs.webkit.org/show_bug.cgi?id=5760

In Apache, disabling keepalive support for Webkit solves this problem.

If the Apache web-server still disables keepalive for Webkit (HTTP persistent connection), this means that each image requires a separate HTTP connection, while Firefox and Chrome can use the already existing connection of the page to also download the images without re-connecting.

As establishing a connection normally is quite slow, then this coupled with a short built-in timeout, may explain the problem that Webkit has with images.

I wonder if your Webkit browsers have the ability to change their User agent identity ?

For example, while knowing absolutely nothing at all about vimperator, I found via google the plugin UserAgentSwitcherLite.

harrymc
  • 455,459
  • 31
  • 526
  • 924
  • Thanks you for your insight, relevant links, and suggestions thus far. I will give some of your suggestions a deeper look through ASAP. However, I can assure you I have given the `User agent` suggestion a shot before; as a matter of I changed it to mozilla 5.0. Either way, I will surely get back to you ASAP on this. Thanks again. – eazar001 Jan 31 '14 at 02:40
  • I tried altering `dwb` user-agent one more time to a `mozilla-gecko` string, and still -- to no avail. I'll give a DNS server switch a test as well to see if that can unveil any clues. – eazar001 Jan 31 '14 at 04:35
  • How strange, using google's public DNS server resolved my issues. I just don't understand, I have been using the same ISP across the state of California with great throughput, however this entire time my DNS was responsible for all of this? So your theory is DNS caching or something like that? Either way ... you have earned yourself a bounty, and I have spent all 100 points of my rep very well [=. Thank you for help @harrymc. – eazar001 Jan 31 '14 at 08:55
  • The problem is therefore proven to be with your ISP. I suggest getting in touch with their Support and ask why you are getting random long delays on DNS queries. You have now all the necessary proofs. At worst you can just stay with google's DNS. – harrymc Jan 31 '14 at 09:55
  • 2
    One other conclusion is the inefficiency of Webkit browsers in that they issue multiple DNS queries for the same website, rather than remembering the first query. Another conclusion is that the ISP's DNS server apparently sometimes cannot handle multiple parallel requests (since the browser probably handles multiple images in parallel via threads), perhaps because they now have more clients but not enough DNS servers. – harrymc Jan 31 '14 at 10:32