Today we pushed out a round of updates to the Netalyzr codebase. Most importantly, we dropped the HTTP content test that downloads the EICAR test virus. Even though this file is completely harmless, some anti-virus systems take it just as seriously as real malware. When detecting the file, these systems cut off all of Netalyzr's connectivity, causing a failure of the test session to complete. Many thanks to our users for alerting us to this problem—it's a great example of the unexpected things we encounter during the tests.
In terms of new features, we have added initial testing of DNS root server behavior. Netalyzr now checks whether it can actually reach all DNS root servers and whether querying them works as intended. We have also beefed up our regression testing and as a consequence fixed a number of result rendering glitches in the test reports.
A few weeks ago the Netalyzr session counter crossed the 400,000 mark. We would like to take this opportunity for a sincere Thank You to all of our users for continuing to run Netalyzr! We always welcome your feedback and suggestions at netalyzr-help@icsi.berkeley.edu.
Yesterday around 4pm PDT the downtown Berkeley area, including ICSI, suffered an unannounced power outage. During the 6 following hours that the local power utility took to restore power, the Netalyzr site remained unavailable. We apologize for the inconvenience.
Around 4am PST today, ICSI suffered an unannounced network connectivity outage that lasted approximately four hours. We apologize for the inconvenience.
As we previously reported in cooperation with the EFF and New Scientist, Netalyzr's results identified multiple US ISPs that appear to monetize their users' web searches using affiliate marketing programs by redirecting some of their users' web searches using services provided by a company called Paxfire.
At this week's FOCI Workshop we presented a paper that describes the DNS error traffic monetization business that companies such as Paxfire engage in, and also in part describe our measurements of search redirections. We'd like to take this opportunity to provide an update on the redirections we observe.
Starting on July 26th, we began to identify web search keywords that triggered redirections. The redirections take place in two stages, a first one using DNS to send the user's HTTP request to a Paxfire-controlled proxy, and the second by the proxy not relaying the requests to the intended search engines but instead returning HTTP redirects through the affiliate programs involved. Using popular domain names provided by Alexa, we identified 165 such keywords. Given interest in the set of keywords from multiple parties, we now make the keyword list available. If you have additional questions regarding our measurements or dataset, please contact us by email at netalyzr-help@icsi.berkeley.edu and we will see if we are able to accommodate your request.
On August 5th, 24 hours after our public disclosure, Paxfire limited or halted the redirections through affiliate programs. We currently no longer observe any HTTP redirections through affiliate programs, suggesting that Paxfire discontinued the practice. However, the following ISPs still appear to redirect some or all traffic destined to Yahoo's and Bing's search engines through Paxfire's proxies:
It furthermore appears as though Hughes (DirecPC) has stopped proxying search requests through Paxfire. For the remaining ISPs we do not possess sufficient data.
We currently do not include explicit testing of Paxfire's keyword-based HTTP redirections in Netalyzr. Until we do so, we would appreciate if tech-savvy customers of the involved ISPs could contact us by email at netalyzr-help@icsi.berkeley.edu.