How does a web page harm thee? Let me count the ways....

As the web grows and diversifies, so do the number of ways in which web pages can harm users (for example). To help counter this threat, Google, Norton, and McAfee maintain "blacklists" that warn users away from potentially unsafe webpages. But what exactly does it mean to get blacklisted by these services? Do the lists agree on which sites are harmful?

When analyzing a blacklist, the primary fact to keep in mind is that the term "bad" is not well defined. As such, each list operates under different criteria. Google, for example, focuses on technical threats such as drive-by downloads. McAfee and Norton take a broader view, and will flag a site based on things like "annoyance factors" or "excessive popups." McAfee also signs up for any mailing lists it finds and records the amount of spam generated. Google limits itself to a binary response marking a site as potentially harmful (e.g., "This site may harm your computer") or not, while McAfee and Norton label websites as "Safe," "Caution," "Warning," or "Untested."

Given these divergent criteria, the first thing that is immediately apparent is that it would be much too simplistic to call all blacklisted sites "malicious," while calling all non-blacklisted sites "safe." To see how similar or different these blacklists are in practice, we took a set of relatively popular domains on the Internet and queried all three lists.

Of the sites that were flagged by at least one of the blacklists:

• 57% were marked as potentially harmful or "Warning," with the other 43% marked as "Caution."


• Google flagged 5.2% with a "This site may harm your computer" label.

• Norton flagged 16.3% of the sites with a rating of "Warning," and another 14.4% with a rating of "Caution."


• McAfee flagged 38.6% of the sites with a rating of "Warning," and another 32% with a rating of "Caution."

Per the statistics above, Google's list was by far the smallest, reflecting its focus on technical threats. Norton had far more "untested" sites than McAfee, partially explaining Norton's lower numbers.

When we compare which sites were blacklisted, however, the results become far more interesting. Of the sites that Google blacklisted:

• Norton labeled less than half with "Warning," half with "Safe," and the remaining ones as "Untested." None were labled with "Caution."


• McAfee labeled a quarter with "Warning," a quarter were "Untested," and the remaining half were safe. None were labled with "Caution."

• McAfee's users lodged complaints about more than half of the sites.

Norton and McAfee's blacklists also didn't agree with each other often. Of the sites flagged between them, only 4% were on both lists. Amazingly, the overlap between all three lists was less than 1%.

Of the sites flagged "Warning" or "Harmful" by at least one list, 61%
were flagged only by McAfee. Only 1% were flagged by all three lists.

These discrepancies shouldn't be surprising, given the fact that the lists employ different criteria and techniques for evaluating sites. But there are also deeper reasons for the lack of overlap: For one thing, the frequency and timing of testing can have a significant impact on the rating a site receives -- if a site is compromised after it's tested by a service, and that service doesn't test it again for another day, week, or even month, that site could end up infecting a significant number of its users while still being marked as "safe" by the service. For another, the diagnostics employed by these services aren't necessarily infallible -- web-based malware is sometimes masked to prevent its detection by some testing services.

So what does all this mean for protecting your business? How should you deal with the fact that there are so many blacklists out there, testing your site on several different criteria and with varying levels of effectiveness? At Dasient, we believe the answer is being proactive about monitoring and protecting your site from web-based malware. To learn how we can help you do that, click http://wam.dasient.com.