Some Limitations of Trojan Scan Tests

Home      What is a Trojan Horse?      How we reviewed anti-trojans     About us

Most reviews of anti-trojan products simply involve scanning large trojan data sets with each product under review and then ranking the results according to the number of trojans detected.  We consider the approach has too many weaknesses and limitations to be reliable.  Here are our reasons:

(1) The use of simple numerical tallies of trojans detected during scanning, assumes that all trojans are equal. That is, one trojan detected has the same importance as any other.  This is far from the truth:

  • some trojans are rarely encountered, while others are prevalent. Detection of what's "in the wild" today is much more important than detecting some long defunct program
  • some trojans are more far dangerous than others. The detection of these is more important than the detection of relatively harmless products
  • some trojans, like polymorphic trojans, are more difficult to detect. A product that detects these is performing better than one that detects simple trojans

(2) Scan tests just test out the scanner component of anti-trojan products.  They don't take into account the memory resident monitor component even though this plays an important role in the product's trojan detection capabilities.

(3) An anti-trojan  product may detect an uncompressed trojan yet fail to find it inside an archive or packed into a compressible executable. Ideally scan tests should look at trojans stored in various forms. They rarely do.

(4) Additionally there is the question of the test set of trojans used.  Different reviews use different sets but which is the best set? Furthermore even if there is a "best" set, will that be the best set at a different point of time?

(5) Finally there is the question of possible bias. It is quite possible to construct a test set that will favor individual products, a fact that needs to be noted when looking at vendor tests or vendor funded tests.

Several vendors feature on their sites their own tests where they have tested their products and others on a test set consisting of every trojan in the vendor's signature database. Not surprisingly, their own product scores 100% detection!

Our advice is to avoid choosing products just on scan test results alone.  Yes, take them into account but also consider:

  • sound design
  • suitability to your needs
  • regular and well researched signature file updates
  • good vendor technical support

 


Advertisements:

The Best Backup Software: 18 backup programs reviewed and rated but only one get "Editor's Choice"

Inkjet Printer cartridges: The best places to buy cheap inkjet cartridges. We looked at 47 seven sites but could only recommend eleven.


 

return