Home      What is a Trojan Horse?      How we reviewed anti-trojans     About us

The aim here to get a measure of how thoroughly the vendors of anti-trojan programs update their signature files. This is of critical importance; the effectiveness of any anti-trojan product is, to a considerable degree, only as good as the quality of its database.

Detection Tests for New Trojans

All programs were tested against a test set of newly released trojans downloaded from hacker sites. The trojans were selected on the basis of download popularity.  The list included the latest versions of of some of the most potent trojans including Subseven, Bionet and Optix.  The test set also included some trojan components rather than complete products. For example AVKillar is a component designed to pull down anti-virus, anti-trojan and firewall defenses prior to the execution of the trojan server itself.

All the anti-trojan products were tested twice. The first time was  approximately one month after the trojan test set was collected. The second test was conducted  approximately three months after collection. All products were tested on the same day and the signature files used were the latest available as at that time.

The procedure used for both the 2002 and 2003 series was essentially the same though in 2003 series we carried out scanner tests alone rather than scanner and monitor tests. This change reflects the fact that modern trojans now routinely pull down monitor defenses, so scanning has taken on an increased performance.

On each occasion the test set and test conditions were the same, apart from the signature file. The test set was left uncompressed and as distributed.  In our 2003 series, we refined the method by manually building custom servers for each of the trojans using the tools supplied in the distributed product. 

For each anti-trojan program, the test set was scanned using the most comprehensive scanning options available for each product.

BoClean has no scanner component so all results were obtained by actually executing the trojans servers or in the case of AVKillar, the program itself.  The 2003 tests involved scanning only and the results shown for BoClean are based on whether the exact version of each trojan was listed in the BoClean database.

After each anti-trojan program was tested, the hard disk was restored from a mirrored copy. This ensured that all programs were evaluated in identical conditions. 

Detection Test for New Trojans -  Results 2002

Detection Test for New Trojans -  Results 2003

Key:   

Need cheap inkjet cartridges? Check out http://www.techsupportalert.com/cheap_inket_cartridges.htm

Conclusions

Let me repeat that not too much importance should be placed on the fact that certain products missed certain trojans as the test set is too small to draw conclusions about detection effectiveness. Besides, these are scanner tests only. The monitor component of the anti-trojan products could have detected the trojans missed by the scanner when the trojans were actually executed.

The best performing product in both years was TDS-3 which achieved a perfect score in 2002 and near perfect in 2003. Most notable is the fact that the 1 month and 3 month results are the same which suggests that the folk at TDS-3 are very quick to add new trojan products to its database.

Trojan Hunter was a notable improver between 2002 and 2003.  This improvement can be correlated with the huge increase in the size of its rules database in the same period. Trojan Hunter's three month test result was particularly impressive and was a close match for TDS-3 while the one month was close but not quite as good.  It would seem Trojan Hunter has really lifted its game in the database maintenance department but still needs more effort to get new trojans speedily into the database if it is to catch TDS-3.

PestPatrol trailed the field in 2002 but showed remarkable improvement in 2003. Full credit to the developer for putting the necessary resources into this area.

BoClean also put in a top performance, though the fact that it was tested under different conditions to the other products raises questions of comparability.

The Cleaner and Tauscan scored about the same as each other in 2002 however in 2003, The Cleaner's performance deteriorated markedly.  We'll be keeping a watching brief on The Cleaner in the coming months and if there is no improvement it will be dropped from our recommended list.

Advertisements:

The Best Backup Software: 18 backup programs reviewed and rated but only one get "Editor's Choice"

Inkjet Printer cartridges: The best places to buy cheap inkjet cartridges. We looked at 47 seven sites but could only recommend eleven.

Anti-trojan Software Reviews Home Page