Comparative tests

The test was made on 10 August-05 September 2009, using Windows XP Professional SP3 on a Pentium Dual Core 2Ghz, 2048MB DDRAM-2.

All programs tested had the latest versions, upgrades and updates and they were tested using their full scanning capabilities e.g. heuristics, full scan etc.

The default settings of each program were not used, in order for each program to achieve its maximum detection rate. Because of this, there is a possibility for the tested programs to detect a few false positives.

All programs were updated on August 10th 2009, between 03.00AM and 07.00AM GMT.

The 562086 virus samples were chosen using Kaspersky, F-Prot, Nod32, Dr.Web, BitDefender and McAfee antivirus programs’ reports. Each virus sample was unique by virus name, meaning that AT LEAST 1 antivirus program detected it as a new virus.

MS-DOS based virus samples were not used.

ALL virus samples were unpacked and the only samples that were kept were the ones that were packed using external-dos-packers (that means not winzip, winrar, winace etc).

The virus samples had the correct file extension using a special program (Renexts) and were unique, according to checksum32 filesize.

Most "fake" virus samples were removed, as well as "garbage" files.

The programs MKS_VIR , PER and IPArmor were not tested because there were no english demo versions available.

The program Extendia AVK was not tested because there was no demo version available.

Thorough mode was not used in VBA32 due to extremely slow scan process and heuristics were set to medium.

The Cleaner’s heuristics were set to medium due to many false positives.

The test was made on 09-10 April 2009, using Windows XP Professional SP3 on a P4 3200 Mhz, 2048MB DDRAM.

All programs tested had the latest versions, upgrades and updates and they were tested using their full real-time monitoring capabilities.

The default settings of each program were not used, in order for each program to achieve its maximum detection rate.

All programs were updated on 09 April 2009.

Specific HIPS (Host-based Intrusion Detection System) tests were used from the website www.testmypcsecurity.com. The test shows the detection rate of the antispyware software against changes in the operating system and not their detection rate against spyware software. Nevertheless, the antispyware software with high HIPS detection have better chances to detect new spyware software.

Rank

1. Drive Sentry version 3.3.0.4 - 84.00%

-.PCTools ThreatFire version 4.1.0.25 - 84.00%

3. Ashampoo version 2.05 - 80.00%

-.Paretologic Antispyware version 5.7 - 80.00%

5. Spy Emergency 2009 version 6.0.205 - 76.00%

6. Sunbelt CounterSpy version 3.1.2416 - 72.00%

7. Spyware Doctor Starter Edition version 6.0.1.440 - 68.00%

8. Spyware Terminator version 2.5.6.316 - 64.00%

9. RegDefend version 2.050 - 48.00%

10. Comodo BOClean version 4.27 - 28.00%

11. Spybot Search & Destroy version 1.6.2 - 24.00%

-.SpyHunter version 3.10.27 - 24.00%

13. Spy Cleaner Platinum version 3.7 - 16.00%

-.StopZilla version 5.0.0 - 16.00%

15. SpywareGuard version 2.2.0 - 8.00%

-.SuperAntispyware version 4.26.1000 - 8.00%

-.SpyRemover Pro version 3.0.4 - 8.00%

18. Ad-Aware Anniversary Edition version 8.0.3 - 4.00%

19. ScanSpyware version 3.9 - 0.00%

-.BPS Spyware & Adware Remover version 9.4.0.8 - 0.00%

-.Malwarebytes Antimalware version 1.36 - 0.00%

-.SpywareBlaster version 4.2 - 0.00%

DETAILED TEST RESULTS (.rar compressed file)

Introduction - What makes our test differ from other tests

Accusations/comments on Virus.gr tests

2002 To 2006 - Comparative Tests

2007-04, 23 April-10 May

2008-06, 1-21 June

2009-04, 09-10 April (HIPS - Antispyware)

2009-08, 10 August-05 September (NEW!!!)

2002-05, 10-12 May

2002-11, 5-10 November

2003-05, 4-12 May

2003-10, 2-12 October

2004-08, 10-25 August

2005-04, 2-16 April

2005-12, 14-22 December

2006-03, 29-30 March

2006-08, 15-25 August

The test was made on 01-21 June 2008, using Windows XP Professional SP2 on a P4 3200 Mhz, 2048MB DDRAM.

All programs tested had the latest versions, upgrades and updates and they were tested using their full scanning capabilities e.g. heuristics, full scan etc.

The default settings of each program were not used, in order for each program to achieve its maximum detection rate. Because of this, there is a possibility for the tested programs to detect a few false positives.

All programs were updated on 31 May 2008, between 09.00AM and 12.00PM GMT.

The 246705 virus samples were chosen using VS2000 according to Kaspersky, F-Prot, Nod32, Dr.Web, BitDefender and McAfee antivirus programs. Each virus sample was unique by virus name, meaning that AT LEAST 1 antivirus program detected it as a new virus.

MS-DOS based virus samples were not used.

ALL virus samples were unpacked and the only samples that were kept were the ones that were packed using external-dos-packers (that means not winzip, winrar, winace etc).

The virus samples had the correct file extension using a special program (Renexts) and were unique, according to checksum32 filesize.

Most "fake" virus samples were removed, as well as "garbage" files.

The program PER was not tested because there was no english demo version available.

The programs Command and Extendia AVK were not tested because there was no demo version available.

Thorough mode was not used in VBA32 due to extremely slow scan process.

The program A-Squared Anti-Malware is anti-trojan/anti-spyware program, not antivirus program.

The program F-Prot was tested using its command line scanner (options fpscan j:\avtest\trobo /adware /applications /output=fpscan_report.log /streams /maxdepth=4 /heurlevel=4) because its GUI kept crashing.

The programs Kingsoft and ZondexGuard were not tested because they could not be updated.

The programs Windows Live OneCare , BKAV and MoonSecureAV kept crashing while scanning the samples.

The program AntiVir is now called Avira AntiVir.

The program Fire uses the exact same engine as Solo.

The program Vexira uses the exact same engine as VirusBuster.

The program BullGuard uses the exact same engine as BitDefender free edition.

The program Avast Professional uses the exact same engine as Avast free edition.

The program AVG Pro uses the exact same engine as AVG Antivirus free edition PLUS the rootkit detection.

The program A-squared Anti-Malware Professional uses the exact same engine as A-squared free edition.

InVircible did not include a "typical" scanner-function and could not be tested.

V-Catch checks only mail accounts and could not be tested.

DOS-Based scanners were not tested.

The following file types were used.

SH, ELF, COM, EXE, PL, BAT, PRC, DOC, XLS, BIN, MDB, IMG, PPT, VBS, MSG, VBA, OLE, HTM, INI, SMM, TD0, REG, CLASS, HTA, JS, VI_, URL, PHP, WMF, HLP, XML, SCR, PIF, SHS, WBT, CSC, MAC, DAT, CLS, STI, INF, HQX, XMI, SIT.

The virus samples were divided into these categories, according to the type of the virus :

File = BeOS, FreeBSD, Linux, Mac, Palm, OS2, Unix, BinaryImage, BAS viruses, MenuetOS.
Windows = Win.*.* viruses.
Macro = Macro, Multi and Formula viruses.
Malware = Adware, DoS, Constructors, Exploit, Flooders, Nukers, Sniffers, SpamTools, Spoofers, Virus Construction Tools, Droppers, PolyEngines, Rootkits, Packed.
Script = ABAP, BAT, Corel, HTML, Java, Scripts, MSH, VBS, WBS, Worms, PHP, Perl, Ruby, Python, WHS, TSQL, ASP, SAP, QNX, Matlab viruses.
Trojans-Backdoors = Trojan and Backdoor viruses.

Rank

1. G DATA 2008 version 18.2.7310.844 - 99.05%

2. F-Secure 2008 version 8.00.103 - 98.75%

3. TrustPort version 2.8.0.1835 - 98.06%

4. Kaspersky version 8.0.0.357 - 97.95%

5. eScan version 9.0.742.1 - 97.44%

6. The Shield 2008 - 97.43%

7. AntiVir version 8.1.00.331 Premium - 97.13%

8. Ashampoo version 1.61 - 97.09%

9. Ikarus version 1.0.82 - 96.05%

10. AntiVir version 8.1.00.295 Classic - 95.54%

11. AVG version 8.0.100 Free - 94.85%

12. BitDefender 2008 version 11.0.16 - 94.70%

13. Avast version 4.8.1201 Professional - 93.78%

14. Nod32 version 3.0.650.0 - 93.36%

15. F-Prot version 6.0.9.1 - 91.87%

16. BitDefender version 10 Free - 91.32%

17. ArcaVir 2008 - 88.65%

18. Norman version 5.92.08 - 87.72%

19. Vba32 version 3.12.6.6 - 87.21%

20. McAfee Enterpise version 8.5.0i - 86.57%

21. McAfee version 12.0.177 - 86.39%

22. Rising AV version 20.46.52 - 85.87%

23. Norton 2008 - 83.34%

24. Dr. Web version 4.44.5 - 82.87%

25. Antiy Ghostbusters version 5.2.3 - 80.23%

26. VirusBuster version 5.002.62 - 77.19%

27. Outpost version 6.0.2294.253.0490 - 75.35%

28. V3 Internet Security version 2008.05.31.00 - 75.23%

29. ViRobot Expert version 5.5 - 74.50%

30. Virus Chaser version 5.0a - 73.65%

31. A-squared Anti-Malware version 3.5 - 71.66%

32. PC Tools version 4.0.0.26 - 69.82%

33. Trend Micro Antivirus+Antispyware 2008 version 16.10.1079 - 67.28%

34. Iolo version 4.325 - 63.98%

34. Panda 2008 version 3.01.00 - 61.41%

36. Sophos Sweep version 7.3.2 - 54.71%

37. ClamWin version 0.93 - 54.68%

38. CA Anti-Virus version 9.00.170 - 51.08%

39. Quick Heal version 9.50 - 47.97%

40. Comodo version 2.0.17.58 - 43.15%

41. Trojan Hunter version 5.0.962 - 31.39%

42. Solo version 7.0 - 21.10%

43. Protector Plus version 8.0.C03 - 20.14%

44. PCClear version 1.0.8.0 - 19.63%

45. AntiTrojan Shield version 2.1.0.14 - 14.74%

46. Trojan Remover version 6.6.9 - 13.49%

47. VirIT version 6.2.94 - 8.63%

48. True Sword version 4.2 - 3.42%

49. Abacre έκδοση version 1.4 - 0.00%

DETAILED TEST RESULTS (.rar compressed file)

The antivirus tests of virus.gr have been accused too many times already during the last years and, after reading an article by Mr. Clementi recently (http://www.av-comparatives.org/forum/index.php?page=Thread&threadID=753 - Andreas says that they are not all coming from him or reflecting his opinion), i feel the need to say this:

a) Not all products are tested equally:

- some products are trial versions, some others are full versions

This is false, all software have FULL capabilities (unless mentioned by virus.gr) during the antivirus test

- some use best settings, some others use "not default settings" (whatever this means); this may be due the trial versions limitations

This is false, all software have the best scanning options to achieve maximum detection

- not all products are updated at the same time/day/week (but this seems now to be solved in the last tests)

This is false, all products are updated at the same day with a few hours difference; read the notes of the latest test

b) Samples:

- samples were choosen by using those products: Kaspersky, F-Prot, Nod32, Dr.Web, BitDefender and McAfee; requirement: unique name. This leads to a major flaw toward KAV and products which use many unique names instead of generic names.

This is false, the test-bed is chosen using more than one AV so is it impossible for the test results to be in favor of one AV

- sample source are mainly other VXer: as VXer use mainly KAV to share their samples, the tests and samples are flawed in favor of KAV.

This is false, the test-bed is chosen using more than one AV so is it impossible for the test results to be in favor of one AV

- there is a huge amount of garbage included in the test-set, even if he says he removed "most" of it.

Most of the samples are gathered from in-the-wild sources and are in fact tested one-by-one, so they are basically among the samples sent by emails to common users or downloaded by “suspicious” sites that promote warez or other illegal software/multimedia; samples are not selected but collected. Nevertheless, IF this accusation was true, it would also make other comparative tests (which also make false positive tests as well as on-demand tests) innacurate and untrustworthy, since there are MANY AV software besides Kaspersky that detect ALMOST all of this "huge garbage collection" as viruses; Now, wouldn't this make all these software also untrustworthy, since they detect so many thousands of "garbage files" as viruses?... Think about it...

- at least some vendors got from them their whole collection in past, as some vendors were able to determine how much garbage is in it (even if some users in forums say that virus.gr do not sends any sample to av vendors)

This is completely false. No av vendor has got the collection owned by virus.gr (at least I am unaware of such a thing). So, I do not see how this was achieved, if so.

- the test-set contains lot of DOS files

The DOS (MS-DOS) files will be excluded in future test, this was my intention since the last test anyway.

c) Miscellanous:

- the Virus.gr site is not a team of people, it is atm 1 person. The tester behind it is a viruscollector (VXer) with very limited programming skills or coding skills in general (which means he can not check for garbage in the test-set, etc.).

This is true, but fact is that virus.gr is assisted by other people when it comes to testing samples for av tests.

The guy seems to have genuine intentions and tries to give his best while doing his tests, but the tests need still improvements and better methods. In any case, the results may be interesting to look at, as they include also some minor AV products which are usually not tested.

This is also true, I am not a professional but this does not mean that the results of the test do not reflect the detection ratio of the antivirus software tested.

Either way, I believe that each test has a different methodology and it is pointless to say that one test is correct and another one isn’t. So, I would suggest that if you do not agree with the methodology used by this site, just visit other sites that also perform antivirus tests. We do not sell neither promote antivirus software; this is not the case for virus.gr.

But if you DO have some comments on virus.gr tests and disagree on the test results, “be a gentleman” and post your question on our forum section instead of making accusations without evidence or without letting us state OUR POINT OF VIEW. This is NOT ETHICAL.