You've probably read about this in other blogs already. At the risk of sounding like a broken record I'll post it here as well as this is really important and I think we should all help spread the word as much as possible. As you may know AMTSO is a non-profit organization made up of a lot of companies from the industry, from independent tests (such as AV-Test, AV-Comparatives, CascadiaLabs, Dennis Technology Lab, ICSA, NSS, PC Security Labs, and West Coast Labs) to antivirus vendors and academia. Visit AMTSO website to view the full member list.
Last month we attended the 5th Anti-Malware Testing Standards Organization (AMTSO) meeting held in Budapest and hosted by VirusBuster. This follows a bunch of other meetings held in Bilbao (Panda), The Netherlands (Norman), Oxford (Sophos) and Cupertino (Symantec). You can read the AMTSO Press Release titled AMTSO to start analysis of Anti-Malware Reviews for the official details.
Most of the work went into validating in a face to face meeting the different documented methodologies and processes which we've all been working on over the last few months. In all, AMTSO has now published a respectable document library about different issues concerning Anti-Malware Testing, and the list keeps on growing.
- AMTSO Fundamental Principles of Testing. A high level overview which covers the 9 principle guidelines to follow while testing anti-malware products.
- AMTSO Best Practices for Dynamic Testing. Probably the first document AMTSO started working on the early days of its foundation. Covers the main issues while running dynamic tests (versus static tests which consist of on-demand scans of many samples).
- AMTSO Best Practices for Validation of Samples. One of the most important and most often overlooked issues of anti-malware testing. How to select valid samples for testing.
- AMTSO Best Practices for Testing In-the-Cloud Security Products. Specially important for products which incorporate this latest method of protection. We were specially interested in this document as you can imagine as some of our latest products such as Panda Cloud Antivirus and Panda 2010 products include cloud-scanning.
- AMTSO Analysis of Reviews Process. Viewed as one of the most important tasks of AMTSO, this document provides insight into the process that AMTSO will follow to review, based on the principles and methodologies published, the different Anti-Malware Tests that are published out there. This process is completely transparent and open to the publlic, so anybody can request a "Review Analysis" of a published test.
One of the most interesting things during these AMTSO meetings is the of openness & sharing of information between what are normally fierce competitors. It's not a very common practice to link to "competitors" sites (and I'm sure I'll get in trouble for it when/if my boss sees this), but I do recommend that you read up some of our colleagues blog posts about AMTSO progress, such as the ones from Sophos, Norman, McAfee, Trend, Avira, PC Tools, Kaspersky, ESET, and last but not least VirusBuster who hosted the event (sorry if I left someone out).