The VPN Consortium issues logos to products of member companies that have passed the interoperability and conformance tests described here. The interoperability logos indicate that a product interoperates with the other products in the test, and the conformance logos indicate VPNC's belief that a product conforms to various parts of the IPsec standards. Each category label under the logo indicates a test that the product has passed.
The VPNC test program issued its first logos in July, 2000. The tests run so far are "Basic Interoperability", "Basic Conformance", "Rekeying Conformance", and "Certificates Conformance". VPNC intends to add additional conformance and interoperability tests for important IPsec features in the future. In order to receive any of the logos, all products must pass the Basic Conformance test.
VPNC test logos are for a particular product line, not for an entire member company. Some companies have many product lines, and each line must be tested for its own logo. Because most VPN companies use essentially identical hardware and software for a whole line of products (with different amount of hardware included in higher-end systems), a logo can cover an entire line of products. VPNC members pay a one-time fee of $1000 for the Basic Conformance logo, but do not pay for any additional category logos.
The Basic Interoperability Test assures VPN users that IPsec systems are generally interoperable with other IPsec systems. To pass, a system has to interoperate with at least three quarters of the other systems that are in the test.
Interoperability is defined as creating a working IKE tunnel between the systems that normal IP traffic can flow through. The tunnel requires TripleDES for encryption, SHA-1 for hash, 1024-bit key exchange, and a preshared secret for authentication. As the term "Basic" implies, every IPsec implementation shipped today should have these features and should be able to interoperate with other IPsec systems.
Each system was set up based on the VPNC documentation profile for the system. Having the test follow the documentation profiles instead of setting up the systems based on the systems' documentation assures that end users can easily achieve interoperability in the same way that VPNC did.
The products from VPNC members that have passed the Basic Interoperability test are:
Full details of the Basic Interoperability test include the technical specification of the steps needed to pass, as well as the trace logs showing that IKE tunnels were set up in both directions.
The Basic Conformance test consists of the member's product initiating an IPsec ESP tunnel to each of the two test gateways. The tunnel requires TripleDES for encryption, SHA-1 for hash, 1024-bit key exchange, and a preshared secret for authentication. As the term "Basic" implies, every IPsec implementation shipped today should have these features and should conform to the IPsec standards with these options.
The products from VPNC members that have passed the Basic Conformance test are:
Full details of the Basic Conformance test include the technical specification of the steps needed to pass, as well as the debugging logs of the tests themselves.
The Rekeying Conformance test consists of setting up the same type of IPsec tunnel as is required for the Basic Conformance test, and then automatically rekeying the Phase 2 SA when it is needed. The Phase 2 SAs must also use perfect forward secrecy (usually called "PFS"). The tester must access a web server behind the test gateway before the rekeying and after the rekeying. As with the Basic test, this must be done on two test gateways.
The products from VPNC members that have passed the Rekeying Conformance test are:
Full details of the Rekeying Conformance test include the technical specification of the steps needed to pass, as well as the debugging logs of the tests themselves.
The Certificates Conformance test consists of setting up the same type of IPsec tunnel as is required for the Basic Conformance test, but using PKIX certificates instead of a pre-shared key for identification. The certificate is checked against the VPNC root certificate and the identity used is also checked. The tester must access a web server behind the test gateway. As with the Basic test, this must be done on two test gateways.
The products from VPNC members that have passed the Certificate Conformance test are:
Full details of the Certificate Conformance test include the technical specification of the steps needed to pass, as well as the debugging logs of the tests themselves.
It is important to note that the conformance tests do not test for interoperability. The conformance logos indicate that the product that got the logo was tested against two different servers, and it passed the test on each server. Thus, the logo indicates that the product interoperates with the servers against which they were tested, but not necessarily with other products that have the same logo.
Although knowing which products conform to the IPsec standards is important, end users need to know which products interoperate in order to make buying decisions. That is the reason that VPNC also has Interoperability logos. As the IPsec industry has found, however, defining and testing interoperability is incredibly tricky. In the past few years, VPNC members have found problems such as:
The Interoperability tests listed above cannot deal with all possible interoperability scenarios. Instead, they focus on the most common real-world scenarios and show how users can recreate the interoperability themselves.
The VPNC conformance test is based on open-source IPsec systems. The reason for testing against open source systems instead of commercial products is to give VPNC members and the people evaluating the conformance logos the ability to see exactly how the tests were done. The two systems used to test for conformance are OpenBSD and KAME. VPNC works with the developers of both of these systems to fix bugs found during the VPNC conformance program and to ensure that the test systems in fact meet the standards themselves.
If you have comments or questions about VPNC's testing, please feel free to send them to Paul Hoffman, VPNC's director, at paul.hoffman@vpnc.org.
