OEMs in the telecom industry invest in verifying the performance and conformance of the high-speed interconnects off their gear. Does the same approach apply to chip-to-chip interconnects, and in other industries?
Firstly, let’s define some terms. Within the router business, as it relates to high-speed I/O, performance testing relates to the raw performance of, for example, an Ethernet port carrying IP traffic. Test equipment often dubbed unceremoniously as “packet blasters” is used to measure packet throughput, latency and jitter. This test equipment loads up a port with packet floods, approaching full line rate (for example, 100Gbps for current Ethernet testers). Then measurements are taken for throughput (how many packets are dropped as the router struggles to keep up with the load), latency (how long it takes packets to transit through the router – a measure of its efficiency at packet processing), and jitter (the time-deterministic behavior of packet forwarding). For performance testing of routers, the Internet Engineering Task Force (IETF) has published specifications, termed “Request for Comments (RFCs)”, that detail how this is measured. An example RFC can be seen here: http://www.ietf.org/rfc/rfc2544.
Of course, the concept of performance testing can be generalized to numerous performance criteria. A good way to consider this is to look at the OSI model, which I wrote about in an earlier blog: https://blog.asset-intertech.com/test_data_out/2011/02/the-test-stack.html. Performance testing at any level of the OSI stack, from physical to application layers, is possible.
Conformance testing, on the other hand, is often used to determine whether a product or system meets a specified standard for interoperability. This kind of testing is sometimes done with an external agency (sometimes the standards body itself) to warrant compliancy. Conformance testing frequently involves “negative testing”: what does the system do in simulated error conditions. A great example of how performance and conformance testing is applied to routers can be seen on Ixia’s website: http://www.ixiacom.com/solutions/testing_routing_and_switching/products/index.php.
So, you may ask, how does this apply to our industry, that of board test? Well, as it turns out, the same concepts fit. Take a PCI Express chip-to-chip interconnect for example. The PCI Special Interest Group (PCI-SIG) holds regular compliance workshops (sometimes termed “plugfests”) that are widely attended by OEMs building PCIe devices and cards. Passing the test at these forums allows usage of the “PCI-SIG COMPLIANT” designation and logo. There are numerous other examples for other protocols and industry forums.
To ensure utmost quality, both performance and conformance testing should be done on products. Doing one and not the other is not a good option. Using PCIe again as an example, if you do physical layer performance testing but not conformance testing, the bus might run cleanly at full line rate but not be compliant to the PCI-SIG specification, and thus not interoperate properly. Alternatively, if conformance was tested successfully but not performance, the bus might work properly with other vendors’ equipment, but the product might not perform optimally (for an example of this, check out my last blog: https://blog.asset-intertech.com/test_data_out/2012/01/your-laptop-are-you-getting-what-you-paid-for.html).
There are, of course, test suites which verify both conformance and performance. We’ll examine these in a later blog.
