Google says its new set of crypto tests has already discovered real bugs.

Google's Project Wycheproof is a new effort by Google to improve the security of widely used cryptography code.

Many of the algorithms used in cryptography for encryption, decryption, and authentication are complicated, especially when asymmetric, public key cryptography is being used. Over the years, these complexities have resulted in a wide range of bugs in real crypto libraries and the software that uses them.

Google's ambition with Project Wycheproof is to ensure that these known flaws are eradicated. The open source project contains a number of test cases that check for these known flaws; currently, there are more than 80 tests for 40 different defects. The project is limited in scope, hence the name that Google has chosen: Mount Wycheproof is a hill that some claim is the world's smallest mountain. Its peak is 237 meters above sea level and just 43 meters above the surrounding plain.

Here's an example of the kind of thing that Google is testing for: one set of tests is for the elliptic curve Diffie-Hellman key agreement protocol that's used to securely share an encryption key between a client and a server without risking that an eavesdropper learns of the key. Both the client and the server have their own private key that they need to keep secret from one another. The algorithm has an implementation vulnerability: if a malicious client structures its key in a particular way, it can trick the server to which it's connecting into disclosing its private key. This malicious behavior can be detected as long as the server performs certain checks on the client's key, but not all libraries do this correctly.

Project Wycheproof includes a test for this flaw, and the development of this test has already yielded fruit. The popular Bouncy Castle crypto library didn't handle the malicious client key situation correctly. Hence, it was vulnerable to leaking the server's private key. Aiming at encryption libraries gives the project particularly high bang for its buck: these libraries are used and reused by thousands of other pieces of software, so their bugs can have an impact that runs far and wide. Fixing these flaws at the source gives a high return on investment.

Presently, Wycheproof only contains Java tests, written to test crypto libraries that use the Java Cryptography Architecture. Java is designed so that crypto libraries can be developed and act as drop-in replacements for one another, as long as they all conform to a particular API. This enables one set of tests to easily be used against multiple different libraries. It's also of particular relevance and interest to Google due to Android's use of the same Java APIs. Google intends to expand the tests to other languages and libraries in the future.