For all the criticism of Microsoft and its security flaws, the software giant has made an impressive turnaround.  While Vista has been derided for a variety of reasons, most would agree that it’s much more secure than Windows XP.  Recently, a hacker conference showed just how vulnerable systems running Mac OS X are, due to their slow rate of patches.  The Mac machine was hijacked within 10 minutes, while the Linux and Windows boxes survived the day.

Now an even worse security flaw has been found in some of the basic code used by a wide variety of Linux security programs.  The error originated back in May 2006 when workers on the open-source security project committed a grave and unrealized error. 

A simple programming error reduced the entropy in the generated program keys created by the OpenSSL library.  Why does this matter?  The OpenSSL library's key generation and other routines are used by the SSH remote access program, the IPsec Virtual Private Network (VPN), the Apache Web server, secure email clients, programs that offer secure internet portals and more.

Just two lines of code created crippling security holes in four different open source operating systems, 25 application programs, and millions of internet-attached computer systems.  The vulnerability was publicly discovered for the first time May 13, after having left the door open nearly two years.  A patch has been distributed, but that can do nothing to repair the damage that has occurred to compromise systems.  Worse yet, it appears that through the installation of compromised keys on other systems, numerous systems not even running the code have likely been compromised.

To understand the error fully, a basic discussion on cryptography is essential.  On a network anyone can peek at traffic, which is bad news for anyone sharing personal information.  However, by using keys, information can be encrypted and then decrypted on the other side by a friendly computer with the proper key.  As a "secure key" is typically 128 bits, which is 2¹²⁸ or about 3.4*10³⁸, the possibility of breaking the key by merely by a brute force attack is out of the realm of modern computing power.  A brute force attack simply involves guessing every single number, but to try to do this on a number of this size would take many years.

However, the system falls apart if the computer can only make a small set of keys, despite the large key size.  To a normal user the key looks fine, it’s the right size, and the data is being encrypted as it’s sent out.  However, to the malicious user they can now use brute force attacks to guess the key and monitor your activity, opening the door to surveillance and exploitation.  This is exactly what has resulted based on the newly discovered error.

The error reduced the number of keys that Linux can generate from 2¹²⁸ to approximately 2¹⁵.  The error was not caught until now because the keys were still 128 bits and to the human eye looked random.  If the system had consistently produced one key, this problem would have been caught, but instead it produced a variety of keys, but a much smaller variety.  The number of keys the system can generate varies with processor architecture, the size of the key, and the type of the key, but all keys using the flawed code will be greatly reduced in their number of possibilities.

Now that the floodgates are opened, a hacker HD Moore of the Metasploit project has released "toys" to help malicious users crack the poor defenseless Linux and Ubuntu boxes.  Moore's website provides lists of precalculated keys based on the bug, to allow malicious users to easily identify vulnerable systems.

Fixing the key problem is not as simple as fixing a buffer overflow vulnerability, another typical security flaw.  As the keys generated our actual files, merely patching the system will not change these files.  Every single key will need to be replaced in a difficult and time consuming process.  Further keys need to be certified and distributed, which takes more time and is error prone.

Debian, the Linux variant used largely by security professionals, and Ubuntu, the variant most commonly used by home users are both affected.  Furthermore, Windows servers may be compromised as well if they are using keys generated on Linux systems. 

Ironically the bug originated from an automated tool known as Valgrind which is supposed to reduce programming bugs which lead to security vulnerabilities.  It found that a block memory was not being properly initialized, meaning that it would contain random information.  The automated tool politely inserted code to clean up the block of memory making it all zeros.  The only problem was that the system was intentionally using the block's unknown to get randomness to generate the keys.  The library also gets randomness from mouse movements, keystroke timings, network packet arrival timings, and even microvariations in hard drive speed.

The Valgrind code caused errors, so the programmers simply commented out all the code, including the other methods of generating randomness on accident.  Only the code which utilized the process ID, an integer ranging from 0 to 32,767, remained to provide randomness.  It turns out the "fix" turned grievous error was not the work of the OpenSSL programmers themselves, but of the Debian team, known for their security expertise. 

OpenSSL developer Ben Laurie raged, "Never fix a bug you don't understand!  Had Debian [sent the bug to us] in this case, we (the OpenSSL Team) would have fallen about laughing, and once we had got our breath back, told them what a terrible idea this was. But no, it seems that every vendor wants to 'add value' by getting in between the user of the software and its author."

One developer more alarmingly points out that the vulnerability has showed a perhaps fatal flaw in the state of the open source industry and in the computer security in general.  One programmer can make a major change which can be blindly accepted by other developers with little understanding of the implications.  This reckons back to controversial statements made by Steve Gibson, a highly respected security consultant, when a major bug was found in Windows.  Gibson suggested that rather than dumb error, it was an intentional attempt to create an open back door.  While hopefully the Linux vulnerability was not maliciously created, the possibility of such a development remains.

Like Alice in Wonderland, it is often amazing to see just far down the rabbit hole goes in terms of the breadth of these kinds of problems.  And this problem is clearly illustrative that unless a more comprehensive methodology of security development is adopted, these problems will only persist and multiply with time.