What a week! I was just on Tampa’s ABC news affiliate discussing the recent Heartbleed vulnerability. Probably my favorite name for a software-flaw in recent memory. You can check that out here. I have to commend them for distilling my commentary into something useful for consumers, but there were definitely some important details lost in translation. This post is geared at addressing some of the more technical points, and to provide remediation strategies for enterprises caught off guard.

Heartbleed is a serious attack, because it affects one of the most critical structural components of modern computing. Secure sockets layer (SSL) is a foundational technology, built on top of TCP/IP and meant to protect sensitive transactions of all sorts. SSL and the cryptography it is based on is baked into a tremendous amount of software. The OpenSSL implementation is a cross-platform open source version of that technology, and those libraries are baked into Linux, BSD, and other distributions. Linux is the most widely deployed operating system on the planet. It sits on personal computers, tablets, laptops, smart phones, servers, and an extraordinary amount of proprietary networking equipment. So the global attack surface of this vulnerability is potentially huge. The scope includes not only core services running on those platforms, but major software packages linked against broken OpenSSL libraries.

We tend to think of OpenSSL in the context of the consumer web, because that is what most people interface with every day in the form of the familiar HTTPS url. That means OpenSSL is linked into major web software like Apache, Nginx, etc. However, the problem is larger because it encompasses the back end infrastructure that companies rely on in their internal networks, as well as the VPN hardware that is suppose to allow secure access to private networks, and the load balancers and proxies that front much commercial infrastructure. This weakens the security posture of many enterprises. That means that software like OpenLdap for directory and authentication services is probably vulnerable. Kerberized services using SSL for transport are probably vulnerable. OpenVPN (a major open source SSL VPN, and arguable one of my favorite pieces of software) is most certainly vulnerable in many configurations. Stunnel-wrapped sockets, secure ftp variants, the list goes on and on. The elements that are supposed to provide strong border security. If you think about the larger picture, that is a lot of code.

As many of you know I am a huge fan of Bitcoin and digital currency in general. We have already seen the release of re-linked binaries for Bitcoin but we can expect that it will take some time for that to be distributed. I don’t think that Bitcoind is very susceptible to actionable information leakage, but I haven’t done a security audit of it either. I would love someone more knowledgeable than I to comment on that.

That is the scope of the problem. So how does Heartbleed work? It lets potential attackers sniff active memory of affected machines in 64kb increments. This potentially can reveal private keys used in transactions, form submission data, or any arbitrarily encoded data. Repeating the attack over a long period of time bleeds out the target. The collected data can than be analyzed for actionable information, such as user names and passwords. Since OpenSSL strives for cross-platform support this means many Unix variants, distros, and more proprietary embedded devices may inadvertently leak information. The OpenBSD developer Ted Unangst has a good write up of some of the technical specifics here and here. There is much debate in the community over whether this is an honest mistake or something more insidious. With the rise of NSA surveillance and clear indications that other software has been tampered with it is certainly a possibility worth exploring. There is also a technical debate over whether custom memory allocation is a desirable feature in security software. I’m not sure where I stand on that except that speed of execution and performance enhancements should be a very distant concern compared with core security.

Let’s talk remediation. Some of you know that I work on a variety of security products myself, SenderDefender is my email attachment security project. It integrates ephemeral messaging with client side encryption to securely transport sensitive information between individuals or companies. Architecturally it is not exploitable by heart bleed in an actionable way, that is there is no way to reveal or modify anything about customer data using this bug. I doubt there are many security services that could say the same, dropbox-et-al could very easily have exposed customer credentials. So how do you address customer impact and remediation. You should have an upgrade plan, security fixes for OpenSSL have been released on all major platforms. Use your distributions update mechanism to make sure you have the latest version. Keep an eye out for the commercial vendors, Juniper, Cisco are lagging on some of their product lines but embedded border security devices can be affected. Look for statically compiled software you are running that may include the outdated libraries, the binary release of Bitcoind is a good example, but tons of commercial and open source software is released in binary form. Finally, once perimeter security has been reestablished, make sure to take a hard look at backed components like OpenLDAP to make sure they are unaffected. Now comes the heart ache, time to reissue SSL certificates for your public and internal sites that could have been compromised, and after that has been done let your customers know that they can change their passwords.

Users are basically at the mercy of the software vendors. Don’t rush to change passwords and update information, this is an eavesdropping attack on the server. Until they are patched, and now that the bug and exploit is widely circulated changing your password is not advisable. Do update your local software, smart phone, tablets, etc.

Major software flaws have long term affects. The Internet is a big place after all, and even though many responsible vendors and service providers are going to be updating their software there will be corners of the Internet and back office systems that remain unpatched and vulnerable for years to come. This particular vulnerability is going in the black-hat toolkit for the long term.