Menu

Heartbleed

CVE-2014-0160High · 7.5

Heartbleed let anyone read up to 64 KB of a server's memory at a time, over and over, through OpenSSL's TLS heartbeat. Private keys, passwords and session cookies leaked, and normal logs showed nothing.

Affected
OpenSSL 1.0.1 to 1.0.1f
Fixed in
1.0.1g
Published
2014-04-07
Known exploited since
2022-05-04

What Heartbleed is

The TLS heartbeat keeps a connection alive: one side sends a message and its length, the other echoes it back. Vulnerable OpenSSL trusted the claimed length, so a tiny message claiming 64 KB came back with 64 KB of whatever sat next to it in memory.

Spot it in the logs

A sensor on the network recorded these. Which lines are the leak?

Heartbleed leaves no trace in normal application logs. Only the network shows it: heartbeat messages whose replies are far larger than normal.

How the attack works

  1. 1The attacker sends a heartbeat that claims 64 KB but carries almost nothing
  2. 2OpenSSL trusts the claimed length
  3. 3It replies with 64 KB of memory: keys, passwords, cookies
  4. 4The attacker repeats it, and normal logs show nothing

Why Heartbleed was so severe

NVD scores Heartbleed 7.5 under CVSS 3.1, and it scored only 5.0 under the CVSS 2 system used in 2014. The number is modest because the bug reads memory but cannot change data or run code. A 2014 study led by the University of Michigan still called it one of the most consequential vulnerabilities since the commercial internet began. It needed no login, worked over a normal TLS connection, could be repeated as often as the attacker liked, and left nothing in normal logs. Each request returned up to 64 KB of the server's memory, and that memory could hold private keys, passwords, session cookies and the data users had just sent.

The reach was huge. OpenSSL is the encryption library behind Apache and nginx, which together served over 66% of active websites according to the Netcraft figures cited by the finders. Netcraft found the heartbeat feature turned on at 17.5% of SSL sites with trusted certificates, about half a million certificates. Sites with it turned on included Yahoo, Twitter, GitHub, Tumblr, Steam and Dropbox. That study estimated that 24% to 55% of HTTPS sites in the Alexa top million were vulnerable at first, including 44 of the top 100.

It was also old and hidden in places people forget. The heartbeat code was added to OpenSSL on 31 December 2011 and shipped in OpenSSL 1.0.1 on 14 March 2012, so servers were exposed for about two years. The same study found more than 70 models of vulnerable embedded devices, such as printers, firewalls, VPN gateways, storage boxes and cameras, plus mail servers and more than half of all Tor nodes. In April 2014 Cloudflare challenged researchers to steal a private key from a test server, and four people did, the first one after at least 2.5 million requests.

Is your version affected?

  • Linux and macOSopenssl version
  • Debian and Ubuntudpkg -l | grep libssl
  • Red Hat and Fedorarpm -q openssl

Affected. Upgrade to 1.0.1g.

How to detect Heartbleed

Heartbleed does not show up in application logs, so detection relies on network sensors, VPN logs and an inventory of OpenSSL versions.

  • Use IDS rules for large heartbeat repliesNetwork intrusion detection rules can spot the attack even though they cannot stop it. In the Mandiant case, the Snort rule named "SERVER-OTHER TLSv1.1 large heartbeat response - possible ssl heartbleed attempt" fired more than 17,000 times. Mandiant expects hundreds of attempts per attack, since each request leaks at most 64 KB.
  • Compare heartbeat request and reply sizesA normal heartbeat reply is about the same size as the request. A tiny request answered with thousands of bytes is a leak. The Michigan researchers extended the Bro (now Zeek) TLS analyzer to read heartbeat messages and flagged a success whenever the server answered with more data than it received.
  • Watch for heartbeats before the handshake endsMany attackers sent heartbeats before the TLS handshake had finished. The Michigan researchers treated any heartbeat seen before the handshake ended, or before the first application data, as an attack attempt. The message type and size travel in the clear, so a sensor can flag this even in encrypted traffic.
  • Review VPN logs for session IP flip-flopsMandiant found stolen sessions by looking for one user session whose source IP kept switching between two addresses, sometimes within one or two seconds. The two addresses belonged to different providers and places. Matching those times with the IDS alerts confirmed the theft.
  • Scan and inventory every TLS serviceList every system that uses OpenSSL 1.0.1 to 1.0.1f, including appliances, mail servers and VPN gateways, not only web servers. A safe check, such as the Nmap script ssl-heartbleed, tells you whether a service still answers oversized heartbeats.
  • Check that certificates were really replacedAfter patching, confirm the certificate was reissued with a new private key and the old one revoked. The Michigan study found that 14% of sites that replaced their certificate kept the same private key, which gave them no protection.

Check a log line

Search the logs yourself

Which heartbeats leaked memory? Compare the claimed lengths.

portal: /var/log/events.log

Fix it, in this order

  1. 1Fix OpenSSL firstUpgrade to 1.0.1g, or rebuild it without heartbeats.
  2. 2Then change what was in memoryReissue the certificate, end active sessions and change the passwords used since the leak began.
  3. 3Never the other way roundA password changed while the server still leaks leaks again.
  • CVE-2014-0224The "CCS Injection" bug, published by OpenSSL on 5 June 2014, let an attacker in the middle force weak keys and read or change traffic. It is a different flaw that needed the next update, 1.0.1h, so servers patched only for Heartbleed were still exposed.
  • CVE-2014-0195A buffer overrun in OpenSSL's DTLS code, fixed in the same June 2014 advisory. It affected only programs using DTLS, but OpenSSL said it could allow code execution.
  • CVE-2014-3566POODLE, published in October 2014, is a different kind of problem: NVD describes a weakness in the SSL 3.0 protocol itself, as used in OpenSSL through 1.0.1i and other products. Upgrading OpenSSL for Heartbleed does not remove it, so it is handled separately.

Timeline

  1. 2014-04-07Heartbleed is disclosed with the fix, OpenSSL 1.0.1g
  2. 2014-04-09The Canada Revenue Agency shuts its online services after about 900 numbers were taken
  3. 2022-05-04CISA adds it to its catalog of exploited vulnerabilities

Heartbleed attacks in the wild

Researchers who kept full traffic recordings from four networks found no sign of Heartbleed attempts before 7 April 2014. After disclosure, attempts began in less than 24 hours: they counted 5,948 attempts from 692 different hosts. The authors add that scanning at other times or on other networks cannot be ruled out.

The first confirmed intrusions followed within days. Mandiant reported that from 8 April an attacker sent repeated heartbeat requests to an organization's SSL VPN appliance, collected the session tokens of logged in users, and took over their VPN sessions. This got around the company's multifactor authentication. In Canada, the Canada Revenue Agency said about 900 social insurance numbers were stolen, and police said the bug was used. The agency shut its online tax filing on 9 April, and on 15 April police arrested a 19-year-old from London, Ontario, who was charged with unauthorized use of a computer and mischief in relation to data.

The largest breach linked to Heartbleed is disputed. In August 2014 the US hospital operator Community Health Systems reported the theft of data on about 4.5 million people, including names, addresses and Social Security numbers. TrustedSec said, based on an anonymous source, that attackers used Heartbleed to pull login details from a Juniper device's memory and then logged in through the VPN. The company did not confirm that account. Mandiant, which investigated, suggested the attackers were a group based in China.

Who found Heartbleed

Two teams found the bug separately. Neel Mehta of Google Security found it first; the Michigan study dates his discovery to 21 March 2014 and his report to the OpenSSL team to 1 April. Engineers at the security testing company Codenomicon (Riku, Antti and Matti) found it while improving their own testing tools and passed it to the Finnish national CERT, NCSC-FI. The dates differ slightly: the study puts Codenomicon's discovery on 2 April, while Codenomicon's own site says 3 April.

Codenomicon registered heartbleed.com on 5 April and explained the bug there under the name Heartbleed. Some companies, including Cloudflare and Akamai, were told early and patched in private, while several Linux distributions got less than a day of warning. OpenSSL published version 1.0.1g and its advisory on 7 April 2014, with a fix prepared by Adam Langley and Bodo Moeller.

Lessons from Heartbleed

Heartbleed turned attention to how critical open source code is funded. In response, the Linux Foundation formed the Core Infrastructure Initiative, funded by companies to find and support under-funded projects that critical infrastructure depends on. In May 2014 it named OpenSSL, OpenSSH and NTP as its first funded projects, paying for two full-time OpenSSL developers and an audit. LibreSSL, now developed inside the OpenBSD project, was also forked from OpenSSL in 2014.

For defenders, the lesson was that patching is only half the response when secrets may have leaked. Keys, certificates, sessions and passwords have to be changed after the patch, in that order. Many sites skipped this: the Michigan study found only 10% of sites still vulnerable two days after disclosure replaced their certificates within the next month. The finders also recommended forward secrecy, which stops a stolen key from decrypting old recorded traffic, yet the study saw only 44% of connections use it in May 2014. And the long tail was real: Shodan still counted 199,594 vulnerable devices in January 2017.

The case

Heartbleed: The Staff Portal

  1. Find what the advisory affects
  2. What could have leaked?
  3. Close it in the right order
Investigate it hands-on

Questions

Why is Heartbleed only rated high, not critical?

NVD scores it 7.5 because it reads memory rather than running code. What it leaks, private keys and passwords, can be just as damaging.

Should I change passwords before or after patching?

After. A password changed while the server still leaks memory can leak again.

What is Heartbleed?

Heartbleed (CVE-2014-0160) is a bug in the OpenSSL encryption library, disclosed on 7 April 2014. Its code for the TLS heartbeat, a keep-alive message, trusted a length field sent by the other side. That let anyone read up to 64 KB of a server's or client's memory per request.

Is Heartbleed still a threat?

Only on systems that were never updated, but those still exist. Shodan counted almost 200,000 vulnerable devices in January 2017, and CISA added Heartbleed to its exploited vulnerabilities catalog in May 2022. Old appliances and embedded devices are the most likely to still have it.

Has Heartbleed been patched?

Yes. OpenSSL 1.0.1g fixed it on the day of disclosure, and the 1.0.0 and 0.9.8 branches never had the bug. Vendors of devices that bundle OpenSSL released their own firmware updates, so each appliance needs its vendor's fix.

How do I check if I am vulnerable to Heartbleed?

Check the OpenSSL version on each system: 1.0.1 to 1.0.1f is vulnerable. For a service you cannot log into, use a safe scanner check such as the Nmap script ssl-heartbleed. Remember mail servers, VPN gateways and appliances, not just websites.

Did Heartbleed affect clients too?

Yes. The OpenSSL advisory says a connected client or server can read the other side's memory, so a malicious server could also read memory from a vulnerable client. Updating the OpenSSL library on client machines and devices closes this as well.

Why is it called Heartbleed?

The bug is in OpenSSL's heartbeat feature and it makes the server leak, or bleed, memory. Codenomicon, one of the two teams that found it, used the name when it set up heartbleed.com to explain the bug.