tl;dr I found a vulnerability in apt that allows a network man-in-the-middle (or a malicious package mirror) to execute arbitrary code as root on a machine installing any package. The bug has been fixed in the latest versions of apt. If you’re worried about being exploited during the update process, you can protect yourself by disabling HTTP redirects while you update. To do that, run:
$ sudo apt update -o Acquire::http::AllowRedirect=false $ sudo apt upgrade -o Acquire::http::AllowRedirect=false
If your current package mirrors redirect by default (meaning you can’t update apt when using that flag) you’ll need to pick different mirrors or download the package directly. Specific instructions for upgrading on Debian can be found here. Ubuntu’s announcement can be found here.
As a proof of concept, below is a video of me exploiting the following
FROM debian:latest RUN apt-get update && apt-get install -y cowsay
When fetching data, apt forks off worker processes that specialize in the various protocols that will be used for data transfer. The parent process then communicates with these workers over stdin/stdout to tell them what to download and where to put it on the filesystem using a protocol that looks a little like HTTP. For example, when running
apt install cowsay on a machine using repos served over HTTP, apt will fork off
/usr/lib/apt/methods/http, which returns a
100 Capabilities message:
100 Capabilities Version: 1.2 Pipeline: true Send-Config: true
The parent process will then send its configuration and request a resource, like this:
601 Configuration Config-Item: APT::Architecture=amd64 Config-Item: APT::Build-Essential::=build-essential Config-Item: APT::Install-Recommends=1 (...many more lines omitted...) 600 URI Acquire URI: http://deb.debian.org/debian/pool/main/c/cowsay/cowsay_3.03+dfsg2-3_all.deb Filename: /var/cache/apt/archives/partial/cowsay_3.03+dfsg2-3_all.deb Expected-SHA256: 858d5116a60ba2acef9f30e08c057ab18b1bd6df5ca61c233b6b7492fbf6b831 Expected-MD5Sum: 27967ddb76b2c394a0714480b7072ab3 Expected-Checksum-FileSize: 20070
And the worker process will respond with something like:
102 Status URI: http://deb.debian.org/debian/pool/main/c/cowsay/cowsay_3.03+dfsg2-3_all.deb Message: Connecting to prod.debian.map.fastly.net 102 Status URI: http://deb.debian.org/debian/pool/main/c/cowsay/cowsay_3.03+dfsg2-3_all.deb Message: Connecting to prod.debian.map.fastly.net (2a04:4e42:8::204) 102 Status URI: http://deb.debian.org/debian/pool/main/c/cowsay/cowsay_3.03+dfsg2-3_all.deb Message: Waiting for headers 200 URI Start URI: http://deb.debian.org/debian/pool/main/c/cowsay/cowsay_3.03+dfsg2-3_all.deb Size: 20070 Last-Modified: Tue, 17 Jan 2017 18:05:21 +0000 201 URI Done URI: http://deb.debian.org/debian/pool/main/c/cowsay/cowsay_3.03+dfsg2-3_all.deb Filename: /var/cache/apt/archives/partial/cowsay_3.03+dfsg2-3_all.deb Size: 20070 Last-Modified: Tue, 17 Jan 2017 18:05:21 +0000 MD5-Hash: 27967ddb76b2c394a0714480b7072ab3 MD5Sum-Hash: 27967ddb76b2c394a0714480b7072ab3 SHA256-Hash: 858d5116a60ba2acef9f30e08c057ab18b1bd6df5ca61c233b6b7492fbf6b831 Checksum-FileSize-Hash: 20070
When the HTTP server responds with a redirect, the worker process returns a
103 Redirect instead of a
201 URI Done, and the parent process uses this response to figure out what resource it should request next:
103 Redirect URI: http://deb.debian.org/debian/pool/main/c/cowsay/cowsay_3.03+dfsg2-3_all.deb New-URI: http://example.com/new-uri
Unfortunately, the HTTP fetcher process URL-decodes the HTTP
Location header and blindly appends it to the
103 Redirect response:
(Note: there are important differences here across different versions of apt. The code above is from 1.4.y, which is what recent Debian uses. Some recent Ubuntu versions use 1.6.y, which doesn’t just blindly append the URI here, however there is still an injection vulnerability into the subsequent
600 URI Acquire requests made to the HTTP fetcher process. I haven’t checked other versions.)
So if the HTTP server sent
Location: /new-uri%0AFoo%3A%20Bar, the HTTP fetcher process would reply with the following:
103 Redirect URI: http://deb.debian.org/debian/pool/main/c/cowsay/cowsay_3.03+dfsg2-3_all.deb New-URI: http://deb.debian.org/new-uri Foo: Bar
Or, if the HTTP server sent:
then the HTTP fetcher process would reply with this:
103 Redirect URI: http://deb.debian.org/debian/pool/main/c/cowsay/cowsay_3.03+dfsg2-3_all.deb New-URI: http://deb.debian.org/payload 201 URI Done URI: http://deb.debian.org/payload Filename: /var/lib/apt/lists/deb.debian.org_debian_dists_stretch_Release.gpg Size: 20070 Last-Modified: Tue, 07 Mar 2017 00:29:01 +0000 MD5-Hash: 27967ddb76b2c394a0714480b7072ab3 MD5Sum-Hash: 27967ddb76b2c394a0714480b7072ab3 SHA256-Hash: 858d5116a60ba2acef9f30e08c057ab18b1bd6df5ca61c233b6b7492fbf6b831 Checksum-FileSize-Hash: 20070
The parent process will trust the hashes returned in the injected
201 URI Done response, and compare them with the values from the signed package manifest. Since the attacker controls the reported hashes, they can use this vulnerability to convincingly forge any package.
Planting the malicious package
In my proof of concept, because I chose to inject the
201 URI Done response right away, I had to deal with the fact that no package had actually been downloaded yet. I needed a way to get my malicious
.deb onto the system for use in the
To do this, I took advantage of the fact that the
Release.gpg file pulled during
apt update is both malleable and installed into a predictable location. Specifically,
Release.gpg contains ASCII-armored PGP signatures that look like:
-----BEGIN PGP SIGNATURE----- ... -----END PGP SIGNATURE-----
But apt’s signature validation process is totally fine with the presence of other garbage in that file, as long as it doesn’t touch the signatures. So I intercepted the
Release.gpg response and prepended it with my malicious deb:
<oops.deb contents> -----BEGIN PGP SIGNATURE----- ... -----END PGP SIGNATURE-----
Then, I set the
Filename parameter in the
201 URI Done response to point to:
The http/https debate
By default, Debian and Ubuntu both use plain http repositories out of the box (Debian lets you pick what mirror you want during installation, but doesn’t actually ship with support for https repositories – you have to install
If packages manifests are signed, why bother using https? After all, the privacy gains are minimal, because the sizes of packages are well-known. And using https makes it more difficult to cache content.
People sometimes get really passionate about this. There are single purpose websites dedicated to explaining why using https is pointless in the context of apt.
They’re good points, but bugs like the one I wrote about in this post exist. And this bug isn’t even special – here’s a different one that Jann Horn found in 2016 with the same impact. Yes, a malicious mirror could still exploit a bug like this, even with https. But I suspect that a network adversary serving an exploit is far more likely than
deb.debian.org serving one or their TLS certificate getting compromised.
(This is all assuming that
apt-transport-https is itself not catastrophically broken. I haven’t audited it, but it looks like a relatively thin wrapper around libcurl.)
Supporting http is fine. I just think it’s worth making https repositories the default – the safer default – and allowing users to downgrade their security at a later time if they choose to do so. I wouldn’t have been able to exploit the
Dockerfile at the top of this post if the default package servers had been using https.
Thank you to the apt maintainers for patching this vulnerability quickly, and to the Debian security team for coordinating the disclosure. This bug has been assigned CVE-2019-3462.