Why does APT not use HTTPS? - <AUTHOR> https://whydoesaptnotusehttps.com/ <comment><USER2> I guess I can copy over a comment I made when this previously made rounds:

I have a few problems with this. The short summary of these claims is "APT checks signatures, therefore downloads for APT don't need to be HTTPS".

The whole argument relies on the idea that APT is the only client that will ever download content from these hosts. This is however not true. Packages can be manually downloaded from packages.debian.org and they reference the same insecure mirrors. At the very least Debian should make sure that there are a few HTTPS mirrors that they use for the direct download links.

Furthermore Debian also provides ISO downloads over the same HTTP mirrors, which are also not automatically checked. While they can theoretically be checked with PGP signatures it is wishful thinking to assume everyone will do that.

Finally the chapter about CAs and TLS is - sorry - baseless fearmongering. Yeah, there are problems with CAs, but deducing from that that "HTTPS provides little-to-no protection against a targeted attack on your distribution's mirror network" is, to put it mildly, nonsense. Compromising a CA is not trivial and due to CT it's almost certain that such an attempt will be uncovered later. The CA ecosystem has improved a lot in recent years, please update your views accordingly.

<comment><USER3> Also, their claim that HTTPS doesn't hide the hosts that you are visiting is about to not be true. Encrypted SNI is now part of the TLS 1.3 RFC, so HTTPS will actually hide one's internet browsing habits quite well. The only holes in privacy left on the web's stack are in DNS.

<comment><USER4> Can you point out where encrypted SNI is in the RFC? I've read the RFC, and I don't recall it being in there. I do see that there is an extension published, which I haven't reviewed in depth.

From a breif review, I see two potential issues:

a) the encrypted sni record contains a digest of the public key structure. This digest is transmitted in the clear (as it must be at this phase of the protocol), so a determined attacker could create a database of values for the top N mirror sites.

b) in order to be useful, the private key for the public keys would need to be shared across all servers supporting that hostname. That's not a big deal for a normal deployment, but it's not great for a volunteer mirrors system -- lots of diverse organizations own and operate the individual mirrors and we need to count on all of those to keep it secure. Also, it adds an extra layer of key management, which is an organizational and operational burden.

<comment><USER5> Yeah, your parent is wrong about it being in the RFC. ESNI is something that they decided wasn't possible and ruled out of scope for TLS 1.3 but then somebody had a brainwave and Rescorla plus some people at Cloudflare wrote IDs and did live fire testing. The drafts are maybe at the "this is the rough shape of a thing" stage, more than ambitions but not a basis on which to announce specific plans.

It's also pointless without DPRIVE. If people can see all your DNS lookups they can guess exactly what you're up to. That's why that Firefox build did both eSNI and DoH

</comment></comment></comment></comment><comment><USER6> The arguments are correct. APT does not need HTTPS to be secure. That said, if APT was designed today I'm sure it would use HTTPS. It's now the default things to do, and Let's Encrypt makes it free and easy.

However Debian, where APT is from, relies on the goodwill of various universities and companies to host their packages for free. I can see that they don't want to make demands on a service they get for free, when HTTPS isn't even necessary for the use case.

Also since APT and Debian was created in the pre-universal HTTPS days, it does things like map something.debian.org to various mirrors owned by different parties. That makes certificate handling complicated.

<comment><USER7> It does not need HTTPS to be _secure_ , but it would need HTTPS to add _privacy_ , for which the protocol has none at the current time.

<comment><USER8> Genuine question: what are you doing with APT that requires privacy?

<comment><USER9> In the past cryptographic export was illegal in the us and many other countries, it can still get you put on a list.

In some countries messaging apps like signal and telegram are illegal.

There is no telling what seemingly benign software will be made illegal in the future for political reasons.

 _Privacy is always a requirement because of these reasons._

<comment><USER10> Size fingerprinting defeats such "privacy". If you need this, use Tor instead.

<comment><USER11> Cause pushing the barrier of how easy it is to detect is a bad thing.

<comment><USER12> This is very asymmetric though. The cost for an attacker is very low. The cost for Debian and it's mirror network is very high.

Given that the cost of implementation is high and the protection is minimal the decision to not do so is reasonable.

<comment><USER13> > The cost for Debian and it's mirror network is very high.

I'm curious how high it actually is. They say it's high, but that could well just be hand-waving. Sure, prior to things like LetsEncrypt those SSL certs would have been a notable financial burden. There's also some extra cost on infrastructure covering the cryptographic workload, but increasingly the processors in servers are capable of handling that without any notable effort.

<comment><USER4> Certificate cost is trivial. Let's encrypt makes it free, but with a small change to the host names (country code.ftp.debian.org instead of ftp.countrycode.debian.org), all mirrors could have been covered with a single certificate. Some CAs will let you buy one wildcard cert and issue unlimited duplicate certificates with the same name. So, that would cost some money, but probably not too much.

The real costs are organizational and technical.

Organizing all the different volunteers who are running the mirrors to get certificates installed and updated and configured properly is work. Maybe let's encrypt automation helps here.

From a technical perspective, assuming mirrors get any appreciable traffic, adding https adds significantly to the CPU required to provide service. TLS handshaking is pretty expensive, and it adds to the cost of bulk transfer as well.

I get the feeling that alot of the volunteer mirrors are running on oldish hardware that happens to have a big enough disk and a nice 10G ethernet. I've run a bulk http download service that enabled https, and after that our dual xeon 2690 (v1) systems ran out of CPU instead of out of bandwidth. CPUs newer than 2012 (Sandy Bridge) do better with TLS tasks, but mirrors might not be running a dual CPU system either.

<comment><USER14> Old hardware will eventually die and needs replacing. I run infrastructure for a CDN setup, and we actually _reeuced_ the CPU overhead with TLS 1.3 + HTTP/2.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER15> Yeesh, as someone who has had to troubleshoot more than a few times HTTP payloads that were mangled by shitty ISP software to inject ads or notifications I would love HTTPS as a default to prevent tampering. I get the arguments against it, but I have 100% seen Rogers in Canada fuck up HTTP payloads regardless of MIME types while rolling out "helpful" bandwidth notifications. Signatures will tell you yes this is corrupt but end-to-end encryption means that the person in the middle can't even try.

<comment><USER16> Likewise, integrity of the download is the primary reason I've switched downloads to HTTPS too. The argument that singed downloads is enough fails to address what the user is supposed to do after the integrity has failed? A redownload can result in the same tampered with file. This isn't hypothetical btw, it happens in the real world, I've had ISPs in Ireland and South Africa damage downloads due to their injections and users don't care if it's their ISP, they get pissed off at you unfortunately.

</comment></comment><comment><USER17> APT many not be using HTTPS. But APT does support HTTPS builtin since version 1.5. I couldn't find any changelog or NEWS for that, but [https://packages.debian.org/sid/apt-transport- https](https://packages.debian.org/sid/apt-transport-https) says so.

I myself uses HTTPS mirror provided by Amazon aws ([https://cdn- aws.deb.debian.org/](https://cdn-aws.deb.debian.org/)). I do so because My ISP sometimes forward to it's login page when I browse HTTP URLs. Also, it does sometime include Ads (Yeah, it's really bad, but it does remind me that I'm being watched).

<comment><USER18> I watched this blow up on infosec twitter and it made no sense, APT has https or even Tor if you really insist on it. Takes 30 seconds to configure.

Storm in a teacup.

<comment><USER19> Defaults matter, as that is what a large majority runs with.

<comment><USER18> Yeah true, but the arguments for tls default ring a bit hollow, to me at least. Someone who really wants the defense-in-depth should probably be switching to onion sources anyway, I was impressed with how quick they were.

As the article says, replay attacks are voided and an adversary could simply work out package downloads from the metadata anyway.

I personally use https out of general paranoia, but understand the arguments for not changing. It's two extra lines in a server setup script.

</comment></comment></comment></comment><comment><USER20> One reason to prefer HTTPS is that in the event of a vulnerability in the client code, an attacker cannot trigger that vulnerability using a MITM attack if HTTPS is in use. One such vulnerability was recently found in apk-tools: [https://nvd.nist.gov/vuln/detail/CVE-2018-1000849](https://nvd.nist.gov/vuln/detail/CVE-2018-1000849)

<comment><USER21> While I agree with your point, a counterpoint is that vulnerabilities in the HTTPS implementation are also possible, and by introducing HTTPS code you are increasing the surface area of possible vulnerabilities.

<comment><USER22> How is this making you more vulnerable that not having it?

It's like saying don't bother having locks on your doors because they are subject to picking... not a great argument.

<comment><USER21> The argument is that the package you are downloading is already signed with public key crypto and verified during the update process. It's integrity is "secured". However there could be bugs in that implementation, and bugs can be exploited to (in one of the worst case scenarios) gain remote code execution during a MITM attack.

A "solution" is to protect the endpoint with HTTPS, making MITM attacks impossible. Except that it's also possible that the HTTPS code could suffer from vulnerabilities which can lead to remote code execution. And if I'm being honest, the code which implements HTTPS is much larger and more complicated than the code which is doing the signature checking in APT right now, so by that measure it's actually a downgrade to something "less secure" since it's just adding on more complexity while not improving security much at all.

In reality I believe HTTPS is more heavily scrutinized than the signature verification code in APT is, and therefore could improve security, and there are additional other benefits to HTTPS aside from added security against implementation bugs (like an improvement to secrecy, even if it's small, and better handling by middleboxes which often try to modify HTTP requests but know to not try with HTTPS requests).

<comment><USER22> I started writing a response but you've thoroughly covered all the bases, thanks for expanding and explaining further.

</comment></comment></comment></comment></comment><comment><USER23> > "Furthermore, even over an encrypted connection it is not difficult to > figure out which files you are downloading based on the size of the > transfer"

Is it really not difficult? I bet if you sorted all the ".deb" packages on a mirror by size a lot of them would have a similar or the same size, so you wouldn't be able to tell them apart based on the size of the dialog.

Furthermore, when I update Debian I usually have to download some updates and N number of packages. I don't know if this is now done with a single keep- alive connection. If it is, then figuring out what combination of data was downloaded gets a _lot_ harder.

Finally, this out of hand dismisses a now trivial attack (just sniff URLs being downloaded with tcpdump) by pointing out that a _much_ harder attack is theoretically possible by a really dedicated attacker.

Now if you use Debian your local admin can see you're downloading Tux racer, but they're very unlikely to be dedicated enough to figure out from downloaded https sizes what package you retrieved.

<comment><USER24> > I bet if you sorted all the ".deb" packages on a mirror by size a lot of > them would have a similar or the same size

Why bet when you can science? :)

      $ rsync -r rsync://ftp.be.debian.org/debian/pool/main/ | grep "\.deb$"
      $ wc -l debian.txt
      1246733 # total number of deb packages
      $ cat /tmp/debian.txt | awk '{ print $2 }' | sort | uniq -c | sort -rn | head -1
       463 1044 # The most common package size has 463 occurances
      $ cat /tmp/debian.txt | awk '{ print $2 }' | sort | uniq -c | sort -rn | awk '{ print $1}' | grep -c "^1$"
      259300 # The number of packages with a unique size
      $

<comment><USER25> As I found at [https://news.ycombinator.com/item?id=18960239](https://news.ycombinator.com/item?id=18960239) there can be duplication which is irrelevant for the point being discussed, as it is one version of a package duplicating another version of _the same package_ , meaning that the size is still a unique identifier of the package. It is worth checking that.

</comment></comment></comment><comment><USER26> I'm somewhat surprised that no-one has (yet) linked to this post [1] by Joe Damato of Packagecloud, which digs into attacks against GPG signed APT repos, or the paper which initially published them [2].

The post makes clear in the third paragraph: "The easiest way to prevent the attacks covered below is to always serve your APT repository over TLS; no exceptions."

[1]: [https://blog.packagecloud.io/eng/2018/02/21/attacks- against-...](https://blog.packagecloud.io/eng/2018/02/21/attacks-against- secure-apt-repositories/) [2]: [https://isis.poly.edu/~jcappos/papers/cappos_mirror_ccs_08.p...](https://isis.poly.edu/~jcappos/papers/cappos_mirror_ccs_08.pdf)

<comment><USER27> Thanks for posting these, I was unaware that they even existed.

</comment></comment><comment><USER28> This argument boils down to two things:

One side: We want performance / caching!

Other side: We want security!

Both sides sometimes argue disingenuously. It's true that caching is harder with layered HTTPS and that performance is worse. It's also true that layering encryption is more secure. (It's what the NSA and CIA do. You smarter than them?)

Personally, I'd default to security because I'm a software dev. If I were a little kid on a shoddy, expensive third world internet connection I'd probably prefer the opposite.

I just wish it were up to _me_.

<comment><USER29> I think it's important to reiterate that HTTPS only adds network _privacy_ about what you download. The signing of repos and their packages means they're guaranteed to be a pure copy of what's on the server.

That same mechanism means you can easily make trusted mirrors on untrusted hosting.

<comment><USER28> No, it also adds security.

If someone steals a signing key then they also need to steal the HTTPS cert. Or control the DNS records and generate a new one or switch to HTTP.

Adding an extra layer of encryption is like adding more characters on a password. Sometimes it saves your bacon, sometimes it was useless and came with performance drawbacks.

If you still disagree with me, that's fine. But I want to hear why you continue to hold this opinion when worked for 1Password during Cloudbleed.

[https://blog.1password.com/three-layers-of-encryption- keeps-...](https://blog.1password.com/three-layers-of-encryption-keeps-you- safe-when-ssl/tls-fails/)

<comment><USER29> In a scalar sense, sure. In a binary "do we have enough security" sense, less so. I realise that's a shitty quality of argument and I could have been more explicit but you can _always_ add more security. Why, for example, aren't you demanding reproducible builds and signed source uploads?

Simply put —and this is, I think, where we disagree— the signing of packages is enough. The design of Apt is such that it doesn't matter where you get your package from, it's that it matches an installed signature.

Somebody could steal the key but they would then either need access to the repo or a targeted MitM on _you_. Network attacks are far from impossible but by the point you're organised to also steal a signing key, hitting somebody with a wrench or a dozen other plans become a much easier vector.

<comment><USER28> The problem with the binary sense is that people misunderstand risk. For example, the blackout in 2003 was partially caused by the windows worm earlier in the day. Even though the computers that were used to control the power grid weren't running windows, the computers that monitored them were. So a routine alarm system bug ended up cascading into a power outage that lasted over a week in some places, including my home at the time. This was classified for a while.

The people that programmed Windows before 2003 probably didn't consider their jobs with the full national security implications.

Then you take something simple, like Linux on a simple IoT device. Say a smart electrical socket. Many of these devices went without updates for _years_. Doesn't seem all that bad, right? Just turn off a socket or turn it on? How bad could it be?

At some point someone noticed that they were getting targeted and and said: "But why?" The reason is simple. You turn off 100k smart sockets all at once and the change in energy load can blow out certain parts of the grid.

The point isn't that someone will get the key. The point is that we know the network is hostile. We know people lose signing keys. We know people are lazy with updates. From an _economics_ perspective why is non-HTTPS justified? Right? A gig of data downloaded over HTTPS with modern ciphers costs about a penny for most connections in the developed world.

To me, it's worth the cost.

<comment><USER29> Although I would not class this as even potentially in-line with Blaster or the _imminent_ death of the internet under an IoT Botnet, I see your broader point. The deployment cost approaches zero and it does plug —however small— a possible vector.

I do think it would cause a non-zero amount of pain to deploy though. Local (eg corporate) networks that expect to transparently cache the packages would need to move to an explicit apt proxy or face massive surge bandwidth requirements, slower updates.

That said, if you can justify the cost, there is absolutely nothing stopping you from hosting your own mirror or proxy accessible via HTTPS.

I'm not against this, I just don't see the network as the problem if somebody steals a signing key. I think there are other —albeit harder to attain— fruits _like_ reproducible builds that offer us better enduring security. And that still doesn't account for the actions of upstream.

</comment></comment></comment></comment></comment></comment><comment><USER30> The claims on this site are easily falsifiable by past bugs on debian's own bug tracker: eg [https://bugs.debian.org/cgi- bin/bugreport.cgi?bug=710229](https://bugs.debian.org/cgi- bin/bugreport.cgi?bug=710229)

I've personally experienced this too: using apt in the presence of a captive portal replaces random bits of `/var/cache/apt` with HTML pages, breaking future updates until you manually find and fix the problem yourself.

<comment><USER31> The reverse argument also works: using HTTPS may lead you to link and expose, say, OpenSSL where you otherwise would not have needed to. OpenSSL has had dozens of vulnerabilities in the past: [https://www.openssl.org/news/vulnerabilities.html](https://www.openssl.org/news/vulnerabilities.html)

Some of these vulnerabilities have the potential for arbitrary code execution, leaving you worse off than the simpler solution based on the verification of cryptographic signatures that has fewer vulnerabilities by virtue of doing less.

The discussion at [https://whydoesaptnotusehttps.com](https://whydoesaptnotusehttps.com) is about the protocol. You can add implementation bug risks to the discussion if you want, but then include the risks from both the approaches being discussed.

<comment><USER32> You've proposed a reverse argument to an argument that was never made. ctz never said anything about vulnerabilities or implementation issues, they said a captive portal is a problem for apt over HTTP but not HTTPS. This is also true of ISPs that like to insert things into HTTP sessions.

<comment><USER31> "bugs"

</comment></comment></comment></comment><comment><USER33> One important factor this article left out is upgrades. If the given HTTPS implementation is broken because of what is now insecure protocols, insecure ciphers etc. Older systems can't update from the mirror if it's updated to use a 'secure' HTTPS configuration while it only supports the 'vulnerable' solution. If HTTPS is left insecure, then it is not much different from using HTTP.

APT's methodology avoids this and as the current signing and protection mechanisms are file based, the worst case scenario is introducing a new file with a new cryptographic signature along side the old schema, to support still updating a system running old security mechanism.

In comparison, trying to run multiple HTTPS servers with different configurations for specific versions of the system being updated would be a significant engineering effort, especially for mirrors.

<comment><USER34> Huh? All you would do is configure the web server running your apt mirror site to serve the same content on both HTTP and HTTPS ports. If the client want to use TLS, they connect to HTTPS. If they want to use plain HTTP, they connect to HTTP. Both sites serve the same content, which is just a series of flat files. AFAIK, the _client_ is responsible for determining the correct versions for the installed distro based on the indices.

This is what many mirrors already do:

[http://mirrors.lug.mtu.edu/debian/](http://mirrors.lug.mtu.edu/debian/)

[https://mirrors.lug.mtu.edu/debian/](https://mirrors.lug.mtu.edu/debian/)

<comment><USER4> If your installed version is configured for https, but is incapable of using TLS 1.2, because it's rather old, at some point soon, a modern mirror would no longer allow it to connect as 2019 (or maybe 2020) seems to be shaping up as the year to kill support for TLS 1.0 and 1.1. Meanwhile, an http config would continue to work.

</comment></comment></comment><comment><USER35> >This can lead to a replay attack where an attacker substitutes an archive with an earlier—unmodified—version of the archive. This would prevent APT from noticing new security updates which they could then exploit.

>To mitigate this problem, APT archives includes a timestamp after which all the files are considered stale[4].

Let's take a look at the repo spec then:

[https://wiki.debian.org/DebianRepository/Format#Date.2C_Vali...](https://wiki.debian.org/DebianRepository/Format#Date.2C_Valid- Until)

> The Valid-Until field may specify at which time the Release file should be > considered expired by the client. Client behaviour on expired Release files > is unspecified.

"Should", "may", and unspecified behaviour.

<comment><USER36> Well, of course the client behavior is under-specified -- sometimes the client is a human constructing a URL to download a .deb in a web browser over a corp- approved proxy, and then hand-installing the package with `deb -i`, bypassing all the security checks. Or sometimes there's a caching proxy (or three) between the client and the server. Or maybe IT has modified apt to only connect to repositories maintained by the IT department, and rejects sources from other domains.

</comment></comment><comment><USER37> Besides the privacy issue of sending package names clear-text, there is a second non-mitigated issue: Censorship.

An MitM could selectively block certain package being installed / update. Imagine using this to prevent: Bitcoin being installed / enforce a ban on crypto without backdoors / block torrent installations.

This doesn't work as well with the 'recognize package size' method because you need to download the entire package before you know the size. Given the need for Ack in TCP, an MitM can't just buffer data until they have the entire package size.

<comment><USER38> > This doesn't work as well with the 'recognize package size' method because > you need to download the entire package before you know the size. Given the > need for Ack in TCP, an MitM can't just buffer data until they have the > entire package size.

All they have to do is corrupt the final packet and the package checksum fails. An attacker only needs to buffer a single packet worth of data.

</comment></comment><comment><USER39> Yes, I just love Debian not being able to update libsexy2 because some stupid filtering box blocks "sex" in URLs.

</comment><comment><USER40> I bet there are many FLOSS advocates who don't read that page as being the result of a cost-benefit analysis. They read it as an inspiring story of the rebels winning one against the https Empire. Because I never see the caveat wrt apt that, "of course, we are a super edgy edge-case that should not be used as a model to rationalize a knee-jerk refusal to use SSL for common cases."

I say this because I've corresponded with such advocates about a completely common case for SSL-- setting up a LetsEncrypt certicate, say. The response I often get doesn't make _any_ sense unless I assume they read a page like this and remembered the feels while forgetting all the relevant details that separate apt from their common case.

</comment><comment><USER41> Even though the packages are signed cryptographically, there are possible risks when using an unencrypted connection.

A man-in-the-middle attack could simply work by serving you a signed, but outdated packages list, preventing your distribution from updating and leaving you vulnerable to security holes. It's the same attack an evil mirror could do as well.

So if you want to be really sure you should probably use two independent mirrors over an HTTPS connection.

<comment><USER42> The website mentions this towards the end (Replay attacks) : _To mitigate this problem, APT archives includes a timestamp after which all the files are considered stale_

The time stamp is described here[1], but it is not clear how the expiration date is decided.

1: [https://wiki.debian.org/DebianRepository/Format#Date.2C_Vali...](https://wiki.debian.org/DebianRepository/Format#Date.2C_Valid- Until)

<comment> julian-klode Well, defining the expiration date is up to the server. Debian picks a week. Ubuntu does not use it, I have started a launchpad branch last year, but um, I don't know launchpad, so it might take some more years ;)

</comment></comment></comment><comment><USER43> [https://onion.debian.org/](https://onion.debian.org/)

Scroll to the end for a very simple how-to.

</comment><comment><USER44> "HTTPS does not provide meaningful privacy for obtaining packages. As an eavesdropper can usually see which hosts you are contacting, if you connect to your distribution's mirror network it would be fairly obvious that you are downloading updates."

It is a dangerous mistake to decide what kind of privacy people need. Privacy should be absolute and without conditions.

What if you live in Iran? Some Ubuntu packages are already inaccessible due to government's pornography keywords censorship. E.g. I can't download "libjs- hooker" from this http link [http://archive.ubuntu.com/ubuntu/pool/universe/n/node- hooker...](http://archive.ubuntu.com/ubuntu/pool/universe/n/node-hooker/libjs- hooker_0.2.3-1_all.deb) from Iran. What if the government decides to censor the "tor" package?

</comment><comment><USER45> Do we now have a custom domain name on a per article basis?

I find it strange to have a site that is just about one thing that is not that important to most people on a custom domain. If there were pages and pages of information then yes this might make sense but there isn't.

Coming soon...

howtotieyourownshoelaces.com

The premise of this article per domain reminds me of 1998 when everyone thought that instead of search engines people would be typing in URLs, e.g. 'yescupofteaplease.com' so URLs like 'pets.com' were seen as goldmines-to-be.

<comment><USER25> Enjoy [http://islinuxaboutchoice.com/](http://islinuxaboutchoice.com/) and [http://heartbleed.com/](http://heartbleed.com/) . (-:

</comment></comment><comment><USER46> As an exploit analyst currently focusing on network traffic, can we stop all this fascination with SSL/TLS? TLS is incredible, but let's use the right tool for the job. Contrary to Let's Encrypt motto, applying TLS to _everything_ can be bad for security.

Let's Encrypt, when are you going to revoke placeimg.com's certificate? The site has been pushing Exploit Kit's malicious payloads since Jan 18 2019 via SSL. Many Flash/IE users are getting infected because most firewalls are unable to peer into SSL tunnels signed by you.

(To be fair, Let's Encrypt is not the only cert authority getting abused (Comodo, yes you))

</comment><comment><USER47> >To mitigate this problem, APT archives includes a timestamp after which all the files are considered stale

How often is this, practically? If I'm understanding this right, each new timestamp would come only with a package upgrade, meaning the time period is quite a long time indeed, long enough for a replay attack to work. I would argue that there should be a mechanism requiring a signed the-latest-package- is-X message updated at least every day or so.

Edit: it looks like this is actually what's going on. The page wasn't clear, but it is a metadata "Releases" file that is timestamped, not the packages themselves.

<comment><USER48> The security repository generally serves up a field in its metadata saying that the data shouldn't be trusted for more than 7 days, if it hasn't changed since 2014 when I encountered this duration as part of my day-job work. It's safe to assume the trusted duration hasn't increased, at least.

<comment><USER47> [http://security-cdn.debian.org/dists/stable/updates/Release](http://security- cdn.debian.org/dists/stable/updates/Release) is 10 days in the future. I still think it should be shorter, as it's pretty conceivable to exploit a 1-day vulnerability in that timeframe, but it's not that bad.

</comment></comment></comment><comment><USER49> Even though replay attacks will be of no use after some time, one could MITM during the vulnerable timeframe to prevent a critical (0day) security update from happening and therefore gaining control over the system, so if you're specially targetable, I believe it's totally worth switching to an HTTPS mirror instead.

Other than that, most should be safe ig.

One annoying thing is that the page name is misleading. apt does use/support https, it's just that Debian chooses for its default mirrors to it be optional.

<comment><USER50> If the attacker can MITM you, he can block your access too.

<comment><USER49> Blocking your access is very noticeable, not having one package upgrade is not.

</comment></comment></comment><comment><USER51> We like to talk about, say, the compromise of integrity and/or authenticity, information leak and so on, but they are not only things TLS/HTTPS was prepared for. Indeed, it is often overlooked that we have two kinds of exploitations in this space. I tend to label them as "active" and "passive". One of the best known passive exploitations is a JavaScript injection from ISP ---Comcast did it in 2017 [1] for example. They alone are typically harmless or annoying at best, but they are indicative of the real security problem lurking around, and often can evolve into active exploitations.

It might be probably true that APT is a simple service that does not require the full TLS capability. But APT is only prepared for active exploitations. Passive exploitations will effectively compromise the availability, by compromising the integrity in the relatively predictable way. I don't think APT is also prepared for passive exploitations---casual users will be much more prone to them.

[1] [https://forums.xfinity.com/t5/Customer-Service/Are-you- aware...](https://forums.xfinity.com/t5/Customer-Service/Are-you- aware/td-p/3009551)

</comment><comment><USER52> [https://usn.ubuntu.com/3863-1/](https://usn.ubuntu.com/3863-1/)

"Max Justicz discovered that APT incorrectly handled certain parameters during redirects. If a remote attacker were able to perform a man-in-the-middle attack, this flaw could potentially be used to install altered packages."

Far easier to do a MITM attack when apt isn't using https by default

Security is best in layers

</comment><comment><USER53> >Furthermore, even over an encrypted connection it is not difficult to figure out which files you are downloading based on the size of the transfer[2]. HTTPS would therefore only be useful for downloading from a server that also offers other packages of similar or identical size.

That seems like a false dilemma imho.

What's preventing APT from splitting up downloads into identically sized chunks of say 4kb?

</comment><comment><USER54> The "Date, Valid-Until" timestamp mitigates replay-attack, cool. But what if a vulnerability is discovered in a package and an MITM attack prevent you from downloading the new patched version, making you believe your system is up to date although it's not ?

<comment><USER10> No, apt then shows a message about failing to download fresh update lists. For security repo, the period it's 10 days.

</comment></comment><comment><USER55> There's also a current and ongoing "discussion" (I say pejoratively) regarding the same issue with VideoLAN. They can't (easily) use HTTPS because they don't control the mirrors. However the package download/update itself is also signed with GPG, thus guaranteeing tamper resistance.

[https://twitter.com/videolan/status/1086672630994927616](https://twitter.com/videolan/status/1086672630994927616)

</comment><comment><USER56> Apparently Windows Update uses TLS without encryption (null cipher) which still provides integrity and authentication: [https://twitter.com/swiftonsecurity/status/74533301869746995...](https://twitter.com/swiftonsecurity/status/745333018697469953). Debian's signing achieves the same result by signing the package list containing the packages' hashes with a known key.

<comment><USER34> As far as I'm aware, all Windows Update packages are also digitally signed.

</comment></comment><comment><USER57> > providing a huge worldwide mirror network available over SSL is not only a > complicated engineering task

Nobody said each mirror couldn't have its own certificate and its own domain. The host names of these mirrors are usually obtained from a trusted source.

HTTPS in addition to GPG signatures does no harm. It also means that one would have to compromise two entities, my distro's GPG key and my mirror's certificate.

</comment><comment><USER58> HTTPS adds very little privacy in this context. The mirror IP address is in cleartext.

Also, a smart observer might be able to detect APT traffic by analyzing the timing.

If you want privacy, APT supports Tor and there are mirrors providing onion services:

[https://blog.torproject.org/debian-and-tor-services- availabl...](https://blog.torproject.org/debian-and-tor-services-available- onion-services)

<comment><USER59> I had no idea APT supports Tor, very cool! I don't think the OP knows that either.

<comment><USER60> Apt has supported HTTPS and tor since forever. Aws offers https mirror. OP is about defaults. If you want you can switch to https or tor or http.

</comment></comment></comment><comment><USER61> Beware of strangers bearing gifts. There is a lot of https scaremongering from a certain subset of people with objectives one finds difficult to understand.

If they were genuinely concerned about privacy, security and surveillance they would have a lot to say about the out of control surveillance economy and its participants including their employers, the security model of browsers, mobile operations systems, javascript, systemic user stalking and hoovering up data.

Yet those stories are mainly driven by people outside the tech ecosystem and within there is mostly silence interspersed with vague economic justifications so there is something truly bizarre about https extremism on the grounds of privacy.

If your packages are already signed why do you need a middleman? That is a better trust model for a distributed Internet than a centralized CA that is not accountable to individuals and can be compromised by power. Here people are neck deep in business models stalking people online 24/7 and tracking their location and some are 'concerned' about protecting the list of packages you download?

<comment><USER62> > Here people are neck deep in business models stalking people online 24/7 and > tracking their location and some are 'concerned' about protecting the list > of packages you download?

That's believable. "There are a thousand hacking at the branches of evil to one who is striking at the root." \- Henry David Thoreau. Nice word "hacking".

Edit: Uh oh maybe Thoreau was pro CA.

</comment></comment><comment><USER63> At a research institute I worked for, we had a proxy server that intercepted http. The antivirus software on the proxy made it impossible to download some security updates once in a while. Fortunately it was possible to change the ubuntu URIs to [https://](https://) to use HTTPS and the broken antivirus sofwate did not intervene anymore.

</comment><comment><USER64> I wish APT used HTTPS if only to be able to tell my proxy "only allow HTTPS" and never have to worry about unencrypted traffic ever. Currently it's the only HTTP trafic my server are generating. Plus it's polluting my proxy logs as instead of having only the domain as it's the case for all the other log, I have the full URLs :)

</comment><comment><USER65> So they don't consider privacy beyond which host I contact, which is weird, because I'm pretty sure which version of packages I have is relevant to an attacker.

And in case of a zero-day exploit it would be really handy not having to wait for some global timeout during which I won't see any updates.

</comment><comment><USER66> I think https by default would be a wonderful addition. I understand the complexities from the project's perspective. I understand the users wants for privacy. If the project not defaulting to https is that much of a privacy issue for you, set up a local mirror then point your boxes to that. Yes, someone could potentially still snoop your local network. But if they can do that, you've got bigger issues to worry about than the fact that they can see what packages you're installing.

[https://www.debian.org/mirror/ftpmirror](https://www.debian.org/mirror/ftpmirror)

</comment><comment><USER67> Those arguments are invalid. Because debian/ubuntu fail to use https regimes like egypt/syria can track people who try install tor and they can cherry-pick block repos based on package name.

I'm sorry for not liking your favorite color and distro. Please deal with it.

And btw they don't digitally sign their package too (they sign separated meta data file having checksum which is not equivalent of embeding signature inside the package and validate it).

Compare that to yum/rpm which use secure https and signed rpm and signed metadata (both the medium and the payload are secured)

<comment><USER68> > _Regimes like egypt /syria can track people who try install tor and they can > cherry-pick block repos based on package name._

They say https would not add privacy in this context because the package size (almost) uniquely determine the package name. Why is this invalid?

<comment><USER37> You can block based on a name because you see the name before the package is sent.

You can't block based on package length, because you need to let the entire update through before you know the length. At that point, it's too late to block. Buffering the entire message doesn't work because TCP expects ACKs.

<comment><USER4> A) you can buffer and send acks to the server and then trickle the data to the client

B) in the interest of memory usage, you could not buffer, and send selective acks to the server -- once you decide to allow it, stop blocking the first data packet, and let the client ack that without the sack and let the server retransmit.

c) b, but for network efficiency, actually let the client receive all packets but the first, and sack them itself --- then when you do allow the first packet, the rest of the packets won't need to be retransmitted.

<comment><USER37> Ah, I confused the hierarchy between TLS and TCP. I thought the acks were protected by TLS, but they are not.

There are still timeout issues with the buffering, but it is a lot weaker defense.

</comment></comment></comment></comment></comment><comment><USER69> My corporate network used a transparent https man-in-the-middle proxy. That is, transparent as long as you're using a Windows machine that has been configured by the help desk and using applications that utilize the Windows certificate store, which doesn't include git, pip, or npm.

If pip or npm used gpg signing of packages rather than https, this wouldn't be a big deal. But as it stands, it's a nightmare on various Windows systems, Linux boxen, and Docker images to get the code you need.

</comment><comment><USER70> There are benefits in switching to HTTPS even if it's not done for security reasons. Namely, HTTP/2.

However, I'm not sure if the benefit in speed would be measurable for apt.

<comment><USER4> HTTP/2 is more likely to be a downside than an upside for apt. You would have additional framing overhead, and I can't see a benefit over pipelined http/1.1 be (which is definitely doable because all the requests are get for static files). Maybe header compression of the small headers could be a very minor win. Multiplexing within a tcp stream is at best neutral for this case, but probably negative: on the client side, it means the possibility of multiple partial files instead of one; on the server, it means more parallel demand on the file system.

</comment></comment><comment><USER71> > HTTPS does not provide meaningful privacy for obtaining packages.

That's a very subjective and circumstantial statement being passed off as fact.

Just because one can list a few scenarios where HTTPS wouldn't prevent a malicious actor from achieving their goals, doesn't mean HTTPS _does_ _not_ increase security/privacy for apt in other situations.

Further, in certain contexts, _guessing_ and _proving_ are not fungible.

</comment><comment><USER57> Lots of reasons to use HTTPS most notable is defense-in-depth.

A good design always aims to have multiple othorgonal security mechanisms in place.

</comment><comment><USER72> It doesn't need to, the actual security is implemented on a per-package basis.

[https://wiki.debian.org/SecureApt#How_to_manually_check_for_...](https://wiki.debian.org/SecureApt#How_to_manually_check_for_package.27s_integrity)

<comment><USER73> You may not want to make it too easy to find out which packages you have installed.

</comment></comment><comment><USER74> Using https is a basic security principle - not using it means you are doing security wrong and someone will be bitten by it.

They can argue and justify why they do it as much as they want, that actually shows a bit of security immaturity on their side.

</comment><comment><USER75> Layering HTTPS on top of the existing authentication system adds the obvious benefit of confidential communication, such that an eavesdropper can't see what packages you're downloading. There's an argument that you can infer the package from the size of download, but this is defeated by downloading partial regular sized chunks of the file with some overlap to mask the actual size.

The only sensible argument against HTTPS seems to be infrastructure cost. This made more sense in the past, but nowadays hardware crypto acceleration (e.g. AES-NI) is commonplace, and certificates can be obtained for free.

Ultimately it's up to mirrors to decide if they're willing to provide HTTPS.

</comment><comment><USER76> Previous discussion:

[https://news.ycombinator.com/item?id=16221563](https://news.ycombinator.com/item?id=16221563)

</comment><comment><USER77> With HTTPS, I need to trust every government in the world to not interfere with CAs.

With GPG, I only need to trust Debian to give me Debian.

Is there really a question about security here?

</comment><comment><USER78> Question: does APT work with HTTP proxies, which e.g. some company networks impose as the only means of internet connectivity?

</comment><comment><USER79> Finding out the packages based on file sizes? Nope - https would hide the sizes of individual files within the entire session.

<comment><USER60> Attacker can simply count the bytes. They can buffer one packet, send ack, and reject it based on bytes transferred. Also it's trivial know when next package starts. Apt doesn't steam all packages at once, it first send first package with http keep alive then waits until client orders the second package.

</comment></comment><comment><USER80> Every other vendor, uses https and also signs packages. Clearly they are all wrong and apt is the only correct one.

</comment><comment><USER81> another option for a "secure" connection is to use the apt onion services from tor [https://blog.torproject.org/tor-heart-apt-transport-tor- and-...](https://blog.torproject.org/tor-heart-apt-transport-tor-and-debian- onions)

</comment><comment><USER80> I guess that explains why I have install ca-certificates, and ubuntu doesn't ship with them.

</comment><comment><USER82> nothing justifies debian making you `apt-get install apt-transport-https` over `http` before you can add your own repos which are https only of course. but hey, it's debian.

</comment><comment><USER83> TL;DR

> _providing a huge worldwide mirror network available over SSL is [...] a > complicated engineering task_

> _A switch to HTTPS would also mean you could not take advantage of local > proxy servers_

<comment><USER84> My local Debian mirrors support HTTPS, and I assume other mirror sites worth their salt do too. Easy enough for Debian to redirect to your local mirror.

<comment><USER85> The local apt-cacher-ng instance I run in my office network cannot be redirected to by Debian, because cannot be aware of it. The apt client will need to build support for local proxies.

As it stands right now, apt-cacher-ng cannot work with https sources.

<comment><USER86> Fedora handles this use case beautifully with MirrorManager, which includes the EPEL repos as well. All of the logic is server side, when a yum/dnf client connects to the Metalink server to fetch a mirror list from our IP block it gets sent our internal mirror - I wish more distros had similar setups.

<comment><USER85> So, the public metalink server is aware of your internal mirror?

<comment><USER86> Yes, the mirror is configured as private and is only served to machines in my IP range - since it's on the internal network it does nobody else good to have access.

<comment><USER85> So every private mirror has to be registered with a central upstream. That sounds unpalatable.

<comment><USER86> It doesn't HAVE to be, I can manually edit my yum configs and specify the mirror - but using MM and Metalink means it automatically gets used.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER87> reminds me of this [http://n-gate.com/software/2017/07/12/0/](http://n-gate.com/software/2017/07/12/0/)

</comment><comment><USER88> I am seeing quite a bit of misinformation about how package managers work so I'd love to share what I have learned. I work with index files on a daily basis, and we might possibly generate more index files than any other organization on the planet. Here is my chance to share some of this knowledge! TLDR/Summary

We can trust the Release file because it was signed by Ubuntu. We can trust the Packages file because it has the correct size and checksum found in the Release file. We can trust the package we just downloaded because it is referenced in the Packages file, which is referenced in the Release file, which is signed by Ubuntu.

Some basic package manager principles

I work with APK, DEB, and RPM based package managers and each of them behave very similar. Each repository has a top level file, signed by the repository's maintainer, that includes a list of files found in the repository and their checksums. When your package manager does an update, it looks for this top level file.

        For DEB based systems, this is the Release file

        For APK based systems, this is the APKINDEX.tar.gz file

        For RPM based systems, this is the repodata.xml file

These files are all signed by the repository's gpg key. So the Release file found at[http://us.archive.ubuntu.com/ubuntu/dists/bionic/Release](http://us.archive.ubuntu.com/ubuntu/dists/bionic/Release) and is signed by Ubuntu and the gpg key is included in your distribution. Let's hope Ubuntu doesn't let their gpg key into the wild. Assuming that Ubuntu's gpg key is safe, this means that the system can verify that the Release file did in fact come from Ubuntu. If you are interested, you can click on the previous link, or navigate to Ubuntu's repository and open up one of their Release files.

Release file

In the Release file you'll see a list of files and their checksum. Example:55f3fa01bf4513da9f810307b51d612a 6214952 main/binary-amd64/Packages

9f666ceefac581815e5f3add8b30d3b9 1343916 main/binary-amd64/Packages.gz

706fccb10e613153dc61a1b997685afc 96 main/binary-amd64/Release

9eae32e7c5450794889f9c3272587f5e 1019132 main/binary-amd64/Packages.xz

5dd0ca3d1cbce6d2a74fcc3e1634ac12 96 main/binary-arm64/Release

The left column is the checksum, then the size of the file, and lastly the location of the file. So we can download the files referenced in the Release file and check them for the correct size and checksum. The Packages or Packages.gz file is the one we care about in this example. It contains information about the packages available to the package manager (apt in this case but again, almost all of the package managers behave very similar).

Packages file

Since we know that we can trust the Release file (because we have proven it was signed by Ubuntu's gpg key), we can then proceed to download the contents of the Release file. Let's look at the Packages file specifically as it contains a list of packages, their size, and checksum.

Filename: pool/main/a/accountsservice/accountsservice_0.6.45-1ubuntu1_amd64.deb

Size: 62000

MD5sum: c2cffd1eb66b6392f350b474e583adba

SHA1: 71d89bd380a465397b42ea3031afa53eaf91661a

SHA256: d0b11d1d27fe425bc91ea51fab74ad45e428753796f0392e446e8b2450293255

The Packages file includes a list of packages with information about where the file can be found, the size of the file, and various checksums of the file. If you download a file through commands like apt install and any of these fields are incorrect, apt will throw an error and not add it to the apt database.

It's time to debunk some myths!

Can an attacker send me a fake Release file?

Sure, but apt will throw it out because it's not signed by Ubuntu (or whoever your repository maintainer is like centos, rhel, alpine, etc)

Can an attacker send me an old index from an earlier date that was signed by Ubuntu that has old packages in it with known exploits?

Sure, but apt will throw it out because it will have a date (in the Release file) that is older than what is stored in the apt database. For example, the current bionic main Release file has this date in it: Date: Thu, 26 Apr 2018 23:37:48 UTC So if you supply it with a Release file older than that timestamp, it will throw it out because it is older than what it currently knows about.

I hope this helps clear the air!

Shameless plug. If you are serious about security and not just compliance, check out our Polymorphic Linux repositories. [https://polyverse.io/](https://polyverse.io/) We provide "scrambled" or "polymorphic" repositories for Alpine, Centos, Fedora, RHEL, and Ubuntu. We use the original source packages provided in the official repositories and build the packages but with memory locations in different places and ROP chains broken.

Installation

Installation is a one line command that installs our repository in your sources.list or repo file. There is no agent or running process installed. It is literally just adding our repository to your installation. The next time you do an `apt install httpd` or `yum install docker` you'll get a polymorphic version of the package from our repository. You can see it in action in your browser with our demo: [https://polyverse.io/learn/](https://polyverse.io/learn/)

What does it do?

Many of the replies in this post referenced an attacker tricking a server into an older version of a package that has a known exploit. We stop this. Even if you are running an old version of a package, with a known exploit, memory based attacks will not work on the scrambled package because the ROP chain has been broken or as we call it "scrambled". So with our packages, you can run older versions of a package and not be effected by the known exploits. This also means that you are protected from zero day attacks just by having our version of the package.

FREE! For individuals and open source organizations you can use our repositories for free. I hope you try it out!

</comment><comment><USER89> Should be HTTPS by default just for the privacy of downloading packages without any intermediaries knowing the package names. It could easily be collected and used in attacks against users of vulnerable versions.

<comment><USER60> Did you even read the article? This is explained at length. Https doesn't give you that benefit since attacker can still single out package based on length and in https address is in plaintext.

</comment></comment><comment><USER90> I feel the HTTPS hysteria is going too far. For some strange reason people started to consider it a security panacea, without often understanding its limitations. Accusations against APT are a perfect example (it's built-in security mechanisms are superior to what HTTPS has to offer).

<comment><USER91> Secure against modification, is not the same as secure against information leakage and privacy.

Using a non-encrypted connection means that it's trivial to work out what packages you download. Using a secure connection at least makes it 1 step harder to infer that information.

However, whether packages should be kept private is all-together another question. I argue that OS updates and packages related to that do not need to be kept private, but applications packages do.

<comment><USER90> > Using a non-encrypted connection means that it's trivial to work out what > packages you download. Using a secure connection at least makes it 1 step > harder to infer that information.

It is trivial still in the case of APT. That's exactly my point: people start believing HTTPS will protect them against many attack vectors it doesn't. That's OK for uninformed people to believe in the magic of a padlock in the address bar, but technical folks should really know better. If I have access to your network traffic and intend to see which packages you download by apt, I will do it irrespective of whether you use HTTPS or not.

</comment></comment></comment><comment><USER92> I don't care for home-brew security, I need something standard.

Also... the firewall at work breaks APT HTTP pipelining. Very very annoying. It would not be able to do so if APT was using HTTPS.

<comment><USER93> The signature method used by Linux mirrors has existed longer than HTTPS has, there's nothing homebrew about it.

<comment><USER92> I trust the file signatures, but if you need to write a full article arguing something is secure then it can be made more secure by making the system simpler and more standard.

<comment><USER94> You are going to become very upset when you discover the simplicity of TLS.

</comment></comment></comment></comment><comment><USER93> One thing that wasn't touched on - the mirror network. There are 100's of mirrors for all the major distros ran by third parties, if e.g. Debian wanted apt over HTTPS, they would need to hand out a debian.org SSL cert+key to all of them.

(And convince them all to take the CPU overhead hit of TLS)

EDIT: Since I can't reply to all the downvoters, I'll add here.. LetsEncrypt does not solve this. [http://us.archive.ubuntu.com/](http://us.archive.ubuntu.com/) \- that goes to likely 10's of different mirrors. Which one will the LetsEncrypt verification call hit?

<comment><USER95> What? No.

The mirrors would just need to install a letsencrypt-compatible client and setup SSL via that.

On any modern CPU since 2011 or so, TLS overhead is below a percent and a decent cheap dedicated box should be able to saturate a 1 Gbps uplink.

<comment><USER4> Actually, no, the mirrors may not be able to verify themselves with let's encrypt.

A hostname like ftp.us.debian.org resolves to many different mirrors, and may not resolve consistently around the world -- if that's the case, let's encrypt will not be able to verify the hosts through http challenges.

Also, 1gbps is pretty small. I can't find any documentation on traffic, but I'd imagine mirrors in popular places are at least on 10gig.

<comment><USER95> I have a 10Gbps uplink on my personal server and I can easily saturate it without getting capped on CPU or RAM.

LE offers other challenges to verify hosts, like DNS verification. DNS verification can be done easily with an external API for mirror owners to hit (most ACME clients offer DNS challenge with the standard update protocol for DNS which can be secure appropriately).

<comment><USER4> Mirror owners don't control the DNS for debian.org.

Debian could get the certificates, but getting the certificates was never the issue -- some CA would be happy to issue certificates for little or no cost to help Debian and gain mindshare. Coordination between 3rd party, volunteer mirror owners and the Debian organization is the issue.

What kind of CPU and traffic patterns are you using to hit 10 gbps of TLS protected traffic?

<comment><USER95> >What kind of CPU and traffic patterns are you using to hit 10 gbps of TLS protected traffic?

Mainly serving a file directory with apache with files ranging between 100M and 1G in size. Should be easily comparable to Debian or Ubuntu repositories.

<comment><USER4> Debian mirrors are going to get a lot of _very_ short connections for people checking if there are any updates frequently. Those are effectively zero bandwidth, but you need to do a full TLS handshake, which is the most expensive part.

<comment><USER95> A dedicated server of decent size should easily be able to handle this kind of traffic.

<comment><USER4> I agree; you would need a dedicated server of decent size to handle this type of traffic.

Debian relies on an assortment of volunteer servers of unknown size, and they're not dedicated.

<comment><USER95> A dedicated server of decent size at the right hoster starts at 25€ a month. That is not that much and I bet most of the volunteer servers are already above this price range.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER96> Another scary thing is that despite not including it, when you do have a need for it you're advised to install apt-transport-https[0]. I can't remember the specific package that requires me to go this route, but it always reminds me "oh they're not using https", I'm surprised they don't pull over SSH or something, doesn't git work with SSH as well?

[0]: [https://packages.debian.org/sid/apt-transport- https](https://packages.debian.org/sid/apt-transport-https)

Edit:

It was VSCode:

[https://code.visualstudio.com/docs/setup/linux](https://code.visualstudio.com/docs/setup/linux)

So Microsoft provides apt packages over https, but Debian doesn't.

<comment><USER97> > So Microsoft provides apt packages over https, but Debian doesn't.

See [https://deb.debian.org/](https://deb.debian.org/) \-- it works just fine.

HTTPS might prevent some kind of surveillance, entities (ISPs, governments, any other MITMs) from determining what kind of packages you're installing, and this might be beneficial, but plain old unencrypted HTTP is often faster if these aren't a big concern (they likely aren't in most developed nations and for most occupations). Lack of encryption overhead as well as transparent proxies being able to serve files are huge boons to this.

<comment><USER98> The article points out that HTTPS offers almost no protection against this kind of surveillance, since packages are almost uniquely identifiable by size.

<comment><USER95> With HTTPS you would be able to use HTTP/2 which means you can multiplex a single connection and once you download more than 2 packages, identifying which you installed is impossible.

With plain HTTP it remains possible no matter how much pipelining you do.

<comment><USER99> Well, maybe; how do HTTP/2 servers allocate bandwidth? If they do it per- stream, you can still identify the size of each package by watching the total connection bandwidth decrease when each package ends.

If they allocate it for the whole connection, then all they get is a total, which still gives the attacker some information (there's a limited combination of packages that sum up to that number).

<comment><USER95> >(there's a limited combination of packages that sum up to that number).

That is a really hard problem to solve. See the knapsack problem.

Imagine I downloaded 5 packages. Debian has about 68'000 packages. That means there is a total of 1 septillion combinations.

If you could check 1 trillion package combinations per second, it would only take 20 thousand years to solve (40 thousand worst case).

<comment><USER100> You are assuming, that all packages are equally interesting (and your math does not account for package dependencies).

In practice, attacker either does not care about your packages, in which case hiding that information gains you nothing, or wants to be alerted, when you (or anybody else) install one of few specific packages. Those combinations can be computed in advance and identified in traffic.

<comment><USER95> Not necessarily, I think you underestimate the complexity of this attack.

Even if you were interested in a few packages, if any additional packages are mixed in or if dependencies are already installed, this problems become a lot harder again.

While HTTPS doesn't make such an "attack" impossible, it makes it very hard and compared to HTTP the attacker cannot inject or replace data (replay attacks are possible with APT on plain HTTP)</comment></comment></comment></comment></comment></comment></comment></comment>


Geodesic Plugin for MapboxDraw - <AUTHOR> https://github.com/zakjan/mapbox-gl-draw-geodesic <comment><AUTHOR> Geodesic plugin for MapboxDraw to draw geodesic lines, polygons and circles. Geodesic calculations are isolated inside the plugin, keeping the developer using the plugin abstracted away from the calculations.

There is another compatible plugin: [https://github.com/zakjan/mapbox-gl-draw- waypoint](https://github.com/zakjan/mapbox-gl-draw-waypoint)

Disclaimer: I'm the author, and development was sponsored by [https://maritrace.com/](https://maritrace.com/)</comment>


Show HN: HoloLens D3D Keyboard - <AUTHOR> Hololens full-size keyboard utility, for D3D build, for in-App username / password / url / comment entries. Demo video: http://www.youtube.com/watch?v=lVKk6iiSlVo
Windows App Store 30-day free trial link (only available on HoloLens): http://www.microsoft.com/en-us/store/p/hololens-d3d-keyboard/9nblggh40s2r
Appreciate your feedback! <comment><USER2> This app does not work on my device.</comment>


Renaissance – control your computer with a musical instrument - <AUTHOR> https://github.com/ColdSauce/renaissance

<comment><USER3> You seem a bit confused as to what the target audience for this is - <USER2> unexpected given the single day the project has been online, but a topic worth discussing nonetheless. Your initial demo is to control a mouse on a Windows desktop, but in the README it states that Android will receive the first release since "desktop apps are virtually dead now." I won't pretend to know what platform this idea might take off on, and I think picking Java gave you a good set of choices, but you should nail down your stance so you and the people interested know where the project is headed.

<comment><AUTHOR> Fair point. I agree that the wording in the README was a bit confusing.

The current readme reflects the direction of the project.

Thanks for the advice!

</comment></comment><comment><USER4> This is amazing! Could be a useful tool to teach kids how to play instruments too, if you worked it into a game.</comment>


What happened to Cory? - <AUTHOR> https://docs.google.com/document/d/15JokX8thp1TxG_I9aodYUxDwRdvdselVQmUwEjOFXGQ/preview# <comment><USER2> Actually I think Cory was in the wrong. And the managers actually __did __everything correctly. They even talked with him, they didn 't even fired him. They just clearly stated that the topic was closed and that he shouldn't have raised it because there was always a negative discussion about the topic which actually led to "performance" problems of certain groups within google.

actually a workplace should be your workplace, not your political debate site.

yes free speech is important. yes it's bad if people discriminate. but did he had other choices than just raising the topic again, in a "google" public forum? yes he could! but he did not, he could've mailed his HR guys or talked with his manager. or both.

but raising the same topic again and again and the same disucssion occurs with no end in sight, is always a bad idea (regardless of the topic).

p.s. i go to work because i love programming and not because i can raise political discusissons over and over again with my co-workers.

</comment><comment><USER3> Sounds like Urs et. al. were less concerned with shutting down pro-diversity speech than they were with Cory's tone, which they seem to believe per se was causing division. Cory didn't seem to be self-aware enough to understand that.

Since the external links are private, it's hard to know if we're comparing apples to oranges; but it's informative to contrast the reaction to Damore's single memo with this fellow's repeated, willful flouting of his superiors' requests to moderate his tone and stop posting long enough for the topic to calm down. It seems like they gave Cory a lot of second chances and leeway.

<comment><USER4> It's proper to flout invalid requests.

</comment></comment><comment><USER5> Current US law about workplace safety proscribes a certain type of professionalism. The kind that says, "We can have a heated discussion on how text based web search works, but not anything else." There are other ways to be professional, but those seem to be difficult to operate usefully in multiperson US companies, and this way isn't bad.

I think it's appropriate to ignore whether a co-worker thinks the moon landing was faked, or what sports team they like, or if those idiots they heard about in the news are a big deal or no deal at all. That's simply not what the workplace is for. To the extent those discussions happen, I view it as a partially-voluntary leaking of their personal lives. Nothing to condemn or engage in, just politely ignore it until your lunch finishes in the microwave. The idea that people would want to express _any_ views in some type of organized way inside their employer's organization makes little sense to me -- I find it unprofessional but not outrageous or anything.

I think it's clear whether something is an on-topic professional comment. All discussion of news stories is a 'no' unless it is e.g. a new regulation on the industry or their customers that was just announced.

</comment><comment><USER6> I don't mean to sound dismissive or anything, but Cory who? There's very little context in the link about what this was or who it was.

<comment><USER7> Context: [https://gizmodo.com/ex-google-employees-memo-says- executives...](https://gizmodo.com/ex-google-employees-memo-says-executives- shut-down-pro-1821996513)

<comment><USER6> Thanks, I had missed that article, and this was definitely missing that extra information.

</comment></comment></comment><comment><USER8> The entire thing is a great case of cultural challenges. From one hand, you'd like to encourage open discussion about everything. Because innovation and out of the box thinking. On the other, discussing some ideas leads to conflict. A culture of free ideas is one of the best ways to attract talent, restricting it will commercially detrimental. So is loosing talent over conflict like this. It is a leadership challenge. A very complex one. But one that does not seem to be handled with a ton of grace. Maybe because there is no good solution

</comment><comment><USER9> Cory is lacking in self-awareness.</comment>


Anil Dash: Don't let Twitter, Facebook, Google be the only game in town - <AUTHOR> http://www.cnn.com/2010/OPINION/01/22/dash.twitter.shutdown/index.html?hpt=T2 <comment><USER2> It'd be interesting to see how far we could take a loosely coupled network for Wordpress installs towards this goal. How actually to make that work is something I've been pondering lately.

Also, I think the work that Dave Winer's been doing around realtime RSS is a huge step in this direction.</comment>


Fully Autonomous Robots: The Warehouse Workers of the Near Future - <AUTHOR> http://www.wsj.com/articles/fully-autonomous-robots-the-warehouse-workers-of-the-near-future-1474383024 <comment><USER2> I was an order picker once in a grocery warehouse, it helped put me through school. I was also a paperboy, a film projectionist, and a bike messenger. All automated now or will be soon. Later I became an editorial illustrator, and when the smartphone hit critical mass I lost 80% of my running gigs over a couple months, print media collapsed very suddenly: that hurt.

So when Uber drivers complain about self driving cars I just cackle maniacally.

<comment><USER3> Sort of a tangent but I agree in the character generation from working. I had it too, and I appreciate having had it. I wonder if this really is part of the reason for later launches for millenials.

</comment></comment><comment><USER4> This is what fascinates me about discussions on subjects such as basic income. BI may not be the right answer and it may not even be viable. However, the writing is on the wall and big changes are coming sooner than we expect.

<comment><USER5> With automated warehouses and autonomous cars and automated fast food and everything else just think of all the jobs lost. I read articles saying the jobs will be offset with new jobs, but no one bothers to explain what. Also when everyone is unemployed how are they going to pay for these automated goods.

When I was in highschool a job was to build character and get you ready to survive in the world. Maybe this is why more kids are living with their parents longer. I don't agree with basic income, because I believe like welfare people will find ways to exploit it. The people who have a drive to work will seem like they are getting less.

I am not opposed to this type of automation. It is just worrisome what impacts it will have that us little guys cannot control..

<comment><USER6> _I don 't agree with basic income, because I believe like welfare people will find ways to exploit it._

The whole point of UBI is to avoid means testing, so there's no reason to game it. Of course, if anything, we won't have actual UBI because it's most likely politically untenable because of the exact perspective you have, but some form of extended welfare, which requires some kind of means testing, which is then gameable.

 _The people who have a drive to work will seem like they are getting less._

If it's truly UBI, then these people can avoid the feeling like they are getting less by not working, or taking a job closer, or starting their own company, that doesn't require driving to work. "I don't want to drive to your office" becomes an acceptable and usable reason to quit your job because you don't actually _need_ a job under UBI.

<comment><USER7> Erg... English is most annoy. I'm pretty sure OP meant "drive" as in motivation, not the car type :)

But yeah, your point it still kind of valid.

Point being, if you are motivated to work then you inevitably get more. If you simply rely on UBI then you get enough for cheap rent and cheap food.

Pretty sure a UBI doesn't allow for a new xbox games and a carton of cigarettes every week.

<comment><USER8> Plenty of homeless people will waste it on cigarettes. I'm actually for it. These are precisely those people for which the currently rampant paternalism is _NOT_ helping. It just further communicates that they are no longer worthy of bettering themselves. This attitude is pervasive and exists as a direct reflection of our neo-liberal capitalistic state. It floats around like pollution in the aether between egos and stifles anyone sensitive to it. Once that's happened the means-testing merely communicates that they are being measured and slotted. That anything known of their condition will only be used to humiliate them further; There is no help. I don't have statistics at hand but these people (hobos) largely aren't reproducing. Can they not just smoke and be comfortable? Why are we so desperate to police the abuses of these systems? Making proper use of them is already it's own reward. Let's just allow people to punish themselves for as long as they feel they need to. Without giving them any more evidence that they ought to...

</comment></comment></comment></comment></comment><comment><USER9> Interesting. The video doesn't show the mechanism by which the little mobile units grab and load a case of something from the racks. That's the classic hard problem with automated storage and retrieval systems. Most of them use special pallets or totes. This one doesn't. That's a big win. Everything else in that video is off the shelf technology.

This is a wholesale warehouse; pallets go in and pallets with different mixes of boxes go out. It's not a fulfillment center where small customer orders are picked. Amazon is working on that, but so far they're only semi-automated. Kiva robots bring the racks to a human picker, who takes an object out of one bin and puts it in another. Amazon has a competition for robots to do that.

<comment><USER10> I bet the little robots act as platforms for individual boxes. Little mobile conveyor belts. When all of the products are relatively large boxes that can be stacked you don't really have to pick things up. That is the big difference between individual customer fulfillment. Amazon has to pack a bunch of odd shaped items. When everything goes comes in and goes out on pallets you know all the components are pallet friendly. It is a nice niche to be in.

</comment></comment><comment><USER11> "... There aren't enough young people coming into the workforce who really want to work in warehouses."

Interesting comment, passing the blame to people. A better comment would have been, "we don't want people to do these type of jobs because we're building a better utopia of the future ..."

<comment><USER12> Who _does_ want to work in warehouses? Haven't we all seen those reports of how uncomfortable it, both physically (hot heavy crowded work) and mentally (agressive metrics and no job security)?

The more uncomfortable the job is the more comfortable I am with automating it away.

</comment></comment><comment><USER13> Mrs Clinton and/or Mr Trump had better figure something out this time round, or next time this country is going to have a large group of grumpy blue collar workers out of a job and a new crop who never got the chance to have one, if they don't figure out quick how to address these structural changes. It won't be one made up of largely rustbelt blue collar workers whose jobs got outsourced overseas or across the border.

This is cutting unskilled and low-skilled labor at the knee. The only bastion left is one that domestic workers are loath to do and that is harvesting and meatpacking --but those will likely fall to automation as well.

This time the giant sucking sound wont be coming from south of the border but rather from the army of robotic workers.

<comment><USER9> Automated lamb deboning system.[1] Starts with an X-ray to find the bones. Laser scanners build a 3D image. Robot saws and knives cut at the right places. The system is kind of slow and rather complicated. About two more generations of development are needed.

Automated fruit picking for delicate fruits is still a research project. There are lots of slow demo systems with robot arms. 1 apple per second is the state of the art.

In practice, most fruit is picked using mechanical systems such as a tree- shaker that makes all the fruit fall off.

[1] [https://www.youtube.com/watch?v=za2dsB0qrMg](https://www.youtube.com/watch?v=za2dsB0qrMg) [2] [https://www.youtube.com/watch?v=TBcWZcjXr-I](https://www.youtube.com/watch?v=TBcWZcjXr-I)

</comment></comment><comment><USER14> Everyone thinks it's the blue collar jobs that will be lost to automation, but I guess a lot of office jobs will be too.

Especially jobs where the data is easily machine readable, processes can be online, and the work doesn't require human negotiation.

</comment><comment><USER15> I don't know why you would complain about it.

1\. Automate all low skilled jobs.

2\. Create technocratic city-states. Maybe seasteading, maybe something else.

3\. Robots won't be able to vote on things like Trump or Brexit.</comment>