Who Killed James I? - <AUTHOR> http://www.historytoday.com/alastair-bellany-and-thomas-cogswell/who-killed-james-i <comment><USER2> On a side note, I clicked the 'subscribe' button to check that offer of £5 for 3 issues. It looks legit, but the 'interesting' bit is that you are forced to use a direct debit to get that offer.

Yeah, right, I'm so going to setup a direct debit (presumably auto-renewable) for 5 quid...

Now I would have subscribed, after my cheap demo 3 issues, and possibly (and perhaps likely) bought a yearly subscription afterward, but now that I've seen that rather questionable trick, I won't.

</comment><comment><USER3> Possibly no one (according to this account), but the rumors that Charles I may have been involved made it much easier to behead him 20 years later.

</comment><comment><USER4> aka James vi</comment>


Ask HN: Are containers really more light weight than virtual machines? - <AUTHOR> I was talking to an expert on containers and asked them "what is the real benefit of containers over VMs" and he said confidently "containers are more light weight than VMs".
But I wonder is that true?
Is "weight" the core value of containers over VMs?  And are containers more light weight anyway? My experience with VMs is that small ones are only 25MB or so - seems pretty light weight to me. <comment><USER2> Depends on how they are built.

If you build a container with everything including the kitchen sync, then that will be pretty heavy.

If you really strip down your VM, you can make it very light.

But, by default, your average container (properly built) will probably be lighter in weight than your average VM.

</comment><comment><USER3> "Weight" in this context refers to execution and memory overhead, not file size.</comment>


The Freedom of Fast Iterations: How Netflix Designs a Winning Web Site - <AUTHOR> http://www.uie.com/articles/fast_iterations/ <comment><USER2> You want a great example of this, look to the man who more or less invented the principle- as I've used news.YC over the past couple of weeks it seems every few days, as simple as the site is, very nice features keep popping up. Nothing disruptive, nothing gaudy, just smooth and steady increase in value. In fact, if you want a great example of a site that's responsive, that knows how to grow a community, that knows how to fill itself with quality community content, that does something people want and does it simply and well... Thanks for leading what you preach.

</comment><comment><AUTHOR> It's cool how even though there is a lot of noise surrounding advice about startups a bunch of precepts have an air of unanimity. "Release early and iterate" would be one of those.

<comment><USER3> Here's a blog post by Aaron Swartz: http://www.aaronsw.com/weblog/rlrr

In there, he proposes that "release early, release often" works in open source, but is less applicable to a wide public who are not expecting something unpolished and incomplete. When you only get one chance to make an impression, the unveiling might need a lot more consideration. Of course, then your product might never even see the light of day because it's been tied up for so long.

This is a different view from that in PG's essay, and it just shows the added complexity in decision-making a startup must go through.

</comment></comment><comment><USER4> I would have appreciated if the article covered few common pitfalls and how NetFlix wrestles them when using the fast iteration cycle.

</comment><comment><USER5> Adding features is one thing, changing the app metaphor is another. I'd say that the latter has to be right at launch.</comment>


How Unreliable is UDP? - <AUTHOR> http://openmymind.net/How-Unreliable-Is-UDP/ <comment><USER2> UDP is just about as reliable or unreliable as IP. It's a shim on top of IP to let unprivileged users send IP datagrams that can be multiplexed back to the right user. (Hence the name: "user" datagram protocol.)

Lots of people talk about a "TCP or UDP" design decision, but that's usually not the relevant question for an application developer. Most UDP applications wouldn't just blat their data directly into UDP datagrams, just like almost nobody blats their data directly into IP datagrams.

Typical applications generally want _some_ sort of transport protocol on top of the datagrams (whether UDP-in-IP or just plain IP) that cares about reliability and often about fairness.

That could be TCP-over-IP, but some (like MixApp, which wanted TCP but not the kernel's default implementation) use TCP-over-UDP-over-IP, BitTorrent uses LEDBAT-over-UDP, Mosh uses SSP-over-UDP, Chrome uses QUIC-over-UDP and falls back to SPDY-over-TCP and HTTP-over-TCP, etc.

The idea that using UDP somewhere in the stack means "designing and implementing a reliable transport protocol from scratch" is silly. It doesn't mean that any more than using IP does.

The question of SOCK_STREAM vs. SOCK_DGRAM is more typically, "Does the application want to use the one transport protocol that the kernel implements, namely a particular flavor of TCP, or does it want to use some other transport protocol implemented in a userspace library?"

<comment><USER3> > UDP is just about as reliable or unreliable as IP.

Ok.

> It's a shim on top of IP to let unprivileged users send IP datagrams that > can be multiplexed back to the right user. (Hence the name: "user" datagram > protocol.)

It's mostly a way to send data in such a manner that you don't need to do the full 'call-setup-data-transmission-and-terminate' that would be required for other virtual circuit based protocols such as TCP when you don't need all that luxury. So for protocols that carry small amounts of data in a manner where retrying is not a problem and where loss of a packet is not an immediate disaster. It's also more suitable for real-time applications because of this than TCP (especially true for the first packet). Because of the fact that there is no virtual circuit a single listener can handle data from multiple senders.

The 'USER' does not refer to unprivileged users but simply to users as opposed to system packets (such as for instance ICMP and other datagram like packets that are not usually sent out directly by applications). So it's not a privilege matter but a matter of user-space vs system modules elsewhere in the stack.

> Lots of people talk about a "TCP or UDP" design decision, but that's usually > not the relevant question for an application developer. It absolutely is.

> Most UDP applications wouldn't just blat their data directly into UDP > datagrams,

They usually do exactly that.

> just like almost nobody blats their data directly into IP datagrams.

You're comparing apples with oranges, IP is one layer and TCP and UDP are on another. So you'd have to compare UDP with TCP and then you're back to that design decision again.

> Typical applications generally want some sort of transport protocol on top > of the datagrams (whether UDP-in-IP or just plain IP), that cares about > reliability and often about fairness.

Fairness is something that is usually not under control of the endpoints of a conversation but something that routers in between influence. They can decide to let a packet through or drop it (this goes for TCP as well as UDP), if a line is congested your UDP packets will usually (rules can be set to configure this) be dropped before your TCP packets will be in spite of the fact that TCP will re-try any lost packets. UDP packets can also be duplicated and routed in such a way that they arrive out-of-order.

> That could be TCP-over-IP, but some (like MixApp, which wanted TCP but not > the kernel's default implementation) use TCP-over-UDP-over-IP, BitTorrent > uses LEDBAT-over-UDP, Mosh uses SSP-over-UDP, Chrome uses QUIC-over-UDP and > falls back to SPDY-over-TCP and HTTP-over-TCP, etc.

Running alternative protocols packaged inside other protocols is a time honored practice. See also: TCP over carrier pigeons and tunneling HTTP over DNS traffic (effectively using UDP). This is not in any way special, it's just a means to an end.

> The idea that using UDP somewhere in the stack means "designing and > implementing a reliable transport protocol from scratch" is silly. It > doesn't mean that any more than using IP does.

It actually comes down to exactly that. If you use UDP as your base _and_ your application requires reliable transmission of data then you're going to have to deal with loss/duplication/sequencing at some other point in your application or put another (pre existing) protocol on top of it in order to mitigate these.

If your application can tolerate those errors (or if they are not considered errors) then a naive implementation will do.

> The question of SOCK_STREAM vs. SOCK_DGRAM is more typically, "Does the > application want to use the one transport protocol that the kernel > implements, namely a particular flavor of TCP, or does it want to use some > other transport protocol implemented in a userspace library?"

TCP is the default for anything requiring virtual circuits if you have demands that are not well describe by that model and/or need real time, low overhead and you're willing to do the work required to deal with UDPs inherent issues (if those are a problem) then you're totally free to do so.

But the question is usually not 'do I need TCP', it usually is 'how do I avoid re-implementing TCP if I need its features'.

It's a tough choice because at a minimum it means that you're going to have to write software for both endpoints.

This is one of the reasons why we see HTTP over TCP in so many places where it wasn't originally intended: it is more or less guaranteed to be well tested and there are tons of tools available to use this protocol combination, especially browsers, fetchers and servers in all kinds of flavors. For UDP that situation is much less rosy and using UDP always translates into having to do a bunch of plumbing yourself.

<comment><USER4> > You're comparing apples with oranges, IP is one layer and TCP and UDP are on > another.

That is what the books say, but I don't think it is right. When you consider that raw IPv4 doesn't work in practice because of NAT, UDP is a defacto minimum internet layer in practice.

<comment><USER3> NAT is a kludge. The existence of NAT does not remove the fact that both TCP _and_ UDP are layered on top of it and there is plenty of stuff happening that is layered directly on IP besides UDP, for instance ICMP.

</comment></comment></comment></comment><comment><USER5> I'll the share the comment here that I posted on the article page.

OP, you should have done latency tests as well. Whether packets arriving out- of-order is unacceptable to an application or not depends on the time distribution of packet delays. If, for example, 99.9% of packets arrive in- order within, say, 10 ms, then that is perfectly acceptable to use for video streaming, for example. Even if that means a packet is 1000 packets late, this just requires a bigger buffer on the receiving end. As long as it's below a certain duration, out-of-order packets aren't necessarily a problem even for live content.

The grouping of 5 packets together and thus checking whether a packet arrive out-of-order more than six places seems arbitrary. If, for example, a packet arrives out-of-order 50 packets late, but it's actually only 1 ms late, then it's not a problem. If it comes along 1 second too late, then it _is_ a problem.

<comment><USER6> Yes, you're right. That was obvious once I got the data. I still thought what I had was worth writing about :)

As you say, that grouping _is_ arbitrary and, it was more likely to include packets that shouldn't have been included (multiple seconds later) than those that did. Though, that's more todo with the fact that I sent so few packets (5-10) in each burst.

</comment></comment><comment><USER7> "I would tell you a joke about UDP, but you probably wouldn't get it."

<comment><USER8> A UDP packet walks into a bar.

A UDP packet walks into a bar.

<comment><USER9> A walks UDP packet bar a into.

<comment><USER10> I think this is definitely a more accurate representation of the joke. Although maybe drop one of the words. :)

<comment><USER11> A walks UDP packet bar into bar.

?

</comment></comment></comment></comment></comment><comment><USER12> I've seen UDP fail drastically (dropping north of 50% of packet loss) on controlled networks. Once you hit a certain congestion on a network or a network buffer fills up you suddenly things go a bit sideways. I'm not sure if switches and OS's consider UDP lower priority then TCP or simply that TCP has mechanisms to recover from packet storms, but TCP continues to "function" where UDP starts hitting the floor en masse.

Conversely I've pondered using UDP packets as a "canary in the coal mine" for networks to monitor it's health.

<comment><USER13> This is how congestion works: queues are not infinitely deep, so the only way to deal with congestion is to (at some point) start dropping packets.

TCP (that is, most TCP flow control algorithms) specifically uses packet loss as an indicator of network congestion / how much bandwidth is available, and will back off and retry dropped packets at a lower rate; hence why TCP continues to function (albeit at degraded performance). UDP has no such mechanism, hence you see firsthand the dropped packets. A well-written application should back off in such a scenario so as not to flood the network, but of course many won't.

(Of course TCP may very well be prioritized over UDP! It's just not necessary to explain your observations.)

<comment><USER14> It's often the opposite. In order to prevent the Internet from melting, TCP does an elaborate dance with all the TCP speakers to back off and avoid congesting the network. UDP does no such dance, and so can monopolize traffic at the expense of TCP.

Obviously this presumes that the UDP protocol in question has some mechanism for handling lost packets. But, for instance, if you're doing lossy video or forward error correction, end users do not deal directly with lost packets.

</comment></comment></comment><comment><USER15> For a fuller picture:

• Timestamps are necessary: UDP loss is bursty, when any component anywhere along the path (including the endpoints) is momentarily too burdened (buffers, CPU, wires) to forward every packet.

• Try packets larger than the 'path MTU' between the two endpoints: any packet larger than that is fragmented somewhere en route, and then loss of any one fragment causes the full UDP packet to be lost.

• Try alongside other traffic on the endpoints, and note that TCP streams can't find their achievable rates (or split the available bandwidth amongst themselves) without hitting the packet-dropping pressure that also affects UDP. So perhaps try with 1, 2, or more TCP streams trying to max wire-speed, between the same machines at the same time.

Note also you can create arbitrarily-high loss rates by simply choosing to send more than the path and endpoints can handle. (Let one side send X thousand in a tight loop; on the other side only check for 1 per second for X thousand seconds. Many will be lost, depending on how much you've exceeded the available bandwidth/buffering between the send and receive.)

<comment><USER16> Avoid timestamps: computer are not perfect, their clocks tends to drifts in non predictable ways (even with NTP). I suspect seen quite real bugs due to clock drifting (and linux non monotonic timer (which impacted python) for a while (but problem is solved)) resulting in packet drops because networks stack don't like to see packets arriving in the future when I was using pyzmq.

[http://lists.zeromq.org/pipermail/zeromq- dev/2013-October/02...](http://lists.zeromq.org/pipermail/zeromq- dev/2013-October/022996.html)

My intuition is we are close to relativistic problems. (Can't prove it, I have to code a kikoo lol form for tomorrow).

</comment></comment><comment><USER17> For the best of both before SCTP exists, if ever, RUDP (Reliable UDP) [1] where needed is what most game developers/engines use on the networking layer such as RakNet [2] (recently bought and open sourced by Oculus), enet [3], and the many based on those.

RUDP or similar systems allow network messaging to ack when needed and unreliable by default which is fine for positional updates with some missing data using interpolation and extrapolation to simulate missing data. Global game states and determining the winner/ending the level might need a reliable call.

With UDP and only some reliable calls you drastically improve real-time performance with less queueing. TCP is ok for turn-based or near-real-time but high action real-time games almost all use UDP with a RUDP twist. Using a mix can also be harmful [4] so the best option is RUDP where needed, default to UDP.

UDP is great because it is a broadcast, almost like TV/radio in that you can show the data you receive and smooth the rest. Although, TV/radio needs every data frame to prevent static/lag but imagine the broadcaster having to error check all connections, it would quickly saturate. UDP allows saturation later. Games can predict movement and smooth out missing data for most game actions, you only need a few points for a bezier to be smooth or you might have variables for speed/direction/forces that you can predict with. Pretty good reliability and lack of ordering are not really an issue, if out of order (timestamp older than last) discard and use next or predict until the next valid message (too much of this leads to lag but normal UDP operations it is enough and actually smoother).

[1] [https://tools.ietf.org/html/draft-ietf-sigtran-reliable- udp-...](https://tools.ietf.org/html/draft-ietf-sigtran-reliable-udp-00)

[2] [https://github.com/OculusVR/RakNet](https://github.com/OculusVR/RakNet)

[3] [http://enet.bespin.org/](http://enet.bespin.org/)

[4] [http://www.isoc.org/INET97/proceedings/F3/F3_1.HTM](http://www.isoc.org/INET97/proceedings/F3/F3_1.HTM)

<comment><USER18> Two other reasons to use UDP for low-latency use cases, even if you cannot handle actual missed packets and thus are fully reliable:

\- If your bandwidth use is small, you can just spam multiple copies of each packet to decrease the chance that a laggy retransmission will be needed. If you're sending packets at a constant high rate, you can instead include copies of the last N messages in each packet, rather than just the new data.

\- You can send packets directly between two NATted hosts using STUN, rather than having to rely on UPnP or manual port forwarding. Pretty obvious, but I only see one other mention of this fact in the thread, and it vastly increases the likelihood of being able to make a direct connection between two random consumer devices.

</comment></comment><comment><USER19> This is interesting, but in my experience, you can't use this data to usefully extrapolate anything. Slightly different hardware, topology, OSes, drivers, system load, network "weather conditions", etc, can radically change the results.

My biggest UDP "surprise" was on a system using UDP but treating it as perfect, because the system designers knew that the hardware bus they were using was "100% reliable". And they were right--the hardware wasn't dropping anything. Too bad that the OSes on either end would discard UDP packets when they faced buffer pressure.

<comment><USER20> Also keep in mind this note: [http://technet.microsoft.com/en- us/library/cc940021.aspx](http://technet.microsoft.com/en- us/library/cc940021.aspx)

Basically, if you send() 2 or more UDP datagrams in quick succession, and the OS has to resolve the destination with ARP, all but the 1. packet is dropped until you get an ARP reply. (This behavior isn't entirely unique to windows btw.)

</comment></comment><comment><USER21> This is really testing "how unreliable are the network routers, cables and network adapters (and drivers!) that I'm pushing traffic through".

How unreliable is UDP as a protocol? Unreliable. This is really a binary state, not a percentage-measurable value.

If you need reliability (in ordering or delivery) you need to layer on top of it and unless your network usage has very specific constraints (eg. low latency is far more important than strict ordering, as with most fast action games) you should almost certainly just use TCP rather than end up badly reimplementing it over UDP.

<comment><USER22> > low latency is far more important than strict ordering, as with most fast > action games

On the other hand, in that situation TCP is the worst choice you can make, as, especially on bad lines, like say Whistler in the canadian mountains, it's easy to get TCP into a state where it builds up a massive latency because it insists on trying to push every single packet through.

<comment><USER23> Agreed. TCP stands for "Transmission control protocol" which started out as file transfer. These days it is exquisitely unsuitable for most things it get used for, even fetching web pages. The delays, retries and congestion controls are set arbitrarily and rarely adjusted. In this modern world of wireless roaming and streaming media, TCP has little or nothing to offer, except that its there.

<comment><USER11> I've long wished for a reliable protocol that was negotiated on a per-link basis. I mean: we have the processing power now. (So, effectively, packets aren't removed from the router's buffer until it knows the next link has the packet in buffer. _Lots_ of implementation details to be gone over, though.)

It seems a mite... silly... to resend a packet from the other side of the world just because your ISP couldn't shove the packet down your internet connection fast enough.

<comment><USER24> When packets are being sent too fast the sender needs to be slowed down. Otherwise the buffers would just fill up. And in traditional TCP, the only thing that tells the sender this is dropped packets.

The smart solution you're looking for is TCP ECNs - a way for the routers to say "I'm buffering it for now, but you'd better slow down". If you're running linux it's a kernel setting you can enable (they're disabled by default as some routers mishandle them).

<comment><USER11> Sorry, should have specified. One of the things this suggests is explicit cache management for individual links. I.e. "I have space for <x> packets before my next ack".

Don't treat links as end-to-end. Treat them as a bucket chain - each link in the chain negotiates with its immediate neighbors only. Currently we do a game of "toss the bucket at the next guy and hope he catches it".

<comment><USER24> It seems like the routes would have to be a lot more static for that to work, negating the big advantage of the internet over traditional circuit switching. Right now each end-to-end link negotiates its own window size and can accept that many packets before acking, and it doesn't matter whether half of those packets go by one route and half of them by another, they just have to all arrive at the end.

<comment><USER11> Not particularly. You can still do all the fancy and not-so-fancy tricks regarding packet routing. As long as each router knows a "closer" router to the destination, you're fine. This is identical to the current setup in that regard.

As a matter of fact, it would probably be easier to make dynamic. (Router A gets a packet for router Z - router A wants to send it to router B, but router B is currently congested, and router A knows that router C is an alternate route, so it sends it to router C instead.)

Now, there are circumstances where this approach is not particularly valid. In particular, on wireless networks. However, TCP over wireless networks isn't exactly great either. (TCP and this approach both make the same assumption: namely that most packet loss is congestion as opposed to actual packet loss.) This approach is for the segment of the network that's wired routers with little to no packet loss disregarding packets being dropped due to no cache space. I.e. this approach is for the reliable segment of modern networks - wireless should probably have an entirely different underlying protocol.

<comment><USER24> Router A knows that router B is congested - but this is actually due to congestion in the link between router K and router L. How does it know which of router C or D would be using the same link? It has to have a global understanding of all the routing paths, no?

Routing the packet to Z and telling you that the path to Z is congested are mirror images of each other; it makes sense to use the same mechanism for both.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER25> The writer is testing over links between data centers. Those tend to have good bandwidth and no "deep packet" examination and munging. Try testing over consumer-grade links, and see what DOCSIS and Comcast are doing to your packets.

</comment><comment><USER26> So UDP is great. I love the simplicity of it, and the fact that it adds very little overhead while being sufficiently low level means you can do all the fun networking things yourself.

What would be great is a "third half" of this equation: a popular protocol a la SCTP. Imagine not having to re-invent message size headers for every damn protocol: the size of your packet is embedded in the datagram just like UDP, but the transmission is reliable just like TCP. HTTP would suddenly get so much better!

<comment> TD-Linux WebRTC actually uses SCTP, but the problem is that SCTP gets filtered out a lot by firewalls and NAT that don't know how to handle it. The solution is to stuff SCTP inside UDP, which works OK because UDP is pretty minimal itself.

<comment><USER27> Sadly, not just SCTP. Anything not-tcp, not-udp, not-icmp (are there others in common use?) tends to get nuked by firewalls. Great fun if you try to run IPSEC with GRE for example. Really sad to have a protocol number in the IP header, only to have firewalls dropping anything but a few whitelisted ones... :-/

On another note... does anyone know if RDP[1] has seen any use lately? I'm guessing it's pretty redundant vis-a-vis TCP in practice?

[1] [http://tools.ietf.org/html/rfc908](http://tools.ietf.org/html/rfc908)

</comment></comment></comment><comment><USER28> The question may have been better phrased as "How unreliable is the internet" . TCP and UDP packets will be dropped at just about the same rate across any network, the only difference is that TCP keeps track, and re-sends any packets that did not arrive.

Each protocol has its own place (and uses)...

<comment><USER23> Almost. The Internet does that. But home routers are notoriously buggy. UDP often gets bad treatment there. E.g. your UDP stream may have large gaps when one of the kids upstairs is uploading to facebook - their TCP traffic gets priority over UDP which is often just dropped. This is because you can generate traffic much faster than the router can dump it to your ISP, and the little buffers fill up.

Also there was an ATT home router that dropped every other UDP packet! Really! Its a matter of, if you don't test it, it doesn't work. And home appliances get tested on delivering web pages etc, not UDP streaming.

</comment></comment><comment><USER29> If you want to know how reliable UDP is, ask a VoIP engineer. The audio component of VoIP calls is called RTP (also referred to as media). Media is passed using UDP, because there's no point in retransmitting voice packets. The time sensitivity of voice communication makes it a frivolous act. By the time the retransmitted packets arrive, the audio stream has already been reassembled and played back to the end user with defects.

I used to run a pretty sizable VoIP network over a DS0-based Carrier Ethernet network. When you have tight control over the network, UDP is extremely reliable. We would go long periods of time with 0% packet loss (as reported by AdTran VQM). When we did see loss, it was normally because of a problem we were already aware of. That's the beauty of DS0. When a DS0-based circuit drops frames, alarm bells sound.

</comment><comment><USER30> The "unreliable" in UDP doesn't mean "expect this to fail", it just means "things are not guaranteed to arrive, and if they do they might not arrive in the right order".

So "make sure you can handle (where "handle" could simply be "safely ignore") failed/unordered transmission" rather than "expect failed/unordered transmission".

If you can't cope with or ignore missed/unordered reception, wrap your comms in a stream based protocol (TCP, one of the other existing ones, or some contraption of your own) that manages reordering and retransmission as needed.

Maybe there is a better word to cover that then "unreliable" which sounds quite definite, but I can't think of one off the top of my head.

<comment><USER31> Arguably "expect this to fail X% of the time" is a more robust approach - because in-order delivery or zero-loss situations are then simply best cases that "just work".

But you can not make work what you can't test.

So anyone embarking on making their own transport protocol should test how their app works with various amounts of packet loss (1%, 5% and 20% are three reasonable marks to try which _may_ happen in real world - especially in the wireless links case), as well as jitter, and variable delay.

To do such testing, it's handy to use a Linux module called "netem" which allows to simulate the delay, jitter and loss:

       http://www.linuxfoundation.org/collaborate/workgroups/networking/netem
       https://calomel.org/network_loss_emulation.html

<comment><USER30> You also need vary the pattern of packet loss as well as the overall frequency: You could lose 20% of your packets over a short period in roughly even distribution (perhaps due to "constant" background noise), or you could lose a sequence of packets amounting to that much (due to a sudden burst of interference on a particular hop of a route).

</comment></comment></comment><comment><USER32> Upon customer request we implemented a UDP-based application protocol on a wireless network that had lots of low coverage areas. We thought they were crazy but it turned out that the overhead of keeping a TCP connection in an area of poor coverage can be overwhelming. The app retried a few times (and of course 802.11 retries on it's own), when responses didn't arrive but in the end the user was the real retry mechanism. To our surprise (not the customer's) everyone was happy in the end.

Packet ordering wasn't an issue in our app, though, so I'm not sure what my point is other than sometimes UDP is reliable enough.

</comment><comment><USER33> The reliability of UDP is mainly going to depend on the end nodes you're trying to connect. The backbone of the internet is very reliable and if you're connecting over ethernet or through some other wired connection you'll see very few dropped packets and thus UDP will be very reliable.

If you're connecting with wireless, expect to see some more dropped packets, and if you're connecting with bluetooth expect to see even more (because of its low power bluetooth is more likely to drop packets than either a wifi or wired connection).

<comment><USER34> Wireless (802.11) has built in re-transmit on a per link basis, so in practice the drop rate isn't as big as you'd expect - the main problem is packet jitter, not loss.

<comment><USER33> Ah interesting, I wasn't aware of that. I think the Bluetooth part is still valid however, as is my point about the loss rate being determined largely by how the end hosts are connected to the network core.

</comment></comment></comment><comment><USER35> UDP, the protocol, is entirely unreliable. The article is testing the reliability of the internet to deliver datagrams.

</comment><comment><USER36> Have two machines (A & B) send UDP packets to a third machine (C). Give them all the same networking hardware, e.g. 1 Gb/sec NICs. Have A & B send at full speed, so a total of 2 Gb headed to C every second, which can only receive 1 Gb. You'll get a lot of dropped packets.

</comment><comment><USER37> It's interesting that a lot of the arguments against UDP are because networks deprioritize the packets or that routers don't handle them. If you know a message is going to be lost if you don't handle it, perhaps it should receive better or at least equal priority. Since TCP is chattier, anything that encourages people to use a lighter weight protocol would then benefit everyone, since there would be more advantages to using the less resource- intensive option. It's an interesting conundrum.

</comment><comment><USER38> This is relevant for Server to server communication : with a fiber link or a copper one.

I'm not sure it's that reliable for a end user on a wireless network.

Still, it's good to know.

</comment><comment><USER39> Nice post. iperf is a pretty good tool for playing around with this stuff. In my limited testing with EC2 instances, packet size and send rate both have very large influences on packet loss. Usually loss will be stable up until some threshhold and then it will go nuts with 40-70% dropped, probably from some buffers somewhere filling up. I'd be curious to see more tests with those as variables.

</comment><comment><USER40> I thought UDP is to TCP as C++ is to Java with regards to memory management. Meaning, TCP will do whatever it can to ensure packet integrity, while UDP pretty much leaves packet integrity up to you.

Is that correct? If so, then regardless of UDP actually appearing somewhat reliable, the spec makes no guarantees that it will be consistently reliable.

<comment><USER23> There's a triplet that loosely describes protocols: { Reliable, Stream, Connection} Each can be true or false. So TCP is { True, True/False, True } and UDP is { False, False, False }. (TCP can be stream or datagram)

There are (were) other protocols possible but these are about all that's supported/tested these days.

Also there's fragmentation/reassembly issues with UDP. If you send a 30K UDP datagram packet, it gets sent as many Ethernet-sized chunks. The receiver is supposed to put them back together. If one is lost, the entire thing is lost. And to boot, Linux didn't do UDP reassembly at all until recently (last year?)

<comment><USER41> Linux has always supported UDP reassembly. Quoting a manpage: "By default, Linux UDP does path MTU (Maximum Transmission Unit) discovery. This means the kernel will keep track of the MTU to a specific target IP address and return EMSGSIZE when a UDP packet write exceeds it. When this happens, the application should decrease the packet size. Path MTU discovery can be also turned off using the IP_MTU_DISCOVER socket option or the /proc/sys/net/ipv4/ip_no_pmtu_disc file; see ip(7) for details. When turned off, UDP will fragment outgoing UDP packets that exceed the interface MTU. However, disabling it is not recommended for performance and reliability reasons."

And they ain't kidding, because there are plenty of layer 3 devices out there who play fast and loose with fragments of UDP packets; it used to be you could assume they'd drop anything that doesn't have a header, so by default UDP on linux doesn't encourage it.

It could be that distributions have been bundling sysctl.conf files with ip_no_pmtu_disc set to true, being that most modern layer 3 devices no longer mistreat UDP so badly; this may be what you're experiencing in the last year.

<comment><USER23> It was on Hacker News; UDP reassembly had never worked in Linux. Maybe I'm hallucinating that? { edit } It might have had something to do with UDP fragment reordering in Linux?

<comment><USER41> Maybe what it was (quoting from memory) is that the timeout for re- ordering/assembly started at 30 seconds (not 120 per the RFC) and only increased up to a limit of 180 when the kernel thought it wasn't something bursty like DNS; some kind of heuristic probably introduced to make NFS pre v4 work better. Maybe the queues were too small when MTUs could be 1500 bytes or less?

<comment><USER23> Here's something I found: [https://lists.openswan.org/pipermail/users/2005-March/004037...](https://lists.openswan.org/pipermail/users/2005-March/004037.html) But that's very old

</comment></comment></comment></comment></comment></comment><comment><USER42> One thing to consider - how quickly do you need to know whether the node on the other end has gone off to see the wizard?

With TCP, it might be a while - you may not get notification that he's gone until he comes _back_.

So you end up with keepalives, and... might as well have used UDP if your media can stand it.

</comment><comment><USER43> Very interesting analysis. By the virtue of the crazy business we've decided to be in (creating a mobile optimized, reliable protocol over UDP), we've had some experience with this... over mobile networks. I can confirm that sending packets at a slow and steady rate, with proper packet pacing (this is all being done by our user level protocol stack using UDP as the underlying datagram sending transport), we commonly see 0% packet loss.

Of course, some of the actual packet loss at the media level is often being masked by the media protocol itself. That's why you'd often see cases where a packet meanders it's way to the destination after 4-5x the normally observed ping latency. This sort of stuff plays havoc with TCP congestion control algorithms (because by then, TCP send side has already decided that the packet is lost and it can _only_ be due to congestion... so it backs the heck off). A lot of our win comes because of doing these things more in tune with how mobile networks actually behave.

</comment><comment><USER44> I've been quite happy with UDP lately. I set up a prototype log analysis system and just to get testing quickly I just sent the CSV log data as is over UDP between servers. It works so well that I still haven't bothered to swap it out with something else.

</comment><comment><USER45> Instead of calling UDP an "unreliable protocol", we called it "best effort."

</comment><comment><USER46> I'm wonder which vendor's routers are re-ordering packets. I know this was a big no-no in the core routers I worked on because of the bad effect it has on TCP. It was one of the things that customers checked in their product evaluations.

</comment><comment><USER47> The whole point of TCP's design is to make delivery reliable, and in order on stressed/lossy networks.

These test reveal exactly what you'd expect, as they are run between servers under prime networking conditions.

</comment><comment><USER48> The title should read: How Unreliable is the network?

The answer is: Very.

</comment><comment><USER49> Doesn't the "U" in UDP stand for unreliable? Joking!

</comment><comment><USER50> "How unreliable is UDP? Let's test with these 5 network paths that I have no control over, and have no idea what their design constraints are...that should be totally authoritive, right?"

Pretty bad test conditions on this, tbh.

</comment><comment><USER51> _Finally_ , a informational discussion on HN..</comment>


Can Digg make a run on Reddit? - <AUTHOR> http://betabeat.com/2012/08/the-digg-bang-theory-can-betaworks-make-a-run-on-reddit/

<comment><USER2> Is digg trying to make a run on reddit? I finally saw the new digg, it felt more like prismatic/other news sites to me.

I associate reddit more with close discussion communities, as broader-versions of forums.

</comment><comment><USER3> The really interesting thing about the redesign of Digg, is in a way it's kind of the first example, of the possibility of seeing gentrification ported to the internet.

I don't think they will be successful, but its a really interesting concept.

One of the hardest parts of any kind of community driven website is to generate the community in the first place. Of course, as the idea of eternal september demonstrates communities deteriorates in quality over time as the community grows to a broader variety of people.

Its interesting then, to try to take an existing community that has settled at the lowest common denominator of content and to try to revitalize it.

If you take the metaphor further, if you think of artists as the cultivators of culture in a city. And the first step of gentrification is an increase in artists (typically seeking lower rent) who is the cultivators of culture in an internet community. Money doesn't really exist on the internet... well at least the classism associated with it (and rent) so what attracts these culture cultivators back?

</comment><comment><USER4> Not if you have to have a FB account to login. They need other options fast.

<comment> k-mcgrady It says on the site FB login is a temporary spam prevention measure while they work on a good account system of their own.

</comment></comment><comment><USER5> They are serving completely different purposes. Why would they be competing?

<comment><USER6> They are both link aggregators. Although, the new Digg looks more like Pinterest or even canv.as than Reddit.

<comment><USER5> But their algorithms for aggregating are very different. Reddit is primarily based on submitted links. Digg is primarily based on links shared on social networks.

<comment><USER6> True. However, at the end of the day, both of these sites revolve around the content "submitted" by users (either directly on the site or across social networks), and they need this content to thrive/survive. So while their purpose/methods may be very different, both sites co-exist in the same ecosystem?

</comment></comment></comment></comment><comment><USER7> I've enjoyed Reddit for years, but I'm about at my limit. Incessant memes, novelty accounts, meta-commentary about novelty accounts, reposts, complaints about reposts, kitten pics, and a million other annoyances that trace back to karma whoring make the site difficult to tolerate. If the new Digg could successfully counteract this trend, I'd be on board.

</comment><comment><USER8> No.

<comment><USER9> Indeed.

blauwbilgorgel gave some good reasons why this prospect seems ridiculous: <https://news.ycombinator.com/item?id=4322197>.

Like outlook.com, the best thing Digg has going for it is that people really want an alternative to the incumbent who sits on most of the market, which in this case is reddit.

<comment><USER10> Because a rushed product is imperfect?

Digg at its highest peak was a grossly imperfect product. If project success depended upon trivia like type="text/javascript", the internet world would be a very different place.

</comment></comment></comment><comment><USER11> It's not even out properly yet and these articles are already starting to pop out?

In its current state definitely not. If they can build even better algorithms and communities than Reddit, maybe. But that's a huge if. And they've got nothing right now.

I do like its design though, which is very different from Reddit, and that means it might serve a different market.

</comment><comment><USER12> Digg is a tabloid, reddit is a community*

* dysfunctional, but community nonetheless.

</comment><comment><USER13> No.

</comment><comment><USER10> They can through algorithms, which is something that is painfully lacking on most social media sites (including HN).

Give users the ability to dislike something strongly enough that anyone who voted it up is excluded from all vote counts/rankings/recommendations as provided to that user for some period of time. While this leads to a filtered world, that is exactly what users expect from sites like these.

Reddit is essentially falling apart for many users (while catering to a new audience) -- in /r/all as well as individual subreddits that see a constant cycle of abandonment as people try to ontologically solve the problem -- as it is increasingly dominated by teenagers.

If I could strongly express a disinterest in every fictitious Facebook picture / iPhone message conversation, and that served as a quality indicator of people whose opinions I don't respect, that would be beautiful.

<comment><USER14> Two things: trivially, you can change the subreddits you are subscribed to and remove the "fictitous FB pic" etc. posts.

Second, and more interestingly: people have different facets that they show in different subreddits. As an example here on HN people prefer tech-oriented stuff. But not everyone here is _that_ into tech. They just sort or organize the content, so they come to HN for in-depth stuff, and (say) Engadget for rumors, and sbnation for sports etc. So someone who upvotes a stupid FB pic in /r/funny might vote the same photo down in /r/politics. If the person is "sorting" content that's appropriate in different situations, it's not useful to discount their opinion entirely.

<comment><USER15> I see this argument a lot, and I have two counterpoints:

1\. I don't want to have to spend time researching subreddits just to figure out what I'd like. Content aggregators are meant to remove steps from the process, not add them. 2\. While the content of smaller subreddits are usually improved, the level of discourse in comments are not. Even if you say 'okay, no meme posts, no meme comments', the fact is that you're going to have a glut of commentors whose first impulse are to crack a joke or make a surface-level comment. The difference in comments between a post on here and a post on /TrueReddit/ are astounding.

(The one pretty ready example to the above are < 100 member subreddits, but those usually have the issue of insufficient content.)

<comment><USER16> And how are you going to tell content aggregators what you like? By upvoting/liking? Doesn't it count as spending time?

I don't want to spend time upvoting 50 submissions to tell the recommendation engine what my interests are (and with the current state of technology it will get my interests wrong anyway). I just want to click one button (subscribe to a subreddit) and that's it.

And when I become less interested in a certain topic (fitness, starcraft) I can just press the unsubscribe button once and never see submissions for that topic again.

<comment><USER15> Ideally -- and take this with a huge caveat, as I'm not exactly in the aggregation business -- measure it by clicks, comments made, and time spent on page/bounce.

Overall, I'd like to see aggregators move to the Pandora model, rather than the Hype Machine one. With Pandora, I can specify a jumping off point (genre, artist, song) but it still refines my tastes from there.

Hype Machine, I've 'liked' hundreds of songs but I still need to manually peruse through specific blogs to find new songs I like. It's unideal.

<comment><USER16> I believe that in the future that's the way to do it, but not right now.

Besides, I'm not a fan of automatic recommendations as they have a lot of drawbacks. Their major problem is: they won't help you find something you didn't know you liked.

<comment><USER10> _Their major problem is: they won't help you find something you didn't know you liked._

This is patently untrue. It is more likely to find you stuff that you didn't know you like than the lowest common denominator strategy.

<comment><USER16> Are you talking about any existing recommendation engine that I can try out to prove myself wrong?

Just how exactly will they do it? I'm really interested, maybe I'm missing something?

Simply speaking, if a system knows all I listen to is classical music, it can recommend me a new classical composition, but it cannot all of a sudden recommend me dubstep (which I might actually like).

<comment><USER10> We're talking about Digg doing this as a differentiation. It is not commonly used because it is very heavy in IO and CPU for individual users (though a site like reddit could take a shortcut just by periodically ramping up some spot high CPU instances and batch generation correlations between users on every subreddit, then staling that until the next run), and sites like Reddit and previously Digg ran on stacks that greatly limited their ability to scale for this.

Regarding music taste, if the system knows that you have a strong correlation with a number of users who like classical but periodically explore new music, what they like will likely be something you might be interested in. This, in practice, is exactly how music discovery happens in the real world (that the discoveries of the people you share musical tastes with are more likely to interest you).

<comment><USER16> What you are talking about is still incapable of recommending (with a high chance of the recommendation being spot-on) new things. In music a new thing would be a completely different genre, not a new song in your favorite genre.

The system you are describing will find people who have a strong correlation with me and who also like:

1) Classical music and Justin Bieber.

2) Classical music and Metallica.

3) Classical music and Dubstep.

And it will recommend me Bieber, Metallica and Dubstep. I might like Dubstep, but I won't like Bieber and Metallica. How is it better than listening to random songs?

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> ... not with 504 Gateway Timeouts. I did get a chance to see it last night and it certainly looked nice, but I'd already seen all the news stories on the front page at that moment, so I haven't quite gotten the point yet.

I'm not sure that FB and Twitter shares are the same as "diggs".</comment>


What do we need corporations for? Former Greek finance minister on Valve (2012) - <AUTHOR> http://blogs.valvesoftware.com/economics/why-valve-or-what-do-we-need-corporations-for-and-how-does-valves-management-structure-fit-into-todays-corporate-world/ <comment><AUTHOR> Previous discussion: ([https://news.ycombinator.com/item?id=4333578](https://news.ycombinator.com/item?id=4333578))

Yanis Varoufakis, who wrote the post during his tenure as "Chief Economist" at Valve, went on to serve as the finance minister of Greece during its financial crisis and negotiations with the EU.

Relevant today in light of the many attempts being made at creating tech coops ([https://github.com/hng/tech-coops](https://github.com/hng/tech-coops)), and decentralizing organizations with blockchain systems.</comment>


Visualizing Ocean Shipping - <AUTHOR> http://sappingattention.blogspot.co.uk/2012/04/visualizing-ocean-shipping.html

<comment><USER2>
        There aren't many truly seasonal events,
        but a few stand out. There are regular
        summer voyages from Scotland to Hudson's
        Bay, and from Holland up towards Spitsbergen,
        for example: both these appear as huge
        convoys moving in sync. (What were those about?)

The yearly trips from the Netherlands up to the North are most likely whalers going out to hunt and coming back with their catch.

</comment><comment><USER3> Does anyone have pointers to anything like this, freely available, for modern shipping? (i.e. after GPS and AIS)

<comment><USER4> Not quite the same, but you can view the position/heading of some ships here: [https://www.marinetraffic.com/en/](https://www.marinetraffic.com/en/)

<comment><USER3> Yep. Use that a lot as a reference. (AIS transponders send quite a lot information and also a lot of people upload photos of vessels that can be viewed at marinetraffic.com)

There are also a few other projects that have overlapping data. (aprs sites, e.g. aprs.fi - more of an amateur radio thing but will still pick up lots of interesting signals)

(Edit: perhaps more interesting to people around here an usb/serial ais receiver can be had for a few hundred dollars and there is a site called [http://www.aishub.net/](http://www.aishub.net/) where you can get access to the firehose provided you cover your area. Disclaimer: I never tried aishub.)

</comment></comment></comment><comment><USER5> let's hope pirates don't have computers.

<comment><USER3> I think the post is more about older ships logs.

For shipping there are rules in place on who are allowed to turn of tracking and under what circumstances.

Also even if "security by obscurity" etc is heavily used it doesn't mean it is a good strategy : )

<comment><USER5> Ahhh :) I might book that cruise now.</comment></comment></comment>


NY Hedge Fund Buys Almost 10% of Palm - <AUTHOR> http://gigaom.com/2010/04/14/harbinger-palm/

<comment><USER2> It freaks me out a bit that hedge funds have so much money. I don't understand how they create that much societal value.</comment>


A Simple Guide to Meditation - <AUTHOR> http://www.theflowstatecollective.com/flow-habits/a-simple-guide-to-meditation <comment><USER4> Meditation is one of those things that people think they're talking about the same thing when actually they're not. It's not just one thing - <USER2> religions and traditions have completely different approaches. Sure, there are some key concepts common to all like "awareness" and "mindfulness" \- but awareness and mindfulness of what? Christians might "meditate" on a passage in the Bible. Theravada Buddhists have the Four Foundations of Mindfulness. There are Zen koans. I have no idea on other forms but I imagine there are way more.

Even among Theravada Buddhist meditation instructors there are clear disagreements on what the experience should be, how concentrated the mind should be, etc.

For example, for something as simple as a pull-up there are questions like: what muscles are being worked? What is the proper form? Pronated or supinated grip? What pace and why?

If I read an article on any subject I expect to see lots of footnotes.

Meditation is something way more subtle than doing pull-ups, yet most guides seem to assume that there is one concrete definition in the learner's mind. Where did the prescribed method come from? What is the goal? To just put instructions or information about any subject without mentioning background or citing any sources is strange to me - is meditation so new, obscure, or insignificant a subject that it doesn't warrant a little more rigor?

TLDR: meditation is a bigger subject than most people reslize.

<comment><USER5> True - meditation means different things to different people. It's obvious the author is referring to the "mindfulness" version though and is trying to do so in simple-as-possible guide form so as not to distract with all the depth and nuance you allude to. I think most people get that there are many different types of meditation. I appreciate the article's simplicity because when I was a once a beginner all I seemed to be able to find was less than simple school- specific (Zen, Shambala, etc) instruction. The problem with the school- specific teachings, IMHO, is that they are all mindfullness-oriented, yet each adds their own special ritual and ceremony which as a beginner you mistake as being essential. At least for me, I've converged on an intentionally Occam's razor style approach aimed at stripping the non-essential. The coolest thing about Bhudda's story to me is he just sort of figured it all out for himself and said each of us should do the same. I've figured out for myself, for example, that it is not necessary at all to sit cross-legged. It makes no difference for me yet standard teachings always strongly encourage it to the point that you feel like you are "doing it wrong" if you don't. Teachers will even say, "if you can't sit cross-legged it's ok..." but when everybody in the group is doing otherwise, the overwhelming signal is it is actually not ok.

I guess one thing I'm saying is the kind of folks drawn to try meditation are often the same kind of folks who will willingly dive down rabbit holes investigating the differences between the schools and entertaining themselves with koans, etc. all the while distracting themselves from actually getting started with the meditation itself. Can confirm; was one of these people. That's why I'd caution against trying to raise awareness of all this complexity to the beginners this guide is intended for.

<comment><USER6> I can't agree with you enough. Also, looking at all the comments, I'm amazed at the complete lack of discussion of love and compassion. Regardless of your personal beliefs, if you're not including either, you're doing it wrong.

Oh, and nothing is as perfect as you believe it to be.

</comment></comment></comment><comment><USER7> Definitely check out Sam Harris' book "Waking Up" (the atheist guy). It was a great introduction to meditation for me.

<comment><USER8> I really enjoyed that book and Harris' approach to meditation. Since Harris is a neuroscientist and an atheist, there's no mysticism in the book. No talk about "feeling the energy".

In addition to the meditation information, he also goes into some neurological theory of thought generation in the hemispheres of the brain that I found quite fascinating.

[edited for clarity]

</comment></comment><comment><USER9> When I first started meditating I found the book The Relaxation Response helpful. It has a one page guide to activating the parasympathetic nervous system and is non-spiritual in its application.

</comment><comment><USER10> For those who are interested in advanced meditation and pushing the limits, I can recommend book "Focused and Fearless". Covers very powerful techniques. I went from unable to sit for 15 minutes to 1 hour of effortless deep meditation within a week. YMMV.

<comment><USER11> Sadly, I never made it to advanced meditation. 10 minutes in and I fall asleep. Used to do that in the shower right after I got home from sports/training. Meditate for 10 minutes under a very light stream of water. The results from doing that for nearly 6 years serves me well every single day

<comment><USER12> Sometimes 10 minutes is all you need. I feel like years of heavy partying wrecked my memory and I've been meditating for about 8 months and I can feel it returning to what it once was.

</comment></comment></comment><comment><USER13> I found a helpful primer to meditation and associated benefits in a good book by Dan Harris - "10% Happier: How I Tamed the Voice in My Head, Reduced Stress Without Losing My Edge, and Found a Self-Help That Actually Works"

Mainly he decouples the practice from the spiritual part of it. so you can go in without expecting a monk to come out the other side.

</comment><comment><USER14> Well, @cko is definitely right; and there are many kinds of meditation techniques. But for one, without getting into too much detail, there is Zazen (without koans), where you are trying to "just be"; and think nothing. Just be, do not cling to any other ideas or thoughts, and when you realize you are thinking, stop it. Just be.

</comment><comment><USER15> Here's a more complete beginners guide: Mindfulness in plain english. I believe its something of a classic.

[http://www.urbandharma.org/udharma4/mpe.html](http://www.urbandharma.org/udharma4/mpe.html)

Scroll down for the ToC.

<comment><USER16> I'm currently reading it (kindle edition.) Highly recommended. The style makes it easy to read (I'm not a native english speaker), it's very down to earth in its tone and addresses potential problems as it goes.

</comment></comment><comment><USER17> Here's my issue with meditation. There's no guide to what progress should look like. As in, everything I read says you shouldn't have any expectations and to just do it and there's a lot of benefit in it. I get that, I've been doing it for over a year now regularly but how can I know if it has any effects whatsoever?

Since I suspect there might be other techniques than focusing on the breath or body scans, I try to follow different guided meditations. After my session, i feel relaxed and my mind slow but efficient. Like getting up after a good nap.

But so does working out at the gym, sort of. Over there, however, you can directly see progress. With meditation...

Perhaps I am looking at this in the wrong way?

<comment><USER18> Hopefully this doesn't seem an obvious response, but to my mind there seems to be at least two schools of thought regarding meditation. These schools are not mutually exclusive, FWIW.

One school seems to see meditation as an end to itself, the other as a tool towards some other end. I can't speak to the former, but for the latter - <USER3> approach you could follow is to identify experts who have written on the area of improvement and who prescribe meditation as a tool. For example, if you're using meditation to improve willpower, then a book on willpower will help you identify whether your meditation techniques are improving your willpower [1].

[1] For example, "The Willpower Instinct" (K. McGonigal) discusses the use of meditation to improve willpower, as well as the 'state of the art' on the subject, so it provides you with some tools you can use to evaluate whether you're making improvements or not

</comment></comment><comment><USER12> Just got back from a meditation gathering from meetup.com and all I have to say is WOW. Powerful, and highly recommended.

Start with 3 min a day right when you wake up (before food, cigarette, anything at all) and then go from there.

<comment><USER19> I tried meditation for a month, all it did was bore me and make me fall asleep.

<comment><USER20> In meditation, mental states and thoughts will play out that are never or rarely allowed to play out. If you are ever so slightly sleep deprived, chances are that sleep will sneak into your meditation. Just watch what comes up in meditation, and act accordingly.

</comment></comment></comment><comment><USER21> > Close your eyes and take one breath in through the nose and one breath out > through the nose.

Should I take this literally? I have a condition which makes it very hard for me to breath through my nose. Is it a requirement?

<comment><USER7> It's not a requirement. The reason many vipassana guides say to breath through your nose is because most people can easily feel the air at the tip of nose and it gives them a clear point of focus. You can breath through your mouth too. Some people prefer focusing on air coming into the stomach instead.

When I have a cold or some other build-up in my nose, I just breath through the mouth, no problem.

</comment></comment><comment><USER22> I practice a form of meditation called Passage Meditation. Though simple, Passage Meditation is definitely not easy. However the results I have seen from myself and others, makes it an extremely beneficial practice.

Here is a free online course for those interested in it.

[http://www.easwaran.org/introductory-passage-meditation- cour...](http://www.easwaran.org/introductory-passage-meditation-course- introduction.html)

<comment><USER23> It would be nice if you elaborated on those results.

</comment></comment><comment><USER24> My favorite guide to meditation is: Being nobody Going nowhere by ayya khema

This subject also highlights how irrational i am. I still remember how beneficial meditation was for me when I practiced it. But getting back into the practice ..... sigh

<comment><USER8> It's like exercise. The benefits are pretty obvious and enjoyable when you're doing it, but putting forth the effort isn't always easy.

I actually try to combine the two practices. I have an exercise bike and I meditate while I'm riding it.

</comment></comment><comment><USER25> A while back, I was looking for a guide to meditation and found this to be really helpful: [http://www.meditationmojo.com/](http://www.meditationmojo.com/)</comment>


How to build a social entertainment website - <AUTHOR> http://www.reddit.com/r/IAmA/comments/ev2zb/i_run_thathighcom_and_it_pays_my_rent_in_san/c1b6ql9

<comment><USER2> He also started <https://www.djangy.com/> [http://www.reddit.com/r/IAmA/comments/ev2zb/i_run_thathighco...](http://www.reddit.com/r/IAmA/comments/ev2zb/i_run_thathighcom_and_it_pays_my_rent_in_san/c1b6u3q)

</comment><comment><USER3> Did PG fake start Hacker News?

<comment><USER4> rms tells me that nickb was widely suspected to be a PG sockpuppet.

<comment><USER5> Ah, this explains why Tara Ploughman has been referred to as nickb's wife.

</comment></comment></comment><comment><USER6> Not quite.</comment>


Stross on Cameron's proposed UK crypto ban - <AUTHOR> http://www.antipope.org/charlie/blog-static/2015/01/ah-politics.html <comment><USER2> Unbreakable crypto is a genie long out of the bottle.

No amount of wishing, legislating, or fear-mongering is going to prevent terrorist organizations from implementing these well-described, mathematically sound, publicly available algorithms.

Since we cannot prevent terrorists from using crypto, the only ones without crypto will be non-terrorists.

This proposal only serves to enshrine within the government the chilling capability and authority to surveil its citizens.

</comment><comment><USER3> The most interesting thing about this story is that almost no-one is talking about what else Cameron said in his speech [0]:

 _Mr Cameron also said that a Tory Government would pass a new law to ensure that the intelligence agencies would be able to track phone calls and internet messages.

He said that this power to track "who made which call, to which person, and when" was "absolutely crucial not just in terrorism but finding missing people, murder investigations, almost every single serious crime.

"What matters is that we can access this communications data whether people are using fixed phones, mobile phones or more modern ways of communicating via the internet." _

I refuse to believe our PM is so technologically ignorant that he believes the 'banning encryption' policy will ever fly. The most plausible explanation to me seems to be that he is trying to rebalance the public's expectations of privacy towards greater surveillance to the extent that slightly less egregious (but still objectionable) laws can be passed in their place. I have a feeling that this kind of rhetoric has been used before and probably has a name.

[0] [http://www.telegraph.co.uk/technology/internet- security/1134...](http://www.telegraph.co.uk/technology/internet- security/11340621/Spies-should-be-able-to-monitor-all-online-messaging-says- David-Cameron.html)

<comment><USER4> It's almost impossible for me to resolve the cognitive dissonance of seeing an educated, by all means literate person completely devalue privacy to the point that it ceases to exist.

To say with a straight face, after the events of the 20th century (and the horrors of Russian and German domestic intelligence services, past abuses committed by the FBI, the current human rights clusterfuck in places like China, Iran, and Syria), that all communication should moderated and monitored by the government?

Can anyone support that idea in good faith? I agree with you that it's an almost unbelievable level of stupidity and one that betrays an ulterior motive. But normally I'd say something like this played into some industry or private-interest wish, but what bank or corporation (much less any commercial tech venture) is pushing for this? What group could this possibly serve in society other than the most high-ranking (and power-hungry) officials in the intelligence establishment?

</comment></comment><comment><USER4> Of course Stross is right on. That Cameron would do such an about-face on the values and ideals the Charlie Hebdo staff died for is disgusting (not to mention, as per the author's excellent point, completely counterproductive).

</comment><comment><USER5> The "illegal guns" the terrorists used in France were brought from _outside of the country_. So now Cameron thinks he can stop a freaking app from entering the country? I mean he can't be that stupid, can he? I don't think he is, because I think this new proposed law has nothing to do with terrorism and has a completely different agenda behind it.

<comment><USER6> I guess the idea is the old "Only criminals use encryption, so you must be a criminal".

Of course, given that the big crime networks are sophisticated this won't stop them. They'll go back to old cyphers and steganography. Same goes for the compulsory handing over of encryption keys, there are now multi-level encrypted filesystems, where you have n keys where each opens a different set of files. Have a few keys protecting embarrassing but not incriminating files and disclose those.

Surely scifi has taught us that totalitarianism never wins?

</comment></comment><comment><USER7> I don't understand why people seem to think he's talking about outlawing encryption. From what I understand he's talking about forcing companies to disclose your messages.

You can still have all the encryption you want on HTTPS, but it then get stored on a server somewhere.

That said, I believe both ideas are abhorrent, so if we have to take some technical liberties to cause enough uproar then so be it.

<comment><USER3> It's an inevitable consequence of these lines of the speech he gave. I recommend you listen to the whole thing to understand it properly in context [0]:

 _" But the question is are we going to allow a means of communications which it simply isn't possible to read. My answer to that question is: no, we must not."_

 _" If I am prime minister, I will make sure that it is a comprehensive piece of legislation that makes sure we do not allow terrorists safe space to communicate with each other. That is the key principle: do we allow safe spaces for them to talk to each other? I say no, we don't, and we should legislate accordingly."_

The only way to guarantee there is no "safe space for [terrorists] to talk to each other" is to ban all means of communication that cannot be decrypted. If we simply ask Whatsapp to record all their messages (and keep the decryption keys somewhere), that still leaves all kinds of encrypted communication applications available (or safe spaces) for terrorists to use.

[0] [http://www.theguardian.com/uk-news/2015/jan/12/david- cameron...](http://www.theguardian.com/uk-news/2015/jan/12/david-cameron- pledges-anti-terror-law-internet-paris-attacks-nick-clegg)

<comment><USER7> I still think that is a rather literal interpretation of what he is saying. Forcing companies to allow access to data seems much more realistic, though admittedly doesn't meet the condition "no safe space for [terrorists] to talk to each other". But maybe he is a complete moron.

Presumably he is going to ban the XOR operation since with a one-time pad...

<comment><USER3> If he forces major companies to provide access to their data, that still leaves minor companies or open source projects to do what they want and therefore still leaves the bad guys with a place to hide. The only alternative is to ban encryption for all but those who agree to allow access by the government (with a warrant of course) which while possible is highly implausible given how much internet traffic is currently encrypted.

</comment></comment></comment></comment><comment><USER8> Is this guy just going to ban everything?

Anybody intent on breaking the law will just [continue to] use steganography and side-step the issue completely. If anything, a cryptographic ban would most likely result in an easier life for terrorists (especially but not exclusively cyber terrorism). One honestly wonders if that's what he wants because it has to be impossible to be _this_ stupid.

</comment><comment><USER9> A comment by Phuzz, one the webpages comment section, says it well.

You can even create spray paints in Counter-Strike which are images, these could be used to hide secret messages.

Point being, anyone can hide a message if they really wanted to without anyone noticing.

</comment><comment><USER10> It's a pity politicians don't need to show even a basic understanding of the material they wish to create laws for.

</comment><comment><USER5> Holy cow this was flagged fast.</comment>


Laptop security: Apple vs. [Insert name here] - <AUTHOR> OK let me tell you my story:
My Mac was stolen, and I used Find My Phone to lock it immediately. A week later, the thief opened the device, and sure he can't access it.
After two weeks, I lost my hope, sure I won't be able to get it back, so I decided to activate the Eraser.
Just an hour ago, I received the notification that the device erasing process has started.
I lost my device and all my data but not my privacy. And I found on Apple support forums that the device will remain locked as long as it still listed in my Find My Phone.
Now I'm planning to buy an x220 and get back to Linux. I was wondering how can I secure a Linux laptop and make it (somehow) harder for the thief to access my stolen device just like Apple did.
Do you have any experience with this? Do you mind to share? <comment><USER3> Long time Ubuntu user here.

The solution to your problem is data encryption. On Linux, there are different levels of data encryption, whether you want to encrypt the whole hard drive or only the /home partition (where all of your private files will be stored).

The ArchLinux Wiki has a very detailed page [1] about all the available options for disk encryption.

Regarding Ubuntu, when you install it on your system, it will ask if you want to encrypt your /home partition [2] or your whole disk [3].

I got my laptop stolen in France last summer, and as much as this was a pain in the ass, at least I didn't have to wonder if the thief had access to my private data (photos, documents and the like) since the /home partition was encrypted. Hell, the thief probably had a WTF moment when (1) he discovered it was not a French version of Windows but an English version of Ubuntu running and (2) when he discovered it was a Taiwanese laptop with this keyboard layout [4]. Good luck to sell that on the French black market :)

[1] [https://wiki.archlinux.org/index.php/Disk_encryption](https://wiki.archlinux.org/index.php/Disk_encryption)

[2] [https://www.howtogeek.com/wp- content/uploads/2012/06/ximage8...](https://www.howtogeek.com/wp- content/uploads/2012/06/ximage83.png.pagespeed.gp+jp+jw+pj+ws+js+rj+rp+rw+ri+cp+md.ic.Q3l_7oXwbw.png)

[3] [http://www.tecmint.com/wp- content/uploads/2016/02/Ubuntu-16....](http://www.tecmint.com/wp- content/uploads/2016/02/Ubuntu-16.04-Installation-Type.png)

[4] [https://c1.staticflickr.com/8/7501/16104079539_00c39c200d_b....](https://c1.staticflickr.com/8/7501/16104079539_00c39c200d_b.jpg)

</comment><comment><USER4> Sorry for your experience.

One standard approach is to set up full disk encryption. A common setup would encrypt every partitions but your /boot partition, so a thief would be unable to access your system if it were powered off. (If you're especially cautious, you can do tricks to protect your /boot partition too, to guard against tampering, but that's beyond the scope of protecting against theft.)

The catch is if the thief steals your powered-on laptop, the system's still decrypted (meaning, the decryption key is still in memory). I'd guess locking your machine is a partial guard (and is what I rely on), but I'd be interested in learning if there's a better method of protection.

ArchWiki has a pretty good overview: [https://wiki.archlinux.org/index.php/Disk_encryption](https://wiki.archlinux.org/index.php/Disk_encryption). I'm happy to try and answer any questions you have.

<comment><USER5> A few questions:

1) > The catch is if the thief steals your powered-on laptop, the system's still decrypted

I think the key distinction is if a laptop's storage is encrypted if it's in some sort of sleep or lock mode. AFAICT most people's laptops are rarely completely off; they either are fully on, asleep, or sometimes locked.

Solutions that secure data only when the laptop is fully off seem almost useless to me; in practice the data rarely is encrypted. Do you know of solutions that address this issue?

2) What about Self Encrypting Drives (SEDs), which encrypt at the hardware level usually by using the industry standard (AFAICT) Opal?

[https://www.trustedcomputinggroup.org/storage-work-group- sto...](https://www.trustedcomputinggroup.org/storage-work-group-storage- security-subsystem-class-opal/)

3) File-level encryption, rather than volume level, would seem to solve the problems in #1. Files are decrypted only when they actually are in use; otherwise they are secured. Therefore on most systems, most data files are secured most of the time. The problem is how to efficiently enter credentials for every file, or every batch of files, the user opens: Type a password every time? What about databases or email (e.g., stored 1 file/msg such as in maildir)? Keep the key on a USB drive that must be inserted and, only when first inserted, authenticated with a password?

Do you know of file-level solutions?

4) The problem with every solution is implementation. Security is very hard to implement, and requires high quality execution to avoid exploits. How do I know that the vendor did it correctly?

<comment><USER4> I glossed over it a bit in my first post, but the data on the drive is actually always encrypted. The system decrypts the data on the fly as it's read into memory, and encrypts when it writes to disk again. This has less of a performance impact than you'd think.

Also, just to be explicit on the user experience: when you boot the machine, one of the first things the kernel does is ask for a decryption password. If you cannot provide that, the system cannot boot further, because everything but the kernel and bootloader (in /boot) is behind the encryption.

1) So as described above, the storage is always secure, regardless of whether the machine is on or not. The rub is that when the machine is in use, the system is actively performing this decryption/encryption. I'm not an expert on the technical side of it (and it probably differs between implementations), but I'd imagine the OS keeps the decryption key in memory. This is functionally an instance of the "it's got to be decrypted sometime" problem, or a variation of the "analog hole" in DRM. At some point, the user will be accessing a decrypted version of the data, and if the attacker is able to take control at that point, he obtains the data.

Let's say I have an encrypted text file (on an unencrypted, regular system setup). To write to or read the file, I need to decrypt it. Maybe a program takes a password and opens a text editing window with the decrypted data. If I care about the security of that data, I'm not going to leave that window open unless I'm actively using it. I understand that if I leave that window open and someone walks by my machine, they'll be able to see the contents, because I left the data in an decrypted state. But I need to have that window open _sometime_ , because I need access to the file myself. It's the same situation with full disk encryption--at some point the data will be decrypted for legitimate use (in FDE's case, only in memory, but still decrypted), and it's up to the user to protect it during those times.

If you lock your machine (using xscreensaver, slock, etc.) and set it to lock when waking from sleep (and whenever else), the attacker must circumvent the lock program before he can access the machine. Ideally this would not be possible (a lock program that lets someone access the system without the password is not a great lock), but there's always the possibility of some vulnerability.

Alternatively, you can always hibernate your machine instead of sleeping it. Hibernate writes the contents of memory to disk and shuts the machine off. When booted back up, the kernel finds the hibernated memory and seamlessly resumes from where you left off. If you set it up properly, the system will write the hibernated memory to an encrypted partition, so the session cannot be resumed without the encryption key. The downside is you have to type the encryption key every time you resume, and my encryption password is a good deal longer than my normal user account password.

In my own practice, if I'm leaving my laptop at a place I'm more worried about theft, I'll hibernate it. In normal use, I'll sleep it and rely on the lock program. Like I said in the earlier post, though, I'd love to hear if anyone has a better approach, or even an analysis of the security of some common lock programs.

2) I have no direct experience with SEDs, but I'm under the impression they decrypt the entire drive when powered on and the password is entered. Or else, they do the same on-the-fly operations I described above. As such, they would be vulnerable to the same attacks as above. Their advantage is transparency to the operating system and better performance. Also see Wikipedia's description of some of their vulnerabilities: [https://en.wikipedia.org/wiki/Hardware- based_full_disk_encry...](https://en.wikipedia.org/wiki/Hardware- based_full_disk_encryption#Vulnerabilities)

3) FDE essentially answers the "how to efficiently enter credentials" question with "at the start, when you first mount the partition" ;). Aside from that, file-level encryption solutions definitely exist, and are commonly used. You can encrypt arbitrary files with the openssl command, and many sensitive files (such as SSH keys) have encryption built into their standard usage (SSH key passphrases). Even when running FDE, I keep a passphrase on my SSH keys, because there's nothing stopping a rogue program from grabbing them during normal computer use. I'd encourage any other extremely sensitive files to have their own protection. To quote tptacek, "FDE does basically one thing for you: it reassures you if your laptop is stolen from the back seat of your car or left in a cab." [1]. Other steps need to be taken to run a secure system.

However, I don't know of a file-level encryption solution that functions exactly as you describe. It would be tricky to implement, for the reasons you described, and others. For example, background programs write to the disk too, and sometimes what they write contains sensitive data. Are you going to enter the key periodically for their use too? And will all these programs play nice with the (comparatively) huge blocking times when writing while you type in the key?

4) Of course, that's the question with any security solution. Many of the Linux solutions are open source, so that's at least a plus, but certainly not any guarantee of security. Short of being a security professional capable of auditing the complete source, you have to rely on project reputation, recommendation, and (ideally) someone else's audit. I'm sure lots of people would like a better answer to this question!

Hope that all helps!

[1] [https://news.ycombinator.com/item?id=9069669](https://news.ycombinator.com/item?id=9069669)

<comment><USER5> Much appreciated; thanks.

I'd add one more item to the difficult-to-avoid vulnerabilities, file and file system metadata. Otherwise a simple directory listing, for a user or a background program, requires authentication.

My guess is that vulnerabilities like that, including the user access hole that you describe so well, are the reason that modern OSes (e.g., on phones) isolate most data so that it is accessible only to certain applications, usually the app that created the data, instead of the old model of all applications having access to (almost) all data. Even if for practical reasons the application needs almost unlimited access to the data, at least you can limit the attack surface to only that app and parts of the OS.

</comment></comment></comment></comment><comment><USER6> Why an x220? That's a six-year old device that predates many modern security features. Secure Boot isn't evil, works just fine with Linux, and TPMs are useful, too. But the most important thing to do is encrypt your drive, which you can do with an x220 just as well (just make sure you get one that supports AES-NI, I believe the ones with a Core i3 don't).

In any case, if you're coming from a MacBook, you're going to hate the x220's display. It's atrocious, with terrible colours and brightness. It's also a SATA2 device, so a modern SSD won't be able to reach anywhere near its full performance. Unless your budget is around $200 I wouldn't recommend such an old machine. But if you do go down that route, you might like r/thinkpad on reddit. Plenty of people there who mod these old devices and have advice on modernising them.

</comment><comment><USER7> Side remark: All the methods exposed here requires you to have a good password. In my case, I am using a Yubikey set with a single long fixed password on it + the normal password I can remember.

So, to login/unlock, I type in my password, plug the Yubikey and press the button. This ensures I have a really strong password.

Because if your password is "1234password", all the provided solutions are of no real use.

</comment><comment><USER8> The interesting part in Apple's FileVault (plus UEFI password) is a Guest access called Safari Only Mode. This mode starts special macOS distro from unencrypted boot partition and allows to run only Safari.app (also terminal.app) The main idea is that thief log in as Guest, connects to WiFi and you can locate your device in Find My Mac. But this mode just allows you to trace device via IP geolocation since macbook never had GPS module like iphones do, so this geolocation is not accurate enough and not useful. You will just see the city or district located from ISP. The real useful information will be WiFi AP MAC addresses (BSSID) located nearby the stollen macbook. Having this information you can quite accurate locate your stolen device and return it!

But I can't add my custom shell script into Safari Mode because of macOS SIP (system integrity protection) that I don't want to disable. Also every major update overwrites changes on boot partition.

I would appreciate for any help with this project. My goal is to build some kind of computrace for macbooks that will be much useful that current Find My Mac.

<comment><USER9> >But this mode just allows you to trace device via IP geolocation since macbook never had GPS module like iphones do, so this geolocation is not accurate enough and not useful. You will just see the city or district located from ISP.

That's not quite true. IIRC, even without a GPS module, macOS sends a list of nearby WiFi APs to the geolocation API, which usually allows for a pretty accurate response (the same you'd get on an iOS device indoors without GPS reception)

<comment><USER8> But I wan't to know the exact list of BSSID/SSID and RSSI of each so I can go to this place and find device by measure signal level of wifi access points.

<comment><USER9> This is exactly what the API uses to triangulate your position.

</comment></comment></comment></comment><comment><USER10> Have a look at [https://www.preyproject.com/](https://www.preyproject.com/)

Open source, cross platform.

<comment><USER11> This is pretty neat, but do note that the remote wipe feature as described by OP requires a subscription: [https://www.preyproject.com/pricing](https://www.preyproject.com/pricing). But I suppose this isn't a huge issue if you're already using full-drive encryption.

<comment><USER12> Those prices are nuts

<comment> angry-hacker Why nuts?

<comment><USER12> $5/mo for something I may never use. It's not even offering replacement insurance. I get all the music I want for double that (spotify) or all the streaming (netflix).

</comment></comment></comment></comment></comment><comment><USER13> Set up your Linux laptop's filesystems to use LVM and LUKS encryption, and just encrypt the whole darn thing. Works with hybernation, too. Here's Slackware's simple guide: [http://ftp.slackware.com/pub/slackware/slackware-14.1/README...](http://ftp.slackware.com/pub/slackware/slackware-14.1/README_CRYPT.TXT)

If you want to be super paranoid, add a keychain usb stick as a required key to decrypt the filesystem so you get 2-factor authentication.

<comment> yjftsjthsd-h > If you want to be super paranoid, add a keychain usb stick as a required key > to decrypt the filesystem so you get 2-factor authentication.

If you do this, make sure to have a backup! Flash drives don't last forever, and are easier to lose than a laptop.

</comment></comment><comment><USER14> Make sure you get a laptop with a self-encrypting SSD that supports TCG OPal. This will give you maximum speed sector-level encryption. Read this post on my nerd-blog: [https://vxlabs.com/2015/02/11/use-the-hardware-based-full- di...](https://vxlabs.com/2015/02/11/use-the-hardware-based-full-disk- encryption-your-tcg-opal-ssd-with-msed/) (no ads, no referrals, really just info) which explains at a high level how SSD-based encryption works.

The open-source msed tool has now been renamed to sedutil see [https://github.com/Drive-Trust-Alliance/sedutil](https://github.com/Drive- Trust-Alliance/sedutil) but it still works the same way.

It would still be possible for a sufficiently advanced thief to secure erase the drive (they need to know how to use TCG Opal to do that), but they will never see your data.

<comment><USER15> My list of SSD models that support built-in encryption, from a few years ago when I was looking into them. Outdated and possibly incomplete, but a reasonable place to start if you're not just going to pick up a Samsung Evo or Evo Pro.

Samsung 840 & 850 drives (EVO and PRO)

Crucial MX100 and MX200, but NOT BX100

Sandisk X300s

Kingston KC300

OCZ ARC 100

OCZ Radeon R7

OCZ Vector 180

PNY CL4111

Intel 520 series (128-bit only? Old model)

Intel 530 series (old)

Intel 535 series

Intel 730 series

</comment></comment><comment><USER16> How does built-in disk encryption on OSX compare to 3rd party solutions?

<comment><USER17> Good question. I've not seen any serious technical analyses of the MacBook. So far only that one must enable an option not to store the keys in memory if one expects to remain secure when the computer isn't powered off.

</comment></comment><comment><USER18> Choose a disk that supports Full Disk Encrpytion. Lenovo has this - [http://www.lenovo.com/support/fde](http://www.lenovo.com/support/fde). This way the encryption/decryption is done by the chip inside HDD and there is no OS security issue that can compromise the data or performance hit on the main cpu. Don't forget to set the HDD password in the BIOS. Everytime system boots, it should ask you for the disk password as first step. If it does not ask the password, then you have not set it up right.

On the Linux OS itself, follow good personal security practices - use strong password, use 2FA (see FIDO devices like YubiKey), disable unnecessary services, install software downloaded from trusted, well-reviewed sources only etc. If you did the HDD encryption above, there is no need to do filesystem encryption again in Linux.

<comment><USER5> > Choose a disk that supports Full Disk Encrpytion ... This way the > encryption/decryption is done by the chip inside HDD and there is no OS > security issue that can compromise the data or performance hit on the main > cpu.

The parent is referring to Self Encrypting Drives (SED), AFAICT. I looked into them a little recently, but I'm not an expert. Consider the following only a starting point:

Beware that not every SED tech is equally secure; some are easily bypassed. The industry standard, and the one I would depend on, is Opal. It usually requires tools in the OS to activate, but I would be surprised if those tools weren't available for major Linux distros.

[https://www.trustedcomputinggroup.org/storage-work-group- sto...](https://www.trustedcomputinggroup.org/storage-work-group-storage- security-subsystem-class-opal/)

Microsoft provides something called eDrive, which AFAICT (I looked at it briefly) integrates Windows Bitlocker with SEDs.

> Lenovo has this

SED tech is a feature of the hdd/ssd, not the computer vendor. The BIOS has to integrate with the SED but I think that is standard, at least in business- class computers (but double-check before you buy!).

</comment></comment><comment><USER19> The X220 comes with CompuTrace and Intel Anti-Theft, which provide the same features on Windows.

One big feature: They also backdoor any new Windows installations on the same device, so if your hacker wipes or removes your hard disk, his new replacement install will be bricked as well.

As such, step one will be hard disk encryption (as mentioned by others) so thieves can't access your data. Step two, if that's not enough for you, is activating either of the two anti-theft measures to brick the device if the hacker tries to reinstall Windows.

</comment><comment><USER20> [https://github.com/cgarduno1garduno/Macbook- Security/blob/ma...](https://github.com/cgarduno1garduno/Macbook- Security/blob/master/README.md)

I found this issue a while back on Macbooks. I'll update my github soon with more details and some images to demonstrate the process. I read this post and I figured I could whip something together and see what people think.

</comment><comment><USER21> As long as you had FileVault enabled (& good password management), that's basically all you needed.

Without FileVault, you can totally still access the laptop and data. Sadly Macs do not have anywhere near the device protection that iPhones do - <USER2> the new ones. But data encryption is what you need.

You can do that same in the installer for Ubuntu Linux. I personally prefer the encrypt home directory option over the full disk option, but there are trade-offs.

<comment><USER5> > As long as you had FileVault enabled (& good password management), that's > basically all you needed.

Does that protect your computer if it's asleep? My guess is most Macbooks are either asleep or on 99% of the time.

<comment><USER22> > You may wish to enforce hibernation and evict FileVault keys from memory > instead of traditional sleep to memory:

      $ sudo pmset -a destroyfvkeyonstandby 1
      $ sudo pmset -a hibernatemode 25

> If you choose to evict FileVault keys in standby mode, you should also > modify your standby and power nap settings. Otherwise, your machine may wake > while in standby mode and then power off due to the absence of the FileVault > key. See issue #124 for more information. These settings can be changed > with:

      $ sudo pmset -a powernap 0
      $ sudo pmset -a standby 0
      $ sudo pmset -a standbydelay 0
      $ sudo pmset -a autopoweroff 0

See [1] for more macOS hardening tricks.

[1] [https://github.com/drduh/macOS-Security-and-Privacy- Guide#fu...](https://github.com/drduh/macOS-Security-and-Privacy-Guide#full- disk-encryption)

<comment><USER23> Apple's own Best Practices doc for FileVault 2 is also useful read for those interested: [http://www.training.apple.com/pdf/WP_FileVault2.pdf](http://www.training.apple.com/pdf/WP_FileVault2.pdf)

</comment></comment></comment></comment><comment><USER24> Thinkpads have BIOS options for full disk encryption (requires password to decrypt at boot time) and boot password (requires password to boot).

</comment><comment><USER25> My laptop was stolen last Friday.

I don't need to be concerned about the thief accessing sensitive documents as I setup full disk encryption (dm-crypt & LUKS) with this kind of scenario in mind.

The only negative I can think of is that I cannot make use of tracking software like Prey due to the entire drive being encrypted. A trade-off that I'm happy to live with.

<comment><USER26> > The only negative I can think of is that I cannot make use of tracking > software like Prey due to the entire drive being encrypted. A trade-off that > I'm happy to live with.

You could have a separate, minimal OS as a honeypot.

</comment></comment><comment><USER27> Full Disk Encryption is the answer for you.

A side question (sorry):

So if you ever buy any mac device, will wiping it (reinstall from USB disk) remove it from the Find My Phone system, or does the previous owner (if they set it up) keep the option of locking the device remotely and holding it to ransom?

<comment><USER28> The seller has to disassociate the device with their account or you have to reset the PRAM. A simple wipe and OS reinstall will not be enough. If you are doing an in-person sale you should insist that the seller login to the machine with you watching, go to the iCloud pref panel and show that Find My Mac is turned off and the system has no associated iCloud account. Since having the device still associated with their account would give you access to their pics et al this is probably a good idea for the seller as well.

</comment></comment><comment><USER29> I remember seeing apps that silently take photos with the webcam and upload them to a location only you can access, every time someone opens the laptop's lid and at regular intervals. May even be trivial to write such an app.

</comment><comment><USER30> If remote wipe is not possible in Linux, Maybe we also need the feature to automatically "destroy all data" when the laptop hadnt been logged in for too long.

<comment><USER31> What could possibly go wrong?

</comment></comment><comment><USER32> Can't comment on Linux - but curious if the location was discovered as well if the erase process had started? And if so, did you report it to the police?

<comment><AUTHOR> The erase process started, but the location was not found. Actually, I didn't activate location access on that device, sadly, I didn't think it could be stolen :( And yes, I did report to police when I lost it, but without the location tracking, it's hopeless.

<comment><USER33> Something similar happened to me with an old location-enabled iPad. The iPad was showing up in a different country, and I was flummoxed on how to proceed as I didn't feel my local police department was going to follow-up with separate country.

<comment><AUTHOR> True. In my case, I think the thief already sell it to another guy. Poor him, he got a locked Mac right now.

</comment></comment></comment></comment><comment><USER34> Intel's management engine can do this sort of thing (at a level below the bios), but I think it can only be activated as part of enterprise tools.

</comment><comment><USER35> There was once this thing called anti-theft or computrace... I remember having a sticker on my Vaio about that.

What was it? Was it any useful?

</comment><comment><USER36> Is there a way to encrypt the hdd, an option like mac on "find my phone", remote erase on windows 10 on dell.</comment>