Introducing a new kind of Wi-Fi system - <AUTHOR> https://blog.google/products/google-wifi/introducing-new-kind-wi-fi-system/ <comment><USER2> Just for everyone focusing on gimmicky features, this is an OnHub, which are intended to just work.

Under the hood, they run ChromiumOS, including things like trusted boot (yes, this home router has a tpm), and the A/B partitions so when you get an automatic update, (which happens during a time of low bandwidth usage), it reboots into the new version in about 6 seconds. The security team is awesome: they pushed an update to all onhubs within 48 hours of public disclosure of a critical RCE earlier this year.

There has been some slowness to expected features, like ipv6, but the PMs have been clear about their goal: they won't include a feature that's buggy. It's exactly what you want in a piece of infrastructure.

<comment><USER3> Wait, OnHub doesn't support IPv6?! That's inexcusable. There's no reason IPv6 couldn't "just work" too. Google Wifi had better support IPv6 at launch.

<comment><USER4> If you require IPv6 so crucially, you probably shouldn't be using consumer grade gear.

<comment><USER5> I've been using IPv6 on a consumer-grade cable modem connection with a Cisco DPC3008 modem and a previous-generation Apple Airport extreme since 2012. There weren't even any difficulties getting it working.

<comment><USER6> There aren't any difficulties if you don't get it working either (yet). The Internet still works with no issues.

</comment></comment></comment></comment></comment><comment><USER7> Lots of vendors already offer this. It's pretty standard for commercial WiFi units. It's just slightly more expensive than dumb routers.

* Eero: [https://eero.com/](https://eero.com/)

* Ubuquiti: [https://www.ubnt.com](https://www.ubnt.com)

* Cisco: [http://www.cisco.com](http://www.cisco.com) (sized for larger buildings)

Does this new Google device come with the Google Fi feature which backhauls all your traffic to Google via a VPN?

<comment><USER8> No. It comes with pretty much nothing but a nice little price break. You get a 3-pack for $299. The Eero 3-pack is $499.00. This is a serious price break in this market, which is currently over-priced. You can get 3 Ubiquities for almost $300. A lot less user friendly, but $500 for 3 units is holding back mass adoption.

Personally, for my home I need just two units. Shame there's no $199 2-pack yet. It would be an instant buy for me.

<comment><USER9> Ubiquiti's Amplifi is $199 for 3 units.

<comment><USER10> Amplifi has 3 models. The high end, the amplifi HD is $350 for a 3 pack

</comment></comment></comment></comment><comment><USER6> Serious question: whose WiFi is not working, to the degree that they think, "I really need to get a modern router to make this Internet thing actually work."

I'm seeing about a dozen fancy modern wifi routers all trying to solve a problem I'm not sure exists.

<comment><USER11> I had nothing but problems with typical consumer hardware. Having to restart routers all the time. Dead spots. When family would visit (both my wife and I have large families) they would connect at least 2 devices each to my wifi (tablet, phone, laptop, etc). My wireless would just tank. They would always ask me why my internet was so terrible.

Last year I bought an Edge router and a higher end Netgear hotspot. Works wonders separating things out and letting me easily add separate "Guest" wireless networks on different channels or whatever using my older devices.

If I were to have done that change now I would've definitely used a device like Google's. But to do something like that would've been outside my budget. I wish I could easily add new hotspots that just seamlessly worked so that I could eliminate the dead spots on the edges of my house and in my yard over time.

Maybe there's something I don't know already and I don't need Google's new Wifi System. But it definitely seems to fix my usecase.

<comment><USER12> Ubiquiti Unifi Access Points just work.

I too have bought a variety of high end consumer devices (for a small business), and they all crash or randomly drop out.

Disable wifi radio on router, plug in Unifi AP, solid connection.

They are annoying to configure, but have been so worth it.

Apple AirPort has also been rock solid.

<comment><USER13> Motion seconded on Unifi. I've just deployed these and they're so far a delight. The PoE (power-over-ethernet) support is great -- if you can get the ethernet cable to the AP's location, you're good. Allows for zero visible cable installations if you have the opportunity to do in-wall cabling.

I did spring for a "Unifi Cloud Key", which is just a small (pack of cards sized), dedicated host for running the Unifi Controller. Makes administration and updates a fair bit easier.

As someone else mentioned, they do require a wired ethernet connection, but the point of that is these are intended as (entry-level) enterprise hardware. Unifi vs. Amplifi is for contexts where 1) the administration overhead isn't a burden and 2) it's preferable to run cable and leave wifi bandwidth available for clients and/or the rich multiple network support (vs. mesh networking). Obviously, #1 means it's not intended for the general consumer market.

<comment><USER14> I wish they didn't do 24V PoE. I know they are coming from a WISP background, but a AP seems like something that should support 802.11af. It's not like it's something exotic.

<comment><USER13> Note that the Unifi AP AC Pro (which I'm using) are 802.3af/at (PoE/PoE+). The AC LR and AC Lite are indeed passive 24v. So there's that option within the line at least.

</comment></comment></comment></comment></comment></comment><comment><USER15> Despite their claims of being designed for user privacy, I'm wary of anything Google puts out. What information does it report back to the mothership?

<comment><USER16> I feel the same way. Was also disappointed with my Chromecast after it just stopped being able to connect to wifi and eventually wouldn't even boot at all. What kind of warranty and privacy disclosures does this solution come with?

</comment></comment><comment><USER17> I'm surprised nobody here has yet questioned the wisdom of allowing Google access to literally all of your internet traffic. This is actually a commercial product that you pay money for, so maybe they're all aboveboard here, but with Google's history of trying to get as much personal data as possible about everybody makes me unwilling to trust them with something like this.

On a different note, is there any reason why someone should prefer this to Ubiquiti's AmpliFi? Unlike Google, Ubiquiti has a long history of making networking gear. The only obvious benefit I see right now is Google Wifi starts at $129 for one unit whereas AmpliFi starts at $199, but that $199 includes 2 "mesh points" and presumably to get the same effect with Google Wifi you'd have to shell out $299 for the 3-pack.

<comment><USER18> I realize it's popular to peddle conspiracy theories about Google in some circles but think about just how many people use Google search, Gmail, Chrome, Android, etc. – do you really think their choice of routers matters more?

There's also an inconvenient truth that such problems are greatly hindered by … Google in their prominent role improving security for average users. If their goal was to spy on people, they'd be the last to push HTTPS, HSTS, HKP, various TLS hardening, etc. rather than one of the first.

<comment><USER19> None of those technologies you listed make it harder for Google to collect information from you, but instead make it harder for 3rd parties to intercept that data in transit.

<comment><USER20> If they were attempting to collect information via your router, those technologies would absolutely hamper them.

<comment><USER18> Exactly right

</comment></comment></comment></comment></comment><comment><AUTHOR> I just bought an OnHub last year. WTF Google?!

<comment><USER21> > I just bought an OnHub last year. WTF Google?!

Do you get upset when a new phone is introduced a year after you bought your last one? A new car?

<comment><USER22> The problem is Google sells products like the OnHub with indications that more is coming. It was strongly hinted that the OnHub was going to be the center of Google's soon to come Google Home solution.

Of course, those of us seasoned with the experience of buying Google products before should know better, but it really doesn't change the fact that people who bought it are absolutely right to feel upset.

<comment><USER2> This is part of Onhub, which can be a part of the mesh, and they both use the same app.

</comment></comment></comment></comment><comment><USER23> I'm a little concerned that an android device is listed as _required_... i would hope all functions of the wifi devices can be configured/administrated from the web or just an internal browser pointed to the device(s).

<comment><USER24> There is no Web interface; you really do have to use the app.

<comment><USER25> Well, that's a deal breaker. I don't own an Android device. There's no excuse for not providing a web interface.

<comment><USER26> I own an Android device and it's still a dealbreaker.

I thought it was cool to deploy a web application from my smartphone but after having done it once or twice, and two or three times having to hotfix production stuff from my smartphone, I can say it's not a device made to configure anything except alarm clocks.

<comment><USER24> In this case, OnHub has pretty much no configuration other than the SSID and password.

<comment><USER27> That's terrible. What about:

\- Proxy settings \- MAC address cloning \- Port forwarding \- Strict security controls \- Wifi bands, 2.4 vs 5GHz (I guess this is less important) \- Logs

<comment><USER24> It does have port forwarding buried under advanced settings but I didn't see the others.

</comment></comment></comment></comment></comment></comment></comment><comment><USER28> My Airports work just fine and have worked for years. I think that the WiFi issues are mostly that people are only willing to pay like $50 for their WiFi router. At this price point I'm not sure it changes anything.

<comment><USER29> If you try to use multiple Airport routers together without Ethernet you get severely reduced bandwidth. These routers (and other similar products) have multiple radios which allow mesh networking without reducing bandwidth as much. In a large house where running cable isn't an option having a mesh network can solve a lot of issues. In a small apartment a single access point is probably sufficient.

<comment><USER28> They have multiple radios and my bandwidth is basically capped by my comcast internet at around 180 Mbps which I get everywhere in the house with 2 airports, one extending the other.

</comment></comment></comment><comment><USER30> I actually just got a "Wi-Fi booster"[1] for my place to have good Wi-Fi upstairs and it's amazing (it even has an ethernet port)

[1]: [https://www.amazon.fr/Netgear- WN3000RP-200FRS-R%C3%A9p%C3%A9...](https://www.amazon.fr/Netgear- WN3000RP-200FRS-R%C3%A9p%C3%A9teur- Wifi-N300/dp/B00MWNWEIE/ref=sr_1_1?ie=UTF8&qid=1475607805&sr=8-1&keywords=wifi+booster)

</comment><comment><USER31> Anyone notice they are using their new TLD?

<comment><USER32> Yeah, I thought they were only using that internally. Well, there used to be an April Fool's page at elgoog.google which was just a mirror-image of the homepage, but that's gone now.

<comment><USER33> I think you mean [http://com.google](http://com.google), which is indeed now just a redirect. I think someone discovered a vulnerability in it.

</comment></comment></comment><comment><USER34> I guess the lay persons question at this point is why not just publish the software and dispel any notion of privacy infringement?

</comment><comment><USER35> The list of features is missing one important item \- Using these we will be better able to monitor your indoor whereabouts.

<comment><USER36> And in specific, lookup Tomographic Motion Detection ala:

[https://www.youtube.com/watch?v=Y8updJWoSxE](https://www.youtube.com/watch?v=Y8updJWoSxE)

[http://www.securityelectronicsandnetworks.com/articles/2014/...](http://www.securityelectronicsandnetworks.com/articles/2014/06/24/xandem- tmd-tomographic-motion-detection-review)

<comment><USER37> This is a nice piece of recent research: Keystroke Recognition Using WiFi Signals

[0]: [https://www.sigmobile.org/mobicom/2015/papers/p90-aliA.pdf](https://www.sigmobile.org/mobicom/2015/papers/p90-aliA.pdf)

</comment></comment></comment><comment><USER38> > The system uses a technology called mesh Wi-Fi (something usually only seen > in expensive commercial installations).

This is misleading. You can create your own mesh networks with several off- the-shelf routers / access points using open source software like batman- advanced [0].

[0] [https://www.open-mesh.org/projects/batman-adv/wiki](https://www.open- mesh.org/projects/batman-adv/wiki)

<comment><USER21> I don't think its misleading to anyone -- the kind of people to whom aftermarket open source software for routers is a reasonable solution won't be misled, and its completely accurate for the people for whom what comes built in with the router is the only thing of interest.

<comment><USER38> But that's different from saying "something usually only seen in expensive commercial installations". And it is still misleading, accuracy of some information has nothing to do with what someone is looking for (a self- contained, plug-and-play mesh network).

<comment><USER21> > But that's different from saying "something usually only seen in expensive > commercial installations".

No, its not. The fact that there are inexpensive alternative means by which it _can_ be deployed, and by which certain segments of the population _can_ and _do_ deploy it, doesn't make it any less true that it is _usually_ only seen in expensive commercial installations.

> And it is still misleading if it tends to mislead one group of people more > than another group.

I don't think it tends to mislead any group at all. I think that the group to whom the exceptions to the "usual" case accurately described here is relevant will be well aware of it and so not misled, and the group who is not interested will also not be misled by the accurate statement, even though they are likely to be less aware of the nature of the alternative.

<comment><USER38> >> But that's different from saying "something usually only seen in expensive commercial installations".

> No, its not. The fact that there are inexpensive alternative means by which > it can be deployed, and by which certain segments of the population can and > do deploy it, doesn't make it any less true that it is usually only seen in > expensive commercial installations.

Well then they should have made that more clear. Expensive + commercial means, well, expensive, and commercial installation means proprietary and hard to access (which, by the way, is also untrue).

> I don't think it tends to mislead any group at all. I think that the group > to whom the exceptions to the "usual" case accurately described here is > relevant will be well aware of it and so not misled, and the group who is > not interested will also not be misled by the accurate statement, even > though they are likely to be less aware of the nature of the alternative.

Sorry, I edited my comment for clarity. Anyway I don't think it's right to just say that because a certain population can't be mislead, it means that the information itself is not misleading.

<comment><USER21> > Well then they should have made that more clear.

"Usually" generally means that they are exceptions, but they are less common. There's no lack of clarity here, you are just ignoring that word.

<comment><USER38> Of course I am ignoring the word "Usually". It's not a substantial word. How much is usually? There are always "uncommon exceptions" to some subject (see the implicit flaw with this sentence? That's because I don't use the word "usually". What kind of an argument can I make without the word "usually"?)

It disregards a whole field of volunteered hard work to make mesh networking technology accessible by people who don't want to buy "expensive, commercial" hardware and software.'

It sends the message: "Don't look further, because it's not worth it -- _because usually other solutions are expensive and commercial_."

<comment><USER21> > Of course I am ignoring the word "Usually".

Then you are criticizing something other than their actual description.

<comment><USER38> > Then you are criticizing something other than their actual description.

No, I'm criticizing the wording. Perhaps you should ask yourself more questions about the construction of these descriptions and what messages they convey, than merely the definition of the word _usually_.

<comment><USER21> > No, I'm criticizing the wording.

Well, except that you are _expressly_ ignoring a key word in doing that.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER39> The problems with this are the same as the rest of the things announced today(except the pixel, in my opinion): Too expensive, and not much to set it apart.

<comment><USER40> I believe Google Home is cheaper than Amazon's Echo too. But in general I agree with you.

</comment></comment><comment><USER41> I've used my Airport Express to extend my Airport Base Station for years. Is this "mesh networking?" How is Google WiFi different?

<comment><USER29> If you ran an Ethernet cable you probably get a similar experience. Otherwise Airport routers use a single radio for communicating with the other access point and clients. This greatly reduces bandwidth.

</comment></comment><comment><USER42> Why are all these mesh Wi-Fi products (Google Wifi, Eero, UBNT AmpliFi, ...) only available in the US?

<comment><USER43> Ubiquity products are available in Europe.

<comment><USER42> "The AmpliFi online store is currently available only for US-based customers."

<comment><USER43> You have to buy from a European reseller.

<comment><USER42> Since the power plug is built into the repeaters, I assume I won't be able to plug it directly into my Australian wall socket.

<comment><USER44> Maybe. I've modded more than one device from the US for AU power by changing 1 capacitor for a higher voltage unit.

I believe European mains is 220 volts and our Australian mains is 230 volts, so it should be fine. I think.

Might need an adapter or change the plug.

</comment></comment></comment></comment></comment></comment><comment><USER45> Anyone know if this is going to make use of modern AQM (fq_codel or cake) like OnHub did? OnHub used the Qualcomm brand name for it: "Streamboost".

</comment><comment><USER46> How is Mesh networking different from a wireless repeater? Yes, there's a nicer UI for this, but are there any technical advantages?

<comment><USER38> The difference between the two isn't in the UI. Wireless repeaters traditionally have a set receiving and transmission point. Let's assume that signal interference / reflection / amplification does no matter, for the sake of simplicity in this example. So if you have a chain of three repeaters A <\--> B <\--> C, and then you move access point C closer to A (closer than B is to A), then the topology does not change (do you see the issue here?). Worse, if you have a longer chain with a few alternatives, then a single point going down (call it X) would require nodes previously connected to X to be reconfigured.

Mesh networks automatically judge the path of least resistance to a destination, and distribute the state of all the access points on a network. If one access point goes down, all the access points previously connected to the faulty access point will find alternatives if there's anything else within signal range.

In this case, if those are meant to be portable access points, you can move them between different rooms without worrying about nodes choosing a suboptimal path.

</comment></comment><comment><USER47> Mesh-based Device Triangulation

</comment><comment><USER48> You can tell this product is made by people who live in suburban McMansions. In a 1 BR NYC apartment, you have no problems with wifi coverage.

<comment><USER21> > You can tell this product is made by people who live in suburban McMansions. > In a 1 BR NYC apartment, you have no problems with wifi coverage.

More relevantly, _for_ instead of _by_ (though its quite possible both are true), but, sure, in a small urban apartment, coverage isn't going to be your big issue, and you aren't going to need a set of routers forming a mesh network.

OTOH, lots of people _don 't_ live in small urban apartments and use the internet.

</comment></comment><comment><USER49> Introducing a new way to spy on you? A WiFi repeater will do the trick...</comment>


Show HN: MicroFly - An Open Hardware, Open Software , 4-inch flying machine - <AUTHOR> http://kck.st/JbZjta

<comment><USER2> I was really excited about this until I realized you're linking straight to your kickstarter page. You lost me.</comment>


Ban Internet Explorer (UK Petition) - <AUTHOR> http://epetitions.direct.gov.uk/petitions/70781

<comment><USER2> No need for spamming government with unproductive petitions: IE 11 is going to be evergreen. Problem solved!

</comment><comment><USER3> Surely we want IE to increase it's market share? I for one am old enough to remember the mono-culture of IE dominated web browsers in the early 2000s, and how bad it was.

More diverse rendering engines makes the web more robust.

Yeah, it makes development harder, too.

</comment><comment><USER4> A better petition would have been to ban <IE10 use within Government organisations.

There's no way in the world that the Govt. could ban IE outright.</comment>


Minefield, HTML5 Massively Multiplayer Online Minesweeper - <AUTHOR> http://mienfield.com/ <comment><USER2> I wonder if it's possible to create a minesweeper layout that can't be solved without guessing at least once or not.

<comment><USER3> All layouts technically fit that description, since you have to guess once for your first move.

Beyond that, though, a hollow 3x3 ring of mines also requires guessing, since there's no way to tell whether the center square is a mine or not. There's lots of other such situations as well.

<comment><USER2> Depends on the implementation - some (e.g. Windows') don't fit there as it is dynamically generated after the first click so the player never loses on the first move other than in "retry this layout" modes.

Interesting point with the 3x3 ring.

</comment></comment></comment><comment><USER4> This is a good concept, and executed well. What is the backend and is this open source?

</comment><comment><USER5> The title of this post got my upvote!</comment>


Get Started with Google's Opens Source Deep Learning - <AUTHOR> http://tensorflow.org/get_started <comment><USER2> Backend is C++. Means it's easier to integrate with existing applications. Looks easier than Caffe. I can't use Theano because I need to be able to compile to a native application.

</comment><comment><USER3> Also see: [https://news.ycombinator.com/item?id=10532957](https://news.ycombinator.com/item?id=10532957) for broader discussion</comment>


The Problem With WordPress - <AUTHOR> http://betagazette.com/2010/01/18/the-problem-with-wordpress/

<comment><USER2> I say it is a better idea to use a well-developed presentation system (like Wordpress) and if you don't like using its user interface use a replacement user interface (example: MarsEdit). Then you are only being heterodox in how you produce articles, but not in how you serve them.</comment>


Files Are Hard - <AUTHOR> http://danluu.com/file-consistency/ <comment><USER3> Pretty much spot on. Local in-kernel file systems are hard, partly because of their essential nature and partly because of their history. A lot of the codebases involved still show their origins on single-core systems and pre-NCQ SATA disks, and the development/testing methods are from the same era. The developers always have time to improve the numbers on some ancient micro- benchmark, but new features often get pushed to the LVM layer (snapshots), languish for ages (unions/overlays), or are simply ignored (anything alternative to the fsync sledgehammer).

The only way a distributed file system such as I work on can provide sane behavior and decent performance to _our_ users is to use local file systems only for course-grain space allocation and caching. Sometimes those magic incantations from ten-year-old LKML posts don't really work, because they were never really tested for more than a couple of simple cases. Other times they have unexpected impacts on performance or space consumption. Usually it's easier and/or safer just to do as much as possible ourselves. Databases - <USER2> local and distributed - are in pretty much the same boat.

Some day, I hope, all of this scattered and repeated effort will be combined into a common library that Does All The Right Things (which change over time) and adds features with a common API. It's not quite as good as if the stuff in the kernel had been done right, but I think it's the best we can hope for at this point.

<comment><USER4> What distributed filesystem do you work on?

<comment><USER3> I'm on the Gluster team.

<comment><USER5> Could you provide any recommendations on reading materials for

1\. File systems (single machine)

2\. Distributed file systems

</comment></comment></comment></comment><comment> psi-squared AIUI, ZFS was explicitly designed to deal with this sort of data corruption - one of the descriptions of the design I've heard is "read() will return either the contents of a previous successful write() or an error". That would (in principle) prevent the file containing "a boo" or "a far" at any point.

It looks like one of the authors cited in this article has written a paper analysing ZFS - though they admittedly don't test its behaviour on crashes. Citation here, in PDF form:

[http://pages.cs.wisc.edu/~kadav/zfs/zfsrel.pdf](http://pages.cs.wisc.edu/~kadav/zfs/zfsrel.pdf)

(edited to add: This only deals with the second part of this article. The first part would still be important even on ZFS)

<comment><USER6> Right, Copy-On-Write filesystems (ZFS, Bttr) are explicitly designed to prevent that kind of corruption by never editing blocks in place, but rather copying the contents to a new block and using a journaled metadata update to point the file at it's new block.

ZFS also includes features around checksumming of the metadata. "Silent" write errors become loud the next time data is accessed and the checksums don't match. This can't prevent all errors, but has some very nice data integrity properties - Combined with it's RAID format, you can likely recover from most any failures, and with RAIDZ2, you can recover from a scattered failures on all drives even if one drive has completely died. This is actually fairly common - Modern drives are very large, and rust is more susceptible to 'cosmic rays' than one might think.

</comment></comment><comment><USER7> There is an easy way to write data without corruption. First copy your file- to-be-changed as a temporary file or create a temporary file. Then modify the temporary file and write whatever you want in it. Finally, use rename() to _atomically_ replace the old file by the temporary one.

The same logic also apply to directories, although you will have to use links or symlinks to have something really atomic.

It may not work on strangely configured systems, like if your files are spread over different devices over the network (or maybe with NFS). But in those cases you will be able to detect it if you catch errors of rename() and co (and you should catch them of course). So no silver bullet here, but still a good shot.

<comment><USER8> I'm surprised rename() wasn't mentioned in the article, it's a well known technique to atomically update a file, which is very practical for small-ish files.

Note that in the general case, you should fsync() the temporary file before you rename() it over the original - but ext3 and ext4 in writeback mode added a heuristic to do that automatically, because ext3 in the default ordered mode would effectively do that and many applications came to assume it.

<comment><USER9> rename is atomic, but it is not guaranteed to be durable. In order for rename to be durable, I've learned that you have to fsync the parent directory.

I was saddened when I learned this. I used to use this old trick for a lot of my systems. I learned it from reading the code of old well-written unix systems programs.

</comment></comment></comment><comment><USER10> "How is it that desktop mail clients are less reliable than gmail...?"

Made me chuckle. I've been told off by a former Googler colleague enough times now to have learned that Gmail is more complex than anyone imagines on a first guess, in order to be "reliable".

<comment><USER11> It is certainly the google service that I use the most. In a decade of quite heavy usage I remember one outage of a 1-2 hours (with no data loss). To me this is the gold standard that the rest of us should aspire to. :)

<comment><USER12> Lately (last year or so) I've started to notice substantial data loss. Either old mails completely missing or large mails being truncated (destroying inline images f.ex.)

So to anyone relying on gmail for safe keeping: Don't.

<comment><USER11> No offense (I have no idea who you are and how much you know), but...

Do you feel you are you competent enough (in e.g. SMTP and MIME) to distinguish between MIME-encoded base64 inline images (super rare) and references to images on external web sites (a lot more common)? Which in the case of old web sites are quite likely to stop working if you revisit old mails.

Did you make the effort to verify that it was the first type when you witnessed this?

<comment><USER13> You really could have just worded that as, 'are you sure those images weren't linked to external websites' without resorting to questioning his competence and trying to cover it with a 'no offense' (laff).

<comment><USER14> That's a good point - that would have been less loaded. (The question remains the same though, doesn't it?)

Also - your suggestion would have been a lot less douchey without that "laff".

</comment></comment></comment></comment></comment></comment><comment><USER15> I've recently been playing with nbdkit, which is basically FUSE but for block devices rather than file systems.

I was shocked to discover that mke2fs doesn't check the return value of its final fsync call. This is compounded by the fact that pwrite calls don't fail across NBD (the writes are cached, so the caller's stack is long gone by the time the get flushed across the network and fails...)

As a test, I created an nbdkit plugin which simply throws away every write. Guess what? mke2fs will happily create a file system on such a block device and not report failure. You only discover a problem when you try to mount the file system.

</comment><comment><USER16> The article's table of filesystem semantics is missing at least one X: Appends on ext4-ordered are not atomic. When you append to a file, the file size (metadata) and content (data) must both be updated. Metadata is flushed every 5 seconds or so, data can sit in cache for more like 30. So the file size update may hit disk 25s before the data does, and if you crash during that time, then on recovery you'll find the data has a bunch of zero bytes appended instead of what you expected.

(I like to test my storage code by running it on top of a network block device and then simulating power failure by cutting the connection randomly. This is one problem I found while doing that.)

<comment><USER17> Wow, 5 and 30 _seconds_ before metadata and data flush? It sounds unbelievably long. If it's true, almost every power loss results in a data loss of whatever was written in the last 15 seconds, on average? Is it so bad?

I'd expect more "smartness" of Linux, like, as soon as there is no "write pressure" to flush earlier.

<comment><USER16> > If it's true, almost every power loss results in a data loss of whatever was > written in the last 15 seconds, on average? Is it so bad?

No, because correct programs use sync() and/or fsync() to force timely flushes.

A good database should not reply successfully to a write request until the write has been fully flushed to disk, so that an "acknowledged write" can never be lost. Also, it should perform write and sync operations in such a sequence that it cannot be left in a state where it is unable to recover -- that is, if a power outage happens during the transaction, then on recovery the database is either able to complete the transaction or undo it.

The basic way to accomplish this is to use a journal: each transaction is first appended to the journal and the journal synced to disk. Once the transaction is fully on-disk in the journal, then the database knows that it cannot "forget" the transaction, so it can reply successfully and work on updating the "real" data at its leisure.

Of course, if you're working with something that is not a database, then who knows whether it syncs correctly. (For that matter, even if you're working with a database, many have been known to get it wrong, sometimes intentionally in the name of performance. Be sure to read the docs.)

For traditional desktop apps that load and save whole individual files at a time, the "write to temporary then rename" approach should generally get the job done (technically you're supposed to fsync() between writing and renaming, but many filesystems now do this implicitly). For anything more complicated, use sqlite or a full database.

> I'd expect more "smartness" of Linux, like, as soon as there is no "write > pressure" to flush earlier.

Well, this would only mask bugs, not fix them -- it would narrow the window during which a failure causes loss. Meanwhile it would really harm performance in a few ways.

When writing a large file to disk sequentially, the filesystem often doesn't know in advance how much you're going to write, but it cannot make a good decision on where to put the file until it knows how big it will be. So filesystems implement "delayed allocation", where they don't actually decide where to put the file until they are forced to flush it. The longer the flush time, the better. If we're talking about a large file transfer, the file is probably useless if it isn't fully downloaded yet, so flushing it proactively would be pointless.

Also flushing small writes rather than batching might mean continuously rewriting the same sector (terrible for SSDs!) or consuming bandwidth to a network drive that is shared with other clients. Etc.

<comment><USER17> > this would only mask bugs, not fix them

If I get a corruption once in 100 outages instead of on every one, I'm satisfied. That it "masks" anything is not an argument at all.

The writes happen in bursts. The behavior of bursts won't change if one more write is done after the burst is over (an example) only a second later instead of waiting 30.

The "delayed allocation" is a red herring: in the optimal case, the software can instruct the filesystem to preallocate the whole file size without having to actually fill the content. If it's not common by some specific applications on Linux, that's the place to fix it.

</comment></comment></comment></comment><comment><USER18> This problem can be fixed. We need to rethink file system semantics. Here's an approach:

Files are of one of the following types:

 _Unit files_

For a unit file, the unit of consistency is the entire file. Unit files can be created or replaced, but not modified. Opening a unit file for writing means creating a new file. When the new file is closed successfully, the new version replaces the old version atomically. If anything goes wrong, including a system crash, between create and successful close, including program abort, the old version remains and the new version is deleted. File systems are required to maintain that guarantee.

Opens for read while updating is in progress reference the old version. Thus, all readers always see a consistent version.

They're never modified in place once written. It's easy for a file system to implement unit file semantics. The file system can cache or defer writes. There's no need to journal. The main requirement is that the close operation must block until all writes have committed to disk, then return a success status only if nothing went wrong.

In practice, most files are unit files. Much effort goes into trying to get unit file semantics - ".part" files, elaborate file renaming rituals to try to get an atomic rename (different for each OS and file system), and such. It would be easier to just provide unit file semantics. That's usually what you want.

 _Log files_

Log files are append-only. The unit of consistency is one write. The file system is required to guarantee that, after a program abort or crash, the file will end cleanly at the end of a write. A "fsync" type operation adds the guarantee that the file is consistent to the last write. A log file can be read while being written if opened read-only. Readers can seek, but writers cannot. Append is always at the end of the file, even if multiple processes are writing the same file.

This, of course, is what you want for log files.

 _Temporary files_

Temporary files disappear in a crash. There's no journaling or recovery. Random read/write access is allowed. You're guaranteed that after a crash, they're gone.

 _Managed files_

Managed files are for databases and programs that care about exactly when data is committed. A "write" API is provided which returns a status when the write is accepted, and then makes an asynchronous callback when the write is committed and safely on disk. This allows the database program to know which operations the file system has completed, but doesn't impose an ordering restriction on the file system.

This is what a database implementor needs - solid info about if and when a write has committed. If writes have to be ordered, the database program can wait for the first write to be committed before starting the second one. If something goes wrong after a write request was submitted, the caller gets status info in the callback.

This would be much better than the present situation of trying to decide when a call to "fsync" is necessary. It's less restrictive in terms of synchronization - "fsync" waits for everything to commit, which is often more than is needed just to finish one operation.

This could be retrofitted to POSIX-type systems. If you start with "creat" and "O_CREAT" you get a unit file by default, unless you specify "O_APPEND", in which case you get a log file. Files in /tmp are temporary files. Managed files have to be created with some new flag. Only a small number of programs use managed files, and they usually know who they are.

This would solve most of the problems mentioned in the original post.

</comment><comment><USER19> I worked at a storage company and the scariest thing I learned is that your data can be corrupt even though the drive itself says that the data was written correctly. The only way to really be sure is to check your files after writing them that they match. Now whenever I do a backup, I always go through them one more time and do a byte-by-byte comparison before being assured that it's okay.

<comment><USER20> This is true. Which is why we really, really need checksummed filesystems. I am very worried that this hasn't made its way into mainstream computing yet, especially given the growing drive sizes and massive CPU speed increases.

<comment><USER21> I run a 10x3TB ZFS raidz2 array at home. I've seen 18 checksum errors at the device level in the last year - these are corruption from the device that ZFS detected with a checksum, and was able to correct using redundancy. If you're not checksumming at some level in your system, you should be outsourcing your storage to someone else; consumer level hardware with commodity file systems isn't good enough.

<comment><USER22> How does checksumming help if the data is in cache and waiting to be written ? For ex: I have 1MB of data, i write it but it stays in buffer cache after written and when you do checksum you are computing the checksum on buffer cache .

On Linux you have to drop_caches and then read get the checksum to be sure. Now per buffer or file drop_cache isnt available as per my knowledge . If you are doing a systemwide drop_caches you are invalidating the good and bad ones.

What if now if device is maintaing cache as well in addition to buffer cache?

Can someone clarify ?

<comment><USER21> How do you know you put good data into the cache in the first place?

There's always going to be a place where errors can creep in. There are no absolute guarantees; it's a numbers game. We've got more and more data, so the chance of corruption increases even if the per-bit probability stays the same. Checksumming reduces the per-bit probability across a whole layer of the stack - the layer where the data lives longest. That's the win.

<comment><USER22> Agree whole heartedly.

I was asking this thinking of open(<file>,O_DIRECT|O_RDONLY); that bypasses buffer cache and read directly from the disk that atleast solves buffer cache i guess. The disk cache is another thing ie if we disable it we are good at the cost of performance.

I was pointing that tests can do these kind of things.

</comment></comment></comment></comment></comment></comment><comment><USER23> Great write-up and probably explains some issues in my apps a while back. Like that my long-time favorite, XFS, totally kicks ass in the comparisons. I agree on using static analysis and other techniques as I regularly push that here. What threw me is this line:

"that when they came up with threads, locks, and conditional variables at PARC, they thought that they were creating a programming model that anyone could use, but that there's now decades of evidence that they were wrong. We've accumulated a lot of evidence that humans are very bad at reasoning at these kinds of problems, which are very similar to the problems you have when writing correct code to interact with current filesystems."

There were quite a few ways of organizing systems, including objects and functions, that the old guard came up with. UNIX's popularity and organization style of some others pushed the file-oriented approach from mainstream into outright dominance. However, many of us long argued it was a bad idea and alternatives exist that just need work on the implementation side. We actually saw some of those old ideas put into practice in data warehousing, NoSQL, "the cloud," and so on. Just gotta do more as we don't have a technical reason for dealing with the non-sense in the write-up: just avoiding getting our hands dirty with the replacements.

<comment><USER24>
        > Like that my long-time favorite, XFS, totally kicks ass in the
        > comparisons.

I think you'll find this an interesting read then: [http://teh.entar.net/~nick/mail/why-reiserfs-is-teh- sukc](http://teh.entar.net/~nick/mail/why-reiserfs-is-teh-sukc)

It's written in 2004 so I don't know how current it is, but it makes the point that XFS makes certain performance & safety guarantees essentially assuming that you're running on hardware that has a UPS with the ability to interrupt the OS saying "oops, we're out of power".

<comment><USER25> 2004 is not current for XFS, that is a decade ago! However, disks finishing writes and not lying about having done it is a critical need for all FS. For some like ext3 you would notice it less as it was flush happy.

XFS is becoming the sane default filesystem for servers as it allocates nodes more consistently than the other current mainstream linux options on multidisk systems. Basically small servers now have more disk space and performance than the large systems of 2004. So XFS stayed put in where it starts to make sense, but systems grew to meet its sweetspot much often.

<comment><USER23> Pretty good analysis.

</comment></comment></comment></comment><comment><USER26> In plan9 mailbox files, like many others, are append only.

All files are periodically (default daily) written to a block coalescing worm drive and you can rewind the state of the file system to any date on a per process basis, handy for diffing your codebase etc.

For a while the removal of the "rm" command was considered to underline the permanence of files but the removal of temporary data during the daytime hours turned out to be more pragmatic.

<comment><USER24> How does Plan 9 deal the equivalent of this append-only pattern potentially causing corruption on Unix if you have multiple writers and the writes are larger than PIPE_BUF (4k by default on Linux)?

Most users of this pattern (concurrent updates to log files) get away with it because their updates are smaller than 4k, but if you're trying to write something as big as an E-Mail with this pattern you can trivially get interleaved writes resulting in corruption.

<comment><USER26> Exclusive locking

</comment></comment></comment><comment><USER27> Surely filesystems are going to go through a massive change when SSDs push standard spinning disks into the history books? They must carry a lot of baggage for dealing with actual spinning disks, much of which is just overhead for super-fast solid state drives. Hopefully this will allow interesting features not possible on spinning disks, like better atomic operations.

<comment><USER28> "IotaFS: Exploring File System Optimizations for SSDs"

 _Our hypothesis in beginning this research was simply that the complex optimizations applied in current file system technology doesn't carry over to SSDs given such dramatic changes in performance characteristics. To explore this hypothesis, we created a very simple file system research vehicle, IotaFS, based on the incredibly simple and small Minix file system, and found that with a few modifications we were able to achieve comparable performance to modern file systems including Ext3 and ReiserFS, without being nearly as complex._

[http://web.stanford.edu/~jdellit/default_files/iotafs.pdf](http://web.stanford.edu/~jdellit/default_files/iotafs.pdf)

<comment><USER29> Yeah btrfs 'ssd' mount option does less for the same reasoning, but still does include checksums for metadata and data because SSDs have at least as much likelihood of non-deterministically returning your data as spinning rust. So even if it doesn't fix the corruptions (which requires additional copies or parity), at least there's an independent way of being informed of problems.

</comment></comment></comment><comment><USER30> I wonder how this approach (single file + log) compares to the other usual approach (write second file, move over first):

1\. Write changed the data into a temporary file in the same directory (don't touch the original file)

2\. Move new file over old file

Does this lead to a simpler strategy that is easier to reason about, where it is less likely for programmer to get it wrong? At least I see this strategy being applied more often than the "single file + log" approach.

The obvious downside is that this temporarily uses twice the size of the dataset. However, that is usually mitigated by splitting the data into multiple files, and/or applying this only to applications that don't need to store gigabytes in the first place.

<comment><USER31> That's not guaranteed to work in the face of crashes. The problem is that the directory update could get flushed to disk before the file data.

This is the fundamental problem: When you allow the OS (or the compiler, or the CPU) to re-order operations in the name of efficiency you lose control over intermediate states, and so you cannot guarantee that these intermediates states are consistent with respect to the semantics that you care about. And this is the even more fundamental problem: our entire programming methodology has revolved around describing what we want the computer to _do_ rather than what we want to to _achieve_. Optimizations then have to reverse-engineer our instructions and make their best guesses as to what we really meant (e.g. "This is dead code. It cannot possibly affect the end result. Therefore it can be safely eliminated.) Sometimes (often?) those guesses are wrong. When they are, we typically only find out about it after the mismatch between our expectations and the computer's have manifested themselves in some undesirable (and often unrecoverable) way.

<comment><USER32> "That's not guaranteed to work in the face of crashes. The problem is that the directory update could get flushed to disk before the file data."

No, it can work, provided that the temporary file is fsynced before being renamed, the parent directory is fsynced after renaming the file, and that the application only considers the rename to have taken place after the parent directory is fsynced (not after the rename call itself).

<comment><USER31> FSYNC is not enough. You also have to make sure that NCQ is disabled:

[https://en.wikipedia.org/wiki/Native_Command_Queuing](https://en.wikipedia.org/wiki/Native_Command_Queuing)

<comment><USER33> OS can use Force Unit Access flag with NCQ to control disk buffering: [https://en.wikipedia.org/wiki/Disk_buffer#Force_Unit_Access_...](https://en.wikipedia.org/wiki/Disk_buffer#Force_Unit_Access_.28FUA.29)

</comment></comment></comment></comment></comment><comment><USER34> The article only mentions Linux/Posix systems, are the same problems also present in Windows/NTFS? I was under the impression that, for example, renames on ntfs were crash safe and atomic, which would make the "write temp file then rename to target" work even if the power is cut?

<comment><USER35> NTFS is not very safe. No data integrity checksums. I think it's about same level as ext4 mostly. Meaning not very good. One shouldn't trust any critical data on NTFS without checksums and duplication.

NTFS consistency checks and recovery are pretty good. But they won't bring your data back.

Microsoft's ReFS (Reliable File System) might give storage reliability to Windows one day. On Linux you can use ZFS or btrfs (with some reservations) today.

[https://en.wikipedia.org/wiki/ReFS](https://en.wikipedia.org/wiki/ReFS)

> ...work even if the power is cut?

If power is cut during NTFS logfile update, all bets are off. Hard disks can and will do weird things when losing power unexpectedly. That includes writing incorrect data, corrupting nearby blocks, corrupting any blocks, etc. That includes earlier logfile data, including checkpoints.

</comment></comment><comment><USER29> The article makes me wonder whether there's enough abstraction being done via the VFS layer, because all this fsync business that application developers seem to have to do can be so workload and file system specific. And I think that's asking too much from application developers. You might have to fsync the parent dir? That's annoying.

I wonder if the article and papers its based on account for how the VFS actually behaves, and then if someone wanting to do more research in this area could investigate this accounting for the recent VFS changes. On Linux in particular I think this is a bigger issue because there are so many file systems the user or distro could have picked, totally unbeknownst to and outside the control of the app developer.

<comment><USER3> That's definitely asking too much of app developers. Every time someone complains about any of this, the filesystem developers come back with a bit of lore about a non-obvious combination of renameat (yes that's a real thing) and fsync on the parent directory, or some particular flavor of fallocate, or just use AIO and manage queues yourself, or whatever depending on exactly which bogus behavior you're trying to work around. At best it's an unnecessary PITA. At worst it doesn't even do what they claimed, so now you've wasted even more time. Most often it's just non-portable (I'm _so_ sick of hearing about XFS- specific ioctls as the solution to everything) or performs abominably because of fsync entanglement or some other nonsense.

We have libraries to implement "best practices" for network I/O, portable across systems that use poll or epoll or kqueues with best performance on each etc. What we need is the same thing for file I/O. However imperfect it might be, it would be better than where we are now.

<comment><USER29> Very rudimentary, but a way for an application developer to specify categories of perform/safety ratio operations. An app developer might have a simple app that only cares about performance, only cares about safety, there'd be a default in between both. Another app developer might have mixed needs depending on the type of data the app is generating. But in this way, if they write the app with a category of A (let's say that means highest safety at expense of performance) and their benchmarking determines this is crap, and they have to go to category B for writes, that's a simpler change that going back through their code and refactoring a pile of fsyncs or FUA writes.

I mean, I thought this was a major reason for VFS abstraction between the application and kernel anyway. It's also an example of the distinction between open source and free (libre). If as an application developer you have to know such esoterics to sanely optimize, you in fact aren't really free to do what you want. You have to go down a particular rabbit hole and optimize for that fs, at the expense of others. That's not fair choice to have to make.

</comment></comment></comment><comment><USER36> One of the naive assumptions that most of us make is that if there's a power failure none of the data that was fsynced successfully before will be corrupted.

Unfortunately, this is not the case for SSDs.

</comment><comment><USER37> This issue is completely fixed by Maildir and was many years ago. Many clients, including Mutt, for example, support Maildir boxes

</comment><comment> david-given One thing I've always wanted to try and never had time is to build SQlite as a kernel module, talking directly to the block device layer, and then implement a POSIX file system on top of it.

It wouldn't solve problems with the block device layer itself, but it'd be interesting to see how robust and/or fast it was in real life.

<comment><USER38> SQLite itself relies on a filesystem for the rollback journal or write-ahead log. So you'd need some kind of abstraction between the block device and SQLite. Might as well just use the existing filesystem and keep SQLite in user space, since that works.

<comment> david-given _cough_

[http://www.sqlite.org/src/doc/trunk/src/test_onefile.c](http://www.sqlite.org/src/doc/trunk/src/test_onefile.c)

<comment><USER39> It all sounded so good until the end of the header where it's described as a demo vfs. :-(

</comment></comment></comment></comment><comment><USER40> This article sheds some light on a problem I've had for years:

The software product I develop stores its data in a local database using H2, stored in a single file. Users tend to have databases gigabytes in size.

After reading this article, I start to understand why my users occasionally get corrupted databases. A hard problem to solve.

<comment><USER41> An easy problem to solve. LMDB never corrupts.

</comment></comment><comment><USER42> While I experienced this pain first hand I'm not sure that the FS deserved a 100% of the blame. There is enough to blame to go around for the userspace, filesytems, block device layer and disk controllers, hardware & firmware.

If you start at the bottom of the stack you have all sorts of issues with drives, their firmware and controllers doing their own levels of caching & re- ordering. Over the years the upper layer developers (block layer / fs) had to deal with hardware that simply lies about what happened or is just plain buggy.

</comment><comment><USER43> It appears that sqlite could be good basis for decrapified filesystem.

<comment><USER41> then you didn't read the article closely. SQLite has plenty of crash vulnerabilities.

<comment><USER43> I just saw zeros in the tables.

</comment></comment></comment><comment><USER44> I don't program much in C, or use direct system calls for files. Mostly I use Java.

Does anyone know if any of this applies to Java's IO operations. I'm sure you can force some of this behaviour, but for instance: The flush method on OutputStream, will it ensure proper sync, or is that again dependent on the OS and file system as described in the article for syscalls?

<comment><USER45> This answers your question:

[http://docs.oracle.com/javase/8/docs/api/java/io/OutputStrea...](http://docs.oracle.com/javase/8/docs/api/java/io/OutputStream.html#flush--)

It's only logical if you think about the bigger picture. Does Java have access to the underlying disk device or does it work with the filesystem? Which component is responsible for the filesystem?

</comment></comment><comment><USER46> I do agree with you, this is my story about the usual stuff: [http://www.sami- lehtinen.net/blog/chaos-monkey-bit-me-a-shor...](http://www.sami- lehtinen.net/blog/chaos-monkey-bit-me-a-short-story-about-transactionality) As well as this classic from SQLite3 guys: [https://www.sqlite.org/howtocorrupt.html](https://www.sqlite.org/howtocorrupt.html) \- As you can see, there are multiple ways to ruin your day.

</comment><comment><USER47> Ah, fond memories of async ext2 corrupting root filesystems beyond recognition... I think we 'invented' 'disposable infrastructure' back in 2002 because the filesystem forced us to... The MegaRAID cards that would eat their configuration along will all your data didn't help either..

Can't remember if we switched to ext3 in ordered or data-journaled mode but it made an immense difference...

</comment><comment><USER48> IIRC I have seen this discussion before. And the answer was do a fsync. But for sakes of performance we want to be able to issue write-barriers to a group of file handles. So we know the commands will be ran in order, and no other order.

<comment><USER41> Correct, we want to be able to write in deterministic order. SCSI has supported this natively for decades. Unfortunately SATA doesn't, and the Linux kernel pretty much doesn't, because it can't rely on the storage devices to support it.

<comment><USER49> Which is just lame, its mostly because the block layer doesn't support file based fencing. A mistake made a decade an a half ago, and no one has the will/political power to fix it.

If the block layer supported it, solving the problem of fencing an ATA device would be as simple as issuing a whole device flush instead of a SYNC_CACHE with range. Which for soft raid devices would make a huge impact because only the devices with dirty data need to be flushed.

Of course the excuse today, is that most scsi devices can't do range based sync either, and just fall back to whole device because no one uses the command. Chicken/egg.

</comment></comment></comment><comment><USER50> Isn't block level duplication/checksumming like RAID supposed to solve this hardware unreliability? I understand that by default RAID is not used on end user desktops.

<comment><USER35> AFAIK, most RAID systems do not have checksums. I may be wrong, but I think RAID5/RAID6 even amplifies error frequency.

It gets more "fun" when you consider many (most? all?) hard disks can get corrupted without checksum failures.

Layer "violators" like ZFS and btrfs do have checksums.

Maybe conventional block / filesystem layering itself is faulty.

<comment><USER51> The layers that ZFS violates were created years before the failure modes were well understood that filesystem-based checksums address. I'm not sure how you _can_ solve these issues without violating the old layers.

In particular: checksumming blocks alongside the blocks themselves (as some controllers and logical volume managers do) handles corruption within blocks, but it cannot catch dropped or misdirected writes. You need the checksum stored elsewhere, where you have some idea what the data _should_ be. Once we (as an industry) learned that those failure modes do happen and are important to address, the old layers no longer made sense. (The claim about ZFS is misleading: ZFS _is_ thoughtfully layered -- it's just that the layers are different, and more appropriate given the better understanding of filesystem failure modes that people had when it was developed.)

</comment></comment></comment><comment><USER52> Good article. You have a typo: looks like you're using Pandoc or something similar, and left out the closing parenthesis in the link after [undo log].

<comment><USER53> Yeah, that's a common error among Markdown writers. It's too easy to forget the last bracket, especially if you are putting a link inside a parenthetical comment in the first place.

Fortunately, it's easy to detect programmatically. I have a little shell script which flags problems in my Markdown files: [http://gwern.net/markdown- lint.sh](http://gwern.net/markdown-lint.sh)

In this case, you can use Pandoc | elinks -dump to get a rendered text version, and then simply grep in the plain text for various things like "-e '(http' -e ')http' -e '[http' -e ']http'"

<comment><USER52> For me, Vim's highlighting and concealing prevents this class of typos. It also makes it more pleasant to read the source, as it hides the links unless I'm editing the line.

</comment></comment></comment><comment><USER54> Does anyone know if there's a recording of the usenix talk given? (Referenced slides)

<comment><USER54> Edit: Found it. [https://youtube.com/watch?v=SVYegdh2CbE](https://youtube.com/watch?v=SVYegdh2CbE)

</comment></comment><comment><USER55> This is an interesting article but the examples given are seriously dated:

Mail servers and clients (MTAs and MUAs) have been using Maildir format to escape this problem since 1996.

Filesystems have evolved. ZFS has been available for ten years.

If you take your data seriously, you do need to make informed choices. But this article isn't targeted at people who won't know about Maildir and ZFS.

The section about relative data loss failures between various applications is great. Again, careful choices: SQLite and Postgres.

<comment><USER56> Wasn't part of the need for Maildir indexability rather than reliability?

<comment><USER55> It provides both. mbox was a known evil 20 years ago, so it's a bad example for the article.

The qmail spec describes queuing and delivery in a filesystem-safe manner.

</comment></comment></comment><comment><USER57> PG was quite proud of having just used the file system as a database with Viaweb, claiming that "The Unix file system is pretty good at not losing your data, especially if you put the files on a Netapp." His entire response is worth reading in light of the above article, if only to get a taste of how simultaneously arrogant and clueless PG can be: [http://www.paulgraham.com/vwfaq.html](http://www.paulgraham.com/vwfaq.html)

PG and company still think this is a great idea, because the software this forum runs on apparently also does the file system-as-database thing.

No wonder Viaweb was rewritten in C++.

EDIT: to those downvoting, my characterization of PG's response is correct. Its arrogance is undeniable, with its air of "look how smart I am; I see through the marketing hype everyone else falls for," as is its cluelessness, with PG advocating practices that will cause data loss. Viaweb was probably a buggy, unstable mess that needed a rewrite.

<comment><USER58> You're spending your Sunday ragging on someone with a throwaway and being hypervigilent about downvotes on your throwaway. Get a grip mate.

<comment><USER59> _being hypervigilent about downvotes on your throwaway_

Well, FWIW, it's working. At this instant his total karma is +10, he only has 2 posts, and the 2nd post is slightly greyed. So that means that his original comment is now in the range of +10.

Sadly, the fact that _I 'm_ replying here probably means that _I_ need to get a life!

</comment></comment></comment><comment><USER60> Frd

</comment><comment><USER61> All of this is great, except the first two sentences:

> I haven't used a desktop email client in years. None of them could handle > the volume of email I get without at least occasionally corrupting my > mailbox.

If I were to get so many emails that it was corrupt my mailbox, I'd first ask myself why, and how to stop that.

<comment><USER62> I wouldn't. If your daily workflow includes a high volume of email, then it does.

What I _would_ ask is: is there a way I can solve this problem without having to totally rearrange how I use email? I'm curious whether the author looked at, say, running a personal MDA that served mail over IMAP, so that it could be interacted with via a desktop email client, without requiring that client to serve as the point of truth. Not to say that corruption couldn't still happen that way, but Thunderbird (for example) can be configured to store only a subset of messages locally, or none at all. With a reasonably fast connection to the MDA, this seems like a possibly workable solution.

<comment><USER61> > If your daily workflow includes a high volume of email, then it does.

Not necessarily. For example, if you have monitoring software sending you all email notifications, you could change that to just write records to a database instead.

<comment><USER62> I suppose I'm giving the person in question credit for being able to see the blindingly obvious.</comment></comment></comment></comment>


Show HN: Lumix Engine – Open source C++ 3D game engine - <AUTHOR> https://github.com/nem0/lumixengine <comment><USER3> Maybe I'm just now getting aware of it but I just recently learned of Godot[1] and now this as well. Seems like we have the chance to replace Unity as the "Community" game engine with a free open source version.

[1] [https://godotengine.org/](https://godotengine.org/)

<comment><USER4> A lot of work still needs to go into those engines to achieve feature parity, specially in regards to GUI tooling.

Just check Unreal's 2017 roadmap.

<comment><USER5> Whoever thinks that all these random 3D engines are worth anything to anyone but hobbyist programmers is delusional to say the least. "gui tooling" (or as I like to call it: "the engine") represents maybe 80 to 90% of value of an engine like Unity or Unreal. I'm sure that anyone who's actually done some project in one of these can confirm what I'm saying. The amount of money and time spent on the tools of UE is simply insane.

<comment><USER6> Unity had no GUI tooling until very recently, yet was already very popular.

<comment><USER7> I think they're referring to the GUI of the actual tools you're using within the engine, not the ability to create a GUI for your project in the engine.

I mean... just look at the screenshot of the OP. That needs some pretty serious work. :-/

</comment></comment></comment></comment></comment><comment><USER8> So... Unreal Engine is already open source and cross platform. I understand they want royalty for revenues > $3000 but that seems ultra reasonable to me considering you get reliable large team to put out release after release.

Does any of the engines mentioned in this thread come close to feature set in Unreal? What's the value here other than creating basic 2D/3D games?

<comment><AUTHOR> You are right that UE/Unity/Lumberyard? is the way to go for 95% of users, but:

* UE may have it's source code publicly available, but it's definitely not free (as freedom to do anything with it)

* UE is very _resource heavy_

* You can not use it to create gambling or military stuff

* 2D possible, but not very nice

* Complicated code

None of the open source engines come very close to UE, but you pay the price for it. Even Unity (without any 3rd party plugins) is not close, but I argue Unity is still very useful.

<comment><USER9> UE4 is only as resource heavy as you want it to be, just as Unity can be resource heavy if you follow certain patterns. Unity and UE4 and many others can achieve quite a bit of parity in performance nowadays. There are scenarios that either one has potential to win out in.

Epic has made great strides in so many areas that if you haven't tried it in a few releases(6ish months?), there is quite a bit of stuff that is better (both in cleanliness/precision but also in performance and overhead)

<comment><AUTHOR> I just tried to open UE 4.14.2 (no scene) - it took 140seconds, how is that not resource heavy? Opening some basic project - 15 seconds. Making a small change in my plugin and compiling it ~2minutes. Intellisense are quite slow, VisualAssist even slower. Donwloading UE - several GBs.

I can recompile my whole engine in 20 seconds (and I am not using precompiled headers), editor launches in ~1 second, basic scene load ~1 second. Small change in source code - compile & link in 3 seconds. Intellisense brokes from time to time when using FBX SDK, but it's fine otherwise and without FBX SDK there is no problem. Engine size with some necessary data - a few tens of MB.

When I am doing something in my game and I have to change something, turn off engine + make change + compile it + start engine + load the scene = few seconds. It's so fast I did not even bother to implement hotreload for native code.

When I'm working on my UE plugin - the same cycle is several minutes.

<comment><USER9> the 140 seconds seems like something is wrong or some weird bottleneck is happening especially for a empty scene. I feel like opening a scene for me is about 10 seconds at most.

I thought you were talking about it being resource heavy relative to unity, not to your own home grown one, which is likely (and obviously) much lighter. I have no doubt in that comparison all the larger game engines have quite resource heavy editor. But the tradeoff is supposed to be that few extra seconds loading it up and few GB downloading it, saves you potentially months of work you don't need to do yourself in a more custom engine.

the 2 minute plugin change also seems really off. I can tweak my plugins and recompile in a few seconds.

Intellisense is much better than it used to, at least on my own machine, but I will admit it was finicky for me for a while. Its usually instant for me nowadays, though I will admit at one point it was ridiculously slow(5-10 seconds). I have no idea what I've changed and I am on the same computer.

no getting around UE4 being quite big, several GB, to download.

</comment></comment></comment></comment></comment><comment><USER10> What the big difference between the OpenGL and D3D renderer? Are they performing and look the same across Windows and Linux? What about MacOS? I guess if it run on Linux I can fiddle with the source and make it run on it?

Look awesome! Always good to have diversity in 3D engine.

<comment><USER11> Generally Direct3D is for Windows and OpenGL is for all other platforms. Sometimes the OpenGL renderer works on Windows as well for easier testing and development.

<comment><USER4> When "all other platforms" does not include gaming consoles, as there is this urban myth of OpenGL support on them.

Only partially true on the PS3 (GL ES 1.0 with Cg), which was largely ignored.

<comment><USER11> Yes, consoles have their own rendering libraries, well for the XBoxOne it's DirectX, but for the PS4 it's a proprietary library (Source: currently working at a AAA studio).

But that is a technicality, because most developers that are making an engine (especially indie or hobby developers) don't have access to development hardware and SDK's.

So it's valid to say that OpenGL runs on all other platforms (that the developer is targeting).

</comment></comment></comment></comment><comment><USER12> Came across this one too the other day - <USER2> [http://xenko.com/](http://xenko.com/)

<comment><AUTHOR> Xenko looks very professional, pity it's GPL.

<comment><USER13> Apparently they allow you to keep your game's source closed. From their FAQ:

> Yes, you can sell your game without any constraints as long as you use the > binary version distributed on the website. If you modify and recompile the > engine yourself, you can still sell your product, but you have to open the > source code of your game and the modifications made to Xenko as stated in > the GPL license.

<comment><AUTHOR> Yes, binary release is not GPL. However the main advantage of open source engines for game developer is their source being opened so you can change it. I wished so many times I have access to Unity source in my dayjob. But in case of Xenko, if you modify it you have to publish the source code of your game - which is a no-go for most game companies.

<comment><USER14> > I wished so many times I have access to Unity source in my dayjob.

Wouldn't that be trap though in practice? Everytime a new Unity version comes around you would have to patch/test your changes back in.

<comment><AUTHOR> Yes, it might be a trap. However you could at least:

* submit a patch

* find out that it's specific behaviour causing the bug and avoid it

* now how to optimized your code

* know how things work - Unity's docs misses a lot of details.

</comment></comment></comment></comment></comment></comment><comment><USER15> Hey, this looks really good. Thanks for the good work.

A question. Please don't get it wrong.

How you going to maintain that? Recently more than anything I'm more afraid of investing my time in some tool will go away in the near future. While UE and Unity 3D are not perfect, neither free (or even cheap). But those teams have all the money to support next releases, and I have some guarantee that they will exist next year.

<comment><AUTHOR> Perfectly valid question. You are right, while there is a high chance that UE and Unity will be maintained, there is no guarantee. In any single person project, especially noncommercial, the chance it will not be maintained is much higher and Lumix Engine is not an exception. The only "proof" that it will be maintained is that I already worked on this 4+ years (as you can see on github) and that I'm making a commercial game with the engine.

If someone is not prevented by something (personal preference, license issue, features) there is no reason to use Lumix Engine rather than UE/Unity. UE/Unity should be the right option for 95% of developers.

<comment><USER16> I do not wish to appear overly cynical (and apologies if this does), but what is stopping me investing all my time/effort into this, and finding out the licensing terms have changed such that it becomes more on par with commercial offerings?

Seems like the only benefit would be short lived, whilst I would take on a lot of the disadvantages of not going to a more commercial offering immediately. I'm genuinely interested. I can imagine this putting a lot of people off "committing".

<comment><AUTHOR> You/I/Anybody can not change the license retrocatively. It means that current source code is MIT and it always will be. I/Anybody might fork the engine under different license or even close source it, but that won't change the fact that current source code is still MIT licensed. Me close-sourcing it is the same as me abandoning it - you can still continue using/developing it under any license you wish.

<comment><USER16> Appreciate the answer, thanks. I guess that would rule out the people who definitely do not even want the possibility of the prospect of maintaining their own engine. I would be curious to know what kind of percentage we would be talking about...

<comment><AUTHOR> No engine is guaranteed to be maintained, not even the UE/Unity. Look at CryEngine, once a big, popular engine. Now with all the CryTek's problems, I would not bet it won't get dumped.

</comment></comment></comment></comment></comment></comment><comment><USER17> What became of ogre3d? [http://www.ogre3d.org/](http://www.ogre3d.org/)

<comment><USER18> There's Urho3D ([https://urho3d.github.io](https://urho3d.github.io)) which seems to be mostly a clone of OGRE. I don't know if any of the same people work on it.

Xamarin have a C# wrapper for it, UrhoSharp ([https://developer.xamarin.com/guides/cross- platform/urho/](https://developer.xamarin.com/guides/cross-platform/urho/)). I'm working with it now and I like it a lot better than Unity so far; but I strongly prefer to work in code rather than doing everything in Unity's IDE.

<comment><AUTHOR> Urho is so much much more than Ogre3D and it's definitely not a clone of Ogre3D. It's full game engine with a lot of features - audio, physics, navigation, ... while Ogre is basically just rendering library. And Urho's rendering is not closer to Ogre than any other rendering solution out there. The only thing they seem to have in common is that people are slowly losing interest in them, which is unfortunate.

</comment></comment></comment><comment><USER19> Very cool technology, both the engine and studio/editor. Impressive, judging from commit stats it seems to be one person project. Kudos!

What happened to FieldsGame? Sounds interesting, but the link on readme page doesn't have any information except it's a physics sandbox of some sort.

<comment><AUTHOR> Thanks. It's mostly one person project. There once was another programmer working on it but now it's just me and from time to time some people I do not know.

FieldsGame is made by 3rd party developer, so I do not know in what stage it is. It is possible it's abandoned.

</comment></comment><comment><USER20> Can it support a large, procedural open world? I tend to think such feature would require its own engine...

<comment><AUTHOR> I do not think there is an engine which can be used to create game such as No Man's Sky without a lot of custom code. This is open source so it can be used as a base for such game.

</comment></comment><comment><USER21> Hey guys do you know Armory3d plus Blender integration [http://armory3d.org](http://armory3d.org)?

<comment><AUTHOR> I've seen it several times, however it's quite confusing what it really is. From their webpage:

> The goal of Armory is to bring real-time rendering to artists. Everything is > fully integrated into Blender, turning it into a complete authoring tool for > real-time content.

and

> Everything is bundled to provide the ultimate game development tool.

That's two quite different goals.

</comment></comment><comment><USER22> Why is are the components limited to 64 distinct types? That sounds low to me.

<comment><AUTHOR> It's a performance optimizations at the moment, since there is much less types. However I know this is an issue for the future and I have a solution prepared.

</comment></comment><comment><USER23> Is there still room for another game engine?

<comment><AUTHOR> I started Lumix Engine years ago for fun and as a test bed for new ideas, which were later implemented in my dayjob. Now I am using it to make a game. I never intended to fill some gap on the market.

<comment><USER24> What's the game?

<comment><AUTHOR> [http://www.indiedb.com/games/on-the-hunt](http://www.indiedb.com/games/on- the-hunt)</comment></comment></comment></comment>


How Police Unions Became Such Powerful Opponents to Reform Efforts - <AUTHOR> https://www.nytimes.com/2020/06/06/us/police-unions-minneapolis-kroll.html <comment><USER2> A monopoly on violence is a hell of a bargaining chip.</comment>


Ask HN: Critique My Paranoia - <AUTHOR> Hello HN. I have a procedure I'd like to use for accessing the internet in a completely anonymous way. Please critique my method ruthlessly.
Step 1: Travel at least two hours from my address, preferably in another state, and purchase a burner laptop with cash and not in sight of any cameras. Also purchase a large capacity SSD and USB.
Step 2: Travel at least two hours in another direction and purchase Amazon, Best Buy or Starbucks gift cards from a small third party shop.
Step 3. Take the burner laptop to a remote coffee shop and use the Wi-Fi, again in sight of no cameras. Download Whonix and use the gift cards to redeem bitcoin and a VPN, making significant attempts to ensure the chosen VPN does not log.
Step 4: Wipe the burner laptop and install the copy of Whonix. Immediately set up the VPN and set internet access to immediately fail and shut off if the VPN or Tor fails. Launder the bitcoin several times.
Step 5: Put a piece of paper over the laptop with a checklist to review every time I go to open it, to remind myself of the various compartmentalization checks.
Step 6: Create many different burner email accounts. Use a new burner email address for every website. User a novel username and password for every one.
Step 7: Only access the internet using cafe Wi-Fi and other networks not tied to myself.
Is this sufficient operational security to remain totally anonymous against every reasonable threat save for an extremely motivated nation state? <comment><USER2> You might still be tracked by device fingerprinting

[https://en.wikipedia.org/wiki/Device_fingerprint](https://en.wikipedia.org/wiki/Device_fingerprint)

Also, I guess you'd have to make sure you used a VPN that didn't keep logs...

<comment><AUTHOR> Great point. I'd probably have to use ad-blocking software, Ghostery and even then do manual oversight on all outbound requests. I could use a sophisticated proxy to do this on the fly when necessary.

</comment></comment><comment><USER3> Instead of traveling hours out-of-state to buy a laptop, how about buying one used, maybe from a pawnshop or a garage sale or one of those one-man-show used computer stores? Its possible that pawnshops have cameras on all the time, but anything in it is likely to be, maybe not "hot", but disowned with extreme prejudice. You'd be putting a cut-out between yourself and the purchase of the laptop.

The Step 5 checklist should include setting a random hardware address for the wireless card or ethernet port. Maybe you can obtain MAC addresses using nmap in one coffee shop, then use them in another shop.

Step 7, get a USB wireless that lets you put a directional antenna with high gain on it, so you can actually be some distance away from the coffee shop, library, etc while you use it.

<comment><AUTHOR> Excellent points...spoofing a MAC address and using an antenna would make things safer and easier. Thanks.

</comment></comment><comment><USER4> There are a lot of things more wrong than black hatting and the drug trade, and this smells at least like something illegal given all the concern about cameras.

My critique, put your energy toward something that you are comfortable being associated with instead.

Good luck.

</comment><comment><USER5> _Step 7: Only access the internet using cafe Wi-Fi and other networks not tied to myself._

I spend a lot of time in libraries on public computers or on public Wi-Fi. Their policies often state up front that there is no expectation of privacy, that staff can check up on your activities if they have reason to do so. I have not paid much attention to policies at, say, Starbucks, but I wouldn't be surprised if they have similar policies. Furthermore, my understanding is that Wi-Fi has pretty big security holes compared to a landline. Plus, if you are a regular, people will recognize you.

I gave up driving years ago and I walk everywhere. This is bizarre and noteworthy behavior in the U.S. People stop me and talk to me and say "I see you walking All The Time..." The degree to which people in cars notice me, recognize me and feel not only free but compelled to speak to me is downright creepy.

So I will suggest that if you spend time very regularly in cafe's using their Wi-Fi, etc. people will not only recognize you, they will feel friendly and curious and like they have some goddamn right to grill you about your life and why you are there all the time and so on and so forth.

I also agree with AnimalMuppet that the lengths to which you are willing to go in order to be "completely anonymous" raise enough red flags that someone, at some point, will take an interest in tracking your ass down and that "someone" may very well be a government agency. So while I get annoyed at how humans are wired and how they conclude they have some goddamn right to grill me merely because they fucking recognize me when I have no clue whatsoever who they are, beyond being annoyed as hell at the whole thing, I don't really need to worry too much because walking everywhere isn't actually a crime, no matter how bizarre and eyebrow-raising it is. But I cannot imagine any reason to go to the lengths you want to go that don't involve serious crimes and most other people will be far more critical of your motives than I am. I am pretty live- and-let-live. On average, other people are much more judgey, butt-in-sky and controlling than I am. So you can bet dollars to donuts that most people will assume you are up to something incredibly evil and that suspicion will fuel their interest in grilling you, tracking you down, etc.

<comment><USER6> _> The degree to which people in cars notice me, recognize me and feel not only free but compelled to speak to me is downright creepy._

If you are in a public location they have every right to attempt to speak to you. Likewise, you have every right to ignore them. At that point - they're being dicks if they do this - but they have the right to continue to try and speak to you. I believe there is a legal extent where this can be deemed harassment but until that point - they have the right to speak to you. I think the legal point is you explicitly telling them to leave you alone, at which point you can contact authorities. Acting uninterested or ignoring them is an implicit message but is not enough, AFAIK. (IANAL)

 _> This is bizarre and noteworthy behavior in the U.S._

You answered why people feel compelled to speak to you and I'm sure you're aware of this. You follow a bizarre behavior. Want to be ignored? Don't stand out. People who stand out get noticed.

Also, legitimate question, how do you expect people to make new friends? Let's go under the following assumptions:

      1) Nobody has a goddamn right to speak to you.

      2) Likewise, you have no goddamn right to speak to anyone else

You can see how that would be problematic for meeting new people and doesn't really jive with most humans (and most cultures) being social?

I'm all for being anti-social and wanting people to leave you alone, especially if you're doing lawful-even-if-bizarre behavior. But to have an expectation that in a social culture in a public place that people will leave you alone seems a little out of touch.

<comment><USER5> The detail you are missing is that there is a huge element of classicism. I walk everywhere. The people stopping me to chat me up are people who drive everywhere. They assume I am poor, which happens to be accurate at the moment but it is not why I walk. They recognize me. I do not recognise them. This creates a power imbalance.

I walk everywhere because of my medical condition. Your advice to not be different if I want to not stand out is one I am, in some sense, fundamentally incapable of complying with.

I am not actually antisocial. But I do see something incredibly problematic in the assumption, that you apparently agree with, that if I do not comply with the car owning cultural standard, other people have some right to butt into my life on grounds that I deserve it for the crime of being weird. I don't agree with you. Furthermore, given the default assumption that no car = must be poor, it is an abusive exercise of power to grill me and expect me to answer your questions.

Speaking to people does not by default have to involve butting into their lives in an offensive and fundamentally disrespectful fashion. Real friends do not start the relationship by pissing all over you.

<comment><USER6> _> I walk everywhere because of my medical condition. Your advice to not be different if I want to not stand out is one I am, in some sense, fundamentally incapable of complying with._

You admitted yourself that the behavior is bizarre for the society you reside in - that's enough to stand out. I'm not saying you're in a position to change your behavior or even that you _should_. But you seem to be aware that it goes against social norms and attracts attention. Yet you turn around and act surprised and disgusted that it does. That seems disconnected to me.

 _> But I do see something incredibly problematic in the assumptioin, that you apparently agree with, that if I do not comply with the car owning cultural standard, other people have some right to butt into my life on grounds that I deserve it for the crime of being weird._

Putting words in my mouth. What I do _agree_ with is that _people have the right to speak to you_ and it is _expected_ when in a public place within a social culture. I simply disagree with your social expectations and consider it more harmful to promote an isolated society over one where an occasional intrusion is beneficial. They can be irritating to an individual at times, but promoting a standard of isolation and ignoring other's is not something I can agree with.

Yesterday, the power was out in my town for scheduled maintenance. I went over to my neighbor's house to ask if they were busy. Did I have any right to know if they were busy or not? No, how rude of me. They politely told me they had nothing to do and were waiting for the power to come back on. I did the oppressive act of asking them if they would like to play Chess until the power came on. Did I have the right to butt into his life like that? No, but he agreed to pass the time. In the end we enjoyed several hours of Chess until the power came back on.

What I should have done is sat in a pitch-black house for 3 hours because I have absolutely no right to trespass onto private property to commit the _revolting_ act of speaking to someone. I'm a terrible person for asking them to essentially be my _entertainment_ while the power was out. I'm also an asshole for discriminating by age and suggesting _Chess_ of all things, because I assumed a 70 year old man would be more familiar or willing to play Chess than Cards Against Humanity or some other time waster.

Perspective matters. Your story made it seem like you felt you were being grilled because people were even _trying_ to speak to you, you assume, because you are walking and that is bizarre. If you want to give it a narrative or illustrate how they were acting classist - _what_ they say is a lot more important than the mere _attempting to speak to you_ part.

I don't agree that we would live in a better society if we ignored one another because we don't have explicit or implicit (ie: being at a social place like a bar or club) permission to speak with someone. Sometimes both parties benefit from unsolicited discussion. Other times one party gets annoyed. It's a trade off - and one I see as worth making.

<comment><USER5> You are completely misunderstanding me and, at this point, I feel like you are intentionally twisting my words. I will try one last time to clarify and then I am done:

If you drive everywhere and you visually recognize a person simply because they walk a lot:

A) It is inaccurate to assume you are "acquaintances." The odds are very poor they recognize you. Seeing me repeatedly as you drive past absolutely does not establish a social relationship of that sort.

B) Driving up to a pedestrian, rolling down your window and bombarding them with personal questions is asshole behavior. (Asking directions is okay.)

C) Assuming that someone is poorer than you, even with reasonable cause, gives you no right to walk up or drive up to them, pepper them with personal questions and expect them to politely answer. It is classist in a really ugly manner.

Furthermore, your replies to me here are personally intrusive and it is off topic. I commented on my personal experiences to make a particular point relevant to the question that was asked. That is not an open invitation for you to question me about my personal life or judge me.

I had two main points: 1) Repeated exposure can cause people to feel an unwarranted sense of familiarity and social bonding that has no basis in reality. 2) Doing something out of the ordinary, no matter how innocent or innocuous, tends to attract interest and/or criticism. --> Those two things can combine in a way that gets really problematic if, like the OP, you do not want people butting into your life.

I am done. I would appreciate it if you would drop it.

</comment></comment></comment></comment></comment><comment><USER7> > purchase a burner laptop with cash

Easy.

> and not in sight of any cameras.

Almost impossible, unless you're buying from a fence. Laptops are high-value items; almost every place that sells them has cameras on the area where they are sold.

> Is this sufficient operational security to remain totally anonymous against > every reasonable threat save for an extremely motivated nation state?

You go to those extremes, and you're likely to motivate a nation state...

<comment><AUTHOR> Two considerations:

1\. I personally know of a large electronics chain that does not have a functional camera pointed at a register you can purchase a computer at. While the rest of the store does, I could try to think of ways around that and bring bags in my jacket that are opaque.

2\. I could try to innocently go to the store on Halloween with a reasonable pretext for keeping a mask on in the store.

<comment><USER8> Use an intermediary to buy the laptop for you in cash.

</comment></comment></comment><comment><USER9> Why not get a prepaid data card? Using tor on a public wifi is suspicious enough.

If you're okay with disabling http (and all plaintext protocols really) then you're better off just using tor instead of a VPN. Keeps the trail of ip connections more distributed. If not you want to make sure you disable them during the process of buying that trusted VPN/S.

<comment><AUTHOR> I have no experience with prepaid data cards. Can I purchase these in cash (easily)? That's a good strategy if so.

<comment><USER9> [http://prepaid-data-sim-card.wikia.com/wiki/United_States](http://prepaid- data-sim-card.wikia.com/wiki/United_States) They will all take cash at retail locations, the only thing is you want to double check that they won't ask from an ID.It's not a requirement but it seems like some will ask. I was asked for my passport when I was in states couple of years ago. Maybe somebody can chime in.

</comment></comment></comment><comment><USER10> Instead of buying burner laptops, buy commodity parts and build your own. They'll be much harder to trace.

Also, keep in mind that the police are much more likely to find you by talking to people than by tracking your online footprint. So you should also fake your own death and never talk to another human again.

</comment><comment><USER11> What about using Tails instead of Whonix? I think all this travel makes it easy to spot you on cameras (traffic, random street cams, etc) which may help to identify.

I would go of less travel (maybe buy everything in close distance but far away/unconnected to your main residence)

<comment><USER11> Also, unless you took all of those security precautions when publishing here, your cover is already blown. Have a nice day!

</comment></comment><comment><USER12> Is this something you do on a regular basis? Do you do something "high profile" when you do this? Or do you do it everytime you access internet?

<comment><AUTHOR> No, I don't do this all the time. I didn't even make this throwaway account in an obfuscated way.

This procedure will only be used for things that I really don't want traced back to me. I'm not a blackhat or drug dealer, but I'd still prefer these activities not be associated with me.

The only tedious bits are preparing the burner laptop's OS for compartmentalization and encrypted/anonymous internet connection (both, not one or the other). Once the machine is set up and I have a suitable amount of bitcoin and the VPN set up I don't need to repeat those processes for months or a year.

However, I really like the idea of getting prepaid data with cash and exclusively using a mobile hotspot. I also like the idea of simply building a computer for this purpose and buying all the components separately.

</comment></comment><comment><USER13> You should visit the Friday Squid posts on Bruce Schneier's blog. There are often good conversations about this kind of thing.

</comment><comment><USER14> All that travelling is pointless, they will simply track your car.</comment>