State Considered Harmful – A Proposal for a Stateless Laptop - <AUTHOR> http://blog.invisiblethings.org/2015/12/23/state_harmful.html <comment><USER2> A read-only memory device for firmware would be helpful. Not flash, not something erasable, but hard ROM, in a socketed device. All the firmware would go in that ROM, and you could take it out if necessary and compare it with other ROMs. There are simple hard-wired ROM comparison devices.

This is the approach mandated by Nevada Gaming Commission slot machine regulations.[1]

[1] [http://gaming.nv.gov/modules/showdocument.aspx?documentid=33...](http://gaming.nv.gov/modules/showdocument.aspx?documentid=3309)

<comment><USER3> Once upon a time, most/many PCs had physical BIOS protection in the form of a jumper on the motherboard that would allow you to put the BIOS into a read- only state. However, we have now had many years where such control cannot be manually asserted by the end user, and the flash just sits there writable (although there are chipset-level firmware write protections, various hacks, like Dark Jedi, have found ways around them). Plus, apparently even when you pull down the WO pin on some flash chips, the hardware setting can still be overridden by software commands. The paper suggests, particularly with the more recent versions of Intel ME, that the PC architecture has now evolved to expect, and perhaps require, access to a writable BIOS (in part, because the ROM stores not only firmware, but also things like configuration settings and data for the new ME-implemented TPM).

Thus, we may not be able to simply go back to a ROM with today's architectures. However, we can give today's systems something that behaves like a writeable flash chip, but is readily (and automatically) reset to a clean/factory state.

<comment><USER4> Some Chromebooks have a "flash write-protect screw"[1] that basically makes the firmware read-only. Acts as a physical protection for flashrom by software (though I am not sure if there are ways to work around it).

[1]: [https://www.chromium.org/chromium-os/developer- information-f...](https://www.chromium.org/chromium-os/developer-information- for-chrome-os-devices/acer-c720-chromebook)

</comment></comment></comment><comment><USER5> This is interesting to me, because this is the same sort of thinking that led to the current iteration of Chrome OS devices. While not stateless entirely as the paper suggests, they clearly have the same train of thought. Without the developer switch flipped, verified boot should entrust that Google (and anyone with its private key) is the sole originator of the code on the device.

I wouldn't suggest using a Chrome OS device for anything where opsec matters, but I find the similarities, at least in thought, striking. Chrome OS's early security benefits were that the device could be trusted if the dev switch wasn't flipped -- and it could be fully wiped and restored on demand if need be. The trusted stick described in the PDF would likely share similar characteristics as far as disposability goes.

<comment><USER6> The issue with ChromeOS is that you know that Google controls everything that's on it – which is not very useful for someone who wants actual security. A state actor could still force Google to install spyware on the devices.

In fact, for chromebooks for education, Google even allows the schools to MitM the traffic of the pupils with no way for the pupil to know it.

Which gets risky when the pupils take the device home, and the school still controls it.

<comment><USER7> How is that any different than an employer forcing their laptops provided to employees through a VPN (where the user is subject to inspection/blacklist/whitelist/etc) all the time? In both situations you have a machine provided to someone for a specific purpose (business/education) that likely describes extremely particular acceptable use cases. MITM traffic on whatever network seems like a valid way to enforce that. Doing your banking on your own machine.

<comment><USER8> These forced MitMs are all very similar, though I don't see how that gets us anywhere. It's not a valid way of enforcing any policy no matter how twisted, because people will root their devices and circumvent the lock-down.

The reason I call it twisted is because I'm wondering why that lock-down is necessary in the first place: you don't want them to "abuse" the device, causing potential damage? This might be a valid concern with company cars due to the life-threatening aspect of driving, but extending this concept to laptops seems abusive. Either way, the risk posed to the school or company's assets can be effectively taken into account by some kind of insurance policy, not by attempting to lock down the device.

<comment><USER9> "So how _did_ the hackers get into the corporate network?" "One of our execs downloaded a game from the Internet that contained malware and then he connected his laptop to the corporate network..."

<comment><USER6> If a single person with a laptop can damage your whole network, you have completely different issues.

</comment></comment></comment></comment></comment></comment><comment><USER10> As far as I can tell this proposal allows to ensure that it is possible to "factory reset" a system in a way that removes all malware, and also allows to "dual boot" OSes without any of the OSes being able to compromise the other (by flashing malicious firmware which then exploits the other OS on a subsequent boot).

It also prevents some more exotic attacks like replacing the BIOS (but not any hardware) with a malicious one as the laptop is delivered, used without network access (but not with network access) and then stealing the laptop and trying to read unencrypted user data leaked by the malicious BIOS.

It is not effective against undetected arbitrary physical attacks (insert a keylogger between keyboard and motherboard) or against persistent software attacks against a single vulnerable OS (persist via the OS autostart mechanism and exploit the OS on each boot).

Having an external stick also mitigates detectable physical attacks (e.g. theft of laptop, or manipulation detected by a broken tamper-proof seal) where the attacker has already stolen the encryption password, since they still won't get the stick and thus won't be able to get the data anyway.

The stick being external doesn't seem to provide much advantage otherwise, since if the laptop hardware is malicious it doesn't help, and if it is not malicious then an internal trusted stick equivalent works just as well.

<comment><USER11> > The stick being external doesn't seem to provide much advantage otherwise, > since if the laptop hardware is malicious it doesn't help, and if it is not > malicious then an internal trusted stick equivalent works just as well.

You can take out the external USB and keep it in your pocket when you go someplace you wouldn't want to carry a laptop (eg. public bathroom). Whether this is necessary depends on how paranoid you want to be.

</comment></comment><comment><USER12> It's an interesting idea, but probably hopeless: all this does is move the goalposts. You're still fundamentally trusting the hardware to read and obey the instructions on your weird "external SPI" device, and if you're willing to trust the hardware then you might as well just put it on the motherboard like today's devices do. If you don't trust your hardware (and I challenge anybody to prove that a motherboard full of chips hasn't had a chunk of flash memory added to it without your knowledge) then you have no reason to think that the code it loaded was the code stored on your "trusted stick".

I don't believe that it is possible to build a secure system which isn't based on trusting the device that you hold in your hands. At some level, you need to have a device which is capable of both UI and computation functions to a sufficient extent to validate whatever transaction you are attempting to sign. You could push that onto a smaller device than your laptop (we already know that phone-sized devices are viable), but you still have to end up at the thing you interact with for signing purposes being a device that you trust.

<comment><USER13> Moving the bar from "This machine can be subverted by anybody who can access the SPI bus" to "This machine can be subverted by anybody who can solder on additional hardware that intercepts and modified bus activity" is still significant.

<comment><USER12> I don't think soldering's really required. If I was attacking the proposed system, I'd just jam a variation on those nifty miniature usb keyloggers into the path of the usb port itself. If somebody was building devices like this, then somebody else would quickly create such a device that could be quickly slipped into place (possibly even one you could just jam into the usb port without opening the case, like the fake slots that ATM hacks use).

(Or rootkit the "trusted" usb stick)

<comment><USER13> What would that get you? The input devices are almost certainly going to be internal and PS/2 or I2C, and if you're the sort of person doing this you'd be using encrypted storage.

<comment><USER12> I mean tapping the external SPI connection at that point. You can't encrypt the first stage that gets loaded into the CPU, so you would simply replace that with your rootkit and then continue as normal until the user types the decryption key into the now-compromised device.

<comment><USER13> The CPU measures the first block of the firmware into the TPM. This is already a solved problem.

<comment><USER12> I'm fascinated by the idea of having a TPM without any on-board storage for it to use. How do you propose that would work?

If you're willing to accept stateful storage for the TPM then I agree this is straightforward, but then I don't think the "stateless device" has been achieved. If you're willing to trust the TPM's storage then you could have just used that to establish trust for everything (which is the status quo on chromebooks).

<comment><USER13> As described in the article, PTT includes a TPM running on the ME. The CPU loads the ME firmware (which is validated against a key on the ME), then starts executing the rest of the firmware (including copying measurements to the TPM).

<comment><USER12> So it just boils down to TPM-protected encrypted storage? That obviously works (because it's how a bunch of devices work today), but it's a lot less exciting... if you can set up a full TPM stack for sealed storage (which we don't have on consumer linux today :( ) then I don't see what attacks this "stateless laptop" defends you against that the TPM doesn't already handle.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER14> I really wish there was a good laptop that runs Qubes OS and has no BLOBS/closed code/firmware at all. I do some research on this every now and then and always come up short. The Librem 15 was the last one I looked into but it had closed components. Something like the Novena can't run Qubes but would otherwise be ideal (I'd gladly give up some battery life, looks and whatnot for pure freedom)

If anyone happens to know a suitable candidate let me know. Wouldn't mind a bit of hardware replacing etc. if the lone closed component could be swapped out etc.

<comment><USER3> I do not believe there is are any Intel chips that give you VT-d (IOMMU) and do not require a firmware blob. Blame Intel for that situation.

I think AMD has open sourced most of their BIOS, and a lot more of their hardware supports IOMMU anyway. Maybe that is a more fruitful direction to consider.

<comment><USER14> You are absolutely right. In fact I didn't know that Intel ME is that evil. Interesting (a bit hard to follow imo) talk from the recent CCC: [https://media.ccc.de/v/32c3-7352-towards_reasonably_trustwor...](https://media.ccc.de/v/32c3-7352-towards_reasonably_trustworthy_x86_laptops#video)

</comment></comment></comment><comment><USER15> I'm probably missing something, but I don't see how this is feasible. Moving all firmware to a device that lives on an external bus means that you must either create a 'trustworthy' distribution channel for all supported firmware (including all system components and peripheral devices), or support only a select few devices and forbid adding any new peripherals. It also means peripheral devices must either be capable of bootstrapping themselves from this chip, or the system must provide a mechanism that 'sends' this firmware to all peripherals before booting the system. I'm inclined to think that the complex interdependency of this boot process seems impractical.

Also, I have to disagree that FPGAs are ideal for the architecture proposed by this paper. Performance and state issues of an FPGA aside, they're field programmable, which seems more vulnerable than 'microcode updates'. Of course, you could just disable field programming, but why even use an FPGA in the first place?

Disclaimer: I believe Joanna is much smarter than I am, so I wouldn't be surprised if my comments are based on a fundamental misunderstanding.

<comment><USER16> Computers already have the type of bus she's talking about. A lot of the low- level/boot level components use SPI/I2C/LPC/etc busses because they're so dead simple to implement. For peripheral cards, PCI-e already supports I2C (well, SMBus technically, but they're close enough), so extension there would also not be terribly difficult.

Finally, regarding FPGAs, they're as programmable as you want them to be. There are a number of applications where they do indeed become write-once chips that just handle what needs handling. Additionally, depending on what you're doing, FPGAs can be more than fast enough -- there are a number of them that support more interesting busses and interconnects, like built in 10-gigabit ethernet. So basically, you end up using the FPGA as a chip fine- tuned and protected based on your needs, not generic needs.

<comment><USER15> My point is not that it's difficult to design a peripheral device that loads its firmware over SPI (or whatever bus). My point is that firmware for all supported devices must be maintained on the proposed external device.

What happens when you add a new peripheral device to your laptop that didn't exist when your read-only SPI-connected firmware repository was created? How do you solve this with less risk than what we have now? Eliminate hardware upgrades and peripheral devices in favor of disposable computers and e-waste?

FPGA: I'm afraid the FPGA argument still doesn't make sense. Sure, the community could create a "trusted" processor or SoC, but why use an FPGA over a custom designed processor?

If the FPGA is reprogrammed at every reboot, we now have to ensure this process can't be exploited. If it's never reprogrammed, why use an FPGA in place of a CPU in the first place?

I appreciate the input and perspectives, but I still don't see how the "laptop" described in the paper is advantageous. There are many promising paths that move us much closer to secure computing, but simply moving firmware around doesn't seem to move us forward.

</comment></comment></comment><comment><USER17> Interesting (and surprising to me) that the word 'laptop' is used instead of 'computer'.

<comment><USER18> It is the formfactor of a portable computer that you could own completely, unlike anything more mobile where at least some of the chips are configured and run by someone else.

Also you would take your I/O with you because it makes no sense if you hardly trust a device you don't let leave your side to interface with some static hardware that any third party could have modified. Current monitors have more processing power than early mainframe computers and more than enough room to hide rf equipment for remote snooping.

</comment></comment><comment><USER19> One random detail that came to mind: I guess the clock would now need to live on the trusted stick?

</comment><comment><USER19> One thing I would love to have that could be a solution to the worst case early boot encryption issues is an external crypto processor that generates and stores long term keys, implements authentication, and only releases temporary keys to the main system (to which it only connects via a simple serial connection). It has enough of a screen and input to receive a password and query the user before performing various actions. That is, something like Bitcoin Trezor but maybe a little more complex input and for more general crypto use. Ideally, such a device could even physically store the trusted stick (or several), although that trusted stick shouldn't interact with the rest of the system differently than any other device for maximum reliability. This way the most sensitive crypto is not performed on a general purpose system and the user could authenticate to the device once and then the device can authenticate the user and provide keys to multiple independent systems without hastle. It is an additional expense so hopefully wouldn't be necessary, but would be one way to solve the early boot encryption problem (if needed and less expensive solutions do not work) in a not completely special purpose way.

</comment><comment><USER20> SD cards have SPI pins, so I recommend SD for Trusted Stick rather than repurposed USB.

I've bought a 512GB SDXC card for the purpose of backing up my laptop, and often wonder whether to use it as a boot device. It's much less vulnerable to theft when it's safe in my pockets, compared to in a bag.

I'd make one small change. Rather than aim for a laptop first, mod a WiFi SD card or other pocket-sized device. The KeyAsic platform (PQI Air Card/Transcend) has been extensively hacked, and Ubuntu can run on it. Client devices (laptop, phone, etc) could connect over WiFi and run VNC through a web browser. It's still vulnerable to keystroke logging on the client, but it would be possible to switch clients halfway through typing important messages. In my opinion the most secure client device would be an iPod running Rockbox, and connecting to the PQI Air Card over serial. My "WiPod" seems like the closest thing we have to a practical pocket-sized open source device, and it lets me share photos from an SD card to my phone :).

</comment><comment><USER21> Non PDF version rendered by github: [https://github.com/rootkovska/state_harmful/blob/master/stat...](https://github.com/rootkovska/state_harmful/blob/master/state_harmful.md)

</comment><comment><USER22> C3TV – Towards (reasonably) trustworthy x86 laptops: [https://news.ycombinator.com/item?id=10833637](https://news.ycombinator.com/item?id=10833637)

</comment><comment><USER23> My OS [https://webconverger.com/](https://webconverger.com/) gets you mostly there. The slate is wiped clean between sessions like [https://en.wikipedia.org/wiki/Privacy_mode](https://en.wikipedia.org/wiki/Privacy_mode) of course.

<comment><USER24> While an OS that doesn't preserve state is an important component of Rutkowska's proposal, and your OS might be one basis for that component, I don't think this is "mostly there" in terms of everything that the paper discusses. Much of what's new in the paper is about _hardware_ issues, especially because it's concerned with firmware attacks that are already being used by attackers like NSA, and that other people clearly understand how to develop in principle.

With these firmware attacks, compromising a device at one point in time may allow the compromise to persist even if the user reinstalls the OS or replaces it with a different one.

One way to see this paper is as a response to

[http://www.slideshare.net/hashdays/why-johnny-cant-tell- if-h...](http://www.slideshare.net/hashdays/why-johnny-cant-tell-if-he-is- compromised)

proposing more details of a safer future platform.

Right now, someone who can briefly get kernel-level control on a machine intended to run your OS might be able to reprogram the hard drive firmware. At that point you have a serious authenticity challenge when booting your OS, because the hard drive can alter the contents of particular binaries at the moment they're read from disk. There are some powerful software-only defenses against this, but if an attacker knows which ones you use, they can probably design an attack that evades those.

</comment></comment><comment><USER25> Done:

[http://cdn.rsvlts.com/wp- content/uploads/2013/02/1671718-sli...](http://cdn.rsvlts.com/wp- content/uploads/2013/02/1671718-slide-p139-4jpg1.jpeg)

+

[https://www.youtube.com/watch?v=m98agJUoCck](https://www.youtube.com/watch?v=m98agJUoCck)

</comment><comment><USER26> I've been saying this about password managers for years: [https://news.ycombinator.com/item?id=9731361](https://news.ycombinator.com/item?id=9731361)

 __ _" The fundamental design flaw of all of these compromised password managers, keychains, etc. is that they keep state in a file. That causes all sorts of problems (syncing among devices, file corruption, unauthorized access, tampering, backups, etc.)."_ __

<comment><USER27> There are a few "stateless" password managers worth considering:

1) [http://www.supergenpass.com/](http://www.supergenpass.com/)

2) [https://www.stephanboyer.com/post/101/hashpass-a-simple- stat...](https://www.stephanboyer.com/post/101/hashpass-a-simple-stateless- password-manager-for-chrome)

(disclaimer: I wrote #2)

</comment></comment><comment><USER28> A laptop without any state sponsored back doors sounds like a good idea.

I'm also for separation of church and state.

</comment><comment><USER29> [http://tinycorelinux.net](http://tinycorelinux.net)

</comment><comment><USER30> Short on time, but I'll say she's looking in the right idea in general. This idea has actually been done before. Removable firmware part used to happen in older machines, too.

 _Far as I know_ , I came up with it first with a proposal on Schneier's blog, etc to put both the CPU and trusted state on a stick or card you inserted into a machine containing only peripherals maybe with RAM. Research CPU's at the time had RAM encryption/integrity to make it untrusted. I was thinking PC Card rather than stick due to EMSEC, storage, and cost issues. I'll try to find the link later today.

It was actually inspired by foreign, airport security compromising stuff. People asked me to develop a convenient solution. So, real problem was physical access to the trusted components. That access couldn't happen but can't keep all our gear with us or away from inspection. A simple chip or PC Card they carried on would be better. The chassis, from laptop to whatever, they could acquire in country or ship separately with inspection. I further imagined a whole market popping up supplying both secure sticks/cards and the stuff you plug them into. Inspiration for that was iPod & its accessories like docks. One more part was that each user could determine how much protection, from tamper-evidence to EMSEC, to apply to their trusted device.

As it sometimes happens, another company showed up with government backing IIRC and R&D on security devices. Their proposed portfolio was very similar. They undoubtedly started patenting all of it. This created a second risk for anyone attempting what I or now Joanna is attempting: a greedy, defence- connected, third party legally controlling pieces of your core business. They usually just rob people but I predicted on Schneier's blog & later here in a heated debate that they could attempt to change or get rid of the product using their patents. Especially true if a proxy for an intelligence agency. We might have just seen that happen with Apple over iMessage but I can't be sure. Anyway, do know there's both prior art and probably patents on these concepts in defense industry.

So, it was a cool concept. It was one of those I was proudest of given it collapsed problems with all kinds of devices to design and protection of one component. That's basic Orange Book-era thinking I try to remember. Unfortunately, after much debate with marketing types, we determined there was a chicken and the egg problem with these [at the time]. The NRE cost would be high to the point you'd want to be sure there was a demand for thousands of them plus people willing to pay high unit prices. Custom laptops were often closer to $10,000 than $3,000 if low volume. My greater market idea was chicken-and-the-egg times a million. That plus risk of 3rd party patents made me back off the idea as nice but not practical.

Since then, what's changed is dramatically lower cost for homebrew hardware or industrial prototyping. Projects like Novena show it can probably be done for lower NRE than before. However, this is security-critical design that needs strong expertise in both hardware (esp analog/RF) and Intel x86. That will up the NRE and odds of them screwing up. ARM or MIPS ("cheaper ARM") might be easier to do but still need HW expert and significant NRE.

So, there's my take. It's a good idea that two of us in security industry already fleshed-out with removable firmware being proven in ancient mainframes. Serious marketing obstacles to getting this done and done securely. A high-level design for the technology, as I did, is pretty straight-forward and will teach one many lessons. It was a good learning experience if nothing else.

<comment><USER13> Having the CPU on an external card makes things significantly more difficult - your connector now needs to break out the entire bus and also be capable of delivering ~100W, and you need a cooling solution that can handle that without the benefit of the greater surface area of the laptop chassis. Joanna's approach is much more attractive in terms of being something that involves very little modification of existing platforms.

<comment><USER30> You might be looking at it a bit differently than me, here. What I was looking at is the central components for CPU & storage are on the card. Power, memory, and peripherals are in the chassis. The card's connectors plug directly into that. So, if anything, I'm re-creating the old situation in towers of pluggable CPU's except it's externally pluggable and the tower is now a laptop with integrated electronics.

Regardless, it was very important to move the CPU out given it's a high chance of targeting or subversion. It is literally the root of trust for computation. I protect it because I assume attackers will be smarter than me and use it against me somehow. Far as cooling, I admit I didn't think much of it for the high-end: just decided on efficient CPU's where that wasn't so much a problem. Think along the lines of the card computers that need no cooling but have good performance.

"Joanna's approach is much more attractive in terms of being something that involves very little modification of existing platforms."

Convenience vs security. Always a tradeoff. I promise you that in physical you'll find the more convenient versions will usually get you screwed. Especially if EMSEC or subversion matters to you. I'm holding off reviewing specifics of her work until she finishes it. No promises that I will but I'd rather wait for finished thing given nature of this topic. I'm writing on the general concept which predates it on paper and partly in real products.

<comment><USER13> > The card's connectors plug directly into that.

In the past a CPU was attached to a relatively low speed bus, and the peripheral interconnects all came off some external chip. These days you've got PCIe coming off the CPU package and memory clocks in the GHz range, so the mechanical aspects of this become massively more inconvenient. Even ignoring that, once you've got storage and CPU on the card, you've basically got a card that's a significant proportion of the size and weight of a laptop. At which point you could just carry the laptop instead.

> Think along the lines of the card computers that need no cooling but have > good performance.

The attempts on that side (such as the Motorola phones that had laptop-style docks available) have been complete failures.

> I promise you that in physical you'll find the more convenient versions will > usually get you screwed

And a solution that's excessively inconvenient will just be ignored.

<comment><USER30> So, to be clear, you're saying modern processors can no longer be physically plugged into a motherboard? That the processor and BIOS chip are physically too big to be isolated into a card-sized container that plugs into such a slot on a laptop? Everything else, including cooling, could be built into laptop part. But this critical part is impossible with today's technology and they all have to be hardwired at manufacturing?

It's strange because my friend's desktop CPU fit into my hand and plugged into place. That was a year or two ago. If that's no longer possible, though, then the CPU can't be extracted into its own device and my scheme can't apply.

<comment><USER13> > you're saying modern processors can no longer be physically plugged into a > motherboard

In a literal sense, yes - laptop parts are designed for SMT only.

> Everything else, including cooling, could be built into laptop part

The point of this design is to allow users to take their state with them when they leave a hotel room without having to worry about the rest of the system being tampered with. You need the removable device to be packaged such that it's trivially removable, fits in a pocket, and is sufficiently hard-wearing that it won't be damaged. Your approach would require it to have a several hundred-pin connector and some means to bind into the cooling design, and that's an incredibly non-trivial engineering problem.

<comment><USER30> Appreciate your elaboration. Seems mine is a no go for mobile, then, if it's Intel chips and such. Embedded-style computers for trusted part are still a possibility. I've seen a card computer put into a laptop for a coprocessor. Hardwire in a KVM-style switch so the coprocessor can be the main processor when necessary. It's naturally removable. This lets key stuff be done on trusted component, safe storage, and even checking of untrusted stuff with what techniques are available.

Just gotta have something that does computation & storage that will not lie to its user.

</comment></comment></comment></comment></comment></comment></comment><comment><USER31> Can this be marked as a PDF?

</comment><comment><USER11> I had a system nearly like what the author describes after the internal SSD drive on my first generation Acer Aspire One died. There didn't seem to be an easy way to replace the drive with a generic SATA SSD (at least none that I was aware of), and a replacement drive was like $80. This is for a $200 netbook, and it wasn't a very good SSD to begin with. So I put Knoppix on a USB stick and basically used that. Since all my stuff is either on my personal server or in the cloud anyway, it was a workable solution. In my case it worked reasonably well, and if I went though the trouble of identifying or putting together a bootable Linux distro with a desktop I really liked I could probably live with something like that as a permanent solution.

I'm not nearly as privacy conscious or paranoid as the author, so I'm satisfied with the convenience of a stateful laptop. I don't even have a screen lock when it wakes from sleep. If you want to use a stateless machine like the author describes, you're going to need a personal server or a cloud provider you really trust to keep your stuff.

Edit: ge0rg had already posted link to non-PDF version.

<comment><USER32> I'm not sure you read the paper? The concern centered on all the flash/firmware compnents such as SPI flash and wifi/BT firmware, not just the OS or hard drive. The idea proposed was to have SPI and all device firmware on a secure USB stick that you always keep with you..

<comment><USER11> You're right. I read the blog post, then skimmed the actual paper. You can already do what she proposes at the OS level, but her concerns go much deeper.

<comment><USER3> It seems like a pretty valid concern. Part of the next generation of rootkits seems to be targeted at SMM-level rootkits (termed "ring -1" by some) that are installed in the BIOS. They are practically undetectable once installed, and can punch through hypervisor protections too.

I think that is also part of the author's concern with Intel ME being present on all systems. It is a separate microcontroller in the chipset that has power on the level of "ring -3" (I believe it is used to implement much of the new SGE instruction set, for example).</comment></comment></comment></comment>


Data Management Tools survey - <AUTHOR> https://docs.google.com/forms/d/1J8F68L3zmNrCn7XMfRNWjeV0rRaUIE2aBbcV7KORv8A/viewform

<comment><AUTHOR> I created a similar survey for a Meetup group I attend, and thought it might be interesting to see what a wider audience says.

I am not a vendor, marketer, or researcher - this is out of personal interest. The results are visible at the end of the survey, and if you optionally send me email address to "survey at hybriddba dot com", I will forward the final results.

 _Edit: removed email address field from survey to avoid publishing addresses for all to see_</comment>


Amazon is trying to start a Holiday on July 15th: Prime Day - <AUTHOR> http://www.amazon.com/b?ie=UTF8&node=11448061011

<comment><AUTHOR> Running more deals than Black Friday is a huge feat for a major retailer. Amazon is clearly trying to make another big Shopping Day (yes, in the midst of summer when it could use it). I bet we'll see this each year if this goes well- and I wouldn't be surprised if other retailers counter with their own events next year - giving this "sale" Holiday credibility.

</comment><comment><USER2> I believe you are confusing _holiday_ with _sale_. Or perhaps you are trying to create fake controversy in order to promote said sale. Well done!

</comment><comment><USER3> CONSUUUUUUUUUME</comment>


Bay Area sentiments - <AUTHOR> https://philip.greenspun.com/blog/2019/02/06/bay-area-sentiments/ <comment><USER2> Bentonville is an odd choice to champion environmentalism, in particular this brand which cares mostly about open space. I'm certain that the amount of developed land, per person, is much much higher there than in the Bay Area. The result is that Bay Area residents conserve more of the nation's land.

<comment><USER3> Having lived in the "greater Bentonville area", I think what he's commenting on is that SF seems like one giant un-ending city coming from there. This is especially true if he traveled there for work and didn't really have time to explore.

</comment></comment><comment><USER4> When authors use their "friends" as rhetorical punching bags, I tend to tune out.

"Look at how enlightened I am, compared to these oafs who I supposedly like, who definitely exist and who also appreciate my bad-mouthing them!"

I'm no fan of SF or the Bay Area but this article is vacuous.

</comment><comment><USER5> Wherein a rant is writ from anecdote upon a brief visit to San Francisco by a wealth conservative. His totally real not fake at all friends opine briefly. At length, straw-men are vanquished courageously.

<comment><USER6> Interesting you assert they're fake.

> "the U.S. (is) the world's most evil nation currently, committing acts > comparable to what the Germans and Japanese did during the very darkest > parts of World War II. "

> "Children don't need two parents"

> "The future is female"

They sure sound like bay area people to me.

<comment><USER3> I live in a city of ~10k people about two hours east of Bentonville, AR. Interestingly, to my ear, they sound like something someone from Bentonville would say :)

I've lived in other, larger places, and it's interesting to me how much the differences between "us" and "them" are magnified to fit the scale of the discussion.

<comment><USER5> This is the critical problem. Hyperbole mixed with invented anecdotes whose falsehood is excused because they "illustrate an deeper truth"

But unfortunately facts presented in an objective way don't get clicks or compulsive repeat visits from the morally outraged.

<comment><USER6> I still haven't seen any evidence these are faked, or anyone reasonably assert they're unusual sentiments.

</comment></comment></comment></comment></comment><comment><USER6> Lots of folks criticizing a 'visitor' for having an opinion of the bay area here: well yes, people inside the Bay Area don't tend to see it the way outsiders do. As an outsider and frequent visitor this is pretty consistent with how I (and most others I've met) see SF: dirty, and full of angry people with extremist positions who've never had to reflect on their own opinions.

West Coast VCs are happier to fly now than they were 10 years ago, there's no reason to want to live in the Bay Area anymore.</comment>


Facebook For iPhone Gets Event Check-Ins, Maps, And Unfriending - <AUTHOR> http://techcrunch.com/2011/04/04/facebook-iphone-event-check-ins/

<comment><AUTHOR> This is really Facebook's answer to Color's Implicit Social Graph. Through "Events" and the ability to "Tag Friends With you", Facebook has just taken its first step in to solving the implicit social graph problem, trying to find out who you hang out with. And they have taken a very low tech approach to doing so, instead of using 20 to 40 of your phone's built in sensors, they simply ask their users to tell FB who they are at the event with.

FB Events will be in the hands of tens of millions on day one, and thats a huge chasm to cross for Color. Though Color's way of figuring out one's implicit social graph is way cooler (and creepier) though. Any thoughts?</comment>


Trying Not to Try - <AUTHOR> http://nautil.us/issue/10/mergers--acquisitions/trying-not-to-try

<comment><USER2> I've found I usually only notice this state in retrospect. I look at the clock and notice that hours have passed. It's an exhilarating feeling, and I think if I was able to recognize it in the moment it would probably ruin it.

</comment><comment><USER3> Ah, time to pull out and dust off my Benjamin Hoff: The Tao of Pooh. I have frequently focused on a different part each time I re-listen to the audio book format...

Doesn't this restatement of wu-wei also reinforce Malcom Gladwell's suggested 10,000 hours?

</comment><comment><USER4> The effortless action described sounds like a flow state. [http://en.wikipedia.org/wiki/Flow_%28psychology%29](http://en.wikipedia.org/wiki/Flow_%28psychology%29)

</comment><comment><USER5> Sorry, but with all my fascination with eastern mysticism, I find this admiration of elegant effortless killing quite troubling. Would we see the same admiring descriptions of how a "mundane task is turned into an artistic performance" if a human was being butchered? Rather reminded me of [http://www.friesian.com/divebomb.htm](http://www.friesian.com/divebomb.htm)

<comment><USER2> I think you're missing the forest for the trees on this one. Whether the guy is an experienced murderer or an expert chef isn't the point here.

<comment><USER5> I understand the point and am no stranger to wu wei and Taoism . I also found the article quite interesting in the science behind it. However, this admittedly off-topic reaction was the first I had to reading the introduction of a philosophy that one usually looks up to. I was wondering if others had the same thoughts.</comment></comment></comment>


Forty and Still a Dev? - <AUTHOR> http://sandordargo.com/blog/2019/08/07/forty-and-still-a-dev# <comment><USER2> Perhaps I've missed something, but is there some unwritten expectation that developers will switch careers or paths by the age of 40? I'm in my mid 40s, tried the management track, and can't see myself ever leaving the world of development. It's not like we're doing something physically arduous in this field; I love to learn and this is a job where always wanting to learn is both expected and rewarded. Why would anyone look at me askance if I'm older than they thought a dev should be?

<comment><USER3> Don't worry you'll be fine. It's the 40 year olds who don't like programming/learning who struggle, and produce the stereotypes.</comment></comment>


Show HN: My startup for book collectors - <AUTHOR> http://www.shelfproud.com

<comment><USER2> This seems to be an online/social version of Delicious Library. We have just under a thousand books in our library - and the only reason they're in Delicious Library is the barcode scanning feature, which makes adding books easy.

Unless you have a similar feature (scan barcodes using the user's webcam to get ISBN, auto-fill details automatically from that), I can't see most people with large libraries taking the effort to enter their details. It's a huge amount of work for very little benefit.

You could also stand to have a walk-through or similar - unless I sign up for your service I can't see how it works, and I don't intend to sign up unless I already know I want to use it.

<comment><USER3> I'm not really the target audience (I own less than 30 paper books, e-books are way more portable, I gave away 90% of the books I had the last time I moved), but I wanted to try your service. I have the same advice: show how the application works before asking the users for the e-mail.

I thought "hey, what if I have 30 huge bookcases, does this application know how to manage the place of the book?" but I could not find the information before I hit the login wall.

<comment><AUTHOR> Hey, both of you! First, thank you for your feedback.

Yes, we definitely have to make a video tour or something like that to show how the website works.

The main difference of Delicious Library is that we are online, so your info is in the cloud, and you can check it from everywhere. Also, one of the most important features (although is not a feature because it's part of the core) it is that you have a public profile, so you can show your books to everyone, only sharing your url (shelfproud.com/username).

Of course, we are working on extra features, mainly for book collectors (the target), like showing interesting eBay and AbeBooks listing, based in the books you have.

<comment><USER2> From an end-user's perspective, the main difference with Delicious Library is that it looks like you're going to make me type in my book's details.

The ease of entering data into Delicious Library is much more valuable to me (and a lot of people, I think) than the 'always available' nature of an online service (which has its drawbacks: you go bust and my data goes with you) and the 'show off' factor from your social/sharing platform.

As well as scanning ISBN barcodes, I'd also look into importing data from existing applications (like Delicious Library) or as CSV/similar files. The early adopter of your service is probably someone who already has some form of cataloguing going on already - take advantage of that and don't ask your users to do a ton of work they've already done somewhere else.

Good luck!

<comment><AUTHOR> Yes, the feature to import from a excel file is already on the way :)

Thank you for all your feedback! :D

</comment></comment></comment></comment></comment><comment><USER4> At 24 I have 300+ books, and the fact that I feel the need to mention it indicates that I am your target audience. The graphic designer in me cringes at your layout and typography. Letters and their arrangement are important when you want to appeal to book lovers. I am not sure if "trophies" work to enthuse the elitist book reader because after all, my leather bound Bibliothèque de la Pléiade version of "À la recherche du temps perdu" makes me superior to any Dan Brown reader. I jest, a bit.

<comment><AUTHOR> You are definitely our target :) Thanks for your feedback.

Of course, book lovers really are into design, cover designs, illustration, etc. But there is not only a unique design which collectors appeal. For example, you have antique book collectors, but also horror book collectors, so we have to target to all of them. That's why we chose a neutral design :)

The trophies are an incentive to upload more books, and can say "I have more trophies than [X]", just like you can do the same in FourSquare. We are working in special trophies for people into special books, like you. For example, the "Antique Trophy" :)

<comment><USER4> I'll keep an eye on it.

</comment></comment></comment><comment><USER5> It looks kinda strange on my monitor:

<http://i.imgur.com/GKwK3.png>

1600x900 Chrome

<comment><USER6> Same problem here, I was just about to post it. Chrome too.

If I remove the 'overflow: hidden' from the #wrapper, it's centered.

<comment><AUTHOR> Thank you for your feedback. We will correct that soon :)

</comment></comment></comment><comment><USER7> At first glance I'm not sure what the difference is from Goodreads, which I actively use and update?

<comment><AUTHOR> The main difference is that ShelfProud is not for book readers, but to book owners. You upload your books, the one you own, and not the one you read. Besides, you get extra stats, you got a public profile to share.

When you collect books, the item you own is unique, although a lot of people could have the same book, the one you own is unique because has something special.

</comment></comment><comment><USER8> What is the the difference with <http://www.librarything.com/> ? Are you targeting the same target? Why would I switch (I m a lifetime member there)?

<comment><AUTHOR> As I said before, the main difference is that ShelfProud is not for book readers, but to book owners. You upload your books, the one you own, and not the one you read. Besides, you get extra stats, you got a public profile to share.

When you collect books, the item you own is unique, although a lot of people could have the same book, the one you own is unique because has something special.

<comment><USER8> I m using Librarything to manage my physical collection of book and to avoid buying 2 time the same book. I still need to add some of the book and I m not sure to include my ebooks.

So if I was both an avid reader and also book into book collecting: would I use both of this applications?

<comment><AUTHOR> You could use ShelfProud, of course, but you won't be able to make comments of the plot of the book, or add your opinion, only details of the book as an object.</comment></comment></comment></comment>


Ask HN: How do I get the most frequent templates from a project's source code? - <AUTHOR> I usually create a lot of code templates based on generic pattern(example for i:=0; i<n; i++) then create a shortcut for it in my IDE. If you used IntelliJ, it is called live template.<p>Since I want to optimize further in my work flow, is there any tool to analyze my project's source code. I want to find out the most frequent code templates from a project's source code. <comment><USER2> [https://xkcd.com/1205/](https://xkcd.com/1205/)</comment>


Git – See differences to master except merges of commits merged into master? - how-to-git https://github.com/understanding-git/merging/wiki/Merging

<comment> how-to-git In the linked wiki page, I show an emulated workflow and end with a question I'm wondering about. My reason for posting this link here is I was hoping someone could answer that question :) Also posted at [https://www.reddit.com/r/git/comments/3dhz8x/understanding_g...](https://www.reddit.com/r/git/comments/3dhz8x/understanding_git_merging_how_to_see_all/)</comment>


GB Studio – Retro adventure game creator for Game Boy - <AUTHOR> https://www.gbstudio.dev/ <comment><USER2> Oh hey, Hacker News! Just wanted to say I've been really pleased with how this project has been received, wasn't expecting it to blow up this much! My Github issues list is going crazy right now! :-D

It's been a side project of mine for quite a while as I was learning Electron and C/GBDK at the same time so don't judge me too harshly on the code hehe.

It's early days for the project so not quite decided how I want to expand it out just yet, maybe to more genres, maybe to allow more JRPG functionality. What ever I end up doing though I want to make sure it's as simple as possible for beginners to make something as easy as possible!

<comment><USER3> Question about the download - I see you have it labeled "win32-x64", does that mean it runs on both 32-bit and 64-bit Windows? Asking because my only machine with a good copy of bgb is an x86 / 1Gb RAM unit.

<comment><USER2> Hi, sorry for the delay replying, weirdly Electron names the output win32-x64 even though it's only for 64-bit, I think the naming convention is platform (win32) then architecture (x64). Either way I've just built a 32-bit version which hopefully will work for you [https://github.com/chrismaltby/gb- studio/releases/download/v...](https://github.com/chrismaltby/gb- studio/releases/download/v1.0.0/GB.Studio-win32-ia32-1.0.0.zip) though I'm not sure how it's going to manage with 1Gb RAM.

</comment></comment></comment><comment><USER4> Hmm, seems like game maker tools for retro consoles are growing in popularity a bit recently. Remember seeing something similar to this for the NES getting some traction on Kickstarter a while back.

Either way, kudos for the project. Nice to see the Game Boy get a bit more attention with stuff like this.

<comment><USER5> I have a theory, of sorts, that popular retro game consoles are the most cross-compatible and future-proof software platforms we have.

Gameboy emulators are relatively simple to make and basically always in- demand. And the Gameboy "specification" can never change—a feature you relied on will never be deprecated or removed.

<comment><USER6> You can also write your software in TeX, that will never change either.

</comment></comment></comment><comment><USER7> Reminds me of NESMaker

[http://www.thenew8bitheroes.com/](http://www.thenew8bitheroes.com/)

<comment><USER2> I was definitely inspired by NESMaker, yeah!

</comment></comment><comment><USER8> This is incredible! And just in time for Ludum Dare 44 this weekend ;)

<comment><USER9> Didn't realize that LD44 was this weekend! Thanks for the heads up, looking forward to compo!

</comment></comment><comment><USER10> The ease and functionality of the environment is fantastic. I especially like the easy visual cues used to denote what attaches to what/where! Really looking forward to this maturing! I'd love to see a more full scripting environment!

</comment><comment><USER11> I'm impressed by the presentation and the project as a whole looks very well put together. I don't even game anymore but when I was a kid this would've been very appealing to get into.

<comment><USER12> Agreed! The landing page/site you have for the project is very nicely presented and I love the little video! It looks like a very fun and easy tool to use

</comment></comment><comment><USER13> Been having a great amount of fun making a game using this, fantastic app!

</comment><comment><USER14> Fantastic! What extra steps would be required to support the GBC? Do you have to provide dual palettes? Or does the GB guess? (are there other differences aside from colour? I guess for the transparent paks)

<comment><USER2> I believe I just need to assign a colour palette to each sprite and background tile but it's something I've not played around with just yet. I kind of like the simplicity of the original greyscale myself but GBC is a highly requested feature so I'm sure I'll look into it!

</comment></comment><comment><USER15> Wow, this looks fantastic -- I've been wanting to work on a simple JRPG for a while but without having to learn RPG maker. There's an obvious caveat here being that RPG Maker, Game Maker and Unity all have a gigantic ecosystem of 3rd party packages and assets that may make them better for pure game making. Perhaps my question is whether there's value in something like this which seems far more minimal?

<comment><USER16> It targets gameboys?

<comment><USER2> It makes ROM files for the original Game Boy and they can run on an emulator or a real console [https://imgur.com/a/oR9aNdK](https://imgur.com/a/oR9aNdK) but it also outputs a web build with touch controls using a fork I made of the js emulator GameBoy Online [https://github.com/chrismaltby/GameBoy- Online](https://github.com/chrismaltby/GameBoy-Online)

</comment></comment></comment><comment><USER17> This is wonderful! Im going to look for my old GB just to spend a few weekends on this.

</comment><comment><USER18> I thought Jeff Gertsman went insane and started a game studio.

</comment><comment><USER19> Looks awesome! I noticed that it doesn't yet support in-game progress save functionality yet, but will definitely give it a try when it does!

<comment><USER2> Saving/loading is definitely on my todo list. The game I built this for originally was only small and was intended to be finished in one sitting but I'll be adding support for saving soon.

</comment></comment><comment><USER20> similar, but to tech about 10 years previous to this was Adventure Construction Set. Very cool, but I lacked the imagination/art skills to make anything compelling with it.

[https://en.wikipedia.org/wiki/Adventure_Construction_Set](https://en.wikipedia.org/wiki/Adventure_Construction_Set)

<comment><USER21> I bought this when it was in stores, but lacked a PC powerful enough to run it! All my elementary school self could do was read the instructions over and over and dream :)

</comment></comment><comment><USER22> Oh, how cool is that?! Will give it try!

</comment><comment><USER23> Wonderful, was looking for something like this to play around with making GB games!

</comment><comment><USER24> This looks awesome</comment>


Neurotheology: This Is Your Brain On Religion  - <AUTHOR> http://www.npr.org/2010/12/15/132078267/neurotheology-where-religion-and-science-collide <comment><USER2> Surely the obvious thing to do would be to compare the brains of religious meditators to atheist meditators?

Isn't the main result they appear to have acquired so far the fact that meditation increases blood flow to the part of the brain responsible for concentrating on things?

Also I'm deeply skeptical due to quotes like this:

 _"One could try to conclude one way or the other that maybe it's the biology or maybe God's really in the room, but the scan itself doesn't really show that," Newberg says. "For neurotheology to really work as a field it needs to be very respectful and open to both perspectives."_

which leads me to suspect that this is just a way of bilking religious institutions out of money to fund fMRI scans and pay researchers' salaries. Nice work if you can get it!

Meanwhile, I should head back to the lab to continue my experiments. Maybe it's gravity which causes the rocks to fall when I release them. Maybe it's the hand of God pushing down on them. We need to be open to both perspectives!

<comment><USER3> Does it not stand to reason that if your field is "neurotheology", you need to be "open to both perspectives"?

<comment><USER4> I'd tip that being open to the hand of God being involved helps to get some of the participants to engage in the project.

<comment><USER2> What legitimate scientific questions can this field answer anyway? What falsifiable hypotheses can it disprove?

Different areas of your brain light up in the FMRI when you're thinking about different things; we've known this for decades, but it hasn't proved much in particular.

</comment></comment></comment></comment><comment><USER5> Last week I just finished 'Religion Explained' by Pascal Boyer - and it is indeed quite good explanation - I recommend that book. In short the theory presented there is about religion as a kind of side effect of having brains trained for what was needed to live. It is quite convincing, maybe a bit falling into the false dichotomy path by trying to present itself as the sole and only theory - while the reasoning there does not in fact eliminate other mechanism adding to the trend of evolving a religious attitude.</comment>


Maps Hidden in Monopoly Helped World War II POWs Escape (2009) - <AUTHOR> http://abcnews.go.com/Technology/monopolys-hidden-maps-wwii-pows-escape/story?id=8605905#.UVHWojfmDK-

<comment><USER2> I'm surprised the enemy camps allowed such large objects to be brought in uninspected.

<comment><USER3> Absent of using an X-ray machine or actually tearing the boards apart, I suspect there wouldn't really be a great way to detect that sort of thing during an inspection. I'm sure they probably searched the boxes, but finding things inside of the actual board would be difficult unless you knew to expect that (then perhaps use a magnet to non-destructively locate files or whatnot cheaply? Maybe simply weighing them would suffice).

</comment></comment><comment><USER4> See also <https://news.ycombinator.com/item?id=4970707>

</comment><comment><USER5> Here's an NPR story about Fort Hunt and the care packages they made for POWs [http://www.npr.org/templates/story/story.php?storyId=9364035...](http://www.npr.org/templates/story/story.php?storyId=93640350)</comment>


Merge branch 'tcp-lockless-listener' - <AUTHOR> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=c3fc7ac9a0b978ee8538058743d21feef25f7b33 <comment><USER2> It's a bit odd how one of the kernel gatekeepers is credited at git.kernel.org as both "author" and "committer", while the actual author's name is hardly evident.

<comment><USER3> That's because this is a merge commit of his own branch that contained the applied version of the patchset (presumably to net-next).

See one of the parents: [http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.g...](http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=e994b2f0fb9229aeff5eea9541320bd7b2ca8714)

</comment></comment><comment><USER4> There was a really good HN thread about this a month ago:

[https://news.ycombinator.com/item?id=10483747](https://news.ycombinator.com/item?id=10483747)

</comment><comment><USER5> Is there any more information on this that is more accessible to someone who is just a web-dev?

<comment><USER6> Your nginx will get even faster! SYN packets (packets with the SYN flag set) are the first packet sent to initiate a TCP connection. A browser connecting to a web server is a TCP connection. So this will enable the linux kernel to handle incoming connection requests way faster. The effect is that high traffic web servers running with already-optimized applications like nginx will become faster under high load, particularly a flood of incoming new connection requests.

If you are trying to weather a TCP based DDoS attack, you have a better chance at your server staying up, provided your connection isn't saturated.

If you're running a load balancer, reverse proxy, port forwarding, or anything else that will receive significant traffic and requires a TCP connection to your server, your server will be faster.

This also affects anything that listens on a TCP socket, like postfix (email), SSH, FTP, memcached, redis, mysql etc..etc..</comment></comment>


A PostgreSQL Extension that allows you to access any other datasource - <AUTHOR> http://multicorn.org/ <comment><USER2> Multicorn developer here, I'll be glad to answer any question.

<comment><USER3> Crap, I'm working on something exactly like this :D Was about to upload it to Github just today, but still wanted to fix a few things.

Yours looks quite more finished, one thing that would be very useful is handling Python to Postgres type conversion, like PL/Python does. After that, the next step is caching I/O functions for the duration of the scan, which PL/Python also does.

Nice to see that the idea made sense for more than one person, hope Multicorn will rock!

<comment><USER2> Sorry to beat you at it ;)

We currently have some very rudimentary python to postgres type conversion, but this area still need a lot of improvements.

You should release your code, I'm sure you have a wide range of ideas worth merging into Multicorn !

<comment><USER3> Actually, the ideal thing to happen would be for PL/Python to somehow become a shared library that provides the typecasting routines, but I'm afraid that's a very big task and difficult to get accepted upstream. It's sad that PL/Python already includes all the code you need to transform Python lists to Postgres arrays, bools to PG bools and so on and every project like this one has to duplicate all that logic.

BTW: in my approach I opted for passing the entire Qual node as a string (using nodeToString) and my plan was to write a Python parser for that format that takes a string and returns a Python object. Not sure how hard that will turn out.

<comment><USER2> I totally agree with you regarding PL/Python. I may try to ask upstream what they think about it.

</comment></comment></comment></comment></comment><comment><USER4> I try to install this extension on Ubuntu 11.10. First of all error in instalation -

gcc -g -O2 -fPIC -fPIC -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wformat-security -fno-strict- aliasing -fwrapv -g -lpythonpython2: -fpic -L/usr/lib -Wl,-Bsymbolic-functions -Wl,--as-needed -Wl,--as-needed -Wl,--as-needed -lpythonpython2: -shared -o src/multicorn.so src/multicorn.o /usr/bin/ld: cannot find -lpythonpython2: /usr/bin/ld: cannot find -lpythonpython2: collect2: ld returned 1 exit status make: __* [src/multicorn.so] Error 1 rm src/multicorn.o ERROR: command returned 2

This is because Ubuntu have - python and python2.7 bin files. I fixed this by creating symlink on python.

After instalation another problem:

$ psql psql (9.1.1) Type "help" for help.

leo=# CREATE EXTENSION multicorn; ERROR: could not load library "/usr/lib/postgresql/9.1/lib/multicorn.so": /usr/lib/postgresql/9.1/lib/multicorn.so: undefined symbol: _Py_NoneStruct

All tested on specialy created for this system: $ python -V Python 2.7.2+ $ psql -V psql (PostgreSQL) 9.1.1 contains support for command-line editing

On this my tests end.

<comment><USER2> Thank you for this report. I'll try to make it work on Ubuntu, expect the documentation to be updated in the next few days.

PS: if you're the 'leopard' who requested a redmine account, it should be activated now, feel free to report it there.

<comment><USER4> Ok, I already added this to tracker. I am very interesting in this extension, its amazing.

</comment></comment></comment><comment><USER5> I have well over 10K messages in gmail.

Will this work?

I was thinking in downloading the whole thing via POP3, creating a Unix mailbox and indexing that.

<comment><USER2> It depends on what your use case is. If you have well targeted queries (eg, with filters on "From" or "Subject" headers, and only query the message headers (not the payload) you can have a pretty good performance.

What is your use case, exactly ?

<comment><USER5> Most of my filtering would be based on the contents of the subject header. I have a site that sends me copies of particular transactions, and these few thousand messages are the ones I'd like to query. Ideally I'd like to have them downloaded for statistical processing.

</comment></comment></comment><comment><USER6> I was at OSCON Data this year that held a talk about the new foreign data connectors piece, it's great that Postgres has them now. Want twitter data? Make an interface to translate twitter data to rows, it's that easy! LucidDB has had foreign data connectors for a while but you have to use Java at least for part of it. I like the generator-style of yielding list-rows shown here: [https://github.com/Kozea/Multicorn/blob/master/python/multic...](https://github.com/Kozea/Multicorn/blob/master/python/multicorn/csvfdw.py)</comment>


A swarm of submarine drones will scour the depths for MH370 - <AUTHOR> https://www.economist.com/news/science-and-technology/21733399-swarm-submarine-drones-will-scour-depths-plane-fantastical-ship <comment><USER2> The "swarm of submarine drones" being "eight autonomous submarines called HUGINs, each six metres long, weighing 1,800kg, and containing a titanium sphere to protect the sensitive electronics therein from the pressure of the ocean's depths."

<comment><USER3> _In prior cruises in the Atlantic, the firm has, according to Josh Broussard, its technical director, managed to scan 890 square kilometres a day using six autonomous submarines. With eight, Mr Broussard thinks that the new mission will be able to manage 1,200 a day—enough to have covered the original search area in just 100 days._

Seems up to the job, and while 8 may not be a swarm of bees, I think for deep sea subs it works as a term.

<comment><USER4> Surely it should be a school?

<comment><USER3> Yeah actually kidding aside, why not that? Maybe the idea is that robots are more like insects than fish, but that seems a bit exotic imo.

<comment><USER5> "Drone" comes from worker bees, so "swarm" is at least consistent. But I found drone misleading here - I thought they'd be airborne. What's the scope of "drone"?

<comment><USER6> More importantly, worker bees aren't called drones.

Drones are males - they don't actually do any work, living exclusively to have sex with a queen in spring. Once mating season is over, they are evicted from the hive to die - they are unable to fend for themselves. I believe it'd be rare to find drones as part of a swarm of bees in the normal case of a swarm.

Worker bees are females. I've never heard them called anything other than a worker bee.

</comment></comment></comment></comment></comment></comment><comment><USER7> This is what a HUGIN submarine drone looks like (from the manufacturer's site):

[https://www.km.kongsberg.com/ks/web/nokbg0240.nsf/AllWeb/B3F...](https://www.km.kongsberg.com/ks/web/nokbg0240.nsf/AllWeb/B3F87A63D8E419E5C1256A68004E946C?OpenDocument)

<comment><USER8> > When you purchase a HUGIN AUV you join the our family.

That's an interesting point of view. I guess it's common with really expensive things?

<comment><USER9> In the first world, it's also common with motorcycles, which are one of the most inexpensive forms of private powered transportation.

The shared traits among motorcycles and an AUV include a more adventurous risk profile, affinity for mechanical devices, and less ubiquitous uptake than alternative forms of transportation.

<comment><USER10> Or even just an expectation that the product is going to require some DIY maintenance, and mutual support from other users. With, for instance, 3D printers, this works in inverse proportion to the expense of the purchase... if you buy a cheap one, you are joining a community - you don't expect it to be perfect, but tinkering and optimizing is likely part of the attraction in the first place. If you buy an expensive one, it's most likely because of a straightforward business need and you're going to rely on the manufacturer for support like any other product.

</comment></comment></comment></comment><comment><USER11> It would be so cool to build one of these things with a collision detection system and a nuclear reactor so that it could crawl the entire ocean. Imagine what it might find. You could potentially become quite wealthy on treasure hunting expeditions alone I would imagine.

<comment><USER12> You wouldn't need nuclear. It could surface for solar/wave generation as needed. That wouldn't necessarily slow it down because the reduced cost would allow you to have more of them operating simultaneously.

<comment><USER13> It seems way more fun to have a nuclear submarine.

</comment></comment></comment><comment><USER14> _Either way, though, the advance of technology may mean that it is the last such mystery. As the oceans are watched with ever closer scrutiny, from space and the depths, it is increasingly difficult for anything to get lost in the first place._

Have things changed this much since 2014? What would happen if the exact same scenario happened again (airliner switches off all transponders and veers off course into an ocean) in 2018?

<comment><USER15> They haven't changed much yet, but the Iridium Next constellation has hosted ADS-B payloads (as well as AIS). Sometime in 2018 these will be available globally, and make it much harder for either ships or planes to be lost outside the range of land-based receivers.

That wouldn't help intentionally or accidentally disabled transceivers as was (probably) the case for MH370. That would require space based radar... which was once but is no longer a DoD project. It'll happen eventually.

</comment></comment><comment><USER16> "Every block of sea floor that the HUGINs map will be examined by three sets of human eyes."

They could publish the data and crowdsource the review process. Also perhaps some machine learning techniques could be applied to indentify most interesting segments to review.

<comment><USER17> Agreed. putting it on Tomnod would be nice (and they're already interested in the search for MH370)

</comment></comment><comment><USER18> Its crazy the amount of money spent on MH370... does anyone know how much it compares to the payout of the victim's families?

<comment><USER19> Also there's not really much "mystery of MH370." It was almost certainly suicide/murder by the pilot. Though I guess there's no harm confirming that.

<comment><USER20> No, it was with the same 'almost certainty' an overheating of a battery back in the load.

<comment><USER19> Ok maybe almost certain is a bit strong but the aircraft maneuvered after losing contact along a path towards the open ocean pretty similar to one the captain had practiced on his flight simulator less than a month before. Not sure how an overheating battery pack would do that. [http://nymag.com/daily/intelligencer/2016/07/mh370-pilot- fle...](http://nymag.com/daily/intelligencer/2016/07/mh370-pilot-flew-suicide- route-on-home-simulator.html)

<comment><USER21> Cabin filling with smoke, everyone panicking...

</comment></comment></comment></comment></comment><comment><USER22> I wonder what other interesting things they'll discover doing the search. Surly they've already stumbled upon lots shipwrecks in the search for MH370?

<comment><AUTHOR> Here's an article about one: [https://www.smithsonianmag.com/smart- news/malaysia-airlines-...](https://www.smithsonianmag.com/smart- news/malaysia-airlines-searchers-find-sunken-ship-instead-180955256/)

ARCGIS had a fairly detailed visualization of the search and the data. At the bottom of the page, there are some examples of other things that were found: [https://geoscience- au.maps.arcgis.com/apps/Cascade/index.htm...](https://geoscience- au.maps.arcgis.com/apps/Cascade/index.html?appid=038a72439bfa4d28b3dde81cc6ff3214)

(HN comments for the latter link: [https://news.ycombinator.com/item?id=14802105](https://news.ycombinator.com/item?id=14802105))

<comment><USER23> Maybe one day they will find captain america

</comment></comment></comment><comment> the-dude AFAIK it is MH370, not MA370.

I am Dutch, we had our MH17. I just checked, apparently MAS17/370 is correct as well.

<comment><AUTHOR> > AFAIK it is MH370, not MA370.

Good catch. I think MH370 was more commonly used, so I've edited the title.

</comment></comment><comment><USER24> It is so disingenuous to say fugro only used 1 auv in the original search so this search will be better. Most of the scanning was done with towfish rated to 6000m using sidescan from three vessels rather than auv's. The article makes it sound like the original search was done with one auv that couldn't get to 6000m.

</comment><comment><USER25> According to amateur debris-finder Blaine Alan Gibson they are looking (mostly) in the wrong area. Somewhere near Diego Garcia should also be investigated, based for example on eyewitnesses in Kuda Huvadhoo in the Maldives. I subscribe to his open mindset; go where the evidence points and don't lock on to any theories based on scant such: [http://thehuntformh370.info/content/blaines-independent- inve...](http://thehuntformh370.info/content/blaines-independent- investigation)

</comment><comment><USER26> [http://archive.is/hrHyJ](http://archive.is/hrHyJ)

</comment><comment><USER27> So much scientifically important data. I hope this is overlap with previous surveys for a comparison. Or that they drop some sensor packs or markers for historical comparison later.

</comment><comment><USER28> It would be super cool if these submarines could be made completely autonomous.

They could come up to the surface to recharge their solar batteries or maybe charge them using turbines rotated by the water movements.

They could remotely relay images and data to a publicly accessible services where they could be crowdsourced for interesting findings.

I think something like this could be useful for more than just the search for MH370 (oceanography, marine biology, etc.)

<comment><USER29> I've actually been thinking about this, and, on a small scale, it should be possibly with almost no budget (i. e. retrofitting an RC model with solar power, an arduino, GPS/WIFI/Cellular). Some thoughts:

\- Submarines aren't great because solar is your best source for power, and you don't want to divide each day to "work" and "rest" because you need to recharge on the surface.

\- Electric propulsion will also require some rather substantial solar setup.

For these reasons, I think either "passive" sensors just following the currents, or sailing vessels, are better candidates at first.

There's also the "glider" concept, where you fill/empty a ballast tank and achieve forward momentum by changing depth while having an aerodynamically- shaped hull. I'm wondering if these could also drag a turbine for electric power, but I fear that idea is coming dangerously close to breaking the laws of thermodynamics.

\- Satellite internet is, unfortunately, too expensive for anything beyond short status codes / coordinates / commands. It's something like 10 cents for every 80bytes. You'd need to navigate to coastal waters and pick up cellular/wifi signals to transmit the payloads.

<comment><USER30> My company offers this exact thing as a commercial service: (Solar-powered Sailing-aaS?) [http://www.saildrone.com](http://www.saildrone.com)

<comment><USER29> Very cool! If I were US-based, I'd apply for a job :)

</comment></comment></comment></comment><comment><USER31> "no find, no fee".

That's pretty bold.

<comment><USER32> My private conspiracy theory (which I don't believe in really): One of the USA Inteligence services knows exactly where the plan splashed down (either due underwater acoustic monitoring ala K-129 submarine, or through satellite monitoring) They can't reveal that information, because that would hint at their capabilities and methods, so they let the information 'slip' to an underwater search company. They 'boldly' go searching and in due course stumble upon the plane. They know their cost is bounded because of the tip- off, and they really do search, just maybe select a pattern which will be more 'lucky' then other possible patterns could be.

</comment></comment><comment><USER33> > Seabed Constructor is the most advanced civilian survey vessel on the planet > today. If its array of technology cannot find MH370, then it is likely that > nothing will

That sentence seemed gushing and overstated. Presumably a large metal plane isn't really going anywhere and if these 8 underwater drones don't find it then if people keep trying they will in the future.

</comment><comment><USER34> I hope they find MH370. At the very least, it will bring closure to families who lost their loved ones.

</comment><comment><USER35> How credible are the conspiracy theories on this? I recall reading about more than one instance where authorities did not cooperate or even misled or otherwise actively impeded investigations.

<comment><USER29> I wasn't following the investigation too closely, but the conspiracy theories I saw weren't too convincing:

Mostly, they highlighted instances of authorities either correcting previous mistakes, or withholding information slightly longer than necessary.

Yet it's quite obvious how these two errors are related: If you're eagerly accused of the former, you'll tend to indulge in the latter.

It's also important to consider that the Indonesian authorities were simply overburdened by such an investigation, were likely to make all sorts of mistakes under stress, and had to content with competing interests that were impossible to satisfy.

Then, this investigation happened to involve dozens of authorities and companies from different countries, making errors in communications, distrust, and cultural clashes much more likely than, say, a US-made aircraft crashing in Colorado.

The debris that was discovered in Australia and Africa seems to, roughly, validate the initial search efforts. While there may be competing theories pointing to slightly different areas of the Indian Ocean to be searched, there's nothing supporting the "real" conspiracy theories that the plane was diverted to Riad and will be used to start WW III.

<comment><USER35> Oh, right, by conspiracy theories I meant something involving foul play closer to it being ditched in the ocean, not those theories that claim it was diverted to somewhere in Asia.

By contradictions I meant the stuff like this [1]. Malaysia got slammed internationally by governments all over the world for their misleading and contradictory statements. It was hard for everyone to explain the particulars away as the being overburdened and stressed. If it had been, I imagine other governments would have been more understanding of that and not slammed them so hard.

One particular incident that left me flabbergasted -- but which perhaps could be explained (though not excused) by miscommunication -- was this one, where I seem to recall (though I can't find the exact article) that they confirmed the flaperon found belonged to the plane _before even the French -- who were the ones doing the analysis -- came to such a conclusion_ [2]:

> On Wednesday, Najib Razak, the Malaysian prime minister, said the flaperon > that washed up on the French territory in the Indian Ocean, since > transported to France for analysis, was from the doomed flight. But his > assertion has not been backed up by the other authorities involved in the > investigation, and the dissonant stances have infuriated many relatives of > those on board the plane, who have waited more than 500 days for concrete > clues into the fates of their loved ones.

> Transport minister Liow Tiong Lai says more possible MH370 debris has been > sent to France for analysis. "Why the hell do you have one confirm and one > not?" Sara Weeks, the sister of New Zealander Paul Weeks, who was on board, > told Associated Press. "Why not wait and get everybody on the same page so > the families don't need to go through this turmoil?"

> The claims and counter-claims continued on Thursday when Liow said a team on > the French territory of Réunion had collected more apparent plane debris. > The claim was contradicted by French officials, who said no new material in > the search for MH370 had been turned over to French authorities.

How in the world they could jump to conclusions before the people actually doing the investigations certainly beats me. I could see it as being a miscommunication, but this was not a one-off incident, and that got harder and harder for the international community swallow the more times it occurred. I recall getting pretty convinced that they were covering up _something_ , though I couldn't go farther and speculate on what exactly that might have been. (When I heard about the misdetected pings I thought perhaps they should have been looking into the Asia possibility too, but only because they had already poured so much effort into other dead end that this seemed worth a shot, not because it seemed otherwise particularly plausible. I dismissed that idea pretty quickly after they found parts from the plane.)

[1] [http://www.straitstimes.com/asia/se-asia/china-slams- malaysi...](http://www.straitstimes.com/asia/se-asia/china-slams-malaysia-for- contradictory-information-on-mh370)

[2] [https://www.theguardian.com/world/2015/aug/06/mh370-search-p...](https://www.theguardian.com/world/2015/aug/06/mh370-search- plane-window-and-more-debris-found-on-reunion-says-minister)

<comment><USER36> > How in the world they could jump to conclusions before the people actually > doing the investigations certainly beats me. I could see it as being a > miscommunication, but this was not a one-off incident.

It is probably just human nature. Many people will continue to think and do what they did before, even after having the error of their ways explained to them. While you might expect a person in a position of responsibility to act more responsibly, it also seems possible that such a person might consider himself to be more competent than average, which would exacerbate the Dunning- Kruger effect - in this case, thinking his hunch about the debris is as good as a conclusion by crash investigators.

A false report of more debris being found might be a misunderstanding of the original find being reported through different channels at different times. An impulsive person, eager to say something, might not consider that possibility before making an announcement.

</comment></comment></comment></comment><comment><USER37> FYI disabling javascript when loading the page disables the paywall. (It actually loads the whole page, then removes the article when it figures out you've reached your article limit)

<comment><USER38> Opening in private browsing mode also seems to work.

</comment></comment><comment><USER39> How do we get ahold of Ocean Affinity?

</comment><comment><USER40> Finally some interesting new tech, every country should have a group of similar things for sea incidents.

</comment><comment><USER39> What area? Near Borneo?

</comment><comment><USER41> i thought they were refferring to lookin for the Argentine sub that went missing more recently.

maybe that one will be next i hope.

</comment><comment><USER42> Somebody alive knows where it really is.

<comment><USER43> That is likely false. If it for any reason nose dived into the ocean at high speed then there may be no witnesses and most of the material would have either vaporized or been shredded into fine particles.

</comment></comment><comment><USER44> Makes one wonder just how much money people are willing to spend to find that plane. Not every mystery is worth unlimited funds to resolve.

<comment><USER45> Like search and rescue operations, these are essentially exercises for defence forces. You can't make opportunities like these.

</comment></comment><comment><USER39> What is search area. Near Borneo?

</comment><comment><USER46> Slightly off topic, but here's my favorite conspiracy theory about MH370: It landed Kazakhstan. Interestingly enough, this conspiracy theory was informative on how satellite pings actually work (whether or not you believe the theory is a different matter)

[http://www.dailymail.co.uk/news/article-2967157/Vladimir- Put...](http://www.dailymail.co.uk/news/article-2967157/Vladimir-Putin- ordered-Russian-special-ops-steal-MH370-plane-secretly-landed-disused-space- port-Kazahkstan-expert-claims.html)

It'll be interesting to see what what comes of this effort.

<comment><USER35> Didn't they pick up broken pieces of the plane wings and such from the Indian ocean?

</comment></comment><comment><USER47> It astounds me that they wouldn't already have used the best equipment on the market? I mean, it's the most prominent underwater search of recent decades, funded by three sovereign states.

My gut feeling is that some people in power do not want this plane found.

<comment><USER48> As a sibling comment alludes, everyone underestimates how astoundingly vast Earth's oceans are.

I did too, until I looked out the window on my first daytime flight over the sea. A seemingly endless stretch of unbroken blue. It was probably the next best thing to being in space and made me appreciate just how much _water_ there is!

And that's just the surface.

<comment><USER49> Everyone also grossly underestimates how much it costs per-day to operate a large offshore services vessel in the open ocean, in terms of salaries/benefits, fuel, insurance, cost of the ship as a whole relative to whatever sort of financing paid for it, inmarsat services, food, etc. There's a reason why submarine cable laying and repair is so expensive...

<comment><USER50> From what I can tell, prior to this latest effort, around $200m dollars have already been spent looking for this plane.</comment></comment></comment></comment>