ProjectPSX – A C# coded emulator of the original Playstation - <AUTHOR> https://github.com/BluestormDNA/ProjectPSX <comment><USER3> > ProjectPSX dosn't use any external dependency and uses rather simplistic C# > code.

Very impressive. This makes it more interesting to see how the drawing code and the controls were implemented in addition to the possibility of it being cross-platform if it using .NET Core.

<comment><USER4> It uses System.Windows.Forms and friends. It is a Windows app.

<comment><USER5> [https://www.mono-project.com/docs/gui/winforms/](https://www.mono- project.com/docs/gui/winforms/) might allow this to be ported easily though.

<comment><USER6> winforms on mono never really worked, and hasn't been updated in about 12 years [https://github.com/mono/winforms](https://github.com/mono/winforms)

</comment></comment></comment></comment><comment><USER7> As a day-to-day C# developer, I'm curious about a few things:

\- Do you ever find that the Garbage Collector causes noticeable pausing in the emulator?

\- Did you do this with 100% managed memory, or do you need to use unsafe code / real pointers?

(Apologies that I didn't read the source code.)

<comment><USER8> Hello There, I'm the owner of this repo. So nice that has ended here somehow. Gonna answer the upper quests...

\- Do you ever find that the Garbage Collector causes noticeable pausing in the emulator?

The GC dosn't cause noticeable pausing on my setup. Even tho it could allocate less.

\- Did you do this with 100% managed memory, or do you need to use unsafe code / real pointers?

In fact it was done without unsafe code or pointers till some of the latest commits. Where i switched to unsafe because performance reasons. On my setup that was arround 10 fps more.

<comment><USER9> Incredible work, congrats!

</comment></comment></comment><comment><USER10> This is great. It would be almost perfect if it didn't rely on WinForms (1) and was built on .NET core from the ground up. Anyway, great project I will keep checking out.

(1) instead it could be a runtime that just gives you an interface to the frame buffer somehow and it's up to the integrator to use WinForms/WPF/Qt etc.

<comment><USER8> That's a very nice idea. I went with WinForms as it was the "stupid easy way" infact on the first i just updated a PictureBox.

There was a time when i had a SDL multiplatform branch but it was noticeably slower so i didn't put much effort on it and it ended deleted.

I mainly focused on the core PSX things and not much on the program itself like the emulator window itself...

<comment><USER11> >> I mainly focused on the core PSX things and not much on the program itself like the emulator window itself...

As you should. Core functionality always comes first. UI can always be messed about with after, when it 'just works'.

Also, fantastic work, here! :)

</comment></comment></comment><comment><USER12> And one in Java: [https://github.com/kilograham/jpsx](https://github.com/kilograham/jpsx)

<comment><USER13> Having a problem finding the "one line" version. Mind directly linking to it?

[https://github.com/kilograham/jpsx/tree/master/src](https://github.com/kilograham/jpsx/tree/master/src)

<comment><USER14> I think you miss read his comment

</comment></comment></comment><comment><USER15> Great job! We need more such examples in managed languages.

</comment><comment><USER16> Very, very cool. Good job! I remember tackling emulating the 6502 was a crazy learning experience, but I did that in C++, but definitely there is value in emulating something this complex in C#. Did you choose emulation because it was interesting or it just seemed to provide the best learning environment for what you were looking for? And have you ever dabbled in PSP emulation? I tried my hands years ago and still have a PSP sitting around with some homebrew I made.

</comment><comment><USER17> This is great! I've often wondered how emulators are actually written - <USER2> understand the theory but it was great to see this, and the linked other emulators this person has also worked on (chip8, gameboy etc).

Super nice and simple to read - e.g. cpu.cs has the thing that does the actual "MUL" and "MOV" etc op codes. Nice :-)

Thanks :-)

</comment><comment><USER18> I'd avoid a name like ProjectPSX to be more distinct from the 100 other PSX emu projects.

PSX.NET or PSX# would be good.

<comment><USER8> Ey, those are real cool names. The name itself was chosen because i already had a ProjectDMG (Gameboy emulator)... I may go with PSX.net when net core 3.0 goes out...

<comment><USER18> They may sound cool but they aren't original lol. Open source C# projects like Paint.NET often take on that convention.

This might sound weird, but make sure .NET is capitalized otherwise people might mistake your project for a website.

</comment></comment></comment><comment><USER19> Very cool work, good job!

</comment><comment><USER20> No external dependencies? So this thing includes the BIOS?</comment>


Ask HN: Automatic Translation, why not? - <AUTHOR>

I'm currently working in a project which was the creation of a series of tools to improve language learning in an original way. I've recently incorpored in the project which was being executed a while ago.
The thing is, some of the people here are convinced that we could make a translator better than any of the translators in the market(google, babylon, etc...).
In my opinion the group doesn't have possibilities of succeding in the topic because of the lack of experience in the field and of course, because it's a really hard problem.
I would like to find material to show them that we're going to lose time and money trying to make something we can't. The approach to the problem is the sentence analysis, identify every element of the sentence being aware of the context in order to choose the correct tranlation of a word.
I've been reading the Stanford course of NLP but it doesn't especify why the approach mentioned above is discarded. <comment><USER2> Consider: "I'll set this here."

How many different meanings can you find for that? What about the classic: "Time flies like an arrow, but fruit flies like a banana."

Sentence parsing is a huge field with a lot of very good work. If you only have "grammatically correct" (whatever that means) sentences, then the "Link Grammar" approach is very good at parsing in context.

Natural sounding language production is extremely difficult. Almost everything machine produced sounds stilted and non-natural.

On the other hand, everyone has to start somewhere, and a truly new approach may yield better results than anyone else's. Unless you take the time, effort, money and risk, how will you know?

<comment><AUTHOR> Thanks for mentioning "Link Grammar", I googled it and I find very interesting information.

</comment></comment><comment><USER3> What is their background? If they have none as you say, it is going to be unlikely.

You mentioned reading the stanford course, there are also video lectures:

[http://see.stanford.edu/see/lecturelist.aspx?coll=63480b48-8...](http://see.stanford.edu/see/lecturelist.aspx?coll=63480b48-8819-4efd-8412-263f1a472f5a)

If they decide to proceed, two good NLP books:

1.Speech and Language Processing

2.Foundations of Statistical Natural Language Processing

<comment><AUTHOR> The group is composed by: a linguist, an "experienced programmer", me, two business guys and a translator.

The "experienced programmer" is what they said he is, but after a week I realised that he's not. I didn't know how to make an HTML parser, so I have to do it for him(I begin programming this year).

Thanks you for the books, I'll give them a try.

</comment></comment><comment><USER4> Think about how often idiomatic phrases come up in English. Then try to translate those phrases into, say, Chinese. And vice versa. Good luck with that.

"Steel rice bowl" makes no sense in English, and "apple of your eye" makes no sense in Chinese.

<comment><AUTHOR> They say we could have a table with a series of idiomatic phrases. Every sentence has to be compared with that table in order to know if its an idiomatic phrase or not. I think that's not a good idea.

<comment><USER2> That's really, really not a good idea. You need to get them to talk to someone who knows for real how hard this problem is.

</comment></comment></comment><comment><USER5> I think that's what everyone tries to do. What your group needs to do is talk to some of the people who have already worked on this.

Then again, there's no reason, a priori, that your group can't build the better mousetrap if they've got the requisite skills and resources.

<comment><AUTHOR> The problem is that we are from Argentina, sorry not to mention that. As far as I know, nobody in my country had a similar project.</comment></comment>


Richard Feynman and The Connection Machine - <AUTHOR> http://www.longnow.org/essays/richard-feynman-connection-machine/ <comment><USER2> I'm always amazed at the breadth and depth of Feynman's genius. My favorite story of Feynman is his calculation of the size of the first nuclear explosion and then driving to just a few miles outside the blast radius (much closer than his colleagues were willing to watch from). That's a level of mental confidence that I can't even begin to imagine possessing.

Surely You're Joking is a great read - one of those "drop what you're doing and read it now" kinds of books.

<comment><USER3> Not only did he drive there, but he watched with no more protection than the glass of the car windshield.

<comment><USER4> ...and ended up with about 18 ultra rare forms of cancer

<comment><USER3> Just two, according to 'kipedia.

</comment></comment></comment></comment><comment><USER5> I had read this before but went ahead and read it again anyway and was again struck by the practicality of Feynman's genius.

However, as always, every genius has their undesirable part:

"The charming side of Richard helped people forgive him for his uncharming characteristics. For example, in many ways Richard was a sexist. Whenever it came time for his daily bowl of soup he would look around for the nearest "girl" and ask if she would fetch it to him. It did not matter if she was the cook, an engineer, or the president of the company. I once asked a female engineer who had just been a victim of this if it bothered her. "Yes, it really annoys me," she said. "On the other hand, he is the only one who ever explained quantum mechanics to me as if I could understand it." That was the essence of Richard's charm."

When I was reading Feynman's biography ( _Surely You're Joking, Mr. Feynman!_ ) one thing that really bothered me was the big difference in his approach to his son and daughter. He seemed to relish how, when his young boy got his interesting games, but sounded quite impatient with his daughter, saying she only wanted the same story repeated to her.

<comment><USER6> I had a different reaction when I'd read _Surely You're Joking_. I thought Feynman had simply observed the de facto difference between his two children; that one happened to be a boy and the other a girl was incidental to the story. (As a mother, I can attest to the fact that it's quite impossible to get a kid interested in something they are not into to start with).

In general, I am a little apprehensive of Feynman's "sexist pig" rep. His younger sister became a physicist largely through his encouragement which tells me that his views on women in science were rather progressive, even by today's standards. He is also known for having sided with a female professor in a sex discrimination suit at Caltech - again, very ahead of his time.

I also recall that at least some of that rep came from his use of stories that were presumably sexist, such as stupid driver (woman), smart cop (man). But according to Feynman, it was his critics, not him, who assumed that the smart cop was male, so the egg was on them.

Finally, a lot of people conflate Feynman's womanizing with sexism, and I think the guy should get a break for trying to find solace in as many women's arms as possible after watching his wife slowly die from cancer.

<comment><USER5> You're right, his children had de facto differences, and that's OK. What annoyed me was his exasperated tone when he mentioned that his daughter liked having the same story read.

As for womanizing, again it shocked that after so little time passed after her wife's death, he jumps into the scene (and with a vengeance).

But maybe, I'm reading too much into this. Here's what Michelle Feynman (daughter) has to say about what kinf of father Feynman was (<http://www.basicfeynman.com/qa.html>):

"Fantastic. Fun, supportive, sweet, silly. He went to great lengths to entertain us. When I was young, the nightly ritual included him scouring the house for the correct stuffed animal, with me rejecting each and every offering until I had the desired one, or until he had made me laugh with his efforts. I'm not sure which was more important. He also made for an excellent radio. I would sit on his lap and turn his nose, and he would make up songs from different radio stations."

<comment><USER7> _As for womanizing, again it shocked that after so little time passed after her wife's death, he jumps into the scene (and with a vengeance)._

It's not like she died suddenly. They had ample time to prepare for her death, and by the time she was gone he was already prepared to move on. He likely went months with the sadness of watching his love leave him slowly, and at the same time he was not satisfying his sexual desires. I can certainly forgive the man for "jumping into the scene" when he no longer had an obligation to his late wife.

</comment></comment></comment></comment><comment><USER8> "By the end of that summer of 1983, Richard had completed his analysis of the behavior of the router, and much to our surprise and amusement, he presented his answer in the form of a set of partial differential equations. To a physicist this may seem natural, but to a computer designer, treating a set of boolean circuits as a continuous, differentiable system is a bit strange. Feynman's router equations were in terms of variables representing continuous quantities such as "the average number of 1 bits in a message address." I was much more accustomed to seeing analysis in terms of inductive proof and case analysis than taking the derivative of "the number of 1's" with respect to time."

Whoa.

<comment><USER3> I would love to see his analysis of the system. I doubt it's been preserved though.

<comment><USER9> If anyone has this, please contact me.

</comment></comment></comment><comment><USER10> A good read every time. This part stood out:

"In retrospect I realize that in almost everything that we worked on together, we were both amateurs. In digital physics, neural networks, even parallel computing, we never really knew what we were doing. But the things that we studied were so new that no one else knew exactly what they were doing either. It was amateurs who made the progress."

<comment><USER11> If everyone is an amateur, isn't the last sentence a tautology?

<comment><USER12> I don't think 'not knowing exactly what you're doing' implies 'being an amateur'.

</comment></comment></comment><comment><USER13> This is my favorite line:

"That sounds like a bunch of baloney," he said. "Give me something real to do."

...advice for the ages!

</comment><comment><USER14> Here's another favorite story from thinking machines: The Rise and Fall of Thinking Machines

<http://www.inc.com/magazine/19950915/2622.html>

For what it's worth, last I heard Sheryl Handler was running Ab Initio. I wonder if she's really the nutcase the article makes her out to be.

</comment><comment><USER15> I thought this was an especially interesting point:

"Every great man that I have known has had a certain time and place in their life that they use as a reference point; a time when things worked as they were supposed to and great things were accomplished."

<comment><USER16> Me too. I instantly knew what that time period was for me.

This concept immediately reminded me of an excerpt from the outstanding book "Rules for Aging" which talks about this and its potential dark side:

"A long happy life last five minutes. One would think that this rule would go without stating, but many people actually believe that a long life of uninterrupted happiness is a real possibility. And they act on this belief! They change families, careers, the structure of their faces, countries, everything, for no more substantial reason than they recall five minutes of uninterrupted happiness in the past, and now they wish to re-create the moment in perpetuity. They even convince themselves that the five-minute period they recall was really five years and giddily substitute the exception (bliss) for the rule (confusion, doubt, misery, fear, confusion, and confusion). Happiness is wonderful, but if you have had more than five consecutive minutes of it, it means you weren't thinking."

That book isn't really about aging, it's more about practical down-to-earth wisdom. I read it for the first time when I was in my early 20's, and it's very amusing and pure gold.

</comment></comment><comment><USER17> "Richard made people feel like a child does, when a grown-up first treats him as an adult. He was never afraid of telling the truth, and however foolish your question was, he never made you feel like a fool."

I don't know much about Feynman but the combination of his brilliance and his popularity has always intrigued me. Surely we can assimilate some of Feynman's charm.

</comment><comment><USER18> I've spent a little bit of time with Danny. He's astonishingly smart. So to hear him talk about someone else like that makes me jealous I couldn't have been around.

</comment><comment><USER19> I cant seem to get to the article but I think I know which article this is. I'm a huge fan.

I especially like how the machine was initially designed so that it required 4 chips per board. But when they were about to start manufacturing the boards, they realized that present technology couldnt fit 4 chips per board.

Feynman it seems modeled the system via set of partial differential equations and then said that he had proved that 3 chips per board were enough and modeled how the system would work even in this configuration.

No one really understood what he had done but they trusted him, created the boards and it turned out he was right.

This story I believe was in an Scientific American a long time ago.

<comment><USER20> Pretty good memory. The article says the numbers are 7 and 5, not 4 and 3, but the idea is the same. I wonder how the conversations went when the others initially overruled Feynman's recommendation and then, months later, were forced to use it and admit he was correct all along.

</comment></comment><comment><USER21> This story never gets too old to re-read.

</comment><comment><USER22> I do wonder if any of the parallel features of their Lisp will ever migrate into newer Lisp / Scheme implementations.

<comment><USER23> StarLisp is basically Common Lisp with parallel-map and parallel-reduce and a special vector data type for them. You can do data-parallel operations over arrays trivially, but it's not clear what array chunk size tradeoff you need for a given problem on today's small core count machines. It might make sense to bring the approach back when there's 100-core machines around; that would basically be a CM-5 on a chip.

In the meantime there's a StarLisp simulator you can play around with: <http://sourceforge.net/projects/starsim/>

When it comes to data parallelism, StarLisp's successor Paralations ([http://www.mactech.com/articles/mactech/Vol.08/08.07/Paralat...](http://www.mactech.com/articles/mactech/Vol.08/08.07/Paralation/index.html)) is much more interesting because it deals explicitly with locality. Paralations' successor, NESL (<http://www.cs.cmu.edu/~scandal/nesl.html>) is implemented on top of Common Lisp, but it's a functional language with ugly syntax.

It would make a lot of sense to have something like StarLisp or APL for CUDA right now. Trying to do data parallelism in C is about the most brain-damaged idea ever. I don't know if anyone is working or interested in that, though.

Overall, I think the MultiLisp (futures) approach is a bigger win for general- purpose parallel programming than StarLisp data parallelism. Doing everything with data-parallel algorithms is hard - you're doing everything with array programming like in APL, but now you need to come up with clever ways to parallelize everything too. But the algorithms themselves are beautiful (read Hillis and Steele's Data Parallel Algorithms, my favorite algorithms paper: [http://docs.google.com/viewer?a=v&q=cache:efZTS1nDfBsJ:c...](http://docs.google.com/viewer?a=v&q=cache:efZTS1nDfBsJ:citeseerx.ist.psu.edu/viewdoc/download%3Fdoi%3D10.1.1.85.1876%26rep%3Drep1%26type%3Dpdf+hillis+steele+Data+parallel+algorithms&hl=en&gl=ca&pid=bl&srcid=ADGEEShW4j_bt4y5VRsTqM7e7IPnAHp1KxQGGH0aHERCb- YZiQBvx8_ERTfrgzb2TsIryUWfBGRUY-JNCEh-reU0BcUfcC8nusO- TyqzlzNelNaeIDsWdgtM7kFLZVuMgIN6_UrDfeOU&sig=AHIEtbRyegDoFIwWgJrFCgdgIb2VlOAseg))

<comment><USER24> > It would make a lot of sense to have something like StarLisp or APL for CUDA > right now. Trying to do data parallelism in C is about the most brain- > damaged idea ever. I don't know if anyone is working or interested in that, > though.

You may well be right, but I challenge you to prove it. I myself am very interested in whether this would work. I have spent many sleepless nights programming GPU algorithms, and wondered if ideas from other programming languages and paradigms (especially functional programming) could be applied to make it easier and more elegant.

The nested data-parallelism approach does look promising on paper, and many people are well aware of the theoretical possibility of this working on GPUs (including people at Nvidia itself), but so far nobody succeeded to make it practical.

So, doing data parallelism in CUDA may be brain-damaged, but the flexibility and performance it delivers is mind-blowing. If you can achieve something comparable using a higher-level functional approach, I will be among your first users, and I'll tell everyone I know.

MultiLisp (AFAIK) is traditional task-parallelism rather than data- parallelism, and would not work well on a GPU.

<comment><USER23> I'm not talking about data parallelism being bad, but C being a bad data parallel language.

<comment><USER24> Of course... read my reply again please.

I am not disagreeing with you, I am saying that I would love to see a better data-parallel language than C (or CUDA to be precise), but it does not exist right now (at least not a practical one that would run on a real GPU with reasonable performance and allow non-trivial nested and hierarchical algorithms).

</comment></comment></comment></comment></comment><comment><USER13> Are any of you aware of more information on the "parallel version of Basic" mentioned?

</comment><comment><USER25> My comment should really be an Ask HN. But is it conceivable to build a CM-5 class computer on FPGA boards ? Any idea how much that would cost if it was possible ?

<comment><USER26> Late CM-5 was 64-node MIPS connected through hypercube or fat tree. I think you can fit several MIPSes into one high-class FPGA - it looks like 4 would fit nicely into Virtex 5 TXT series, not so sure about more.

I think that you will need custom boards for interconnection.

So it will be some costly project.

Plasma, a simple MIPS-I variant: <http://opencores.org/project,plasma>

<comment><USER25> Thanks and upvoted for the reply and the link. Did not expect my comment to picked up, given I joined so late in the thread. In terms of raw FLOPS I think one can comfortably top the CM-5 machines with current commodity hardware. But it would have been pretty cool to have a personal CM-5 to play with.

</comment></comment></comment><comment><USER27> yes you all are right and even i would like to revisit this</comment>


Hackers vs. Consultants: How not to find an idea (Part 2 of 3) - <AUTHOR> http://behindthescenes.midtownrow.com/

<comment><USER2> This was a much better article than the last one. In fact, I'd go so far as to say the first one should have just been skipped. Everything that needed saying was in this one, and completely removed all my objections from the first.

</comment><comment><USER3> Although some parts of the posts are a bit too stereo typed for my me, I really like the general approach. Its an interesting clash of opinions / cultures, catchy to read. I hope to hear more from you guys!

</comment><comment><USER4> 'Catch up on Part 1 of this series' link is not working. I am getting 404. I know I can scroll down for previous post, just wanted to let you know.

Cheers

<comment><USER5> Sorry for the broken link, and thanks for looking out. Should be fixed now.</comment></comment>


How Exercise Changes Our DNA - <AUTHOR> http://well.blogs.nytimes.com/2014/12/17/how-exercise-changes-our-dna/?ref=health

<comment><USER2> [https://hn.algolia.com/?q=exercise+dna#!/story/forever/0/exe...](https://hn.algolia.com/?q=exercise+dna#!/story/forever/0/exercise%20dna)</comment>


The maths of the paper disproving conspiracy theories don't add up - <AUTHOR> http://littleatoms.com/david-grimes-conspiracy-theory-maths <comment><USER2> A personal theory i have is that the "Sceptics"community is somewhat comprised of slightly socially naive stem educated people (sorry). They simply do not fathom the ruthlessness of the political or business world. Hackers are a notable exception, they usually have they usually are pretty adept at seeing propaganda and cover ups.

The notion that "i can prove that conspiracies are false with my math" makes me cringe the same way it makes me cringe when the sceptics community "Proves religion wrong", completely disregarding any psychological or anthropological explanations for such a phenomenon.

A good chunk of the highly educated people from the economic fields, Law, Public Relations and business world are much more manipulative and opportunistic than most "science people" understands.

Go to any of the hippest (and most expensive) night clubs in a european city and you will meet these kinds of people everywhere, earning huge amounts of money doing dubious business deals or polishing the images of morally questionable partners.

You won't meet many tech people there, but the ones you will meet will be in ad-tech, data-reselling, or affiliate marketing.

You won't se any nerds these places, as you won't se any nerds at PR or upper class "fund raising" events, where the small scale conspiracies are pretty obvious.

<comment><USER3> Having spent time in both the tech world and the politics world, in my experience it is the conspiracy theorists who are naive.

Generally the common denominator in conspiracy theories is that there is a group of people who are secretly in control and are purposefully causing things to happen to further their own particular agendas.

As you point out, it is true that a lot of powerful people really _are_ trying desperately to further their own agendas. But the false part is that there is a small group who is really in control. The truth is, no one is in control.

Sounds scary, right? Which explains the attractiveness of conspiracy theories.

<comment><USER4> It depends a lot on your definition of 'conspiracy theory'. If you only consider crazy babblings about martians, sure, but...

Do you remember that before Snowden, the whole NSA thing was a tinfoil-hat worthy idea? And what the NSA does goes beyond what moderately tinfoily people thought.

"Conspiracy theory" is often a good way to get rid of the pursuit of truth.

<comment><USER5> A conspiracy theory is merely a theory without corroborating evidence. The reason conspiracy theorists tend to be disbelieved is that, more often than not, their assertions are a matter of faith, and when they're right, they're right in the way a stopped analog clock is right twice a day. If someone's paranoid fantasies happen to correspond to reality, that doesn't make them more trustworthy.

So while conspiracies shouldn't be dismissed out of hand, they also shouldn't be given special credence, either, and skeptics should be assumed to be part of an attempt to suppress the pursuit of truth.

That said, A lot of skepticism is as much an attempt to reinforce confirmation bias as is conspiracy theory - and one could say that conspiracy theory is a form of skepticism about the apparent natural order of things. So both sides tend to suffer from a tendency to not actually care about the truth when that truth could contradict their prejudice.

<comment><USER6> > A conspiracy theory is merely a theory without corroborating evidence.

It is interesting that the only term that we had to refer to a "theory about an agreement between persons to deceive, mislead, or defraud others of their legal rights or to gain an unfair advantage" became a synonym for lunacy, while leaving us with no short way to express the former concept.

A conspiracy theorist might suggest that we can be controlled with language.

<comment><USER5> To be fair, the more lunatic type of conspiracy theories are the ones most people hear about. The X-Files might be more to blame for that perception than the government.

Assuming of course, the X-Files wasn't secretly a propaganda campaign intended to discredit conspiracy theorists by subconsciously associating certain theories with fiction. But then again, the government has used urban legends and conspiracy theory as a cover for its own operations before, i'm personally certain they planted the Roswell story (and retraction) in order to cover up a more mundane secret project and retrieval, but just never expected it to go as viral as it did.

But even so, conspiracy theorists are half to blame for their own reputation at least.

</comment></comment></comment></comment></comment></comment><comment><USER7> It seems that the original paper didn't consider participants' incentives to keep the secret.

> All three are based in the United States, two in law enforcement or security > services where secrecy is part of the job description and the cost of > breaking it is extreme.

There must be differences between conspiracies where the conspirators agree that the secrecy is proper or beneficial, and conspiracies where people are forced into it or simply become aware of the secret by chance or as part of a job. For that matter, there must be differences between conspiracies where people are trying to unmask them (for example, national security reporters who have heard a rumor about the existence of a secret program and are investigating to follow up on it) and conspiracies whose existence hasn't been hypothesized by outsiders.

Just today the Washington Post reported on internal CIA use of deception against its own staff:

[https://www.washingtonpost.com/world/national- security/eyewa...](https://www.washingtonpost.com/world/national- security/eyewash-how-the-cia-deceives-its-own-workforce-about- operations/2016/01/31/c00f5a78-c53d-11e5-9693-933a4d31bcc8_story.html)

> Senior CIA officials have for years intentionally deceived parts of the > agency workforce by transmitting internal memos that contain false > information about operations and sources overseas, according to current and > former U.S. officials who said the practice is known by the term "eyewash."

In this case, there could be hundreds of people who think they know the truth about something, but really only a handful do. (The Post explicitly says this.) So even if one of those hundreds of people reveals what they know, the conspiracy won't really be revealed.

</comment><comment><USER8> For any interested, I wrote an entire essay and some arguments in favor of conspiracy theory on Schneier's blog here:

[https://www.schneier.com/blog/archives/2013/06/the_psycholog...](https://www.schneier.com/blog/archives/2013/06/the_psychology_7.html#c1503021)

I argue that conspiracy, or key elements of it, is a natural part of human behavior. You can see it in all kinds of legitimate things. You can also see it in many criminal activities. A subset of it would be what we traditionally call a criminal conspiracy good enough to leave only breadcrumbs. The conspiracy theories... one's using good investigation rather than cherry- picking... have to find and tie together these breadcrumbs to derive the hidden activity.

Academics almost exclusively tend to analyze why people must be wrong-headed if they investigate conspiracies, err, criminal activity. Instead, they should look at those that were proven right and wrong to identify data points for criteria or heuristics to help investigators get it right more often. What constitutes good evidence of a probable conspiracy vs what is just bias of researcher? A valid question and form of research.

However, it's nonsense and defies common sense to have their assumption that conspiracies don't happen and investigating one is equivalent to mental illness unless you have a confession in hand from perps.

<comment><USER9> I agree that there are probably some conspiracies, even statistically its bound to happen. People always have something to hide. Whats ridiculous though are the infowars type of sites that literally state everything is a conspiracy and that every action a government or cooperate official makes is to enslave people and make more profits.

<comment><USER10> These sites utterly fail to understand the difference between an active conspiracy and merely _coincident interests_ for certain things to happen.

For example, if an industry is predicated on certain things happening or not happening (e.g. military contractors being biased towards war) or denying certain facts (e.g. oil companies playing down global warming) it may seem like a widespread conspiracy but it's actually a bunch of independent actors with the same motivations or bias.

There's also the fact that a large conspiracy needs a large motivating factor. A group of criminals conspiring to steal something has a very clear reward. The government spraying massive amounts of chemicals from airplanes to do...stuff...is hardly a compelling reason.

<comment><USER8> "These sites utterly fail to understand the difference between an active conspiracy and merely coincident interests for certain things to happen."

That's a good point. This happens a lot with oligopolies, too. Yet, there has been evidence that some of them collude in secret often through intermediaries to expedite this process. You usually see this with lobbyists but sometimes outright scandals like RAM price-fixing. So, coincident interests doesn't auto-eliminate possibility of a conspiratorial explanation but certainly should be the _default_ belief for scheming, self-interested behavior.

"The government spraying massive amounts of chemicals from airplanes to do...stuff...is hardly a compelling reason."

I smirked at the wording. The amount of mythology around chemtrails mostly makes it a good example. Yet, you're off the mark on this one just as they are.

[https://en.wikipedia.org/wiki/Operation_Sea- Spray](https://en.wikipedia.org/wiki/Operation_Sea-Spray)

So, you can replace "stuff" with testing of bio-weapon systems and studies of disease impact on unwitting subjects with legal immunity. Not sure if any of the chemtrail claims or evidence overlapped with that as I didn't research it much. Yet, it's a fact that the U.S. military secretly (for a while) flew planes over U.S. cities that doused them with both chemicals and biological agents to support our biological and chemical weapons capabilities. There were many groups involved in that. They even blocked further investigation of all the incidents and those effected. So, we know what's revealed is a fraction of what went on but I'm not going to speculate size of that fraction. I just know there's more.

This brings me to another facet of the problem: supporting irrational stuff with rational stuff. The fact that certain conspiracies and lies happened before seem to increase the likelihood people will believe a similar claim. "Of course they're doing chemtrails on us: remember Sea-Spray!? Why wouldn't they still be doing it?" That doesn't logically follow but does in many people's minds. Hell, it often does in reality so much we have the meme that "History Repeats." I think allowing recurring and similar rogue behavior in industry and governments plants many seeds for other, false beliefs to show up. Another reason to put an end to any schemes we identify. Affects signal- to-noise ratio of our ability to detect sneakier schemes.

Good news is the biggest bullshiters in conspiracy claims are usually _really_ full of shit. Obviously. They practically out themselves. Helpful to researchers like myself as we just filter them away.

</comment></comment></comment></comment><comment><USER11> Conspiracy's clearly don't exist, except when the US government is found guilty of them in court of law:

[http://www.nytimes.com/1999/12/09/us/memphis-jury-sees- consp...](http://www.nytimes.com/1999/12/09/us/memphis-jury-sees-conspiracy- in-martin-luther-king-s-killing.html)

[https://en.wikipedia.org/wiki/United_States_House_Select_Com...](https://en.wikipedia.org/wiki/United_States_House_Select_Committee_on_Assassinations)

<comment><USER12> The argument in the paper (and I admit I find the analysis unconvincing) was the a successful conspiracy requires a relatively small number of people involved to successfully carry out and to go a long time undetected, not that they don't exist.

</comment></comment><comment><USER13> If you throw everything together it's uninformative.

1\. Obviously, some things are successful "conspiracies" to continue nefarious activities despite what would be public aversion: An obvious example is the clergy sex abuse case.

2\. Obviously, other supposed conspiracies are bullshit: Chupacabra, Moth-man, LGMs at Area 51.

3\. Other things are a kind of readily identifiable, if you are historically literate, forms of opportunism: The rise of the security state and the neocon wars after 9/11, for example. 9/11 wasn't an "inside job" but it was cynically exploited about as far as possible. From Winston Churchill to Rahm Emanuel, politicians have known not to waste a good crisis. "Cui bono?" Yeah, the people who jumped on it and exploited.

</comment><comment><USER14> Good point, worth skimming [if you've heard of the original article].

Maybe a better direction to go would be "how long until leaks occur" given the number of people on a project. This isn't that different [although it's less click-bait-y than mentioning conspiracies], and you'd have lots of examples from industrial products.

How often do Apple product releases leak? How long does it usually take for them to leak?

There should at least be enough data to do something interesting.

<comment><USER15> >How often do Apple product releases leak? How long does it usually take for them to leak?

how many of those are actually leaks and not "guerilla" marketing like movie trailer leaks.

<comment><USER16> Wouldn't they leak every time if intentional? The fact that pics only rarely get released in advance implies the leaks aren't authorised.

<comment><USER17> Not necessarily, a constant reliable stream of information isn't as 'interesting' as more random results.

I think the right terms for this are "variable reward" and "variable interval" here:

[https://en.wikipedia.org/wiki/Reinforcement](https://en.wikipedia.org/wiki/Reinforcement)

</comment></comment></comment></comment><comment><USER18> "Paper disproving the concept of conspiracy theories found to be erroneous; spawns multiple conspiracy theories and adds credence to innumerable more"

</comment><comment><USER19> So, he never actually said it was a cumulative failure curve. I'll admit a graph of the probability of failure per year (non-cumulative) may be less expected, but doesn't necessarily indicate an error on the authors part.

And the bit about his estimates of the number of conspirators in the NSA.. he says the exact same thing: "In the PRISM case, the figure of 30,000 comes from total NSA staff. In reality, the proportion of those employed would would have knowledge of this program would likely be a lot less but we take the upper bound figure to minimize the estimate of p."

</comment><comment><USER20> It always annoys me when reading about this paper, that the Climategate (parts 1 & 2) incident is ignored.

It doesn't prove that there's a climate-change conspiracy, but it was a bona- fide attempt by someone who had access to internal documents to whistleblow.

I don't understand why NSA - Snowden is an example of a conspiracy being exposed but UEA - Climategate is not.

Unless of course, it's career suicide for an academic to come within a mile of being seen to portray climate change as a conspiracy. Which could be taken as more evidence that there is a conspiracy.

<comment><USER21> > _I don 't understand why NSA - Snowden is an example of a conspiracy being > exposed but UEA - Climategate is not._

Because when Snowden's documents were analyzed the evidence matched reality, while when the "Climategate" emails were analyzed all that was found was normal researcher chatter?

Seriously, the critics of the emails quoted a few phrases out of context that sounded bad but weren't and couldn't point to any change made to data or methods that wasn't already publicly documented. Then a bunch of independent bodies went through the emails and data and agreed all was good.

Mainstream rejection of a story is _not_ evidence that that story describes an ongoing conspiracy.

You might as well ask why Obama - Birthers is not considered an example of a conspiracy being exposed. Unless of course, it's career suicide for a politician or journalist to come within a mile of being seen to portray his citizenship as a conspiracy. Which could be taken as more evidence that there is a conspiracy.

<comment><USER20> my point has nothing to do with whether you believe climate change to be a hoax or not, or whether it is a hoax or not.

The point is that someone exposed a bunch of internal documents in an attempt to whistleblow what they considered to be a conspiracy.

The reason there's no parallel with the Birther thing is that no-one in the Obama camp attempted to whistleblow.

The reason there is a comparison with Snowden-NSA is because Snowden was a whistleblower.

The paper in the article holds up the climate change conspiracy theory as "it can't be a conspiracy because no-one attempted to whistleblow it" while completely ignoring that someone DID try to whistleblow it.

Again, whether or not there was actually any conspiracy doesn't matter. Someone "on the inside" thought there was and attempted to whistleblow. It needs to be included in the list of failed conspiracies regardless of whether it actually was a conspiracy or not.

<comment><USER20> thanks for the knee-jerk downvoting too.

At no point have I criticised anything about climate change, or even suggested that it might not be the most urgent pressing thing that human society faces.

But the automatic downvoting/criticising happens to any post that dares to mention anything like Climategate. I had to think twice before mentioning it because I knew I'd lose some of my precious internet points.

If this happens here, where there are no consequences, what happens in academia, where the penalties of being sceptical are so much more serious, even career-threatening?

<comment><USER21> > _If this happens here, where there are no consequences, what happens in > academia, where the penalties of being sceptical are so much more serious, > even career-threatening?_

Looks like some of your points were restored, but I'll note I downvoted you for the above. It's ridiculous to assert some relationship between this message board and academia, or that downvotes are responding specifically to a single cherrypicked idea from that post.

It ends up being complaining about downvotes with a not so subtle insinuation that the subject wasn't given a fair shot (when it was actually _widely_ and very publicly investigated) and you're begging the question by asserting that any dismissal of the question is evidence of dismissal of the question for a specific reason, when it could just be that some questions have already been addressed and so are no longer worth more time than it takes to give a downvote.

</comment></comment></comment></comment></comment><comment><USER22> it's was a coverup attempt!!! :-)

</comment><comment><USER23> Are there massive conspiracies? Just look at the set of religions in the world to control people-- and you will already have your answer.

<comment><USER24> Conspiracy is usually something that is non-obvious. The sociodynamics of religion are quite obvious, and closer to a mass hysteria.

The parish provides a sufficient population that enables social proof to convince people automatically of the correctness of believing and belonging. It's not that the priesthood makes people believe things - it's the people, and the social proof and the social pressure that does this. The priesthood gains political power (which all men yearn, consciously or not), and when co- operate with the state provide a channel of mass control and communication. Thus the people perpetuate the belief, and the state has no incentive to intercede.

</comment></comment><comment><USER25> The title isn't grammatical. Math and Do must agree.</comment>


Allocation for last batch of IPv4 addresses begins - <AUTHOR> http://www.ripe.net/internet-coordination/news/about-ripe-ncc-and-ripe/ipv4-chart-updated

<comment><AUTHOR> To make it clear, the RIPE represents: Europe, the Middle East and parts of Central Asia. APNIC(The Asia Pacific) have : 0.9126 /8 blocks ARIN (American) have : 3.3552 /8 blocks LACNIC(Latin American and Caribbean) have : 3.2192 /8 blocks AFRINIC (Africa) have : 4.1416 /8 blocks</comment>


Cobol turns 60: Why it will outlive us all - <AUTHOR> https://www.zdnet.com/article/cobol-turns-60-why-it-will-outlive-us-all/ <comment><USER2> Free COBOL on the web: [https://www.thefreecountry.com/compilers/cobol.shtml](https://www.thefreecountry.com/compilers/cobol.shtml)

COBOL has been ported to many different platforms.

Like COBOL for GCC: [http://cobolforgcc.sourceforge.net/](http://cobolforgcc.sourceforge.net/)

</comment><comment><USER3> I used to program COBOL and it is a bit of a pain on mainframes. There is Open COBOL for Windows, GNU/Linux and Mac. It is now known as GNU/COBOL [https://sourceforge.net/projects/open- cobol/](https://sourceforge.net/projects/open-cobol/)</comment>


Asynchronous Programming in Rust book - <AUTHOR> https://rust-lang.github.io/async-book/ <comment><USER2> There isn't _too_ much activity on this book [1] but I definitely think that more documentation about async programming in Rust is needed. Just recently I wanted to do something in async Rust and it's just such a PITA. I'm writing Rust since 3-4 years now and async throws me back to those first days where I didn't know how to cope with the error messages. Hopefully async/await syntax will improve this experience, but even then I think that documentation is needed. The futures crate is severely underdocumented. I'd love to have an example snippet next to each combinator etc.

[1]: [https://github.com/rust-lang/async- book/commits/master](https://github.com/rust-lang/async-book/commits/master)

<comment><USER3> The documentation is asynchronous.

I'll show myself out.

<comment><USER4> await oriented pedagogy

</comment></comment></comment><comment><USER5> Is it just me, or is the fact that the most important part:

[https://rust-lang.github.io/async- book/getting_started/state...](https://rust-lang.github.io/async- book/getting_started/state_of_async_rust.html)

Is missing, somewhat ironic?

Feels very much like the state of async matches the state of the guide. :P

What _is_ the state of async? Is it close? Is it still changing with the futures 0.3-beta not finalized?

Are we six months away? A year?

<comment><USER6> You can track the progress of the remaining issues here:

[https://areweasyncyet.rs/](https://areweasyncyet.rs/)

<comment><USER5> I know, but I still struggle to get a handle on the state of it really.

What _is_ going on with futures 0.3? Why is everyone still using 0.1?

How does that relate to these issues?

It superficially appears like the whole async story is still in a concept stage...

<comment><USER7> It's the open source approach to upgrading. All the cool kids are focused on version N+1, which doesn't work yet. The users still on version N don't get support any more because only losers use version N. You see this pattern frequently in open source. The Python 3 debacle spent five years in that state.

Commercial products tend to avoid this. Sales of version N go way down before version N+1 is generating revenue. Overall revenue drops during the transition. That's not good.

<comment><USER8> Note that this isn't what's happening here; tokio is explicitly supporting "version N" in your terminology. Which is why your parent is asking why people still seem to be using the "old" version.

(Also, there's a compatibility layer, so even the people that want to play with the shiny new N + 1 can do so, even though it's not explicitly directly supported.)

</comment></comment></comment></comment></comment><comment><USER9> This approach to async programming feels like a much more leaky abstraction than the 'it's basically semaphores' stuff for m:n threads. Though being able to do so much as a library is nice.

How does async translate calls to other async functions? Is refactoring into smaller async functions less efficient? If not, how does it deal with (possibly indirect) recursive function calls? Does it give up or select a loop breaker?

And what is the purpose of the pingpong between executor->Waker->push onto executor?

I am also still unsure what the approach to multithreading might be. Multiple executors with work stealing or one dispatch executor with worker threads or something else still?

<comment><USER8> > How does async translate calls to other async functions?

There's nothing special going on. Remember, async on a function is something like

      async fn function(argument: &str) -> usize {

to

      fn function(argument: &str) -> impl Future<Item=usize> {

so, when you call an async function, you get a Future back. That's true even if it's inside of another async function.

> If not, how does it deal with (possibly indirect) recursive function calls?

Recursive calls to async functions will fail to compile: [https://github.com/rust-lang/rust/issues/53690](https://github.com/rust- lang/rust/issues/53690)

That said, see that discussion; the trait object form will probably eventually work.

Heavy recursion isn't generally Rust's style, since we don't have guaranteed TCO, so you threaten to overflow the stack and panic.

> Is refactoring into smaller async functions less efficient?

That's a complicated question. It really depends. I don't think it should be, thanks to inlining, but am not 100% sure.

> And what is the purpose of the pingpong between executor->Waker->push onto > executor?

Right now, the best resource is [https://boats.gitlab.io/blog/post/wakers-i/](https://boats.gitlab.io/blog/post/wakers-i/) and [https://boats.gitlab.io/blog/post/wakers- ii/](https://boats.gitlab.io/blog/post/wakers-ii/)

> I am also still unsure what the approach to multithreading might be.

You have options! Tokio now does multiple executors with work-stealing by default, in my understanding.

<comment><USER9> Thanks for your answers!

From the second blog post I actually found [https://github.com/tokio- rs/tokio/pull/660](https://github.com/tokio-rs/tokio/pull/660) which switched tokio from 1 reactor+worker threads to n reactors with work stealing.

</comment></comment></comment><comment><USER10> This book is largely a work in progress, still -- note all of the TODO's. Async-await designs have to stabilize before the rest is done.

</comment><comment><USER11> Sounds like an interesting book, the Rust Programming Language book was a great read ([https://doc.rust-lang.org/stable/book/](https://doc.rust- lang.org/stable/book/))

</comment><comment><USER12> How much performance is gained by going async instead of blocking threads on modern hardware?

Skimmed through [https://vorner.github.io/async- bench.html](https://vorner.github.io/async-bench.html). If I understand it correctly, one get about twice the performance with async.

Is this correct? Seems like a compromise (code complexity vs performance) not worth taking.

<comment><USER13> async isn't only about performance, but has other advantages, like reduced resource consumption. In addition to that async io also gives you better control over how to cancel io reads and writes on systems where the IO is not interruptible.

But you are correct, if you don't have a specific need, async is generally harder than using threads for concurrency. Ideally the async/await work in Rust is going to make that trade-off less extreme than it is today, which may mean more people will feel comfortable using it as it should reduce boiler plate.

<comment><USER14> > but has other advantages, like reduced resource consumption

Could you expand on that? I've never heard that mentioned about async before.

<comment><USER8> You can think of a task as being a thread, but it has one single allocation that's the exact possible stack size. No more, no less. This uses less memory than spinning up a thread with the default stack size. Yes, you could use the proper APIs and get the correct size too, but you have to figure that size out by hand for each thread. It just implicitly happens with tasks.

<comment><USER12> Default stack size for new threads is 2 MB, [https://doc.rust- lang.org/std/thread/#stack-size](https://doc.rust-lang.org/std/thread/#stack- size). That means total_memory_usage = 2 MB * number_of_cores, which is not so much on modern hardware.

<comment><USER12> ...and of course you know that, being on the Rust core team, just adding the specifics (I did not know the numbers myself).

<comment><USER8> Hm, maybe I misunderstand what you're getting at; you're talking about one thread per core, not one thread per unit of work? Sure, if you only have that few threads, then it's not that big of a difference, but if you want to spin up a few hundred thousand of them...

<comment><USER12> > you're talking about one thread per core, not one thread per unit of work?

Yes, a thread pool, consisting of one thread per core/computing unit. The units of work are then scheduled between the threads. Units of work here being some kind of IO, e.g. servicing HTTP requests.

> but if you want to spin up a few hundred thousand of them...

Hm. Thought there was a limit for work that can be done concurrently by the CPU, based on the number of cores/hyper-threads available. Found this on threads and IO performance [1], it seems to make the same point.

What kind of work load is common to spread over so many threads (on the same machine)? Does the OS switch efficiently between hundred of threads on regular CPUs? Genuinely interested.

1: [https://www.jstorimer.com/blogs/workingwithcode/7970125-how-...](https://www.jstorimer.com/blogs/workingwithcode/7970125-how- many-threads-is-too-many)

<comment><USER8> Okay so, there are lots of ways to do this kind of stuff. A threadpool is a pretty classic one. Apache being the poster child here in an HTTP server context.

> Thought there was a limit for work that can be done concurrently by the CPU,

Right. But in an IO bound scenario, the CPU isn't doing work; it's waiting on IO. So, because threads are generally heavy, you don't want a ton of them, taking up memory, doing nothing.

But, when you have lightweight threads, you can spin up one per connection. This ends up being simpler, and you don't have the large memory usage. This is what nginx does, in a sense. It still has a worker per core, but each of those workers can handle thousands of requests simultaneously, because it's all non- blocking.

That limit to concurrent work is exactly why non-blocking architectures are so important, and task systems fit into them really nicely.

<comment><USER13> Excellently said, Steve. This is a great thing to know in this context, "Latency numbers ever programmer should know": [https://gist.github.com/jboner/2841832](https://gist.github.com/jboner/2841832)

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER15> I think I'm correct in saying you don't _need_ tokio for async but it seems all non-toy code uses it. Are there any alternatives to tokio out there for writing real async code or is the idea to build everything on it? As if it was std, but it's not... right?

<comment><USER8> Sorta, kinda. One big example of when you wouldn't use Tokio is when you don't have an operating system.

Tokio is a good, default choice, but some projects may have different needs.

</comment></comment><comment><USER16> Also known as: collaborative multitasking.</comment>


The meaning of AMP - <AUTHOR> https://adactio.com/journal/13035 <comment><USER2> All of this is true, but unfortunately, none of it seems to matter. Because almost every major web property relies on Google, AMP has continued to grow and thrive, and it shows no chance of stopping any time soon.

It actually doesn't even matter what Google's marketing folks say about it. As long as Google controls the web, whatever Google wants is what websites are going to do.</comment>


Tips for running HBase on low-memory EC2 - <AUTHOR> http://dev.hubspot.com/blog/hbase-tutorial-5-tips-for-running-on-low-memory-ec2?utm_campaign=hbase&utm_medium=hackernews&utm_source=social

<comment><USER2> Thanks for writing this. We're also running a large HBase cluster on EC2 and frustrated with the poor suitability of instance offering (there's literally nothing cost effective for HBase in the current generation).

I am curious why you chose c1.xlarge over m1.xlarge, which is cheaper and has double the memory. In my experience HBase tends to be more memory bound than CPU bound.

<comment><USER3> We did initially use m1.xlarge, a few years ago when we first started with HBase. I no longer have the hard numbers available, but while I agree HBase is memory hungry, our m1.xlarge servers were constantly pegged for CPU. For a while we ran both types, and eventually phased out the m1.xlarges entirely.

We run pretty frequent compactions due to our read and write patterns, so of course if you do not need to compact as often you might get much better mileage out of the m1.xlarge.</comment></comment>


Ask HN: Do you have cyber insurance? - <AUTHOR> Does it make sense for a startup? <comment><USER2> Do you mean insurance against security breaches? If yes, I don't know how this would work in practice. Please elaborate your question!</comment>


iPhone 8 – 64GB Prices in 31 Countries - <AUTHOR> https://blog.prisync.com/iphone8-64gb-worldwide-prices/ <comment><USER2> Kind of intrigued why they used hourly gross minimum wage to determine how many days one would have to work to buy an iPhone 9. Why not median or mean?

<comment><USER3> Agreed. Median salary would have been way more meaningful.

Although it is interesting to see that despite the US's absurdly low minimum wage, the purchasing power actually stacks up pretty well with other countries, even those with absurdly high minimum wages.</comment></comment>


Desktop Computer Kicks Supercomputer Out of Top Place on Pi Record List - <AUTHOR> http://www.dailytech.com/Desktop+Computer+Kicks+Supercomputer+Out+of+Top+Place+on+Pi+Record+List/article17293.htm <comment><USER2> Much discussion already: <http://news.ycombinator.com/item?id=1032063>

<comment><AUTHOR> Whoops... my search fu is seriously rusty today.

<comment><USER2> I commented on the first version(s) so I knew it already existed.</comment></comment></comment>


WPEngine Under Large Scale DDoS Attack - <AUTHOR> https://wpenginestatus.com/intermittent-connectivity-issues-in-one-us-datacenter-5/ <comment><USER2> Damn, WPEngine's been under a lot of attack lately. They do hold a lot of valuable assets in their servers. Say what you will about Wordpress, but it's essentially the ultimate sales and marketing tool of the 21st century.</comment>


Why we chose Mongo + Hypertable - <AUTHOR> http://brownbag.springmetrics.com/2012/03/i-think-were-going-to-need-a-bigger-boat/

<comment><USER2> What do they use for hosting?

As Jay Kreps (works at LinkedIn, created Voldemort DB) would wonder, they went from a service managed by Amazon to what?

[http://blog.empathybox.com/post/19574936361/getting-real- abo...](http://blog.empathybox.com/post/19574936361/getting-real-about- distributed-system-reliability)

<comment><USER3> We're still on EC2, though we've learned to distribute servers across availability zones.

</comment></comment><comment><USER4> "Map/reduce operations aren't exactly blazing fast, and are single-threaded (though they will parallelize over shards)." This sounds extremely pointless with it being single-threaded, but whatever.

<comment><USER3> Its single-threaded nature for map/reduce is its biggest liability in my opintion. It will apparently be fixed in the next version (<https://jira.mongodb.org/browse/SERVER-4258>)

Right now, large map/reduce operations just bring the thing to its knees. So we just don't map/reduce with it. The query features are really quite nice, and we do the rest in hypertable.

</comment></comment><comment><USER5> This is very offtopic, but what is the deal with the header pictures in this blog? It's just random photos of office couches.

<comment><USER3> They're pictures of our office in Durham, NC.</comment></comment>


This is the exploit equivalent of that guy who played the perfect game (2008) - <AUTHOR> http://www.matasano.com/log/1032/this-new-vulnerability-dowds-inhuman-flash-exploit/

<comment><USER2> Someone makes an argument against checking malloc, talking about the overhead it adds.

Does anyone have a problem with using a malloc wrapper?

      void *Malloc(size_t size) {
        void *r = malloc(size);

        if(r == NULL)
          /* do things to quit the app */

        return r;
      }

edit: yay for " " code blocks. Hacker News++

edit again: I wasn't clear. I meant overhead in terms of lines of code it added. I'm asking if it makes sense to wrap it up in a nice upper-case call that just terminates the program if the worst happens.

<comment><USER3> From my understanding, his problem was that the malloc wrapper being used not only checked but also returned -1 if it was null (I assume some other arbitrary number otherwise) which would add extra computation time.

Out of curiosity, I have always used assert to do malloc null checks. I realize it is just a macro which gives a conditional to exit but is at all hindering in terms of performance?

eg:

      void *Malloc(size_t size) {

        void *r = malloc(size);
        assert(r != null);

        return r;
      }

<comment><USER4> That's probably the same as the non-assert check EXCEPT that release builds might disable assertions and then you're in trouble.

<comment><USER5> Right. Plus as a matter of style, assertions are traditionally distinguished from error handling. Assertions are used to catch programmer errors; error handling is for when something happens that is nevertheless within the contract of the library (malloc can return NULL, fopen can return NULL, etc).

</comment></comment></comment></comment><comment><USER6> This is a dupe and over a year old

<http://news.ycombinator.com/item?id=164725>

<comment><AUTHOR> Apologies. I'm new to using searchyc and didn't expect it to miss the URL.

[http://searchyc.com/submissions/http%253A%252F%252Fwww.matas...](http://searchyc.com/submissions/http%253A%252F%252Fwww.matasano.com%252Flog%252F1032%252Fthis- new-vulnerability-dowds-inhuman-flash-exploit%252F)

</comment></comment><comment><USER7> Am I on Reddit? Seriously, what's with the headline?</comment>


Ask HN: I'm Looking for Interview Question about Java Around Web Development - <AUTHOR> Hello, currently we are hiring for a Java Developer (middle skill) and looking for "good" interview questsions, however I'm actually a little bit clueless, about what are good questions. We don't want the task last longer than maximum of 15 minutes and it should be around Java and Web development.<p>We will probably start with Fizz Buzz, but after that we wan't to be a little bit more specific to our problems instead of asking algorhitmic questions. I tought about creating a small guestbook with java (framework doesn't matter). would that be a reasonable choice? <comment><USER2> What aptitude are you trying to test for? Do you already have a code base in Java you might use for inspiration? You mention framework doesn't matter, but the difference in what you can accomplish in 15 minutes is significant if you're comparing, say, core Java EE and Grails.

Agree with brudgers that 15 minutes is short. You're going to need a fill in the blanks type of assignment, where they import a project into their favorite IDE and can just add methods to a skeleton class. This means you need to test importing/running in at least Eclipse and IntelliJ, and maybe a regular text editor (Emacs/VIM/Sublime) with a simple build/deploy/test script (Maven or Gradle).

In other words, you have to factor away all the time sucking annoyance associated with spinning up a project you haven't touched before.

This is a hard problem, would be interested in discussing it further with you.

</comment><comment><USER3> Fifteen minutes does not seem like long enough for a professional to set up a development environment or to flesh out the problem for which a guest book app is possibly a reasonable solution.

To put it another way, perhaps a mid-level developer be thinking about parameterizing FizzBuzz instead of quickly hard coding magic numbers and strings.

Good luck.</comment>


Pervasive cross-correlations of various traits' genetics - <AUTHOR> http://drjamesthompson.blogspot.com/2015/11/genetic-story-jumps-ahead.html <comment><USER2> It's not surprise that at least some of these traits are normally distributed in a population AND intercorrelate. Multifactorial traits are the result of many independent genes. You get enough hits of genes on a finite genome describing a trait of certain complexity, and another one of similar complexity, then there's bound to be overlap.</comment>


On the Cruelty of Teaching Computer Science (Dijkstra) [pdf] - <AUTHOR> https://www.cs.utexas.edu/users/EWD/ewd10xx/EWD1036.PDF <comment><USER2> "the subculture of the compulsive programmer, whose ethics prescribe that one silly idea and month of frantic coding should suffice to make him a life-long millionaire"

that was eighties. now we say billionaire</comment>


Thermal images reveal what parts of a screen were tapped - <AUTHOR> https://www.theatlantic.com/technology/archive/2017/03/hot-hands-smartphones/519069/?single_page=true <comment><USER2> Isn't this used for skimming ATMs?

What's new about this?

</comment><comment><USER3> Why haven't lock screens gone with floating/moving numbers yet.

Or at least a pin pad that randomizes number positions. MMOs have had this 'comfort' feature since the late 90s as a antikeylogger feature.

</comment><comment><USER4> LG phones have this optional feature to scramble the digits on the software touchpad so they aren't in the regular order (but a random one). This would be a start I feel; they would need to also actually get a picture of the software keypad instead of just an IR picture your screen however long after

</comment><comment><USER5> Good thing I always carry my phone in the back of my jeans. Hot pocket, FTW.

</comment><comment><USER6> The video they showed seems faked. Why do his other presses and swipes he makes after entering the password not show up in the thermal image? Why would they fake this?

</comment><comment><USER7> Without comment on this specific article, there was a video of this doing the rounds last year using a FLIR ONE on a card terminal. I subsequently saw another video debunking it. It only worked if you super cooled the pin pad, there was evidence of that in the original video; it was totally un-workable with a normal room temperature keypad.

I imagine though it's likely highly variable though depending on the thermal conductivity and emissivity of the surface being touched, as well as the actual room temperature and the quality of the thermal sensor.

EDIT: This random video seems to agree with the above, again, no idea how credible. Highly variable on factors. [https://www.youtube.com/watch?v=8Vc-69M-UWk](https://www.youtube.com/watch?v=8Vc-69M-UWk)

</comment><comment><USER8> useless attack since many phones are already randomizing number pad each time is displayed

</comment><comment><USER9> no shit

</comment><comment><USER10> This is nothing new. I remember an article/paper about the same thing but with pin pads.

Keep in mind, it only works for a very short amount of time as the surface cools down quickly. I don't think it's an attack vector you should worry about.

You can just as easily shoulder-surf the user as he unlocks the phone. It works even better with the "stroke gestures" that is common on android devices - most users don't disable the "draw line" option... it's much easier to spot and remember than a pin pad entry :D

<comment><USER11> Sometimes you can unlock people's phones by holding them up to the light and seeing where the grease marks are, if they use the dot pattern unlock or a small pin. Got a few people with this in high school.

<comment><USER10> I was about to mention this in my post but wanted to keep it short :D. The smudge marks are probably as good of an indicator as the heat signature, considering that they don't degrade with time. It also works better with the pattern-type unlock screen on android (but not so well with the pin-type).

That reminds me, you could build a system that records and highlights the smudges (by using a circular arrangement of LEDs and a fixed camera, or by taking pictures from multiple angles under a single light source). It should make a good weekend project, maybe I'll try it :)

</comment></comment></comment><comment><USER12> This was a common mechanic in the splinter cell games, you'd wait for a guard to go through a door with a keypad and you'd put on your thermal vision goggles to find the 4 numbers. I guess that now that thermal cameras are becoming more ubiquitous this has jumped closer to reality. Back then I can't imagine there were thermal cameras that were small and cheap. Nowadays you can get a flir lens for your phone at about $200 - $300 if I recall correctly.

<comment><USER13> Cat Phones (yes, Caterpillar Tractor) offered the first phone with an IR imager. They intended this for industry and construction, where you're sometimes looking for things that are overheating, or heat leaks. It combines the visual and IR images, so you can see the outlines of what you're looking at. Their customer base has a real use case for this feature.

Like other Cat phones, but unlike the USB-port devices, it's not fragile.[2]

[1] [http://www.catphones.com/en-us/](http://www.catphones.com/en-us/) [2] [https://www.youtube.com/watch?v=mVPku- xItv8](https://www.youtube.com/watch?v=mVPku-xItv8)

<comment><USER14> I used a CAT phone for hiking for a while and at first I was quite impressed by the sturdiness (dropped it on rocks multiple meters a couple of times). But there are some design flaws where the connectors are just covered by plastic plugs that leak water after a while and its not waterproof anymore. Would not buy one again.

</comment></comment></comment><comment><USER15> I recommend that everyone use an extended-length PIN for your phone. Both Android and iPhone support it. Mine is 12 digits; a bit of extra time, but vastly more difficult to brute-force or shoulder-surf.

<comment><USER16> Why digits and not a real password? I have found that typing 10 letters is not too inconvenient on a phone.

<comment><USER17> The size of the alphanumeric keyboard is a real problem for me when trying to enter a password to open my device.

Sure, I can do it, but it really slows me down, and makes the value of the password a lot less to me.

I'd rather use a longer and more complex PIN on a much larger keyboard. Preferably one that re-uses at least some of the numbers, so they might have an idea of how long the PIN is, but they might have a harder time figuring out what the correct order of the numbers is.

At least, that's my current view. That might change tomorrow. ;)

</comment></comment></comment><comment><USER18> Most of this risk is mitigated with finger print auth like Touch ID. Though, nothing is perfect.

<comment><USER19> Fingerprint auth exposes you to court-ordered unlocking though.

<comment><USER20> not necessarily, if you use several sequential fingerprints in a certain order, and have a hard lockout after X failures. My phone knows 6 of my fingerprints already; shouldn't be hard to get that additional level of "something you know" in addition to "something you are".

<comment><USER20> I wonder why someone would downvote this without reply? It's a perfectly reasonable suggestion, would be trivial to implement, and would be both a password and a biometric. That's two-factor authentication.

Somebody's a real humbug.

</comment></comment></comment></comment><comment><USER21> Easy fix. Number pad should come up with numbers in different random order each time.

<comment><USER22> That'll break for people who use "shapes" on the keypad.

<comment><USER23> Imagine numpad in your mind to play shape on to extract sequence

<comment><USER24> I can't say if this is a serious answer or not.. But either way it does hold something potentially valuable. Show a numpad and have a "connect the numbers" passphrase, just like you draw the shame now. And you connect X numbers that you choose yourself

Next you can just shuffle the numbers each time, making the pattern random whilst still using the shapes.

<comment><USER25> I guess s/shame/shape/ is in order, took me a while to parse that. :)

<comment><USER24> Oh yeah, that makes a lot more sense doesn't it! ^^

</comment></comment></comment></comment></comment></comment><comment><USER26> Solution add a feature to warm up the screen to exactly 98 degrees to avoid detection. Learned this from sneakers.

<comment> alex- Another (cheaper?) solution would be to randomise the location of the keys each unlock attempt.

It would cause me a massive problem - I basically only remember my pattern

<comment><USER27> One of my banks does this for online PIN entry and it's amazing how much it slows you down not being able to do it via muscle memory.</comment></comment></comment>


Dropbox: The first dead decacorn - <AUTHOR> http://alexdanco.com/2015/08/24/dropbox-the-first-dead-decacorn/?hn=true

<comment><USER2> Not convinced. Slack users are not the only ones who use Dropbox.</comment>


Unison File Synchronizer - _o-O-o_ https://www.cis.upenn.edu/~bcpierce/unison/ <comment><AUTHOR> From the changelog:

> Further improvements to the OS X GUI (thanks to Alan Schmitt and Craig > Federighi).

Here are Craig's commits:

\- [https://github.com/bcpierce00/unison/commit/48f8e1b27edbe2df...](https://github.com/bcpierce00/unison/commit/48f8e1b27edbe2df0a6da272b6694f0d522c710b)

\- [https://github.com/bcpierce00/unison/commit/6645d1793ce843f6...](https://github.com/bcpierce00/unison/commit/6645d1793ce843f6fe163c7ad68dc1932ad89500)

Around the same time, Dave Abrahams, now of Apple's Swift team, makes an appearance on the unison-hackers mailing list.

<comment><USER2> Wow. I've been using Unison for ages and had no idea Craig had had a hand in it.

</comment></comment><comment><USER3> I've been working on a spiritual successor to Unison for a few years called Mutagen [1]. It aims to provide more flexible synchronization and tighter integration with filesystem watching, enabling ~real-time remote editing with your local editor of choice. It also adds SSH-style network forwarding, so you can forward network traffic to/from remote systems to access remote applications without exposing ports. It currently supports SSH and Docker containers, but additional transports are coming.

I'd be really happy to receive feedback if you have a chance to try it out!

[1] [https://github.com/mutagen-io/mutagen](https://github.com/mutagen- io/mutagen)

<comment><USER4> Haven't dug into it yet, but quick Q: does this allow the use case of using a local IDE to interactively develop/run code against a docker environment running in a remote server? (so two "jumps")

<comment><USER5> That's actually one of the primary use cases at the moment. The way Mutagen works with Docker is by using `docker cp` to inject one of its "agent" binaries into the container and then communicating with that binary over `docker exec`.

You can also orchestrate your setup a bit (e.g. to work with Docker Compose) using Mutagen's new orchestration infrastructure. You can find an example of this at the bottom of the Mutagen homepage: [https://mutagen.io](https://mutagen.io)

</comment></comment></comment><comment><USER6> Is this project still going? I used to use it a lot for duplicating my project directories between my lap PC and my home PC. It was so powerful and so useful at the time. Honestly I thought it had died.

I use Syncthing now. It's really useful.

<comment><USER7> I recently switched from Syncthing to Unison, after getting tired of Syncthing either taking forever to sync anything or getting stuck altogether, with very little visibility or control over what it was doing. So far Unison (with fsmonitor) has 'just worked', with files updating immediately on save as I'd expect.

(That said, one of the problems I encountered with Syncthing seemed to be that it wouldn't properly reconnect when I switched networks, which is necessary since it runs as a persistent daemon. With Unison, so far I've been starting it up in the foreground in a terminal as needed, so it hasn't had to provide the same functionality. But I expect I'll eventually set up some type of auto- reconnect, which will hopefully work better than Syncthing did.)

<comment><USER8> Huh, I've had the opposite experience with Syncthing. It's basically just worked, it's been bulletproof. I'm not sure why the disparity.

<comment><USER9> Syncthing can behave oddly, sometimes it works great and others it just starts looping and doing nothing.

Of course I've not tried it in a while because of those issues so it could be fixed now.

</comment></comment></comment></comment><comment><USER10> I can't praise Unison enough. I've been using it every day for over 10 years and is easily one of the most useful pieces of software I've ever come across. I've donated multiple times over the years.

<comment><USER11> I'm a maintainer of the Unison package in Fedora and I'd love to know which version(s) of Unison you use day to day.

In Fedora (and I think this applies in Debian too) we have to maintain 3 versions because Unison isn't interoperable across minor releases. For this reason we package 2.13, 2.27 and 2.40, and I think there is discussion about packaging the latest release too. Keeping these ancient (esp 2.13) versions going is a pain to say the least.

<comment><USER12> This incompatibility is one of the reasons why I reduced my dependency on it, though I still use it for minor things.

The last straw came when it turns out that unison was incompatible with the _same_ version, if that version had been built on a different system. I can't remember the details but there was some library version difference that results in a unison that has the same version number but wouldn't talk to one from another system without crashing.

Drove me crazy.

<comment><USER13> That can happen because unison uses OCaml's built in serialization, which isn't guaranteed to be stable across OCaml versions. It doesn't often change, but the representation of arrays was changed in OCaml 4.02, which meant unison compiled with 4.01 couldn't sync with unison compiled with 4.02. Debian bug discussing the issue: [https://bugs.debian.org/802919](https://bugs.debian.org/802919)

<comment><USER14> > which isn't guaranteed to be stable across OCaml versions

Oh, so OCaml is a toy language from academics. I've read so many good things about it over the years, but no one mentioned that aspect of it. There should be a law about putting a warning in Big Red Letters on the box so someone doesn't make the mistake of using it in a large, long lived project.

<comment><USER15> > Oh, so OCaml is a toy language from academics.

What? Why did you conclude so?

> but no one mentioned that aspect of it.

Really? Did you open the documentation? It's stated explicitly.

[https://caml.inria.fr/pub/docs/manual- ocaml/libref/Marshal.h...](https://caml.inria.fr/pub/docs/manual- ocaml/libref/Marshal.html)

> There should be a law about putting a warning in Big Red Letters on the box > so someone doesn't make the mistake of using it in a large, long lived > project.

It's documented, and it's not intended to be used for serialization, you use it in very particular cases, for example for faster object sharing with IPC for multicore. We use Json or S-expressions or other stuff for that purposes.

I really don't know why unison people don't switch to a proper serialization.

</comment></comment></comment></comment></comment></comment><comment><USER16> A few years ago, I was working in a company that used a shared drive on a local Windows network. Several analysts and people from other departments would use the drive to store shared price lists, tables, and other documentation. Part of my job was to keep these data files updated with info on a daily basis.

The problem with trying to edit these files was that everyone logged in on the network had access to them and could change them arbitrarily. Or they could even just inadvertently lock the files if they left them open on their PCs. There was no control or change management.

I decided to keep the 'canonical' versions of the files on my PC at work and use Unison to sync them over to the shared network versions of the files once a day. Unison would instantly tell me if anyone other than me had changed the files, and I could investigate further. It was a huge relief knowing that I had proper control over those files.

</comment><comment><USER17> Unison's cool, but I hit some major roadblocks with it, namely: it can't run on an heterogeneous network [0,1]. As many others have already suggested, syncthing[2] is the new best tool out there for multi-directional sync.

[0] [https://wiki.archlinux.org/index.php/Unison#Version_incompat...](https://wiki.archlinux.org/index.php/Unison#Version_incompatibility) , [https://groups.yahoo.com/neo/groups/unison- users/conversatio...](https://groups.yahoo.com/neo/groups/unison- users/conversations/topics/11439)

[1] [https://try.popho.be/byeunison.html](https://try.popho.be/byeunison.html)

[2] [https://syncthing.net](https://syncthing.net)

</comment><comment><USER18> Unison has been an indispensable tool for me for many years. I keep my desktop and laptop in sync by using an always-on Linode as a sync server.

Something I love about Unison is the fact that I can sync files from anywhere on my filesystem without having to move them or replace them with symlinks.

I can also sync subsets of my files by defining multiple profiles. I have Unison set up to default to a "common" profile that includes nearly everything, but I can explicitly request a sub-profile if I want. I also have a separate profile that syncs the profiles themselves, so I can make profile changes on one machine and run `unison profiles` to upload them.

</comment><comment><USER19> Unison efficiently and reliably syncs my website (static blog mostly) between OS X and FreeBSD. It's one of those extremely underrated pieces of software that just quietly does an excellent job, yet most people never heard of it and continue to struggle with rsync or similar (syncthing fulfills a somewhat different role). Perhaps because it's written in an oddball language, or because the website isn't flashy enough.

I often evaluate a wide range of software before choosing what I consider best for a job, and many years ago, Unison came out way ahead in such an evaluation. It never failed me.

</comment><comment><USER18> The Development Status section of the manual mentions a follow-on project called Boomerang¹. It looks like the GitHub repo² only had activity for about 2 years, then stopped. Is that a dead project?

¹[https://www.seas.upenn.edu/~harmony/](https://www.seas.upenn.edu/~harmony/)

²[https://github.com/boomerang-lang/boomerang](https://github.com/boomerang- lang/boomerang)

</comment><comment><USER20> I use Unison every day to synchronize my work computer and my home computer, using an USB thumb drive as the intermediary. The thing I like the most about this arrangement is that I don't need a direct SSH between home and work, that there is no cloud service involved, and that the GUI lets me double check what files are going to be written to/from the usb drive before it synchronizes.

There is one issue that I have run into though. If I format the USB as fat32 it can't properly store permissions. And if I format it as ext4 then I need to make sure that my numeric user ID is exactly the same in both my home and work computers (otherwise the permissions get messed up).

Is there a way to make my arrangement more robust so that I can sync files through and USB drive, without needing to use the same numeric user id in all my computers?

<comment><USER21> > Is there a way to make my arrangement more robust so that I can sync files > through and USB drive, without needing to use the same numeric user id in > all my computers?

$ chown -Rf ufo.users /media/usbdrive && unison

<comment><USER20> Would I need `sudo` for this chown command?

<comment><USER21> Yes, you very likely need to be root. You could also perhaps add in fstab that you mount the USB drive as a specific uid. See man mount and man fstab and search for " uid" (with the space, else you search for uuid as well).

</comment></comment></comment></comment><comment><USER22> I used to use unison to periodically sync files between a host filesystem and a VM filesystem as it was a lot faster - in terms of filesystem performance when not syncing - than using the shared folder functionality provided by the VM.

</comment><comment><USER2> I've been using Unison since... forever.

A hint regarding running the latest Mac binary on Mojave: if the GUI crashes on you, it's likely to be an issue regarding syncing file _permissions_ and not just files.

This has been biting me for a couple of months now whenever I sync my development tree between Dropbox and OneDrive, since Windows' WSL tends to set wonky 0x777 permissions on entire file trees, and it is usually those that cause Unison to crash.

Otherwise, I've been using it to sync 400K+ files without incident.

</comment><comment><USER23> Unison is great for keeping multiple machines in sync. I eventually switched to Dropbox for reasons I don't recall, but it might be a good time to reconsider. Also, I'd forgotten, but apparently I even made a (poor) attempt at writing a Unison UI of OS X, some 11 years ago... [https://github.com/logandk/autoson](https://github.com/logandk/autoson)

</comment><comment> the-peter I used and loved Unison for years, but I found myself wanting something that just worked invisibly with no admin effort so I switched to Resilio (Bittorrent Sync). Both work for my purposes. Anyone else try both of these and have feedback?

</comment><comment><USER24> How difficult would it be to build a Dropbox clone on top of this?

<comment><USER25> Pretty hard. Unison is nice, but...

\- Unison is written in OCaml, which is (probably) a perfectly fine language but not commonly used

\- Unison synchronizes files to files, but for a Dropbox-like system you really want deduplication for space savings (i.e. server-side storage is a bunch of pointers to content-addressed blocks.)

\- in general, it's not clear to me that the client is really the hard part of Dropbox. Note e.g. the part where Dropbox now runs its own data centers for cost reasons.

\- there's a million fiddly things to get right, and Unison hasn't had _that_ much usage

<comment><USER26> > there's a million fiddly things to get right, and Unison hasn't had that > much usage

Unison is the _only_ bidirectional sync tool that I trust to get the details right. It is backed by a formal model with various proofs of correctness. Such models are also easily representated in OCaml; which I can assure you is more than a fine language, especially if one cares about correctness. Dropbox has struggled to get these details correct in the past (see the paper "Mysteries of DropBox", by Unison's author Professor Benjamin Pierce). TLDR, Pierce teaches these Python hackers how to fix their broken code.

<comment><USER27> > _Unison is the only bidirectional sync tool that I trust to get the details > right. It is backed by a formal model with various proofs of correctness._

That's just about the sync process/stages, the easy part that can actually be formalized.

The "million fiddly things" are about OS and filesystem issues, incompatibilities, and so on, and Dropbox has a hugely larger test base for those things...

<comment><USER26> I disagree. POSIX, although somewhat dated, has provided a good enough abstraction layer for filesystems and OSs. My proof of this is the number of different and successful filesystems for Unix/Linux. If the abstraction didn't work, everyone would be forced to use the same filesystem.

The issues and subtleties are with bidirectional sync. It is not "the easy part". Dropbox didn't get it right in the past, we have no proof they have it right now.

<comment><USER27> > _POSIX, although somewhat dated, has provided a good enough abstraction > layer for filesystems and OSs._

First of all, POSIX semantics are not what Windows support. Second, even where available, POSIX is a tiny part of the possible issues. Adequate for naive apps that need to open or write some files, not for a reliable sync tool.

[https://danluu.com/deconstruct-files/](https://danluu.com/deconstruct-files/)

<comment><USER26> The issues discussed in that link, such as concurrency, lack of atomicity and failure are all factors that make bi-directional sync a hard problem. Yes POSIX provides few guarantees and so that burden rests on the application code above. The formal model must consider the race conditions and failure modes, it is not "the easy part". I do acknowledge there is a modelling gap to overcome and some problems with POSIX, but I disagree that its the hardest part of the problem.

</comment></comment></comment></comment></comment></comment></comment><comment><USER28> What a fantastic tool!

A long time ago used unison to keep file uploads on 2x primary web servers[1] in sync. It worked like a charm. With IIS on Windows!

</comment><comment><USER29> How does Unision compare to Rsync, is one a part of the other or both are different tools for syncing files?

</comment><comment><USER30> Daily user since beginning of this year. Help keeps my laptop in sync with my remote dev environment.

</comment><comment><USER31> Not to be confused with Panic's Usenet/NNTP client:

* [https://en.wikipedia.org/wiki/Unison_(Usenet_client)](https://en.wikipedia.org/wiki/Unison_\(Usenet_client\))

<comment><USER32> Nor with the Coda replicating filesystem, which is not to be confused with Panic's code editor of the same name.

<comment><USER33> Nor with a certain super cool language of the future [https://github.com/unisonweb/unison](https://github.com/unisonweb/unison)

</comment></comment></comment><comment><USER34> Unison does not support hard links!</comment>