Syncer: fast stateful file/disk data syncer written on Go - <AUTHOR> https://github.com/stargrave/syncer <comment><USER2> I assume the title has a typo and that it's supposed to say "written _in_ Go".

With that said, as a user of a tool/library/service, why should I care what language it's written in?

I want to know if a data-syncing tool is reliable or not. I know rsync is, but OK that's not always super-duper fast.

I know volume-streaming with btrfs and ZFS works wonders, without the need to do manual book-keeping of a secondary state and hoping you ALWAYS keep that secondary state up to date. I doubt this out-performs the FS just doing the job itself, and it sounds much more prone to user error.

So what does this offer over those? It sounds like it has less features and is not as reliable. Being written in Go offers me nothing I don't already have.

<comment><AUTHOR> I answered in another comment on that question. The main point of course is not a language. ZFS incremental snapshots and its send/receive are great, but my task is to create binary identical image of hard drive with bootloaders, partitions and so on. I have got SSD that can breake and slow USB connected HDD. I want to sync them several times a day and be able to trivially switch them in the case of failure. ZFS is only a filesystem. I need to prepare harddrive with bootloaders to use ZFS send/receive. I do not want it. Raw dd is what is needed, but it is slow -- the only reason this utility is written.

<comment><USER2> > ZFS incremental snapshots and its send/receive are great, but my task is to > create binary identical image of hard drive with bootloaders, partitions and > so on.

If you use UEFI, bootloader is just a normal file on a normal (FAT) partition so no special magic is needed, but fair enough.

ZFS typically handles partitions its own way so no need to sync that.

But really: If you just want to keep to full drives in sync, at block-level so one can at any point be a stand-in replacement for the other, it sounds like what you want is RAID0. Isn't that what you want?

And with UEFI you can boot those RAID-volumes, even though they are soft-raid volumes.

Not saying your need doesn't exist, but that existing solutions which are more standardized and requires less administration seemingly already exists.

<comment><USER3> I believe if you raid0 an HDD and an SSD you lose fast writes because the OS waits for it to be written to all disks in the array.

Maybe depends on the implementation, I I'd imagine it would degrade performance.

</comment></comment></comment></comment><comment><USER4> It's late and maybe I missed something looking through the code, but what happens if you don't have write permissions on the directory that the statefile is in? The copy operation is performed but then the state is lost when the statefile rename fails.

<comment><AUTHOR> At first, temporary file created in current directory. So if directory is not writable -- temporary file creation at the very beginning will fail.

<comment><USER4> Ah, that's what I missed. I guess there's still a possibility of permissions changing during the transfer, but I don't think there's anything you'd want to do to handle that other than error out.

<comment><AUTHOR> In that case I will print the message telling that something went wrong but temporary statefile is alive: [https://github.com/stargrave/syncer/blob/master/syncer.go#L2...](https://github.com/stargrave/syncer/blob/master/syncer.go#L205)

</comment></comment></comment></comment><comment><USER5> Suggestion: Don't set a default for dst. That looks really dangerous.

<comment><AUTHOR> Agreed. Fixed. Thanks for the suggestion.

</comment></comment><comment><USER6> A former colleague has written something quite similar[0] few years ago, which we used to backup encrypted disks (actually snapshots of encrypted disks to be precise).

[0] [http://chunksync.florz.de/](http://chunksync.florz.de/)

</comment><comment><USER7> Nice, but in which use cases it may be useful?

<comment><AUTHOR> I have got fast SSD that can break down every day. I have got USB hard drive. I want to sync data every day, twice a day between them as fast as possible (raw dd takes an hour). If SSD is broken then I just plugin USB hard drive and continue working (with some minor data lost).

ZFS has incremental snapshots, it is fast, but I need to restore bootloader, partitions and everything corresponding to be able to bootup and work again.

<comment><USER8> Scratching your own itch is the correct path to success, in my books. Keep up the good work.

</comment></comment></comment><comment><USER9> this is great. I wonder how it would perform when I need to sync 10mil. files. Rsync and anything else breaks down at this point, well it does work but it takes forever and rsync bloats to 10-20GB memory foot print and after a few hours starts streaming updates.

<comment><AUTHOR> syncer works on block level, with raw byte sequences. It knows nothing about file systems. Everything is limited with sequential read/write speeds of your hard drives.

It takes blocksize-size memory block for each CPU in your system. I have got 4 CPUs and work with 2 MiB blocks: program will take 8 MiB of RAM.</comment></comment>


Why Pattern Discovery Matters - <AUTHOR> https://www.logsense.com/blog/why-pattern-discovery-matters <comment><USER2> [https://github.com/logpai/logparser](https://github.com/logpai/logparser) has a nice collection of research on this topic, in case anyone is interested.</comment>


What just happened to Hacker News? - <AUTHOR>

What just happened to Hacker News? <comment><USER2> We're getting a working demo of the _one_ type of political news story that _all_ (well, most) HN readers can get worked up about.

(As a friend of mine once said, "the big political question of the 21st century will be how to deal with too much data." And this is your baseline case of the issue going political this decade.)</comment>


Finnish Experts Analyze Deep Impact of Nokia Decline - <AUTHOR> http://www2.hs.fi/english/archive/news.asp?id=20020830IE5

<comment><USER2> This analysis is from 2002. Today Nokia is ~1% of the GDP, not very significant.</comment>


10% of a population is the tipping point for intellectual disruption - <AUTHOR> http://news.rpi.edu/update.do?artcenterkey=2902&setappvar=page(1)

<comment><USER3> This seems more interesting than it actually is. In effect, it seems like the authors are saying, "exponential growth is like real fast!"

The model seems very one-sided. I can't access the original paper, but it seems they have 4 states for each node:

1a) Committed to opinion A

1b) Has opinion A

2) Has opinion B

3) Has both opinion A and B

In this model, there is no "committed to opinion B" -- which makes this useless as a model for things like climate change, evolution, or civil rights where both sides have a committed population. Since this is the case, opinion A can only grow, and at some point will hit a threshold that takes over the whole population.

I assume they built the model this way because if it were balanced, you would likely get random outputs, depending on how the graph is arranged. And random outputs don't make good headlines. :)

<comment><USER4> Your state 3 should be "Has opinion B but has been exposed to a node in state 1". In that state, the next exposure to different node in state 1 would change the node to state 1b. True believers win by being immune to contrary opinions.

I doubt the researchers were motivated by headlines. Rather, they put together a simplistic model and got a simplistic result that is surprising only for the consistency of the tipping point across their chosen variations of the model. For example, other (field) research shows that the common reaction to opinions or facts that contradict one's opinions is to argue, which tends to cement one's opinions further. This doesn't necessarily imply one starts out committed. It also ignores confirmation bias, which limits exposure to contrary opinions. Etc.

As others point out, if their model was accurate, there could be no long-term stalemates, like progressive vs. conservative, both of which have significantly more than 10% true believers. Nonetheless, it is suggestive, particularly as it describes relatively isolated communities and as it implicitly distinguishes between belief systems which insist on absolute commitment (e.g., Evangelicals, Tea Party) vs. those which are satisfied with mere agreement and tolerate dissent (e.g., Ecumenicalists, Independents).

</comment></comment><comment><USER5> > "When the number of committed opinion holders is below 10 percent, there is > no visible progress in the spread of ideas. It would literally take the > amount of time comparable to the age of the universe for this size group to > reach the majority," said SCNARC Director Boleslaw Szymanski, the Claire and > Roland Schmitt Distinguished Professor at Rensselaer. "Once that number > grows above 10 percent, the idea spreads like flame."

So how does a small group reach 10%? This description of their model appears to claim that it is impossible for the beliefs of the majority to change (because _every_ group begins with below 10% adoption of their ideas).

<comment><USER6> This appears to be Yet Another "we threw some code together and published the results without feeling obligated to check our model against the real world" study. At least if they did any sort of real-word verification, the article doesn't mention it.

The assumptions strike me as unreasonable, the results worthless because the details matter a _lot_. Another prime candidate for "we tuned our unverified model until it did something publishable", too.

<comment><USER7> That passes publishable, it doesn't have to be final verdict on the subject. It should be taken as is. They mentioned observing the revolutions which happened really fast and would fit in the model.

<comment><USER6> Yes, the low bar of publishable is part of my point. I think I failed to communicate my main point, which is that you can make these iterative models do _any damn thing you please_. It's a fun sort of entertainment for programmers, kind of like those "evolve a cart that runs over random terrain" things that pop up every so often... but without proving a correlation to the real world, it's useless. I can make very small tweaks to their model and make the threshold 20%. I can make very small tweaks to their model and make the threshold 5%. How do you judge which of these three results is correct? Or how do you judge if a single numeric threshold is even meaningful in the real world?

No, seriously, don't just blip over that. How do you verify this result? How do you tell which is correct? There is no evidence in the link that anybody has asked that. Maybe they did, but it's not a negligible detail, it's very important!

Here, run one of them in your browser: <http://www.openprocessing.org/visuals/?visualID=6599> Have fun yourself seeing how very small tweaks, well below the real-world noise threshold, can have profound effects, the source is right there.

It's not an interesting result, without some sort of verification against the world, and cherry picking a few current events and making vague allusions to how it sort of acts like a system you threw together isn't verification in the slightest.

<comment><USER8> This may be a useless post since it just reiterates, but I feel strongly that I should re-emphasize that yes, _very small tweaks_ can make the threshold 20% or 5%, so the number is essentially pulled out of a hat.

<comment><USER9> Finance types refer to this as a "sensitive" model -- it's not robust in the face of small changes to parameters.

A good way to explore these problems is to perform monte carlo simulations over the set of possible parameters and then derive general behaviour from that -- looking in particular for 'catastrophic' features in the phases space.

Later you can narrow your data set down based on empirically observed parameters.

As an undergraduate I wrote a standard grid-based model of epidemics which does this. So it's not as though it's rocket science.

</comment></comment></comment></comment></comment></comment><comment><USER10> I think it matters WHO the 10% are.

Its been shown in various experiments that a significant percentage of the people go along with the group even when the group is obviously wrong. I think it was at least 60%.

So that leaves 40% or less that are capable of adopting something new. You'd have to get half of them to sway the followers. That's 20% or less, which 10% certainly is.

But if your 10% are in some kind of subculture and most are followers, they aren't going to be influencers of the larger society.

If the 10% are all in the media and politics, I think the change is likelier.

It also depends on the nature of the change. A change that requires significant inconvenience will be harder than deciding Pluto is no longer a planet.

<comment><USER11> This all sounds very likely to me. But is there a source for this information?

<comment><USER10> There's the Asch conformity experiments, where people went along with the group with no overt pressure and with nothing at stake:

      http://psychology.about.com/od/classicpsychologystudies/p/conformity.htm

The Milgram experiments on obedience to authority are also relevant. 65% of subjects went along with what amounted to torture of another person:

      http://en.wikipedia.org/wiki/Milgram_experiment

While these experiments weren't about the spread of ideas, conformity and obedience are certainly at work for the vast majority that simply repeat what they hear. You can have a perfectly sound, logical case, even with physical evidence and many will not trust their own judgement if its contradicted by an authority.

But it sort of has to work this way. There have to be followers if there are to be leaders and leaders are necessary if groups are to compete with each other, which was the case even before modern humans showed up.

</comment></comment></comment><comment><USER12> I think the population of atheists (and near atheists) is at or above 10% in the US. Judging from my perspective in North Carolina, the population of devout Christians is at least 10%.

I wonder what this model would say about ideas that oppose each other.

</comment><comment><USER13> This is based on a computer model/simulation. One of the assumptions in the model was that if a person talks to 2 people who hold a different opinion than him/her, they will then change their opinion. I am not sure how realistic that assumption is. Do you change your opinion or belief if just 2 people disagree with you?

<comment><USER14> Counterexample to their assumption: religions.

<comment><USER8> any reasonable model would consider atheism as a religion when it comes to the mechanics of its spread.

<comment><USER15> I have no idea what that is supposed to mean. Religion spreads when people tell other people about the religion. You can't believe in Jesus (or whomever) until they tell you who he is.

Atheism, on the contrary, simply means "the lack of theistic belief". Anyone who has not subscribed to a religion is technically atheistic. I'm not sure how many "atheist missionaries" you've met, but do you really think that atheism spreads by people convincing other people?

All babies are atheists, by the way.

<comment><USER16> And your post is founded on scientific fact not belief?

</comment></comment></comment></comment></comment><comment><USER17> Isn't the "tipping point" in the law of diffusion of innovation (<http://en.wikipedia.org/wiki/Diffusion_of_innovations>) somewhere between 15-18% market penetration?

</comment><comment><USER13> It seems like the RPI.edu server hosting the original article is down - <USER2> probably to the link appearing on Hacker News..

<comment><USER18> Seems to be the same article if not similar: Minority rules: Scientists discover tipping point for the spread of ideas [http://www.physorg.com/news/2011-07-minority-scientists- idea...](http://www.physorg.com/news/2011-07-minority-scientists-ideas.html)

</comment></comment><comment><AUTHOR> These slides might shed some further light on the research models and application domains: [http://www.rpi.edu/dept/phys/sicsin2011/NetSci2011_Sameet.pd...](http://www.rpi.edu/dept/phys/sicsin2011/NetSci2011_Sameet.pdf)

</comment><comment><USER19> So what happens if there are multiple groups, each larger than 10% and all holding mutually exclusive beliefs? Say for instant, 40% of the population of a country is Catholic and 45% is Protestant? Civil war, I guess?

<comment><USER7> I think the new opinion has to come from 'in-group'. I mean, it would be needed that suddenly 10% of the catholic population thinks that 'Hey, protestants are better'. Then, they might influence their friends to change.</comment></comment>


New ransomware is targeting Windows and Linux PCs with a 'unique' attack - <AUTHOR> https://www.zdnet.com/article/this-new-ransomware-is-targeting-windows-and-linux-pcs-with-a-unique-attack/ <comment><USER2> If Linux ever got even 10% of the desktop/laptop market, it would be effective to target it, and it would be extremely vulnerable.

I've had Linux servers hacked because I was a few days late at patching my WordPress....

<comment><USER3> oh! I thought Linux had some inherent / in-built architectural feature that made viruses impossible .

</comment></comment><comment><USER3> >It's an unusual form of ransomware because it's written in Java, deployed as a trojanised Java Runtime Environment and is compiled in a Java image file (Jimage) to hide the malicious intentions.

are there any options in browser settings / adblock extensions to block this thing ?

</comment><comment><USER4> Can someone eli5 what this means for regular desktop users of linux distros?</comment>


Why I'm Not Working on my Startup (yet) - <AUTHOR> http://foohack.com/2008/08/why-im-not-working-on-my-startup-yet/

<comment><USER2> Having a bit of a runway is good, and real-world job experience can pay dividends. Some suggest that you'd get the same out of a failed startup, but both routes have their advantages.

One thing I learned, sadly later than I wish I had -- before starting a startup practice living cheap. Start really considering if you really need that iPhone or that new monitor, buy nothing but generic stuff, have one beer at the bar instead of three, get a house-mate. Cook!

As I was building towards going full-time on the startup I managed to get my cost of living down to less than half of what it was, with no major lifestyle changes. That'll give you a nice pad and the skills to stretch it when you do go full-time.

You can also spend the time learning; nothing teaches like experience, but you can spend time building up your contacts, researching your field, learning as much as possible about startups, setting up infrastructure, searching out good co-founders -- so that when you're ready to run you can go at it full speed.

<comment><USER3> Living cheap also has benefit in that it often forces you to assess what you need and want in life and ultimately what is really important to you.

There is a certain pleasure to be had to free oneself of the materialistic lifestyle we are often indoctrinated into following through nurture, especially in many western countries/cultures (e.g. process of society and education often teaches get a job, buy a house, get a car, get better job, bigger house, newer car etc. much of which is not really necessary).

When you earn enough not to worry about food and bills, you can really begin to do the things you want and learning to live cheap certainly helps.

The freedom is liberating and truly the best things in life are free (or often surprisingly cheap).

</comment></comment><comment><USER4> Most assumptions you have can only be verified by you.

My advice is: try doing something on the side first, something that people can actually use, share it with friend and see how you feel about feedbacks. I found for myself that the greatest motivator is your will to build something useful; your major test will be how you bounce back from negatives.

How many hours you put in, when do you work even whether you should leave your job or not will depend of your level of motivation, it's hard to build a startup without sacrifices.

<comment><AUTHOR> _My advice is: try doing something on the side first, something that people can actually use, share it with friend and see how you feel about feedbacks._

Well, what I've found is that most of my side projects don't get very far, because I invariably end up pulling a bunch of long nights at my main paying gig, and that saps the motivation considerably.

Also, maybe I need ruder friends, but I've found much better results sharing stuff with strangers. They're more honest. Friends are biased by their desire to make you feel good.

<comment><USER5> Post it HN. I dont think HN readers are biased(even if some of them are your friends) :)

<comment><AUTHOR> I'm not so sure. The posters may not be biased, but that doesn't mean that the comments aren't. I've seen some ideas that were just not all that great get lavished with praise.

It's possible that I just missed what was so useful about the site in question... but it's also possible that those who didn't like the site didn't comment. I didn't like the site, or indeed care about it at all, and I'm not the type to want to hurt someone's feelings, so I didn't comment on it.

Friends or not, I think the best feedback is likely from users who have the problem you're trying to solve. The best question is probably "why _wouldn't_ you use this thing?"

<comment><USER6> Why not build and solve a problem _you_ have?

<comment><AUTHOR> Actually, I have two side projects that are exactly that. But I keep ignoring them. That's the problem :) My user is getting upset with me, and about to look to my competitors.

<comment><USER6> Perhaps the problem isn't painful enough?

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> This is _exactly_ what I planned when I left school. Cool! My plan was to work for 5 years at a job I at least LIKED- put away enough to live for 1-2 years without doing anything at all- and then work on whatever startup idea I figured would be best at the time. Why did I choose 5 years? Honestly, it was because most companies vest employees in their retirement plan of choice after 5 years.

Fast forward to 5 years later (last June 15th) and I'm ready! Money saved, credit card balances at 0, no debt, no obligations! I've even started to work on some of the code for my idea. The only thing I didn't count on was the housing market... so now I just have to hang around my current job until my place sells. anybody want to buy a condo in Chicago?

Seriously though- from where I'm sitting, it's a great plan. Give yourself the opportunity and ability to do something amazing, and chances are you will! Not to mention, if you are at least managing your money successfully, and not going in to debt- you're doing vastly better than many Americans already.

Good Luck!! :cross posted from comments:

</comment><comment><USER8> I feel that I've learned an incredible amount since I left my job and started my start-up - and I think that's largely an indication of how little I actually knew when I jumped in.

If I can offer anything for consideration, it's this:

When do you want to make your mistakes? Now, or in a few years? While working for a large company, or while living off your savings? While doing stuff "on the side", or while doing it "for real"?

I am not advocating one choice over the other, but this might be another useful angle to approach the problem.

<comment><AUTHOR> Thanks, that's actually helpful.

Personally, I don't see "living off my savings" as a bad time to make mistakes. Webdevs are usually in short supply, and we're past the first speculative bubble, so "get a job" will likely always be an option. I can "retire" every 5 years :)

</comment></comment><comment><USER9> There's more than one way to do it, and this is a more deliberate path. good plan, seriously. I quit my job early this year, have enough savings to last for a while (it's easy when you live like a grad student), and am exploring ideas.

</comment><comment><USER10> It's clear from your post and comments that you're a very smart guy. I can tell you think so too, but I'd bet you're even smarter (relatively) than you realize. I agree that if you follow your plan, you're likely to be successful, if you don't give up just a bit too soon and go back to the solid, dependable paychecks.

My only two cents concerns your patience. I'll admit, my only experience working for someone else was a year-long internship right after college, but despite it being a sweet gig in a beautiful place, I hated it, and it was all I could do to make it to the end of the year. Since then, it's been only startups for me, and it's so incomparably different and better that I just have to weigh in that your 3 year plan is a little too patient and risk-averse for me.

It sounds like you love your job already. That's definitely key. But if you wait another 3 years to go for it, and find out you're fantastic at it and are having the time of your life, won't it seem like you should have given it a go even sooner? For instance, by getting a cofounder willing to pay the company bills for a while as an even trade for your fantastic ideas? Seems worth considering if you'd both rather start sooner than later. Making your own stuff full time is so _fun_.

</comment><comment><USER11> Bootstrap your startup to beta while still at Yahoo. Sure you'll have to put up with not working on what you love 100% of the time but it'll be better than 0% and probably close to 100% seeing how you seem to like Yahoo o.k.

I am in a situation very similar to yours - good employer, o.k. hours and an overdue startup. I am doing this right now. It's long days and busy weekends and try as I may, a couple of days a week I don't get any sleep. But there is both satisfaction and a reduction in risk in doing this for me. For me, it's not the financial risk that matters as much but I just wasn't sure I would be able to maintain my focus and enthusiasm doing just a startup - amongst other reasons because my cofounders are non-technical.

On the other hand the fact that my cofounders have funded the startup, pay me to do this (on top of my salary at bigco) and handle all the boring non- technical tasks while still letting me be an equal part of strategy and fundraising - all these make my situation a bit different from yours.

</comment><comment><USER12> I encourage you to work on one or more of your ideas sooner rather than later, even while working full-time.

You seem to be at the stage in your life where you could pursue the YCombinator program (or similar programs). This is one way to fund your ideas without going the self-funded route.

You seem to have the skills that you'll always be able to get jobs at Yahoo or elsewhere. You don't really have much to lose.

<comment><USER13> Actually, my reading was he (like me) was well _past_ the stage of life where YC funding would be helpful. Someone correct me if I'm wrong, but a first round YC bet is something like "cost of living for two college students for a summer". To an established professional with a stable job, this is noise: most of us could fund such a company for three months right out of our bank accounts.

The benefit to doing the YC thing is the exposure to sources for later funding rounds, and the networking that comes with being part of the community. The money isn't the draw for anyone but a brand new graduate. It's just there to make sure no one starves while working on the project.

<comment><AUTHOR> If the choices are: 1) get funding, and do it now, and give up even 1% of my company, or 2) wait a year or two, and do it on my own dime, but own it 100%, I'm leaning towards #2.

<comment><USER14> I don't necessarily want to own 100% of my own company, but I want to own at least 51%. I think that Y-Combinator leads to individual ownership that is far less than 51%. Even the founders together will have less than 51%. Am I wrong?

<comment> cypress-hill are the lot of you fourteen years old? or just that old mentally? you're sitting around talking about how you will divy up your little hypothetical companies...did you read richie rich comics as a child?

1\. get an idea someone gives a shit about. 2. find someone to put money behind your idea. 3. then you can get your rocks off playing god with the stock options. without 1,2...talking about 3 is just mental masturbation

<comment><AUTHOR> Hi, troll.

<comment> cypress-hill oh i'm sorry, do you own this thread?

</comment></comment></comment></comment></comment></comment></comment><comment><AUTHOR> I've been absolutely floored by the quality and quantity of comments and advice from all of you, here and on my blog. I've reworked my plans and expectations a tad, and gained a lot more confidence about some parts that I was shaky on. Thanks, HN!

</comment><comment><USER15> This is the hacker demographic I will be targeting should I become wealthy enough to finance my own YC-inspired investment firm.

</comment><comment> cypress-hill so sad. his whole life has boiled down to fetishisizing work

how about this for a plan - get married and have kids. you'll have a regular sex partner, and when you are sixty, you won't sit around wondering why you are alone

and if you save some money, take a winter off and be a ski bum. one year cruising between whistler, aspen, half the crazy ass monster resorts in the alps...beats the hell out of pissing into the wind in the website game

<comment><AUTHOR> Hi, troll.

<comment> cypress-hill keep swinging. sorry not everyone is in the mutual support echo chamber.</comment></comment></comment>


Module that casts photorealistic shadows - <AUTHOR> http://indamix.github.io/real-shadow/

<comment><USER2> "Photorealistic" is a big, big stretch here. Why is the shadow the colour of the object that is shadowing it? Why doesn't the softness of the shadow relate to the distance to the light source? Why don't adjacent objects affect each other shadow colours? Also, the inset shadow doesn't seem to work - the cursor can hover directly over shadow area.

It mostly seems to be a css shape-repeater to give a 'shadow-like' visual appearance in a dynamic direction.

I mean, I'm not trying to put down the writers js/css skills, but when I read "Photorealistic shadows", I think of something like LuxRender. So, probably, just marketing skills.

</comment><comment><USER3> Well I didn't _notice_ any performance issues, but based on the other comment I checked out perfmon, and yeah, its pegging a core.

Maybe less sprites on the demo page?

<comment><USER4> It's laggy as hell for me on a i5-3427U. Not a beefy machine, but unless your target audience is gamers with tricked out PCs...

<comment><USER5> its incredibly slow on my mbpr with chrome 27.

<comment><USER6> Runs just fine here on a regular mid-2012 MBP15. I can see one thread/core is working full time, but the animation is buttery smooth.

<comment><USER7> Yep! Macs rule now :)

</comment></comment></comment></comment></comment><comment><USER8> This is a cool idea :-) You should make a demo with just one object though, so it doesn't lag so much.

</comment><comment><USER9> I bet if you `s/box-shadow/text-shadow/` a lot more people will find ways to use it.

</comment><comment><USER10> Perfect for making me click the back button when your landing page performs abysmally slow.</comment>


A visual chronicle of Tokyo's disappearing jazz bars (2016) - <AUTHOR> https://thevinylfactory.com/features/tokyo-jazz-joints-visual-chronicle/ <comment><USER2> I was a fan of Tokyo (and Kyoto and Osaka's) "Old man bars" after stumbling into a few on my first few visits there. Having discovered Tokyo Jazz Joints on Instagram before my last visit, I used some of their writeups for inspiration on where to check out this time around.

While I have been both shouted and clapped out of a Japanese bar and told a bar was closed when it was clearly open (both immediately upon walking in, presumably because of my gaijin appearance), I've never had a negative experience in a jazz bar in Japan. If it's your intent to quietly enjoy a drink and maybe some salty bar snacks, you will be treated with a low-key hospitality I'd describe as 'genteel' - it's now a quality I look for in bars here in America because it is totally my speed.

<comment><USER3> I liked the concept of "old man bars" \- tough a lot of UK pubs would fit that description, but not really a "spoons" pub

<comment><USER4> Oh, spoons. Where the whole family can come in to hang out with the town drunks all day.

Experienced this outside of York, more country side.

<comment><USER5> _> to hang out with the town drunks all day_

I hate spoons. It's the worst of the corporate world (race to the bottom, conformity, lies, undercutting family businesses) sold as "what the working- classes really want". It's the McDonalds of alcohol and tabloid-culture, with a Scrooge-like owner to match.

</comment></comment></comment></comment><comment><USER6> The bars may be disappearing but Jazz itself is alive and well in Japan. I heard it _everywhere_ in Kyoto & Tokyo. I'm not sure why this is but it's a lot more pleasant than the top 40 stuff you hear playing everywhere in the US.

<comment><USER7> If you're into fusion jazz at all, two of the best bands of all time hail from Japan: Casiopea and T-square. Their influence is felt deeply in lots of 80s/90s Japanese PC and Video game soundtracks. The keyboardist for Casiopea also composed all of the station jingles for the subway in Japan, which all together make one song.

\- [https://youtu.be/S0Xm1PWb07o](https://youtu.be/S0Xm1PWb07o)

\- [https://youtu.be/4DCWEJzsrJo](https://youtu.be/4DCWEJzsrJo)

\- [https://youtu.be/wBD1_mrRUIk](https://youtu.be/wBD1_mrRUIk)

<comment><USER8> > The keyboardist for Casiopea also composed all of the station jingles for > the subway in Japan

This is amazing! I had no idea this was the same person. I love Casiopea.

<comment><USER7> Yeah, check it out!

[https://youtu.be/nSG5IkRA9BE](https://youtu.be/nSG5IkRA9BE)

<comment><USER9> Whoa! This is awesome. Made me Google again (hadn't since my first trip to Japan) for any melodies I could find as a download to use as a ringtone/text tone (not that my phone is ever off vibrate mode).

Thanks!

</comment></comment></comment></comment></comment><comment><USER10> Reminds me of stories from Haruki Murakami, which were laden with jazz references. He apparently worked at a jazz bar before writing his novels.

<comment><USER11> More than that, he owned a coffee shop/jazz bar.

<comment><USER12> Called _Peter Cat_ and it was in Kokubunji (western Tokyo).

</comment></comment></comment><comment><USER13> I could get lost in this world.

I lived in Japan for a couple of years. A Japanese friend treated me to a Jazz Bar a couple of times. It was long ago... I only remember requesting a Billie Holiday song, Stars Fell On Alabama, I think. It was the perfect antidote for the relentless energy of Tokyo's public life.

</comment><comment> enahs-sf Swing in Shibuya is a real treat. I stayed in an Airbnb in the same building and found it by accident. I sat in there all day ripping cigs, drinking highballs and listening to old school bangers. The guy who owns it, Suzuki-San tends the bar and picks the tracks. He also plays trombone in a jazz band. I probably shaved 3 years off my life in the course of three hours but I'd stop in here anytime I'm in Tokyo.

</comment><comment><USER14> If you visit Tokyo / Yokohama, I can recommend Dolphy for some live Jazz. [https://dolphy-jazzspot.com/](https://dolphy-jazzspot.com/)

<comment><USER15> Smack in the middle of Nogecho, one of the coolest nightlife areas I've ever seen.

</comment></comment><comment><USER16> As a person who just returned from Tokyo and sometimes plays with traditional jazz bands, I'm going to say Tokyo has a great atmosphere and thirst for jazz but rarely did I ever hear anything comparable to New York or New Orleans. It's not lack of talent, Japanese musicians are incredibly talented. I think it's more a lack of exposure to jazz pre-1950 or outside of the lounge genre context. It's even hard for some professional American musicians to get because of trad jazz's distinct feel.

Throw on Basie's "Jive at Five" or Ory's "Savoy Blues" and you can get an idea of the genres (swing / nola jazz).

<comment><USER17> Japan's exposure to jazz extends well beyond the "lounge" setting.

It is home to some incredibly important participants in the free and avant- garde jazz scenes.

</comment></comment><comment><USER18> I would love to visit some of these if travel ever becomes possible again... Although, you know it always seems a little intimidating to wander as a stranger into a very specialized, private bar or restaurant in Japan, and if not speaking Japanese (but that's a different topic).

I came across an episode of "Japanology" on Youtube about cafes (which is a surprisingly interesting series produced by NHK -- each episode has a generally heart-warming or touching twist to the general theme it covers). And the ending is exactly about a jazz cafe that closed but was brought back for 2 weeks by its loyal fans some years later. I imagine some of the ones described here are in the same boat.

[https://youtu.be/UF0dnPwqXqY?t=1326](https://youtu.be/UF0dnPwqXqY?t=1326)

<comment><USER19> You miss so much of Japan w/o knowing Japanese. Good thing that it's pretty easy to pick up and the JLPT N5 - N1 is really a great standard to compare against and see where you land.

I lived in Tokyo for a year and passed JLPT N3 with a breeze.

Bars are private and intimate, if you know how to speak Japanese, a world really opens in front of you and you easily make friends. Especially if you enjoy Japanese music.

Try it! Especially w/ some of your favorite Japanese songs. You'll be surprised how quickly you'll pick it up!

<comment><USER20> >Good thing that it's pretty easy

I think you're underselling the difficulty a bit considering that Japanese is in the hardest class of languages as ranked by FSI [1]. Granted this probably includes literacy as well; if you only want to learn the spoken aspects it's probably not much harder than the category III languages.

[1] [https://www.state.gov/foreign-language- training/](https://www.state.gov/foreign-language-training/)

<comment><USER5> The spoken aspect is really easy, by European standards, but you need a ton of practice to re-learn the entire vocabulary from scratch - no hints, no guesses. That requires _a lot_ of patient effort, for our modern attention- deficit-by-default sensibilites.

</comment></comment></comment></comment><comment><USER21> If you know Golden Guy (Burning Man camp), this seems straight up the kind of thing that should be there. (I already posted it to the Golden Guy FB group).

And yea it's a takeoff on Golden Gai.</comment>


Transparent ponzi scheme ICO raises $120k in 5 hours - <AUTHOR> http://ponzicoin.co/home.html <comment><AUTHOR> His Ether wallet shows $130k already and rising: [https://etherscan.io/address/0xe3f64dc522a66405c51d96aae2342...](https://etherscan.io/address/0xe3f64dc522a66405c51d96aae234217a03502bb4)

</comment><comment><USER2> Hilarious.</comment>


Ask HN: Cannot find a website where you could hire sysadmins short term - <AUTHOR>

I recently saw a cool website on HN where you could hire sysadmins for small bits of work. HN Search is failing me. I can't seem to find it anymore. Can anyone remember what it was? <comment><USER2> Today I wrote about something similar:

[http://blog.steve.org.uk/it_is_unfortunate_that_many_compani...](http://blog.steve.org.uk/it_is_unfortunate_that_many_companies_need_the_same_sysadmin_jobs_carried_out.html)

If you did (re)find the site please do post it as a comment. I like the idea of remote-sysadmin, and did ponder the value of advertising myself for it, but I couldn't quite believe there would be a regular market for it.

</comment><comment><USER3> I did not see that but if you find it, please post it here. I'd like to know as well.

<comment><AUTHOR> Yep, it looks really cool. I forgot to bookmark it. It looks like it was a curated list of sysadmins that you could hire for short term min projects but also retainers as well too I think.

I need to move a WordPress blog from Hostgator to a VPS, but I'm not too hot on setting that all up. I can follow the Digital Ocean community guides ok, but I'm worried I'm missing something. Better to have someone who knows what they are doing. I don't trust people on odesk or elance.

<comment><USER4> Didn't see the website but give me a shout if you want and i'll try to help you depending on your requirements :).

Btw. why would you trust more someone on a website or HN over someone on Odesk or Freelance?

<comment><AUTHOR> I tend to look at HNer comment history. You can usually tell quite quickly if someone seems sound or not.

I also don't like the ODesk / freelancer sign up process (credit cards etc).

<comment><USER4> I see and yes that works for HN but i doubt it will work for most of the other sites then. Buy me a .lb of coffee and i'll move your blog :)</comment></comment></comment></comment></comment>


Every week a new person becomes the curator of the NHS Twitter account - <AUTHOR> https://www.england.nhs.uk/atnhs/# <comment><AUTHOR> Submitting this because HN has talked about similar ideas in the past, and because it's interesting to be able to talk to a wide range of people involved in different bits of the English NHS.</comment>


No Man's Land: Where on Mars Should Astronauts Go? - <AUTHOR> http://www.scientificamerican.com/article/no-man-s-land-where-on-mars-should-astronauts-go/ <comment><USER2> Personally I would pick somewhere along the Valles Marineris since the landscape photography would be out of this world.</comment>


The History Behind the Science of Stress - <AUTHOR> http://www.npr.org/blogs/health/2014/07/07/325946892/the-secret-history-behind-the-science-of-stress

<comment><USER2> This is a very important story, which I heard on the radio while driving yesterday. Most of what we think we know about stress and its effects on health, and especially about "Type A personality" and health, is skewed by how the research programs were developed in the 1950s through the 1970s. Various forms of adverse environmental influences that can be categorized as "stress" do have general health effects, but not all stress is bad at all levels of stress, and sometimes stress co-occurs with other influences on health that are actually more important than stress itself.

A free article on the Pub Med Central website, "The 'Father of Stress' Meets 'Big Tobacco': Hans Selye and the Tobacco Industry," Am J Public Health. 2011 March; 101(3): 411–418.

[http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3036703/](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3036703/)

doi: 10.2105/AJPH.2009.177634

tells more of the background discovered by health researchers digging into the trove of tobacco industry documents that were disclosed as part of the tobacco industry settlement with state governments in 1998.

<comment><USER3> To avoid the politics, something that turned up in later research is that the thing that correlates to bad outcomes (i.e. heart disease) is hostility more than ambition.

</comment></comment><comment><USER4> And to think the science of sleep is just really getting started. Stress and sleep are two sides of the same physiological coin and basal responses at that. Sleep well and manage stress to optimize health, productivity, and happiness. Yet, I know I'll spend the next decade showing the data to support that sentence.

<comment><USER5> We should all hope health wearables arrive sooner. The sooner we get devices that accurately capture and measure personal health data (e.g., stress levels, cholesterol, blood pressure, glucose levels), the sooner we consumers can identify cause-and-effect relationships and determine the optimal behavioral patterns for maintaining health.

<comment><USER4> It depends on what you want. Accuracy and price are at cross purposes. Our biggest fear is the glut of cheap devices now is undercutting the long-term value for true health applications.

<comment><USER6> You can have an accurate step counter at a low price. The problem is that the step counter is being sold as a device that knows how well you sleep and how many calories you burn.

</comment></comment></comment></comment><comment><USER7> A wonderful read on th subject is Robert Sapolsky's "Why Zebras Don't Get Ulcers": [http://www.powells.com/biblio/1-9780805073690-1](http://www.powells.com/biblio/1-9780805073690-1)

</comment><comment><USER8> No mention of eustress. [http://en.wikipedia.org/wiki/Eustress](http://en.wikipedia.org/wiki/Eustress)

<comment><USER4> What is the _objective_ data to support that concept?

<comment><USER8> There is a list of references at the bottom of the article.

</comment></comment></comment><comment><USER9> This article mentions that there was tobacco money and then does a lot of hand waving insinuation that Selye was wrong. I don't think that's the case. As far as I understand it he remains a giant in the field and his work continues to be built on.

Selye was willing to say some people were overstating the role of tobacco in isolation as a cause of disease. So? I happen to think he was right.

I read Sapolsky's "Why Don't Zebras Get Ulcers?" it confirmed Selye's major ideas. It was extensively end-noted with modern research.

</comment><comment><USER10> What I've read and understood is that the stress associated with low social status (i.e. open plan offices for a programmers, whose work requires more focus and privacy) is extremely toxic (depression, anxiety disorders, cardiovascular load) but that stress _itself_ isn't inherently toxic. For example, if you work 70 hours per week but you enjoy the work, it doesn't hurt your health. Technically speaking, all exercise is _stress_ but not unhealthy (in fact, the opposite).

If anything, our bodies interpret our social status by stress, which means that good stresses (e.g. a workout, a hard problem) can have positive effects.

I don't think it's even controversial that the negative stresses coming from, e.g., open offices and subordinate work contexts, are extremely toxic. It's just that physical stress isn't _prima facie_ dangerous.

<comment><USER11> I suspect that this varies a lot by person and their internal attitudes. I don't associate open plan offices with low social status, nor do I let other people define what my social status is. As a result, this is not a major source of stress for me. Now, I put all sorts of pressure on myself about living up to my own personal standards, but the particular concerns that you raise and assume are universal barely even register in my head.

<comment><USER12> I used to get stressed out in open plan offices in my 20s. I was only running at ~25% productivity compared to when I worked on my own projects at home or took the day off and worked from home. The reason I was stressed was because of my own expectations. My employers have never had an issue with my performance. It's just I knew I should be able to get 3-4x as much done each day and I was beating myself up over it.

These days I don't mind that I am much less productive than I could be. I do a couple of hours of really focused work early in the morning before the rest of the office arrives and I use the rest of the day to socialise, network, go to meetings, help the junior devs, and research.</comment></comment></comment>


Bagel: Large-scale graph processing on Spark  - <AUTHOR> https://github.com/mesos/spark/pull/48

<comment><USER2> I dogfooded Spark by writing a distributed machine learning aligner. It is tightly integrated with Scala. It obviously still has a long way to go, but it does provide some very nice properties, e.g.:

\- Support map-reduce style computation model (i.e. map, reduce, groupByKey style computations). It is important to differentiate map-reduce infrastructure (e.g. Google MapReduce, Hadoop MapReduce) from the computation model (e.g. map/reduce function).

\- For many use cases, minimal change required going from serial code to distributed, parallelized code.

\- Very nice REPL interface that's very convenient in exploring data.

\- Much better support for iterative (machine learning) computations through in-memory cache of data sets and minimal per iteration scheduling overhead.

\- Good integration with existing infrastructures, e.g. Hadoop HDFS

To find out more about the framework: <http://www.spark-project.org/>

<comment><USER3> Is Spark really as fast in real use, as the graph on the site claims? I don't think I've ever seen anything beat Hadoop by that margin before.

<comment><USER4> We currently use Spark for generating reports that scan the same data set multiple times. We got a 20-30x performance gain over hadoop+hive. Spark's ability to keep data sets in memory works very well for our use case.</comment></comment></comment>


Estimate the size of your English vocabulary (2014) - <AUTHOR> http://vocabulary.ugent.be <comment><USER2> Took it once and was very conservative on words I didn't "know". Got 49% with no false positives. Upon inspection if I'd answered yes to everything I thought was a word, I'd have gotten much higher with no false positives. So I took it again and answered yes to everything I could guess the meaning of and got 89% with no false positives.

An example of something I said yes to on the second pass that I'd have said no to on the first: I guess superexcellent is a word, the meaning is very clear but I don't know that I've ever seen it used before and would never consider using it myself.

<comment><USER3> I had the inverse issue. I answered yes to too many non-words, despite the fact in hindsight they weren't weren't a word - but I thought they sounded correct enough that they _could_ be used as a word.

Went back and also got a higher score by being less trigger happy with my right finger.

<comment><USER4> Yeah - somehow "humanly" is a word while "autereric" is not. Cats sit humanly, but filmmakers don't have auteureric predilictions.

I got 87-0 being strict and 93-17 by including "maybes", and I'm not sure which is more reasonable.

</comment></comment></comment><comment><USER5> This test made me feel quite the blatherskite with my hemineglect of words a bibliolatrous individual like myself surely should know. It quite made me turn a pale shade of ecru; my vocabulary thinner than a shingling. Perhaps the ratiocinative thing to do would be to rusticate for a while, and form the cullet in my mind into something more .... ah, forgive my aposiopetic nature, for I must schuss off somewhere else.

<comment><USER6> Why waste time say lot word when few word do trick?

<comment><USER7> "Brevity is the soul of wit." -WS

</comment></comment></comment><comment><USER8> I wonder if this test is trying to detect something else than what it pretends to be. Dislexya, mispellings... It didn't feel like a vocabulary test.

<comment><USER9> Obviously it's trying to detect left handedness

<comment><USER10> I think it might actually just swap the buttons if you are left handed

<comment><USER11> I'm left handed and it used F for "no" and J for "yes".

</comment></comment></comment></comment><comment><USER12> 88% with 1 false positive, but it was 'wargish' which I think should be accepted as a real word.

To be fair, I did also hit yes for words that I could work out the greek/latin roots for.

\---

I know the intent of this test, but English has a special problem in that there is no real definition of what is and is not an English word. For example, there are a large number of words on this test that come directly from another language and are used as a term of art in a specific field.

See: mora, furuncle, cercal

Then you get transliterations like dacoit.

Are units of currency automatically English: piastre

Another genre is words that may be technically English, but would never be used - see delegable (delegatable would be used)

Then you have several alternate spellings from British English/American English/alternative transliterations.

<comment><USER13> I would consider "wargish" to be a word meaning "in the style of a warg", with warg being a word for wolf that appears in modern English works (Tolkien being the obvious one).

<comment><USER3> Agreed. If an author described a character to me as "wargish", I would instantly have an idea in my head of what that person looked like - ugly, brutish, and long-faced like a wolf. It should then I think, by extension, be classed as a word; or at least not be used in a test like this.

</comment></comment></comment><comment><USER14> As a native english speaker, answering only known words: 80% true positive, 0% false positive.

As a native english speaker, answering for known or _plausible_ words (with empty profile to not contaminate the test): 96% true positive, 23% false positive.

<comment><USER15> German, 56 % true positive, 0 % false positive, could propably get about 90 % of the word non-word decisions right by guessing, the non-word are pretty non- English. There werde also a hand full of true-positives were I tended to think I know the meaning but decided to err on the other side because of the promised heavy penality.

PS: Right-handed

</comment></comment><comment><USER16> It doesn't estimate the size of your vocab. It measures your propensity to distinguish wordy looking things from nonwordy looking things.

</comment><comment><USER17> What counts as "English vocabulary"? I definitely don't count the name of the currency of Malaysia. It seems like they're relying on random selection from thefreedictionary and user feedback, but their take on "English word" vs "other language that happens to use the Latin alphabet" differs from mine.

<comment><USER18> I got "gaucho" and that scored as an English word. I'm not sure I've ever heard it used other than to talk about Spanish speaking cowboys.

<comment><USER19> I got "halal". Definitely a word you hear as an English speaker, but not a word that came from English.

I have a feeling Scrabble players do well on this test.

<comment><USER20> What does "came from English" even mean? At some point all the words in English came from something else, like French, Latin or old Norse.

</comment></comment></comment></comment><comment><USER21> I felt smart having only missed 9 real words. Then I went to look them up. The definition for affricate made me feel real stupid again when I didn't understand almost every third word in the Google provided definition:

>a phoneme which combines a plosive with an immediately following fricative or spirant sharing the same place of articulation

<comment><USER22> I wouldn't feel bad...those (phoneme, plosive, fricative, spirant, and affricate) are specialized vocabulary words from a specific domain (linguistics). I only know them because I have an interest in the topic. Other than _phoneme_ , they describe how words are pronounced and/or formed in the mouth.

<comment><USER23> Another linguistics enthusiast checking in. To expand on this, a "plosive" is a sound made with the tongue stopping the airflow and then releasing it (think "t", "p", "k" sounds), while a fricative involves continuous airflow (think "s", "f", and so on). What that definition is saying is that an "affricative" is a sound that combines two of these made with the tongue in the same position. In English, an example is the "t-ch" sound at the beginning of the word "chat". Other languages have different examples, like the "t-s" sound written as ц in Russian.

</comment></comment></comment><comment><USER24> I got 80%, although I got penalized for "prostuttion" which I accepted by accident. I think it's a bit much to penalize people for accepting misspellings of real words. After all if someone wrote _" In Germany prostuttion is legal"_ the meaning is perfectly clear.

<comment><USER11> I got two slightly misspelled words and it didn't mark them as wrong, so this whole test seems wildly inconsistent(not to mention it seems to have some foreign words and even names marked as real english words according to some other commenters...).

</comment></comment><comment><USER13> This was a cute test, but it dinged me for not "knowing" the word "toreador". Except I do: it's a Spanish word, not an English one. I also don't consider it a loan word, because the English word for "toreador" is "bullfighter".

<comment><USER25> This will speak to your soul, then: [https://en.wikipedia.org/wiki/Uncleftish_Beholding](https://en.wikipedia.org/wiki/Uncleftish_Beholding)

</comment></comment><comment><USER26> 89% with no false positives.

I wonder how tightly coupled this will be to education, gender, age, and especially handedness. Will righties rule and lefties drool, or will an unthinkable upset occur? I expect the U.K. to dominate because all the smart movies use U.K. actors, full of $2 words.

<comment><USER18> Hah, leftie here with the exact same score as you. I'm probably going to be thrown out from any statistics they derive as an outlier because I'm a high school dropout programmer who ravenously read books in his youth.

<comment><USER27> 86% here with no false positives but with a four beer handicap and over 60yo but still "in the top level" (which has no definition). So I'll take this and skedaddle. Cheers :)

</comment></comment></comment><comment><USER28> 84% - 0%: Non native speaker. Have to brag :) And I beat my native speaker partner who had 86% - 13% = 73%. :)

</comment><comment><USER29> I know the non-words are really "not English", but I wonder how similar the sentences would be if asked to "use in a sentence".

</comment><comment><USER30> 73% with 0% false positive. Most of my false negatives were "Can you really stick that many suffixes on this word?"

</comment><comment><USER31> There were words like "bate" and "earliness" that I figured could have worked and would be legal but considered they could have been made up. Still happy with 79% - 0% as native English ( well Aussie)

</comment><comment><USER32> problem I have with this test, is a slight dyslexia when hitting the appropriate key. Several times I knew I hit the wrong one a moment after, but had no way to go back and correct it.

Also, in some cases I noticed that there was what appeared to be a real word, but it was slightly misspelled, so I'm sure that counted as an incorrect input.

<comment><USER33> I didn't see how to estimate the size of your vocabulary, I just saw a percentage. Did I miss something? Like I want to know something like, based on your answers were estimate you know 8311 words.

Edit: oops replied to wrong parent

<comment><USER34> It is saying what percent of ALL words you know. So just take the total count of English words and multiply by your percentage.

<comment><USER35> I don't think this can be right. the Oxford English Dictionary has over 300,000 main words, and I got a score over 80%. That would suggest I have a vocabulary of 240,000 words, which seems a bit high, to say the least!

<comment><USER36> Their FAQ claims 60,000 ("The word list should contain almost all English words...") so 48k+ works known, which is somewhat more believable. OED 2nd Edition (1989) has 171k full, current entries [1] (and another 47k obsolete, which I will disregard), so 137k+ words, which is quite a few but perhaps not beyond the realm of imagination.

[1] [https://en.oxforddictionaries.com/explore/how-many-words- are...](https://en.oxforddictionaries.com/explore/how-many-words-are-there-in- the-english-language)

</comment></comment></comment></comment></comment><comment><USER37> There's a definite grey zone of wordy word-like strings that are being cobbled together from chunks of other words, where it's very easy to imagine a possible definition for a word that sounds like that. Also, once or twice, I'm noticing things that feel like proper first names.

Meanwhile, the upper 10% of words includes highly specialized terms from niche sections of fields such as geology, chemistry and medicine, which arguably, are not words, even if you've seen them before. In some respects, a proper name isn't a word, and so, is a corporate invented product name really a word? Isn't that also a proper noun like "John" or "Texas"?

In this 10% grey area, I see a few words that I know to be words, but I cannot immediately tell you the definition, and if I were to try and use them in a sentence, I'd be able to apply the word class (noun, verb, adjective, adverb) within the structure of a sentence. But, I don't actually know the word. I only know that it's probably a word, but since I can't guess the definition or use it, I should dump it as an unknown, but more often than not, I'm getting away with some of these, mostly because after multiple passes, I can detect how the non-words are being generated, and the real words possess a more obvious validity of structure.

That said, I'm able to hover at or above 90% with occasional disembiggening penalties for over-cromulence.

</comment><comment><USER38> Discussed at the time: [https://news.ycombinator.com/item?id=7949313](https://news.ycombinator.com/item?id=7949313)

</comment><comment><USER39> A lot of the time it's easy to guess what real words are based on what appear to be structured correctly. I scored much higher guessing (even though I had a couple of false positives) then when I answered truly.

Also a lot of the complex words are medicine/medical which makes sense since there are an awful lot of them but isn't necessraily representive of 73% (my score) of the words I will encounter outside of medical literature.

</comment><comment><USER40> 87% with no false positives. In my view many of these are not what I consider English 'words' - plant names, trade marks and archaic words. Still fun though.

</comment><comment><USER41> This is a neat format but I wouldn't trust the results. Some of the "real" words are not actual English words (I got a Spanish word and an acronym) and some of the "fake" words are real, but obscure (for example, "coath"). Still a lot of fun, which seems to be the point.

</comment><comment><USER42> A few words (kilocycle, advertency, implacability, teacupful, outcrop) I would accept in day to day use, but I wasn't sure if they were legal conjugations of the root words. In fact firefox's dictionary does not have advertency.

</comment><comment><USER43> Today I learned that "effing" is actually considered a word nowadays.

<comment><USER44> "effing and blinding/jeffing" is something old people say, I imagine "effing" by itself as a minced curse is far older.

</comment></comment><comment><USER45> The test, that really estimates the size, not just the percent: [http://testyourvocab.com/](http://testyourvocab.com/)

</comment><comment><USER46> I misinterpreted the test, since I assumed they were all real words and only selected the ones I at least could figure out the likely definition for. That said, I'm still feeling pretty good about my results, only missing a few real words (of which I recognized 2-3 of them, but didn't know the meaning), and had one non-word which was very similar to two real English words.

</comment><comment><USER47> I don't understand the purpose of the test result.

So basically I know 83% of the English words... that were asked in this test. If next year 1000 new words are introduced to the dictionary, that doesn't automatically mean I know 830 of them. In other words the minimum size of my vocabulary is 83 words. Nothing more. Nothing less.

<comment><USER48> If you poll a hundred voters and 70 of them say they're voting Wolf and them say they're voting Sheep, do you know nothing more than at least 70 people will vote Wolf and 30 Sheep?

</comment></comment><comment><USER49> For the record, I found 67% of the words, and said yes to 0% of the nonwords. I proceeded conservatively, only saying yes when I have seen the word before, not when I guessed it should be a word.

They say this is fairly high for a native speaker.

I gave my mother tongue as German, so is this "fairly high" for a native German speaker?

It felt too easy.

</comment><comment><USER50> 84% with no false positives. I missed mostly medical lingo, one legal term and "stitchwort" which I did suspect was a plant but I had no idea which one.

This is pretty much what I expected, since as a non-native speaker; though even if you told me what stitchwort was in Finnish I'd still not know what it actually is besides a plant.

</comment><comment><USER51> 90% / 0% Native English, I was guessing most of the time. Really surprised the medical words didn't screw me over. If it sounded and looked like an English word, I would answer Yes. I answered No to anything that seemed even a little odd. I think if I retook the test my score would vary wildly...

</comment><comment><USER52> Didn't realize there where non-words in the tests and was getting disheartened at how many words I don't know...

</comment><comment><USER53> I wonder how they take into account the kind of people who take a test like this are likely to have a good vocabulary. I suppose that might not be true but I don't think I'd do something similar for a test that measured something along the lines of matching faces to pop musicians.

<comment><USER26> I bet the only truly important correlation is how many books a person has read. Too bad they didn't ask that.

"Would you say you've read less than 10, less than 100, less than 1000, or all the books ever?"

<comment><USER54> I've read probably around 500 books in my life (and I'm 30), and I only got 70% with zero false positives.

I think it depends on the type of books one reads, not quantity.

<comment><USER55> It makes me sad that between your and the parent comment we take 100, 500 and 1000 to be a lot of books to have read in a lifetime.

</comment></comment></comment></comment><comment><USER56> 96%. The two I missed were misclicks (e.g., I accidentally hit 'no' on 'parboil.')

A number of those were intuition, though.

<comment><USER57> Going on intuition I figured that "boxthorn" and "razorbills" were undoubtedly English words but not knowing them I followed the rules and answered no. They both are, apparently, English words. 87%.

<comment><USER56> Part of the 96% comes, I think, from the fact that I did doctoral work in English lit, which gives me a better "feel" for those sorts of archaisms. I can't say that I knew all of the words that cropped up in the test, but many of them sounded plausible on the basis of having read quite a bit of pre-19thC stuff.

</comment></comment></comment><comment><USER36> I found the test significantly easier on subsequent attempts, as I familiarized myself with how strict (as it turns out, not very) their definitions are - I wonder if they will do any correcting for this?

(Scores for reference: 70, 77, 87, 84, 84, 84)

</comment><comment><USER58> 93% with 3 false positives, all of which were literally me being a dumbass and hitting the wrong button because my hands REALLY wanted yes to be on F.

Which means nothing except that I've seen a lot of words and remember them.

</comment><comment><USER48> If you're interested in this you might also like [http://testyourvocab.com/](http://testyourvocab.com/)

I'm working on something similar for German at the moment.

</comment><comment><USER59> Non-native speaker, got a score of 63% with 1 false positive, so that makes it 60%.

However, it said that it is relatively high for a non-native speaker. How did they reach this conclusion?

<comment><USER36> From their FAQ, "...a proficient native speaker will know some 40,000 words of the list (i.e., 67%) ... [for second language speakers] our estimates range from 6,000 words (10%) for a medium proficiency speaker to 20,000 words (33%) for a high-proficiency speaker."

So well above their estimates, but they don't extrapolate on the source of those numbers - they might be available somewhere at [http://crr.ugent.be](http://crr.ugent.be)

<comment><USER59> Aha, thanks for that data. Idk, somehow I feel second language speakers would perform better than 33% Thanks for that link too

</comment></comment></comment><comment><USER60> 84% but penalized 3% for saying "steepy" is a word.

<comment><USER61> Turns out it is: [https://www.merriam- webster.com/dictionary/steepy](https://www.merriam- webster.com/dictionary/steepy)

</comment></comment><comment><USER62> That was fun!

My score:

You said yes to 63% of the existing words.

You said yes to 0% of the nonwords.

This gives you a corrected score of 63% - 0% = 63%.

I already knew I wasn't a wordsmith so that's not awful. About what I expected.

<comment><USER63> For science! 67% for me.

My brain small. Sad!

<comment><USER64> Don't worry, I got less than that, but 0 bad guesses!

I'd bet there is a bias from those reporting scores here -- a humblebrag, for science, of course!

<comment><USER65> As a none native speaker (English would be my 3rd language at best, maybe 4th) I expected to dramatically underperform compared to most scores here, but:

You said yes to 76% of the existing words.

You said yes to 0% of the nonwords.

This gives you a corrected score of 76% - 0% = 76%.

Which I find oddly high.

</comment></comment></comment></comment><comment><USER66> 77% existing, 7% non existing.

"This is a high level for a native speaker."

I'm happy because I'm Swedish. I would like to see a histogram also by native language.

</comment><comment><USER67> Didn't estimate my vocabulary, just scored results of my test.

From the description I was hoping for a statistical estimate of how many English worsd I know...

<comment><USER36> From the FAQ: "The master list includes 60,469 words and 304,275 nonwords. From these you get a random sample of 70 words and 30 nonwords. The word list should contain almost all English words (non-inflected forms)."

So just multiply by 60k for how many you know. You've also got the info you need for stuff like stddev.

<comment><USER47> That's not how that works. To obtain an absolute number the words have to be ranked by frequency and then the 100 words have to be picked so that frequent words appear first and less frequent words appear at the end. The estimator then knows whether you know a 90%, 99%, 99.9% or 99.9% percentile words and uses that to estimate that you know enough words to cover x% of all words that will appear in a text. Then you use that percentile on your word ranking to count how many words are below that percentile.

The posted test site does nothing of the sort.

Imagine the test only asked you a single 90% percentile word. vocabulary.ugent.be would tell you: You know 100% of the words. The process I described would tell you: You know 90% of the words.

Obviously both tests become more accurate as the number of words increases but the vocabulary.ugent.be test requires you to test all 60k words to get an accurate result. The second test will probably need less than 1000 words to be dead on.

[http://testyourvocab.com/](http://testyourvocab.com/) seems to be a pretty good implementation of the approach I described.

<comment><USER36> The test you described is useful for determining how many words in a text you will know - i.e. what percentage in a given set of English. That's probably a more useful number, but it's not what parent asked: "I was hoping for [...] how many English worsd [sic] I know."

> Obviously both tests become more accurate as the number of words increases > but the vocabulary.ugent.be test requires you to test all 60k words to get > an accurate result.

They could _never_ get an accurate result for words known in a body wih this test - it needs a separate dataset of word frequency. That's not what they're testing.

</comment></comment></comment></comment><comment><USER68> I wonder how much of this study is permanently skewed by being on HN and it's specific audience - as well as all the redos.

</comment><comment><USER69> Weird that they provide a button to improve your score.. wouldn't that mess with their results if many people retest themselves

</comment><comment><USER70> 96% true positive, 0% false positive.

Agreed that this seems like a measurement of something other than vocabulary.

</comment><comment><USER71> Few attempts with around ~60%. I noticed my response times for known words were quite higher those in question.

</comment><comment><USER72> 73% with 0% non-words. While I was taking it, it felt like I was doing worse, but not a bad result overall.

</comment><comment><USER73> Interesting, found the word not from Merriam-Webster dictionary: araliaceous.

</comment><comment><USER74> 73% with 0% false positives, ESL

I am curious about how common is having a ~75% score

</comment><comment><USER75> I really wish they gave the histogram of scores.

83% (86% known - 3% false positive)

</comment><comment><USER76> Russian, 66% counting the 7% penalty for non-words.

</comment><comment><USER77> 87% and no false positives. Native speaker. Fun!</comment>


High School Club Builds Self-Driving Vehicle - <AUTHOR> http://theinstitute.ieee.org/ieee-roundup/blogs/blog/high-school-club-builds-selfdriving-vehicle <comment><USER2> This paragraph is interesting:

For the most part, I'm self-taught. I took a series of C programming language classes a few years ago from Stanford's GiftedandTalented.com program. I learned the Python programming language on my own. In eighth grade, I took advanced-placement calculus and statistics courses, which enabled me to take multivariate calculus in ninth grade. We are fortunate at my school because we have a professor from Foothill College who teaches this course. This class turned out to be useful for understanding machine-learning algorithms.

I've finished Udacity's online machine-learning engineer nanodegree. I also interned at Nvidia, in Santa Clara.

\-----

The opportunities available for self-education now are just fantastic.

Kudos to all those involved.

<comment><USER3> Wait. So, a high school student interned at Nvidia?

<comment><USER4> Don't know if it's still the case but when I was at Microsoft in the late 90s this was common, and they were given real work.

<comment><USER5> Is there something more to the kids being software prodigies? When I was young, there weren't that many. If there were, they didn't make the news. Today, it seems like they are more frequent and that many are in software. Has there maybe been a societal change that allows more to blossom?

<comment><USER6> Broader access to technology? There weren't many online courses or cheap computers 10 years ago

<comment><USER5> After I typed it, I thought about the price aspect. Prodigies that made the news, back when I was younger, often came from wealthier families. Today, prices are quite low for a bunch of things.

That does count as broader access, methinks. So, I suspect you're right. There are many, many sources of inexpensive education. I don't know if that would, statistically speaking, increase the actual numbers/percentages - but if it doesn't, it would almost certainly enable them to go further.

Ever wonder what it'd be like to have had access to this stuff when you were younger? Personal computers didn't arrive until I was an adult.

Anyhow, kudos on the kids. That is impressive and certainly better than I can do.

<comment><USER6> I can't really wonder what it'd be like to have access to this stuff - I'm 17 myself, and am from a moderately wealthy family in the Bay Area so have had exposure to technology basically my whole life. I definitely think the culture of the area and growing up with a computer in my room contributed to my passion for tech.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> I mean this in no way to belittle the students accomplishments... certainly they have done something that is extraordinary for highschool and something even trained engineers might find difficult.

But one takeaway from this article is how amazing it is that open source, the availability of hardware/electronics for reasonable prices, and open online courses where people can learn at their own pace have all come this far.

It is incredible to me for instance what is out there for object recognition algorithms and models now. And sensor hardware that used to cost thousands can now be had for a few hundred dollars.

Again, kudos to these students. They deserve praise for what they accomplished. But I am a little jealous I did not have access to what they have when I was a kid :)

</comment><comment><USER8> These types of headlines always remind me of The Pareto principle, in that 20% of the effort will achieved 80% of the result.

Getting the last mile (no pun intended) is what's extremely difficult and time consuming, and what separates the novice from the master.

Good luck to these kids and their bright future. I hope they bring us the last 100% necessary for self-driving technology.

</comment><comment><USER9> This is really cool; I'm glad to see this kind of technology reaching younger students. In grad school, we actually built something similar for undergrads to use as part of a robotics course. The idea was to give teams an identical platform, and then race them at the end of the semester to see who did the best job at programming it.

If anyone is interested, we open sourced all of the design files, with instructions on how to build a car of your own for around $900 (at the time, this was several years ago): [https://vaderlab.wordpress.com/roscar-robot- stock-car-autono...](https://vaderlab.wordpress.com/roscar-robot-stock-car- autonomous-racing/)

Here is a video of the results from the semester. One group was able to max out the speed on the car, which at scale is equivalent to 128 MPH: [https://www.youtube.com/watch?v=jwRjv3D7lGo](https://www.youtube.com/watch?v=jwRjv3D7lGo)

<comment><USER4> What a fantastic resource!

</comment></comment><comment><USER10> Worth noting FRIST Robtics competition is also a great place to get students involved in STEM and involved in team building, since running a FIRST team requires fundraising, design logos, order t-shirts, promote team event, so there are things for students who are not interested in the engineering part to do too, while learning more about STEM. Also parents are usually involved too... so more bonding time and more school-parent engagement.

I co-founded FIRST Robotics team in my high school (all started because I wanted to run a club for college app lol). Then the following year after we won the All-Start award, our school created an "engineering program", and our teams (yes, teams), battle with other nearby teams :) in NYC during off season.

</comment><comment><USER11> In Michigan, the lack of regulation around autonomous testing can be great. At first, you just needed a $30 manufacturer plate to do autonomous testing anywhere in Michigan, but I'm pretty sure they recently got rid of that requirement.

Level 2 autonomy is not exceedingly difficult to achieve either. If your car has LKAS/ACC or similar features you can port something like openpilot[0] over to your car.

[0] [https://github.com/commaai/openpilot](https://github.com/commaai/openpilot)

<comment><USER12> comma employee here - we have bounties for a handful of cars!

[https://comma.ai/bounties.html](https://comma.ai/bounties.html)

</comment></comment><comment><USER13> I love this story. I hope the snobs at r/machinelearning get a good read of it. I've seen HS kids there ask questions and get talked down to instead of encouragement. Those people want to keep ML their exclusive little club and apply intimidating terminology to simple concepts to keep people out. Congrats to these kids. Middle fingers in the air to the snobs.

<comment><USER14> This is so true. The difference between an ml expert and a novice is often less than a few percent on accuracy.

<comment><USER15> the difference is that the novice is nearly always combining models and code written and trained by the expert.

</comment></comment></comment><comment><USER16> I know this is off topic but I've been recently digging into geometric algebra. Boy is linear algebra a shit algebra compared with GA. It's particularly well suited for computer vision and what not. So check it out whatever if you are trying to get into it. Things actually make sense!</comment>


Online Gift Registry - <AUTHOR> http://thegiftregister.com.au/gift-shop/ <comment><AUTHOR> You need a great gift idea now? No problems. You've come to the right place. Just click a category below to see our collection of trending and unique gift ideas. All sections updated regularly.</comment>


Toyota, Lexus owners warned about 'relay attacks' - <AUTHOR> https://www.cbc.ca/news/canada/ottawa/toyota-lexus-relay-attack-1.5380947 <comment><USER2> There are many solutions like the one below.

[] [https://keylessprotector.com/](https://keylessprotector.com/)

</comment><comment><USER3> Some of the mitigations for this high tech security system are hilarious. Park a less desirable car in front of the target car.</comment>


Spolsky lives in a Reality Distortion Field - <AUTHOR> http://hsivonen.iki.fi/rdf/

<comment><USER2> The author admits he lives in his own reality distortion field where everyone uses Safari or Firefox and a Mac. The reality is that 80% of the world still uses IE, and anyone trying to make a living on the web should think long and hard before ignoring that fact.

Joel isn't arguing that IE8 should be or deserves to be the standard, he's simply arguing that like many things Microsoft puts out, there are too many people using it to ignore.</comment>


Live demo of Asana, a new product from Facebook co-founder Dustin Moskovitz - <AUTHOR> http://asana.com/2011/02/asana-demo-vision-talk/ The actual product demo begins at 11:20.<p>Might have to wait a bit for the clip to load as Vimeo doesn't allow you to play from an arbitrary point of the video. <comment><USER2> They spent a year writing a web framework and a few months writing a little project management app in it. The product looks pretty lame right now.

Hopefully they learned their lesson and they'll do a public release and start iterating.

</comment><comment><AUTHOR> The actual product demo begins at the 11:20 mark.

Might have to wait a bit for the clip to load as Vimeo doesn't allow you to play from an arbitrary point of the video.

</comment><comment><USER3> The explanation they provide about the data model and approach to the solution seems well thought out. However, despite their emphasis on simplicity I couldn't help but think it seemed thoroughly complicated when the demo finally started.

<comment><USER4> Amen to that, the product looks overly complicated, especially for a v1.

</comment></comment><comment><USER5> Do you also think launching their "collaborative task and project management software" for free initially is a clever marketing move?

I think it has potential, if the software is good, to put pressure on many other project management tools out there.

However, one could argue that paying a few bucks every month is not an issue at all for most companies and they will instead go with popular products from 37 signals and others. After all managers want the "cool" product. And 37s is great in selling the cool things.

<comment><USER3> I think Asana's approach is a good one. I'm always for someone that is willing to challenge the orthodox approach of the status quo. Besides, from their demo it seems as though they're aiming to appeal to more than just businesses.

</comment></comment><comment><USER6> anyone tried podio.com? That is pretty well done already with it's own appstore etc

</comment><comment><USER7> What I would like to see is XMPP integration.

</comment><comment><USER8> Looks like Wave :)

<comment><USER9> Yeah, funnily enough, at the beginning when he is naming all the other collaboration tools out there (including google docs) he doesn't mention Wave....

</comment></comment><comment><USER10> fuck, it's a social feed-reader for work stuff. I hate work, but this looks good.</comment>