Watch [Best Buy] Write [Their] New API Documentation - <AUTHOR> https://medium.com/best-buy-developers/watch-us-write-our-new-api-documentation-36051f794138 <comment><AUTHOR> Yes, I'm not happy with this article's title. Basically I thought that Mailjet made a brilliant choice in open sourcing the process of crafting their API documentation[1], and Best Buy has decided to follow suit.

[1] [https://www.mailjet.com/blog/introducing-new-mailjet-api- gui...](https://www.mailjet.com/blog/introducing-new-mailjet-api-guides/)

<comment><USER2> Hey Eric, I'm a developer evangelist at Mailjet and we're so happy you loved it! Thanks a lot for the kind words.</comment></comment>


Show HN: Hack interviews by studying questions recently asked by companies - <AUTHOR> http://techinterview.club <comment><USER2> Repost (from the same day): [https://news.ycombinator.com/item?id=16169971](https://news.ycombinator.com/item?id=16169971)</comment>


Answers about Android - <AUTHOR> http://www.bbc.co.uk/news/technology-20754182

<comment><USER2> "Believe it or not, we started work on the iPlayer radio app for Android on the same day as the one for the iPhone, but we're still resolving a number of issues [...] That worked out of the box on Apple, but not on Android."

</comment><comment><USER3> writing a program so it runs cross platform on ~800 devices and ~100 different operating systems is more work than writing a software for a single device and software.

Android is defragmenting the market, because it allows you to write a single software for both samsung touchwiz and htc sense.

iOS is fragmenting the market because you have to write a seperate app for it.

The fragmentation complaint makes no sense, because Android did not spawn all those different devices. Android just allows you to target all of them with the same code.

</comment><comment><USER4> than*

Sorry, just a pet peeve.</comment>


Show HN: Go statistics library with 100% code coverage and no dependencies - <AUTHOR> https://github.com/montanaflynn/stats/blob/master/README.md <comment><USER2> If you are calculating means, variances etc using the textbook formulas, you are doing it wrong. Don't just add up a bunch of floats and divide by N to calculate an average. Don't use random people's high school homeworks for statistical analysis. Write things with dependencies so they are correct.

<comment><AUTHOR> I'm not sure why you think using standardized formulas is incorrect, there are certainly cases to use a mean average. There are other times you may want to use geometric mean, harmonic mean, mode, median, etc... and the library doesn't force anyone to use a specific function for the analysis, it just provides the tools to be used as needed.

Saying this is my high school homework is quite rude, it's a project I've been working on for several years along with several other contributors. It's cloned hundreds of times a day and quite obviously of benefit to many people and organizations.

This library is used as a dependency in many projects like cockroachdb, mongodb, etc... and has a lot of tests to verify it's correct. Not having dependencies ensures that the entire library is well tested and removes the need to track down bugs far down the dependency graph.</comment></comment>


Ask HN: What are the best IT war stories? - <AUTHOR> <comment><USER2> At my first developer job, I accidentally left off the 'Where X=X' in a SQL update statement. My manager didn't catch it in time and pushed it from staging->production.

The next day, the damage was done when the first client started using the new features I just implemented. I stayed until midnight pouring over code with my manager to make sure there were no more mistakes.</comment>


Data Driven Hiring: How RJMetrics Hired an Assistant - <AUTHOR> http://info.rjmetrics.com/blog/bid/52535/Data-Driven-Hiring-Hiring-an-Assistant

<comment><USER3> If you were deriding some executives it'd be another thing. But I'd steer clear of mocking the little guy on your corporate blog.

I know it isn't your intention but your somewhat mocking commentary at some of those candidates may be taken in a bad way and reek elitist - <USER2> qualities you really want to project as a founder, imo.

Example: _Computer illiteracy wasn't one of the job requirements._

A kinder way to state this would be "We really need someone who is proficient with the computer."

<comment><AUTHOR> Noted, thanks!

</comment></comment><comment><USER4> People are so sick of filling out insipid job application forms these days (only to never hear back anything), you can't blame someone for not taking it seriously every time. The situation is probably far worse for admin assistant jobs.

However, I applaud that you took the time to put together a simple project assignment to evaluate candidates and then used that as a basis for a phone screen.</comment>


Show HN: Convert Website Visitors into Customers - <AUTHOR> https://www.engagerhq.com/ <comment><USER2> Usefull app will give a try</comment>


All courses on Udemy are $10 at the moment - <AUTHOR> https://medium.com/discountcoupons/black-friday-sale-on-udemy-goes-live-1e3d6456a189 <comment><USER2> Given the business model followed by Udemy, most of the courses I've previewed are underwhelming.

But I'm sure there would some hidden gems in there, any suggestions from HN crowd?</comment>


Optimize Code or Upgrade Memory? - <AUTHOR>

Myself and my partners are building a very large online comparison site.
There are currently millions of products in the site and when we move to compare other things there could theoretically be 100 million or more.
The database is taking a hammering at the moment and the site keeps going down.
My partner is a tech nerd and wants to throw more hard drives and ram at it but i want him to optimize the code more.
Memory and ram is very cheap now so this is actually a good short term solution and we will need to upgrade eventually anyway.
My problem is that i personally think even though the memory and ram is a cheap solution we should spend the time to optimize the code a lot more even though this could cost many expensive hours of work combined to a few dollars for more ram.
What does Ycombinator think? <comment><USER2> Do you have new features that could be implemented with that time?

There are a lot of other variables as well, but at a high level, if there are new features that could generate revenue, then I personally would throw hardware at the problem while building an iterative plan to deal with optimization.

</comment><comment><USER3> It is not an either/or choice. Sometimes you need to do both: throw hardware at the pb _and_ optimize the code.

Sometimes you need do do everything possible to scale up.

</comment><comment><USER4> Do some quick tests on your app to figure out what part of your app is the bottleneck. Rememeber the 80/20 rule applies here as well. You might get a lot of performance improvement by focussing only on few key areas. DO NOT, EVER, OVER OPTIMIZE YOUR APP. If you need to spend lot of time to rework then do it in phases rather than taking a big bang approach

</comment><comment><USER5> What does "my partner is a tech nerd" mean?

Does he know what he's talking about, when it comes to code?

Do you know what you're talking about, when it comes to code?

Who knows more?

Why do you not trust his suggestion to throw more hardware at the problem?

Why does he think more hardware will matter?

</comment><comment><USER6> Both.

Never underestimate the effect of speed on traffic. Throw everything you can at it.

</comment><comment><USER7> Both.</comment>


Irish vet fails oral English test needed to stay in Australia - <AUTHOR> https://www.theguardian.com/australia-news/2017/aug/08/computer-says-no-irish-vet-fails-oral-english-test-needed-to-stay-in-australia <comment><USER2> "One of the first pieces of legislation passed in the new Federal Parliament was the Immigration Restriction Act. Now known as the infamous White Australia Policy it made it very difficult for Asians and Pacific Islanders to migrate to Australia. This Act stated that if a person wanted to migrate to Australia they had to be given a dictation test. The dictation test could be in any European language. So a person from China or Japan who wanted to live in Australia could be tested in one or all of French, Italian or English languages."

Granted this was well over a hundred years ago however I feel that the Australian attitude to migrants is still quite unfriendly. They're making it more and more difficult to migrate there even if you have roots in the country especially if you are from a non English speaking country.

Source: Am Irish and went through the Australian immigration process years ago.

<comment><USER3> Don't be mistaken, Australia has the 12th highest immigration rate of all countries per capita. [1]

and the 8th highest immigration of all countries, regardless of population. (also [1]).

Australia has a massive, massive amount of immigration.

[1] [https://en.wikipedia.org/wiki/List_of_countries_by_net_migra...](https://en.wikipedia.org/wiki/List_of_countries_by_net_migration_rate#Countries)

<comment><USER4> edit: I was wrong regarding Wikipedia's data.

Australia has a cap on permanent immigration per year of around 200,000. That Wikipedia page is claiming 45 per 1,000 inhabitants, which gets you closer to one million net migration. That conveniently ignores the far more important number: how many people are actually allowed to stay on a permanent basis?

Further, about half those ~200,000 allocated per year go to people already in Australia.

Back in reality, Australia is extremely restrictive when it comes to allowing permanent immigration. It has a lot of temporary work immigration.

[https://insidestory.org.au/how-many-migrants-come-to- austral...](https://insidestory.org.au/how-many-migrants-come-to-australia- each-year/)

<comment><USER3> That wikipedia page stats are over a 5 year period, for every country

So the 1,023,107 listed from 2007-2012 is 204,621 per year.

Sounds spot on to me.

> _Further, about half those ~200,000 per year go to people already in > Australia_

Sure, but they are not permanent in Australia, and immigration means they became permanent. It's not really relevant where they started from.

Australia is allowing ~1 million people each 5 years to becoming permanent, which is a huge number... in fact 8th in the world overall.

It's the same for every other country, so the comparison to other countries is still spot on.

<comment><USER4> Correct you are. I thought the wiki reference was annual, having to do with the far larger temporary work immigration that Australia allows.

</comment></comment></comment></comment></comment><comment><USER5> Reminds me of a Belgian guy who was in the papers a few years ago, who had to pass an 'integration test' before he could get Dutch citizenship. Part of that test is a language test (people from the north of Belgium, i.e. Flanders, speak Dutch - it's the same language as the one spoken in the Netherlands. So, like the woman in the OP, he had to take an exam on his mother tongue, and with much less of an accent than an Irish accent is as compared to the Queen's English). If you have a degree (even just a high school degree) from a Dutch language school, you're exempt, but this guy didn't so he wasn't (this exemption was put in to basically avoid situations like this, under the assumption that the vast majority of people would have a high school degree).

Either way, he failed the test a few times; reason being that this 'integration test' doesn't just cover language skills, but also things like 'knowledge of Dutch society'. For example, you get 3 pictures of a front yard of a house, with varying degrees of clutter on them (like a children's bike and a garbage bin with the lid open). Question being: how are you supposed to maintain your yard, like in picture A, B or C? Most (Dutch) people I've given the test failed it because the questions are so ridiculous. It's quite lulzy actually. Well not for that Belgian guy I presume.

<comment><USER6> Hang on now...I thought only non-EU people must do the inburgering nonsense? Or does it also apply to anyone who wants to get citizenship?

<comment><USER7> EU citizens can travel freely regardless of citizenship, but getting citizenship in another country is a separate process.

<comment><USER5> Yeah this. You can live anywhere in the EU but every member state is free to determine criteria for getting citizenship.

</comment></comment></comment></comment><comment><USER3> I'm Australian and got very close to needing an English test to get Residency in Canada. I was getting residency because I was working as an Engineer and my company was sponsoring me. I have an Engineering degree from an English- speaking country (Australia), and had been working for the company for ~2 years, of course in English. Both of those things counted for nothing.

I would have had to to fly ~2500km, pay for the tests etc. (I was living in Northern Canada.. weekend trips to Alaska means driving due South)

I spoke to multiple people on the phone and in person (in English) and not a single one was able in any way to do anything to intervene. There was no possibility of me doing the test over skype video chat to avoid the expensive flight

Rules are rules, don't dare to interpret their intent. This applies doubly when dealing with Immigration.

Canada have now tightened the rules significantly, and it's my understanding if I applied now, I would be forced to take that English test.

<comment><USER8> Also Australian in Canada. I sat the mandatory English test, with a number of others that were native speakers.

It is a ~$300 test too.

<comment><USER3> ugh, sorry to hear that.

Out of curiosity, did you ace the test? I at least was hoping to do that.

<comment><USER8> Highest mark: 12/12

</comment></comment></comment></comment><comment><USER9> I live in Dublin and I can barely understand native Irish people either. I side with the computer.

<comment><USER10> "I live in Dublin and I can barely understand native Irish people either."

People in Dublin itself are widely regarded to have a neutral-sounding, easily understandable English. Indeed, Dubliners are conspicuously prominent in ESL teaching around the world for that reason. It is when one gets out of Dublin that accents begin to be more challenging for people accustomed only to mainstream US or UK pronunciations.

<comment><USER11> Very much depends. Some of the middle-class neighbourhoods produce a fairly neutral accent, but most dubliners speak in a very heavy accent, arguably one of the heaviest in the country.

<comment><USER12> You've clearly not been to Cork [edit..] or Kerry!

<comment><USER11> Yeah, both are pretty bad (I was born and raised in Ireland), but the north Dublin accent holds a special place in my heart for sheer badness.

</comment></comment></comment></comment></comment><comment><USER13> "Pearson has categorically denied there is anything wrong with its computer- based test or the scoring engine trained to analyse candidates' responses."

Surely that lack of self-questioning makes them categorically unsuitable to do the job. By refusing to acknowledge that there's room for process improvement, that signals that they don't care about quality.

</comment><comment><USER14> Seems a bit mad to make a private company the gatekeeper.

<comment><USER15> I have no idea why this news is titled "Computer says no". For the speaking part it's humans evaluating the results. That said you are right on point about private company being the gatekeeper.

IELTS raised the fees more than 300% in the last few years. They now cost around $330 per exam and they always invariably fail you the first time around. You are allowed to take as many tests as you need. Conveniently you pass after a few tries.

I know competent English teachers who grew up in Australia failing this test few times. And equally I know very few migrants from third world countries passing the test even if English is their first language.

It's more of a blatant money making scheme than anything else and quite obviously the department of immigration must charge IELTS for allowing it to be one of the English examiner.

Just follow the money and you have all the answers.

<comment><USER16> > it is the only one that uses voice recognition technology to test speaking > ability

> Other test providers have said they use human assessors

No, the article clearly states that the test in question used voice recognition on a computer. Other companies use human evaluations, however this one didn't.

<comment><USER15> This is what I disagreed with. Afaik, everything else is marked by computers but voice was always handled by humans.

Unless this is something very recent in which case I am not sure how computers can handle accents or ambiguous words. Even google's voice to text has so many errors on untrained voice models

</comment></comment></comment></comment><comment><USER17> The issue here is that whoever drafted the rules did not think to offer an easy way to appeal the results. If the person in the article could have just said "can I have a human do this next time" there wouldn't have been a problem. This is what happens when non-technical people that have too much faith in technology draft policies.

</comment><comment><USER18> This is a story about how voice recognition has failed and should not have been used in this situation.

I was very surprised to hear that this was even permitted a method for the outsourcing provider, but then again Australia has a poor and somehow still worsening relationship with immigration, so perhaps I'm more accurately "re- dissapointed" by this revelation (as an expat Australian).

<comment><USER19> If this was the US, an unsatisfactory vendor would almost certainly be there because of some political crony situation. Sometimes merit has no bearing on goverment procurement.

</comment></comment><comment><USER20> For me it seems like a conspiracy of immigration (in different countries) where whatever rules are passed down to them, they'll going to bend them to let as much ineligible people as possible while taking pride in rejecting obvious good fits. And then you would wonder why citizens distrust immigration to provide healthy addition to their society.

Not unlike some professors who take an easy pass on dullards while throwing all their wrath at students with signs of self-learning. Those people are actively sabotaging the system they're part of.

<comment><USER14> >For me it seems like a conspiracy of immigration (in different countries) where whatever rules are passed down to them, they'll going to bend them to let as much ineligible people as possible while taking pride in rejecting obvious good fits

I think you're just making things up.

</comment></comment><comment><USER21> What I don't understand is why vets are considered in short supply in Australia. Graduates vets out of uni struggle to get jobs here.

<comment><USER5> 'Hairdresser' is also considered to be in short supply. Here's a list of all occupations (in case you didn't know): [https://www.nwivisas.com/nwi- blog/australia/australian-skill...](https://www.nwivisas.com/nwi- blog/australia/australian-skilled-occupation-list-2017-2018/) . This list had a bunch of occupations removed earlier this year; bad news for everybody looking to emigrate to Oz to take up a lucrative career in picture framing, because "Picture Framer" is no longer eligible for a 457 visa.

I realize that your actual remark is to the 'why' of some professions being on this list, I'm just pointing the above out for non-Australians like myself who might be amused by these things when seeing them for the first time, like I was.

<comment><USER22> Chicken Farmers are in short supply apparently.

<comment><USER21> Chicken farmers are not in short supply. What is in short supply are people willing to work as chicken farmers for chicken feed (sorry for the pun).

</comment></comment></comment></comment><comment><USER23> [https://youtu.be/5FFRoYhTJQQ](https://youtu.be/5FFRoYhTJQQ)

<comment><USER24> For the curious:

Comedic bit about voice activated elevators. In Scotland</comment></comment>


Windows Task Manager Runs Doom (896 Cores) - <AUTHOR> https://youtu.be/hSoCmAoIMOU <comment><USER2> It's kinda amazing that these continue to get shared, despite even the first few videos by him being fairly clearly debunked as fake, e.g. because he got things like the capitalization the the "Task Manager" wrong. More details: [https://www.reddit.com/r/pcgaming/comments/hsryyn/windows_ta...](https://www.reddit.com/r/pcgaming/comments/hsryyn/windows_task_manager_runs_doom_896_cores/fyclcpx/), [https://www.reddit.com/r/windows/comments/hlo6nd/pokemon_the...](https://www.reddit.com/r/windows/comments/hlo6nd/pokemon_theme_rendered_in_windows_task_manager/fx1dwvd/)

</comment><comment><USER3> Fake but still cool: [https://www.reddit.com/r/pcgaming/comments/hsryyn/windows_ta...](https://www.reddit.com/r/pcgaming/comments/hsryyn/windows_task_manager_runs_doom_896_cores/fyclcpx/)</comment>


Full-stack developers are in fact stuck at mid-level. Don't go down that path - <AUTHOR> https://habr.com/en/post/436596/ <comment><USER2> That is a load of rubbish. I've been coding for various platforms and languages for over 20 years. My understanding of how software is built surpasses any language specifics, however I'm very proficient at a senior level with both front (JavaScript) and backend (Python) development and I can jump into Go, Java or PHP when I need to and set up complex server infrastructures. What I do, absolutely requires me to fully understand all sides of the software product. Not just for my clients, but for my own businesses which I could build because of this understanding. I'm not special here. Many full stack devs are very good (to a senior level) on both ends of the stack. This article is the writer's projection of his own failure to progress while dancing in both weddings. There should be an ethical disclaimer on "advice" given in authoritative tone that are purely based on one's own bad experience.

<comment><USER3> When you say

> My understanding of how software is built surpasses any language specifics

do you mean that sure, you don't understand all the language specifics, but that's unimportant because you understand the high level? Or do you mean that you keep track in detail of developments in each of the five languages you've listed (to the level of detail of reading all their release notes and testing every new feature, knowing about the characteristics of all different flavours of index in SQL databases, and knowing best practices for managing state in React components, as suggested in the article)?

If the latter - if you truly have an encyclopaedic knowledge of every tool you use - then, well, congratulations, but I hope you realise that nearly all of us who work with a diverse array of technologies never achieve that mastery.

But if, on the other hand, you're casually conceding that you _also_ don't have the low-level mastery of the tools you use that you would if you worked with only one of them every day, then... maybe you shouldn't be disdaining the author as a failure for failings that you also have?

<comment><USER4> What does being "senior" mean? I think it can entail many different things, from understanding the business, to work relationships with your stakeholders, to architectural expertise, and on and on. Compared to these, knowing the latest and greatest updates to any given programming language is fairly minor, especially when it's not the one you're working with at that given moment. You can catch up JIT when you approach a given problem on a given language instead.

There's no reason to think you came be senior in multiple languages. Even if they were unrelated and separate, you could do it, it would just take multiples of time. But the fact is, they are related. Concepts apply, and actually enhance when you apply them, across different paradigms. It doesn't work in a way that easily conveyed in passing comments (otherwise being a senior might be way easier!), but as you progress, your pattern matching brain will start to see and expect certain things from the languages you work with. That's what makes you am effective programmer in a language. The sense that "there has to be a better way to do this". Not knowing immediately what that way is.

<comment><USER2> When I hire, I never say "senior" or "junior". I don't care for those titles. They mean nothing. I meant "senior" in the "conventional" sense to match the language to that of the original post.

<comment><USER4> Likewise. Is it not still worth defining the inherent characteristics, even if it's not a job title?

<comment><USER2> Characteristics yes. As in what will one be doing during working hours, or what one delivers. However, I've seen "seniors" who really had no business being in software development. No idea how they claimed the "title". While at the same time I see what the industry labels as "Juniors" due to number of years in the job market, being so amazingly good at what they do that they put said seniors to shame. Yet the "business" needs those labels to justify a pay- grade of sorts. This is wrong in my opinion. Compensation should match the ability, not the number of years someone had the fortune of being employed at this or that capacity or how good they can blag through an interview.

</comment></comment></comment></comment></comment></comment><comment><USER5> I consider myself "full-stack" because my average work week nowadays seems to include a constant mix of four kinds of development: back-end performance- intensive rendering services (C++ inner loop kind of stuff), back-end "devops light" service design work and glue scripts, back-end automation work (random shit like PHP/Hack scripts to wrangle data), and plain old front-end (SPA React nowadays).

The post author seems oddly upset that someone corrected his TypeScript design choices. If you want to do full stack, you can't take feedback personally. I regularly get code reviews where I've done something that's standard practice in one language but a no-no in another, and I just didn't remember. (PHP/Hack is like ground zero for these gotchas.) That's absolutely ok — that's why we do code reviews after all!

My real expertise is the C/C++ level stuff, but I write the other code because I'm basically the domain expert. The value of that code is that it's doing something useful and nobody else had to be assigned to write it; not that it's "ideal PHP" or "beautiful React" right off the bat. That can always be tweaked and refactored in code review.

Summary: in "full stack", you can't afford ego about your code, but you need conviction about the value of your contributions. That often means you need to spend time understanding stakeholders or user needs rather than e.g. reading TypeScript release notes.

<comment><USER6> > I write the other code because I'm basically the domain expert

I think this is a key point. In my experience there are two paths for engineers to senior level positions: (1) management, which is usually easier (more openings available, engineers may find it unattractive) but slowly moves one away from technology development, or (2) domain expertise which is technically harder to achieve (need to be _recognized_ as expert) but often provides more freedom and allows staying science / technology focused.

Each path requires some (different) soft skills which are more important than maintaining either a super broad or a super focused "stack". My 2c.

</comment></comment><comment><USER7> Sounds like the author is blaming his shortcomings on full-stack development and is making a poorly thought out generalization of what full-stack development entails. Thinking that he can best people who boast many years of experience puts him right where he belongs in terms of experience and being a laughing stock.

More likely than not, he is stuck at mid-level because of his lack of humility and arrogant approach to seniors, traits that would make a terrible technical manager.

<comment><USER8> > I once cranked out a design based on abstract classes in TypeScript, and got > ridiculed because apparently nobody does it this way in TypeScript. I > certainly pretended that my colleagues were hopeless idiots. It used to help > before, but that time it left me with a bad aftertaste.

seems like both he and his coworkers would need some help in humility.

<comment><USER9> Are you suggesting that abstract classes are a good idea in TypeScript? Because to me, abstract classes suggest that you're doing a lot of inheritance, which is the opposite of a good idea in JS.

<comment><USER3> Why? Old-school pre-ES6 explicit prototype wrangling was confusing, but what's wrong with using inheritance if you're writing in TypeScript (or, for that matter, a JavaScript version with classes)?

<comment><USER10> Too much abstraction can be bad as well, many times I've written a nice class diagram to discover no one really needed to reuse certain functionality, a function would have sufficed :P

</comment></comment></comment></comment></comment><comment><USER11> I'm assuming this is aimed at people wanting to be hired by companies with large teams as a developer where you can focus mostly on a single area?

If you're working freelance, as a consultant or building a product, people are paying you to spare them the technical details. They don't care what language you claim to be a guru in and you shouldn't shoe horn in your preferred tech stack when something else is more appropriate. Also, if you frame yourself as just "an expert in language X", you're only setting yourself up as a replaceable commodity.

I think having the skills to understand business goals combined with the ability to design and develop solutions where you're flexible about what tech stack is used is much more valuable to businesses.

Also, I don't understand the false dilemma of having to choose between being a "jack of all trades and master of none!!!" vs "super guru in one area". It's very practical to get to expert level in several languages and frameworks, especially given the similarities between many of them. If you spend all your time on one language/framework, you're going to quickly reach diminishing returns in terms of what you're learning vs how useful in general it is.

How much better are you really going to get at Python after spending 1 year, 5 years and 10 years with it? Surely at some point you're just learning obscure tricks or the ins and outs of libraries that are going to be outdated in a year or two anyway? Why is that valuable compared to being more well rounded and learning transferable skills?

If someone has been programming for a long time and they only list they're highly proficient at one language, that's a red flag to me they lack perspective and are unlikely to be able to architect solutions that require the bigger picture.

<comment><USER12> _I think having the skills to understand business goals combined with the ability to design and develop solutions where you 're flexible about what tech stack is used is much more valuable to businesses._

Very well-said, and needs to be repeated here every time someone posts on HN " _What language, framework, shiny-new-object should I learn to get the best job..._ "

</comment></comment><comment><USER8> I'm a full stack developer. I thought I was going to agree with the article, at least for some parts, but I actually disagree completely.

What the article says is true, but you could replace fullstack with any other role, name the article "You'll never be the best" and the article would still be true.

Even if you specialise in something, there will still be others that are better (and if they ridicule you, they are just assholes) and more things to learn anyway.

The author is mentioning a lot of specific technologies (react, databases, python) that, even as a pure "backend engineer" or even a "database specialist" would still face the issues mentioned in the article.

Specialising and going deep is great, as staying broad and doing more stuff. You just need to understand where you stand and what your strengths and weaknesses are. You can always specialise more, and more, and more. Specialise in backend, or actually .NET development, or actually databases, no I mean SQL, actually I mean scaling SQL databases, etc.

</comment><comment><USER13> Don't listen much to this guy. Full stack developers aren't doomed to fail, the only thing that happened here is that _this_ guy failed miserably. He failed because he focused on the wrong things.

If you want to be a full stack developer, the quickest way to get there is to build an entire application all through it's layers by yourself. Front end, backend, and databases. And the more complicated and demanding that application is the faster you will learn. Learn how to do migrations and backups, learn security concepts and how to harden your application, learn about the intricacies of UX and UI design. Know the seven layers of OSI and get your hands dirty with them. Learn actual system programming. Stop using fucking ORMs for your SQL.

Don't do what this guy did and master languages deeply and then think that's what makes you full stack. You could actually get away with lesser understanding of languages and technologies if you are very skilled in knowing how they fit together.

<comment><USER14> This is a good technical approach but if you get this far, your salary won't match the value delivered by your actual skillset.

Companies know this. There's not appropriate compensation for people who can get comfortable with this breadth of stuff. Fullstack is currently "exploited" because of this IMO.

<comment><USER15> That is true. Your salary will most likely not reflect your actual value, but will be way below what you should actually earn.

But - if you do it right (and with a bit of luck), you can work yourself into a position in which you've got a lot of freedom to choose what you want to work on, and that freedom is worth a lot in itself. Because you're the one guy that can "get everything to work smoothly together" and that is able to deliver entire working solutions, while practically all the others may be able to get a single part into good condition, but they stumble when it comes to getting to something working in a greater context. Their solutions will inevitably be less polished than yours.

People are not indifferent to this. It takes a while, but they notice it. And you will be in high demand because your solutions have this really nice tendency to "just work". And nobody will really understand why it is that way. Which means you'll be able to largely direct what you're going to work on next, because since nobody has a real clue why your stuff "just works" all the time, they are doomed to rely on your opinion when it comes to where your specific skill set is most valuably used next.

<comment><USER16> This is the direction that my career is taking. Definitely the benefit is you eventually get to work on the good projects that you want to because people trust you to deliver a level of work they don't seem to get very often.

I've always felt relatively underpaid compared to what I wind up bringing to the table, but I would say there have been subtle but noticable perks in other ways and it's definitely translated to comp over time too.

It clicked for me a while back that the vast majority of devs just working on line of business applications have never and will never put together an entire enterprise grade system end to end by themselves in their career. So all you really have to do is (a) have a relatively good head on your shoulders and (b) just hack away at building out a fully fledged system end to end in your own time. A sort of reference architecture of your own. Then when you need to implement stuff at work, you've already solved the problem once before so instead of taking a day to wrap your head around how a particular part of it works, you just whizz through it relatively quickly. It looks like magic from the outside, but it's nothing more than being prepared ahead of time.

</comment></comment></comment></comment><comment><USER17> The author doesn't understand the actual value of a full-stack developer.

A full-stack developer is not a 10x ninja developer in every language/technology in the "stack". A full-stack developer is a developer that has good enough understanding of a stack of technologies (say Django for the backend and Vue for the frontend, maybe Swagger in the middle, and PSQL for the database) to help the team fix integration issues between each layer of the stack.

<comment><USER18> "Full-stack" is the jack-of-all-trades. Ergo master of none.

<comment><USER19> In truth it's more like jack of all trades, master of one or some. 20 something years ago you didn't hear the term full stack, you did everything — database to UI — because that's just what you did.

These days as a consultant, I tell people I'm full stack with more emphasis on the back end.

<comment><USER16> I like to say "some parts of the stack are fuller than others"

</comment></comment></comment></comment><comment><USER20> The comments here are (predictably) negative, but it takes a lot of self- awareness and courage to not only say, "I used to think X, but now I think that was a mistake" but also to publish it publicly and invite feedback and ridicule.

<comment><USER21> Though the author seems to be laser focused on career progression and $$$ (I suspect due to cultural bias's)

"I had practical skills in all aspects of professional software engineering." I don't see any layer 1-3 skills there.

I would expect a full stack ie layers 1-7 developer to be able to design and get implemented a small network.

<comment><USER16> Not really?

It typically just means front-end, back-end, database. Not networking? I know it says "full" but it's not that full.

<comment><USER22> They also have to build their own computer and a small generator that creates electricty, and probably their own wifi router. otherwise they're mediocre and don't understand how stuff REALLY works.

<comment><USER21> I do seem to recall as a child making a small generator Dad got me a kit designed for kids for Christmas on year.

Though I think Layer one assumes power and the pys media exists :-)

<comment><USER23> Power is assumed, yes, but it deals with physical media directly. That's what 1 is. You make or design the RF level hardware. Layer 2 is where PHY is a separate entity - driver stack starts at the lowest point of this. (Used to be higher but now more things are offloaded to firmware or hardware.)

</comment></comment></comment></comment></comment></comment><comment><USER24> I think on one level the author has a point. Fullstack developers make sense for early startups who can only hire 1 or 2 developers, or who only are 1 or 2 developers. However once you start growing and probably need to rebuild your app, it's time to hire multiple specialized developers.

If an established company is recruiting fullstack developers it could be because there is a manager who thinks, "Why hire two developers when I can hire one?" Which is magical thinking as there is always a trade-off.

</comment><comment><USER16> This guy is painful. What a junior attitude. And please never "retrain for management". Just don't. Please don't. You are exactly the wrong kind of person for it.

</comment><comment><USER25> If you expect to be a generalist by somewhat half heartedly knowing how to use a number of technologies you're doing it wrong. What makes you a generalist is not just being able to use tools, but rather understanding how they work. That means for web frontend work you should rather step towards understanding how the dom and v8 works instead of jumping on the newest hyped framework or instead of just learning react's api maybe also understand how and why they are building their own virtual dom. If you do backend work you eventually want to look into compilers and understand the difference between interpreting, compiling and jit-ing code. If you do database things you may want to start thinking about how you would implement your own and touch on things like transactions, consistency, how a query optimizer works or what structures data is stored in. Not that you will ever be tasked with building your own frontend library, language, compiler or database; but all that helps you immensely to understand and evaluate the tradeoff space and also quickly jump on the newest hyped thing if it actually is a good idea.

</comment><comment><USER26> The codebase at my job is usually complex - Python, Ruby and Go, along with JavaScript/Typescript and a little bit of Java and R. I used to be in the "all programming languages are basically the same" camp but jumping back and forth between technologies has given me more appreciation for the details. Some knowledge I've wished I had at my fingertips: semantics of Go channels including all the edge cases, the scoping semantics of Ruby blocks, anything in the JavaScript standard library, the ecosystem of Java build tool and helper libraries. Having to stop and look up things lowers my development velocity compared to a specialist. In the other hand, as a generalist I can trace a request across component boundaries and go as deep as needed into a system to diagnose issues. So I think it depends on what kind of projects you are working on. A "serial generalist" that goes from one tech to the next is always going to be behind the curve, but that isn't really what I would think of as a full stack developer, either.

</comment><comment><USER27> Another side of this is that there's demand for full stack competency. Even if it isn't perfect comprehensive knowledge, it's enough to get difficult jobs done. I think this person was striving for perfection where it's not really feasible for most people, or even necessary for most jobs.

I've tried several times in my career to focus on a skill but because I was able to do many other things, I was asked to by my employer. I rarely say no. My last job was originally supposed to be 'make these products faster' with a focus on the backend, but it turned out the frontend and anything in between the two was as much a part of the problem. Not only did I fix issues on the server, but I re-architected the frontend to a degree, moved all assets to a CDN, significantly improved the js code's tree shaking, code splitting, and lazy loading, tackled minutiae like compressing images better, using a faster and more secure alpine image, and the list goes on.

I didn't really want to do that stuff, but I could and it needed to be done. And the reality is that even though I'm not a master of anything I do, no one else at that company had done those things despite having the opportunities (indicating they didn't know how or didn't even know it needed to be done), and the main application I worked on was sped up a ridiculous amount. I remember we had accounts that took an actual honest to dog minute or more to perform an action, and when I was done it did the same thing in a few seconds. It was a game changer.

The people I worked with were awesome at what they did, and in a sense I was only standing on their shoulders. But they don't understand the entire stack well enough to connect the dots like I did. So that's where full stackers really shine, I think. Sometimes I think I get too much credit on teams for my contributions because all I do is make sense of things and break it out into tasks that most of the team is already able to do. But I suppose there is real value in bringing the bigger picture together for your team.

</comment><comment><USER10> Once my architect wanted everything on python so I obliged, because of the size of the data I went with spark, but since we were using python I had to interface with it using pyspark, which still uses python 2, after a few months we start having all this problems with the py4j bridge in spark collapsing and us having to restart the service every few hours.

Fed up with the situation I rewrote the thing in Java/Scala and we haven't touched the service ever since.

My point here is that using the right tool for the task can mitigate a myriad of issues, all the python knowledge in the world was not good when working with a java tool.

What is needed is balance, not every language is good for everything, not every framework can do it all and certainly not one engineer knows it's all (either the individual contributor or the manager)

<comment><USER28> hi, when did all this happen? isn't pyspark almost on par with scala implementation now ?

</comment></comment><comment><USER29> As a fellow full stack developer, I can relate. I have been called out for using Optional too broadly in Java because I was trying to treat it like Rust, and I too need to research specific syntax and API on a daily basis. On the other hand, when our company was bought out and I had to change technologies fast I think it was an asset. A year and a half in and I am clearly a senior dev again. Not the owner of any domain anymore, but like the article says, someone they come to to bounce ideas off of, and someone who can identify the cross domain architecture deficiencies with a concept of how to fix them. Also because I have to research constantly, once in a rare while I stumble across an elegant solution the domain experts weren't aware of.

</comment><comment><USER30> At my last startup, we tried full stack development in conjunction with pair programming. Fullstack dev is hard essentially have to be great at both back and front end and some UX. Pairing helped. In the end many didn't want to continue preferring to specialize. That was a shame because like pairing, full stack is a nonstop dev flow, no pauses for talking to other 'end' nor code review delay.

Because it's hard to build that culture it's ew rely done and so there's no market for it. What full stack means in the market is front end with a little back end and doesn't pay specialty rates. Actual full stack is a specialty just so rare it's not even 'a thing' in the minds of employers.

<comment><USER31> > What full stack means in the market is front end with a little back end

So much this. I believe in full stack development skills, that someone can have them, but I don't believe in the job role of full stack developer.

At small medium scale, there's just so much more front end work on a lot of apps. Let's say 70%. My experience with full stack as a job role is that it turns into "what backend change can I quickly make in order to get back to the front end work which is most of the work." Which then means no ones manning the ship / design / scope creep / perf risks etc of the backend. I think separate roles lead to more even evaluation of best feature implementations.

</comment></comment><comment><USER32> Unless one literally does not care about computers AT ALL, and is content with front-end development, having no hobbies involving any OTHER kind of development etc, one will end up being a bit of a "full stack developer".

I, frankly, don't trust career coders who don't have an active interest in the broader activities around coding, software, hardware, maybe HDL's and FPGA's, or low-level stuff with microcontrollers etc.

as it appears that a career coder who specializes and then expresses no further interest in branching out is merely milking a perceived cash cow and not going to show an interest in how even the specialized piece they are building inter-relates with the other pieces of the puzzle.

<comment><USER33> > I, frankly, don't trust career coders who don't have an active interest in > the broader activities around coding

I think that's unfair and ends up feeling a lot like signaling to me.

Some of those things I'm very interested in, but there are tons of development things I'm just not excited about. I don't care about containerization trends, about kernels, not all that interested in chasing the new shiny front-end stuff. I am curious about FPGAs but don't have enough reason to go farther than that. I know devs who do not care about machine learning, about databases, whatever, but are very adept at their areas of interest. If I talk about my hobby work with robotics and microcontrollers and another dev isn't interested at all, I'm not going to judge their general skills.

Expecting people to be excited about anything related to computers, hardware, general engineering isn't very realistic. It's such an amazingly broad cross- section of fields and interests.

<comment><USER34> I think OP was saying he would be suspicious of a coder who was not interested in at least one area outside their professional domain, not any given area. If that's the case then I have to say I agree with them.

</comment></comment></comment><comment><USER35> One problem is that people think about seniority as expert level in something.

In my terminology, when you are a senior you need to be a near expert in whatever you call yourself a senior within.

If that's .NET/C# like the author refers to, you must know how all (the esoteric) features of the language (e.g C#) behaves, you must know IL, WinDBG for advanced debugging and the things along those lines.

So many developers claim something that they are absolutely not, which is experts or senior in a technology.

It's IMO much harder to become an expert in something than to learn a subset of a set of technologies. Much harder actually.

<comment><USER16> I would class your definition as expert rather than senior. It's a level above what I'd expect.

Then again seniority has many dimensions. I often see a lot of senior devs who have mid level skills or mindset but they have become domain experts or experts in the product and seniority is conferred that way. I don't necessarily think that is wrong or undeserving either. I guess if we are talking absent any business/domain knowledge then seniority is judged more closely off raw technical familiarity. For me it's about system design/architecture. Having a strong set of principles that guide your decisions etc.

</comment></comment><comment><USER32> Here's some additional unsolicited advice: Work on your own skills instead of dishing out advice on how to approach systems you appear to only superficially understand :)

</comment><comment><USER36> I also have the same question about how to improve my technical skills. I don't think it is related to full stack or not. I found that most of the companies require developers with mid-level skills only. I don't have enough chance to improve because there are no difficult problems in my job. I would say if you think you are not good enough, simply move to companies which give chances to solve difficult problems.

<comment><USER37> The thing with programming, contrary to pretty most other fields, you have any possibility to do something diffcult in your free time!

</comment></comment><comment><USER38> Personally, I'd prefer to have a broader albeit less competitive set of knowledges as a full stack (this should be defined, though) developer. I do not see it as a flaw.

Back to my backend days I was on the brink of the burnout but switching to frontend gave me breath of fresh air. When I found a modern frontend as a total mess I started to dig into devops and it surely improved my competencies as a backend either.

</comment><comment><USER22> I see no reason why a backend web developer wouldn't know html/css and pure js to an acceptable level. You don't have to be a Node.JS ninja on the frontend but definitely have a good grasp of what's going on on the frontend. People who can only do one thing will have a harder time leading teams or going to management.

</comment><comment><USER39> I have a feeling that all developers working on product would become full- stack developers eventually. Probably the dichtomy of backend v.s. frontend would become product v.s. infrastructure. Product developers support business units while infrastructure developers support product developers.

</comment><comment><USER40> One take away, from the article and comments here. There is no industry agreement as to what a junior or senior role is. Everyone seems to place importance on a different set of skills.

</comment><comment><USER41> I suspect the author really wants to quit management and roam the world on two-week niche contracts, but is too afraid to do so? </s>

</comment><comment><USER42> Is a snarky, jpeg code snippet at the beginning of an opinionated tech article the new snarky, pop-culture gif?

</comment><comment><USER43> this is nonsense

</comment><comment><USER44> I agree with the first part of the title, the only thing I've read.

I completely disagre with the second part: full stack developers are not stuck because full-stack is bad per se, they have chosen to not master one single skill, preferring instead to learn many of them, with a varying level of confidence, from master to "I can learn it, if needed".</comment>


U.S. Relaxes Limits on Use of Data in Terror Analysis  - <AUTHOR> http://www.nytimes.com/2012/03/23/us/politics/us-moves-to-relax-some-restrictions-for-counterterrorism-analysis.html?_r=1&hp

<comment><USER2> Headline is a bit euphemistic in how it describes a civil rights trampling police state power grab:

> The guidelines will lengthen to five years - from 180 days - the amount of > time the center can retain private information about Americans when there is > no suspicion that they are tied to terrorism, intelligence officials said.</comment>


Ichiro Suzuki's return to the Seattle Mariners won't resolve his internal battle - <AUTHOR> http://www.espn.com/espn/feature/story/_/id/22624561/ichiro-suzuki-return-seattle-mariners-resolve-internal-battle?sf183894966=1 <comment><USER2> that was magnificent. does a great job illuminating the dark side of the sort of singular, compulsive pursuit of perfection that we tend to fetishize around these parts... his path is certainly not one for the faint of heart.

<comment><USER3> I don't even like baseball but that was an excellent article.</comment></comment>


How many replacement characters? - <AUTHOR> https://hsivonen.fi/broken-utf-8/ <comment><USER2> A well laid out case. Given the churn the new change in preference it might cause in the ecosystem, it's probably worth reverting, especially since apparently no real benefits beyond "it feels right" (according to this accounting) have been put forth.

It may cause the committee to lose a little face, but less than digging in your heels over a decision that has no ramifications to you but does to others, and you have no real justification for. Hopefully they see that and acquiesce, or at least come back with a well thought out rebuttal that isn't dismissive.

</comment><comment><USER3> I completely agree with the author on this. The old Unicode 9.0 best practice made sense for a UTF-8 decoder that consumed input using a state machine, and like the article says, if you accept the correct byte-ranges in each state as in Table 3-7, your state-machine-based decoder will implicitly reject every kind of invalid sequence, including overlong encodings, encoded UTF-16 surrogates and encoded out of range characters. A state machine also makes other things trivial, like validating UTF-8 input without outputting characters and writing a streaming UTF-8 to UTF-16 converter. Another property is that it rejects invalid input as soon as possible, eg. as soon as it consumes input that can't possibly be part of an invalid sequence.

This is an example of a particularly elegant UTF-8 decoder which uses a state machine: [http://bjoern.hoehrmann.de/utf-8/decoder/dfa/](http://bjoern.hoehrmann.de/utf-8/decoder/dfa/)

The proposed Unicode 11 best practice suits a decoder that checks for overlong sequences, encoded surrogates and out of range values as a sort of post- validation step, after consuming all the bytes of a potentially valid UTF-8 sequence. Not only is this different to the behaviour of every existing decoder except for ICU, it also seems less elegant, more complicated and more error prone to me. If I understand correctly, even bytes like 0xc0 and 0xf5, which never form part of a valid UTF-8 sequence, won't be rejected immediately in this kind of decoder.

The article makes a pretty solid argument for why this difference in behaviour matters, even though it's just a best practice and not an official requirement. The two key points in this for me are that most existing UTF-8 decoders produce identical results matching the Unicode 9.0 best practice, and that there was an actual bug in Chrome when two internal UTF-8 decoders produced differing results. I think I'd make a stronger conclusion though: Not only should they keep the current recommended best practice, they should also elevate it to a requirement in order to prevent bugs like the one in Chrome from happening in future. Most existing UTF-8 decoders are already compliant, and it would be a nice property of UTF-8 if all byte sequences, including invalid ones, decoded to the same sequence of codepoints in every decoder.

</comment><comment><USER4> How I have implemented things is that when the UTF-8 decoder encounters an invalid sequence, it retreats to the beginning of that sequence. It then converts the first byte of that sequence to a replacement characters, and consumes it. Then it resets to its initial state and begins decoding starting at the following byte.

Basically we are saying "no match occurs for a valid UTF-8 pattern at this input position; let's do error recovery by dropping a byte, pooping it out as a replacement character and trying again."

I have used the surrogate pair range U+DC00 - U+DCFF for replacement characters. On output, I convert these back to individual bytes. Thus the end- to-end decode+encode is binary transparent: any byte string can be decoded to a sequence of code points, some of which may be replacement characters, and that sequence will encode back to the original byte string. (This requirement cannot be achieved if multiple bogus bytes are collapsed into one replacement character.)

Well, that's not the full story: to have this transparency property, we also need to ensure that when some U+DCXX occurs by means of a valid UTF-8 pattern, we nevertheless treat it as invalid. I.e. there is a rule that if the UTF-8 decode works, but a U+DCXX code-point emerges, then we retreat to the start and drop a byte as a replacement character, as if a bad code had been seen.

<comment><USER5> Your UTF-8 decoder is identical to just "emit a REPLACEMENT CHARACTER for every bogus byte".

Also, the use of surrogate pairs mean you actually don't have a UTF-8 decoder at all, you just have something that's similar but produces a sequence of potentially-invalid unicode scalars.

> _we also need to ensure that when some U+DCXX occurs by means of a valid > UTF-8 pattern_

It can't. Trying to encode surrogate pair codepoints in UTF-8 is strictly invalid.

<comment><USER6> > It can't.

Why did you cut off "we nevertheless treat it as invalid."? You're misreading that line, and actually in agreement on what to do. It's the bit pattern that is valid, which is how you can parse the code point and see that the code point is invalid for UTF-8.

<comment><USER5> I cut you off because you basically said "and if we see this thing that is defined as invalid, we treat it as invalid". Which is something that all UTF-8 parsers do, so there's no point in calling it out like this is special behavior.

<comment><USER6> That's not me.

The point is that this variant parser accepts many things that are normally errors and turns them into faux-surrogates, so it's worth restating that it rejects surrogates in the source file.

<comment><USER5> My apologies.

In any case, here's what OP said:

> _Well, that 's not the full story: to have this transparency property, we > also need to ensure that when some U+DCXX occurs by means of a valid UTF-8 > pattern, we nevertheless treat it as invalid. I.e. there is a rule that if > the UTF-8 decode works, but a U+DCXX code-point emerges, then we retreat to > the start and drop a byte as a replacement character, as if a bad code had > been seen._

But this whole paragraph is wrongheaded. A U+DCXX codepoint _cannot occur_ as a result of a UTF-8 decode, because it is defined as invalid. Even just that last bit there, "as if a bad code had been seen"... a bad code _was seen_! This paragraph makes me question whether OP actually understands UTF-8 decoding at all.

<comment><USER6> I think the meaning is clear. They're talking about _part_ of the UTF-8 decoding process, the part that actually decodes leading/trailing bytes into arbitrary 21 bit numbers. What term would you use for that?

<comment><USER5> Well, I wouldn't even bring it up, because the UTF-8 decoding process by definition cannot produce code points in the surrogate pair range, as those are illegal to encode in UTF-8. But if I must, I might talk about the "bit pattern" and the integral value that results from interpreting it. I certainly wouldn't talk about code points resulting from a UTF-8 decode.

<comment><USER5> Somebody is apparently upset about what I said. Why?

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> The author found it hard to "find the right API entry point in Go documentation".

For the record, Go produces one U+FFFD per byte, not per maximal contiguous run, when iterating over bad UTF-8. This is part of the language specification, not just a library, although the standard libraries follow this behavior. For example, in the standard UTF-8 library, [https://golang.org/pkg/unicode/utf8/#DecodeRune](https://golang.org/pkg/unicode/utf8/#DecodeRune) says that the size returned is 1 (i.e. 1 byte) for invalid UTF-8.

The relevant language spec section is [https://golang.org/ref/spec#For_statements](https://golang.org/ref/spec#For_statements) and look for "If the iteration encounters an invalid UTF-8 sequence, the second value will be 0xFFFD, the Unicode replacement character, and the next iteration will advance a single byte in the string."

Example code: [https://play.golang.org/p/OLIWcjLIvF](https://play.golang.org/p/OLIWcjLIvF)

I'll note that both Go and UTF-8 were invented by Ken Thompson and Rob Pike. I'm sure that the Go authors were aware of UTF-8's details. (Go also involved Robert Griesemer, but that's tangential).

<comment><AUTHOR> Thank you. I was looking for something that takes a potentially invalid buffer of UTF-8 and returns a guaranteed-valid buffer and failed to find a function like that.

(And, indeed, Go is an interesting case due to its creators being the inventors of UTF-8, too.)

<comment><USER7> Yeah, there's not really a guaranteed-valid buffer concept in Go. Even if you have valid UTF-8, you still have to iterate over it to e.g. rasterize glyphs, and iterating over possibly-bad UTF-8 is no harder than iterating over known- good UTF-8.

If you want to compare to other UTF-8, validity alone isn't always sufficient. You often have to e.g. normalize anyway, and normalization should fix up bad UTF-8. Again, a guaranteed-valid buffer type wouldn't win you much.

</comment></comment></comment><comment><USER8> Thought: what if Unicode decoding was "lossless" in the face of errors, such that the replacement characters _represented_ the bitstring of non-decodable bytes? (E.g. 256 reserved-codepoints for each possible octet value, that render as e.g. "[FF]" in a box; and then another 255 for the set of 7-bit, 6-bit, 5-bit, etc. overhangs.)

<comment><USER4> been there, done that; it is very useful.

      1> [(file-get-string "/bin/ls") 0..15]
      "\x7F;ELF\x02\x01\x01\xDC00\xDC00\xDC00\xDC00\xDC00\xDC00\xDC00\xDC00"
      2> (file-put-string "foo" (file-get-string "/bin/ls"))
      t
      3> (sh "cmp /bin/ls foo")
      0
      4> (sh "sha256sum /bin/ls foo")
      a90ba058c747458330ba26b5e2a744f4fc57f92f9d0c9112b1cb2f76c66c4ba0  /bin/ls
      a90ba058c747458330ba26b5e2a744f4fc57f92f9d0c9112b1cb2f76c66c4ba0  foo
      0

There is no need to handle any fractional bytes if the original input is a sequence of bytes; so many whole bytes have to be recovered, not so many whole bytes plus three bits or whatever.

<comment><USER9> I might be sleep deprived... but what exactly is this script doing?

You're making a copy of /bin/ls into foo, and sha256sum the copy and the original.

        $ head -c 15 /bin/ls
        $ cat > foo < /bin/ls
        $ cmp /bin/ls foo
        $ sha256sum /bin/ls foo

I don't get it.

<comment><USER4> > _You 're making a copy of /bin/ls into foo_

By getting its contents as a character string formed by passing the binary through a UTF-8 decoder, and writing out that string via the UTF-8 encoder.

</comment></comment></comment></comment><comment><USER10> > The proposal is ambiguous about whether to do the same thing for five and > six-byte sequences whose bit pattern is not defined as existing in Unicode > but was defined in now-obsolete RFCs for UTF-8 [...] If five and six-byte > sequences are treated according to the logic of the newly-accepted proposal, > the newly-accepted proposal matches the behavior of ICU.

Regarding 5- and 6-byte sequences, perhaps the Unicode Consortium in their ambiguity and the ICU in its implementation are allowing for their possible return to Unicode. One day in the far-off future when UTF-16 finally dies, it will be feasible to increase the codepoint repetoire back up from 1 million to 2 billion, which is easy to implement in both UTF-8 and UTF-16.

</comment><comment><USER11> This raises what appear to be important points. Has it been formally submitted to Unicode in some way?

<comment><USER12> Yeah, there's a long discussion about this on the official Unicode mailing list recently. You can read it at [http://unicode.org/pipermail/unicode/](http://unicode.org/pipermail/unicode/)

<comment><USER13> Link to the thread: [http://unicode.org/pipermail/unicode/2017-May/thread.html#53...](http://unicode.org/pipermail/unicode/2017-May/thread.html#5389)

</comment></comment></comment><comment><USER14> It seems like only harm is that "implementations have to explain themselves" and one Chromium bug.

<comment><USER5> The Chromium bug is a demonstration of the fact that different behaviors in different implementations can lead to real bugs. There's no reason to think that one Chromium bug is the only time this will ever matter.

<comment><USER6> It says in the spec that the number of replacement characters can vary, so it's hard to blame Unicode for a bug caused by two parsers making different numbers.

(Unless the argument is that there should be an official required number, which is a different discussion entirely.)

<comment><USER5> No, you're right in that it's perfectly legitimate for multiple parsers to behave differently here. But if there's one behavior that nearly all parsers have standardized on, that's very valuable because it makes it a lot easier to use two different parsers without a problem.</comment></comment></comment></comment>


Ask HN: What are some geospatial tools I can use to solve this problem? - <AUTHOR> I'm the tech lead on a new project that is out of my domain experience. At my company we have a fleet of drivers who checkin via an app with their lat/lng coordinates. We have pickup locations (whose locations are already known) and dropoff locations (whose locations are not known until an order is placed). We have a google maps for business license so we can query for directions with traffic durations and address to lat/lng. The key problems this service needs to solve are:
1) Given a new order, who will be the fastest driver to arrive at the pickup?
2) Use our own historical driver data to estimate drive times in the future.
3) Caching and solving similar problems
4) In general, using our own data so we are not just querying google for everything
So far I've looked at postgis, simple caching in redis, redis-geo, and solr spatial search. I'm just curious if anyone has any ideas on the topic. Thanks! <comment><USER2> pgRouting[0] is an extension for PostGIS that provides some routing functions.

looks like they had a further extension solving a very similar problem (Dial- a-ride-problem, DARP[1]) in their 1.x versions. I can't find it in their current docs, but it may be just an issue of how the reference it. Take a look through their stuff, it sounds like it'd be very helpful.

[0] - [http://pgrouting.org/](http://pgrouting.org/) [1] - [http://gis.stackexchange.com/questions/5330/route- optimisati...](http://gis.stackexchange.com/questions/5330/route-optimisation- for-multiple-vehicles) (see first answer)

</comment><comment><USER3> If you want to do it in-house, you can geocode the customer addresses through geocoder (using Google's API) and use SOLR or elastic search to store the latitude and longitude, then do geospatial queries with all the drivers' current coordinates to find the closest driver.

I believe SOLR can handle pretty fast indexing with soft commits, so latency of indexing new records shouldn't be an issue. Not sure about elastic search.

You can always pay Google

Google Distance Matrix: [https://developers.google.com/maps/documentation/distancemat...](https://developers.google.com/maps/documentation/distancematrix/)

</comment><comment><USER4> Postgis probably easiest/best overall. You don't need to do routing or TSM? In a native form for one use spatialite works.

The routing piece is the hardest. But if you don't have to do that then you just need something that stores groupings. For that matter if the points are a finite list that gets loaded as new then you don't even really need to store it as spatial data. Just list of drivers, destinations, times and you should be able to determine who gets where fastest.

</comment><comment><USER5> osm

<comment><USER4> OSM if you mean open street map is just display. Osrm-- open street routing might work if you need routing. But postgis has routing modules and you could call Google for the routes and store.</comment></comment>


Ask HN: Is AI going to replace web developers? - <AUTHOR> Looking at the trends, AI seems to be breaking new boundaries but is it possible that web development jobs in the future may be handled by AI? <comment><USER2> TL;DR: Yes it's possible, but it doesn't actually matter if it happens or not.

An AI creating websites is definitely possible, but will it actually replace people? I don't see that happening (at least not at a large scale). It's much more likely to be a tool that developers use rather than a tool that replaces developers. Or possibly something similar to Squarespace. It might be able to manage content on its own and build webpages, but what happens when there's a bug? What happens when you want to innovate and create something new? An AI would have trouble accomplishing that, so web developers would still play a role.

Now the more important question in my mind isn't whether or not they will replace jobs, but rather does it matter if they do? And my answer to that is no. There's two scenarios. A scenario where AI completely eradicates all developer jobs, in which case AI has probably eradicated a majority of today's current jobs and we would adapt as a society. Either by instituting some changes like Universal Basic Income or by developing new job markets (probably geared more towards the arts or things we can't yet imagine). The second scenario is that AI is introduced more as a tool or a simple solution to web/app development. In that case, there will still be a need for devs to do the more complex stuff. There will also be other development fields to move into, like Machine Learning Engineer for example.

</comment><comment><USER3> Doubtful. Most of development is discovering, understanding, and refining requirements that may not all be known in advance, and then translating those requirements into deliverables. This is an intricate process subject to information asymmetry, and AI/ML and related fields are fundamentally reliant on a trove of accurate data.

</comment><comment> a-saleh I think it will be another tool in the toolbox.

I.e. I remember that few years ago, large part of web-dev was getting a page design in image files and you sliced and diced them and created proper html, maybe with few placeholders, with css optimized both for desktop and mobile.

Then I have seen a thesis on my uni, that automated all of that with genetic algorithms. You just threw an image at it and few hours in it spews out optimized web-page. I wouldn't be suprised if current photoshop has a button for it :)

<comment><USER4> Tangentially related to this point, when tools like Rails came along, it seemed that web applications would become trivialized to the point where "anyone can do it". In an environment where anyone can be a web developer, do you need a web developer? It turns out you do, fortunately.

Now, Artificial general intelligence (AGI) is a threat to everyone's jobs, just about, but I can't see an AI short of AGI replacing a web developer's cognitive processes.

</comment></comment><comment><USER5> It won't likely replace people any time soon, but if it's anything like other technologies, it will make workers more productive (reduce jobs) or the process easier to learn (larger pool of worker -> lower wages).

</comment><comment><USER6> Very probably, if web development is seen as a closed game with a limited set of rules and numerable outcomes.

</comment><comment><USER7> Yes it will. The question is, when?

</comment><comment><USER8> You may have heard of The Grid, a new crowdfunding campaign that would revolutionize website design by incorporating Artificial Intelligence as a design tool. The service is similar to SquareSpace or WIX but include algorithms that essentially analyze a user's content and imagery to calculate the ideal website design based on a set criterion. Any new content that gets added will result in the entire site reorganizing itself on the fly. Sound too good to be true? Perhaps. But interactive designers and developers have been worried about pre-made templates and DIY client services for years. Some of us fear being replaced. Some of us welcome our new robot overlords. Some of us, myself included, see these new services as a chance to evangelize that there are extremely deep, unique User Experience challenges that a web template (created by a bot or not) just can't solve. In the Beginning Human-Robot Interaction has been a topic of both fascination and fear since the time of antiquity. Homer's Iliad described the creation of automatons by the Greek god Hephaestus, who built consciousness into golden statues. Aristotle proposed that human thought and logic could be mechanized. Nineteenth century inventors and artists perfected self-operating toys and machines. These concepts would be lifted to a new plane with the invention of the programmable digital computer in the 1940s and subsequent founding of the field of AI research at Dartmouth College in 1958. Our relationship with AI has always been pretty tenuous. As in all instances of paradigm shifts and technological revolutions, we're at odds with making our lives easier while feeling a dull sense of displacement or irrelevance. Harkening back to Isaac Asimov's Three Laws of Robotics, we've attempted to safeguard ourselves from AI completely replacing or harming us. But replace may not be the right sentiment as it relates to the Web. We've seen design trends, languages and methodologies come and go. Most often they're superseded by far more efficient ways of doing things and conveying information. Could the introduction of Web AI fall into the same path of progress or are we entering a new era of design? Not for Everybody The Grid promises to free us from the shackles of design and code. It promises that all of our problems are to be solved via a set of constraints and templates, whether they're conceived from an algorithm or otherwise. While I think AI and template sites will benefit from small businesses and casual users who are on a tight budget, they completely ignore all of the work that User Experience designers and information architects have done in solving problems from an empathetic, strategic and tailored point of view. When we talk about web design, we're talking about something much more far-reaching than what the end-user sees. Since the design is freed from static, printed paper, the screen is able to contextually change what is displayed based on environment and feedback. How we get to that point of context is the task of a UX designer. Defining who the user is and his or her goals, and what a client's business goals are, helps inform the final design. It's beyond the ability of an algorithm to figure that out. We just aren't there yet. We haven't successfully taught machines how to think and behave like real human beings, so until we can do that, we have to do the dirty work ourselves. And web designers love those really tough problems. All of the nuances, the complexity, and the compromise. Another issue with an AI-based website is how much control we really have over its output. You essentially abdicate the big decisions for the system to make, resulting in a site based on its own assumptions and programming limitations. (Side note: the code the AI spits out is so poor that you're likely going to be dinged on SEO crawls!) I don't think we're ready to hand over our thought processes as they relate to defining a unique face and voice online either. Although interface conventions are becoming more homogenized, the need to outline a brand's differentiators in the interactive space are at an all time high.</comment>


It's 2059, and the Rich Kids Are Still Winning – Ted Chiang - <AUTHOR> https://www.nytimes.com/2019/05/27/opinion/ted-chiang-future-genetic-engineering.html <comment><USER2> > _In a society increasingly obsessed with credentials, being genetically > engineered is like having an Ivy-League M.B.A.: It is a marker of status > that makes a candidate a safe bet for hiring, rather than an indicator of > actual competence._

Not specific to fancy MBA programs, but sometimes credentials means more than a safe bet, but also a fraternity or cabal.

Imagine that, by default, there's a level playing field for everyone, but a subgroup of people collude to trade favors amongst themselves. That group will have an advantage over everyone outside the group who are playing by the egalitarian rules.

<comment><USER3> Imagine that, by default, there's a level playing field for everyone, but a subgroup of people share some common background and therefore they can form trusted connections amongst themselves at lower cost vs the overhead to establish such connections between any two parties who don't know each other from Adam. I claim this not nefarious situation is indistinguishable from your sinister finger pointing.

<comment><USER2> How do you mean "trust"? Possibilities include:

1\. Trust that the person is capable at particular things. (This you might get from credentials, which perhaps also defines a group, and accomplishments.)

2\. Trust that the person is truthful with you, and will honor agreements with you. (This you might get from their reputation within your network/group.)

3\. Trust that the person will give preferential treatment to you/group, and that they do this _because_ there is mutual advantage to a subgroup doing that, separate from trust in ability and honesty. (Maybe sometimes this is called loyalty to the group, though you might say you trust in their loyalty. Or maybe it's merely understood.)

I'm pretty sure I've seen #3 happen in business/opportunities, read about it in news, and heard people talk firsthand about it, in ways not always covered by #1 and #2.

</comment></comment></comment><comment><USER4> >Yet we continue to ignore this because it runs counter to one of the founding myths of this nation: that anyone who is smart and hardworking can get ahead. Our lack of hereditary titles has made it easy for people to dismiss the importance of family wealth and claim that everyone who is successful has earned it.

Who your parents are has always mattered. Parents being able to give their children an advantage in the world is, in general, a good thing. Even the preamble to the Constitution of the United States talks about securing the blessings of liberty to ourselves and our posterity.

It is human nature to want to work hard to better your offspring. It is one of the main motivating factors for many people. Any attempt to remove this advantage is bound to fail.

We need to stop looking at individual opportunity as much as look at the bigger generational picture. As many have pointed out rags to riches in a single generation is very rare. However, rags to riches in 2 or 3 generations is not uncommon (see especially the experience with Asian immigrants). We need it so that anyone who is smart and hard working can end up better off than their parents, and ensure a better starting position for their own kids who if they are also smart and hard working end up even better.

Any Procrustean policy of trying to make it irrelevant who your parents are is always going to end in frustration.

<comment><USER5> One change I'd like to see is in the opposite direction: We (U.S. society) should stop assuming that someone who has wealth is automatically more meritorious than someone without. We've taken the concept that "someone who is smart and hard working CAN improve their situation" and flipped it to be "someone in an improved situation MUST be smart and hard working".

We should also watch out for what we consider "smart". Someone that is a complete manipulative and exploitative jerk can leverage that to become quite wealthy. They will defend it as "smart", and perhaps it is, but if don't consider that behavior valuable, we should stop pretending the wealthy cases are exceptions.

And of course, to agree with some of your points, we shouldn't toss the baby with the bathwater, nor paint with too wide of a brush. That said, the American narrative and American reality are somewhat dramatically out of sync right now, so some adjustment seems prudent. When the U.S., champion of socio- economic mobility, is ranked rather poorly at the same compared to other developed nations, we become delusional hypocrites, not champions.

</comment></comment><comment><USER6> This article isn't too surprising. Modern studies only find a small positive connection between IQ and lifetime success (0.25 correlation). Even so, there is no single element predictor that can better measure success compared to IQ. I would hope that by 2059, social scientists can find an accurate way to determine how being rich contributes to "winning".

<comment><USER7> Do they? My experience reading them doesn't indicate that at all, IQ is enormously important in determining lifetime success it's just there are different plateaus.

Once you achieve a certain IQ threshold (be it 115, 130, 145, whatever) then other factors play a bigger role but before you hit that threshold your outcomes statistically will be much worse than even the lowest performers who do make the cutoff.

</comment></comment><comment><USER8> This assumes that applying edits is hard and expensive while finding what to edit is easy and can be done without multiyear tests. But the reality is exactly the opposite.

More likely outcome is that genetic editing companies will be paying poor people to be able to test effectiveness of therapies on their children, and instead of direct payments will use long term contracts taking percentage from the child's future income.

And if you think this business model is immoral and horrible, notice that this is the business model governments are meant to perform with taxes and social payments, they just fail miserably because of their monopoly.

</comment><comment><USER9> I watched a very interesting TED talk related to this the other day. The premise is the question "Where in the world is it easiest to get rich?". The answer won't surprise you if I tell you the answer will surprise you.

[https://www.youtube.com/watch?v=A9UmdY0E8hU](https://www.youtube.com/watch?v=A9UmdY0E8hU)

<comment><USER10> Define rich.

<comment><USER9> He does.

</comment></comment></comment><comment><USER11> The way things seem to be heading, I'm afraid we're going to have much bigger problems by 2059 to worry about.

<comment><USER12> Unless the rich are going to start building domed cities for themselves, they might be better off making their designer children smaller, vegan, and tolerant of lower levels of oxygen.

<comment><USER13> How about robot children: \- No oxygen required \- No food required \- Can be made quite small \- 50% lower chance of uprising when compared to humans

<comment><USER8> "No food required" is quite nice. Technology for full robots may require quite some time, but before that it is probably possible to create artificial stomach that can replace your stomach and gut, and instead of eating 3 times a day and going to toilet you simply could replace a bottle once a week.

</comment></comment></comment></comment><comment><USER14> Thank you.

<comment><USER15> I clicked on the "web" link under the title and opened it in a private tab, which loaded Google search results, then I clicked on the NYT link and it worked.

Edit: the site reloaded to cut the article then say I'm in Private mode and to log in. I just went back, opened the NYT result again, and stoped loading the page once the text appeared.</comment></comment>


Grace Hopper organizers struggle to practice the diversity they advocate - <AUTHOR> https://techcrunch.com/2016/10/20/anita-borg-institute-grace-hopper-diversity/ <comment><USER2> It sure is easy to tear down the work of others when you can't do something important yourself.

It seems a reasonable question as to whether or not race is an important metric for an event that seems to be primarily devoted to gender diversity. This goes back to the whole debate about "white women's feminism" vs. the rest.</comment>


Richard Mack, the Republican challenger to SOPA's author - <AUTHOR> http://news.cnet.com/8301-31921_3-57373735-281/meet-richard-mack-republican-challenger-to-sopas-author/

<comment><USER2> I think this is as good opportunity as taking on GoDaddy. It's not every year you get an opportunity to pull down people in power who work against those they were elected to serve. If Lamar Smith goes down, I bet proponents of future bills will be a tiny bit more careful about introducing one sided legislation like SOPA. Right now, there is not political penalty for those who does stuff like this.

</comment><comment><USER3> Promising interview, but then he quickly devolves into the typical soundbites and catchphrases popular in politics (e.g., tax, borrow and spend, borrow from china, big government, big brother, moderate Republican) and doesn't really say much as a result.

<comment><USER4> Yeah, that's the feeling that I got also. Although given the financial backing of his opposition, perhaps he feels that he needs to assault Lamar Smith on any occasion that he can.

Also, it seems like his main audience is other Republicans, who will understand and be concerned about what he means by those criticisms. To a Republican the interview is informative, but to non-Republicans it's probably off-putting.</comment></comment>