Ask HN: Why does Google need so many employees?  - <AUTHOR>

<comment><USER2> That's what I thought before I joined. Once I got into the swing of things and started doing actual work, I was like "Goddamnit. We need _more_ employees. There's too much work to do!"

The simple, non-confidential answer is that running a global service used by billions of people entails facing a lot of problems, on a daily basis, that most developers will never face. And that makes everything move slower.

Have you ever internationalized an app? It's not just a matter of translating every string in the UI, although that certainly is a pain in the ass. You also need to handle right-to-left text and mirrored UIs for Arabic/Hebrew/Farsi, and English text mixed in with that mirrored text (this is common in Israel) which requires a directionality reset. Did you know that you're not supposed to bold Chinese characters? The typographic convention is to make them red for emphasis - but _only_ Chinese characters, English text embedded in Chinese characters (eg, "President Obama" in a Chinese news article) should still be bolded as necessary, so you can't just use a global CSS rule. You know that several European countries format a million bucks as $1.000.000,00 instead of $1,000,000.00, right? And dates go DD/MM/YYYY instead of MM/DD/YYYY, and your date parsing routines have to understand that as well as your date display routines. Ever dealt with Russian plurals? In English, you know something is plural based on whether it's one or many; in Russian, it's based on whether the last digit of the number ends in 1, 2-4, or 5-0, and sometimes depends upon its position in the sentence. Can you do case-insensitive comparisons in multiple languages?

And every single UI engineer needs to be familiar with these pitfalls. A lot are hidden behind frameworks and utility functions, but then every single UI engineer needs to learn the frameworks and utility functions, and understand the purpose behind the libraries so they know when they're appropriate to use.

Do you worry about efficiency when you program? Most programmers these days grab something like Python or Ruby whenever they need to do something, but doing a complex parsing routine in Python can easily run 100x slower than doing it in C++, and at Google scale that means 100x more machines. There are a bunch of open-source libraries for the type of internationalization problems above - but the ICU libraries can easily pull in 50M or so of data files into RAM. If you're running on 20K machines (number pulled out of a hat), that's a _terabyte_ of additional RAM consumed because you wanted to use an open-source library. Do you care about end-user latency? Most webdevs these days take for granted that they'll have JQuery or Prototype available to deal with cross- browser differences. One of the first things I did when I joined Google was try to convince them to allow JQuery for development, and as part of that I ran an experiment to figure out what the actual user impact would be. It turned out it would've _doubled_ end-user latency of the time, and so we had to keep writing everything from scratch based on browser primitives.

Then there're all the issues surrounding running a massively distributed system. Machines fail, packets get dropped, and you have to handle those failures gracefully. It's impossible to upgrade a distributed system all at once without downtime, so every time you make a change that crosses a component boundary, you need to make the new component backwards-compatible with the old one and run both versions of the code in prod until the whole fleet has been upgraded. Many familiar algorithms that work on one machine don't parallelize effectively, so you're limited to what you can compute. Even if they do, the standard libraries for them usually assume a single flat address space, and so you have to really understand the algorithm and how to convert it to a distributed network of computers, and then reimplement it yourself.

If this sounds exciting, we're still hiring. :-) If it doesn't, well, if your startup gets big you'll have to face these problems anyway. Hopefully you can hire someone who finds them exciting.

<comment><USER3> That's all true, but I fear it might be painting a slightly unrealistically positive picture :-) There are also other kinds of reasons for why everything moves slower than one would hope, and they aren't quite as interesting.

Some system your project depends got deprecated again, and you need to decide whether to spend the time to migrate it to the replacement now, or hope it continues working until the replacement has been deprecated too. The development configuration files for other teams systems that you need to inherit from to bring up a full test cluster for your own system changed again, causing mysterious errors. Hopefully early on, rather than in the output of a data pipeline that takes 20 hours to run.

Or the system your working on is somehow relevant to the outside world. There's a legal/political/PR problem, and actual engineering effort is needed to implement the minimal hack to get rid of the problem as soon as possible. Or your project has inexplicably and apparently without anyone's knowledge been marked as falling under Sarbannes-Oxley, and now some change needs to be explained in great detail to a SOX auditor. Which would of course be much more important than any work you were actually hoping to get done :-)

The code base is so massive and there are still so many unexpected dependencies that even with distributed and cached compilation, just the processes responsible for the distribution might be enough to drive your machine to its knees. Of course there's a process in place to try to keep the dependencies from growing out of control, but that's then going to make your life much more difficult if you're working on the inconvenient parts of code nearer the center of the dependency graph, and thus more strictly regulated.

Also, there have historically been a lot of misguided projects that in practice failed but somehow didn't die. Usually they will get killed eventually, but they do tend to grow larger and live longer than they should. Maybe that's changed lately.

This is, of course, an unrealistically negative view. But at least in my experience these kinds of issues were more significant drags on productivity than the fun kind of issues you listed. Though that could be just because the boring kind of drag is much more noticeable. Google's a great place to work at in any case.

<comment><USER2> Yeah, I've run into all of those at times. There are some problems that are just inherent to working at a big company.

Most of the really frustrating ones are fairly avoidable if you (or your tech lead, if you're not leading the project) are reasonably adept at playing the big-company game. My usual reaction to deprecation of critical infrastructure is to complain loudly and persistently, and I can usually get a stay of execution for 6 months to a year, enough for someone else to be the early adopter and work out the kinks of the replacement, then move my own project over to it once it's stable. I've saved many hours for my team by continuing to use deprecated software until the replacement has been deprecated, and then switching straight to the replacement's replacement once it's stable.

The massive codebase is reasonably easy to navigate once you become adept with CodeSearch. Build performance problems are persistent, but I solve that by getting a new machine every time the opportunity presents itself and using them headless for builds & demos. I also open them up to my team so that other frontend engineers I work with can use them...I've got a mini server farm under my desk.

The legal/political/PR stuff can't really be helped, but I find that exhilarating in its own way, since you get to see how those systems work. I've learned to hate the Chinese government since working at Google, though.

BTW, all of these are potentially problems at a startup using open-source software and doing interesting things. It's just that there:

1.) Instead of worrying about your OS libraries being deprecated, you pull the version you want into your own source tree and never touch them, at least for several years. Any attempt to upgrade to a new version is met with intense pain, and so you push the problem off and hope that either your company will go bankrupt or you'll get bought by some big company and the programmers there will rewrite your software entirely.

2.) Instead of complying with the legal problems, you ignore them. If somebody pays attention to you, you're sued out of existence and all look for new jobs. If they don't, you go about your merry way and hope nobody looks too closely at you.

3.) Instead of your dependencies growing out of control....aww hell, every company I've worked at that's more than 6 months old has had dependencies that were out of control.

4.) Instead of being on a misguided project that in practice failed but didn't die, you're on a misguided project that will fail and die spectacularly. And then you get to do it again for the next startup.

Startups can be a lot of fun too, but understand, you're screwed regardless of what you do. :-) Engineering is hard. Let's go shopping!

</comment></comment></comment><comment><USER4> I'm a former Googler who managed the deployment of their content distribution network. Outside of the codebase challenges that nostrademons articulated very well, Google has to deal with various big company concerns including shipping and logistics, human resources, legal issues in the many countries it operates in, hardware development, datacenter management, etc. To do this they have many talented people across disciplines. I was constantly looking for people to join our team while still maintaining the high level of ability and innate Googliness that Google looks for in an employee. In many measurements Google is still understaffed. Check out Royal Pingdoms profit per employee. [http://royal.pingdom.com/2011/05/17/apple-staff-profit- per-h...](http://royal.pingdom.com/2011/05/17/apple-staff-profit-per-head/). The difference between 2008 to 2010 is incredible and factors in growth in employees. Google still needs to hire... not for world domination but to maintain and improve the services that we love.

</comment><comment><USER5> <http://www.google.com/intl/en/about/products/index.html>

Each one of those 50-odd services could be a full-fledged company on its own with a few hundred or even a few thousand employees (in fact, many of them were). I would even bet that as separate companies, they would add up to a lot more than the estimated 20,000 Google employees.

<comment><USER6> After reading what you wrote about each one being its own company, then seeing that Google Toolbar still exists, I started thinking:

"We're delivering the next generation in browser toolbar software"

Oh dear god.

<comment><USER5> That is actually a really good example. Look at the competition, Conduit ([http://techcrunch.com/2008/01/16/toolbar-company-conduit- rai...](http://techcrunch.com/2008/01/16/toolbar-company-conduit- raises-8m-more/)), a company that literally has a few hundred employees (150-499 according to Glassdoor). I figure that Google has about the same number as the competition does for each service, so the Chrome team would roughly match Mozilla (~300 employees), the Docs team would match Zoho (~1500 employees), Maps would match MapQuest (~150 employees), etc.

<comment><USER7> Not to nitpick, but Zoho also has portfolio of several other products including Accounting, CRM, Helpdesk, Network Manager among others. And their total employee strength as you said is ~1500.

</comment></comment></comment></comment><comment><USER8> They waste a lot of engineering resources, but almost everyone somehow works long hours. They could be _even_ more efficient.

<comment><USER9> And you are quoting this from?

<comment><USER8> I had a short and painful stint there, buddy (its been over a year, though).

Hopefully, given all the press, things are a bit better now.

</comment></comment></comment><comment><USER10> The answer is simple: World Domination.</comment>


Worried About Swearing Too Much? Science Says You Shouldn't Be - <AUTHOR> https://www.discovermagazine.com/health/worried-about-your-foul-mouth-swearing-could-actually-be-good-for-you <comment><USER2> Ironically, the fact that many people don't use profanity is what gives it power. Telling people that it's okay to say taboo words only works as long as most people don't listen. Once the f-bomb becomes commonplace it's no longer an expedient way to express strong emotions. Similar to building up a tolerance to caffeine, the effect is diminished if you use it a lot.

If saying something benign like "peanut butter fudge" made you grip the pliers harder then the effect wouldn't be due to the power of words but rather it would be due to your core muscles engaging and bracing your body.

So I'm going to do my part and not encourage anyone to use profanity, and I'll only use it when I need to.

<comment><USER3> I think the reduction in power is _the best_ reason to use profanity to your hearts content, maybe its some kind of language accelerationism, but the very idea of "profanity" always irked me; it always felt/does feel extremely arbitrary. "This word is bad because it's bad"

</comment></comment><comment><USER4> I went through a phase where I tried to curb my casual profanity and use "dag nab it" or "confound it" or the like.

One day I stubbed my toe and came out with a "gosh fucking darn it!!!!"

<comment><USER5> And, as usual... [https://xkcd.com/75/](https://xkcd.com/75/)

</comment></comment><comment><USER6> Archie : How very interesting. You're a true vulgarian, aren't you?

Otto : You are the vulgarian, you fuck.</comment>


Inside the Attack that Crippled Revision3 - <AUTHOR> http://revision3.com/blog/2008/05/29/inside-the-attack-that-crippled-revision3

<comment><USER2> DoS attacks are illegal. If this were some kid in his basement, the FBI would have long since busted down his door and hauled him off to juvie.

What's with the double standard?

<comment><USER3> It's more complex than that. I don't know the details of this instance (Link is down), but the bittorrent protocol is pretty bad - you can just add any IPs you wish into a swarm and say they are sharing particular files. So launching DDOS attacks is pretty simple. There is also no easy way to trace back who injected those IP addresses into the system.

</comment></comment><comment><USER4> Nice. Have infrastructure that can launch SYN floods without a human triggering it, and then don't have any data centre staff on call during the weekend. I hope they get into deep trouble for this.

<comment><USER5> Ditto, I've been following the media defender debacle since September of last year (when the e-mails were leaked) and it's never ceased to sicken me. Altho their business model of 'fighting fire with fire' is the only way I can think of to properly combat many of the existing issues it's still shady to profit off of blatantly illegal activities.

At least what tpb provides is legal in Sweden.

</comment></comment><comment><USER6> He makes the point that what if this happened to a hospital or other critical server. While that point is valid, they seem to only be going after Bittorrent ports, so you would think no important servers would accidently be targeted. However, many colleges nationwide are BT trackers for Linux and other open source projects. How embarrassing would it be if MediaSentry illegally took down one of the colleges it said was not enforcing the law.

</comment><comment><USER3> Posting info about what I assume is a DOS attack, on the domain being attacked, seems a little silly. (Link isn't loading for me).

<comment><USER7> It's on the front page of slashdot now.

<comment><USER3> Ah thanks for that.

</comment></comment></comment><comment><USER8> disgraceful, pitiful even. at a time when content publishers should be competing on quality or service, and building strong, happy brands, instead they risk everything employing black hat companies to destroy everything that disagrees with their existing business model. I synpathise with labels whose content is being stolen, honestly, I do - but nothing good can come of the private sector using destructive activity to try and protect its position

</comment><comment><USER9> This page half loads then Firefox (3 beta) switches to its typical error page, but saying "The page you are trying to view cannot be shown because it uses an invalid or unsupported form of compression." Anyone else seeing that?

</comment><comment><USER10> It's interesting that there is a market for illegal hackers and only the Media companies seem to be hiring them.

<comment><USER11> Or the other actors manage to hire competent crackers and do not get caught.

</comment></comment><comment><USER12> What's Revision3?

<comment><USER13> They want to be the Internet's TV providing a wide range of podcasts that appeal to the tech community.</comment></comment>


GNU.org down - <AUTHOR> http://gnu.org/ <comment><USER2> So post it in HackerNews to make it popular and keep it down :D

<comment><AUTHOR> Better the people read it here and maybe click once then people trying emacs elpa updates 10x before they realize the GNU.org is down I guess ;)

</comment></comment><comment><USER3> Slow news day ?

</comment><comment><USER4> It's up again.

</comment><comment><USER5> still</comment>


Magpul Flashlight Gun Transformer Concept [flash movie] - <AUTHOR> http://gizmodo.com/365738/from-flashlight-to-flash-fight-with-the-fmg+9-concept-from-magpul <comment><USER2> The FMG-9's been all over the gunnie blogs lately. As LawDog put it: "Now, that's just flat neater than kitten toes." (<http://thelawdogfiles.blogspot.com/2008/03/want.html>)

Admittedly, it's just a concept gun and, even if it weren't, entirely unlikely to survive the political and technical trials of getting to market. But still, how friggin' cool is that?

It's amazing what people can do with a glock, some ingenuity, and one too many screenings of the transformers movie.</comment>


Ask HN: How can I ask for increment? I am feeling underpaid - <AUTHOR> Currently I am making 12$/hour. I graduated one year ago. I have really good experience. I did nine internships during my bachelor studies. Worked with CERN, AWS, Hackerrank like companies. I also have part time work experience for more than a year. I don't know about my value in market. What should I do? <comment><USER2> You can always ask for more, the worst thing that can happen is that they say "No". I would suggest you put together a list of successful projects and tasks you have worked on and present that to your manager with an ask for a raise. Make your case that you have demonstrated value and think your compensation should better reflect your value.

I don't know where you are in the world, but where I am (in Pittsburgh, PA, USA) a jr dev with 1 year of experience and a good track record could ask for $20-25/hr without issue. You also don't mention what work you have been doing, that can change the money equation a lot.

Negotiation is a difficult skill to learn, but it can be very rewarding. Good Luck.

</comment><comment><USER3> Which location is this?

> I don't know about my value in market

Search for similar jobs or talk to recruiters. They'll tell you roughly what you should be earning.</comment>


Spacecraft may have captured dust particles from beyond our Solar System - <AUTHOR> http://arstechnica.com/science/2014/08/spacecraft-may-have-captured-dust-particles-from-beyond-our-solar-system/

<comment><USER2> The article is worth reading, for anyone who was going to skip it. It's especially interesting to read about the aerogel method and techniques used to discount many of the particles captured.

"seven dust particles caught by the Stardust spacecraft likely came from the [interstellar medium]"

Must be a fascinating project to work on given the attention to detail required and the rarity of subject material.</comment>


Creating a QR Code step by step - <AUTHOR> https://www.nayuki.io/page/creating-a-qr-code-step-by-step <comment><USER2> I appreciate the steps, but this fails to answer a question which, imho, is way too often omitted in tech tutorials, and that's "Why?".

Sure, I understand these are just the steps to create a QR code and I could just Google why the QR code needs a masking pattern, but that's besides the point. A good material would explain why I need it, instead of just saying it should be done. Even one sentence would be sufficient.

<comment><USER3> The utility of the masking pattern becomes clear when following the next steps, which calculate "penalty points" for dot patterns in the QR code that might occur naturally, but are known to make it harder to read and more prone to erroneous reads. The masking pattern that, when applied to the "clean" content, generates the least problematic output is chosen.

I found this pretty self-explanatory while just following the individual steps. However, a few sentences detailing the higher-level algorithm and its purpose right before the step which first introduces the masking pattern probably wouldn't be a bad idea either.

<comment><USER4> Yeah, I once implemented a very simple AR tag mechanism -- using Binary Golay codes [1] which are super simple to understand.

It turns out that you need a mask, otherwise you can too easily get a tag that is entirely (or close to) white or black. I was fortunate with a small code space (only 12 bit) that I could get away with a single mask, it never occurred to me that I could have used multiple masks (at the cost of a the bits to encode the choice)

I looks like extra complexity, but it solves a very pragmatic problem

[1] [https://en.wikipedia.org/wiki/Binary_Golay_code](https://en.wikipedia.org/wiki/Binary_Golay_code)

</comment></comment></comment><comment><USER5> During quarantine I designed a paper-based system for making QR codes by hand. Everything is very papery and bureaucratic, but also fun. Tables to look up bytes, long forms for calculating Reed-Solomon, and even transparent sheets of paper for comparing different masking patterns.

My goal is to film a video of generating a QR code without any computers or even calculators. It's possible, just might take two hours or so.

<comment><USER6> That's awesome. Reminds me of Ken Shirriff computing sha256 with pencil and paper...

[http://www.righto.com/2014/09/mining-bitcoin-with-pencil- and...](http://www.righto.com/2014/09/mining-bitcoin-with-pencil-and- paper.html)

<comment><USER7> It's a great way to learn. In my cryptography classes we did all of the algorithms by hand (often shortened, so 16 rounds of something would be dropped to 3 or min bytes dropped) on pen and paper as homework. Hard but it really did make a lot stick in my mind about how things worked.

<comment><USER8> It's also a great way to debug.

I've written video/image codecs and when things go subtly wrong, decoding the raw bitstream by hand and comparing carefully with the decoder output has been the best way to figure out the problem. It can be quite tedious, but as the phrase goes, definitely "builds character".

Or as someone else put it, "How can you tell the computer what to do, if you can't do it yourself?"

</comment></comment></comment></comment><comment><USER9> I found this video to be a lot clearer and with the combination of the website makes understanding easier: [https://www.youtube.com/watch?v=KA8hDldvfv0](https://www.youtube.com/watch?v=KA8hDldvfv0)

</comment><comment><USER10> Seven years ago I had the wildest ideas of creating photo realistic QR's. Sadly, I got pulled away to other projects with higher priority. The unanimous advice given to me at that time was to hold publishing so I could exploit it financially. Until today neither of both have changed.

I'm not going to let it catch more dust and decided to open-source it under Affero license.

Creating a photo realistic QR requires two steps. The first is to create a 93x93 dithered monochrome image. The dithering is calculated to preserve the mandatory QR framework/timing bits and generating the CRC.

The second step is adding colour information as a 186x186 dithered layer. The colour palette is created using Spacial Colour Quantification which uses the 93x93 QR image as constraints on the available colour range.

SCQ also stabilises palettes used for animations (a service not made available on the site).

During the years the site has been prone to bitrot. I managed to recover most of the missing parts parts from backups. Most of the functionality has been restored and I'm still working with higher priority projects.

[0] [https://www.qrpicture.com](https://www.qrpicture.com) [1] [https://github.com/xyzzy/qrpicture](https://github.com/xyzzy/qrpicture)

</comment><comment><USER11> Though QR codes are very interesting technically, they were designed to be used by devices with orders of magnitude less computational power than is currently the norm, and much lower resolution optics as well. This is a good thing from an error-prevention perspective, but I wonder if a more modern QR code could be designed for today's smartphones.

I think an interesting new standard would take advantage of higher resolution cameras and OCR functionality to make data transparent to the consumer. I personally wonder what the hell is going to pop up on my phone every time I use a QR code... Text? URL? Link to an app store? Who knows. A standard which marks text as "data" would let a user know what they're saving, and let them point a camera in the general direction of some text, and all the relevant info be extracted and organized. (Think addresses, receipts, etc.) I already use my phone to remember lots of things - like I take a picture of the back of my DSL router for the access codes printed there, or take a picture of my bike tire to remember the size of the tube I need, etc.

Or if transparency isn't important, given we have 4K cameras now, imagine if there was a popular standard using steganography! I can imagine movie posters having showtimes, director and actor info, URLs for streaming it online, etc. all baked into the image.

<comment><USER12> Most of your critique would be handled by proper input sanitation and allowing the user to review and confirm the best / default action.

For a modern QR code I'd prefer something that allows more data and which also mandates support for some compression algorithms and data filters.

Old QR codes are optimized for shrinking serial numbers, barcodes, and the like to a very small size on a product. That's a good use case and I feel they've sufficiently solved that problem domain.

A modern QR might include more like a very tiny web-page, with a subset of allowed entities for security. An icon, or a link to an icon, image (links, maybe bitstream embed?), and simple links.

I would ideally love to be able to hand out small files as blob patches on a business card, printed on a standard laser printer (300 DPI max resolution, assume 100 to 150ish DPI). Maybe two files on the back half. One a reasonable resume and the other a pack of public key data.

Edit, back of napkin math: In the US that would be about 3.5in by 2in and in the dot density about 1.5 in square for each of two squares. At 300 that'd be 450 by 450 dots, or 150 by 150 in the minimum. 22500 to about 200000 bits, not including ECC. The upper density I'd like only offers about 25K of data. This might be enough for a compressed mostly text document or some well-packed PQC safe keys and en EC crypto key or two for current use.

[https://en.wikipedia.org/wiki/Post- quantum_cryptography#Comp...](https://en.wikipedia.org/wiki/Post- quantum_cryptography#Comparison)

</comment></comment><comment><USER13> For those interested in the creation of the QR code. Japanese TV Station NHK World have a 15 minute documentary charting it's creation - [https://www3.nhk.or.jp/nhkworld/en/ondemand/video/2072031/](https://www3.nhk.or.jp/nhkworld/en/ondemand/video/2072031/)

It also features interesting details such as how they figured out the ratio of the marker squares should be.

<comment><USER14> Make sure to have a QR code reader on hand as you watch this. The video is full of Easter eggs hidden in the dramatizations.

</comment></comment><comment><USER15> This was a good exploration of your approach. The interactive usage inline with description was very helpful. As some have pointed out, some expansion in certain areas could improve it, but still good.

As a side note, I'm looking into using your libraries. They do seem fairly solid and efficient.

Thanks for putting this out there!

</comment><comment><USER16> Previous discussion: [https://news.ycombinator.com/item?id=18360847](https://news.ycombinator.com/item?id=18360847)

</comment><comment><USER17> An artisanal QR code goes over the error correcting code by manually calculating the long division: [https://www.quaxio.com/an_artisanal_qr_code.html](https://www.quaxio.com/an_artisanal_qr_code.html)

</comment><comment><USER18> >The math behind computing the Reed-Solomon error correction codes is omitted because it is long, tedious, and not very interesting.

LOL! That's probably the most difficult part of creating a QR code, the rest is just framing/padding the data which is pretty obvious.

<comment><USER19> I'd argue that while Reed-Solomon is used in QR codes, the math behind it is not central to explaining how to create a QR code, and there are plenty of material available on Reed-Solomon.

Specific framing/padding of the data is what makes a QR code. This may be simpler than explaining how Reed-Solomon codes work but it is not obvious.

I think it is good that this article stays on point.

<comment><USER20> Other discussions on Hacker News and Reddit have requested me to show the Reed-Solomon ECC calculations too (see [https://news.ycombinator.com/item?id=18370829](https://news.ycombinator.com/item?id=18370829) ).

mytailorisrich is correct; Reed-Solomon is used by many different standards, so it's not difficult if you learned it elsewhere. Whereas other parts of the QR Code standard are idiosyncratic, i.e. unique to it. If you want to see for yourself, go learn how Data Matrix works (another 2D barcode) and notice how little knowledge you can carry between the two standards.

As for showing the Reed-Solomon calculations, I stand by my statement of them being long and tedious. Moreover, it requires background knowledge of finite field arithmetic (as opposed to ordinary arithmetic), which increases the explanation length and makes the calculations even more difficult to follow by hand.

The actual Reed-Solomon encoding is conceptually simple. First you choose a generator element. Then you create a divisor polynomial by multiplying monomial terms involving powers of the generator. Then you encode the message as a polynomial, and finally divide the message by the divisor to yield the remainder polynomial, which is the final RS ECC data.

If you want to see the cryptic verbosity of tons of numbers, here are examples: [https://www.backblaze.com/blog/reed- solomon/](https://www.backblaze.com/blog/reed-solomon/) ; [https://www.youtube.com/watch?v=H2LlHOw_ANg](https://www.youtube.com/watch?v=H2LlHOw_ANg) (AES)

</comment></comment></comment><comment><USER21> This is a great resource. I used it along with another practical tutorial ([https://www.thonky.com/qr-code-tutorial/](https://www.thonky.com/qr-code- tutorial/)) to complement the official QR code standard when I wrote my own QR code encoder in Common Lisp: [https://github.com/mare5x/LispQR](https://github.com/mare5x/LispQR)

</comment><comment><USER22> Along similar lines is there any guides on making more artful QR codes? I've seen ones that aren't square but still scan and found them fascinating.

<comment><USER23> It basically amounts to cranking up the error correction and seeing how much you can get away with changing before it stops reading correctly.

</comment></comment><comment><USER24> How small can you make a QR code until it can't be scanned by a phone?

<comment><USER25> That's dependent on the zoom factor and focus distance of your lens and therefore really hard to say in general.

In theory, you could even make a micrometer-sized QR code and scan it using a phone-microscope-attachment - so if you're really trying to push it, probably smaller than you can reasonably print/manufacture.

For a more practical usage, when we worked with QR-Codes, about 3cm by 3cm (or 1.2 inch) seemed to be the smallest unit not causing trouble; smaller than that and problems with focus started appearing for some clients and scan times increased. Of course, that's only anecdotal evidence.

</comment></comment><comment><USER26> That's really cool! Thanks for sharing it!

</comment><comment><USER27> ... or you could just use an "Oh By" code.[1]

No app, no reader, and you can just chalk/write/scratch it anywhere you like.

This message, for instance, is at: 0xH6VYP7 [2]

[1] [https://0x.co/](https://0x.co/)

[2] [https://0x.co/H6VYP7](https://0x.co/H6VYP7)

<comment><USER20> What happens when the company goes out of business?

What happens when you try requesting random short URLs to peep on other people's texts?

<comment><USER27> "What happens when you try requesting random short URLs to peep on other people's texts?"

I think most use-cases involve a code that you chalk up, or write, or paste publicly ... I am not sure that the codes being secret will be a common use- case.

There's a fair amount of space in a 6-digit alpha-numeric code, though (36^6, roughly) ... that's a lot of brute forcing ...

<comment><USER20> 36^6 = 2,176,782,336

<comment><USER27> Correct - thank you.</comment></comment></comment></comment></comment>


Ask HN: Company offers to pay for sth to improve our skills, what to propose? - <AUTHOR> My boss told me our company wants to invest on our education/skills: improving and broadening our skills, mastering something specific, etc. He gave me a few examples: buy books, attend a conference, etc. But he didn't have plenty of ideas and asked me to propose something, and I do not know what to propose.
If given this opportunity, what would you guys propose?
Some background: I work in a startup in Berlin (we broken even with our first product and are now working on a new one). We are a small team (<10), and each one of us works on at least a few areas (mobile, backend, devops, etc.).
Thanks! <comment><USER2> If you guys are using AWS Amazon in organizing a series of presentations, check [https://aws.amazon.com/start-ups/loft/](https://aws.amazon.com/start- ups/loft/)</comment>


'Star Wars' T-Shirt Displaying Blaster Gets Seventh-Grader in School Trouble - <AUTHOR> http://www.techtimes.com/articles/115957/20151214/star-wars-t-shirt-gets-seventh-grader-in-school-trouble.htm <comment><USER2> Just be glad it isn't under a "Zero Tolerance" policy. My son's friend got suspended for a year and had to attend alternative school in 6th grade. He brought an air soft gun to school to play at his friends house.</comment>


Ten Predictions for 2030 - <AUTHOR> http://venturefizz.com/blog/ten-predictions-2030

<comment><USER2> Egregious typo alert: "telephone polls" in item #8, referring to the slender, vertical wooden members that keep electric, cable TV and telephone service wires aloft.</comment>


Wireless power from WiFi  - <AUTHOR> http://www.engadget.com/2010/01/09/airnergy-wifi-power-system-gives-rca-a-reason-to-exist-video/

<comment><USER2> See also <http://news.ycombinator.com/item?id=1043296></comment>


"Millennials believe stormy weather can interfere with cloud computing" - <AUTHOR> http://www.citrix.com/news/announcements/aug-2012/most-americans-confused-by-cloud-computing-according-to-national.html

<comment><USER2> I knew dates could be awkward, but still... "Interestingly, an additional 17 percent have pretended to know what the cloud was during a first date."</comment>


A Public, Private War: How Government and Tech Can Better Prepare for Conflict - <AUTHOR> https://cltc.berkeley.edu/2019/12/17/a-public-private-war/ <comment><AUTHOR> moar context:

 _> The US government & tech sector have a complex history of betrayal & mistrust in matters of war & peace, from Snowden to Maven. The past is prologue & interviews over the last 2 years show a path forward. My paper, "A public, private war", is out today. /1_

source: [https://twitter.com/jonathanreiber/status/120701653344197017...](https://twitter.com/jonathanreiber/status/1207016533441970177)</comment>


Schooling in adolescence raises IQ scores (2011) [pdf] - <AUTHOR> http://www.pnas.org/content/109/2/425.full.pdf <comment><USER2> Doesn't this just mean that IQ Scores don't measure what they're supposed to? My understanding is that IQ scores measure how smart you COULD be, all things considered. But if you can increase this with schooling, isn't it just how smart you are right now?

<comment><AUTHOR> From my own experience, you're able to prepare for IQ-tests. I scored slightly below average (below 90 actually) on the IQ-test I did with 7. Average (104) on the one I did with 16. And actually made it in to Mensa with a result of 144 on the IQ-test I did aged 18. I was in a good mood the day I did that IQ- test, and learned math, memory techniques etc. a few weeks before. So, IQ- tests really aren't that reliable.

<comment><USER3> Around here those tests seems to be careful to avoid maths, memory etc and only try to test your ability to recognize patterns or find missing solutions.

<comment><USER4> So you practice patterns to raise your score.

<comment><USER3> No, not really. Never felt I had time.

I also think they try to introduce elements that makes it hard to prepare for the test.

That said if someone has never taken one of these then getting used to how the test works and doing a sample test could help them get a better score right away.

</comment></comment></comment></comment></comment><comment><USER5> IQ scores have to do with a lot of things. IQ can lower due to:

\- not meeting basic nutrition requirements during gestation and early life

\- not consuming enough iodine. iodine gets added to salt to ensure people consume it.

\- exposure to lead and other toxic substances.

...and mental stimuli.

Some people (notably alt-right Twitter accounts), promote that idea that certain races have significantly lower IQ, spreading maps of average IQ per country. It is my duty to call out that as wrong. Having the aforementioned factors in consideration quickly refutes that idea.

Some people point out that there are genetic markers associated with higher IQ. To be honest, we haven't listed all of them yet. So I would stay quiet before pointing out that certain population doesn't have certain SNP.

<comment><USER6> >not consuming enough iodine. iodine gets added to salt to ensure people consume it.

source?

Also, it's ridiculous to argue that genetics don't affect IQ. There are many genetic disorders that impair cognitive skills.

<comment><USER7> You just rebutted an argument he didn't make.

<comment><USER6> Woops, you're right.

</comment></comment></comment></comment><comment><USER8> What really raises the score is practising on IQ test type of riddles and questions.

</comment><comment><USER9> It would be interesting to see some sample tests. From the data section: _The measure of IQ used in the analysis was obtained from tests of cognitive ability administered by the Norwegian military to all draft-eligible men at approximately the age of 19 y as part of the universal military draft in Norway. The data from these tests have been widely used and interpreted as IQ scores for research purposes._

Also interesting, the historical summary of the "distributed" nature of the Norway educational system: _The reform [...] required that all municipalities provide their youths with an additional 2 y of standardized education in the eighth and ninth grades. [...] The reform was introduced at the municipality level, the lowest of three administrative levels in Norway, with the other two levels being the national level, which has responsibility for higher (tertiary) education, and the county level, which has responsibility for secondary education. [...] Following the reform, the new type of middle school was administered by the municipalities. Each separate municipality was able to introduce the full compulsory schooling reform after local officials submitted a reform plan to a national committee, provided national funds [...]_

It sounds like they used a bottom-up approach rather than a plan forced from on top, which seems like a good thing.

This makes me want to learn more about how school systems / school boards work. It seems to be a city-level organizational structure, but there are provincial/state, and federal rules too. This is a lot of APIs: you pay federal, provincial, municipal, and school taxes, and then some portion of your taxes comes to the schools. Surely there's room for simplification...

</comment><comment><USER10> IQ scores usually increase with education in youth but as an individual becomes an adult the child age scores drop off.

</comment><comment><USER11> Guys before you take this too seriously, PNAS is a publication repeatedly criticized by stats experts like Andrew Gelman. NHST techniques are bad, and there are many papers showing almost zero effects of education.

Plus your prior on this should be pretty low given Turkheimer's 3 laws of behavioural genetics. Especially since education is more like a shared environment variable rather than a unique environment variable.

<comment><USER12> You shouldn't invoke Turkheimer's three laws paper (which is a good paper) to make a statement with which Turkheimer would disagree. Turkheimer certainly agrees with the factual observation that formal education (which is part of the environment not necessarily shared by siblings in the same family) can raise IQ. You can look up a paper on which he is a co-author for more details. (I correspond with Turkheimer from time to time, and know many of his co- authors and colleagues.)

The link to the paper here is from Eric Turkheimer's faculty webpage at the University of Virginia, which I visit often to read his latest publications.

[http://www.people.virginia.edu/~ent3c/papers2/Articles%20for...](http://www.people.virginia.edu/~ent3c/papers2/Articles%20for%20Online%20CV/Nisbett\(2012\)%20.pdf)

<comment><USER11> Turkheimer's laws are not the property of the man himself! I think it's pretty safe to say there's no clear winner on what the effects of schooling is _on the margin_ at least.

You're right to correct me on school being part of the nonshared environment that's my mistake, you probably know more than me how complicated it is to extract meaning from the nonshared environment. Also the temporary nature of the improvements are very confusing. Thanks for the paper.

Here's a summary by SSC (not really an expert but worth reading): [https://slatestarcodex.com/2016/05/19/teachers-much-more- tha...](https://slatestarcodex.com/2016/05/19/teachers-much-more-than-you- wanted-to-know/)

</comment></comment></comment><comment><USER13> I'm curious why this was downvoted.

<comment><USER14> Please don't change the subject to downvotes. It never works, which is why HN has not one but two guidelines against it: [https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html). Also, as typically happens, that comment is no longer net-downvoted.

We detached this comment from [https://news.ycombinator.com/item?id=13180415](https://news.ycombinator.com/item?id=13180415) and marked it off-topic.

<comment><USER13> But I wasn't complaining about being downvoted, nor was I inviting anyone to downvote me. A comment not written by me was downvoted by someone who didn't seem to be represented in the replies to it, and I really did want to know why, out of actual curiosity.

<comment><USER14> I appreciate the curiosity part, but it really is off topic.</comment></comment></comment></comment>


What do political speechwriters have against paragraphs? - <AUTHOR> http://nextness.com.au/insights/what-do-political-speechwriters-have-against-paragraphs/ <comment><USER2> (A) What looks good when written/printed and what sounds good when spoken frequently have little to do with one another.

(B) Modern journalism has the same problem, only worse due to (A).</comment>


node-jsc: A Node.js port to the JavaScriptCore engine and iOS - <AUTHOR> https://github.com/mceSystems/node-jsc <comment><USER2> Hi,

I'm Koby Boyango, a senior researcher and developer at mce, and I've created node-jsc, an experimental port of node.js to the JavaScriptCore engine and iOS specifically.

The project also includes node-native-script, NativeScript's iOS runtime refactored as node-jsc native module, allowing access to native iOS APIs directly from javascript.

See the iOS Demo Project for an example of using both node-jsc and node- native-script.

Feel free to check out our repositories, ask me anything or share your ideas\suggestions :)</comment>


Dashboard UI: 9 Best Practices for Designers, Developers and Product Owners - <AUTHOR> https://adamfard.com/blog/dashboard-ui <comment><USER2> As I am developing a general-purpose Dashboard [1], this is really helpful :)

[1] [https://github.com/darekkay/dashboard](https://github.com/darekkay/dashboard)</comment>


Show HN: Tamper – Devtools extension. Edit remote files locally - <AUTHOR> http://dutzi.github.io/tamper/

<comment><AUTHOR> I built this tool because I had a lot of production issues where I needed to make some changes and test them. I couldn't use devtools' Elements or Sources panel since I had to make this changes available as the page loads (inline script changes, css url changes, etc...).

I tried using Fiddler and similar alternatives (Charles, Burp), but:

1\. when running it on the Mac you need to run a VM

2\. Fiddler shows you ALL the requests made by you're machine.

3\. Moving between Fiddler, Chrome and Sublime is just annoying.

Since it's running a proxy server (it's actually based on [http://www.mitmproxy.org](http://www.mitmproxy.org)) on your machine, it's also very useful when you want to test things on mobile devices.</comment>


Is it legal to rewrite credit card magnetic strip data? - <AUTHOR> http://www.quora.com/Is-it-legal-to-rewrite-credit-card-magnetic-strip-data

<comment><AUTHOR> Dynamics have had the tech offered by Coin for a really long time, but for whatever reason, they switched from offering a credit card to a rewards card:

[https://www.dynamicsinc.com/](https://www.dynamicsinc.com/)

I'm guessing the reason is mostly PCI compliance. They also hold a number of patents on reprogrammable cards which makes me wonder if Coin have licensed them or not.</comment>


Top Programming Languages: Python extends its lead, Assembly enters the Top Ten - <AUTHOR> https://spectrum.ieee.org/at-work/innovation/the-2018-top-programming-languages <comment><USER2> Good list but I think GitHub's list matches more closely to what I see people using: [https://octoverse.github.com/](https://octoverse.github.com/)

In particular, GitHub's list has JavaScript as #1 and I've seen a lot of frontend and middle-tier work moving to JavaScript via React, Node, Angular, Vue, etc.

Also, how is Assembly in the top 10 on the IEEE list??? I get that every machine runs it but I don't see a lot of people programming in assembly these days...

<comment><USER3> IEEE focuses a lot more on embedded and industry stuff IIRC so that would basically kill Javascript outright. Python is fairly popular there as is Assembly (I maintained a Assembly based production line controller from the 80s in a previous job)

</comment></comment><comment><USER4> This looks somewhat different to me than other rankings I've seen lately. Python is a nice language but putting it in first place kind of raises a lot of questions about the methodology since it is nowhere near that spot in essentially all similar rankings I've seen, ever. Also, python for embedded development?! I'm sure people do that but it hardly sounds like a typical use case for python.

<comment><USER5> With the "robot" framework, python is fantastic to test embeded systems.

<comment><USER6> You just brought out bad dreams, having been one of their guinea pigs, back when it was HTML based.

I hope it is more user friendly nowadays.

</comment></comment></comment><comment><AUTHOR> Link to interactive web page: [https://spectrum.ieee.org/static/interactive- the-top-program...](https://spectrum.ieee.org/static/interactive-the-top- programming-languages-2018)

Python is #1. C++, C, Java and C# at 2,3,4 and 5 respectively. Javascript at 8, and Go at 9.

</comment><comment><USER7> I feel like the IEEE is more focused on embedded and industrial software. At least, when I was in that industry that's the last time I saw people with ieee.org e-mail addresses. And that explains why this list may appear weird to the HN audience. And this is a great reminder that what we (HN visitors) see is just our own filter bubble.

</comment><comment><USER8> The whole thing with ranking programming languages and gauging technology trends is warping the minds of budding programmers. It's like they are watching a market ticker to figure out what to learn, instead of just asking them selves what they like doing.

</comment><comment><USER9> > First, Python is now listed as an embedded language. Previously, writing for > embedded applications tilted heavily toward compiled languages, to avoid the > overhead of evaluating code on the fly on machines with limited processing > power and memory. But while Moore's Law may be fading, it's not dead yet. > Many modern microcontrollers now have more than enough power to host a > Python interpreter.

And more importantly, you can use Python and template frameworks to generate C code for your embedded targets. We don't write C manually these days.

<comment><USER10> > We don't write C manually these days.

I do.

</comment></comment><comment><USER11> HTML is included, yet it is less "popular" than Assembly? First I hear of "Processing" and it is more popular than SQL?

While they explain their methods here: [https://spectrum.ieee.org/static/ieee- top-programming-langua...](https://spectrum.ieee.org/static/ieee-top- programming-languages-2018-methods)

Could it be, that languages like Assembly are searched more often than others, but are not necessarily popular?

</comment><comment><USER12> A slightly odd selection of languages - some seriously obscure stuff in there, yet missing some really obvious ones.

<comment><USER13> I think it's because it's the IEEE, so the choice of language is screwed a bit towards hardware engineers, and developers of embedded devices.

The languages most would think is missing would most likely be languages like Swift and perhaps Rust. As popular as these languages seem, how many shipping applications really use them? Not that many in the grand scheme of things I would say.

</comment></comment><comment><USER10> > Programming Languages > HTML

Can the list even be taken seriously?

<comment><USER14> Or Arduino...

</comment></comment><comment><USER15> Disappointed Erlang didn't make the list in 2018 :(

<comment><USER11> Erlang is 37.

[https://spectrum.ieee.org/static/interactive-the-top- program...](https://spectrum.ieee.org/static/interactive-the-top-programming- languages-2018)</comment></comment>


DNC launches tech marketplace for Democratic candidates - <AUTHOR> https://techcrunch.com/2018/04/30/dnc-launches-tech-marketplace-for-democratic-candidates/ <comment><USER2> Are the tech companies pre-approved by the DNC?

<comment><USER3> One wonders why not a set of github links to open source tools...</comment></comment>


Discovery of 8000 year-old wine production in the Middle East - <AUTHOR> https://phys.org/news/2017-11-earliest-evidence-winemaking-team-year-old.html <comment><USER2> Interesting discovery. However:

Georgia is in Europe, not in the Middle East -- in terms of geography, history, and present culture.</comment>


Apple Logic Pro X free trial is available - <AUTHOR> https://www.apple.com/logic-pro/ <comment><USER2> Ableton now also offers 90 days of Live free trial</comment>


The rise of American authoritarianism - <AUTHOR> http://www.vox.com/2016/3/1/11127424/trump-authoritarianism <comment><USER2> The one-sentence summary quote: "... the GOP, by positioning itself as the party of traditional values and law and order, had unknowingly attracted what would turn out to be a vast and previously bipartisan population of Americans with authoritarian tendencies."

Good article. The wave is what's fascinating this year, not the surfer (Trump).</comment>


GoDaddy is "suboptimal" for hosting WordPress - <AUTHOR> http://andrewgarrison.com/blog/godaddy-is-suboptimal-for-wordpress/

<comment><AUTHOR> It wasn't covered in the blog post, but it is also interesting how slowly WordPress performed compared to DotNetNuke. I never felt there was a performance problem with my web site until I moved from DNN to WP. I suppose it could also be GoDaddy's MySQL servers vs their SQL Server servers. Maybe their MySQL servers are bogged down.

Edit: added a missing word.</comment>


Twitter meets TensorFlow - <AUTHOR> https://blog.twitter.com/engineering/en_us/topics/insights/2018/twittertensorflow.html <comment><USER2> > Machine learning enables Twitter to drive engagement, surface content most > relevant to our users, and promote healthier conversations.

One that wants to manipulate your mind, one that echochambers your discovery, one that censors arbitrarily.

<comment><USER3> Do you remember hiw bad YouTube comments have been? And it's significantly better lately?

That happened due to ML.

Not everyone is out there to use ML for nefarious purpose. And there are multiple use cases for ML.

<comment><USER4> At least for music videos, comments are still pretty shitty. It feels like more than half of them are one of the following patterns:

\- "Like if you're watching in June 2018?" [2k upvotes]

\- "Most underrated artist of our time" [5k upvotes]

\- "Wow, mad respect for <artist>" (written by verified account by other artist trying to promote their own content)

<comment><USER5> Why are those bad comments? The first one is a bit ridiculous, but the others are fine. They're _comments_. They're not discussion points. YouTube isn't a forum or a social network. There's nothing compelling people to leave intelligent or even constructive points. The comment section is for people to note what they think. That's it.

When people say these sorts of things are bad I think it's because their expectations and understanding of a comments section are wrong.

<comment><USER4> It's just super repetitive, and actually good and original comments get buried because they don't match the hivemind.

Of course, it's a question of taste what comments should be more visible. If the community actually prefers reading the same stuff over and over again, then the current state is probably justified, and it's just a moronic community.

</comment></comment></comment></comment></comment><comment><USER6> I always enjoy browsing the Twitter Engineering blog because of 1) the granular detail that they give 2) the concise, organized format that the information is presented, and 3) because I love their creativity. However, the first sentence of this article "Machine learning enables Twitter to drive engagement, surface content most relevant to our users, and promote healthier conversations. As part of its purpose of advancing AI for Twitter in an ethical way, Twitter Cortex is the core team responsible for facilitating machine learning endeavors within the company" is a complete turn off. Keep it technical. Leave the Marketing/PR out of it.

\- Regular blog.twitter.com/engineering reader

</comment><comment><USER7> This is very confusing and meandering. It gives flow charts and lists of steps that don't map to my experience building deep learning models at scale, and spends a strange amount of time passive aggressively dismissing Lua Torch and extolling virtues of TensorFlow that aren't very important.

As with all of these purported pipelining systems, I'm skeptical and happy to let a bunch of other people deal with the headches of making it adequately general for a few years before I'll even start caring about grokking it for my use cases.

In the meantime, creating build tooling, data pretreatment tooling and deployment tooling is pretty valuable for me to understand business considerations and make sure all my modeling & experimentation aren't just time wasting ivory tower projects, particularly in terms of customizing performance characteristics on a situation-to-situation basis, free to design the deployed system without a constraint to a particular serving architecture.

It also makes me very disinterested in applying to work for the Cortex team, because even though the article is talking about DeepBird v2 as a means to free ML engineers to do more research, it seems pretty obvious that there's a huge surface area of maintenance and feature management for this platform. Your job is probably going to be _less_ about research, which is scarce work that people compete over anyway.

Possibly attractive for people who just like deep C++ platform building, which is an internal drive not often found in people wanting to solve business problems with ML models.

<comment><USER8> > Possibly attractive for people who just like deep C++ platform building, > which is an internal drive not often found in people wanting to solve > business problems with ML models.

And the world has plenty of people who are not interested in trying to solve business problems with ML models, but are rather interested in the engineering side of ML. I am one of those people. My current work (at cortex) involves improving the time taken to train the types of models not usually described in research papers.

If anyone is interested in machine learning but wants to build high performance training / serving platform, message me over twitter (@pavan_ky), Cortex is hiring.

<comment><USER7> Personally I am very interested in both how models can help solve business problems and how to make effectively engineered tools for machine learning.

In my work I spend a lot of time on new deep learning architectures or experimenting with modifications or fine-tuning or ensembling.

I write a lot of container and Makefile tooling to ensure experiments are reproducible and results have identifiers that map back to the full set of data, software and parameters.

I also write a lot of backend server software to wrap trained models in a web application, mostly in Python, and do a lot of work with Cython after profiling to target only those spots of the code that reveal actual performance bottlenecks in terms connected directly to a specific business problem's latency or throughput requirements — as in, not taking the huge premature optimization step of assuming a whole system needs to be written in C++, and instead using profiling and case-by-case diagnostics to know when to write something as an optimized C extension module callable from Python for very specific and localized sections of code.

My experience has been that there is such a lack of transparency about how deployment will work, how performance will work, etc., when using cookie cutter pipeline approaches, like sklearn Pipelines, TensorFlow serving, Fargate, etc. You'll always need to break some assumption of the pipeline, layer in new diagnostics, debug latency issues, etc., on a case-by-case basis.

99% of the time, ease of specifying a new model or articulating an experiment is not hard, requires little dev work, and only represents about 10% of the actual work needed to explore a model's appropriateness for a given problem at hand.

The rest requires very specialized control and visibility to basically perform application-specific surgery on the pipeline, customizing and tailoring many aspects, from how multi-region deployment should look to how optimized the web service code should be to whether to use asynchronous workers or a queuing service to stage and process requests, to optimizing preprocessing treatments, to instrumenting some extra New Relic metric tracking that the pipeline isn't extensible enough to just specify in some config, and so on.

What's been most important is that the deep learning engineers on the team, who are _researchers_ , are also excellent system engineers at all those topics too and display a high degree of curiosity towards them, and absolutely do not look at it like "boring work" that distracts from the experimentation they would rather do. Their value add is not driven by spending more time experimenting — that's virtually never the case. Their value add is in _both_ knowing the details of the deep learning models intimately while also knowing the deep details of implementation, optimization, deployment, and diagnostics.

In that sense, tying model development to a cookie cutter pipeline framework, whether Spark or sklearn or something custom in-house, is something I believe strongly is an anti-pattern.

</comment></comment></comment><comment><USER9> Maybe they can use ML on the user feedback to figure out what features people actually would like on the platform, instead of trying to figure out relevancy, which no company has ever successfully done. /s

Is there a recent in-depth article somewhere about Twitter's internals? It must be frustrating working on features very few people want to use.

I still can't edit a tweet I just posted, any video looks absolutely horrible for the first 10 seconds, the timeline is a mess, third-party developers that make interesting/much better clients are getting stomped on, and harassment is running mostly unchecked on the platform. Yet I read interesting development articles on the engineering blog. Wasted talent?

I really like the Twitter engineering blog articles, but it seems like it's just an HR tool.

</comment><comment><USER10> This blog post describes twitter's move from Lua Torch to Tensorflow. I am surprised to see it so highly ranked on the front page because there's very little content here. Basically, they describe the sorts of data structures they use and list a couple of advantages of Tensorflow vs. the out-of-date Lua Torch framework.

<comment><USER11> It's probably on the front page because a lot of people are deciding between PyTorch or Tensorflow for their company's production ML pipeline, and this is an article about making exactly that decision. Even if they didn't present any actual analysis, just the sentence "Twitter's ML team, which had previously used LuaTorch, recently evaluated PyTorch and Tensorflow and ultimately decided to standardize on Tensorflow" would be of interest to a lot of people.

</comment></comment><comment><USER12> This post says all you need to know about Twitter.

"Machine learning enables Twitter to drive engagement...".

The very first thing they mention is engagement. They don't care about quality, or what users want, or to foster communication. They care about one thing and one thing only: engagement. More clicks, more likes, more moving around the interface.

</comment><comment><USER13> > given the migration of the Torch community from Lua to Python via PyTorch, > and subsequent waning support for Lua Torch, Cortex began evaluating > alternatives to Lua Torch

I'm a bit out of the loop, has PyTorch really become so much more popular than Lua Torch, in the short time span since it launched? And is it true that most of the original Lua Torch community switched to PyTorch?

<comment><USER14> PyTorch is rivaling TensorFlow in popularity now. Lua Torch is basically dead.

<comment><USER13> Thanks, I saw many articles about PyTorch lately, but didn't realise that most people had switched for good.

</comment></comment></comment><comment><USER15> Off topic, but the article mentions "deep learning at scale" which triggered me. Is this use of the term "at scale" something new(ish)? As far as I know "at scale" means something like "in the appropriate amount". Here it seems to be shorthand for "implemented in a scalable manner". This use seems all over the place now. Is there a native English speaker who can comment on that?

<comment><USER16> I am not a native English speaker, but quite familiar with the use of "at (large) scale" in similar context.

It means that one has deployed it/is using the tool with a non-trivial (could be anywhere from few 100s to few 1000s of machines) amount of CPU and/or data. In the context of serving, "large scale" could also mean the number of queries/second hitting the serving layer.

</comment></comment><comment><USER17> Brought my following down to 9 accounts last week and enjoying the flow again. I have always used Twitter passively, only re-tweeting 3-4 interesting stories a day. Hoping TensorFlow for healthy conversations does not spoil this sort of 24/7 breaking news from trusted sources, now. What I fear the most, is having my flow made of one tweet from sources I choose & one tweet TensorFlow suggests to me.

<comment><USER18> > I have always used Twitter passively, only re-tweeting 3-4 interesting > stories a day

That seems fairly active to me. In almost 10 years I did my first re-tweet last year.

<comment><USER17> You're right, of course, ehehe. Just avoiding to express personal opinions, so that no flamewars or endless arguments waste your day, is not passive enough if you still sign up once and sign in daily. Therefore, the next step is deactivating my account for good while following trusted sources without logging in at all? Twitter cookies would still track my activity, though.

</comment></comment></comment><comment><USER19> What could possibly go wrong?

</comment><comment><USER20> Did twitter drop Scala? Wonder how the story is with ML and Scala

<comment><USER21> No,they have not. Especially on the machine learning teams

</comment></comment><comment><USER22> Twitter has become a tragedy.

</comment><comment><USER23> PR sucks. It looks like PR.</comment>


Doing: CLI tool for remembering what you were doing and tracking what's done - <AUTHOR> https://github.com/ttscoff/doing#doing <comment><USER2> Looks very helpful. It would be great to hear a testimonial from anyone using this in their everyday workflow.

<comment><USER3> Not this tool but I use jrnl.sh with a similar work flow. Some days more than others - basically a quick brain dump when it feels like I run out of cache. Works great for me, especially with iTerm2 where I have a global shortcut for a mini shell to drop down so I can add notes without messing around with tabs or windows

<comment><USER4> Same here, I find it extremely useful to jot down stuff and just have an overview of the day. Do you use any other scripts for it, for it like analysing your tags etc?</comment></comment></comment>


Google Glass: The Opposition Grows - <AUTHOR> http://news.cnet.com/8301-1023_3-57574607-93/google-glass-the-opposition-grows/ <comment><USER2> I am a gadget geek just like any nerds out there. I like the concept of Google Glass but the practical application scares the shit out of me. Google Glass in its current form is every advertisers/data-miners perfect wet dream.

I can see myself smacking someone for taking videos/pictures of me without my consent, by holding a camera right in my face. Google Glass almost legitimizes that kind of behavior. "Oh I can record our conversation because the camera is inside the glass and look its shiny."

Like the video of this guy holding a camera to random people's face: <http://www.slashgeek.net/2013/02/26/google-glass-privacy/>

NOT OK!

<comment><USER3> The video doesn't prove google glass is invasive. It just shows that people don't tolerate jerks too well. Even without the camera, having some weirdo come up and stare at you without having the decency to respond to you is of course going to annoy people. Nothing to do with technology.

<comment><USER4> It also ignores many of the things you can currently do. I've recorded more then one meeting by simply leaving my iPhone on record mode in my pocket. Audio is frequently more damning then video, and anyone can do that these days without giving anything away.

The right to privacy in public is generally regarded as a right to casual privacy - in the sense that many people can hear and may capture or accidentally record what you say, but won't publish or broadcast these events.

It seems the problem with Google Glass is more that it's forcing people to consider what has been true for a very long time: that wherever you go, chances are you're recorded in dozens of ways, by dozens of people, and the only actual defense is essentially social ettiquette in handling this content (and the fact that most people just don't care about who you are or what you do).

</comment></comment></comment><comment><USER5> I used to withhold face-palm reactions to opinions that differed from my own on the premise that _all_ views deserve to be heard, but there is so much ridiculous opposition in the world now, that it's become burdensome.

Even reading this trash article was in just about all respects, a horrible waste of time.

<comment><USER6> I really have no idea, based on your short comment, what your position is on omnipresent surveillance. I, for one, have no problem supporting bars that promise an ass kicking to anyone who shows up wearing google glass - I'm hoping we all quickly agree that, with clear exceptions (repair jobs, field surveys, photo shoots, reporters) - that they won't be an appropriate social use of technology.

<comment><USER7> > I, for one, have no problem supporting bars that promise an ass kicking to > anyone who shows up wearing google glass

Bars can just ban photography and ask their customers to not have Glasses on without the need of any sort of ass kicking. I would never go to a bar that advertises ass kicking as a feature - independent of whether I have/wear Glasses or not.

</comment></comment></comment><comment><USER8> For anything really, sides rally amongst themselves. We have differing opinions on issues all the time. I have a side concern about how we consider technology, and science in general. I really think it should be elevated to something more like free speech. That the dangers of not producing and experimenting with technology far outweigh fears of it harming ourselves. Well maybe not a belief but a hope. Something to strive for.

</comment><comment><USER9> What would life be like if everyone wore a spy camera that transcribed all conversations and uploaded them to various internet services all the time? If you knew everyone you ever talked to recorded it all and put it on the internet?

Once your data is recorded, you can't take it back 30 years down the road if you decide you don't like the direction the company has taken, or if privacy laws have changed to no longer be in your favor. The data is already out there. If you think changes in the TOS are bad now...

And your info won't just go to Google. There will be apps. You'll "Post this conversation to Facebook" or something equally stupid. Now fifty different niche websites are accessing things you said to your friend that you never uploaded. (Also, remember seeing in the news that some jobs require your facebook password as part of the interview process? What if that became more common? What if it wasn't just facebook, but everything you ever did, said, or saw happen?)

Our personalities will change, too. We'll become accustomed to constant and total self-censorship

People's data will be subpoenaed too, or just made available for general monitoring in some dictatorship somewhere. Maybe you once had a conversation with someone who later is investigated for something, and out of context your conversation sounds terrible. Now they investigate you. This investigation won't lead to a conviction, but it might fuck your life up for a few months. And that's an optimistic scenario.

I don't really believe it's stoppable though. We're not taking smartphones back. This is just a smartphone strapped to your head. We probably will tell people not to wear them in certain places, but how long will that last? We'll get used to it, like we got used to the TSA. Technology is only going to get more ubiquitous. Maybe we can fight fire with fire. The cypherpunks have been doing it for a while.

Play devil's advocate sometimes. Google Glass may end up being the best thing ever, but we should always question things. If the answer to the question is good, then it doesn't matter that we asked, but if it's bad... we'll be damn glad we did.

</comment><comment><USER10> I think this was a far better summary of the opposition to Google Glass:

[http://gawker.com/5990395/if-you-wear-googles-new-glasses- yo...](http://gawker.com/5990395/if-you-wear-googles-new-glasses-you-are-an- asshole)

"If You Wear Google's New Glasses You Are An Asshole"

</comment><comment><USER11> _> They say that it will herald a world in which "privacy is impossible and corporate control total."_

They have this temporally reversed. This world heralded Glass. (Unfortunately. Stallman could say, "Told you so.")

</comment><comment><USER12> Like all things, it would have some good use and bad use. But technology WILL evolve, and many people would adopt. It ultimately settles at a healthy balance.

In general though, privacy has been dead on the net for a long time now. In life outside the net too, the guy opposite you could have the camera on while he holds the phone in his hand. So yes, some people would have issues with this, but generalizing that response to a majority would be exaggeration.

</comment><comment><USER13> Theres alot of new tech i am really excited about. Next generation smartphones & tablets, things like Leap Motion/Kinect/MYO, cheap and tiny computers etc.

But Glass isnt really one of them... I cant shake the feeling it will end up in the same strange spot as bluetooth headsets and it just has alot of potential issues with privacy, usability (voice only?) and general appearance.

</comment><comment><USER14> "There's no way of even knowing if someone else is recording you through their glasses from somewhere in the cloud."

I wonder if the author makes sure to turn their monitor to face the wall when they've finished working so hackers can't film them while they sleep.

</comment><comment><USER15> stopthecyborgs? Looks like more microsoft astroturfing. The backlash is growing in microsoft's dreams only.

</comment><comment><USER16> Resistance to technologies like Glass is pointless. They're going to happen, and a whole lot more to boot, like it or not. People running around demonising Google or Glass are just silly - Google is just one of the few global entities with the necessary resources and vision to consciously make the next big steps in a pretty plain sequence.

The real question is adaptation - as Clay Shirky says, it's all about learning how to work _with_ the wind, not against it.

</comment><comment><USER17> One of the things that hurt the Segway's early adoption was that it was legislated against its intended use as a walking replacement -- they were legislated against (or rather, considered powered vehicles) by many nations and local governments. They are still illegal to drive on sidewalks in the UK.

If enough people block the use of Glass, it could have an adverse effect on its sales and growth.

<comment><USER18> Were laws really the reason why the Segway didn't take off?

I figure it didn't take off for the same reason non-overly obese people don't use Rascal carts. People were worried about being perceived as lazy. They're also really expensive, so you'd be perceived as snobbish as well.

Just look at Gob's character in Arrested Development. They didn't stick him on a Segway because he's a prudent person that needs to get to places in a hurry.

<comment><USER19> I've always thought Segway's failure was simply because it created more problems than it solved.

It was expensive, large, you had to find somewhere to park it, you had to remember to charge it, it came with an aura of snobbishness (at that price tag, not entirely undeserved).

And it solved... what? In suburban America it doesn't go fast or far enough to get you anywhere in reasonable time. In urban America the distances people walk aren't _worth_ all of the above compromises.

The Segway makes no sense to the suburbanite whose main mode of transportation involved their garage door opener. The Segway also makes no sense to the urbanite who can walk 100 feet to the closest grocery store.

It was a marvelously cool technology looking to solve a problem very few people actually had.

<comment><USER11> It solved problems for cities and city planners. A Segway user could be like an enhanced pedestrian. You could have fewer stores serve a larger populace. It didn't get to do this because it didn't solve its own chicken and egg problem.

<comment><USER20> There are other, better, technologies in the same solution space, e.g. bicycles, which are more useful, familiar, _vastly_ cheaper, more reliable (e.g. don't need charging, simpler all-mechanical design), and already have tons of existing infrastructure. Bicycles have drawbacks (e.g. the "what do you do with it" issue common to almost all vehicles), but nothing that the segway doesn't _also_ have.

The segway solved nothing. It was a neat toy.

</comment></comment></comment></comment></comment><comment><USER21> It's good to see there's some sense left in the world.

</comment><comment><USER22> Resistance is futile - but I applaud the effort.

</comment><comment><USER23> What a shame, you are going to miss on all these ads that Google has them lined up for you. Google Glass apps might even sell the fact that your voice is coarse today and convince Johnson & Johnson for $1.50 off a cough medicine. Or use an app analyze your toilet bowl before flushing to see if Kellogg's cereal would benefit your diet. Or suggest some new makeup to your girl friends, maybe it's the lighting but it seems like they could use some new makeup. They'd love that. Who wouldn't, right? You are so hip.

Google Glass was designed by people that make a living by tracking you, your friends and by annoying the hell out of you with advertisements. That is the fundamental problem, it solves their problems, not yours. For almost everyone, there is no need to walk with glasses on, unless the designer of such glasses needs to know every step you take. Need directions? Check the phone and check it again 5 minutes later. No need to look like a glasshole.

<comment><USER7> Disclosure: I work at Google.

I wanted to explain myself when I downvoted you, but I am just going to say that I would rather be a glasshole than be that bitter.

<comment><USER23> _Disclosure: I work at Google._

Or changing the world one ad click and one snoop at a time. (since you started it)

Fine. Being bitter, assuming I am, can be transitory. Once your sell your soul for a salary, you can never get it back.

You have a couple of years before "Googler" becomes a toxic word. Maybe less, given Larry's focus on squeezing every penny out of the ecosystem, so plan accordingly.

<comment><USER19> I downvoted you, and I don't work at Google.

Get off the high horse - tell me what you do for a living and I can find just as morally damning words to say about you. Unless you are curing cancer at an orphanage for paraplegics, the morality of your career is, like the rest of us, more gray than it is black or white.

So stop it with the meaningless emotional rhetoric and attack someone's arguments on its own merits. To pretend that all Google does is shove ads in your face, and that their only accomplishments to date have been so, is either brutally ignorant or blatantly and deliberately disingenuous.</comment></comment></comment></comment>


'Jobocalypse' set to automate as much as 70% of jobs - <AUTHOR> http://www.dailymail.co.uk/sciencetech/article-2542113/Will-robot-jobocalypse-make-YOU-obsolete-2014-year-droid-takes-job-say-experts.html

<comment><USER2> this story has been told 1000 times - the canonical outcome is that more technical jobs (programming, repair, etc) will be created to maintain and create the robots. I'm pretty sure that the dynamics of the economy and governments would shift were we to reach 70% unemployment too, towards more of a leisure-oriented existence.</comment>


Ruby Code Analysis Tools and Practices - <AUTHOR> https://speakerdeck.com/dimazhlobo/ruby-code-analisis

<comment><AUTHOR> I'm sorry. English isn't my first language. I'm ashamed of that typo. But I can't change title on speakerdeck because it will break links :(.

</comment><comment><USER2> I know it's quite low but Analisis made me chuckle, sleep deprivation at it's best.

</comment><comment><USER3> I did knew only few of them, so it's nice, someone covered this many tools.</comment>


CSS usage on the web platform - <AUTHOR> https://developer.microsoft.com/en-us/microsoft-edge/platform/usage/ <comment><USER2> It would be much much useful, to show which ones are actually properly supported by IE, probably would be less work.

</comment><comment><USER3> transition is at 0%?

<comment><USER4> If you click through, they explain what's going on. [https://developer.microsoft.com/en-us/microsoft- edge/platfor...](https://developer.microsoft.com/en-us/microsoft- edge/platform/usage/css/transition/)

"Wait, transition is used only 0% of the time? That can't be right. This data is for a CSS shorthand (eg: background) which is normally expanded to the appropriate longhands by the browser's engine and placed into the CSS Object Model. In some instances this can not be done, such as when a site uses custom properties. This results in confusingly low numbers, but the data for the shorthand is actually in its appropriate longhand. Click the link below to see the data for all of the longhands that map to this shorthand."</comment></comment>