My 7-Year Old Daughter's First Programming Experience - <AUTHOR> http://dancali.io/blog/a-74-step-account-of-my-7-year-old-daughters-first-programming-experience <comment><USER2> How young can a child learn to program?

While I'm not ready to indoctrinate my four-year-old into the monkhood of true geekery while he's getting in his prime running around time, I was curious to see how logically he could think. My layman's knowledge was that children began properly interpreting and creating rules around the age of six, while four was still wild imagination territory.

For my experiment, I created an extremely minimal programming game, like a very stripped-down version of RoboRally: [https://dl.dropboxusercontent.com/u/5458223/robot_factory.jp...](https://dl.dropboxusercontent.com/u/5458223/robot_factory.jpg)

The image depicted is doctored. We had already completed four playthroughs the night before with a similar setup, however. He was able to figure out how to program the robot to collect all of its pieces (three Duplo blocks comprised the robot; successful completion of the robot wins the game). I demonstrated the rules once, and then let him try to play by himself with minimal guidance. One interesting strategy he discovered was placing the "instructions" (the cards with arrows) on the game board in order to create a path. [https://dl.dropboxusercontent.com/u/5458223/robot_factory_re...](https://dl.dropboxusercontent.com/u/5458223/robot_factory_redux.jpg)

A few weeks later, we played it again, and I taught him how to make new "instruction cards". We're not yet at the place where we've done anything with variables and abstraction. Apparently he liked the game well enough, because he independently recreated it at school:

<comment><USER3> A child using language functionally to get what he or she wants is essentially a form of programming. Example:

        1. result = ask_parent_or_candy(mom, nicely)
        2. if result != candy:
        3.     ask_parent_for_candy(dad, nicely)

I studied analytic and philosophy of language, and i feel that many people in CS don't see the forest through the trees when it comes to programming as such. While some code (e.g. C) handles much more technical aspects of what is actually happening, scripting languages are much better suited to the human mind, and i think creating a natural language language (<\-- that seems a bit redundant) is an achievable goal.

If i had a child, i would start her out on python, because python is essentially in english, and english is the only code i'd prefer to write, given the choice if there were a formalized english scripting language.

Even in python there is a bit of jargon that i think is an unfortunate holdover. Int vs float may be necessary, but "string" is a poor term for what is essentially text. I'm not saying we should change anything, because coding is becoming more like natural language organically, but again, if i were to teach my child coding (and i don't want to step on any cuteness of this story), i think a child would understand python better than anything with brackets and semicolons.

Full disclosure: i genuinely like python better than most languages.

<comment><AUTHOR> I actually thought about what I should start with if I were to teach coding and you're right I would probably start with something that's interpreted first of all. I was thinking Ruby , perhaps Python, you're right.

But the crazy thing is that I didn't plan the whole thing. It was more of a "lets build an app together" type of thing. My kids and I have been talking for a while about building an iPhone game together so that's where the whole iOS thing came from.

But you raised a good point. If you had a choice, what would be the best language to start off kids with? And maybe you're right, maybe there should be a new language built just for this purpose.

Maybe a bunch of us coding dads can create an open-source mini-lang of some sort with the sole purpose of teaching kids to code.

<comment><USER3> I figured you didn't start out with the intention, but yes, i agree that building a simple language for kids would be a very good idea. Perhaps you wouldn't even need words at all. The entire point of generative grammar, if i'm not mistaken, is that logic is pre-programmed into the brain.

<comment><AUTHOR> You know a long time ago I used to "code" in this language called LabView and it was all graphical. It was designed for electrical engineers. I remember working with electrical engineers that had no idea about coding and they would be able to build complex systems in LabView. You would drag and drop a box and attach it to another box with a connector and that would be a loop, etc. Really awesome. Maybe something visual like that, with colors, animations etc would work for kids.

</comment></comment></comment></comment></comment><comment><USER4> I don't have kids, but even if I had this entry would still be totally worthless for me. It's touching and nice, but can anyone learn anything from it? I don't really think that there are any useful tips on teaching 7 year old anything, I would even say that it's quite opposite, how could anyone post code like this in such context: @property (nonatomic) IBOutlet UILabel* label; ?

<comment><AUTHOR> I hear you, but if you check the timeline of the lesson, I didn't teach her every step of the code, that would be too much info off the bat. I wrote a bit of boilerplate code and her lesson - as I mentioned in the post - was just those 3 lines where we talked about variables mostly.

<comment><USER4> Have you considered that XCode and Objective C may not be the right tools for the job?

<comment><USER5> Something like Scheme would be appropriate.

<comment><USER6> Scheme with graphics (i.e. a LOGO module) sounds very good. Kids are much more impressed by getting pictures to appear than seeing a number output.

</comment></comment></comment></comment></comment><comment><USER7> This has been a super fun thing in our house: [http://scratch.mit.edu/](http://scratch.mit.edu/) It's also something a 7 year old can use on his or her own to learn concepts and make fun stuff.

<comment><USER8> For Android devices, there's also MIT App Inventor, which is a very scratch- like environment for writing mobile apps, available here:

[http://appinventor.mit.edu](http://appinventor.mit.edu)

In addition to basic I/O and animations, it's able to deal with a lot of the device capabilities, including SMS, GPS, dealing with the camera, and so forth.

Unfortunately, I don't know of any similar environment for programming iThings --- Apple fairly famously rejected a Scratch port from the app store in 2010:

[http://wiki.scratch.mit.edu/wiki/Scratch_Viewer](http://wiki.scratch.mit.edu/wiki/Scratch_Viewer)

although a "Scratch player" for Android is in alpha, and apparently, people are also working on pure HTML versions.

<comment><USER9> There is an app called Hopscotch for the iPad which is of similar structure to scratch. I can't recommend scratch highly enough, the kids at my local coderdojo really do some great stuff with it.

</comment></comment></comment><comment><USER10> This is brilliant and amazing to read.

What comes to mind is, we need a list of analogies in the most simplest form so that we can explain fundamentals like variables to kids. Hell, anyone who doesn't know how to code really!

75\. HN reads and smiles

<comment><AUTHOR> I think you nailed it my friend. That's exactly what was going through my mind as I saw myself trying to teach her what a variable is. My boys were playing with Legos and I was drinking coffee so I'm like alright, let's try that. We went through other analogies that I didn't blog about but one of them was how the whole IDE window (XCode) is like a table that you draw on and how a program is like a drawing you make in art class. So the purpose of coding is to use an IDE to create programs just like in art class you draw on a table and create art work. Also, just like you need pencils, colors, etc, in coding you need variables, methods, etc.

Kids get stuff like this really fast.

<comment><USER10> I'd be interested to see some more examples you come up with if you continue to teach your kids.

I'm re-learning basic trig right now for drawing OpenGL stuff, and I was trying to think what I could have told younger-me on why I should be paying attention in high school for this topic.

When I was in high school, it would have made me more attentive if people said, "Hey, this stuff that finds circle arcs and radius, you can use this in real life if you decide to get into gaming, graphics, .. (and other cool topics that would excite me)."

<comment><AUTHOR> Yeah it'd be interesting for me too to explore this further. Stay in touch via my twitter @idancali and I'll post stuff as soon as I get a chance. Until then, check out this post I wrote recently: [http://dancali.io/blog/you-too- can-become-a-computer-program...](http://dancali.io/blog/you-too-can-become-a- computer-programmer-heres-your-first-and-most-important-lesson-in-computer- programming)

It's about how I decided to pursue coding as a career when I was in high- school.

I think the main thing is to tell younger kids that coding is an art just like drawing and that there is no right or wrong way of doing stuff. They should not be afraid to break things and experiment and they should absolutely have fun building things, seeing things come to life. That's what got me into coding in the first place and I think that's what makes a world of difference. The rest, the variables and things like that are secondary.

</comment></comment></comment></comment><comment><USER11> I think it's great that the author found something that worked for him and his daughter, but this particular example seens a little complex. The first code I ever wrote was something like

10 PRINT "HELLO"

20 GOTO 10

That's 3 or 4 steps. No IDE, no GUI, no variables. A program I could re-create completely on my own the next day.

<comment><AUTHOR> Hey I hear you. When I started coding I was 9 and I did something like that in BASIC. It was awesome. I was actually considering what to start out with and I just went for iOS off the bat to see what happens. I also think kids these days pick up stuff way faster than 20-30 years ago. It's insane how fast they pick up things like touch interfaces, smartphones, tablets, etc for example.

<comment><USER12> Same here. I have a one year old now, and was just thinking how sad it is that there isn't an equivalent of a C-64 any more. Turn it on, _bam_ , a BASIC REPL in less than a second. Give the kid the language manual (all of 10-20 pages) and off they go.

Today, there is all this cruft they have to be shown (Cmd-R, XCode, etc.). If they screw up they can't just reboot.

There are benefits of course. Knocking out the power cord doesn't destroy the work in progress. There are non-volatile media. Screens have colors :-)

</comment></comment></comment><comment><USER13> I wrote a post some time back on teaching my kids to code, thought I'd include it here in case anyone found it useful.

[http://richiekhoo.com/post/18723300657/coding-with- kids](http://richiekhoo.com/post/18723300657/coding-with-kids)

</comment><comment><USER14> Every now and then I do "math" with my son (5 years now).

I just use a terminal and "bc". He loves to type in numbers and + or * and see the result. And for + we "guess" ahead of time what the result will be.

What amazed me early on: bc has variables. I explained it to him and somehow he immediately got it. You say A is 5 and B is 1 then A + B is 6.

His attention span is usually around 5 minutes, but each time it's fun.

<comment><AUTHOR> Good stuff and congrats. I'm always amazed to see young kids get into coding and math. I think that's awesome.

I was so pleasantly surprised when she got the whole variables thing. I think drawing analogies to real life stuff is a sure way to get kids engaged, especially if the props you use are toys. About the attention span - yeah I noticed my 5 (almost) year old wanted to try when he saw his sister having fun, but then he got bored after a couple of minutes.

</comment></comment><comment><USER15> My 6 year old and I often boot up the old 8-bit machine that I have reserved in a corner of my lab just for this purpose. I have to say - there is no better way to teach a 6-year old to code than to fire up a BASIC-running 8-bit machine from the 80's and let 'em at it. Something so satisfying about hearing him read out his code .. "10 ... do this .. 20 .. do that .." ;)

</comment><comment><USER16> That's one of doing it...the other would be something like this [1]. Not ashamed to say I had to beat the whole thing the first time I saw it because it was so brilliant. And I normally hate pointless puzzle games.

[http://light-bot.com/](http://light-bot.com/)

<comment><USER17> More apps should have a video showing you the gist of what's going on like they do.

</comment></comment><comment><USER18> Check out Hopscotch:

[https://itunes.apple.com/us/app/hopscotch-coding-for- kids/id...](https://itunes.apple.com/us/app/hopscotch-coding-for- kids/id617098629?mt=8)

</comment><comment><USER19> The author correctly says 'pieces of lego' into the cup - but previously says 'playing with their legos'. I'm sure they wouldn't say 'the children were looking at the sheeps', and then later on say 'the children counted the sheep'.

For whatever reason, this irks me when reading as a grammar error. It's like a klaxxon sounding in the middle of the sentence.

<comment><AUTHOR> I know what you're saying, and the funny thing is that when I wrote that, I actually paused and thought about the proper way of referring to LEGO. So if you google "playing with legos" there's a ton of articles that use that expression, such as this one on HuffPost for example: [http://www.huffingtonpost.com/2014/01/28/google- legos_n_4681...](http://www.huffingtonpost.com/2014/01/28/google- legos_n_4681201.html)

So kids 'playing with their legos' is valid given the fact that LEGO has become a household name and LEGO can refer to the brand, the game itself, while Legos, or legos can refer to the LEGO blocks themselves. It's sort of like using Google as a verb ([http://en.wikipedia.org/wiki/Google_(verb)](http://en.wikipedia.org/wiki/Google_\(verb\)))

Thanks for reading btw

PS. English is my third language so a grammatical error here and there wouldn't be that bad anyways :)

<comment><USER19> 'Legos' is a very North American term so you may have just picked up an (incorrect) slang.

This was an interesting discussion if you're genuinely interested: [http://english.stackexchange.com/questions/10839/legos- not-l...](http://english.stackexchange.com/questions/10839/legos-not-lego)

Most North Americans will have been brought up with the incorrect usage, and will defend it strenuously, but they are wrong. The correct plural is Lego, or, if you want to get specific (as you did in the article), lego pieces or lego bricks. The company spent many years trying to teach correct usage but gave up in the end. But correct is still correct, and if english isn't your first language, there is still time!

Even in the verb usage 'kids playing with their lego' \- you don't pluralize the term.

</comment></comment></comment><comment><USER20> That was totally awesome + cute. I have 5-month old girl and can't wait to teach her to program! :)

<comment><AUTHOR> Awesome, let me know how that works out :)

</comment></comment><comment><USER21> I like the focus on variables and values - very cool! congrats!

<comment><AUTHOR> thanks dude, I do believe that's the first fundamental concept that makes most sense in a 101 course/lesson. If you get that, you can move on, if you don't, then you have no ground to stand on</comment></comment>


Ask HN: What is your primary source of news in 2018? - <AUTHOR> <comment><USER2> In addition to HN and a bucket full of RSS feeds, I am reading the Federal Register quite a lot [https://www.federalregister.gov](https://www.federalregister.gov)

It mostly started as a work thing (gotta look for grants sometimes), but it has turned into a pretty informative source for what the government is doing.

</comment><comment><USER3> I listen to NPR on my (short) commute and read a couple Financial Times articles a day on average. I also read several finance, economics, and statistics blogs and newsletters; while those probably don't count as news, between all of those sources and HN, I manage to do a decent job of keeping up with politics and current events to the extent that I care about them.

<comment><AUTHOR> Nice. I too have interest about politics but news.google.com is what I use to get latest updates about it.

<comment><USER3> To clarify, I'm actually not all that interested in the day-to-day of national politics - I campaign and vote for the candidates that I support, but I largely don't care about the specific ways in which Trump is embarrassing the country at any given time. My impression is that news.google.com is a good source for that sort of thing, however.

</comment></comment></comment><comment><USER4> I don't have a primary source any more.

I use HN, obviously, but also browse some other tech sites.

I tend to look at the free online-papers, the Guardian, the Independent. I get a fair amount of news headlines from Twitter. I think in 2018 it is easier and yet more important to use a range of sources

<comment><AUTHOR> :) twitter

</comment></comment><comment><USER5> HN, browsing news.google.com, and a pile of RSS feeds consumed using RSSOwl. Also, Facebook, Twitter, Reddit, and G+, to some extent.

<comment><AUTHOR> Do people still use G+? :O

<comment><USER5> Why don't you login and check for yourself?

<comment><AUTHOR> Oh yes :p

</comment></comment></comment></comment><comment><AUTHOR> Upto what extent HN provide the useful news from day to day life? Can somebody tell about it. Please.</comment>


Sustainable Sources of Competitive Advantage - <AUTHOR> http://www.collaborativefund.com/blog/sustainable-sources-of-competitive-advantage/ <comment><USER2> I have been following Morgan Housel's (the OP) writings for a while. He has written some excellent essays. In terms of explaining important ideas, they come close to "PG-Level" (Paul Graham) in writing. If you are interested in reading more from Morgan, I recommend the following:

1\. The greatest story ever told: "Capitalism's success is built on a belief – a story – that, given the right incentives, people can work together to solve problems. It's worked magnificently. It's the greatest story ever told."

[http://www.collaborativefund.com/blog/the-greatest-story- eve...](http://www.collaborativefund.com/blog/the-greatest-story-ever-told/)

2\. The Reasonable Formation of Unreasonable Things "Few things have as much impact on your lifetime investment returns as the decisions you make during bubbles."

[http://www.collaborativefund.com/blog/the-reasonable- formati...](http://www.collaborativefund.com/blog/the-reasonable-formation-of- unreasonable-things/)

</comment><comment><USER3> Isn't willingness to fail a way of learning faster? Willingness to wait was also justified in terms of learning.

All the arguments seem to boil down to learn faster, adapt faster. Evolution's a story as old as time.

</comment><comment><USER4> How sustainable is having proprietary information? Assuming the information is valuable today or will be in the future. I guess less so today than in the past due to leaks and information spreading in general. But I am not certain if this makes a practical difference. The willingness to fail and wait touch on the value of having rare information.

See this blog post and the associated paper: [https://www.schneier.com/blog/archives/2015/09/the_declining...](https://www.schneier.com/blog/archives/2015/09/the_declining_h.html)

</comment><comment><USER5> Is there any way to express this advice in a more fundamental, perhaps more abstract, framework? I often find more enlightening than off-hand advice without any clear underlying structure.

<comment><USER6> Maybe there is no clean, abstract framework to express all this. Succeeding in business isnt as simple as methodologically following steps 1 to 10.

On the other hand, I found this advice very succint and expressed elegantly. "Be more empathetic than your competitors" is such an elegant idea and captures the meme "Eat your own dog food and be your own customer" perfectly.

<comment><USER7> The biggest and most successful companies show little evidence of empathy.

One of the most sustainable strategies is to create an oligopoly or de facto monopoly [1], and then squeeze customers for cash as hard as you can.

You will eventually run out of lock-in, but you can have a very profitable time before that happens.

A lot of people (me included) find this strategy morally objectionable, and of course if you're a customer, it's a huge pain in the ass when you're on the wrong side of it.

But as a business strategy it has obvious benefits. If you can create lock-in, where are your customers going to go?

The reality is that today's business environment selects against empathy at large scales. Making customers happy is fine for small scale service providers, but beyond a certain size, it's far more effective to work hard to eliminate customer choice.

You might wonder if business would work better if empathy was valued over cut- throat monopolistic land grabbing. That's an interesting question, and it's tempting to suspect that it would - not just in terms of customer satisfaction, but in terms of technological stability and speed of innovation.

But that's not the business environment we have today.

[1] Ideally in a limited market, so there's no danger of formal antitrust action.

<comment><USER6> Monopolies don't interest me because I can never become one. There is little practical value talking about how to become the next Time Warner or Verizon.

Becoming a popular "small" service is attainable however and it's hard to build a successful one unless you have empathy.</comment></comment></comment></comment>


A New Theory of Western Civilization - <AUTHOR> https://www.theatlantic.com/magazine/archive/2020/10/joseph-henrich-weird-people/615496/ <comment><USER2> Interesting although I would dispute assuming that others follow their mental framework as a Western or EIRD thing in itself. It seems to be more a baselining effect. I recall Chinese nationalist attempts at "outragous unthinkable insults" that were downright tepid even in countries without the equivalent hard 1st Ammendment protection.

Really everyone has to work from some reference framework even if it is "other cultures tend to think the same as ours/different" as an assumption.</comment>


Show HN: Heaptrack, a fast heap memory profiler for Linux - <AUTHOR> https://github.com/KDE/heaptrack <comment><AUTHOR> See also the release announcement for v1.0.0: [https://www.kdab.com/heaptrack-v1-0-0-release/](https://www.kdab.com/heaptrack-v1-0-0-release/)

And the initial proof-of-concept announcement: [http://milianw.de/blog/heaptrack-a-heap-memory-profiler- for-...](http://milianw.de/blog/heaptrack-a-heap-memory-profiler-for-linux)

Feedback welcome, patches even more so: [https://phabricator.kde.org/dashboard/view/28/](https://phabricator.kde.org/dashboard/view/28/)</comment>


Helvetica vs. Arial WebApp - <AUTHOR> http://tumult.com/hype/gallery/Helvetica_vs_Arial_WebApp/Helvetica_vs_Arial_WebApp.html <comment><USER2> This is possibly the worst UI I've ever come across. I had no idea what to do or how to do it. What is this even supposed to help me accomplish?

<comment><USER3> Welcome to _mobile first_ web development.

This tool seems to help you learn about Arial and Helvetica and how they differ despite their similarities. Personally, neither is a very suitable font these days unless you want to explicitly invoke either a Microsoft or an Apple 'feel', respectively. These fonts are tainted with those connotations by decades of association.</comment></comment>


Show HN: AutoProctor – Proctoring of Online Tests Using TensorFlow - <AUTHOR> https://www.autoproctor.co/ <comment><USER2> Have you thought about an upgraded service to have you review students data manually? Customers might pay a premium to have a human look at it so they don't have to choose between looking at it themselves and trusting your AI.

<comment><AUTHOR> Good point. There are other proctoring services that offer a human proctor. So, at one level, we would be competing with them.

But, more importantly, we tell our teachers that they should think of AutoProctor as more of a preventive tool. Just knowing that they are being monitored means that students are much less likely to cheat. Teachers tell us that student grades have dropped significantly since they started using AutoProctor. So, at least as of now, it seems to be working.

Another area we would have to worry about with a human reviewing the evidence is the Privacy. For now, only the algorithm and the kid's teachers have access to the data. Opening it up to third-party contractors would mean having to deal with all the issues that brings along.

</comment></comment><comment><USER3> Fantastic! Cannot wait to try it out!

<comment><AUTHOR> You can try it by clicking on the 'Try It Yourself' button on the homepage!</comment></comment>


Forthcoming OpenSSL releases - <AUTHOR> https://mta.openssl.org/pipermail/openssl-announce/2016-February/000063.html <comment><USER2> Use LibreSSL: [https://en.wikipedia.org/wiki/LibreSSL#Security_and_vulnerab...](https://en.wikipedia.org/wiki/LibreSSL#Security_and_vulnerabilities)

Support the OpenBSD team:

[http://www.openbsdfoundation.org/campaign2016.html](http://www.openbsdfoundation.org/campaign2016.html)

that makes the LibreSSL.

<comment><USER3> LibreSSL can't be used everywhere OpenSSL is currently used. Notably (and intentionally) missing is FIPS 140-2 support, which most (almost all?) commercial product that uses OpenSSL will rely upon for selling a FIPS 140-2 compliant or validated product.

Personally, I find Network Security Services (NSS) much better designed than OpenSSL/LibreSSL and wish more things would use it. Notably better is the use of an actual database to store objects. This really helps when you are using certificate revocation lists (CRLs) which may be huge blobs that change frequently.

<comment> X-Istence Wasn't there recently an announcement that OpenSSL was planning on removing FIPS 140 support in the near future and possible add it back at a later point in time?

Specifically mentioned here, OpenSSL 1.1.x won't have FIPS support: [https://groups.google.com/forum/#!searchin/mailing.openssl.d...](https://groups.google.com/forum/#!searchin/mailing.openssl.dev/FIPS%7Csort:date/mailing.openssl.dev/k-71_d9ql_g/s1ItwMr4BAAJ)

<comment><USER3> OpenSSL 1.0.x will likely remain supported (by someone, perhaps if not OpenSSL) for a long time due to this decision.

</comment></comment></comment></comment><comment><USER4> Glad they are fixing these but damn it is scary every time I see OpenSSL in a headline anymore.

btw

         HIGH Severity.
         This includes issues that are of a lower risk than critical,
         perhaps due to affecting less common configurations,
         or which are less likely to be exploitable.
         These issues will be kept private and will trigger
         a new release of all supported versions.

</comment><comment><USER5> And we just finished rebooting everything for glibc…

<comment><USER6> I wonder what kind of new internet will emerge from the ashes of everything we've been using since the 1970s, when it finally all goes up in flames in the next few years.

<comment><USER5> I think you underestimate the sheer tenacity of crappy solutions.

<comment><USER7> I think you underestimate how badly things are failing.

<comment><USER5> You'll need a really, really, _really_ massive meltdown before people are willing to fork over the money necessary to restart from scratch. Not even nuclear meltdowns have historically convinced people to do that. Not even global warming is convincing people to do that.

</comment></comment></comment></comment></comment><comment><USER8> It's kind of unsettling to know there is a known vulnerability (at least known to some) out there and is going to stay unpatched for a couple of days. On the other hand, it is kind of nice to be able to brace for it mentally.

Does anybody know why the update is announced a couple of days in advance? Are e.g. maintainers of corresponding packages in Linux distros or *BSD given access to the code ahead of time so they can build new packages?

<comment><USER9> >Does anybody know why the update is announced a couple of days in advance?

For server operators to be prepared. (And I would prefer if they would narrow the timescales more for that.)

>Are e.g. maintainers of corresponding packages in Linux distros or *BSD given access to the code ahead of time so they can build new packages?

Yes, but that doesn't require a public announcement.

<comment><USER8> Thanks for clearing that up!

</comment></comment></comment><comment><USER10> Eager to see how this LibreSSL comparison chart will be updated: [https://wiki.freebsd.org/LibreSSL#LibreSSL_.28and_OpenSSL.29...](https://wiki.freebsd.org/LibreSSL#LibreSSL_.28and_OpenSSL.29_Security_Vulnerabilities)

<comment><USER11> The "totals" seems a bit off...

16 + 7 = 19?

5 + 31 + 12 = 36?

<comment><USER8> It is possible that some vulnerabilities affected both LibreSSL and OpenSSL. In those cases, the total would be smaller than the sum of its parts. However, that is just speculation on my part.

<comment><USER12> "total" in context is for a single project, the things beying summed are different levels of severity.

<comment><USER8> Oh. In that case, I have no explanation to offer other than "math is hard", which does not sound very plausible in this case.

</comment></comment></comment></comment></comment><comment><USER13> Mirror available here, in case the official site stops loading: [https://marc.ttias.be/openssl- announce/2016-02/msg00001.php](https://marc.ttias.be/openssl- announce/2016-02/msg00001.php)</comment>


Improving YouTube video thumbnails with deep neural nets - <AUTHOR> http://youtube-eng.blogspot.com/2015/10/improving-youtube-video-thumbnails-with_8.html <comment><USER2> I wonder if they included sexy images in their negative training sets -- many videos accrue millions of views (and ad dollars) by having a few frames of cleavage interspersed with other (often derivative) footage.

It would be great if their algorithm picked a thumbnail that reflected the _entire_ video, not just a few frames specifically chosen to game people's compulsive clicking.

<comment><USER3> Most of those are just manually selected thumbnails by the uploader. After uploading, YT gives you 3-5 thumbnails you can choose from.

Also, partnered accounts are allowed to upload custom thumbnails (which can be any image, not necessarily even a screenshot from the video).

<comment><USER4> > After uploading, YT gives you 3-5 thumbnails you can choose from.

Can you pick an arbitrary video frame, or only one of the suggested thumbnails?

<comment><USER3> It automatically captures 3 different thumbnails (I guess using the algorithm in OP) and lets you select any 1 of those 3.

<comment><USER5> I presume they use the image selection as training data too—if not that seems like awfully low hanging data fruit.

</comment></comment></comment></comment></comment><comment><USER6> It looks like they prefer images with a few large faces near the center of the frame. That's probably the right answer for social media. (Plus a cat recognizer.) Used on news footage, you probably get the talking head rather than the news event.

<comment><USER7> We can't guess as to how the NN is preferring images, but it looks to me like it's preferring images with a high entropy in certain regions

</comment></comment><comment><USER8> There's an outside company that was working on this: Neon Labs ([https://www.neon-lab.com/](https://www.neon-lab.com/)).

Their insight is that not only are there images that are "high-quality", but also images that are positive. Positive images get more clicks, over just a decent image. I wonder if that information is encoded in the RNN in some way.

(This is where I'd normally rant about RNNs and other ML techniques hiding this information from their creators by locking it up inside the black box, but I'll save that for another day.)

</comment><comment><USER9> They've got to be training on more inputs than mentioned. For example, is one or a close set of times in the video linked externally and generating traffic? Grab the entire set of frames from that time period and run it through the quality classifier, there might be iconic frames from that section that people are looking for.

Are people re-watching a small segment of the video? Try classifying individual frames from that segment or just before. Of course, those are often action moments that result in smeared motion and artifacts and may not result in a quality thumbnail.

These ideas also only come into play when a video has been live for a while, after the uploader has initially picked a thumbnail. Maybe a "We have some new thumbnail suggestions for you, take a look" alert or message?

</comment><comment><USER10> So, in an article about image processing, why not include nice big beautiful images, that get even bigger when you click on them?

I click on the low detail inline images, and they stay the same disappointing size and reveal no further detail.

They're all, like 600px X 200px? Am I being greedy for want of gigantic images, upwards of 3000px wide?

I suppose it _is_ an article about thumbnails, after all, so maybe I shouldn't be so surprised.

</comment><comment><USER11> Seeing this run through an equivalent of the deep dream visualizer could be really interesting -- what _are_ people looking for in thumbnails? I'm having difficulty imagining what features would even be relevant in such a situation.

<comment><USER12> I'm guessing: "sharpness" of image, good saturation, presence of (smiling?) human faces, non-human mammals facing the camera, bare human skin (?)

(I agree that'd be cool.)

</comment></comment><comment><USER13> Meanwhile, I still can't edit a playlist while playing it.

edit: constructively put - there's simpler stuff to fix UX and match user patterns still isn't there?

<comment><USER14> When you have a big system, the most consistent argument against working on one thing is that you should be working on something else, this is true for everything in the system, because everyone has a different opinion on what that thing is.

For example: why should you spend time working on the playlist playback when youtube could instead spend time working on automatic categorization, content creators have to manually create playlists, even if they sequentially number their videos. Youtube shouldn't waste their time on playlist editing when it could be doing the right thing automatically.</comment></comment>


Google Co-Founders Talk Regulation, Innovation, And More In Fireside Chat - <AUTHOR> http://techcrunch.com/2014/07/06/google-co-founders-talk-long-term-innovation-making-big-bets-and-more-in-fireside-chat-with-vinod-khosla/

<comment><USER2> Google is working on something new. [http://geekthem.com/google-co-founders- talk-regulations-inno...](http://geekthem.com/google-co-founders-talk- regulations-innovations/)</comment>


Ask HN: Canadian HS student getting job in SF? - <AUTHOR>

I am senior in highschool (17 years old) going to university next year. Before heading off to university I would really like to spend the summer in San Fransico.
Over the past 4 years I have worked at several (3)startups doing mainly data science work. One of  startups got acquired and the other 2 are still going strong. I also do some freelance web development work on the side, mainly to pay for school.
I would just like to know what the best course of action would be for me. I know its hard to get a job for one summer, 2 months is a small amount of time. So, the company would have to take a risk on me.
I was thinking of renting a place on AirBnB to save some money. My father has a U.S citizenship so I have the whole work Visa situation under control.
In general, how feasible do you guys think this is? I have the year to prepare for the trip. <comment><USER2> First - that sounds awesome. Just generally, I pretty much always think an experience like that is worth so much, regardless of if it goes as expected or not. So if you can make it happen, try it!

Onto the details - It will be expensive. AirBnB is probably also not going to be very cheap. Your best bet is probably to find a sublease from someone on Craigslist. You can probably get a room at a decent price doing that, but it would need to be done in person to avoid CL scams.

For the job I probably don't have the best perspective to give you a perfect answer, but generally I think it would depend most on what your expectations are and what size the company is. A small startup probably won't have the time or energy to expend on a 2 month temp. A larger startup might, but the challenge would be to get their attention.

You're best bet might be to apply to some sort of internship program - I think I've seen a couple larger startups have things like that. Not sure about the pay though. Or you could try to find a company that needs contract/freelance work.

<comment><AUTHOR> Thanks a lot!

Craiglist has this stigma associated with it so I generally try to avoid it. But maybe I will give it a try!</comment></comment>


If you want to be massively successful, do not set ambitious goals - <AUTHOR> https://medium.com/accelerated-intelligence/if-you-want-to-be-massively-successful-do-not-set-ambitious-goals-according-to-studies-affa9cd39f5d <comment><AUTHOR> Has anyone read the book this is based on ("Why Greatness Cannot Be Planned")? I'm worried that this article is making a leap from "most successful people got there by pursuing novelty rather than goals" to "pursuing novelty is a better way than goals to become successful."</comment>


Siri Can Open Up Your New iPhone to Attack  - <AUTHOR> http://www.msnbc.msn.com/id/44966147

<comment><AUTHOR> "To prevent someone from accessing Siri and taking advantage of what you thought was your own private, personal assistant, go to the Passcode Lock screen and disable access to Siri on locked phones."</comment>


Show HN: MyBucket.co, a bookmarking app - <AUTHOR> http://mybucket.co

<comment><AUTHOR> This is bookmarking app that I built and run for several months now. And I'd like to know if it can be useful for HNers.

It's a point and click bookmarking tool without the busy work of tagging, etc. And yes, it does have migration tool from delicious.

Some of the nice features it have:

* There is no plugin to install. Just bookmarklet.

* The app does not ask for password.

* Bookmarked images are shown as thumbnails.

* Bookmarked videos are shown as mini-clip.

* You can come back to your bookmarks by spawning the widget, when you want to.

* full-text search on your bookmarks.

<comment><USER2> It's neat, but... I do miss tagging. Even if it's tedious, once I have a few hundred links stashed away, widly varying in nature, I'd love a way to organize them a tiny bit.

Tags were a great tool towards that goal. Not perfect, but worked well enough.

<comment><AUTHOR> Can full-text search help you in organizing the bookmarks?

<comment><USER2> Not really, no, I'm afraid, unless I can add a description of some sort to my bookmarks (at which point, it's not much different from tagging, at least in my case) - neither the page titles, nor the urls are good search material for my use case.

<comment><AUTHOR> Ah, MyBucket.co scrapes the whole html and indexes the text for search.

The text is put in description field which you can edit.

Perhaps that answers your concern?

<comment><USER2> Ah, I see. Hrm... that might work. I'll certainly test it for a while.

Thanks!

</comment></comment></comment></comment></comment></comment><comment><USER3> clickable: <http://mybucket.co/></comment>


An Entrepreneur Who Took a Chance on Herself - <AUTHOR> http://www.nytimes.com/2010/07/11/jobs/11pre.html?_r=1 <comment><USER3> Obviously NYT was having a slow day. However, what I'd like to point out is the discrimination: Do you think the same piece would be created if the entrepreneur in question was an average looking guy? People always talk about how women get discriminated against in CS, EE and the like (which _is_ true to a point) while refusing to see the advantages of being a woman in these areas.

When we were interviewing people for positions, I always sensed somewhat more enthusiasm before interviewing a woman in my colleagues, even if it's a _phone interview_.

You may find these comments Summers-like, but let me know if you have not experienced similar things in your work environment.

<comment><USER4> Disagree.

The NYT site didn't make it easy, but I just browsed about 12 months worth of this column(?) "Preoccupations."

Female and male profiles were about 50-50, and everybody else seemed pretty average looking.

Less than half the time is it a profile of an individual, the rest of the time it's a profile of a "trend" (This Hashrocket pair programming article that was on HN a while back, is from this same NYT column: <http://www.nytimes.com/2009/09/20/jobs/20pre.html?>)

All the rest of the profiles were equally as interesting-but-not-mind-blowing. (Though I'm less critical than you about the slow-news-day part. That's "slice of life" journalism. It's always like that, heh. In that regard, every day is a slow news day. )

I suggest a study: One where when a post's author/subject is a female, we count how frequently her appearance is brought up when it's not otherwise the topic of the post, versus when it's an average looking guy posting about how he organizes his email folders/ moved to git/ almost took VC.

Does that stuff get called a slow news day?

<comment><USER3> An interesting addition to your proposed study would be this: in addition to other people, how often does the person covered push their appearance. This was my main objection to this lady, when you go to her web page, her modeling and vanity shots are predominant, e.g. this is how she chooses to represent herself. She may be thinking that this would be advantageous in a male dominated profession.

<comment><USER4> Heh, I didn't check out her site. It sounds obnoxious. (And I see from your other HN comments, we share a distaste for another cheeseball who flaunts his/her appearance, w/multiple "in action" shots all over his/her website, who I won't give further attention by typing his/her name :D).

But I'm simply objecting to you using the word discrimination, and suggesting an average looking guy would not get this piece written. I even covered the "citation please" angle (not that you said that, but so many people on HN abuse that construct as a way to discredit pretty reasonable ideas, without having to provide a citation themselves.)

I'll take your word that she's pushing her appearance, just not that it's responsible for her coverage.

And my "proposed study" tone was maybe too snarky, I apologize. (I try to avoid that), I just had to give you a hard time about using "discrimination" and "average looking guys" in the same posit. Are we on the same site, seeing the same stories that make the front page?! (Said in friendly tone, poking as much as fun at myself for reading, as HN for posting 99% of its content about/by average looking guys, usually comparing Android + iPhone.)

<comment><USER3> I agree with you, "discrimination" was perhaps too strong, a subliminal preferance, perhaps?

BTW, this is by far the best/nicest response I got to one of my comments.

</comment></comment></comment></comment></comment><comment><USER5> It is a well-written and interesting article. So unlike some of the comments below, I wouldn't chalk this up to some sort of discrimination (in favor of women)

It is true that her history/accomplishments aren't necessarily extraordinary. A few years ago, in a salaried job, she was "was on track to be earning six figures within three years." Now, she has one iPhone app that is doing well in the app store (the app never made it to the top 3 in the Music category, but it is currently at #7 in Music)

I think that her accomplishments not being extra-ordinary may make the story more interesting to a lot of people. The fact that it is #5 on HN is a testament to the story's relevance. Kudos to Prerna Gupta for a well-written article, for generating positive PR for her company and also for taking the risk of leaving her salaried job to take up entrepreneurship.

She shares the last attribute with many of us on HN :)

</comment><comment><USER6> Oh now I remember who the entrepreneur actually is. She started one social networking site in India called <http://yaari.com> which was absolutely spammy. Mailed tons of spam endlessly - probably that's why it didn't work out and now that site simply shows her personal blog.

</comment><comment><USER7> She's a Stanford grad, entrepreneur, surfer, model and apparently she has been published in the NYT. She also created this diet, which looks interesting: <http://thenaturalfoodsdiet.com/about.php>

I had no idea human beings like this existed. I feel inferior.

<http://prernagupta.com/>

<comment><USER8> There are tons of people who have bios like this, especially in San Francisco. You quickly come to realize that most of the people who have done 50 million different "impressive" things in a short period of time are either dilettantes who seek the most recognition for the least work investment, or people who have one or two serious interests, and a strong ability to promote all of their other minor achievements.

(Protip: my vanity meter goes off the scale when the self-promoter is a woman who prominently features her amateur modeling portfolio on a personal website, or intermixes pictures of herself in a bikini in otherwise unrelated content.)

I think the thing to learn here is that noise level for self-promotion is set pretty high. If you think you're bragging too much about an accomplishment...well, you probably aren't. ;-)

<comment><USER9> Any time someone lists "Microsoft Word" as a skill on their resume (other than a secretary, and then I'd expect a lot more detail) my gut reaction is to assume they're the kind of person who enjoys listing accomplishments no matter how deep or meaningful.

<comment><USER10> Add to the list those who write: ASP,XML,ORACLE,SAP,Javascript,AWK,GREP,PERL,.Net and few more capitalized abbreviations

In fact beauty is in minimalism. Such as Google which just say C++,Java and Python

</comment></comment></comment></comment><comment><USER11> I find it mildly interesting that the NYT article omits CS from her dossier. Does branding her as purely an Economics grad make her more of an 'everyman' and thus result in a more relatable story?

<comment><USER10> yeah CS would mean she is geek, shyaa. They must keep the whole econ major models can design an Iphone app feel!

</comment></comment><comment><USER12> _Maybe I value my time more than my net worth. Maybe my fear of boredom outweighs my fear of failure. Or, maybe I have an irrational belief that I will succeed against all odds. Whatever it is, I find the risk of entrepreneurship to be not only worthwhile but also necessary for fulfillment. Work is no longer work. It is life, and a good one._

Is it just me or do those last paragraphes sound like beautiful poetry?

<comment><USER13> Sounds more like banal, run-of-the-mill inspirational speech (not that I have anything against that)

<comment><USER3> Relief! So I wasn't the only one thinking that. It sounds like a warmed up version of Timothy Ferriss-like mantra.

</comment></comment></comment><comment><USER14> She gets points for cajoling a fluff self-promotion piece out of the nytimes. That's not easy.

<comment><USER15> And _very_ effective, I was stunned how much sales a mention in the NYT can bring. Wish I could repeat that for my next startup.

</comment></comment><comment><USER16> Her other venture was Yaari.com, which was a scammy dating site aimed at Indians. It was big with some Indian VC's here in Bangalore but eventually got shut down.

</comment><comment><USER17> the social networking site if I'm not mistaken is www.yaari.com (targeted towards indians) -- as someone who's closely followed the indian startup scene, i seriously doubt if they had anywhere close to 2 million users. besides, think about it - if you have close to 2 mill users, why would you just abandon it ?

</comment><comment><USER18> _...I started Khush Inc., which makes an iPhone music application called LaDiDa. It's a kind of reverse karaoke — it creates background music when people sing lyrics into a microphone..._

Someone managed to make Microsoft SongSmith into something that doesn't suck?

</comment><comment><USER19> One of the more interesting profiles to be featured on NYTimes - <USER2> for the fact that she could easily have had an easy high-lifestyle life financed by her consulting career and gave it up to make her own destiny.

I know someone who worked for the Monitor Group - which is the unnamed consulting group - she was super sharp.

</comment><comment><USER20> The title sounds like a tautology. Don't all entrepreneurs take a chance on themselves, by definition? :)

</comment><comment><USER21> In spite of the positive outcome for that woman, do you find that article having a depressing effect for you? She was on track for "six figure salary in 3 years" and worked for very reputable enterprises, and still she was not able to come up with any realistic business-model for herself.

An iPhone app while definitely an achievement may as well serve as a proof of a "normal" web-oriented startups going out of fashion.

<comment><USER21> These who downvote silently are the freaks afraid of expressing their opinion.

Isn't the whole purpose of a website like HN to have discussions?

</comment></comment><comment><USER10> Sorry but I really find her website too cheesy!

Someone who claims to have 5 Million $ worth web company should know better.

Also her modelling portfolio is like HFCS worst than cheesy.

<comment><USER22> haters gonna hate! srsly?

<comment><USER10> dude please check it out. <http://prernagupta.com/> its nor functionally oriented nor asthetically oriented nor minimalistic.

Its a wordpress blog with different pages done by making them categories, the top menu bar gives it a feel of the website.

Yet its just a wordpress blog. I have seen much better cheap templates for wordpres.</comment></comment></comment>


Apple, Google, Facebook condemn GCHQ proposal to eavesdrop on encrypted messages - <AUTHOR> https://www.cnbc.com/2019/05/30/apple-google-and-whatsapp-condemn-gchq-ghost-proposal.html <comment><USER2> The GCHQ article in question: [https://www.lawfareblog.com/principles-more- informed-excepti...](https://www.lawfareblog.com/principles-more-informed- exceptional-access-debate)

>We're _not_ talking about weakening encryption or defeating the end-to-end nature of the service. In a solution like this, we're normally talking about suppressing a notification on a target's device, and _only_ on the device of the target and possibly those they communicate with. That's a very different proposition to discuss and you don't even have to touch the encryption.</comment>


A startup's plan to sell solar like cell phones - <AUTHOR> http://gigaom.com/cleantech/a-startups-plan-to-sell-solar-like-cell-phones/ <comment><USER2> From a marketing perspective this seems absolutely genius.

From an engineering perspective, it seems completely backwards. The coupling between the actual cost of the system and what the user pays is minimal: drawing more power doesn't actually increase wear on the solar cells. Really, all they have done is wired a meter whose sole purpose is to waste electricity if the user isn't paying.

If it works, and if it successfully brings electricity to people who would really benefit, I can't argue with it. It just seems so very wrong from an engineering perspective.

</comment><comment><USER3> Doesn't this model work for cell phones because they are really cheap and the provider doesn't care if they get lost/stop paying? Afaik solar panels are expensive to make so how does this make any sense?

<comment><USER4> A 50W solar panel isn't all that costly (150-200$). This model is just a way to 'loan' these devices for consumers who can't afford even those. You will be surprised by how much utility these small capacity solar panels might provide even for those Indian consumers connected to grid (if they provide 2 lamps/ 1 fan as mentioned). This is because in rural areas, power cuts of around 8 hours/per day are very common.

</comment></comment><comment><USER5> I wonder what their overhead is for going to rural areas to install/service these devices.

<comment><USER6> Probably not too bad since labour costs in rural areas are very low. I'm not sure what they are now, but unskilled labour used to be about $3 a day. I remember when I was in India seeing guys climbing telephone poles with no safety equipment whatsoever, just raggedy street clothes and flip flop sandals.

</comment></comment><comment><USER7> Curious how they would keep them from not just ripping off the meter?

<comment><USER8> I wonder why they chose a pay-as-you-go system depending on how much energy they use.

I think this would be much better as a rent-to-own program with a flat monthly fee.

<comment><USER9> Seasonal work, summer hours, school and other needs take precedence over what a family spends money on, on a month-to-month basis. Pay-as-you-go electricity is common in some western countries and it is not unusual for there to be days where the family cannot afford to top-up until pay-day again, where food etc comes first as a primary need.

</comment></comment></comment><comment><USER10> Cell phone can be cut off at anytime so as making it easy to link its access to continuous payments. The company has little leverage against the cell panel users due to non-payment.

<comment><USER11> These look like pre-paid power-minutes or something similar. So you pay for 500 minutes of power, and when you've used it all the power stops. When the solar panel knows when you've paid for however many minutes are required to "own" the device you stop having to buy minutes for it.

So they could run the risk of people just not paying for the device anymore because they can't afford or don't want it anymore, or tamper with it to try and get it working without paying it off. They'd still have the 10% initial fee plus any minutes they paid for already.</comment></comment>


Girls and Computers - <AUTHOR> http://rmurphey.com/blog/2012/03/25/girls-and-computers/ <comment><USER2> The thing that stands out in many of these stories are these three points:

       * Accessible price point,
       * Instantly usable to write code,
       * a place to go when the capabilities ran out.

It is very challenging to reliably buy this today. When my daughter wanted a computer a gave her a VAX 4000/VLC, it had BASIC, C, FORTRAN, and COBOL installed and could run adventure. I got it for free from a junk pile.

I'm going to try and fix that. [edited for formatting]

<comment><USER3> I would add one thing:

      * impoverished environment

This, I think is crucial. Given a Linux machine with a rich GUI or an iPad, kids focus on the eye candy and start playing with those things. A (relatively) limited environment at the same time focuses them and makes them creative.

<comment><USER4> I'd add another thing, both good and bad: turning if off returned it to a known state.

This meant you could experiment with it, to the point where it broke, secure in the knowledge that recovery was as far as a quick flick of the power switch. The negative was that for complex projects, where you wanted to preserve the state of the machine, you had to play cassette roulette.

<comment><USER5> Tape drives for Commodores and Acorns at least, were pretty reliable, just slow.

<comment><USER4> The VZ-200 (contained a TRS-80 rom) was atrocious!

</comment></comment></comment></comment></comment><comment><USER6> It's interesting to me how little difference there is between her experience and my own (male) early experience with computers.

Though on second thought, it's not at all surprising.

<comment><USER7> I think we have a tendency to assume "people like us" are, well, people like us. Some things are more universal than we think.

</comment></comment><comment><USER8> I'm not sure if I can see the correlation between the story being told and the fact that the author is female. It seems to be briefly mentioned in the first and last paragraphs.

<comment><USER9> > I'm not sure if I can see the correlation between the story being told and > the fact that the author is female.

I saw a quip the other day that was along the lines of "Hacker News posters consistently make the mistake of assuming that, because a post shows up on Hacker News, the author is somehow 'making a big deal out of it'".

The "correlation" is that the author, after a spate of sexism stories regarding women in tech, got to thinking about how she got started as a woman in tech, and wrote a blog post sharing the story.

It's a slice of a story of someone's life, nothing more, nothing less. Don't try to read too much into it looking for larger correlations and grand overarching Big Deal Points.

<comment><USER10> You're probably right, but the post is titled "Girls and Computers", which kind of seems to imply some deep, and general, theorizing about girls and computers.

<comment><USER9> > You're probably right, but the post is titled "Girls and Computers", which > kind of seems to imply some deep, and general, theorizing about girls and > computers.

Or that she's been spending some time thinking about "girls and computers" after the news stories about girls and computers and it made her think of her own story.

That's how I read it, anyways. Obsessing about the "deeper meaning" of a 3 word title and whether or not it is the best description of the content seems a bit pointless.

<comment><USER11> Article titles should match their content.

<comment><USER9> I'm not convinced it doesn't, but either way, this is pedantry of the most unproductive sort.

</comment></comment></comment></comment></comment></comment><comment><USER12> A lot of the responses here seeking a modern alternative to this experience seem to be focusing on hardware. In a way that's not surprising but I think it's a bit of a shame. That's because I think we have a platform today that can rival the ease and immediate feedback of those early computers: the web browser. There are probably a couple things needed to complete the picture.

1) A nice basic library that can serve as an immediate stepping stone to the UI. There's probably some out there already that are very beginner friendly.

2) Some kind of REPL/IDE like browser extension to make it easy to dive in right away. Something a bit easier for kids to wrap their head around than the developer tools of today, but also incorporating a basic editor so they can edit in place, save files etc.

I'm not really sure about the form of (2) or how vital it is, but it certainly wouldn't hurt.

<comment><USER13> I'd second this suggestion. Another great 'feature' of this platform is that it's trivially easy for the learner to share his/her creations with friends.

</comment></comment><comment><USER14> I doubt it will be possible to recapture the golden age of early PC's and their ability to get kids programming. The computing landscape has changed, and educational practices must change with it. We have to assume kids will have ipads as their first computer and treat them accordingly.

Kids these days are web users first and foremost. You have to let them cross the gap between visiting sites and creating sites. There's plenty of opportunity for sites that let kids create stuff in javascript and share it with their friends. What's so different from animating a canvas using javascript and animating a tv screen using basic?

</comment><comment><USER15> I guess Children of Tech savvy parents become hackers...</comment>


Ask HN: How Do You Come Up With Blog Post Ideas? - <AUTHOR>

It seems like every one in technology has something to say, and many of them have their own outlet for saying those things (that end up getting posted on HN!)<p>What is your source of inspiration for deciding what to put on digital paper and share with the world? <comment><USER2> I don't blog a lot so far, but I do have a list of possible topics somewhere. I tend to add to it when:

\- I feel like I've come to realise something new (for me) that others might find insightful too. These don't turn into blogs often, but it's helpful to have it written down somewhere nonetheless.

\- An opinion of mine keeps popping up in my mind and I feel it would be relevant to others.</comment>


Bose wants your kids to build their own Bluetooth speakers - <AUTHOR> https://techcrunch.com/2016/06/28/bosebuild/ <comment><USER2> * Its guidance software is only available for apple products.

* It's 150 USD for around one hour of play time.

Apart from that: I like it! Looks great and for sure would've interested me as I were in their target age. Also appears to be very rugged.

edit: product link: [https://build.bose.com/](https://build.bose.com/)

</comment><comment><AUTHOR> What I really like in their video is that it's a girl who is building the speaker. As a male software engineer, it's really refreshing to see companies do their best to bring more diversity into our field.</comment>


Why aren't you commenting your code? - <AUTHOR> http://sethetter.com/why-arent-you-commenting-your-code/ A refresher on some of the major benefits of commenting throughout your code. Meant to serve as a reminder for those still struggling to make it a habit. <comment><USER2> Comments - like documentation - are a liability. Most projects tend to treat them as supporting artifacts, but they are actually unidentified risks. Did you account for the time spent either rewriting documentation/doxygen/comments as you go (frequent chunks of small work) or an after the fact round of document backfilling (single chunk of large work) in your schedule?

No one even reads those documents. The client claims they are to assist future developers - internally or another vendor - but would you trust a Word document or a .cpp file when everything is on fire and you need to figure out a system's behavior? Same thing applies for code comments.

There is one source of truth in software, the code. You can pile on the design documents, wikis, and architecture diagrams but at the end of the day, what is coded is what gets executed. Make the code easy to understand and reason about. And then, if you really want to be nice to future developers, throw in a single page overview of the codebase (README) that gives someone an entry point.

Edit: I will just add the caveat to preempt the inevitable - this does not apply to public facing documentation of an API/framework/library etc.

<comment><USER3>
      > Comments - like documentation - are a liability.

Tests, too. But you wouldn't advocate not having tests, right?

<comment><USER2> It all comes down to: does the benefit out way the costs?

A well-maintained suite of automated tests provide an enormous benefit. A brittle set of Selenium scripts costs a project more than paying a Tester to manually run scenarios.

<comment><USER3> Yeah, I agree, it's a question of cost vs benefit.

A bunch of tests that check trivial cases is not helpful at all, a well- maintained suite of tests is very useful.

What I mean is that a bunch of documentation that only states the obvious is not helpful at all. A set of well-maintained (!) code comments can make navigating unfamiliar code much, much easier.

</comment></comment></comment></comment><comment><USER4> The thing about comments is that they don't get maintained. So the comments end up diverging from the implementation. The tests all pass even if the comments are wrong, so nobody cares unless you are particularly diligent about code reviews of EVERY commit.

Also this should go without saying but I still see comments like this:

      // loop over the collection
      for (var foo in bar) {
        ...
      }

which are not helpful and just clutter the code.

<comment><USER5> It's often said, "A comment is a lie waiting to happen".

<comment><USER6> And a line of code is bug waiting to happen, especially if no one knows why it is written in the weird way it is.

<comment><USER5> In theory your unit test would pick that up ;)

</comment></comment></comment></comment><comment><USER7> Yeah, but what about

        // $set $foo to $bar
        $foo == $bar;

        // print out value of $foo
        echo $foo;

        // is $bar the same as $foo? if so, return true, otherwise return false
        if ( $bar === $foo ) { // check if the same
            return true; // great, it's true!
        } else { // here we check if it isn't true
            return false; // nope, not true, let's return false
        }

I have nothing against comments where something needs to be documented, but not everything needs comments as most code is essentially self-documenting.

Edited for clarity.

<comment><USER8> It's completely uncontroversial that those comments are unnecessary. When you see them you remove them; no reason not to write any comments at all.

</comment></comment><comment><USER9> As mentioned in Clean Code, beautifully and concisely code doesn't need much of commenting. It's pretty much self-explained.

<comment><USER6> And over here in the messy real world with weird dependencies and legacy junk and performance-critical regions, we have comments.

</comment></comment><comment><USER10> 99% of the time when you are writing a single line comment you should instead be either renaming a variable or using the extract method refactoring using some abbreviated form of the comment in the new name.

For the remaining cases it's more important to make sure that you're documenting the "why" of something in the code. If you find yourself often in the situation where making changes to the code requires modifying the comments to keep them up to date that's a big sign that you aren't writing comments appropriately.

<comment><USER6> ComputeGeometricMeanPriceChange2YearWindow.

There is something to be said for the spacing that comments allow.

<comment><USER10> Start there, then continue refactoring and renaming. I've rarely run up against a situation where an unwieldly long name was actually irreducibly so.

</comment></comment></comment><comment><USER11>
        // It was hard coding it
        // so it should be hard reading it

</comment><comment><USER12> I find comments useful in two main cases:

1) When I'm not all that familiar with the language. This allows me to grok the purpose and actions of code quicker.

2) When the action being performed contains more lines than fit in the screen. Knowing a bit more context going in can make what follows more clear.

For 1, it's a matter of who you intend to read/modify the code after you. For 2, that's probably more a case for splitting that code into multiple functions.

From that, I think comments are best used by poor and great programmers, but the middle of the road may not get much benefit out of them. If you don't know how to split your complex bits of code into multiple chunks, or if you expect most people coming after you to now be as experienced as you, comment.

</comment><comment><USER13> I started writing code in a more literate style recently (where the code is a document with markdown interspersed with the code segments) and found that writing what I want to do first makes the development process much easier than just delving into code. Feels like writing a paper in many ways (you write the outline, flesh with your thoughts on how it should work, and then start writing code)

I put this tool together on a whim (although the github rendering is somewhat strange because I took liberties with the fenced code blocks): <https://github.com/niggler/voc>

<comment><USER14> I think the next level for you is to do the thinking before coding that your doing now, but just skip the commenting part...

<comment><USER13> I used to plan in my head / on whiteboard, but that doesn't scale well when you want to start handing code off to your employees.

</comment></comment></comment><comment><AUTHOR> I feel like I'm seeing a lot of discussion about what should be commented. Just like with tests, this can be hard to put your finger on. Even more so, the level of detail a comment needs to have for it to be useful/useless will vary for each developer reading/working on the code. In my opinion it's better to be overly helpful than it is to leave it all difficult to understand.

Also, it's obvious that outdated comments are terrible. It is my expectation that if comments are going to be in the code at all, they should be maintained just like the rest of the code.

</comment><comment><USER15> Commenting methods can often be redundant. But I find it is almost always a good idea to comment classes that contains any degree of complexity. Too often when digging into a new project I see a class like "ItemHelper", and have to read a few dozens of lines to realize something that could be summarized like: "Converts X-data to a format that Item can parse".

It makes it so much easier to get introduced to a code base, and comments like that seldomly needs to be updated.

</comment><comment><USER5> You shoudl read Robert C Martin's, "A clean coder". [http://www.amazon.co.uk/Clean-Coder-Conduct-Professional- Pro...](http://www.amazon.co.uk/Clean-Coder-Conduct-Professional- Programmers/dp/0137081073)

</comment><comment><USER16> Sometimes documentation is just a way to (try to) make up for poor code.

I usually pretend I can't document my code while developing. I'll sure add some anyway.

This doesn't apply to public APIs. There, you of course need to be more explicit.

</comment><comment><USER17> Anyone have an example of well commented source code?

<comment><USER3> [https://github.com/postgres/postgres/blob/master/src/backend...](https://github.com/postgres/postgres/blob/master/src/backend/optimizer/util/clauses.c#L3574)

In general, most of Postgres code.

<comment><USER18> I prefer a far more stilted comment style otherwise you have trouble remembering the code between the comments and tend to scroll a lot more. I mean /* The error cases here shouldn't happen, but check anyway */ is really not that helpful.

Also, all the long descriptions before function calls are mostly redundant. If i want to know what a function does I can go there and look, but it really should have a sufficiently descriptive name that after the first time I don't need to check again most of the time. Unless something non obvious is going on.

<comment><USER3> I think it is helpful to know if the check is just a sanity check or if there's an already known set of conditions can lead to that particular error.

The function comments I also find very useful. Read a few of them and see how much information they carry. Preconditions for the function, reasons why it does what it does, assumptions it makes... These functions are used from throughout the code and it's important to document them well.

I'm purposefully not quoting specific parts of the file, because of course if you look at each and every one of them, you'll find a few that could be improved. But the OP asked for a well commented code base and if PostgreSQL is not one, then I don't know what would be.

<comment><USER18> I could argue that comments are best for exceptional behavior so if it's normal to do a lot of sanity checks there is little reason to comment on each one. However, my point was saying something like:

      if (error) //sanity check

Saves space and get's the same point across without padding the line count.</comment></comment></comment></comment></comment>


Review new web app startup: book search engine and file storage - <AUTHOR>

Hello,
I am interested in you ideas, comments and impressions on my new startup: book search engine and PDF file storage and sharing platform at http://findpdf.net. With more than 250 million files available from Google search API and over 1300 books saved by users it is really useful in finding documentation, articles, tutorials and ebooks.
A set of tools like: read online PDF viewer, download file, convert to text (BETA) and store & share button via PDFbox.
Check it out and let me know if it useful for you, if you see some potential or what features you will need or recommend to add.
Best. <comment><USER2> Clickable: <http://findpdf.net></comment>


Ask HN: Best practices for log format? - <AUTHOR> We're currently going through a bikeshedding discussion about the log format that our microservices will generate.
Developers without Ops responsibilities want the logs to be as readable as possible to aid their development process (traditional timestamp, severity, component, action and a quick description in a single line and, depending on the type of event, maybe a JSON object in many lines with details about the request, possibly with blank lines and indentation to make everything very easy to read).
Ops people want the files to be easily parseable for injection into ElasticSearch. They want to avoid complex configuration and want more flexibility to generate reports. If logs were generated in a single line in JSON format, they would be happy.
It seems there is no way to please everybody. Are there any standard formats that microservices-oriented architectures are using these days? Is there a standard at all? How to approach this? <comment><USER4> Do not invent your own formats.

There is an RFC for structured logging [1]

Also, journald does structured logging, plus indexing and searching like a simple database and it's designed for your use case. It can receive the logs and forward them using a connector for ElasticSearch [2]

[1] [https://tools.ietf.org/html/rfc5424](https://tools.ietf.org/html/rfc5424) [2] [https://www.loggly.com/blog/why- journald/](https://www.loggly.com/blog/why-journald/)

<comment><USER5> RFC5424 Syslog defines some the common stuff all logs should have: hostname, app name, datetime, message, priority.

Many logging tools (Splunk, Loggly, etc.) consume syslog. Syslog works over UDP and TCP. Java Logback has a syslog appender. Linux systems have rsyslog for forwarding, writing, etc.

You'll wind up including your own custom structured data of course (IP address, response time, user id etc.), but it's nice use a recognized standard for the standard parts. (Kind of like, say, using status code, cache-control, and content-type in HTTP.)

<comment><USER4> The RFC defines key-value structured logging.

[https://tools.ietf.org/html/rfc5424#section-6.3.5](https://tools.ietf.org/html/rfc5424#section-6.3.5)

</comment></comment></comment><comment><USER6> If you are micro-services based architecture, then it is very important to track and log a request through the multi-service calls and relate the logs.

Syslog with a common UUID set at the entry point (typically at proxy or load balancer level e.g. nginx) of the request and then logger will log the UUID as the part of UUID parameter of Syslog.

Syslog then can be parsed and also form JSON objects for the poeple in Ops and send it to a service. Basically you could have one more internal micro-service to handle logs from Syslog and then parse it and send it wherever Ops would need.

<comment><USER7> Along those lines (and possibly what the OP was looking for as well), what are the conventions around key names including namespacing? Or what are the common environment variables you capture for a given service, i.e. the boilerplate stuff.

<comment><USER6> Syslog has very detail key name sand namespacing ([http://www.iana.org/assignments/syslog-parameters/syslog- par...](http://www.iana.org/assignments/syslog-parameters/syslog- parameters.xhtml)) which can be filled in with application related data and this can be then parsed by any PaaS logging platform / OSS logging tool

I think syslog is a widely accepted log format and most of the tools which revolve around log metrics do support syslog and more so with parsing tools

</comment></comment></comment><comment><USER8> General rule: writing logs is a means, not an end, so whoever reads it most gets to decide what it should look like.

Corollary 1: allow developers to use whatever they see fit for DEBUG logging.

Corollary 2: require single-line json for everything else (if your developers have to read lots of production logs, you probably have a bigger problem than your log format)

You will need something to have ops filter DEBUG lines out.

Options:

\- disallow having DEBUG logging in production

\- separate log file for DEBUG logging

\- start each line with the log level

Also: give the developers a tool that converts the 'one json line per event' file to what they want. Configure _less_ and other tooling to know about it.

Optionally, tweak the json serializer used in logging to always output the time stamp, severity, and short description at the start of the line.

</comment><comment><USER9> Have you looked into [https://brandur.org/logfmt](https://brandur.org/logfmt)?

Having lived with various points between the 'awesome to read' and 'simplest to parse' I currently favor simpler parsing and investing in tooling that makes manual parsing of logs easier.

Of course the decision should include the scale of the microservices. There would be no benefit in optimizing too much for readability if the number of requests to your services is so high that you would need complex filtering before you can look at them.

Another way to cut down the bike shedding is to go with a library that wraps around the format you choose. We built GitHub.com/clever/Kayvee and have changed the format a few times. However since everyone uses the kayvee libraries at Clever it's not too crazy to change the format.

Hopefully this was useful. Apologies for the disconnected paragraphs - <USER2> public transit :)

</comment><comment><USER10> The only thing I can add to everyone else's replies is: VERSION IT.

This will allow you to write a parser which parses all versions of your log format, whichever such versions are – based on the version… and allow you to amend the log format at any time without worry that suddenly your old logs aren't going to be compatible with new ones.

"we" use this even for Apache access log files, and it's one small thing which has helped a ton.

</comment><comment><USER11> Converting a structured format into a human-readable form is relatively easy. Parsing unstructured logs is hard.

</comment><comment><USER12> > traditional timestamp

As as sysadmin I implore you use ISO 8601 format, preferably with the UTC offset.

<comment><USER13> Do you prefer the full format (2017-04-27T09:43:04,0010+0100) or a related but more readable format (2017-04-27 09:43:04.0010 +01)?

<comment><USER14> My personal preference would be sans TZ, since everything I can control is set to UTC immediately.

<comment><USER15> Technically "Z" postfix is a TZ identifier for UTC time. It's short and makes the timestamp compatible with RFC 3399:

        2017-04-27T09:43:04Z

<comment><USER16> Not only technically, but also practically, since many tools, libraries and service APIs accept and produce the Z as the TZ identifier for UTC.

</comment></comment></comment></comment></comment><comment><USER17> As part of my current role, I setup and managed a unified logging infrastructure as part of a SIEM install. We're talking ingesting logs in the order of TBs a day (Lots). Here are my observations:

1\. Syslog type logs are great when the volume is low, but field extraction can be a pain (unless you're a Regex savant)

2\. Structured logs are nice to deal with. Fairly easy to read and a dream to parse. Make sure you have a good ETL process in place to process these logs so they are searchable. 3\. If you have a lot of logs, they are only as good at the usability of the search on that corpus of text.

I use Splunk to ingest the logs and once the extractions are in place, it is a dream to search. Other options like ELK are available and work pretty well too.

Show the devs the advantages of having structured logs indexed and ready to search. No more: SSH > cat | grep | tail | less | more

Real-time functionality and the ability to alert on certain conditions is an added bonus.

</comment><comment><USER18> If you're using a configurable logging library like logback for Java, it is fairly easy to have your application support both structured logging directly to Elasticsearch, and logging a more human readable format to disk or stderr for development.

It would be nice if there was a standard format you could just ship to a logging service, but the closest seems to be syslog, which is somewhat janky.

Re-parsing logs after they have been formatted for human consumption feels like an anti-pattern motivated by anemic logging libraries.

<comment><USER19> Exactly this. Configure two log handlers - one sending data to ES and another to disk.

Alternatively, give the ops team what they want (more information) and then generate log files that please the devs from ES query output

</comment></comment><comment><USER20> JSON-formatted log files can be pretty-printed by tools like the Logfile Navigator:

[http://lnav.org](http://lnav.org)

That should make the decision a lot easier.

</comment><comment><USER21> We generate structured logs, I.e. key-value pairs. Depending on the environment they are either formatted with logfmt or JSON. This serves our use cases quite well.

</comment><comment><USER22> This is a node module but the idea can be applied generally ([https://github.com/trentm/node-bunyan](https://github.com/trentm/node- bunyan)). Basically you output the logs in the format ops require(Which i assume would be structured) and for devs put a beautifier that parses the logs and prints in the format they require.

<comment><USER23> Also this: [https://github.com/pinojs/pino](https://github.com/pinojs/pino)

</comment></comment><comment><USER24> Not directly answering your question but

1) fluentd[0] can be used to parse diverse log formats

2) You can set an environment variable, e.g. `ENV`, and output to a more human readable format when `ENV=development`.

[0] [https://github.com/fluent/fluentd/](https://github.com/fluent/fluentd/)

<comment><USER25> Fluentd's regexp parsing will be PITA to maintain, both because regexps only give match/no match, which is difficult to debug, and because of how to test new regexp set with Fluentd.

Liblognorm library is a much better choice. Regular rsyslog installation comes with liblognorm already, though I prefer a standalone log parsing daemon (and thus I wrote logdevourer).

</comment></comment><comment><USER26>
        maybe a JSON object in many lines with details about the
        request, possibly with blank lines and indentation to make
        everything very easy to read

Pretty-printing the JSON is a presentation concern - it shouldn't be baked into what's being written to the logs.

</comment><comment><USER27> JSON feeded into ElasticSearch works pretty well for us. I was a bit sceptical about it a first moment since I was used to grep classic logs in /var/log/, but it's actually really good. Kibana allows you to have a bunch of dynamic criteria for the logs, and filtering obviously JSON is better than randomly grepping a thing in text. I cant think of any other solution than files in /var/log, and a thing from top of my head which annoys me in text logs - <USER3> you dont know when exactly something happened (eg. before midnight or after a midnight?), you have to grep two files (expecting you have one log file per day). Also to grep the files, you need to have an access to the servers, but the fewer people have the access there, the better.

</comment><comment><USER28> Make one team responsible for maintaining logs and they get to decide the "how". This is most likely Ops. Devs are now a customer with requirements and Ops should find a solution that will both meet their needs and the needs of devs. That will at least stop the bikeshedding.

</comment><comment><USER29> This academic paper has some interesting insights about what distributed systems (e.g. Hadoop) log in order to be able to reconstruct what happened after a problem occurs.

[http://www.eecg.toronto.edu/~yuan/papers/zhao_stitch.pdf](http://www.eecg.toronto.edu/~yuan/papers/zhao_stitch.pdf)

In particular,

"\- log a sufficient number of events — even at default logging verbosity — at critical points in the control path so as to enable a post mortem understanding of the control flow leading up to the failure.

\- identify the objects involved in the event to help differ- entiate between log statements of concurrent/parallel homogeneous control flows. Note that this would not be possible when solely using constant strings. For example, if two concurrent processes, when opening a file, both output "opening file", without additional identifiers (e.g., process identifier) then one would not be able to attribute this type of event to either process.

\- include a sufficient number of object identifiers in the same log statement to unambiguously identify the ob- jects involved. Note that many identifiers are naturally ambiguous and need to be put into context in order to uniquely identify an object. For example a thread iden- tifier (tid) needs to be interpreted in the context of a specific process, and a process identifier (pid) needs to be interpreted in the context of a specific host; hence the programmer will not typically output a tid alone, but always together with a pid and a hostname. If the identifiers are printed separately in multiple log state- ments (e.g., hostname and pid in one log statement and tid in a subsequent one) then a programmer can no longer reliably determine the context of each tid be- cause a multi-threaded system can interleave multiple instances of these log entries."

Some of this may be duh-obvious, but it results in a naming scheme where all important objects in the system have unique, printable ids.

Couple more observations: Hadoop logs tend to fit on one line (they can be long lines, though), except in the case of exceptions. Exceptions are almost always logged, even if they represent an ordinary situation that is recovered from in the error handling code. (Many software failures in these systems occur because a standard, recoverable exception is thrown in one place, improperly handled in the error handling code, and then this causes a much more severe exception further down the line.) All exception logs include a multi-line stack trace.

</comment><comment><USER30> Imo json gives you the most flexibility and adds as much metadata the developer (applocation side) and ops (in case you use an orchestratiom such as kubernetes, etc) need.

Eg in kubernetes it's pretty common for applocations to log everything in json, kubernetes will then wrap the app logs in another layer of json that includes information about the container name, id, host ip, etc.

Microservices and containers will most likely require you to have some centralized way to view your logs (kibana etc) so ops and devs can create their own views.

Even in the basic case of tailing log files you can use a jq alias with a custom format to convert the json logs to something colorful and readable.

Ps. On mobile so i apologise for possible mistakes

</comment><comment><USER31> We developed a common logging module that we include in our micro services to log to json. It also takes in environment variables so we can include the deployment related information to identify and query the logs once they get centralized to splunk.

</comment><comment><USER32> All applications should generate JSON logs.

The transport system (syslog usually) will take care of time stamping and some meta data.

application => (JSON text) => syslog daemon => (syslog message) => graylog/ELK/splunk

</comment><comment><USER33> Log format is the least of your problems (FWIW: ElasticSearch is happy to to take JSON.)

A more interesting question is "can you actually _learn_ from your logs?" Most logs are just isolated data points in time. I've built a logging system called Eliot for Python ([https://eliot.readthedocs.io](https://eliot.readthedocs.io)) that lets you generate logs with _causality_ : it's a tree of actions where you can see which action caused which other action, and whether each action failed. Actions can span processes, so it's also a tracing system.

This makes debugging complex systems _much_ easier because instead of "X happened. Y happened. Z happened." you can see that Z happened _because_ of X, but Y was unrelated.

(Output is JSON, easy to push into ElasticSearch.)

I'm sure there are other logging systems out there with similar features, or other approaches that work (combination of service-specific logging and protocol tracing, say). But the basic point is you should be thinking about what your goals are for the logs and what you want to learn from them... formats are easy to change and transform, but lost information is often lost forever.

</comment><comment><USER34> My preference is:

2017-04-27T02:46:20.762-07:00 [$level] $message

Where $level is one of [debug, info, warn, error] and $message is a string log message.

If your working with distributed systems may want to add $hostname between the date and level.

Full example:

2017-04-27T02:46:20.762-07:00 web4 [warn] unable to connect to mobile interface

</comment><comment><USER35> Being in a situation where logs are not read, or can't be read, is very similar to not having logs. So it's important to emphasize formatting and productivity when it comes to logs.

JSON can be useful. You can use jq ([https://stedolan.github.io/jq/manual/](https://stedolan.github.io/jq/manual/)) and common unix commands to parse logs (grep to filter, wc -l to count results, sort|uniq to get unique results, colrm to truncate lines, cut to get fields)... using those can be as powerful as using a query language.

Then, you can use a log aggregator to have access to all your logs. ElasticSearch/Kibana can be useful for that. Commercial services such as Splunk or SumoLogic can also be very helpful.

</comment><comment><USER36> Don't forget log parser if you're using Windows.

</comment><comment><USER37> why not generate the logs as json with a key called 'humanreadable' that contains an easy-to-ready logging statement that the devs will love?

</comment><comment><USER38> If folks want a fancy format they can always use AWK to reformat the beautifully readable developer logs. Anyway why use Elastic Search when you could use Splunk?

<comment><USER39> Probably because Splunk is ridiculously priced, even for enterprise software.

</comment></comment><comment><USER40> On one really large project with lots of microservices that generated requests to lots of other microservices, we ended up using JSON logs and some convention. We had libraries in all the languages we were using (php, node js, go, python, c / c++) that followed the convention.

The convention was simple. Every log MUST have at least these three parameters: --action, --r, and --t, and one optional one: --p.

\--action would be the "subject" of the message with a dot notation. A lot of times it's simply class.method or class.method.something.finished.

\--r is the "request" hash. This hash represented everything that has happened in the current process to handle whatever was asked of it. So for a web request, the --r would start as the request came in from nginx, and the --r would end when the response was sent back.

\--t is the "thread" hash. If the calling process sent a --t parameter, use that. Otherwise, generate a new one.

\--p is the "parent" hash. If the calling process sent a --r parameter, set that as --p. Otherwise, don't add it.

All of this was automated in the logging libraries. So generally a developer would simply call Log.d('whatever', {something: 'something something'}) or Log::d('whatever', ['something', 'something else']);

Requiring those four params allows us a couple fantastic things. First, highlighting the --action field made it a lot like looking at your inbox. Every message had a subject; This made for very simple skimming. Second, when hunting errors, it was very easy to find all the other related log messages by searching for the matching --r(equest) value. Third, for larger processes that sometimes involved tens of servers, it was easy to generate a tree of the entire request from start to finish, by simply using the --t(hread), --p(arent), and --r(equest) values.

These logs were all sent to rsyslog, which forwarded all logs to a central server (Also running rsyslog), which kept up beautifully. Besides handling the collection and transport for us, it was great to be able to watch the logs centrally - and then on each server when necessary - just by tailing /var/log/syslog. I had written a couple parsers for our logs on the central server, but eventually, as the project grew, our logs became far too much to handle with simple tailing apps, so we went with ELK (Elasticsearch, Logstash, Kibana).

Back then we used Logstash for the conversion, but on recent projects, I've just used rsyslog's @cee plugin, which skips the "L".

And, over time, we'd added a few more indices that worked well with kibana dashboards and tables. The two most useful were --i and --ms.

An --i parameter was simply an incrementing int starting at 0 for every new --r(equest hash). Lots of times our logs came in out of order, so having an explicit incrementing value in the log messages was really handy. An --ms parameter was for milliseconds. Any log message that had timed something included an --ms message. That was most excellent for dashboards.

Also, all hashes were simply truncated SHA1 hashes. For instance, on the API server, we explicitly took the server IP, request URI, headers, parameters and a timestamp (to the ms), hashed it, and truncated to 12 chars. Any collisions were generally spread out apart enough in time that they weren't an issue.

</comment><comment><USER41> What part of making everyone happy are you struggling with? Logging is simple, don't over-complicate it, just dump plaintext to a file.

If Ops really need those features they can have a process read the logs and ingest them into whatever system they want. Plaintext with the fields the devs want (time stamp, severity, etc) is easily parsable. Don't underestimate what you can do with tail and a few scripts.

<comment><USER42> Ops is maybe trying to help developers and increase security (no login on the boxes needed anymore to look at logs). Better workflow for (certain kind of) alerts, which may then get more complicated. Maybe kibana dashboarding. That in turn will help developers tracking logs for each requests (as someone mentioned above, with a common UUID passed around).

And in there lies their mistake: provide the functionality (single access point to logs, alerting on logs) and make the developers work to gain access to it like it was another service. You want ES access? Parse your logs, I'm not going to write filters for you (I can help, but you are going to do the bulk of the work). This guarantees logs in a parsable format in less than 3 months of developers being oncall, particularly if you have a container scheduler (we're talking about microservices right?) and use ASGs (and your policy for upgrades and box issues is "blow up the box and create another").

<comment><USER41> > Ops is maybe trying to help developers and increase security (no login on > the boxes needed anymore to look at logs).

Can this be handled fine by having tail run as a daemon and forwarding all log entries to somewhere that the developers do have access?

> Better workflow for (certain kind of) alerts, which may then get more > complicated.

This is likely to be highly specific as to what sort of alerts you want, but will tailing the log and running it through an awk script work? I assume you'll have to do something much like this with any alert management tool.

Tools like kabana like nice, but this is one of those areas where I think people might be going for the shiny solution (that looks great to management) instead of analyzing what they really need.

<comment><USER42> You can use whatever system you want (syslog can drop the logs in a single box, no need for horrible tail-based concoctions), as long as you mantain it and it's not Ops responsibility to fix your AWK scripts that look for events in logs from 3 different services for the same request and the same customer (and/or respond to the alerts those script generate at 2am after Bob forgot to update it to parse correctly a new log message - we have our own fires to fight already).

I'm not saying one should go straight to ELK. There are other ways, but at the end of the day you are going to implement a similar stack, guaranteed, and you are going to regret using freeform logging instead of a sensible structured format.

<comment><USER41> It sounds like Ops want all the control but to do none of the work? Maintaining scripts like this is there job.

<comment><USER42> I'm sorry: it's your application, your alerts, your responsibility. Not Ops.

<comment><USER41> If it's my responsibility then I need access to the systems, otherwise I can't perform excercise that responsibility.

How does alert creation and maintenance not fall under ops? They sound like the sort of ops people that devops teams were created to replace.</comment></comment></comment></comment></comment></comment></comment>