Advice From An Old Programmer - <AUTHOR> http://learnpythonthehardway.org/book/advice.html <comment><USER2> _People who can code in the world of technology companies are a dime a dozen and get no respect. People who can code in biology, medicine, government, sociology, physics, history, and mathematics are respected and can do amazing things to advance those disciplines._

This is a nice sentiment but as someone who has been a programmer for biology and medical research, it's not true. It's just like any other mediocre programming job, but your office is a folding table under a decommissioned fume hood. Physicists and mathematicians who program are so underpaid they dream about the glamour and glitz of working in a cube on Wall Street. There are bad jobs with technology firms, but by far the programming jobs with the most "respect", most fun and usually highest pay are with tech firms and well funded startups.

<comment><USER3> Hmm, you really got a problem with this don't you?

[http://www.reddit.com/r/programming/comments/i1j67/advice_fr...](http://www.reddit.com/r/programming/comments/i1j67/advice_from_an_old_programmer/c204vpv)

There you are on reddit, saying the same thing, and ignoring the fact that you're confounding your experience as a Biology postdoc with you use of code. Hopefully, as a scientist you know what confounding is. :-)

P.S. I'm the author.

EDIT: point at the exact comment.

<comment><USER4> It was really interesting for me to look at all of the different directions me and my fellow students took after completing our CS degrees. I always joke that I 'sold out' with my programming skills for going to work at a news/media company to build their websites, whereas a couple of the guys a I know are doing a much more honourable thing by working in Biology and Space Tech.

I think it's unquestionable that they are the ones who are deserving of more respect, but I still think that it's not just our jobs that define how well we've used our programming skills. We're always hacking away on our own little projects, and hopefully carving our own paths, creating useful software along the way (something you excel in, Zed). I think that having the drive to do this commands more respect than any specific career choice.

</comment></comment></comment><comment><USER5> Once upon a time, I played poker for a living. The most important thing I learned was that the least important factor in how much I earned was my skill at poker. The most important skill was table selection, my decisions over which games to get involved in and who to sit next to. A close second was managing my resources, making sure I was playing within my financial and emotional means.

If you want to be world champion or mix it up with the legends in Bobby's Room, you need to be a phenomenal player, but being great at poker won't stop you from being broke and miserable. Everyone in poker knows a 'That Guy', a bona fide poker genius who keeps fucking their life up because they care too much about the game. The archetypal example is Stu Ungar; He won three world championships and over $30 million, but died aged 45, penniless and alone in a crappy motel because he just couldn't get his life in order.

Like poker, software is intrinsically meaningful only as an intellectual puzzle. It can be fascinating and beautiful and it can send you mad. Fall into the trap of believing that poker is a worthwhile pursuit of itself and you end up in the trap - living out of your car or on the crappy end of the Vegas strip, some weeks a millionaire, some weeks a bankrupt. You see them come and go, the bright-eyed but slowly dimming college dropouts who could be living the life of Riley if only they knew something outside of poker, if only they stopped trying to beat the hardest games for the sake of it.

Your usefulness as a developer is only indirectly related to your ability to code. There are bona fide geniuses working in poverty and obscurity, there are utterly mediocre programmers doing amazingly useful and important work. Github is overflowing with brilliant, painstaking solutions to problems that just don't matter. Your most important skill as a developer is choosing what to work on. It doesn't matter how lousy a programmer you are, you can still transform an industry by simple merit of being the first person to seriously write software for it. Don't spend good years of your life writing the next big social network or yet another bloody blogging engine. Don't be That Guy.

<comment><USER6> I seldom write any reply on hacker news. But this time I feel I have something to say.

The fundamental question behind all these arguing is, what can be considered useful? In the other word, what we do, couldn't be considered a waste of life?

Philosophers have been arguing about this for 2000 thousand years. So I doubt there's going to an absolutely correct answer in near future. What I can offer, is just my personal choice.

Whether we like it or not, whatever we do, whatever we've build, good or evil, will vanish sooner or later. The human beings will extinct, the earth will become quite, the sun will extinguish. It's just matter of time. If nothing we do could make any sense in the long term, how can we consider something useful not?

Like any living things on this world, we, humans, are born to have many instincts, like to survive, to reproduce, to play. And when we don't obey these inner demands, we feel unhappy. So we eat, drink, sleep, f __ _, and make money. However, that's because what we do is useful, or important, or has any meaning behind them.

Some people might feel happier when they can attain something that are _useful* for others, which usually bring them more money, food, prestige, and many other things. But some might be satisfied enough by resolving some intelligent puzzles. As long as people don't end up in a crappy motel because they couldn't get their life in order, I guess these 2 attitudes towards life are both OK.

<comment><USER7> Although I like the grandparent's anecdote and up-voted him, I would have made the same argument as you did against his point. It's just confusing and therefore unsatisfactory to use usefulness as a tool to decide how to spend your life.

The reason why one shouldn't be That Guy is not because spending all your life playing Poker, or doing Maths, or playing Chess (as in the case of Bobby Fischer) just for the sake of it doesn't benefit others, but because when your level of obsession reaches a certain point, the point of diminishing returns, you just get confused -- you can't tell whether you are still getting pleasure doing it anymore. You've been off-balanced with other parts of life for too long that you've lost the ability to approach your passion purely from intellectual curiosity, as you used to.

So the bottom line is, solving intelligent puzzles is fun, as long as you don't abuse it.

</comment></comment></comment><comment><USER8> My guess is that I've been coding for longer than Zed, and I don't remember _ever_ thinking it was boring, and a lot of the stuff he says might be true for Zed, it sure as hell isn't true for the rest of us:

"Programming as a profession is only moderately interesting. It can be a good job, but if you want to make about the same money and be happier, you could actually just go run a fast food joint."

If this is actually true for you, and you can actually code, you really need to be making brighter career moves.

"People who can code in the world of technology companies are a dime a dozen and get no respect. People who can code in biology, medicine, government, sociology, physics, history, and mathematics are respected and can do amazing things to advance those disciplines."

Well, maybe. But this sounds like random riffing from someone who hasn't held a senior, stable position in an actual tech company nor worked as a programmer in any of the long list of disciplines. I've met plenty of people who were the duty programmer in a non-CS area who were treated like shit.

But what would I know? At around the time I was messing around with software pipelined SIMD string and pattern matchers, Zed was pursuing the real business of a Working Programmer, which is to say, writing rants and offering to rent a boxing ring to fight people who made physical threats to him (which, I admit, is kinda cool).

<comment><USER9> I definitely got a different, more inspiring read out of this post. I'd sum it up as "programming for the point of programming will eventually reach a local maximum of enjoyment; focus instead on the creative potential and possible reach of the art programming has enabled you to create." In other words, "make something people want", and from that you will really receive fulfillment.

The true, lasting satisfaction that you'll look back on 20 years from now isn't the number of programming languages you've learned, but what you've created and the impact those creations have had on those who have used them.

<comment><USER3> You'll find that this part of the book gets interpreted by different people in different ways. Some folks, usually those who've invested their life in _only_ programming, take it offensively and assume I'm insulting them. Others who are beginners or maybe having a hard time with programming find it inspiring. Still others just find it weird.

To each their own.

<comment><USER10> Your post reminds me to time when I did my first freelance project. I want to create software/applications that people can use, and can make their life easier.

Thank you Zed.

</comment></comment></comment></comment><comment><USER11> Though there is some wisdom in what Zed says here, I am getting quite tired of the 'actually, programming itself (especially as a job) is boring' meme, as if saying that marks you out as some sort of world-weary but insightful practitioner of the arts who has somehow risen above the mere trivial and seen it for the dull plod it really is.

Please stop telling me what I think + feel about my craft - _you_ feel that way, _you_ want to do it down, fine, but don't act as if it is some sort of immutable truth, especially when you are talking to people new to the craft which I think is quite unethical, frankly. Why assume your own experience/opinion is somehow necessarily correct and ought to be dictated as hard-won truths, when your entire schtick is 'dissecting others' logic'?

No matter how many times I read these sorts of things, nothing changes the fact that I find programming a joy, the whole thing of being able to render machines of such incredible power + complexity to do stuff at all a miracle and know it is something I want to spend as much time doing as I can for the rest of my life - no jaded naysayer, not even an 'internet famous' one can change my personal experience of the thing, and I am absolutely nowhere, a nobody in a CRUD job, and not even that good of a programmer, but my passion and love for it remains.

A funny thing about this is that no matter what pursuit you can think of, there are always those in it who say 'actually the reality of this is horrible', people who take that attitude exist everywhere in every field, no matter how wonderful and joyous it seems and often actually is. I think that's something to contemplate - if it were all true then nothing would be worth pursuing.

There is a lot of bullshit spoken about programming, lots of crap aspects to the reality of it, _lots_ of crap aspects of dealing with other people in trying to get things done, but people seriously - this thing is amazing, let's be grateful and celebrate it for once, please.

<comment><USER3> If you read what I actually wrote, instead of what you wanted to hear, you'd see that I said the _profession_ is boring. Why would I write a book teaching people programming if I thought it was boring? Coding is awesome fun and my favorite thing, but damn the job is really super boring.

<comment><USER11> With all due respect, I did read TFA -

"I have been programming for a very long time. So long that it is incredibly boring to me."

But you do of course later say:

"This doesn't mean I think programming is boring, or that you will think it's boring, only that I find it uninteresting at this point in my journey."

And, of course:-

"Programming as a profession is only moderately interesting. It can be a good job, but if you want to make about the same money and be happier, you could actually just go run a fast food joint."

To me that all seems a little contradictory, but ultimately seems to say to me 'actually, after all these years I've seen through all the naive bullshit and _realised_ that it is, in fact, rather boring'.

There are obviously crap and boring aspects to any activity, but how on earth can you say the act of doing something is interesting, but the profession itself is not? Do you mean to say that everything in the profession _other than_ the programming is boring? Well I'd disagree with that too, though less emphatically.

</comment></comment></comment><comment><USER12> You know, half the reason Zed Shaw is so well known is not just that he is a prolific coder, but that he is quite an excellent writer as well.

Well done Zed. I always appreciate what you write.

<comment><USER13> Well, other than that he's completely mistaken about the career advice he offers here. I'd be very curious to know if he's ever been the highly- respected programmer in a science lab, or personally knows anyone who is. My science friends tell me the job doesn't exist; any code that needs writing, a grad student will do it sufficiently well (on top of their lab work) for no more money.

<comment><USER14> I'm not brave enough to speak in extremes/absolutes like "completely wrong" on a topic like this.

My experience aligns with his advice. As a finance student, I was selected for a research project because of my database/SQL knowledge, which got me mention in the Journal of Finance when my prof. was published, and which later paid unexpected dividends. Later, knowing to code in a finance/analyst job ultimately positioned me for a startup that has left me much better off financially than if I had stayed in my job. I also have a friend in the pharma research field that did very well because he could write code.

(Note that it's not always the money that is the pay off. Sometimes it is an opportunity that leads to other opportunities that would have never been open to you had you not been able to code, as in my case. )

So, in my experience his advice has proven to be right on.

<comment><USER13> But was he a pharmacist that could code, or a programmer who showed up one day and said, hey, need any programs?

<comment><USER14> The former.

</comment></comment></comment></comment></comment><comment><USER15> >Finally, I will say that learning to create software changes you and makes you different.

This is so very true in my experience. The way I think has fundamentally changed since programming became the primary professional/pastime activity in my life about two years ago. I'm 32, and everything from my politics to my decisions about where to live has been effected. (Whether I'm using the proper choice between 'effect'/'affect' is still a challenge...)

Programming has taught me more about critical thinking in two years than I learned in the previous 30. I can't stand many of the things I used to because of it, but thoroughly love many of the things I used to find painfully boring.

<comment><USER16> Seymore Papert, in _Mindstorms_ :

"By deliberately learning to imitate mechanical thinking, the learner becomes able to articulate what mechanical thinking is and what it is not. The exercise can lead to greater confidence about the ability to choose a cognitive style that suits the problem. Analysis of "mechanical thinking" and how it is different from other kinds and practice with problem analysis can result in a new degree of intellectual sophistication. By providing a very concrete down-to-earth model of a particular style of thinking, work with the computer can make it easier to understand that there is such a thing as a "style of thinking". And giving children the opportunity to choose one style or another provides an opportunity to develop the skill necessary to choose between styles. Thus instead of inducing mechanical thinking, contact with computers could turn out to be the best conceivable antidote to it. And for me what is the most important in this is that through these experiences these children would be serving their apprenticeships as epistemologists, that is to say learning to think articulately about thinking."

[http://www.amazon.com/Mindstorms-Children-Computers- Powerful...](http://www.amazon.com/Mindstorms-Children-Computers-Powerful- Ideas/dp/0465046746)

<comment><USER17> More here: <http://dailypapert.com/>

</comment></comment></comment><comment><USER18> _Programming as an intellectual activity is the only art_

Programming isn't an art.

Programming is a craft.

The difference is that you can use a craft to make art, but crafting isn't always artistic.

Let's not get over our heads and consider ourselves artists because we can code...

<comment><USER3> Oh, I'm so glad you, the one arbiter of all that is art, is able to correct our misunderstanding of the definition of "art".

The truth is, you can't define art, so in the same way I can use paint to paint my house, or use paint to create a portrait, I can use programming for art or just as a day job.

Finally, everything that requires skill has craft, even art.

<comment><USER19> while op is possibly being pedantic, aren't you agreeing with him?

shin_lao is suggesting that painting itself is not an art, but a craft. what you do with the paint defines whether or not it is art.

likewise, your code can create something menial the same way it can create something artful.

being able to program makes you no more of an artist than being able to put paint on a brush. whether or not being able to put paint on a brush makes you an artist, of course, is subjective. :)

<comment><USER3> I have no real idea what he's talking about, since like most pedants, what he talks about isn't really relevant to anyone but him.

</comment></comment></comment></comment><comment><USER20> Some good points buried among juvenile posturing, I guess the "old" part must be irony. Would you take programming language advice from someone who has learned the language in "a day or a week"? I realize the point is the language doesn't really matter but statements like that and the general tenor of the post don't inspire me.

<comment><USER21> No, most languages are isomorphic up to semantics. the outliers today are the logic languages, the HM typed languages and the macro languages.

javascript, qbasic, java, c++, c, c#, ruby, python, perl, they are all very similar, some with their knobs exposed, some without, some with bad environments, some with better, some with closures, some without.

Once you grok the inner similarity of software, it's just not very interesting to learn new languages.

<comment><USER22> "At the time that I wrote this book I knew about 20 programming languages and could learn new ones in about a day to a week depending on how weird they were."

He's saying the entire range is a day to a week. The ones that are isomorphic would take a day or two, I agree.

But he's saying the ones that are _not_ isomorphic also take just up to a week, and according to other discussions with him, it's not because he's already an expert in those other types of languages.

</comment></comment></comment><comment><USER9> I have to admit, I came into this discussion in support of the chapter and the author (<http://news.ycombinator.com/item?id=2664502> for example), but after reading Zed's replies I'm turned off by his elitist replies, such as on multiple occasions telling people that they "can't read".

Clearly there's a lot of support (443 votes and counting), but resorting to this sort of insulting just doesn't lend anything to the conversation. And I think it's a worthwhile conversation to be had.

</comment><comment><USER23> > Programming as an intellectual activity is the _only_ art form that allows > you to create interactive art.

What about oral storytelling? Or just (re)telling jokes? Improv and related performing arts? Stage magicians? Mimes? Community murals?

<comment><USER24> While the same thoughts crossed my mind, I think he probably meant art in the age of mechanical reproduction.

Still, programming affords a combination of the widest reach and lowest barrier to entry: it rarely requires the artist, performer(s), or specific location(s) to be consumed as a medium.

<comment><USER23> I agree with what he meant, but I dislike his use of the word "only." I hate to be really anal, but that kind of thing always sticks out in writing to me.

"Hard distinctions make bad philosophy." as John McCarthy (of Lisp and AI fame) would say.

</comment></comment></comment><comment><USER22> If he thinks he can learn any programming language in a week, he's still ignorant about programming languages.

I challenge his knowledge of/about Haskell, Agda, etc.

Also, I am sure there are plenty of "old programmers" who are still excited about programming languages, and think they matter.

<comment><USER3> I've gone through <http://learnyouahaskell.com/> and it wasn't too hard. About the only difficulty is the pointless and contradictory jargon like "point free" or the hand-waving about Monads and how they violate their purely functional execution model. Never heard of Agda, but then I sort of don't care.

Obviously, there are old programmers who are into programming languages, but they usually sit around writing crappy programming languages that nobody can use because they never ask anyone if their programming languages are usable. You know, kind of like Haskell.

<comment><USER25> The mark of whether you "know" a programming language isn't whether you've gone through a tutorial; it's whether you can write useful programs in that language.

<comment><USER3> Uh yeah, there's this thing called experience which allows me to learn a language well enough to realize it's a total load of bullshit. Haskell was exactly like that.

<comment><USER26> Zed, why do you think Haskell is spoken of so highly if it's so crap? Or are you being hyperbolic? I'm not in a position to judge, but when Peter Thiel gives a 19-year-old $100k to do a Haskell project, that gets my attention.

Really nice effort on this book, I wish I could teleport it back in time to my past self.

</comment></comment></comment></comment></comment><comment><USER27> I got bored with programming so I started to break out into more mathy things like machine learning and computational linguistics. It's the best of both worlds.

<comment><USER3> I got bored with programming so I started studying guitar. Way more fun.

<comment><USER7> I would also recommend board games, such as Go, or Chess (actually, let's just pick Go; Chess is undeservedly popular, you know, like Java).

</comment></comment></comment><comment><USER28> Who is the old programmer that Shaw interviewed for this section? He should have credited him.

</comment><comment><USER29> "they can go to hell"

"people who tell you it is are just jealous that you have picked up a skill they never in their wildest dreams could acquire"

As much as I get worked up by his writing, I think Zed Shaw always goes a little over the line...

</comment><comment><USER30> The piece is definitely tainted by the author's personal experience but the closing message is uplifting and empowering,

"Finally, I will say that learning to create software changes you and makes you different. Not better or worse, just different. You may find that people treat you harshly because you can create software, maybe using words like "nerd". Maybe you will find that because you can dissect their logic that they hate arguing with you. You may even find that simply knowing how a computer works makes you annoying and weird to them.

To this I have one just piece of advice: they can go to hell. The world needs more weird people who know how things work and who love to figure it all out. When they treat you like this, just remember that this is your journey, not theirs. Being different is not a crime, and people who tell you it is are just jealous that you have picked up a skill they never in their wildest dreams could acquire.

You can code. They cannot. That is pretty damn cool."

</comment><comment><USER31> It seems likely to me that the dual specialization strategy Zed describes is beaten by the strategy of being an opportunist who has the ability to code as one of the tools in their toolkit.

[http://www.ribbonfarm.com/2007/12/05/the-fine-art-of- opportu...](http://www.ribbonfarm.com/2007/12/05/the-fine-art-of-opportunism/)

</comment><comment><USER32> For all his faults, Zed Shaw is really a great personality to have in the programming world. He loves to code, but doesn't get caught up in the minutia and cargo-culting.

</comment><comment><USER33> _Programming as a profession is only moderately interesting. It can be a good job, but if you want to make about the same money and be happier, you could actually just go run a fast food joint._

Zed appears to be saying that _he_ isn't happy as a professional programmer. Some people really enjoy it.

</comment><comment><USER34> I'm just going to start linking to Shoshin every time I see a weary, old coder blogger post: <http://en.wikipedia.org/wiki/Shoshin>

</comment><comment><USER35> While I'm yet a young programmer, the primary paid experiences/internships/jobs I've had with programming have been in the biomedical/bioinformatics field. My first day on the job? Did probably all the work they'd expected of me the entire summer (all using a simple "programming" trick with Excel) then proceeded to write them a few program to make their data-entry/processing jobs a lot easier and more precise. I was like a magician.

It was pretty cool.

I've had nowhere near as much experience as Zed or many of the people replying on here, but my experience in different labs and situations has been similar.

On another related note; I was talking with an international finance/investment profession when I expressed my dual interests in finance and neuroscience. She immediately made the point that I'd be infinitely more valuable in finance with a hard science background (particularly in financing the field my background was in).

Moral: Domain knowledge and niche/cross-discipline expertise is valuable.

<comment><USER36> Addendum to your moral: get a job in finance and don't waste your time making CRUD apps and being Super IT/DBA Guy at some uni's bio research lab. Much better pay, and if you're lucky you can train on in maths and the like to move up from day trading to being a full on quant. _Then_ you can have the midlife crisis after you've amassed some capital and connections to start your special startup in SoCal.

</comment></comment><comment><USER37> For all the flack that Zed gets, this is actually one of the sanest/most inspiring things I have seen him write.

Truer words have never been said.

<comment> ryan-allen You may also like this essay of his: <http://zedshaw.com/essays/master_and_expert.html>

</comment></comment><comment><USER38> I think this becomes apparent to anyone with experience with multiple languages and multiple platforms. The language is just the tool you use to accomplish the goal. Its good practice to learn new and different languages and paradigms just to see how they work and how you can take the good and add it to your arsenal.

</comment><comment><USER39> I just don't like when he calls himself an old programmer. He is only like 10 years older than me - oh, wait..

</comment><comment><USER40> As someone who's been trying to hire good software engineering professionals for some time now, I'm going to tentatively agree with, "people who can code in the world of technology companies are a dime a dozen"; however, I'm going to posit that, in the world of technology companies, good software engineers who are passionate, diligent, and bent on improving themselves and their field are rare. So, please do not despair because you've read the Old Programmer's advice and you're not fluent in biology, medicine, government, sociology, physics, history, or mathematics. I can say that even as a generalist, you can help advance the field of software engineering, and I represent one technology company that will value you for that.

</comment><comment><USER41> I found the book that contains this chapter quite impressive. It represents the only attempt I've seen to document the procedure many self-taught programmers followed to learn programming by exploring. At first, reading the instructions about carefully typing in the exercises, it took me some time to realize the point (and I don't think someone would get much value out of _only_ typing in the exercises); however, once I read some of the "Extra Credit" exercises it made perfect sense as an introduction to learning by exploration. If typing in the exercises and running them doesn't give you an irresistible urge to tweak them and see what happens, you probably won't find programming fun.

</comment><comment><USER42> >People who can code in the world of technology companies are a dime a dozen and get no respect.

That may be a bit exaggerated, but...

>People who can code in biology, medicine, government, sociology, physics, history, and mathematics are respected and can do amazing things to advance those disciplines.

This is quite true. I'm involved a bit with a few circles outside the whole web-tech sphere, and it's quite different. Someone who understands the web and technology can create things that others cannot. Being involved in a certain field gives you knowledge and social contacts in the field. Put those together, and the business ideas you can come up with can be ones that can really change things.

</comment><comment><USER43> > People who can code in the world of technology companies are a dime a dozen > and get no respect. People who can code in biology, medicine, government, > sociology, physics, history, and mathematics are respected and can do > amazing things to advance those disciplines.

So true. I'm a aeronautical engineer and notice things everyday that could be revamped through coding. The problem is the vast majority of aeronautical/mechanical engineers aren't comfortable with code, and if anything broke or any improvements were needed after I left, no one could do anything with it. So we're back to our "database" of 200+ Excel files and nested folders for CAD "version control."

</comment><comment><USER44> I think a craftsman is entitled to reflect upon his work and profession and not have that introspection be neat an clean for everyone else's consumption.

There are moments of doubtful introspection and unwelcome truths that are insightful and illuminating. Stop and listen to the perspective and perhaps just spend a moment with it... That is want it means to be human.

Spouting of some knee-jerk reaction shows you are not even beginning to listen and haven't begun to hear what this person is emoting. There is a lot more here that is heartfelt and true than simply what is written.

</comment><comment><USER45> This is a fantastic piece of writing. Was actually very inspired by it.

<comment><USER46> I totally agree, it's quite an inspiring article.

My dad suggests that I take over his business which I might but I'm still sticking to programming as a career. May be I won't be earning a lot of $$$ but atleast I'll be doing something that I totally enjoy.

<comment><USER47> You should learn to apply your knowledge of programming to his business.

I find that that when knowledge of programming and software systems is combined with business, one has the potential to create a highly profitable autonomous system which only requires the initial investment of time (and hard work, of course).

</comment></comment></comment><comment><USER48> Zed's a strong cup of coffee and guys like him generally play a really healthy role in the community they are a part of, driving discussions in directions they might not have taken and playing the part of Gadfly at Large -- every community needs a Zed to keep from becoming complacent.

"Rails is a Ghetto" notwithstanding, Zed's writing is always worth engaging even when you strongly disagree. In that respect he's always reminded me of Steve Albini.

What Albini would do with Fret Wars, however, is another question entirely.

<comment><USER49> Actually, I tend to think "Rails is a Ghetto" is one of his most beneficial works. It was angry and juvenile at times, but I think it did help make rails grow up. It also called out a lot of the fanboy-ism in the community and helped people who actually cared about engineering to rise.

</comment></comment><comment><USER50> _Programming as an intellectual activity is the only art form that allows you to create interactive art._

I would venture to say that music composition is also interactive. Music, to me, is art twice: first in the composition and second in the performance. Somewhat like programming, composers create something and send it out into the world so that others can use it, give it life, perform it.

</comment><comment><USER51> I agree that it's more important to have some additional domain knowledge than to be some superprogrammer. I'm not the best programmer but I have some (human) language learning knowledge and thus could set up my own business at <http://www.chinese-course.com/> Best thing you can do as a programmer is be your own boss.

</comment><comment><USER52> _People who can code in the world of technology companies are a dime a dozen and get no respect_

Wow, where have you been coding? I am not as proficient (and DEFINITELY not as prolific) as you are, but I've only had that feeling once working as a programmer in Singapore, and that was because the boss in question happened to be an asshole.

</comment><comment><USER26> Oh please. You can't make as much money running a fast food joint and you will not be happier, unless that's in your blood. (I know one of the owners of a super-high-grossing KFC and have run a small business myself.)

Programming is a cushy, stimulating, high-paying job - and don't you ever forget it.

<comment><USER26> And, your risk in running a fast food joint is much, much greater. Franchise fees of $100k+, managing employees, avoiding lawsuits because your parking lot has a bump in it, and being in the shop everyday versus...

learning a language is free + telecommute, flexible deadlines + server crash leads merely to whining + Idea Guys have to pay developers and not the other way around.

</comment></comment><comment><USER53> I'm surprised than in this whole wall of critique, almost nobody has pointed out about the last 3 paragraphs.

I don't think "They can go to hell" is a good enough advice. There are nerds out there who are struggling socially, may be even virgins and I think they deserve better advice.

</comment><comment><USER48> If you haven't watched Peepcode's Play by Play with Zed Shaw, you should. Really insightful stuff.

</comment><comment><USER54> Since the best programmer, the most intelligent person or the best working people can be the lowest paid person, I think that what you must learn is to sell your skills.

Compare Linux to MS, is not about what you do but about how you sell it.

</comment><comment><USER55> <http://en.wikipedia.org/wiki/Narcissism>

Coding and software engineering are not the same knowledge. Coding is just a small part of the big discipline. The difference between software engineer and a coder with even years of practice is the same as between a poet and a full- time typist.

Memorizing syntax (and may be even some standard idioms) of 20 different languages doesn't make you a software engineer, and of course, you can code something after a day to week of practice, but it will be a mere amateur coding, not a programming.

Of course, such kind of coders are dozen for a dime, because having a dictionary doesn't make you a translator or even speaker of the any human language. After memorizing some basic words and you can use some very basic vocabulary and copied sentences from a phrasebook, but you cannot read even a news paper, let alone a non-fiction or a scientific book. You cannot even watch a TV - they're speaking too fast.

The idea behind reading other people's code is not about finding familiar syntax or code blocks, or even use Ctrl-C, Ctrl-V it is about appreciating and understanding the ideas which leads to this exact code and not the other, why this and not that, which is impossible without engineering knowledge.

No amount of memorized syntax can compensate the lack of practice. You may learn the syntax of 50 programming languages, it is not a substitute for years of practice with at least one of them.

Memorizing standard idioms also does not make you a programmer. You should understand why there idioms emerge and remain here, which ideas are behind them, which technology, what are advantages and disadvantages. This knowledge makes you a software engineer, and this knowledge will give you respect of others.

Producing yet another python tutorial, without ever realizing that keeping it short, full of idiomatic examples, and ability to grasp the big picture in an couple of hours is a huge advantage of the classic tutorial <http://docs.python.org/py3k/tutorial/> doesn't make you a teacher of humanity.

And of course, language matters. just look at this <http://hyperpolyglot.org/scripting> and try to appreciate the differences in readability and self-explanatory between PHP and Python/Ruby. Try to grasp why carefully chosen syntax of Python/Ruby is different from chaotic approach of PHP. Unfortunately we can't see the bloatedness of Java syntax on the same page.

<comment><USER3> Wait, so you link to the wikipedia page on Narcissim, then spend 9 paragraphs pontificating on my experience and how you know all about me in a way that assumes everyone cares what you think?

Right, I'm the narcissist.

<comment><USER55> I don't know what this way of expressing my ideas assumes. If you have something to say about these ideas - you're welcome.

</comment></comment></comment><comment><USER56> I found the discussion about domain experts interesting, as if there is no domains within CS. Wouldn't domain expertise in networks, machine learning, DSP, data bases, operating systems etc. be of any value.

</comment><comment><USER57> the first time i read this i remember thinking to myself, "oh boy, i wish i'd read this 8 years ago. i would have still been a programmer, but i would have chosen a different degree..."

</comment><comment><USER58> Interesting response:

[http://dekorte.com/blog/blog.cgi?do=item&id=4871](http://dekorte.com/blog/blog.cgi?do=item&id=4871)

<comment><USER3> I find that the hallmark of my writing is that people seem to find their own issues in reflected in what I say. This is a great example of that.

</comment></comment><comment><USER59> > At the time that I wrote this book I knew about 20 programming languages and > could learn new ones in about a day to a week depending on how weird they > were.

Funny. I've been programming quite a bit too and feel comfortable in a handful of languages. Knowing what's happening in the compiler and in the hardware or vm (runtime) is a different story. Being able to understand what programs do no matter what language they're written in is also a different story. We all know 20 different languages.

</comment><comment><USER60> I like the point mentioned that we should avoid being allied to a particular language and reflect on solving the problem.

</comment><comment><USER61> Lovely end to a book. To say other forms of art are not interactive however is a little short sighted.

</comment><comment><USER62> Isn't this the same guy who complains about not being able to find work and how pissed off he is that Google only offers him "junior system administrator" jobs?[1]

Not sure I'd want to get my career advice from him.

[1] [http://techcrunch.com/2008/01/01/zed-shaw-puts-the-smack- dow...](http://techcrunch.com/2008/01/01/zed-shaw-puts-the-smack-down-on-the- rails-community/)

</comment><comment><USER31> "People who can code in the world of technology companies are a dime a dozen and get no respect."

O RLY?

[http://blogs.forbes.com/nicoleperlroth/2011/06/07/winners- an...](http://blogs.forbes.com/nicoleperlroth/2011/06/07/winners-and-losers- in-silicon-valleys-war-for-talent/)

<comment><USER3> O HAI! You found an article about programmers getting respect in Silicon Valley. You totally proved me wrong, since it's exactly like Silicon Valley in...New York.

<comment><USER31> Good to know, but if I'm a coder who wants respect, it seems like it'd be easier to move across the country than develop an entirely new set of skills.

</comment></comment></comment><comment><USER63> you're the man, Zed

<comment><USER3> Thanks, I appreciate that you liked it.

</comment></comment><comment><USER64> zed shaw taught me python, cured my hemorrhoids and showed me a how to be a real man.

<comment><USER3> Must have been hard for you to get those rimmjobs on your massive hemorrhoids.

</comment></comment><comment><USER65> hah, well shit way to spoil the end of the book for me!

</comment><comment><USER66> Zed Shaw is trolling again. His ego is so boring.

Good that I stopped reading the rant when I found out it was him.

</comment><comment><USER67> I'm clapping my hands right now! Good text :)

</comment><comment><USER68> > People who can code in the world of technology companies are a dime a dozen > and get no respect. People who can code in biology, medicine, government, > sociology, physics, history, and mathematics are respected and can do > amazing things to advance those disciplines.

Totally

<comment><USER69> I disagree. People who code in the world of technology companies get respect, and quite a lot of it. Getting paid for your work is a form of respect.

<comment><USER70> Getting paid is only a form of acknowledgment that you worked, but it in no way denotes respect.

It simply says " to the best of our knowledge s/he did <insert type of job here/> work for the duration of this pay period.

I agree that some people do get respect in this industry, but too many of us are simply cogs in a wheel.

<comment><USER69> What's so bad about being a cog in a wheel, if you're enjoying your life?

<comment><USER71> I think that's precisely one of the the points Zed is making. That is, that coding is a cool, rewarding pursuit. But that you shouldn't expect a coding profession to necessarily be the either cool or rewarding.

If you're happy in that profession, great. I don't think the chapter in this book says you shouldn't be, in fact I think it would suggest that you have the right disposition for such a profession and are therefore fortunate.

</comment></comment></comment></comment></comment><comment><USER72> _What I discovered after this journey of learning is that the languages did not matter._

This sounds a lot like my journey. I was overwhelmed with the details of ~20 programming languages and data formats. Eventually it actually got rather frustrating to work with all of them. Then I discovered Lisp.</comment>


How a New Jobless Era Will Transform America - <AUTHOR> http://www.theatlantic.com/magazine/print/2010/03/how-a-new-jobless-era-will-transform-america/7919/

<comment><USER2> I'm old and generally pessimistic. But I've seen enough recessions to know that when you've read the seventeenth article about the never-ending bottom (as I did during the reagan era, during the early 90s when Germany and Japan were going to destroy the US economy, and post dotcom collapse), I can't help but think it's time that the pundits will all be contradicted on the upside. Just like they were on the downside.

Nations that have demographics and immigration policies that promote population growth are going to succeed comparatively over the next 20 years. The US isn't at the top of the heap there, but its close.

<comment><USER3> _immigration policies that promote population growth_

It's more subtle than that, and to a certain extent it's zero-sum. Countries compete to attract skilled immigrants, and there is a finite supply of doctors, engineers etc willing to relocate per year. As we see in the UK there is a much larger supply of people who simply shop around for the best welfare then claim "asylum". The brutal truth is that it's in no country's best interest to admit people who are unable or simply unwilling to contribute to their economy.

<comment><USER4> I find this 'welfare scrounger' outrage coming from britain laughable. Britain has the least generous welfare system of any rich european country, yet you get the most compaints from there.

<comment><USER3> It's because we're the most accomodating. Seriously, try speaking to the Swedish or French government in anything other than the national language. Only the UK will automatically hire a translator. Generous welfare is no use if you can't claim it. That's the secret of most European countries.

Why do you think all those people _already_ in a safe country, France, are trying to get from Sangatte to Dover? It's not "asylum" once you're already safe!

</comment></comment></comment></comment><comment><USER5> The thing that has always amazed/disgusted me is how our education system trains people to be passive consumers, and to leave school "looking for a job", as if jobs somehow grew on trees and would eventually fall when ripe.

Until people learn from the cradle that they're not owed a job by anyone, but have to create value of some sort, either for someone else, or for themselves, we'll have this same sad situation.

Someone "stuck at home" is in the perfect situation to start creating value that other people are willing to pay for: mowing lawns, baking bread, making furniture/simple machines in their home workshop, using whatever raw materials and machinery their forebears have amassed and are freely available, but, most of all, using their brains and their willpower.

</comment><comment><USER6> I wonder if there is a way to profitably exploit this?

Presumably bad news in the hiring market for new graduates is good news for YC: when the alternative to YC is picking up the phone and having Google and Microsoft getting into a bidding war over you, startups look insane, when the alternative is being a "parasite single" (wonderfully evocative Japanese English coinage for "lives with Mom and Dad"), having your ramen paid for for a few months must look pretty attractive. Granted, the vast majority of people capable of starting startups are probably much more employable than the average college graduate or the average inner-city dropout.

Is there anything the rest of us can do? "10% discount on lifetime wages if you hire during a recession" seems like it would matter if I were hiring, but I'm not. Does anybody have any other clever ideas?

I've got one: I can't offer anybody full-time employment but if somebody could vastly decrease the friction of employment I'd love to have employment available as a service. Something like what the India/Philippines virtual assistant companies perform, except with a more consistent level of quality.

Working for it wouldn't be the most fulfilling existence in the world, but it would pay for rent, food, and loan payments while folks look for more permanent opportunities, and the virtualized nature of it would make it more conducive to that than e.g. trying to get a job in food services. Plus, unlike your local On The Border, I'm not going to ask "So Joe, are you going to be available in 3 months or will you be going back to school?" because I have no particular need for Joe to be personally available in 3 months -- I don't have to train Joe, so it doesn't matter to me if I'm working with a new Joe come fall.

Currently I do a wee bit of this via freelancing, but freelancing requires friction from me -- I need to find freelancers, supervise them, pay them, find replacements when they flake out, file tax documents for them, etc etc. That's a whole lot of grunt work for something which is supposed to save me from doing the grunt work. I want the API to grunt work.

<comment><USER7> that was my first reaction, but this article is extremely long. wishing there was a "tl;dr;" section so i could figure out the conclusion in a sentence or two. bookmarked it for now, will come back to it.

<comment><USER8> It's extremely long? Maybe I'm a bit older than most readers here (I'm 38), but has the average readers attention span shrunk that much?

Nuance cannot always come in bite-sized chunks. BTW, I read the article (on a plane - the print edition, no less) - it's worth the time.

<comment><USER9> What's this "print" you mention?

<comment><USER8> The Atlantic

It's a very good magazine.

</comment></comment></comment></comment></comment><comment><USER10> My view is that the article is on the optimistic side.

It accurately describes the best case. It admits that nobody knows how bad it will be. But it fails to talk about known structural problems that we still face.

We have coming credit crunches in commercial real estate when ARM loans reset, prime home mortgages when ARM loans reset, private equity when principal comes due on a ton of LBOs, local government as their tax base continues to decrease, unfunded obligations increase, and they continue struggling under a major load of debt, and local governments as well as their pension funds suddenly lose large amounts of money due to overexposure to the private equity debacle.

Of course we have bright points like bank profits, right? Wrong. The only reason that banks are recording a profit is that during the credit crisis they were allowed to stop marking debt to market. The major banks are all sitting on a ton of future losses that are not properly accounted for. But since their unrealistic accounting recorded a profit, they are paying themselves enough money that the bankers (if not their employers) will ride out the coming crises.

I could say more but I'll stop. I'm depressing myself.

<comment><USER11> I could not have said it better myself, it seems like no one is talking about the commercial real estate market just like no one dared say it on the eve of the housing crash. I have been watching commercial rent plummet in the last few months and cant help but feel that it is starting.

</comment></comment><comment><USER12> I'd love to see the ideas generated by a Y Combinator RFS for harnessing millions of unemployed/underemployed workers.

Amazon Mechanical Turk is one part of the picture, and it opens up the door to more. Anything that makes it cheaper to create work, hire employees/freelancers, or find jobs is valuable in this economy. Or ways for people to earn money besides the mainstream labor economy. Or something that can generate value from activities that people do with their idle time while they're unemployed - whether it's constructive like Wikipedia or entertaining like Farmville.

<comment><USER13> That's an excellent idea. One of the things that I really would like is a way to earn money without screwing over a transitional employer by jumping ship the soon as a real job comes along.

Ideally it would be open to anyone and everyone all the time commitment free. At 3 AM, I have no idea how this magical service would work. Ask me after some sleep and coffee ;-)

</comment></comment><comment><USER14> There must be a template article out there for journalists. When it's a slow news day you fire it up, fill in the blanks, and a few minutes later you've got an article about how 20 somethings live with their parents, have no self direction, and feel of sense of entitlement.

I think the same template has been in use for generations.

</comment><comment><USER15> If you are receiving unemployment, I can't think of a better time to start your own business. Great companies will be born out of this era that were bootstrapped all the way. I think frugality and a focus on the bottom line is a renewed theme. Even if you fail miserably you'll be light years ahead of everyone else.

<comment><USER16> You would think so, but in fact the unemployment "insurance" agencies of some (many?) states are overtly hostile to using your time to build a startup rather than unsuccessfully seeking a position as an employee.

In Connecticut, for example, the statutes say revenue from self-employment should offset unemployment benefits, but the state agency has chosen the highly creative interpretation of "revenue" to include _potential_ capital gains your efforts represent, and the notional amount of salary you should be paying yourself for the work you do for your startup, including, say, writing a business plan.

It does seem like they are shooting themselves in the foot, but the unemployment "insurance" fund in CT is already living on debt.

<comment><USER17> That does not make any sense.

Maybe there is a way to make the snake eat it's tail. After your payroll paycheck bounces call up the state employer regulators and see if they will make your paycheck whole.

Then post the whole thing to the web as a performance artwork

</comment></comment></comment><comment><USER18> Bit of a catch-22 with harnessing the underemployed. The longer the economy maintains large numbers of underemployed, the less consumption in non- essential goods and services, and the less people buying whatever it is you sell, leading to the more underemployed, and the less consumption ... But if consumption increases, right when you can actually use all that labor they start to dissapear to proper jobs.

</comment><comment><USER19> The idea of total jobless-ness makes me think of a short sci-fi story: <http://marshallbrain.com/manna1.htm>

</comment><comment><USER18> ojbyrne, you raise a great point. But still, and perhaps naively, I find myself bothered by this recession in particular. In previous recessions the backbone of America -- our immigration policy, culture of competition, education and infrastructure -- made it virtually certain that new indstries, wherever they form, would center in the US. In our present state our Universities are still the best in the world and our immigration policy, albeit under fire, is still flexible enough to attract the best and the brightest. Yet I know no less than three people (one with two successful exits and a third company in the worlds in Boudler, CO) leaving California because they want a better education for their children and the high schools are a mess. Our infrastructure is extremely old, and as to the culture of competition every year we are more protectionist in policy. It is not certain to me that we will dominate the next generation of technologies the way we did with automotive, computer, pharma and information in the past decades.

</comment><comment><USER20> That link crashes my Firefox for some reason. Anyone else having the same problem?

</comment><comment><USER4> It's the fault of the teacher's unions. 12 years of school gets you no employable skills. If they simply started teaching programming in grade 8 to everyone by grade 12 everyone would be employable (since there's an unlimited supply of programming jobs).

<comment><USER21> Programming jobs are not unlimited. There's just a big demand for programmers and lack of supply. If every wannabe musician/artist/celebrity wanted to be a coder we'd certainly feel the strain.

<comment><USER22> marshallp has a point. There is a limit to the demand for food; after a while, you are full. There is no hard limit to the demand for technology, because the value that technology adds is the ingenious solving of problems, and it seems that everyone always has problems.</comment></comment></comment>


HyperDex can now act as a stand-in for MongoDB - <AUTHOR> http://hackingdistributed.com/2015/01/12/more-mongo-than-mongo/ <comment><USER2> I appreciated the candor of this note in "What's The Catch?"

 _HyperDex does not have anywhere near the market penetration of MongoDB. This means that it 's easier to find a developer who has used MongoDB than one who's used HyperDex_</comment>


How The Apple Tablet Could Ruin Computing  - <AUTHOR> http://www.popsci.com/gear-amp-gadgets/article/2009-08/how-apple-tablet-could-ruin-computing

<comment><USER2> He says that Apple could lock down the tablet like it does the iPhone, only allowing Apple approved apps. And that would ruin computing. Well, I wouldnt buy an Apple tablet. Someone else will sell me a computer. Maybe it wont be as stylish, but it will run what I want. It aint so, that whatever Apple wants, the rest of us will gladly accept.

</comment><comment><USER3> Ugh, after reading that article I have that same feeling as when I accidentally stop the remote on fox news.

</comment><comment><USER4> This doesn't make any sense. The only way this speculation could ruin computing is if it somehow also meant that all other computers stopped working.

</comment><comment><USER5> In the same sense that Bing 'killed' Google, and HTML 5 is going to 'kill' Flash, and Firefox 'killed' IE, then yes - Apple is going to kill computing.</comment>


A Mathematical Model for the Determination of Total Area Under Curves - <AUTHOR> http://care.diabetesjournals.org/content/17/2/152.abstract

<comment><USER2> The article is from 1993. Is this different from the usual graphical integration methods? Was this a interesting idea in 1993??</comment>


Why don't people buy Bitcoin at one exchange and sell at another? - <AUTHOR> http://bitcoin.stackexchange.com/questions/12670/why-dont-people-buy-at-one-exchange-and-sell-at-another <comment><USER2> The lag is too high. Rarely can you buy coins, get them transferred in to your wallet, transfer them in to the new exchange and sell them before the price has changed.

One of the biggest problems with Bit Coin is the delay. 5 minutes to move money between wallets is not uncommon. If you want to pay me, so I can pay the waitress we need 10 minutes.

Doing a check out at the grocery store that takes 5 minutes is not going to go over well with the masses.

Figuring out how to make the markets more fluid will likely be the biggest challenge for BTC, after is solves its issues with having huge swings in price such that everything in the grocery store would have to be retagged every hour to make pricing work. :-)

<comment><USER3> All you have to do is keep some money in USD and BitCoin in reserve on all the exchanges. That way you can buy some BitCoin on one exchange and then sell the same amount on the other at the same time, and then transfer over at your convenience.

<comment><USER4> The imbalance is not random; MtGox always has a higher price so your money would be flowing in one direction and you'd end up with all your money in MtGox USD that's very difficult to access.

<comment><USER3> It goes in both directions. If you are running low on USD at BitStamp, you just deposit money there (assuming deposits take an acceptable amount of time) and withdraw the same amount from Mt. Gox (which might take a while). You still made money. Are you looking for a never-ending money cycle?

You are creating a cyclical process of arbitrage. If arbitrage traders have large enough reserves they should be able to pretty consistently close the gaps, which means your opportunities will be few and far between anyway, giving you plenty of time to prepare.

When buying and selling BitCoin becomes more fluid, that is.

<comment><USER4> There's no cycle if you can't withdraw USD from MtGox at all.

<comment><USER3> Well MtGox clearly has a problem with USD withdrawals, which is why this arbitrage opportunity exists in the first place and doesn't seem to be closing.

If they fix the USD withdrawal problem, which I suspect they will do as they start to lose business to the other, better exchanges, expect arbitrage to obliterate the price difference between MtGox and other exchanges.

</comment></comment></comment></comment></comment></comment><comment><USER5> It's not possible today because the infrastructure just isn't there. You'll experience trading lag which is just too risky and it offsets the profits. Also, getting money off an exchange takes forever.

Someone organized a fund on bitcointalk which tried to arbitrage prices on mt.gox and bt-china (This is cumbersome because you need physical presence in china to be part of bt-china). It sounded promising but it failed and everyone had to be refunded.

[https://bitcointalk.org/index.php?topic=330209.0](https://bitcointalk.org/index.php?topic=330209.0)

In fact, the reason the BTC price on Mt.Gox is so high is because you can't get your money off there in a reasonable amount of time. This means that to get your money off of Mt.Gox you do not send a sell order to convert to USD, you instead transfer the BTC to an external wallet and sell it elsewhere. This leads to more buy orders than sell orders on Mt.gox (leading to higher BTC/USD prices).

</comment><comment><USER6> The topic of arbitrage comes up quite frequently in Bitcoin discussions; it's a good question and the answer is fairly straight forward. There aren't many choices for exchanges with a reasonable amount of transaction volume. Most people flock to these exchanges because they're well known, trusted, and you can trade large volume. The problem with these exchanges is their popularity - everyone using them means longer waits for transactions to be processed. In order to perform arbitrage effectively you need to be able to trade quickly. Any semi sane person will realize the risk in trading with 5 minutes lag, nevermind two weeks. The gain isn't worth the risk.

</comment><comment><USER7> Let's just be crystal clear about this, people are doing it. The prices would be vastly different if they were not. This route could be direct or indirect, but it's happening none the less. At this point you won't get any money out of Mt. Gox unless you have a JP bank account, which requires physical presence there.

</comment><comment><USER8> One reason not mentioned here is that its probably more attractive right now (with the upward price movement) to just buy and hold Bitcoin rather than take the arbitrage and cash out on another exchange.

</comment><comment><USER9> so the currency with "no transaction fees" avoids the free-market implications of arbitrage that would prevent price-fixing..... through the serendipity of two-way transaction fees?

</comment><comment><USER10> More interestingly: Is anyone doing BTX<->Litecoin arbitrage?

</comment><comment><USER11> Well 404 people starred this github repo that does it... [https://github.com/maxme/bitcoin-arbitrage](https://github.com/maxme/bitcoin- arbitrage) I added campbx support to a pull I did and tried it out a while back. It was kinda cool. So yeah, you can do it. The problem is, unless you have limitless liquid $$$ at both exchanges you're probably going to hit bottom on one of them sooner or later. At that point you're talking time and fees before you can get more cash in.

</comment><comment><USER12> You can easily buy at BTC-E and sell at Coinbase for a profit. The spread between the two is routinely high enough to limit the price fluctuation risk to acceptable levels these days.

</comment><comment><USER13> I'm certain people do. It's called arbitrage.</comment>


If you use GNU grep on text files, use the -a (--text) option - <AUTHOR> https://utcc.utoronto.ca/~cks/space/blog/unix/GNUGrepForceText <comment><USER2> I just looked through the GNU grep history to see when it suddenly started being able to decide halfway through a file that it is binary after all; this is since 16 September 2014, so fairly recently. Before that, it just checked the first few kilobytes to decide, and didn't change its opinion afterwards. To me, this is a very nonintuitive change.

<comment><USER2> Why is it that in recent years, with this and the more recent ls quoting fiasco, maintainers of longstanding UNIX utilities suddenly got the urge to fix what isn't broken?

<comment><USER3> What's the 'ls quoting fiasco'?

Actually I recently found that coreutils and ls behave fairly well with funny filenames:

Here is an invalid utf-8 byte and then a valid utf-8 sequence

        $ x=$'\xce\xce\xbc'
        $ touch "$x"

You can list it:

        $ ls
        ?μ

And here 'ls' does better than other tools that display filenames. It shows the invalid byte and then keeps decoding with error recovery:

        $ ls --escape
        \316μ

However GNU stat (which I think is also in coreutils) does something similar, but weirdly messed up:

        $ stat *
        File: ''$'\316''μ'

(it looks like it's outputting a valid shell string, except with extra quotes)

\-----

Most command line tools are not aware of stuff like this. For example you can touch "x$ANSI_TERMINAL_CODES" and if you do "bash x??" or "python x??", then your terminal will change color because of the escape codes printed back to the terminal.

I just changed Oil to use a well-defined format I called QSN (quoted string notation):

[http://www.oilshell.org/blog/2020/04/release-0.8.pre4.html#t...](http://www.oilshell.org/blog/2020/04/release-0.8.pre4.html#the- highlights)

It adapts Rust's string literal syntax to express arbitrary byte strings precisely and losslessly. (JSON can't express arbitrary byte strings.)

The QSN encoder does UTF-8 _decoding_ with a specific error recovery mechanism. So it's basically like what ls and stat do, but it's more precise.

(If anyone is interested in QSN, please contact me. I think it's more generally useful in a lot of places. It's something we already do but it's precise like JSON.)

<comment><USER4> They broke it in 2016.

[https://www.gnu.org/software/coreutils/quotes.html](https://www.gnu.org/software/coreutils/quotes.html)

At least with Gnu, you can recompile your own, non-broken version, which is the only saving grace of these stupid, trendy changes.

<comment><USER3> The old behavior was the bad one; the new behavior is good. And that link explains why very well.

I guess I should have figured that oblique references to "ls quoting fiasco" is shorthand for "I don't understand what's wrong but I'm angry about it..."

(On the other hand I would say the grep -a issue is bad both before and after because either way it relies on autodetection. The fundamental issue there is that there is too much variance in encodings, which isn't easy to fix. Luckily UTF-8 is growing in popularity, and it doesn't have this issue because it doesn't require metadata for extremely common operations like "find ascii substring".)

<comment><USER5> "The old behavior was the bad one; the new behavior is good"

If you're a human, yes. If you're a script, it breaks you in half. If you're a script that has to run on various versions, then maybe it's time fix yourself and use find. You're a sophisticated script after all, not one of these who require a human with a debugger. Modern culture may not appreciate that little 'compat' thing, but it is essential if you want something to continue to work and not just stop and wait for someone's educated guesses. Good software doesn't point fingers at you, it just works. I remember how recently I wanted to check network interfaces on some machine and commanded 'ifconfig'. Now it's called 'ip a', and there is no ifconfig. I can guess the reason – ifconfig was bad and ip is good. There is also an eternal "FAT" label issue in unetbootin app, which resurrects every time Apple changes its fdisk output format (in every release, as it seems). The workaround is to run it with a cli option – a very thing that unetbootin was created for to skip. This is what makes systems so much fun. Without all these cool things, we would just sit there and cry over our uselessness.

ed: I read below that ls does that in interactive mode only, maybe it's not that bad then.

<comment><USER6> You can't just expect compatibility about random things - that's why we have formal contracts: standards, specifications, documentation. A human-readable output of any app, in particular, should never be assumed to be stable, or to have a specific format (even if observations imply it), unless its docs specifically say otherwise.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> I mostly run into this on searching my environment: "set | grep whatever", now needs an "-a", possibly because of escape codes added to the environment a decade ago.

Maybe the fix would be to only activate the "detect binary files" code if stdout isatty?

Because it is a nice feature when I do a big grep to find something among my home directory or the entire filesystem. It is certainly annoying to get binary garbage in my terminal. Or maybe the binary detection could get smarter, maybe making the determination on a match-by-match basis ("This line I'm about to output is a kilobyte and half of it is non-printable", say).

Though, ack-grep doesn't seem to avoid putting binary garbage on my terminal, so maybe reasonable to switch to something that isn't so clever? Most of my terminal greping is done with ack these days, so I'd probably be happy with gnu-grep disabling this cleverness.

<comment><USER8> Usually you want this feature no matter where the output is going. Adding "-a" sucks, but it's not obvious how else this could work (and still be backward- compatible).

IIRC the grep heuristic only considers a short prefix of the file. If the garbage comes later, you lose. Unfortunately, this makes things seem a bit unpredictable.

<comment><USER9> > IIRC the grep heuristic only considers a short prefix of the file. If the > garbage comes later, you lose. Unfortunately, this makes things seem a bit > unpredictable.

This was changed about five years ago to just keep looking. Which makes things a bit unpredictable in a different way.

</comment></comment></comment><comment><USER10> Does someone know if using grep on a binary file is somehow defined by POSIX?

At a glance, I couldn't find a reference on the grep page:

[https://pubs.opengroup.org/onlinepubs/9699919799/utilities/g...](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/grep.html#top)

<comment><USER11> It isn't. The page says:

> _The input files shall be text files._

"Text file" is defined in [https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1...](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_403) :

> _A file that contains characters organized into zero or more lines. The > lines do not contain NUL characters and none can exceed {LINE_MAX} bytes in > length, including the <newline> character._

<comment><USER10> Thank you :-)

Interesting, especially the part about the LINE_MAX. Even though it kinda makes sense, I would never have thought that having a very long line makes a file a non-text file when all characters are 'normal' characters.

<comment><USER12> Id kind of like that. Grepping in a directory also containing minified js (with ten thousand character long lines) is a pain

</comment></comment></comment></comment><comment><USER10> Actually, I never considered log files untrusted input, but as this example shows, it would be wise to do so.

<comment><USER13> FWIW, attacks like Javascript or SQL injection via logfiles are hardly unknown. Log files are plenty scary. ;-)

<comment><USER14> Someone did a POC of a CSRF against the admin interface for Cisco routers. They sent garbage packets that sufficiently confused the 'hex editor' display view in the admin web pages in such a way that it made a request to another page, changing permissions.

</comment></comment></comment><comment><USER15> I see this and suddenly it clicks! That is exactly why I couldn't import an SQL dump that I tried importing for days now, that is filtered with grep. Wow.

And I was wondering all the time why mysql reported this strange error "SQL error in Binary file" when the .sql file was clearly a text file...

</comment><comment><USER16> I started using ripgrep a few years ago and haven't looked back. It's way faster, automatically excludes .gitignored files, and just has a bunch of common sense functionality.

[https://blog.burntsushi.net/ripgrep/](https://blog.burntsushi.net/ripgrep/)

</comment><comment><USER17> This 'feature' is especially irritating when one uses grep on some text files with legacy (non UTF-8) encoding, but has locale with UTF-8 encoding. The grep decides that regular text file is binary just because there are byte sequences that are not valid UTF-8 sequences.

<comment><USER18> How... How can grep possibly work on a strange non-utf8 encoding if you don't say to it?

<comment><USER19> If you're grepping for ASCII strings, then the UTF-8 pattern will match the Latin-1 file.

<comment><USER18> how can you expect to find ASCII strings in the parent's "text files with legacy (non UTF-8) encoding"?

<comment><USER19> Because some of them, like Latin-1, which i mentioned in the comment to which you replied, are supersets of ASCII.

</comment></comment></comment></comment></comment><comment><USER20> Also precede the file list by `--`. A very confusing thing can happen if a file happens to begin with a dash... (You can intersperse options like `-e pattern` among file names, if for some reason you wanted to do that.)

<comment><USER21> But that's not grep specific and generally a good idea, especially in scripts that get the file names from their command line, some input file or god knows where.

</comment></comment><comment><USER22> Have you thought about environment variable GREP_OPTIONS? [https://www.gnu.org/software/grep/manual/grep.html#Environme...](https://www.gnu.org/software/grep/manual/grep.html#Environment- Variables) You can define it at the beginning of the script.

<comment><USER23> > As this causes problems when writing portable scripts, this feature will be > removed in a future release of grep, and grep warns if it is used. Please > use an alias or script instead.

</comment></comment><comment><USER24> Love ack for source code and text docs: [https://beyondgrep.com](https://beyondgrep.com)

<comment><USER25> I used ack for a while, switched to ag (I don't remember why? FOTM maybe?), and finally ended up with ripgrep. If you haven't tried ripgrep you definitely should. It has almost completely replaced gnu grep for me.

<comment><USER26> I did the same but stayed with ag. For me, like with its contemporary find replacement fd, it's the UX that provides the most benefit.

The speed benefit isn't really a huge factor; working how I'd expect, omitting ignored files, being able to specify file extensions to search, plus simple editor integration. Amazing.

<comment><USER25> I keep meaning to replace find, but never get around to it. I will check out fd. Ty.

</comment></comment></comment></comment><comment><USER27> Also works well on non-text files, similar to `strings`! I don't think it works as well, but can still be useful for quick checks.

<comment><USER28> I normally do:

strings file | grep search_pattern

<comment><USER29> But remember, always use "strings -a"! [http://lcamtuf.blogspot.com/2014/10/psa-dont-run-strings- on-...](http://lcamtuf.blogspot.com/2014/10/psa-dont-run-strings-on-untrusted- files.html)

<comment><USER30> It's the default since 2014:

[https://sourceware.org/git/gitweb.cgi?p=binutils- gdb.git;a=c...](https://sourceware.org/git/gitweb.cgi?p=binutils- gdb.git;a=commit;h=7fac9594c41ab180979bdf5927ff7f7e1d13a9e9)

edit: I would say that if you are doing forensics on an untrusted binary and you are not using a dedicated VM for it then you are not careful enough. objdump, nm are still attack vectors, not to mention debuggers and disassemblers.

</comment></comment></comment></comment><comment><USER31> Oh man, I've had this issue before and I just chose to nuke the logs and try again, thinking they were corrupted!

</comment><comment><USER32> I normally use -I to skip binary files

</comment><comment><USER33> Why is this even a thing?

This is a serious anti-feature as far as I can see.. can someone clarify otherwise for me?

<comment><USER18> Try `cat /usr/bin/*` in your terminal

</comment></comment><comment><USER34> > 'LC_ALL=C'

Wuff, reminds me of the completely incompatible difference between BSD sed and GNU sed

</comment><comment><USER35> shoutout to [RipGrep]([https://github.com/BurntSushi/ripgrep](https://github.com/BurntSushi/ripgrep)), which is generally faster, has more intelligent defaults (searches cwd by default, ignores files matching .gitignore), and can search through only certain text files (like your .java and .py files, say). Not affiliated, just found it worth the effort to learn some slightly different flags, though many are the same as normal grep.

<comment> ainar-g > ignores files matching .gitignore

Maybe it's just me, but that sounds like a bad default. I can definitely imagine people being confused by that.

<comment><USER36> They are. That's why it's always mentioned in the first few sentences of docs (man page, --help, README). With that said, this default is one of ripgrep's defining features and is something that users consistently report as one of their favorite things about ripgrep.

You can disable all smart filtering (gitignore, hidden, binary) with `rg -uuu foo`. That will search the same stuff that `grep -r foo ./` will.

</comment></comment></comment><comment><USER37> Drum banging time: another good use for the UTF8 BOM.</comment>


Man's hoard of nearly 5,000 guns shows ease of amassing arms - <AUTHOR> http://www.reuters.com/article/2015/12/03/us-usa-guns-stockpile-insight-idUSKBN0TM2LU20151203#xfyllGgqh89J7dls.97 <comment><USER2>
       hundreds have had their serial numbers removed

That, in itself, is criminal activity.</comment>


Ask HN: What would you change in Linux if building from the ground up? - <AUTHOR> POSIX, GNU and Linux are great systems that set up our computer to run the programs we love. But they are all based on legacy systems that go back to the original mainframes and first computers.
We have never done a complete reset from the ground up. But history is littered with incomplete attempts. This is because driver support and other niceties of modern systems would take enormous time to reinvent.
But if you could start over and you were in charge, how would things work differently? <comment><USER2> In 2017 the gap is in IoT. The Linux + 32 bit processor w/ MMU + WiFi stack adds up to a minimum cost of $30 for anything that connects to the internet.

Something super-lightweight that has a good network stack, doesn't require a conventional MMU, TLB and all that stuff that explodes the die area. (but ideally has some memory protection)</comment>


Show HN: Dweet.io – Twitter for machines, now with alerts - <AUTHOR> http://buglabs.tumblr.com/post/94633748906/dweet-tm-now-with-real-time-alerts-and-notifications

<comment><USER3> Full Disclosure: I work for a competitor of theirs as an embedded engineer.

Question for any bug labs people. How are you guys running the alert code? I assume it's something long the lines of a node sandbox that just runs the bit of JS, I've only looked into that stuff breifly. I was also looking at using really light weight interpreter called Duktape for a personal project. It has got a really similar C interface to Lua, but that ended up being way too much work and I abandoned it. I also recently found an erlang js interpreter that looked really interesting too, can't remember what it was called but it was from one of the erlang nosql databases, they used it for doing map reduce.

Also, why did you guys decide to go with chunking for the push over something like long polling or websockets? I assume the websockets was because of device code complexity, but longpolling seems more standard/less of a hack to me.

I always love seeing how others implement these kinds of things.

<comment><USER4> Hey azdle, thanks for the question and being honest. Do you mind me asking who you work for? Just curious more than anything.

To answer your questions about the alert code, yes it runs in a sandbox, but we also parse it and take it through a pretty lengthy pre-check process to weed out any code that might cause performance bottlenecks or general security problems.

As for websockets, we actually do support them (well, technically socket.io) but just don't make it very clear in the documentation. If you take a look at our open-source javascript client ([https://github.com/buglabs/dweetio- client/blob/master/dweet....](https://github.com/buglabs/dweetio- client/blob/master/dweet.io.js#L367)) you can see that we're actually using it there.

Hope that helps.

<comment><USER3> Sure, I work for Exosite[1].

I'm mostly curious about this because we're working on getting something similar into our API right now (the subscribe part, scripting has been there for awhile now) and I've been thinking a lot about this stuff recently, I've probably been pushing the most out of anyone here.

Did you guys have a reason for going for chunked over long polling though?

[1][http://exosite.com](http://exosite.com)

<comment><USER4> Not exactly sure what you mean by long-polling in this context. Do you mean a client making calls to the API at regular timed intervals to check the status? Or do you mean the server making calls to an endpoint URI when an event occurs?

<comment><USER3> Basically, it's:

1\. Client makes a request, somehow indicating that it wants to do long polling (url, header, etc.).

2\. Server receives the request, but does not reply, the socket is left open.

3\. An event occurs, then the server finally responds to the initial request with information about the event.

4\. Go to 1.

So it's basically polling, except the server just seems to take a really long time to respond.

In the end, both ways are completely valid methods, I was just wondering if there was any particular thing that swayed you either way.

</comment></comment></comment></comment></comment><comment><USER5> Nice! At first, it looked a lot like [https://data.sparkfun.com/](https://data.sparkfun.com/) which hosts a similar service. (Sparkfun made all the code available with an OSS license at [https://github.com/sparkfun/phant.](https://github.com/sparkfun/phant.))

But it seems that Dweet.io adds subscriptions and alerts to the mix - <USER2> pushing data out to subscribers or when certain conditions are met. That's a nice addition.

</comment><comment><USER6> Thank you for the follow up post!

Here is a direct link: [http://beta.dweet.io](http://beta.dweet.io)

<comment><USER7> I've been using Dweet for some time on a lot of different fun and hacky projects, I just wanted to say good job!

<comment><USER6> Thank you for the support!

</comment></comment></comment><comment><USER8> The link should probably be directly to the post

[http://buglabs.tumblr.com/post/94633748906/dweet-tm-now- with...](http://buglabs.tumblr.com/post/94633748906/dweet-tm-now-with-real- time-alerts-and-notifications)

<comment><AUTHOR> argh, sorry about that. can't edit it :(

<comment><USER8> No worries. Just dropped that in there in case dang sees this thread. Or at least it's here if someone finds this in the future.

<comment><USER9> Thanks! We changed it.</comment></comment></comment></comment>