Connections Episode 1: The Trigger Effect (1978) [video] - <AUTHOR> https://archive.org/details/james-burke-connections_s01e01 <comment><USER2> Utterly brilliant. If you haven't seen it before then do watch the whole series. It may be over 40 years old, but its still as relevant as ever.

The opening piece about the fragility of our modern technostructure was eye- opening when I first saw it and still packs a punch today. And especially right now.

Spooky co-incidence: the initial pieces to camera were made from the World Trade Centre. Then in the sequence on the power cut he talks about Scandinavian Flight Nine-Eleven being in danger of hitting the towers of down- town NY.

Also in Episode 4, look at the size of that military sat-nav! For bonus points, type the coordinates into Google Earth to see where he is sitting.

<comment><USER3> Is it really that old? I remember seeing it in the 90s and it didn't look that old. I loved it, I wish there was a remake or something similar more current. May be someone can add commentary on the old series updating it with current knowledge. I would watch that.

<comment><USER4> There was a Connections 2 series in 1994 which is the one I remember watching and loving as a teenager: [https://www.imdb.com/title/tt0370117/](https://www.imdb.com/title/tt0370117/)

<comment><USER2> Connections 2 wasn't a patch on the original. Connections 3 was just milking a dead cow.

However _The Day the Universe Changed_ is very good.

<comment><USER5> +1 for "The Day the Universe Changed", I watched it a few years back and I loved it (and my wife too).

</comment></comment></comment></comment></comment><comment><USER6> Undoubtedly James Burke is the reason I give a shit about history at all today (without which one cannot really have a meaningful grasp on the current world or its inhabitants, IMO). I was lucky enough to be exposed to these at a young age.

I have a directory on my file server with local copies of everything he ever released.

If you've never watched this or The Day The Universe Changed, please do. (I say this as someone who hates TV.)

<comment><USER7> What do you mean by _TV_ in this context?

Edit: oops, hit _reply_ by mistake.

I cannot stand having my television viewing experience interrupted by irrelevant advertising, and because of this I do not have anything in my house that connects to the antenna on the roof the previous occupant(s) of this dwelling installed.

 _Television shows_ themselves, or whether they're called? _Series_? Damn there are some good ones!

<comment> mark-r The first time I saw Connections it was broadcast by the local PBS station. No commercials.

</comment></comment></comment><comment> john-tells-all Another wonderful series is "The Secret Life of Machines". Cartoonist Tim Hunkin explains how machines work, with lots and lots of models and cut-aways of real life devices. And animations! And moving sculptures! His humor is pervasive and a delight.

[https://www.youtube.com/watch?v=baFaEvBywGc](https://www.youtube.com/watch?v=baFaEvBywGc)

<comment><USER8> These overviews give you just enough knowledge to start debugging a broken household machine. I love it. If anyone's looking for more videos from this series here's a playlist of the complete series: [https://www.youtube.com/watch?v=KDpNQQqdSh8&list=PLByTa5duIo...](https://www.youtube.com/watch?v=KDpNQQqdSh8&list=PLByTa5duIolYRtq45Cz_GmtzfWJyA4bik)

</comment></comment><comment><USER9> James Burke, and Richard Rhodes as well, are the reason I began to see history as something important and useful, a way of understanding both "how science happens" as well as "why our society is the way it is".

This knowledge, in the way it is presented, leaves the viewer or reader with a sense of agency, that what they do can either intentionally or not cause outcomes with great consequence.

It leaves one feeling one can do important things.

<comment><USER10> Thanks for mentioning Richard Rhodes. I highly recommend his book _The Making of the Atomic Bomb_ to anyone who is interested in a mix of science and history.

</comment></comment><comment><USER11> Yes, so much good stuff in this series. Sagan must have been inspired when he released his "Cosmos" a year or two later.

But this one has that central thesis that ties all the episodes up with a bow.

Not sure I cared as much for Burke's follow-on series but maybe I need to give them a second watch. They probably couldn't measure up to his original "Connections" masterpiece anyway though.

</comment><comment><USER12> Funny enough but, there was just a Reddit front page gif clip of Burke narrating his Connection with a perfectly timed NASA rocket launch. Nice to see him introduced to a new generation and some of the commentors miss the cold War referances. "The difference between a Moon Rocket and an ICBM is the top 15 feet"

[1] [https://v.redd.it/p8mc0yjzu4s41](https://v.redd.it/p8mc0yjzu4s41)

</comment><comment><USER13> Another link with all 3 seasons included: [https://archive.org/details/ConnectionsByJamesBurke](https://archive.org/details/ConnectionsByJamesBurke)

p.s. Like many of you here, this show _deeply_ influenced how I look at the world. This content should be shared far and wide.

</comment><comment><USER14> I remember, when I was a child, my father brought the Connections video game home. It was one of the first games I played on our home PC. The game was similar to Myst, but had a lot of nice explanations and history lessons in there.

</comment><comment><USER15> How timely. I found this series after watching The Ascent of Man, which I found after watching Cosmos. Highly recommend all three, especially the Ascent of Man.

James Burke has a way of conveying a sort of fundamental optimism about society that I find invigorating. I remember listening to an interview with him from the past couple years and after all this time, he still sounds as sharp as ever.

</comment><comment><USER16> As a side question:

I remember seeing a TV science series in the 80's that had Richard Feynman explaining how a 30k calorie pyramid of jelly donuts on a barbecue grill was the equivalent energy a Tour de France cyclist expends in a day.

What show was that? I can't find anything.

<comment><USER17> Maybe PBS "The Ring of Truth" - not Feynman though, Philip Morrison.

[https://youtu.be/Nk8CQNThbc0](https://youtu.be/Nk8CQNThbc0)

<comment><USER16> Thank you! That's it at 32:20. He and Feynman look similar, and it was 30+ years ago.

</comment></comment></comment><comment><USER18> I'm typing this comment atop a 1978 fourth printing of CONNECTIONS I just pulled off my shelf, after seeing this post. A prized possession, found used in rough shape. It was printed in the US, by Little Brown.

<comment><USER2> I've got a first printing from new. Nyaah!

No dust cover though.

</comment></comment><comment><USER19> Brilliant series. I remember watching it when it was released. Recommended.

</comment><comment><USER20> His Scientific American column was pretty good too.

He did a good TV special on climate change around 1989-90 that's on YouTube somewhere

</comment><comment><USER21> Here's Burke's version of wikipedia, for those interested: [https://webbrain.com/brainpage/brain/C6015FA0-82BF-F1FA-9D05...](https://webbrain.com/brainpage/brain/C6015FA0-82BF-F1FA-9D05-0EA9FD7F845E#-2751)

To me, Tom Scott is the person best taking up Burke's torch.

</comment><comment><USER22> At least as of a couple years ago this was available on netflix too. We used it in homeschooling, wonderful series.

</comment><comment><USER23> Other shows from that era worth a watch:

The Ascent of Man

Civilisation

There's also Connections series 2, although it is not as good.

Plus any of BBC Adam Curtis documentaries...

</comment><comment><USER24> One of my favourite shows. Found their copy of the best documentary ever created "Dangerous Knowledge", but it's incomplete:

[https://archive.org/details/DangerousKnowledge_201903](https://archive.org/details/DangerousKnowledge_201903)

</comment><comment><USER25> Burke's After The Warming, 1989

[https://www.youtube.com/watch?v=RfE8wBReIxw](https://www.youtube.com/watch?v=RfE8wBReIxw)

</comment><comment><USER26> Connections was such a great show. Thx for this. Wish I could find the old black and white Dr Who episodes on YouTube as well.

</comment><comment><USER27> We had a teacher in the 8th grade that basically forced us to watch the whole thing. I liked it then. Love it now.

</comment><comment><USER28> How hard would it be to restore the video quality? Could it be done with consumer software?

</comment><comment><USER29> Thanks, just watched two of them... gotta be careful or I will lose my whole Saturday!

</comment><comment><USER30> Fascinating and timely. I've always loved the series.

</comment><comment><USER31> Yay I love this series!</comment>


Safari Is Experimenting with an API That Could Limit Cookies to Logged-In Users - <AUTHOR> https://adexchanger.com/privacy/safari-is-experimenting-with-an-api-that-could-limit-cookie-storage-to-logged-in-users/ <comment><USER2> Have fun with the browser trying to figure out who is logged in...</comment>


The first piece of AI-generated art to come to auction - <AUTHOR> https://www.christies.com/features/A-collaboration-between-two-artists-one-human-one-a-machine-9332-1.aspx <comment><USER2> First at Christies public auction? Sure. can Buy that.

First sold for more than the tokenistic price? I can't prove it, but I am very unsure this is strictly true. its the first in the 21st century modern experience, but I would suspect there are works by earlier 1970s onwards AI which have traded for real cash. This is the most visible? sure. Doesn't mean works haven't traded before now. I'm sure somebody like Paul Allan who bankrolled much of the babbage build for the SF computer history museum would have early works.

People were playing with this stuff from the sixties onwards. People have been exploring machine mediated art for years.</comment>


Open Letter to Spotify Re. Accessibility - <AUTHOR> https://gist.github.com/duncan-bayne/9daa251ab5a507b3099d

<comment><AUTHOR> Spotify got back to me with a boilerplate reply, & I replied in turn to ask that the issue be taken seriously.

[https://gist.github.com/duncan- bayne/9daa251ab5a507b3099d#co...](https://gist.github.com/duncan- bayne/9daa251ab5a507b3099d#comment-1437892)</comment>


This is the Moto X. Can it save Motorola? - <AUTHOR> http://www.theverge.com/2013/8/1/4578890/this-is-the-moto-x <comment><USER2> No.

[http://en.wikipedia.org/wiki/Betteridge's_law_of_headlines](http://en.wikipedia.org/wiki/Betteridge's_law_of_headlines)

</comment><comment><USER3> If Moto's management can convince Google that having tens of thousands of employees making a nice but not special handset is a worthwhile use of Google resources, then yes.

But, in isolation, hells no. The moment this product was revealed, the product management process at dozens of 2nd and 3rd tier OEMs swings into action: "Make one like this, but faster and cheaper."

$200 is a nice price, but it isn't a stunning price. Moto isn't going to drive anything out of the market except, maybe, the Lumia 900, also $200, made by that other legacy high-cost OEM scrambling to stay in the top 10. Perhaps the lesson of Moto X is that it is a phone Nokia could have been making.</comment>


Show HN: AirText – Android Messaging for MacOS - <AUTHOR> https://www.airtextapp.com/ <comment><USER2> Now if only there were a third party app for 'iOS messaging for PC.'</comment>


VR social productivity app Bigscreen raises $3M with Andreessen Horowitz - <AUTHOR> https://techcrunch.com/2017/02/24/vr-social-productivity-app-bigscreen-raises-3m-in-round-led-by-andreessen-horowitz/ <comment><USER2> After wasting money on 2 other VR desktop apps, I was shocked that Bigscreen was free and 10x better than the other ones.

Hope they can put the investment to good use. I'm definitely not opposed to paying them when the final version is released.

<comment><USER3> Bigscreen founder here. Thanks for the kind words! If you have any bugs or feature requests, please let me know! firstname.lastname@example.com

</comment></comment><comment><USER4> I have never used Bigscreen (just had access to a Rift for a couple weeks) but I'm not surprised by this investment. When I played around with it, despite the several gaming options available, the most compelling experience I had in VR was just a pretty mundane casino game where I played poker. That above all else sold me on the potential of VR. As long as people think about VR just as immersive gaming, the potential is going to be limited. But what the casino VR experience taught me was that even something completely mundane like playing a game of cards could be far more immersive and more entertaining in a VR setting; if I had 5 friends over at home with me, I would've much rather plugged in with them into this beautiful virtual casino then dealt with shuffling cards at home in real life. Can't wait to be able to get the hands on my own VR device to see everything that Bigscreen has to offer!

<comment><USER5> To be honest that kind of scares me. What will happen when VR is commonplace and we are continually using it to escape reality.

Not to say we don't do that already, but the gradual degradation of reality is truly quite interesting.

<comment><USER6> Hopefully the goggles will become comfortable and we ll be able to take them off easily. We ll get used to the escape. It will become as commonplace and unremarkable as headphones

</comment></comment></comment><comment><USER7> I can't wait for VR resolution to increase so things like this work really well.

From my experience using the Oculus dev kit 2 there was too much of a "screen door" effect and it was hard to read text on a virtual monitor vs reading text on a real life monitor. It wasn't practical to use a virtual monitor to say, write code or surf the web.

But resolution will only get better! At that point will things get fun and I can see people eschewing monitors for VR "monitors".

<comment><USER8> As someone who has used both devkits of the Oculus and now owns a consumer- version Vive, I have to say the jump in resolution after the Oculus devkit 2 was pretty major. The resolution isn't great for reading anything other than big text and you can still make out the pixels when you look for them, but I'd no longer describe the resolution as having the "screen door effect" that was so major in the Oculus devkits.

I've enjoyed using Bigscreen for watching videos and movies, checking a couple headlines, and for vr-chatting with friends. I hope the headset's resolution is better in the future so that I can do more activities while in it, but as it is is pretty cool.

<comment><USER9> You can absolutely read small text if you supersample at 1.5-2.0x.

</comment></comment></comment><comment><USER10> This reminded me of a project by Sun Microsystems called Project Wonderland. They created a virtual world in which you could attend meetings and look at slides together with other avatars – although this was not in VR.

Here is a demo video: [https://youtu.be/-CFOGDBFKrk](https://youtu.be/-CFOGDBFKrk)

</comment><comment><USER6> They re doing what secondlife should have done long ago. I wonder if they have a platform for games.

</comment><comment><USER11> I've used bigscreen before, and while it might be hard to read small text, it is still a really nice app. I don't use the "multiplayer" feature, but what is really nice is to switch up my work environment at home from time to time. It also eliminates a lot of distractions.

<comment><USER3> I'd love to learn more about how you're using Bigscreen (and how we can make it better for you). Could you email me? firstname.lastname@example.com

</comment></comment><comment><USER12> In a virtual environment wouldn't you want a program like Blender to have a 3D interface?

And are couches and living room environments a skeumorphobic ornament to help the transition to VR?

<comment><USER13> A lot of us are working on 3D interfaces for design tools, but the difficult part seems to be making it bearable for longer sessions (hours). Some examples:

Medium [https://www.oculus.com/medium/](https://www.oculus.com/medium/)

Tilt Brush [https://www.tiltbrush.com/](https://www.tiltbrush.com/)

Quill [https://www.oculus.com/experiences/rift/1118609381580656/](https://www.oculus.com/experiences/rift/1118609381580656/)

MARUI [https://www.marui-plugin.com/](https://www.marui-plugin.com/)

Simplex [http://www.gradientspace.com/simplexvr/](http://www.gradientspace.com/simplexvr/)

Gravity Sketch [https://www.gravitysketch.com/](https://www.gravitysketch.com/)

Unity EditorVR [https://blogs.unity3d.com/2016/12/15/editorvr- experimental-b...](https://blogs.unity3d.com/2016/12/15/editorvr-experimental- build-available-today/)

Unreal Engine VR Editor [https://docs.unrealengine.com/latest/INT/Engine/Editor/VR/](https://docs.unrealengine.com/latest/INT/Engine/Editor/VR/)

Modbox [http://store.steampowered.com/app/414120/](http://store.steampowered.com/app/414120/)

Mindshow [http://mindshow.com/](http://mindshow.com/)

and my own tool Sketchbox [http://sketchboxvr.com](http://sketchboxvr.com)

So far it seems like the traditional CAD players (Autodesk, Dassault, Adobe) are focused on using VR for visualization, rather than creation.

<comment><USER14> Man this will be the matrix or definitely Caprica.

</comment></comment></comment><comment><USER15> Does Bigscreen let you view modern web content in VR? If so, what are they using under the hood to do that?

<comment><USER3> Bigscreen lets you view _any_ content in VR that you can currently view on your computer, not just web content.

You can see your entire desktop on floating virtual screens, which enables you to use your any desktop app (including but not limited to your web browser).

</comment></comment><comment><USER16> Interesting to see a VR company working fully remote

</comment><comment><USER17> I haven't been able to type in the other screen viewer app I got on the rift. Does BigScreen offer that?

</comment><comment><USER18> I hope they're working on the street protocol for the metaverse!</comment>


The OCB2 authenticated encryption scheme (ISO standard) has been broken - <AUTHOR> https://eprint.iacr.org/2018/1040 <comment><USER2> The first thing you want to know about this is that OCB3, which supersedes OCB2 and is going on 10 years old now, is not affected by this attack. The IETF RFC for OCB is OCB3.

Also, Internet cryptosystems generally don't use OCB at all, despite its elegance and performance, because of IPR issues.

The second thing you want to know (and you want to know it waaaaaay less than the first thing) is that this breaks authentication, not confidentiality; you can't use the attack to directly "decrypt" OCB2 messages, just to forge messages derived from a leaked plaintext/ciphertext pair.

This is a big deal for crypto people, though; see 'pbsd comment for more.

<comment><USER3> Link to pbsd's comment: [https://news.ycombinator.com/item?id=18350968](https://news.ycombinator.com/item?id=18350968)

</comment></comment><comment><AUTHOR>
        We present practical attacks against OCB2
        an ISO-standard authenticated encryption (AE) scheme.

        OCB2 is a highly-efficient blockcipher mode of
        operation. It has been extensively studied and widely
        believed to be secure thanks to the provable security
        proofs.

        Our attacks allows the adversary to create forgeries
        with (almost-known) single encryption query.

What I find most fascinating is that OCB2 is a scheme for which there has been a security proof since 2003. I am neither a cryptographer nor up-to-date with the state of the art in cryptanalysis, but at first glance, it seems like most attacks discovered these days rely on some kind of side channel or unspecified aspects of the protocol that the implementations get wrong. Even djb (cryp.to) is spooked: [https://twitter.com/hashbreaker/status/1057791485016526848](https://twitter.com/hashbreaker/status/1057791485016526848)

<comment><USER4> For anybody else wondering about the reply to that tweet:

mosh is using AES-OCB, but seems to be using OCB3 from what I can tell:

[https://github.com/mobile- shell/mosh/blob/944fd6c796338235c4...](https://github.com/mobile- shell/mosh/blob/944fd6c796338235c4f3d8daf4959ff658f12760/src/crypto/ocb.cc)

<comment><USER5> Yes, Mosh definitely uses OCB3 and is apparently unaffected (just added a FAQ to the website). Still pretty scary though -- this could easily have been Mosh's first real security vulnerability and it would have sucked. We first implemented Mosh's crypto in 2011, shortly after OCB3 was released. If I had started working on Mosh six months earlier, I probably would have picked OCB2.

<comment><USER6> What reasons lead you to select any OCB variant in 2011? Not trying to be snarky -- I am genuinely curious, given the patent landscape at that time and lack of any open source grant.

<comment><USER5> Reasonable question!

(1) OCB has long had a patent grant for GPLed software ([https://github.com/mobile-shell/mosh/blob/master/ocb- license...](https://github.com/mobile-shell/mosh/blob/master/ocb- license.html)), so the Rogaway patents were a non-issue even in 2011. (Of course there are other patents on authenticated encryption, e.g. IBM's Jutla patents. I understand why OCB gets singled out because of the Rogaway patents, but my understanding is that all AE modes, and perhaps most computer programs in general, likely implicate some patent.)

(2) The crypto experts told us that OCB was the best AEAD mode and that if we are going to put all our eggs in one basket, we should pick OCB. I am pretty happy with the choice and with the lack of traditional cipher negotiation in Mosh.

(3) Implementation quality strongly favored OCB. At the time, OpenSSL had no AE modes at all, so we probably would have had to vendor and ship an implementation of something not-OCB, or use a non-AE construction. I understand OpenSSL had a lot of pain implementing GCM in a robust way over the subsequent years -- if we had depended on OpenSSL or tried to use its AE modes, we would have inherited that pain.

Given our position of wanting to pick _one_ mode and lock it in (which seems to have paid off), I don't think we could have made a better choice in 2011 than AES-OCB. I might make the same choice today, but would look harder at the other available AE modes in OpenSSL.

<comment><USER6> Thanks! The details really help me understand the options available and why the choice was made. I appreciate your taking the time to answer the question thoroughly.

</comment></comment></comment></comment></comment></comment><comment><USER7> So it turns out that the attack "was possible due to the discrepancy between the proof of OCB2 and the actual construction" (according to the paper). That's a scary thought.

I wanted to learn a little bit more about progress towards verified implementations (i.e. deriving the implementation automatically from its proof) and found a nice summary here:

[https://crypto.stackexchange.com/questions/34304/formal- veri...](https://crypto.stackexchange.com/questions/34304/formal-verification- in-cryptography/34326#34326)

<comment><USER8> This is what makes me so uncomfortable about numerical computing (e.g. CFD): how do you know you got it right? If you implement a fluid simulation, it might _look_ right... but _is_ it right?

Of course, there's proof assistants etc that can construct an implementation, but even in this cryptography context, where it's desperately worth it, they're too difficult/too much work.

Unless you really understand the maths deeply, it's difficult to be sure you got it right (and even if you do). Especially when it's not easily decomposable into independently verifiable modules.

The funny thing is, when you write straightforward code, and try it a few times (even, enshrine those as "tests"), you will feel that it is right. But there's not really that much certainty that you really did get it right.

Leading me to believe the crucial thing isn't how difficult it is to get it right, but how much it matters. Software "bug reports" are ubiquitous. Not so for cryptography!

<comment><USER7> > This is what makes me so uncomfortable about numerical computing (e.g. CFD): > how do you know you got it right?

Indeed. Lack of associativity and distributivity when working with floating point numbers gives the whole thing a feeling of spooky witchcraft.

It's precisely the same feeling that I get from Javascript and PHP's weird notions of equality ("==" vs "===").

</comment></comment></comment><comment><USER9> OCB2 is not a block cipher; it is an authenticated encryption scheme built on top of one.

The scary thing here is not so much the error in the proof, which does not have many repercussions beyond OCB2, but that it went 14 years without being discovered. This despite the OCB2 paper, which introduced XEX, being highly influential. Every time something like this happens, confidence on the security of all "provably secure" schemes is undermined.

<comment><USER10> > Every time something like this happens, confidence on the security of all > "provably secure" schemes is undermined.

Put in an other way, provably secure schemes only guarantee what had been proved.

</comment></comment><comment><USER11> OCB2 is not a cipher, but a mode, like CBC, GCM, XTS, etc.

<comment><AUTHOR> Edited the title, thanks!

</comment></comment><comment><USER12> Where is it used and what is under threat now?

<comment><USER6> [Edit: OCB in general] is almost entirely unused due to the patents around it. Instead, people ended up using GMAC (e.g., AES-GCM mode) for the most part.

(Another good AEAD alternative today is Chacha20+Poly1305, standardized in IETF RFC 8439 (née RFC 7539).)

<comment><USER2> It is also virtually entirely unused because it was superseded by OCB3.

<comment><USER6> Ah, sorry for the confusion. I was referring to the entire group of OCB modes, not just OCB2. Edited my comment to reflect that.</comment></comment></comment></comment>


Crowdpac – See what US politicians really stand for based on donations and more - <AUTHOR> https://www.crowdpac.com/

<comment><USER2> So every issue is constrained to a left-right slider? How idiotic. Most of the issues listed do not break down anywhere close to such a simplistic two party model. For example, the intelligence/surveillance issue is presented as a liberal=freedom/conservative=security dichotomy when there are plenty of liberals who prioritize security over freedom and conservatives who are the reverse.

It only took me 3 issues until there were no matches.

</comment><comment><USER3> Great idea! I've actually wished for something like this before.</comment>


What laptop bag do you use? - <AUTHOR>

My current messenger bag does not fit my macbook pro 15 inch very well. What bags do you all recommend? <comment><USER2> <http://www.pacdesigns.com/>

The site is ugly and unusable, but the bags are absolutely stellar. I have the Pro Ultimate and I use it with my 17" unibody Macbook Pro in a Booq Viper ballistic nylon sleeve. I used to use top of the line Booq bags and ditched them entirely for the Ultimate Pac Bag. It's 3-4 layers with truck tarpoline liner in the middle. The fabric is Dupont Cordura Plus, which is the best you can get. The bag is very very waterproof. I've been caught outside biking home in thunderstorms with torrential downpours and I got home with the contents of my bag completely dry.

If you get one of the two biggest models, opt for the secret compartment. It's so well hidden that I had to contact the company to find out where it was on the bag.

If you anticipate weight, definitely get the x-strap option. You can carry a small child or medium-sized dog comfortable in one of these bags. See this image: <http://www.pacdesigns.com/images/offtoworkpac.jpg>

It's not a cheap bag, but I have no doubt that I'll have this thing 10-20+ years from now. It is rock solid.

</comment><comment><USER3> Bullfrog Carrier Bag from Brooklyn Industries: [http://www.brooklynindustries.com/index.cfm/fuseaction/produ...](http://www.brooklynindustries.com/index.cfm/fuseaction/product.detail/_/Carrier- Bullfrog- Bag/productID/1d65fbb4-a282-46ff-8780-1bf06a0c903f/categoryID/eb161d31-fcfb-4d00-bdb6-8addacd9e584?productOptionIDs=9fc8413d-035b-42b9-8c05-202af4ba70dd)

Every day for the last 4 years, I've carried a Thinkpad around NYC in this thing.

It quickly converts between two different configurations: (a) comfortable and back-friendly double backpack strap and (b) a more professional-looking single shoulder strap.

Has a padded laptop compartment, a main compartment, and two big external pockets.

Only complaint is the latches are plastic and have finally started to break. I love this backpack enough though that I'll probably take it to the tailors.

</comment><comment><USER4> I use a Crumpler Skivvy[1] for my MacBook Pro 15". Great bag and durable as hell, but this model is made to carry little of anything else—and opening the flap on mine can get loud with the amount of velcro it has, which they seemed to have doubled in size since? If you need to lug around alot of stuff you might want to size-up to their Considerable Embarrassment[2] bag.

[1] [http://www.crumpler.com/CA/Laptop-Bags/Laptop-Messenger- Bags...](http://www.crumpler.com/CA/Laptop-Bags/Laptop-Messenger- Bags/Skivvy-M.html?LanguageCode=EN&SKU=SKY000-B00150) [2] [http://www.crumpler.com/CA/Laptop-Bags/Laptop-Messenger- Bags...](http://www.crumpler.com/CA/Laptop-Bags/Laptop-Messenger- Bags/Considerable-Embarrassment.html?LanguageCode=EN&SKU=CET001-Y00150)

</comment><comment><USER5> Targus 17" CityGear Miami Messenger Laptop Case (TCG200) <http://bit.ly/gsbveL> Goes for about $40 @ Amazon.

I've got a 17" Dell so big, you could hit someone over the head with it and kill them. This was the only "slim" case I could find that it would fit in easily. I'm hoping to move on to a 17" MacBook Pro later this year and this case should work for it as well.

</comment><comment><USER6> I have the very popular Jack Spade Nylon Canvas Computer Field Bag (Olive). Can not complain, a great all around laptop and messenger bag.

[http://www.jackspade.com/shop/product.php?productid=20355...](http://www.jackspade.com/shop/product.php?productid=20355&cat=331&page=1#)

</comment><comment><USER7> I usually get inspiration from either thefancy.com or etsy.com for these things. It's fascinating how much beautiful self made vs. industrial made stuff you can find. Here's one rather fashionable MB15" bag <http://www.travelteq.com/>

</comment><comment><USER8> I got a $30 Swiss Army bag from Target several years ago. It fits a 15" laptop just fine and has room for two textbooks in another pocket. I feel like I've gotten my money's worth out of it and would recommend it in a heartbeat.

<comment><AUTHOR> Thats a nicer price that what I have been finding. I've been looking at <http://www.tombihn.com/page/001/PROD/300/TB0825> and <http://www.sfbags.com/products/cargo/cargo.htm>

<comment><USER8> You'd be overpaying at those prices, unless you care about the label. Go to Target, check out their luggage section, grab a $30 bag and go on one hell of a bender with the savings. That will make you far happier than a $220 messenger bag will.

</comment></comment></comment><comment><USER9> Outdoor Products Power Pack. BIG backpack (2500 in^3). Fits a 17" MBP with no trouble at all (in fact I've stuffed two 17" laptops into it at the same time). Has a great organizer and many cool features.

</comment><comment><USER10> I use my wife's old laptop bag. It's yellow with a blue and white floral pattern. Looks kind of like a diaper bag. Nobody is going to steal that baby.

I would like to get a skull iron-on patch for it.

<comment><AUTHOR> haha, thats awesome.</comment></comment>


No time for testing the system going to handle India's biggest Tax Reform - <AUTHOR> http://www.business-standard.com/article/economy-policy/no-time-for-software-testing-now-gstn-chairman-navin-kumar-117062600040_1.html <comment><USER2> The title seems a bit misleading.

> We did the testing on the basis of the rules that came in December. After > that, some changes were made to the rules. Those changes we have absorbed > now, so there is no time to do beta testing for that. Other than that, we > have done all other tests. So, we are fairly confident of the system.

Not saying the project and testing has been managed perfectly, but there has been time for testing unlike the title suggests.

Any large project will have bugs, especially with late changes and additions such as the ones described in the article. At least they plan to make relatively swift releases of new versions.</comment>


Ask HN: where to go from here with my app? - <AUTHOR>

I'm looking for feedback on my product as well as on the business model.
I am about to launch Afford-It, an app for Android built with jQueryMobile and PhoneGap. Its  function is to figure out for the user what part of their balance they can actually spend, and what part they need to reserve for recurring bills later in the month (like rent, power, cable, insurance).
My business model is to ask something like $3 in the Android market for the app. There are existing budget apps going for that price (after a free trial) and those are terrible apps that ask you to enter every single expense and income.
Perhaps a bank, or Mint, will come along and hire me to pimp up their online banking with my app's functionality. But that would be bonus, not baseline.
My USP is that with my app, you don't need to enter all your pastexpenses: after a one-time setup it only needs your current balance to give you meaningful advice, plus a list of upcoming bills.
There are admob ads in a few of the pages of the app, but I don't expect much income from them.
There are some basic things that will need to wait for v2: - entering incidental future incomes/expenses - patterns other than monthly (i.e. biweekly pay, saturday shopping trip, etc)
My current roadmap is to add these functionalities to my (cross-platform) code, meanwhile get it on iPhone and iPad, and maybe do a Dutch translation (I'm living in the Netherlands, so it might be worthwhile to push a Dutch version where I have better in-person access opportunities)
My questions are these: - Where should the product go from here? - How does the business model sound to you?
The online demo is at: www.afford-it.me/demo. You can log in with username hackernews and password reviewers. <comment><USER2> _after a one-time setup it only needs your current balance [...] plus a list of upcoming bills._

The demo stopped working for me in the browser after it asked for the initial balance ("next"-button did nothing).

Either way it sounds like I'd have to constantly update your app with my purchases and expenses and income? Why would I want to do that?

<comment><AUTHOR> Sorry, I forgot to mention: you need to open it in Chrome (or Safari), and specifically not in Firefox. The app uses localdb which Firefox doesn't support. (I hope this was in fact the cause of your problem).

As for constantly updating: no! The whole point of this app is that you only need to enter your current balance. From your balance and your recurring bills (one-time setup), it calculates when your next income is due, which bills need to be paid before that, and that leaves as a remainder your free-to-spend amount.

<comment><USER2> Hm. I'm afraid I don't see the point.

So I tack in my income, my recurring expenses, and then it tells me my disposable income per month (a static figure)?

What is the advantage over just tapping those figures into a calculator or an excel sheet (which is probably easier to update later)?

</comment></comment></comment><comment><USER3> If you are charging $3 for the app, you shouldn't have AdMob ads in it. Offer a lite version with stripped functionality and the advertisements to complement the full paid version.

Sidebar: UX need to be tightened up and polished - take a look at <http://mobile-patterns.com/> for inspiration.

<comment><AUTHOR> Thanks for your comments, Rich. It seems to me that the current version is already pretty stripped. Any bits in mind that seem "Pro" to you?

Thanks for the mobile patterns site. I'm aware that buttons and lists are not quite consistent in shape and function across the app, haven't really pinpointed a problem though.

</comment></comment><comment><AUTHOR> PS: Please open the app in Chrome (or Safari), not Firefox. In Firefox it will break due to a lack of localdb support.

<comment><USER2> It didn't work in Chrome for me either (OSX, Chrome 17).</comment></comment>


Halliburton tries to patent patent trolling itself - <AUTHOR> http://boingboing.net/2008/11/17/halliburton-tries-to.html

<comment><AUTHOR> Maybe this could be the way out of this patent nonsens ?

Patenting patent trolling by one company devoted to this, so that it could devote it's own existence to troll patent trolls.

That would a delicious and pure irony.

<comment><USER2> Yeah, Halliburton is going to save us.

More likely they'd want to license their "technology" to other patent trolls.</comment></comment>


Why Software Development is a Craft - <AUTHOR> http://blog.8thlight.com/articles/2009/10/22/why-software-development-is-a-craft

<comment><USER2> Although many of the people who have been attracted to this movement, mainly because they are passionate about programming, I can't help feeling that this metaphor like many others does not describe software development either. I'm not sure that it is that useful.

For me software is becoming more varied and ubiquitous it is some systems will need engineering vigour, others perhaps are design problems, others have heavy linguistic elements, others are mathematical and many are built from building blocks. Moving through the software profession I think you meet characters that in other worlds would be anything from lawyers through bankers to builder/plumbers.

[http://www.kheper.net/topics/blind_men_and_elephant/index.ht...](http://www.kheper.net/topics/blind_men_and_elephant/index.html)</comment>


The ghost factories: histories of automata and artificial life - <AUTHOR> https://www.tandfonline.com/doi/full/10.1080/07341512.2020.1757972 <comment><USER2> From the questoining-assumptions-I-didn't-know-I-had department.</comment>


Shepard Fairey Pleads Guilty Over Obama 'Hope' Image - <AUTHOR> http://artsbeat.blogs.nytimes.com/2012/02/24/shepard-fairey-pleads-guilty-over-obama-hope-image/

<comment><USER2> The article does a poor job of explaining _specifically why_ he may see jail time. Misuse of a copyrighted image in itself is rarely cause for imprisonment (at least today).

Fairey is in hot water _not_ because of the copyright issue, but because of the falsified evidence he provided as part of his lawsuit against the AP. So everyone understands the series of events, Fairey saw the heat coming and filed suit against the AP pre-emptively. In that suit, it is now clear that he did broke some laws completely unrelated to the copyright infringement. This statement is based on the quote (from this article): "...later admitted that he had been mistaken and had tried to conceal his mistake, by destroying documents and fabricating others."

So, Fairey, like others before him, will not suffer the greatest penalty from his canonical crimes, but for his subsequent actions attempting to avoid the consequences and lying to the courts.

</comment><comment><USER3> Shepard Fairey needs to find himself a new legal team. If his current legal team could make a good 'fair use' defense for the Obama image, then Fairey probably wouldn't feel the urge to destroy evidence in order to win his case.

Meanwhile, his legal team was out issuing 'cease-and-desist' letters over the Obey Steelers baby dolls. <http://gawker.com/5166157/obey-trademark-law>

</comment><comment><USER4> I don't think that photographs of publicly elected officials should be copyright-able in this context: it should be fair use.</comment>


Panic Resets Oil Prices - <AUTHOR> http://www.businessweek.com/bwdaily/dnflash/content/oct2008/db20081010_772519.htm <comment><USER2> And the grass was green and the sky was blue.

'Circle of life' FTW.</comment>


Fire TV: Everything You Need to Know About Amazon's Streaming Box - <AUTHOR> http://gizmodo.com/amazons-fire-tv-everything-you-need-to-know-1556889628 Will this shake things up? <comment><USER2> And I just bought the new Roku Streaming stick...I'll be curious to see how they stack up.</comment>


Ask HN: What music helps you code better? - <AUTHOR> Techno? House? Classical Piano? Pop? No music? What helps you code better? <comment><USER5> Mostly music with no lyrics. That's key.

I'm currently enjoying brain.fm set to 30 minutes and synced with a 30 minute pomodoro timer. I enjoyed the trial enough to purchase a lifetime membership (using a discount code) as I feel much more productive, especially when combined with some Bose noise-cancelling headphones. I've spent more money on less productive things so I figured "why the hell not?". It may just pay for itself.

If not brain.fm, then I'd prefer to listen to movie soundtracks, such as Another Earth, I Origins, Manglehorn, or The Wilderness.

Some artists I've enjoyed while coding:

Piano: Ludovico Einaudi, Dirk Maassen, Johann Johannssen, Olafur Arnolds, Joep Beving

Electronic: Tricky, Moderat (particularly the instrumental album).

Heavy Rock: Earthless, Russian Circles

Post Rock: Sigur Ros, Explosions in the Sky, This Will Destroy You

<comment><USER6> I agree about 'no lyrics', at least in my experience, being the key to coding productively while listening to music.

What I'm wondering is, what is it about music WITH lyrics, which interferes with coding? ... is it the language processing side of the brain being overloaded or just distracted from the task at hand ( which would be translation...right? ) i.e. coding == actively translating tasks and concepts into a machine's language.

<comment><USER7> For me it works well to listen to music with lyrics that I know so well that I dont even really listen to the lyrics anymore.

In general I feel like music works better if I know it well.

I like to listen to more distracting music though when doing a "boring" task that does not require that much brain processing itself.

<comment><USER5> Good point! A boring task would definitely free up some of that brain processing.

My coworker watches tv shows on his second monitor and listens to murder- mystery podcasts all day since his job mostly entails cranking out web banners and print ads of various sizes. He could pretty much do it with his eyes closed, as he's been doing it for so long. Not to mention he's just damn good at his job.

I guess it all depends on how much of your personal processor you can spare for your current task.

</comment></comment></comment></comment><comment><USER8> I listen to New Retro Wave and outrunner almost exclusively.

I have also found that video game soundtrack remixes can help to preserve focus as well.

</comment><comment><USER9> For me, the only genre that reliably works is "minimal" music, like stuff by Steve Reich[1] or Gas[2]. My definition of this genre would be music with no lyrics that has highly repetitive structures (differentiating it from "ambient" music, which I find too sleepy to be invigorating).

[1]: Music for 18 Musicians [https://www.youtube.com/watch?v=ZXJWO2FQ16c](https://www.youtube.com/watch?v=ZXJWO2FQ16c)

[2]: Koenigsforst [https://www.youtube.com/watch?v=ehNGoyMf9kU](https://www.youtube.com/watch?v=ehNGoyMf9kU)

</comment><comment><USER10> I did an entire project listening to the Steely Dan box set in constant repeat. Something about the precision and clarity of the studio work does it for me. I'll still fire it up when I need to slam something out.

</comment><comment><USER11> John Cage's 4'33 on repeat.

</comment><comment><USER12> Mostly ProgMetal/Djent. Some artists inspire me a lot while coding:

      - <USER2>
      - <USER3>
      - David Maxim Micic
      - Arch Echo
      - Animals as Leaders
      - <USER4>
      - Scale the Summit
      - Angel Vivaldi
      - ... etc – a.k.a. their related artists

There's also this album I used to listen to while reading Blood Meridian and nowadays while coding:

      - Hex: Or Printing in the Infernal Method by Earth

Hope they inspire you too ;-)

<comment><USER13> You and I have similar tastes. I'd also like to through in some names for this genre... \- Cloudkicker \- Pomegranate Tiger \- The Helix Nebula

Outside of this, I really like video game music while coding. Anything from the Playstation 1 era, namely the Final Fantasy 6-10 soundtracks, both OST and orchestra versions. Nobuo Uematsu is incredible.

<comment><USER12> Nice additions! Your taste is good, sharp! And WOW, Nobuo Uematsu is really incredible. Thanks for the recommendations!

</comment></comment></comment><comment><USER14> Posted in HN a few weeks ago [0], Someone wrote a pretty good article on the tricks they use to be more productive developers. At the end, they cover music genres.

From personal experience, I agree with everything he says. Music with less or no lyrics keeps me focused more on coding. When a song starts playing with lyrics that I enjoy, it distracts me from what I am working on. I particularly enjoy Lo-Fi Hip Hop[1], because there typically are not very many lyrics in the music and it typically has white noise in the background to help reduce distracting noises.

[0] - [https://raddevon.com/articles/focus-for-better-work-life- bal...](https://raddevon.com/articles/focus-for-better-work-life-balance-as-a- web-developer/)

[1] - [https://www.youtube.com/watch?v=FlKlmgksvgQ](https://www.youtube.com/watch?v=FlKlmgksvgQ)

</comment><comment><USER15> Ambient, lo-fi, chillstep and garage is the best. I love these so much that I produce music for coding myself. These are my best playslists [https://open.spotify.com/user/1167646885/playlist/0adabJ0Yu4...](https://open.spotify.com/user/1167646885/playlist/0adabJ0Yu4hRQ50cakvNNg?si=Jq1ls_FDR3GeNY9uoIpi1Q) [https://open.spotify.com/user/1167646885/playlist/04uAJE4hhK...](https://open.spotify.com/user/1167646885/playlist/04uAJE4hhKVupggI29x2xy?si=1e3UgpPHSS6zuR61cA0Z3g)

And my own EP [https://open.spotify.com/album/40Td5u6cbC8iApy1VMopCu?si=uqv...](https://open.spotify.com/album/40Td5u6cbC8iApy1VMopCu?si=uqvnTZvwTp2rtMotXKHOiw)

<comment><USER16> thanks for sharing!

</comment></comment><comment><USER17> Check out [http://www.focusatwill.com](http://www.focusatwill.com) \- music designed / curated by neuroscientists for focus

</comment><comment><USER18> It depends on the day, but for some reason when I really got to get stuff done, I listen to the theme for Ghostbusters II on repeat all day, Bobby Brown's "Control." I wonder if my coworkers hear me listening to the theme from Ghostbusters II all day on repeat and think I'm weird, but I get so much done with Ghostbusters II theme song. I get too hot to handle, too cold to hold.

</comment><comment><USER19> Anything with a decent tempo (except for Dance Music - too much thoomp thoomp) that is decent but that I don't know the words to. The important part is not knowing the words. If you know the words it can pull you out of the zone without you realizing. As for type I have no specific type, I'm all over the place, just as long as it follows the above criteria.

<comment><USER20> Haha, that "thoomp thoomp" genre is called "unsa unsa" in some ex-Yugoslav counties.

</comment></comment><comment><USER21> The Dwarf Fortress background music.

Game music (mostly) is designed for a related purpose - setting a mood whilst also encouraging you to focus on the task at hand (whether it's shooting aliens or squashing bugs). Film scores are similar.

</comment><comment><USER22> For coding, it's usually classic music (no lyrics) but while researching, I'll put on techno. No matter the genre though, I will use MusicChoice which is the streaming service that comes with my cable tv subscription.

</comment><comment><USER23> The Social Network Soundtrack [https://www.youtube.com/watch?v=yydZbVoCbn0](https://www.youtube.com/watch?v=yydZbVoCbn0)

</comment><comment><USER24> Any music that blocks me from external noise but at the same time doesn't distract me. Anything without lyrics is best but I have found Classical Piano works the best for me.

</comment><comment><USER25> [https://www.mixcloud.com/prototype202/](https://www.mixcloud.com/prototype202/)

</comment><comment><USER26> Depends on the situation, but I've found that hard rock has helped on many occasion (AC/DC, Steppenwolf)

</comment><comment><USER27> I've been listening to Animals as Leaders recently it's instrumental and is kind of Jazzy, and Heavy Metal.

</comment><comment><USER28> House / Tropical House when typing away, no music when designing

</comment><comment><USER29> I used to write university assignments under the influence of Rush.

</comment><comment><USER30> Swedish death metal

<comment><USER31> Gilfoyle?

</comment></comment><comment><USER32> musicforprogramming.net is pretty good

</comment><comment><USER33> down-tempo.

If I'm not doing architecture, emails, or thought provoking tasks, I can get away with lyrical music.

</comment><comment><USER34> "The Dark Knight" OST

</comment><comment><USER35> Downtempo (Tycho etc)

</comment><comment><USER36> SomaFM Groove Salad

</comment><comment><USER37> Nordic Death Metal

</comment><comment><USER38> Hardcore Gabber.

</comment><comment><USER39> Synthwave.

</comment><comment><USER40> Silence.

</comment><comment><USER41> beatbasement / somafm

[http://serv1.beatbasement.com:9622/](http://serv1.beatbasement.com:9622/)

[http://ice1.somafm.com/secretagent-128-mp3](http://ice1.somafm.com/secretagent-128-mp3)

[http://ice1.somafm.com/defcon-256-mp3](http://ice1.somafm.com/defcon-256-mp3)

[http://ice1.somafm.com/deepspaceone-128-mp3](http://ice1.somafm.com/deepspaceone-128-mp3)

[http://ice1.somafm.com/thetrip-128-mp3](http://ice1.somafm.com/thetrip-128-mp3)

[http://ice1.somafm.com/cliqhop-256-mp3](http://ice1.somafm.com/cliqhop-256-mp3)

[http://ice1.somafm.com/earwaves-128-mp3](http://ice1.somafm.com/earwaves-128-mp3)

-

sleepbot

[http://sc13.shoutcaststreaming.us:8194](http://sc13.shoutcaststreaming.us:8194)

[http://sc7.shoutcaststreaming.us:8044](http://sc7.shoutcaststreaming.us:8044)

-

bbc radio [http://bbcmedia.ic.llnwd.net/stream/bbcmedia_radio2_mf_p](http://bbcmedia.ic.llnwd.net/stream/bbcmedia_radio2_mf_p)

(bbc radio one) [http://bbcmedia.ic.llnwd.net/stream/bbcmedia_radio1_mf_p](http://bbcmedia.ic.llnwd.net/stream/bbcmedia_radio1_mf_p)

(bbc radio fourfm) [http://bbcmedia.ic.llnwd.net/stream/bbcmedia_radio4fm_mf_p](http://bbcmedia.ic.llnwd.net/stream/bbcmedia_radio4fm_mf_p)

(bbc radio five live) [http://bbcmedia.ic.llnwd.net/stream/bbcmedia_radio5live_mf_p](http://bbcmedia.ic.llnwd.net/stream/bbcmedia_radio5live_mf_p)

(bbc 6music) [http://bbcmedia.ic.llnwd.net/stream/bbcmedia_6music_mf_p](http://bbcmedia.ic.llnwd.net/stream/bbcmedia_6music_mf_p)

(bbc 1xtra) [http://bbcmedia.ic.llnwd.net/stream/bbcmedia_radio1xtra_mf_p](http://bbcmedia.ic.llnwd.net/stream/bbcmedia_radio1xtra_mf_p)

(bbc radio four extra) [http://bbcmedia.ic.llnwd.net/stream/bbcmedia_radio4extra_mf_](http://bbcmedia.ic.llnwd.net/stream/bbcmedia_radio4extra_mf_)

(bbc world service) [http://bbcwssc.ic.llnwd.net/stream/bbcwssc_mp1_ws- eieuk](http://bbcwssc.ic.llnwd.net/stream/bbcwssc_mp1_ws-eieuk)

</comment><comment><USER42> I listen to old french house classics. It's an odd choice and occasionally the lyrics or samples distract me, but I figure the potential for distraction is always there and I'd rather be yanked from concentration into my most pleasant memories.

I also like some trance music or instrumental movie scores if they're chill enough.

But it kinda depends on what kind of code im writing. If I'm knee deep in the guts of some poor program with disassembly and an exploit and pointers floating in my head, I have a pair of those bose noise canceling that I can wear for complete silence.

I don't think music is necessarily a good frame for productive coding. At least not on its own. When I sit down with an intention, I'm just hearing ambient office noise until something a bit more interesting catches my attention, I kinda feel that urge to put my seatbelt on, Spotify is just a reflex, and I'll get to debugging or whatever I'm doing.

I know some guys who have like playlists or whatever queued up for work, but that's not really how I work. Hell, I know another guy in my department who wears noise canceling headphones all day every day. That's just how he likes it.

</comment><comment><USER43> Mostly soft music without lyrics.

\- Luv letter / Flower Dance \- Sakuya2/3 \- REBORN -Dr.ミンチに会いましょ \- 失, 焚梦 |A Chinese musician 十指流玉 \- A Little Story (etc...)

some Japanese Team Shanghai Alice soft musics</comment>


Node v7.9.0 Released - <AUTHOR> https://nodejs.org/en/blog/release/v7.9.0/ <comment><USER2> Just a reminder that this month towards the end (or the next one if it slips) the version 8.0.0 is going to be released, but it won't become LTS until after another 6 months.

I am building a project ( [https://serverjs.io/](https://serverjs.io/) ) based on modern Node.js (ES6+) and cannot wait for it to come. IMHO the future is looking great.

</comment><comment><USER3> Lol, love commits that reference voodoo... [https://github.com/nodejs/node/commit/d4d6986551](https://github.com/nodejs/node/commit/d4d6986551)

</comment><comment><USER4> Is there anywhere a human readable list of what's new?</comment>


Tech Journalism Is Less Diverse Than Tech - <AUTHOR> https://oonwoye.com/2020/07/31/tech-journalism-is-less-diverse-than-tech/ <comment><USER3> Having had a significant other that worked in journalism, this makes sense to me. Journalism has become a profession which is much easier to follow if you have an "elite" background. It can be a prestigious job, but with a low salary for decades. To break into it you often have to earn your stripes and work your way up. In "elite" journalism--major newspapers and outlets--this often includes very low paying or non-paying internships in some of the most expensive cities in the country. It's a lot easier to do this if you have parents that can pay your rent or max out their non-taxable gifts to you each year.

To even get these internships it helps to go to one of the top 5 J-school graduate programs, which are expensive ($30k+ tuition/year) and in high cost of living cities (except Mizzou). Due to the racial wealth gap in the US-- which was caused by systemic racism--this means that upper-class White people are going to be the ones that can best outlast this system and end up making it as a journalist.

<comment><USER4> There's also an extreme supply-demand mismatch in journalism (as there is in acting, music, and similar fields), meaning that many persons in the field take a lot of their "income" from the profession via non-monetary sources: [https://jakeseliger.com/2019/11/19/have-journalists-and- acad...](https://jakeseliger.com/2019/11/19/have-journalists-and-academics- become-modern-day-clerics)

</comment></comment><comment><USER5> Note that linked study [0] uses too high white percentages for Facebook.

> As of 2020, the percentage of white employees at Facebook is 63.2%.

It links to this [1] document where if you click on "whites" in the "US ethnicity" section, there is indeed a 63.2% percentage. But that number corresponds to the green curve which concerns people in leadership roles. The number for all facebook employees (including the technical engineers, etc) is the blue one, which only has 41%. In the image, facebook would thus be closer to the other tech companies and further away from the journalists.

It's also interesting to note that the percentage of nonhispanic whites in the US population is 60.1% [2], so they are actually underrepresented in big tech companies (and overrepresented in tech journalism). Didn't know that!

[0]: [https://tech-journalism-diversity-report.github.io/code/](https://tech- journalism-diversity-report.github.io/code/)

[1]: [https://diversity.fb.com/read-report/](https://diversity.fb.com/read- report/)

[2]: [https://www.census.gov/quickfacts/fact/table/US/PST045219](https://www.census.gov/quickfacts/fact/table/US/PST045219)

<comment><USER6> To add to that, that's counting Jews as white. When you don't, two very unfortunate things happen: first non Jewish whites are now the most underrepresented category, and two you end up sounding like Charlie Chaplin with a mustache.

<comment><USER7> Most Jews in America are Ashkenazi and are essentially German, Polish, Ukranian for them most part.

i.e. 'Zuckerberg' 'Sandberg' 'Goldman'.

They are genetically not different from other Europeans other than for some possibly some minor specific linkage way back. That wing of the diaspora has been in Europe for at least 1000 years, originating in the area from Netherlands down to Frankfurt and spreading Eastward towards Russia.

Even many Sephardi jews, from the Med (i.e. Spain, Greece, Turkey), are more European than they are Middle Eastern at least genetically.

<comment><USER6> The point here is to show the silly consequences of categorizing people by "race", not to argue about the minutiæ of how to properly do race discrimination.

<comment><USER7> So I think that's a fair point, at the same time I don't think 'race' is quite a silly thing, just not a hugely important thing.

If race is silly, then is BLM a silly concept?

We could say the same thing about 'silly culture' or 'silly language' or 'silly attitudes' or 'silly politics' ... so that we can just all focus on the _truly important_ things like our 'iPhones'.

At very least it's history, and interesting on that level, more controversially, it's part of our identities at least in a broad sense.

I find this [1] basically fascinating.

[1] [https://en.wikipedia.org/wiki/Genetic_studies_on_Jews](https://en.wikipedia.org/wiki/Genetic_studies_on_Jews)

</comment></comment></comment></comment></comment><comment><USER8> I remember a photo of a Gamergate group as compared to all of the HuffPo journalists together. The latter was nothing _but_ white women, as compared to the actually varied racial and gender composition of the former. Physician, heal thyself, I guess.

<comment><USER9> You may be thinking of this much-lampooned tweet with a photo from a Huffington Post editors' meeting:

[https://twitter.com/lheron/status/733758898855940098](https://twitter.com/lheron/status/733758898855940098)

It looks even worse at a few years' remove.

<comment><USER10> I'd invite everyone to take a look at the picture in above link, but I don't think it's "nothing but white women" as the grandparent comment suggests.

Maybe it's not this picture, but just want to make sure the criticism is on the right thing.

<comment><USER11> I guess one of them is asian? Maybe there's a very white latina as well? Can't tell.

</comment></comment></comment></comment><comment><USER12> we should be promoting equality of opportunity not equality of outcome - <USER2> dont understand how this chart with %of white people makes any sense at all - did you do multivariate analysis to see what factors are a concause for someone to be hired?

<comment><USER13> Equality of opportunity is really close to equality of outcome. If two groups are offered the same opportunity, but one is pressured to take it and the other is pressured to take something else, the opportunity is the same but the fact that the outcomes are different speaks to something else going on, like the pressures in my example. Given the history we have, inequal outcomes can pretty frequently be directly tied to issues that still don't provide the kinds of agency we want. I think of it as equal opportunity being a lever, while equal outcome is a measurement.

For example, suppose you find some data about outcomes in different groups with covariate X. Maybe there's an outcome gap between groups which goes away when you account for X. That doesn't mean the disparity in outcomes is fair. It just punts that question to a new question of whether the disparity in X is fair. And so on.

<comment><USER14> This is a vast assumption, and one that often produces inequalities more than it reduces it. In reality, people's preferences often do vary between groups even without discrimination or inequalities of opportunity. E.g. over 90% of pediatricians are women, and there's no evidence that this is due to discrimination against men. The representation of women in technology is actually highest in countries with less opportunities for women, and lowest in countries with the greatest equality between men and women. Efforts to force an equal outcome in the context of unequal preferences does not reduce inequalities of opportunity, it produces inequalities.

<comment><USER13> I would say the same about the case that you're making. It's a vast assumption that produces more inequality than it reduces. It's a question of nature versus nurture. Our preferences are a function of our expectations and the expectations of us. I agree that the underrepresentation of men in pediatrics, for example, is a function of preferences. But I think those preferences are functions of nurture more than nature. If men were the primary caregivers for children, I expect the representation would be flipped.

Taking today's society and culture as a given, yeah I agree with you. But I think that's a problem with the prevalent culture.

<comment><USER14> If these disparities are attributable to culture, we'd expect different cultures to produce different outcomes. This is not the case. Women make up the majority of pediatricians in all societies that give men and women equal opportunity to become doctors. Men outnumber women at least 2:1 in all countries. And what variability does exist is actually inversely related to equality: women are _more_ likely to be in STEM in patriarchal countries, and less likely to go into STEM in egalitarian countries.

<comment><USER13> > If these disparities are attributable to culture, we'd expect different > cultures to produce different outcomes. This is not the case.

This isn't good evidence. We don't have cultures where birth control was invented hundreds of years ago and then they evolved independently. Or where a knowledge economy mattered more than raw strength for any reasonable time. Biology mattered so much more across cultures for most of history, so the fact that they all have patterns like this isn't really evidence that it must not be culture.

<comment><USER14> I think missed the crucial part of the previous comment: when birth control was invented and countries became more egalitarian, the disparities between men and women in many professions _increased_. Societies where women still aren't permitted good accesses to reproductive control see higher rates of women in STEM. Had birth control been invented earlier, current trends indicate that disparities of women in STEM would be even greater.

</comment></comment></comment></comment></comment></comment></comment><comment><USER15> Sorry but "18% other" just doesn't seem fair to write. Come on, an article about diversity that just groups ~1/5 into "other"...

</comment><comment><USER16> Tech human resources, legal, and recruiting are overwhelmingly white women, to wit no one cares. Truck drivers are 94% men, no one complains about that. The business and political community is hyperfocused on software engineering jobs for two reasons. Corporations want political support for H1-B. And, in some cases, STEM workers have high importance in influencing the political world as we saw with Facebook in the 2016 elections. Therefore, the political world is hyper-focused on those specific jobs to hire people that they think are sympathetic to their political parties, usually along race and gender lines.

<comment><USER17> In my opinion is a very strange that HR is overwhelmingly female. It seems to me that many companies consider HR to be just administrative bureaucracy. Yet talent development and organizational psychology have been pioneered by male researchers and psychologists.

<comment><USER18> HR are administrative bureaucracy. They are neither expected nor trained to do anything with organizational psychology. Nor have power to change organization.

They do know some psychology, but more of tactical "how to calm this person right now".

</comment></comment></comment><comment><USER19> May I suggest a follow-up article with the title _' Journalism is politically less diverse than its audience'_ in which a rallying cry is heard to shake up the market? To do anything else would be rather hypocritical given the great drive for diversity in everything and anything.

</comment><comment><USER20> Maybe I'm getting more conservative and irritable in my old age, but I've really gotten saturated and tired of diversity, diversity, diversity -- "we should reflect who our customers are", etc. As if that's the durable standard and goal of our society. I just don't buy it.

Here's what I'm _for_.

I'm for removing barriers to anyone being able to be interested, educated, and encouraged to do the things they want to do. And we should work to make it easier for people to be identified as having skills and talent in whatever field they choose.

Then let the chips fall where they may. This push for equality of outcome + diversity is political bullshit of the moment. Every field and industry and job will reflect the demographics of who is interested in doing that job, the pipeline of people who seek to become educated and qualified for it, and the proportions of who succeeds at showing their skills. You will not escape that truth -- and trying to do so will cause more frustration and political backlash than working within that truth.

Fashionable as it may be to latch on to a message right now, I don't think the majority of people buy the idea that diversity in itself means that any selective field _ought_ to equal the proportions of the groups you choose to divide general society into. People can simply see that it's not true as an outcome. And that movement will lose.

<comment><USER21> Thinking about this with an engineering mindset, how can you know that you are offering equal opportunities of you don't measure the outcomes? What can you measure that shows men are as likely to be given a tech job as women, if not the outcomes of men and women working there? For do you know you've made it as easy for a black student to be a great software developer as it is for a white student, if you don't look at great software developers and measure how many are black or white?

<comment><USER20> How do you know if the measurements you're taking at the end of the process are valid to interpret, if you don't know the inputs?

I'm not willing to trust the one (or act on it in a way that distorts what could be an otherwise fair/unbiased process) without knowing the other.

My problem with this approach is that people are too willing to stop halfway and claim that the outcome looks biased, without seeing what the inputs were. And cause (or seek to cause) massive change to people's lives based on that.

<comment><USER21> I absolutely agree, you need to try to measure boy side of the equation, not only the outcomes.

</comment></comment></comment></comment><comment><USER22> This is important information and an interesting point, but I'm not surprised that it's less highlighted. Tech companies are some of the most powerful entities in the world. They pay their employees extremely high salaries, make billions of dollars and control how basically everybody goes about their life, from using the internet to interacting socially.

Tech journalism...is none of those things. Oh sure they have power in that they can critique the companies and expose their issues. But their influence is not on the same level as tech companies' influence. Their journalists are not getting paid high salaries. They're not controlling our daily lives.

Journalism as a whole? Diversity is definitely a hot topic there. Tech journalism? Ehh.

Of course we should care about diversity throughout society. But that's not how it works. We care about diversity when there is power, either an intense amount of it or an intense lack of it. We care about the diversity of Ivy League schools and the diversity of prisons. We don't care about the diversity of baristas. Therefore we don't care about it in tech journalism.

<comment><USER23> The point is that we care about stuff that journalism says we should care about.

</comment></comment><comment><USER7> In this particular case, it's not about white people's representation, it's Asian. Asian people are highly represented in tech, and they are not in journalism. That's the qualifying issue. So understanding why that disparity exists is the key, it has little to do with 'diversity policies' or anything like that.

</comment><comment><USER24> I'm hardly surprised they don't practice what they preach. The majority of journalistic outlets have rapidly descended into yellow journalism, lies, doxxing and slanderous allegations.

Journalism is a race to the bottom in quality, and journalists absolutely don't put the effort in anymore. Hitting the streets and risking their lives to get that one big story just isn't a thing anymore, now they get all their information from Twitter and engage in some pretty egregious conduct against ideological opponents.

ABC News covered up allegations against Epstein. NPR falsely accused innocent assault victims of being "right-wing extremists", then refused to retract it. New York Times thinks doxxing controversial people is okay. All of them vehemently support censorship, and some even call for the prosecution of their own sources.

These institutions don't need to be saved or reformed, they need to be burnt to the ground.

<comment><USER25> _ABC News covered up allegations against Epstein_

And NBC covered for Weinstein, see Ronan Farrow' book.

</comment></comment><comment><USER26> interesting, one assumed that journalism per se is diverse than other fields.

<comment><USER27> One reason could be that journalism typically requires a college education and then doesn't pay that well. Many media internships are also unpaid. That means you need some source of "runway" in the form of previous savings or family wealth to get started and build your career.

<comment><USER28> On the other hand tech is one of those industries where someone from say even a journalism degree can get a job after a short bootcamp (and there are many free ones if you are a woman or minority)

</comment></comment></comment><comment><USER29> I think the article is right to raise the point about diversity in tech media. However, posting it here and some of the comments just feels like whataboutism.</comment>


When setting an environment variable gives you a 40x speedup - <AUTHOR> https://news.sherlock.stanford.edu/posts/when-setting-an-environment-variable-gives-you-a-40-x-speedup <comment><USER2> Story time.

This took place around 2009. Back then I was working for Rapid7, on their network vulnerability scanner Nexpose. The codebase was mostly Java and was relatively large at 1M+ lines of code. We had many unit tests. Running the entire test suite took up to 40 minutes on Windows, and 20 minutes on Linux (Windows was always about twice slower on everything: building, product's startup time, etc.) The company had grown quickly to at least 30-50 software engineers. The problem was that every time one of them ran a build on his or her local machine (which happened multiple times a day) it would have to run the test suite and waste up to 40 minutes of this person's time. 40 minutes × dozens of engineers = lots of inefficiencies in the company.

I loved solving performance issues so one day I remember arriving at the office and making it my mission to investigate if there was an easy way to speed up the test suite. Our build system was based on Ant, and our tests used the JUnit framework. After a little time profiling Ant with basic tools (top, strace, ltrace), and taking a few Java stack traces, I realized that most of the wasted time was not actually running the individual tests, but many instances of the JVM kept being started and destroyed between each test. Our Ant build file was running the JUnit test with fork=yes, which was required for a reason I don't recall at the moment. This forks the JVM for running the tests. Then a little googling lead me to this:

[https://ant.apache.org/manual/Tasks/junit.html](https://ant.apache.org/manual/Tasks/junit.html)

While reading this documentation, I stumbled upon an unknown parameter to me: forkmode. What does it do?

" _forkmode: Controls how many JVMs get created if you want to fork some tests. Possible values are perTest (the default), perBatch and once. once creates only a single JVM for all tests while perTest creates a new JVM for each TestCase class. perBatch creates a JVM for each nested <batchtest> and one collecting all nested <test>s._"

Our Ant build file did not set forkmode, so it meant we were forking a new JVM for every test!

I immediately tried forkmode=perBatch and... the test suite ran 10× faster! 40 minutes down to 4 minutes on Windows. And Linux ran it in 2 minutes instead of 20 minutes. I told my boss right away but he was unbelieving. He asked that I check with our most-experienced Java developer. I showed him my 1-line patch speeding the test suite 10× and he said "I guess you are right, we can commit that." By lunch time the fix was committed and everyone loved me :)

<comment><USER3> That's the time to ask for a raise!

<comment><USER4> You're gonna be very disappointed in the workplace if you expect a raise every time you're just doing your job.

<comment><USER3> If he was just doing his job the implication is that no one else was doing theirs at the same company.

<comment><USER4> Don't you think it's far fetched to say that every single developer at the company is useless because they didn't look into ant build settings?

<comment><USER3> I'm playing devils advocate, but the point is he used initiative. He wasn't asked to improve the build, he was curious and found a solution.

Is that worth a raise? Probably not on it's own but I was saying it's a good time to ask for one if you are secretly thinking you should get one as you have a good good forward.

On a meta level a company that doesn't taskmaster or track everyone's time usage will get this kind of result from time to time. Curious professional just making things better. Continuous improvement.

</comment></comment></comment></comment></comment></comment><comment><AUTHOR> This was written by my coworker, Kilian Cavalotti. We have two main file stores: $SCRATCH is a Lustre cluster, which is extremely performant but doesn't do well with lots of inodes. And $HOME is a multi-node Isilon, which can handle all the inodes, but is not as performant. And we have users who sometimes like to put many files in single directories.

You can find more information about us at [https://srcc.stanford.edu](https://srcc.stanford.edu)

And you can find more information about Sherlock at [https://www.sherlock.stanford.edu](https://www.sherlock.stanford.edu)

<comment><USER5> Tell your colleague I think emojis make the article look like it was written by a 12 year old.

IMO. I guess I'm just an old fart.

<comment><USER6> It is how most millenials write nowadays

<comment><USER7> It's how people trying to write like millennials write

Postfix emoji, eurgh

</comment></comment></comment></comment><comment><USER8> > Having thousands of files in a single directory is usually not very file > system-friendly, and definitely not recommended.

I wonder, is this specific to the situation when I use "ls", or is it in general?

I recently worked on a project where I need to store many small files on ext4, although these files are not read/written by human. I came across suggestions to group files into subdirectories and not put them all in one directory. Is there evidence that it's actually worth it on a modern filesystem?

EDIT: by "many" I mean 10+ million files.

<comment><USER9> For future reference, next time you need to store millions of related small files, it might be worth checking if SQLite is an option instead of using the filesystem: [https://www.sqlite.org/fasterthanfs.html](https://www.sqlite.org/fasterthanfs.html)

Performance aside, I wish NodeJS had a way to pack all its little turds into one SQLite file, out of sight. (Admittedly I hadn't thought to look for such a thing until now... Preliminary results are negatory.)

<comment><USER10> You should benchmark SQLite before recommending it like this (for future reference). I think it's a very elegant library, and I admire the development philosophy that went into it. However, every time I've tried to use it for something that needs to be fast it ended up being a painful mistake.

<comment><USER11> Could you show us your methodology for your benchmark? We don't know under which circumstances you experienced degraded performance.

I've seen a lot of people just missing an index or something, and their DB then ran fine, so that's why I ask.

<comment><USER10> I don't work for that company any more, and I couldn't have legally showed the code even if I did.

It's not an index thing though. Try inserting 10 million rows into a simple table. No foreign constraints or anything. Some of our "documents" had 300 million rows. Handle the primary key however you want (let the DB do it, or generate your own). Use whatever SQLite API calls you want (precompiled statements, whatever). In a case like this, adding other indexes can only slow it down.

There are a few options to disable integrity and other safety constraints. They help speed things up a little, but it's all painfully slow in comparison to a writing simple CSV file.

The same is true on reading. You can parse an entire CSV file more quickly than "SELECT *" from the table.

I've tried several time to use SQLite as a document file format. It performs worse than CSV and JSON. It performs much worse than just an array of binary records. The devil is in the details, and assuming each file is not a single row/record, I wouldn't be surprised if 10+ million files in a structured directory tree performs better too.

<comment><USER9> Well, I certainly agree about benchmarking versus other options, especially if I/O proves to be a bottleneck. The other option above was not a single CSV or JSON-structured file, but millions of small files stored on the filesystem (with its requisite indexes, permissions, access logs, etc). And the comparison is not with the file contents being splayed out into a relational structure in SQLite, but just one flat key-blob table for storage and retrieval. It's possible that a multi-gigabyte CSV file would be faster still, depending on actual access patterns and how much effort you want to devote to this bespoke CSV database.

<comment><USER10> > The other option above was [...] millions of small files stored on the > filesystem

I don't have any direct comparisons of SQLite to this approach, but other projects I've worked on did have a simple directory tree organizing medium size chunks of data by date over 20+ years. We had one "DB" where everything was stored by the degree of latitude and longitude it fell into, and another where things were stored by date. Both were gigabytes in size at a time when that was considered a large data set, and it was very fast to open only what you needed.

Depending on the problem, this can be a very good solution. It was trivially accessible from any programming language (including ones that didn't exist when the DB was started), and it was trivial to migrate to new operating systems (some of this started out on VMS).

I like SQLite quite a bit, but it's not always the best solution to storing tabular/relational data.

<comment><USER9> How many files are you talking about? That's the relevant variable, not the amount of data or whether it's text, tabular or relational data, images, audio, etc. E.g. you can copy a single 10 GB file to another drive faster than 1 million files totaling 1 GB, all due to filesystem overhead. If there's a filesystem where that's not true, I'm interested. :)

<comment><USER10> > How many files are you talking about?

The numbers won't seem significant by today's standards. In the one case (stored by date), maybe 150 thousand files, each a few megabytes.

> you can copy a single 10 GB file to another drive faster than 1 million > files totaling 1 GB

True, but I'll bet you can create (or read) a thousand files with a thousand records each faster than you can insert (or select) a million records into (or from) a SQLite table.

<comment><USER9> 150,000 is solidly in the realm where SQLite tested faster than the filesystems in the link above, although their files were only a few kilobytes. It's almost certainly different (worse) for multi-megabyte files. But what I've been trying to convey is that the number of records in the SQLite table will be identical to the number of files. If you need to parse the file contents, you'd parse the BLOB just the same. The difference is in how you interact with the disk.

Is reading X-thousand files containing a thousand records each (or one thumbnail, or one HTML dump, or one JavaScript function, or whatever) faster than SELECTing exactly the same number of BLOBs containing exactly the same data? It's worth considering and testing once the number of files starts affecting performance or even just becomes a pain to deal with. If it turns out that storing many files is still a better fit for a particular application, that's cool too. Nothing is a panacea.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> I started reading this thinking it would talk about LC_ALL/locale related, but not too surprised to see another env variable throwing a wrench in the works.

<comment><USER13> for context you refer to the fact that

        LC_ALL=C grep string hugefile

is often magnitudes faster (also works for other shell utilities) - this is due to unicode handling that is either complex due to the nature of unicode or just performances badly on glibc - probably it's the complexity.

<comment><USER14> Seems analogous to xml parsing in Perl, where I realized a 10x speedup in parsing by switching from a general-purpose XML parser to a set of regular expressions.

<comment><USER15> [https://stackoverflow.com/questions/1732348/regex-match- open...](https://stackoverflow.com/questions/1732348/regex-match-open-tags- except-xhtml-self-contained-tags/)

If you read nothing else, make sure you read the first answer there.

<comment><USER16> If you want to extract just the values of a single tag or similarly trivial things or the XML is a very restricted set, regexes are just fine for parsing.

<comment><USER17> it's really not

<comment><USER18> If you want to extract strings from XML that arrives in a well-known format that meets certain expectations that may not be rigorous, then regexes can be fine.

</comment></comment></comment></comment></comment></comment></comment><comment><USER19> Fave speedup story, in the 80s a good friend had written a custom accounting system in CBASIC and he came to me in regards to one client, a boat charter service management business that maintained a set of 'virtual double-entry books' for every one of its 200+ member yachts. The monthly process was taking 4 hours to complete. Could I improve on that?

Well in a couple hours I discovered that his routines assembling the main batch file did repeated lookups in other files without caching any operation results, and because it was MS-DOS the OS didn't cache many sectors either. So the hundreds-of-thousands of preparatory operations were waiting for the hard disk platter to come around again, for each. Yes, even hard disk platter speed was a significant factor in those days.

So I added a 15 element array that cached lookup operations in memory. From 4 hours to 15 minutes, Thank You Very Much.

<comment><USER20> May I contribute with an hybrid optimization. Some workers were tasked to remove duplicate in excel files by hand. They'd have to create filtered view for each entry, Excel being slow it took hours. Some woman had a 3000 lines spreadsheet and was about to lost it. I wrote a 8 lines VBA to count duplicates, just enough so she could filter anything that had count > 1 and then delete things on her own. Turning multi-day job into a two clicks operation. Probably my most happy lines of code in my entire life.

</comment></comment><comment><USER21> "he mentioned his laptop was 1,000x faster than Sherlock to list this directory's contents"

"that's a 40x speedup right there"

So... what about... the rest?

<comment><USER22> Good point! But sometimes a large-scale parallel filesystem can't beat a local SSD on certain patterns.

Also, we didn't see any timings for that 1,000x claim, so let's say it's an estimate. :)

<comment><USER23> It's a great article, thanks.

But .. do people actually use `ls` to list so many files? I mean, it'd scroll off the terminal anyway.

<comment><USER24> I admit to being guilty of the odd

      for i in $(ls)
      do
        Something dumb
      done

<comment><USER25> Out it interest why not xarg or find's exec

<comment><USER26> Not GP, but I often find myself writing making statements (which is awkward with find), or using syntax that find struggles with.

</comment></comment></comment></comment></comment></comment><comment><USER27> The irony here is that instead of just unsetting the alias that is proxying `ls` to `ls --color=auto` the solution was to leave the coloring in place but try and hack the override env var for defeating that behavior.

ie, `var foo = 1 + 1 - 1` instead of `var foo = 1`

Wouldn't it have been more straightforward to open one's .bashrc/.bash_profile and remove the alias?

<comment><USER28> Sure, but they WANTED colors. The just wanted them to be fast.

<comment><USER22> Precisely. We want colors. We're refined people.

</comment></comment></comment><comment><USER29> I was once flown out to look at a significant performance problem. The customer could not do a days worth of work in a day. We'd told them to buy a fully loaded Sun E15k, and it couldn't keep up.

I instrumented everything (recompiled everything so that the binaries spat out gprof data, iostat output, vmstat output, sybase monitoring) and it quickly became obvious that the DB driven config was configured in a way we'd not anticipated. The fix was easy (just use a btree in a couple of places instead of a linear array walk), but then we had a huge problem.

With the fix, the CPU bottleneck became a 15% CPU utilisation. The system had been sized based upon our advice. Whoops. What happened next was political and I won't go into it. Deliberately vague on the industry and customer because of the 7 figure cost involved!

<comment><USER30> So before diagnosing the problem you made the customer spend huge amounts of money based on a guess. Not the least bit surprised.

</comment></comment><comment><USER31> I really enjoyed the writing style. This was fun to read. Thanks!

<comment><USER32> I feel the opposite, I'd rather they get to the point in the first paragraph then follow it up with casual examples of how they arrived at the solution.

<comment><USER22> True, that could have used a TL;DR, agreed.

<comment><AUTHOR> But then people would miss the rainbows!

</comment></comment></comment></comment><comment><USER33> The tl;dr here is that "ls" can be much faster if you disable colorizing files based on the their file capabilities, setuid/setgid bits, or executable flag.

        LS_COLORS='ex=00:su=00:sg=00:ca=00:'

<comment><USER34> And if you don't want angry fruit salad colors in your cool hipster retro- Matrix semitransparent tiling-window desktop theme:

      unset LS_COLORS
      alias l='/bin/ls -a'
      alias ll='/bin/ls -alF'

<comment><USER35> I'd argue that for the vast majority of manual usages of ls, the maybe second or so saved listing directory contents by removing colors will be dwarfed by the additional time it takes the human to parse its contents, without the additional context provided by those colors.

<comment><USER34> I'd agree with you: judicious and consistent use of color should give better HCI performance than the colorless method, for the majority of people.

But it's very visual and aesthetic, and maybe it's best considered a user preference, which is why the tongue-in-cheek rationale, when I mentioned the option.

<comment><USER36> Good points. And for those wondering, "HCI" means Human Computer Interaction (had to check).

<comment><USER34> Thanks. HCI (and human factors engineering) were the original areas of study concerned with these questions. UX, which is more popular at the moment, seems to have more emphasis on visual appeal and marketing psychology, rather than on effectiveness for the user's goals. You could see "dark patterns" as an extreme of this shift in intent.

</comment></comment></comment></comment></comment></comment><comment><USER37> I thought this was about setting NODE_ENV=production, and reading other comments others know different ways. Seems there are many ways to get a system an order of magnitude faster with little, but deep work/knowledge.

</comment><comment><USER38> It may be worth mentioning that this sort of thing isn't just a problem for whoever is running it, because it may have a bad effect on other users if you hammer the metadata server(s). It's necessary to have monitoring which can identify the jobs responsible and take action. The Lustre system here was clobbered for several days because the admins had no means of doing that, despite all the facilities provided for and by Lustre.

</comment><comment><USER39> I suspect something like "ls | cat" would speed it up as well. Probably tests if stdout is a tty before bothering with stat() and color logic.

<comment><USER22> True, but then you would loose all the coloring. By targeting only the attributes that generate additional syscalls, you can keep the vast majority of the colors and get a nice speed bump. Win win.

</comment></comment><comment><USER40> I was expecting some new flag about getdents bigger buffer size.

[http://be-n.com/spw/you-can-list-a-million-files-in-a- direct...](http://be-n.com/spw/you-can-list-a-million-files-in-a-directory- but-not-with-ls.html)

</comment><comment><USER41> An often-neglected related trick is the "dir_index" feature of ext3/ext4

</comment><comment><USER42> The author removed the coloring, because the default setup is for ls to list files with colors. He uses an environment variable to do that.

Wouldn't it be simpler to use \ls instead of ls? This way you get the "basic version of the executable", not some alias. I always use \command instead of command in my shell scripts, because you never know which aliases a user set up.

<comment><USER43> But aliases are not supposed to be available in subprocesses/scripts, unless you use shopt -s expand_aliases. Even if they were, you cannot ask all your users to rewrite their scripts to use \ls instead of ls.

</comment></comment><comment><USER44> The conclusion seems unsatisfactory. Why is lstat so much slower on their system than on the laptop?

<comment><USER45> Compute clusters generally use distributed file systems, over the network...

<comment><AUTHOR> Exactly that. We have 2000+ users, and 1000+ compute nodes. Each compute node does have a certain amount of local SSD (`$L_SCRATCH`) for job-local storage, but for everything else we have to use some sort of network-accessed file system. For us, that is NFSv4 for longer-lasting data (`$HOME` and `$GROUP_HOME`) to the Isilon over Ethernet, and short-term data (`$SCRATCH` and `$GROUP_SCRATCH`) to Lustre over Infiniband.

(Yes, Infiniband! Hello, Mellanox!)

</comment></comment></comment><comment><USER32> tl;dr version

LS_COLORS='ex=00:su=00:sg=00:ca=00:'

</comment><comment><USER46> Wow. Miss this for decades. Life waste so much on colour.

</comment><comment><USER47> Why are people so lazy at naming? [https://en.m.wikipedia.org/wiki/Sherlock_(software)](https://en.m.wikipedia.org/wiki/Sherlock_\(software\))

What ever happened to meaningful names that described the particular, not the meme?

<comment><USER17> like GNU?

<comment><USER47> I refuse to defend any gnu or unix naming conventions.</comment></comment></comment>