Why the Boeing 737 will never take the place of the 757 - <AUTHOR> https://thepointsguy.com/news/737-never-replace-757/ <comment><USER3> Having worked for 3 years designing 757 flight controls, it's a special plane for me, too. I always enjoy finding the bird I'm booked on is a 757. Some of the guys I worked with on it were an engineer's engineer. I lost my fear of flying through working on the 757.

<comment><USER4> > I lost my fear of flying through working on the 757.

That's refreshing. I have a friend who worked on the A380 and she says she'll never set foot on one.

<comment><USER5> For what it's worth, I work with many people deeply involved in the A380 design from the beginning (and other Airbus aircraft), and they don't pass a chance to fly it.

<comment><USER3> The Air Force would send the chief mechanics up on check rides. It's a great way to ensure they did their job well :-)

</comment></comment></comment></comment><comment><USER6> The 737-MAX is the plane equivalent of Windows XP Service Pack 8.

> JetBlue is among airlines planning to use the [AirBus A321LR] on routes > across the pond, serving Western Europe from New York and Boston.

This has to be a gut punch for Southwest, who has famously relied only on the 737. They're close to hitting their ceiling on domestic expansion with the 737 and can't compete on US-EU routes because of their fleet.

I like that the author points out the realities of modern 737 performance - just because it _can_ reach coast to coast destinations doesn't mean it _should_.

Thats not to say that the 757 is perfect. From what I've heard, maintenance is significantly more complex than the 737. And as the article pointed out, it takes way too long to board/deplane for the short hops that the 737 dominates now.

<comment><USER7> Southwest as far as I'm aware had shown no interest in transatlantic routes, nor do I think they've hit their domestic growth limits - there are still lots of places they don't fly.

<comment><USER8> Once Southwest gets over the hurdle of flying something other than 737s, it'll have transatlantic ambitions.

<comment><USER9> But flying a single kind of plane has a lot of advantages, doesn't it?

It seems that switching to a new plane would mean a drastic economic change.

So the transatlantic ambitions should be a cause to go past that, rather than a side effect.

<comment><USER10> Having a single source for planes also has disadvantages, as the recent fiasco demonstrates.

The CEO is on record saying that they will be evaluating having a second source of planes next year:

[https://theaircurrent.com/airlines/southwest-wants-a- small-a...](https://theaircurrent.com/airlines/southwest-wants-a-small- airplane-from-boeing-or-airbus-and-that-could-reshape-aviation/)

</comment></comment></comment></comment></comment><comment><USER11> Isn't this exactly what technical debt is? The term originated in software engineering, but it seems to be exactly what Boeing has if we transfer the terminology into aerospace.

> is a concept that reflects the implied cost of additional rework caused by > choosing an easy (limited) solution now instead of using a better approach > that would take longer

737 rated pilots are abundant, production lines are ready and paid for and orders come in as long as the product works. There is huge incentive to do just small fixes if you can't see technical debt in the books. When it shows up in the bottom line company is already paying interest.

</comment><comment><USER12> The points guy is a terrible source largely full of mistakes and poor information.

For instance: " If built, the 797 would bridge the range and capacity gap between the narrowbody 737 family and the much larger 787 and 777 families — a slot occupied by the now-geriatric 757 and 767."

The 787 and the 767 are the same size to a rounding error. The 787 is basically a drop in replacement for the 767 but much more efficient.

And further, the 737MAX and the A321LR (and A320neo) and even the A220 are designed to replace the role of the 757 in air travel. The US market wants smaller aircraft that are more efficient and fly longer range to open up more frequencies. They're getting them.

> " I may be biased, since I fly it, but you might undoubtedly call it the > most versatile jetliner Boeing has ever built. It's a medium-capacity, high- > performing plane able to turn a profit on both short and longer-haul routes"

Even a quick perusal down Wikipedia would tell you the 757s are incredibly inefficient aircraft, getting a second wind a cargo haulers due to their way outsized power to weight ratio. And even with all that they can't make regularly scheduled east coast transatlantic flights without often stopping along the way if there are temperatures or headwinds.

As for domestic travel the sheer inefficiency of the planes makes them a poor contender. Really only the full-transcon corridor made any sense at all and even those are being replaced. JetBlue went all A320. They're by no means "able to turn a profit on short and medium haul flying" solely by virtue of them being scheduled. They're often flying them for utilization reasons. Can you even call any of this flying "profitable" when American had a 3% margin last quarter across their entire route network, lost money on domestic and made it up on international?

The 737 will absolutely take the place of the 757, and already was scheduled to prior to the MCAS issues coming to light.

<comment><USER3> > the 757s are incredibly inefficient aircraft

Ironic considering the genesis of the 757 was a new wing and new engines that produced a 35% improvement on fuel efficiency over the 727 it replaced.

<comment><USER12> Oh definitely but that was almost 40 years ago (launched in 1981, designed even earlier). Progress is a wonderful thing!

</comment></comment></comment><comment><USER13> Never understood why the 757 wasn't more popular?? It had the best performance in its class while still being economical. Maybe back in its day there was too much overlap between it and the 767, but from talking to pilots who have flown it, the 757 handled like a fighter jet, and passengers seemed to love it too.

<comment><USER14> If the 757 were still in production today, it would still sell OK.

There is now a big demand for aircraft that size for certain routes (US transcontinental, and second-city Transatlantic, especially the transatlantic)

What happened is that laws changed to allow twin-engine aircraft to fly routes much farther from the nearest alternate landing site, making 757s suddenly great vehicles for Transatlantic flights between second cities. Of course, this happened shortly after production of the 757 ended.

<comment><USER15> _If the 757 were still in production today, it would still sell OK._

ETOPS helped the 757 into the international role sure, but the 737 and A320 are more fuel efficient and benefit from being able to share crew.

<comment><USER14> 757-200 has about 800 nautical miles more range than the longest-range 737NG.

737-900er, the longest-range 737NG only has 3000 nautical miles of range, which is _just enough_ for TATL flights. The 757-200's slightly longer range opens up many additional city pairs without having to upgauge to a 767.

Of course, now that Boeing has lost a couple of years due to massive failures (like the 737-MAX), Airbus has had time to catch up -- and the A320neo-based A321-XLR finally surpasses the 757-200 in this space.

However, the XLR seats about 70 fewer passengers than the 757, so the 757 will probably still live on for a little while longer.

If Boeing hadn't killed the 757, and then immediately start sucking, there might've been a refresh to a 757 MAX-like aircraft that sat 225-290, had a 4500nmi range and was efficient. But, you know, we got the real timeline instead.

<comment><USER15> _737-900er, the longest-range 737NG only has 3000 nautical miles of range, which is just enough for TATL flights. The 757-200 's slightly longer range opens up many additional city pairs without having to upgauge to a 767. _

The 737-900ER doesn't have the range of the 757 but it will use less fuel on the routes both can do.

 _However, the XLR seats about 70 fewer passengers than the 757, so the 757 will probably still live on for a little while longer._

For airlines that already have 757s, maybe. The XLR and 757-200 both offer nearly identical seating (with the nod going to the XLR in one class config) and range. The 757-300 can seat about 40 more in one class config, but you take a pretty hefty range penalty. Ultimately the XLR is pretty much the direct replacement for the 757.

The XLR (and 900ER and MAX) also have one big advantage that no revamp of the 757 will offer: commonality. You can share flight and cabin crew between the 318/319/320/321 (CEO and NEO) and between the 600/700/800/900/900ER (NG) and 7/8/9/10 (MAX).

The Airbus lineup (320/330) is further compelling because going between the narrow and widebody product requires relatively little training.

 _If Boeing hadn 't killed the 757, and then immediately start sucking, there might've been a refresh to a 757 MAX-like aircraft that sat 225-290, had a 4500nmi range and was efficient. But, you know, we got the real timeline instead. _

Boeing killed the 757 because nobody bought it. Same with the 767. In fact the only reason the 767 line is still open is because the USAF bought the KC-46. Meanwhile 757 operators like Iceland Air were moving away from the 757 to the MAX. The 757 was a great plane, but it was the wrong product at the wrong time.

Personally I think the 737 ought to die. I don't like it as a passenger, and I can't imagine it being fun in the legendarily noisy and cramped cockpit on a long haul flight.

<comment><USER14> On the XLR, you're right. I was misreading Wikipedia's capacity tables -- the XLR and the -200 are virtually identical in passenger capacity

</comment></comment></comment></comment></comment></comment><comment><USER16> Boeing has botched this so badly that the replacement for the B-737 will probably be the Comac C919. It's not clear how good an aircraft it will be, but the financing is very favorable.

[1] [https://en.wikipedia.org/wiki/Comac_C919](https://en.wikipedia.org/wiki/Comac_C919)

<comment><USER17> > Boeing has botched this so badly that the replacement for the 737 will > probably be

It will definitely be the A320 family, which has none of these problems, comes in sensible size variants already, is used extensively every day around the world, is price comparable, and isn't going to spook Western consumers in the same way that a Chinese plane will.

</comment></comment><comment><USER18> Is there a way to select for the aircraft when booking a flight?

Price, flight times, and minimal number of connections are the main focus; but is there a website that can add in aircraft type?

I realise types can be swapped, but with historical data there should be very high probability of prediction of aircraft type at booking.

I flew a 757 back in the 1980s and have fond memories of it. This article made me realise that a spate of flying in 737s might have been what put me off flying for so long.

<comment><USER19> Most airline websites and search engines will show you which type of aircraft a flight is on in the results. Alternatively, Google Flights or ITA Matrix can show you. You don't always get to find out easily which variant of an aircraft it is, e.g. a 757-200 vs. a 757-300 might both show as "757."

<comment><USER18> So there's a gap in the market if you want to be able to filter by aircraft.

I can imagine adding another dimension for searching flights could have an impact on usability though.

<comment><USER19> ITA Matrix is my favorite powerful flight search engine. It lets you filter by aircraft, among many other things. You have to learn what essentially amounts to a really obscure query language to use it effectively.

<comment><USER20> Google Flights uses ITA Matrix tech (bought by Google a few years ago) and it even gives you a link to buy tickets straight from the airline. Has pretty much the same usability features of ITA Matrix and none of the cruft of Orbitz et. al.; highly recommend it.

<comment><USER21> Some power user features of ITA aren't available on Google Flights still - <USER2> example, it lets you construct specific weird routings or layovers that Google will often optimize away. For instance, if you want a 24 hour layover in a city, or if you want to fly a less efficient direction to your destination to build mileage, or whatever.

You can use bookwithmatrix.com, though, which parses the output and will generate a ticket for booking on an online travel agency like Expedia.

<comment><USER20> Thanks for the recommendation! I hadn't heard of it bookwithmatrix.com.

</comment></comment></comment></comment></comment></comment></comment><comment><USER22> I am sure there will be all sorts of (correct) actuarial arguments that driving my car to work is more deadly than flying in the v2 737, but I still don't want to ever fly in a 737. It's got the mark of Cain at this point. I doubt I'm the only one that feels that way.

<comment><USER23> As I understand it, all 737s pre-MAX are fine; you want to avoid the MAX planes because of MCAS and the new engine placement.

<comment><USER15> _As I understand it, all 737s pre-MAX are fine; you want to avoid the MAX planes because of MCAS and the new engine placement._

Yeah, just don't pay too much attention to the structural problems that Boeing is having with the 737 NG (cracked pickle forks). The inspection threshold keeps getting lower and the scope of the inspections keeps growing.

<comment><USER24> What a branding disaster, tack on the name of reliable series of crafts to a flying suicide machine and destroy the whole line.

I'm sure many customers will avoid 737s in general due to confusion.

<comment><USER25> Branding is like leverage. There's a downside that goes with the upside.

</comment></comment></comment></comment></comment><comment><USER26> This article raises an interesting point in the 737 MAX discussions. Far too much the press coverage about the MAX has concentrated on whether the MAX can be made to fly safely - which I think it can be, but not, whether it is actually the right design for the tasks it is meant to do. And it does not surprise that stretching a smaller design wouldn't do the trick so well.

I am not an aviation expert, so the article was a bit short on details which make the 757 better than the 737 MAX, but I assume it is the wider body and probably better sized wings etc.?

<comment><USER20> 707, 727, 737, and 757 all have the exact same cross section. One of the advantages of the 757 is that it sits higher off the ground and can accommodate the large engines necessary for a longer fuselage. More generally, 757 is designed to be longer, with larger engines and wings, and higher fuel capacity. 737 had to have these features bolted on as it was stretched, fitted with higher bypass (more fuel efficient) engines, and given larger wings and fuel capacity.

</comment></comment><comment><USER27> 321LR will be very very successful if Boeing decides to scrap the 797 project.

</comment><comment><USER28> > What would it be like on the westbound leg, I wondered — a longer flight, > from a shorter runway, in the face of winter headwinds?

Wouldn't colder (denser) air and a headwind make the plane require less runway?

<comment><USER29> The author isn't worried about the takeoff speed in that quote; he's thinking about the increased duration of the flight as stiff winter headwinds (at cruising altitude) decrease the aircraft's groundspeed.

Prevailing surface wind direction doesn't vary much with season.

</comment></comment><comment><USER30> I always like flying on a 757 because of the middle-ish door position. It breaks up the interior and makes boarding and deplaning more pleasant.

I far prefer the MD-80/MD-90/717 to the 737. The rear mounted engines make most seats (except the very back) far quieter and landings always seem smoother. Obviously they're a smaller plane than contemporary 737s, but it's still far more comfortable than the cramped conditions of the 737.

</comment><comment><USER31> Will not touch technical side of plane. I am not so proficient in this field, but it's one of the best looking passenger jet what humankind have designed. Now only cseries/a220 have done it rigth

</comment><comment><USER32> 737 < 757! The argument holds!

</comment><comment><USER33> well, one's a narrowbody and the other a widebody with much higher PAX, so didn't really thing this was a point that needed to be made...

<comment><USER34> The 737 and 757 have identically sized fuselages. In fact, they are both basically the same fuselage as the original 707.

</comment></comment><comment><USER35> The 737 max is what happens when the bean counters are in control of an engineering firm.

<comment><USER36> The bean counters will always be there. Alternatively, one can think of the 737 MAX (and prior new 737 models) as an example of regulation imposing perverse incentives. ( [https://en.wikipedia.org/wiki/Type_certificate](https://en.wikipedia.org/wiki/Type_certificate) )

<comment><USER37> Interesting point, but what would be the alternative? Are you really suggesting that pilots should be able to switch from e.g. the 737 to the A320 without any kind of additional training?

<comment><USER36> Type certificates are not about pilots; they're about airframes. Maybe skim the link?

> A type certificate signifies the airworthiness of a particular category of > aircraft, according to its manufacturing design ('type'). It confirms that > the aircraft is manufactured according to an approved design, and that the > design ensures compliance with airworthiness requirements.

And to be clear, since it seems my comment was also not totally clear: I'm not saying aircraft regulation / Type Certification is bad in general. Just that this particular implementation currently has unintended perverse incentives which lead to the 737 MAX.

<comment><USER38> I'm generally blown away at the audacity people have to suggest casually changing any FAA regulation, considering how good they are at their jobs...

<comment><USER36> I don't see how you parse that out of my comment without some extremely uncharitable interpretation. I haven't suggested any particular change, nor that any particular change be carried out "casually," without due consideration.

<comment><USER38> You were being uncharitable, why are you surprised it was interpreted that way?

<comment><USER36> Do two wrongs make a right? Please keep an open mind going forward.

<comment><USER38> Please don't tell me how to behave going forward, and if this is your way of admitting you did something wrong, you've really got to work on that.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER39> Instead of building an NMA with a limited market, Boeing needs to build a 737 replacement.

</comment><comment><USER40> Smith is spot on with his analysis of why no number of mods and modules will make the 737 into a 757.

Here on HN I have ranted about the wobbly bobbly handling, obsolete hydraulics, and noisy crappy cockpit of the 73, versus the modern and comfortable 75.

Pay attention to what the article says about takeoff and landing performance. We used to load the 75 heavily for trips between NY La Guardia and Chicago Midway, or Midway to Washington National. We could NOT load a 737-NG heavily due to field length or climb limits. Boeing made a bad, bad decision to push the 737 instead of advancing a fresh design.

Just wait and watch. They won't abandon the 737 until one or two more are lost due to runway overruns or something else related to its shortcomings. That is WITH better than average pilots, all the sooner to happen with typical ab initio crews.

<comment><USER41> I also like the intellectual idea of taking an older airframe (767-200) that is suitable for the task and modernizing it once, rather than applying yet another level of patches to something already pushed far outside of it's intended design envelope.

<comment><USER42> But why is it so hard to develop a new plane now, compared to patching an old one? One would think computers could make a single engineer hugely more effective nowadays than back then. Constraints, generative design, accurate simulations. Additive manufacturing, CNC machines and composites can make manufacturing a lot simpler too. Has it actually reduced any cost? Interest rates are really low too.

Is it that new planes being developed have much stricter regulations? If so, that's a problem, the incentives result in the public being less safe.

The A220 ran so much over budget that the whole program was sold to Airbus. Yet it seems like a really good airplane from what I've read. 787 and A350 were also terribly expensive.

Something similar has happened with nuclear plants. They seem to take much longer to do than way back and are so expensive to be barely profitable. A friend commented that one reason is because top talent doesn't see the nuclear industry as a "cool" thing or a viable career anymore.

There is one additional problem in airplanes and nukes. They are seen as prestige projects by governments. This makes it hard to make regular money in the industry as you always have subsidized competition.

<comment><USER43> It was always hard to develop a new plane. For all that the author waxes rhapsodic about the 747's 2 year design to test flight, the 747 development process very nearly killed Boeing (it brought Boeing far closer to the brink than the a380 did for Airbus).

<comment><USER42> That's true! This book, "The Jetmakers" is available online and it talks about these things, but in the 1945 to 1972 time frame. It is a good read.

[http://www.generalatomic.com/jetmakers/index.html](http://www.generalatomic.com/jetmakers/index.html)</comment></comment></comment></comment></comment>


Where Wal-Mart Failed, Aldi Succeeds (2011) - <AUTHOR> http://www.nytimes.com/2011/03/30/business/30aldi.html?pagewanted=all <comment><USER2> Something I've been working on in my spare time is a database of grocery staples (about 200 items, plus 80 common cleaning/toiletry items), their costs (per item, unit price, per package), and where you can get them for that price. Regular everyday price, not sale prices.

Aldi not only outcompetes most supermarkets -- including ones that are traditionally considered inexpensive like Market Basket -- but can go toe-to- toe with warehouse clubs and bulk services like Amazon Subscribe & Save. Having spent far more time in supermarkets than I'd care to admit, it's pretty obvious as to why:

* The stores are smaller. They pay less in rent.

* They have a limited selection of inventory. Don't expect lots of pre-prepared food, and fancy pre-made items. In fact, expect very little. It's mostly staples with some extremely common packaged convenience foods.

* As a result of the first two items, they don't need much in the way of staff to cover the store. They're spending their time restocking the store as stuff runs out, so things _don 't_ run out. However if you want some items, your best bet is to go at open.

* Which is 9AM. They also close at 9AM. Not running 24 hours means lower costs.

* They have few registers. My local store has four. However they don't bag your groceries. Your groceries get dumped into another cart at the end of the register, and you go pack it yourself. The speed in which a person can be checked out is VERY quick as a result.

* You pay $0.25 for a cart, and get it back when you return it. The upside is no one is outside rounding up carts.

* They provide no bags. At all. Though they'll happily let you take the empty boxes items came in.

Compared to the local BigCo supermarket chain, they come in at about 40% cheaper, even if you're buying store brand from the BigCo chain. On many items, they undercut both the local BJs and Costco, despite not selling that kind of quantity. They do it by providing exactly what you need at a grocery store, and nothing you don't.

Even compared to Market Basket -- which I do love and still go to for some things -- they still easily come in 15% cheaper. There are some things I can't get at Aldi (so I'll go to Market Basket, as I'll go to Trader Joe's to get high quality produce easier), but damned if they aren't a super efficient setup.

<comment><USER3> > Don't expect lots of pre-prepared food, and fancy pre-made items. In fact, > expect very little. It's mostly staples with some extremely common packaged > convenience foods.

In my experience, I would have to disagree. For what I have seen at the two nearest stores to me, 50% of the store's inventory is boxed, canned, and frozen foods. True, they do have your staples (your meats, veg, and dairy), but to say they have "very little", at least in my experience, is simply not true. In fact, the wife and I get most of our boxed and canned goods at Aldi.

> However if you want some items, your best bet is to go at open.

I've been at our stores pretty much at random hours and have never been disappointed in the quantity of inventory. It sounds like you might be visiting a mismanaged store.

> They provide no bags. At all. Though they'll happily let you take the empty > boxes items came in.

Not true. Again, at least where I am at. They offer paper bags for $0.10 and heavier reusable plastic for $0.30 (to the best of my recollection). They also offer the washable heavy-duty fabric variety as well.

<comment><USER2> > _Don 't expect lots of pre-prepared food, and fancy pre-made items. In fact, > expect very little. It's mostly staples with some extremely common packaged > convenience foods. In my experience, I would have to disagree. For what I > have seen at the two nearest stores to me, 50% of the store's inventory is > boxed, canned, and frozen foods._

I should've clarified more on the "fancy pre-made items". Most of the larger national chain supermarkets have grown more and more into this. In particular, one location in my area (in an affluent neighborhood) has dedicated over 1/3rd of its floor space to "hot and ready to eat" take-out style food, prepared on- site, and priced at $10/pound (excluding things like sushi). It's a common sight to see a cart filled with nothing but basically take-out boxes, bakery items, bottled water, and alcohol.

That particular store in that particular location, if I included frozen meals, and boxed/canned meals, it's literally 3 out of every 4 square feet.

> _However if you want some items, your best bet is to go at open. I 've been > at our stores pretty much at random hours and have never been disappointed > in the quantity of inventory. It sounds like you might be visiting a > mismanaged store._

No, this is more of living in a major city, and the location's relatively small size (the adjacent CVS is about 50% larger). I would compare the size of its produce area to 16-20% of that of a typical chain supermarket. As it result, it depletes _very_ quickly. This means that the highest quality stuff is gone within 10-15 minutes of open.

I wouldn't say that's unique to Aldi either. If you don't go to Market Basket close to open, don't expect high-quality produce to be there when you show up, and some items may be gone entirely. The reason being is the area MBs are always packed except right at open.

<comment><USER3> Thanks for the clarification, I understand where you are coming from now.

For pre-made, as you describe, yes, Aldi has nothing like that beyond maybe some sort of yogurt parfait. Stores like Kroger, Whole Foods, Giant Eagle, and the other major supermarkets reflect what you describe. It seems that my wife and I encompass a shrinking minority or people who eat home-cooked meals more often than not.

As for inventory availability, I guess it does depend on the store, as ours have been fairly good with keeping stock available. This is in a city (Columbus, Ohio) of 800,000+ with the 'burbs encompassing another million or so. We are becoming quite a foodie town, and I think that reflects a lot on the availability of fresh produce. We have many ethnic supermarkets as well, much larger than Aldi, that have a VERY wide assortment of foreign fruits and vegetables. I would have to say we probably have a lot of brokers here.

</comment></comment></comment></comment><comment><USER4> It is well-known that Aldi pays its emloyees pretty well (at least in the usual European countries). But they have few employees per shop. These are expected to work really hard and long hours and are expected to do everything necessary to keep the shop running (complaining that something is not part of what you were hired for is not allowed). In German TV there have been complaints about the hard work and long working hours, but none about the pay.

The pay is even so good that when Aldi expanded to Switzerland, there were lots of people who feared or complained preliminarily that Aldi would reduce the average wage level of the region. The opposite happened: After Aldi opened some shops in Switzerland there were people who complained that Aldi pays more for shelf stockers than lots of other professions after four years of hard apprenticeship:

> [http://www.tagesanzeiger.ch/wirtschaft/AldiLoehne- > provoziere...](http://www.tagesanzeiger.ch/wirtschaft/AldiLoehne- > provozieren-Lehrlinge/story/30678140)

<comment><USER5> I tried to work at an Aldi once, back in the late 90's. Their pay was nearly what the local factories made (Chrysler, Delco [electronics for GM], and Subaru a bit further away). They rarely hired folks at full time, however: Most worked 25 hours a week, but I think still had some insurance & vacation.

The most telling sign of the work environment: They had low turnover. Most retail seems to be the opposite.

</comment></comment><comment><USER6> Things have changed a bit for ALDI since 2011.

They're sinking $1.6 billion into store expansion and renovation. I live near one of the prototype remodel stores and it's really nice.

[https://www.ft.com/content/41644020-ee28-11e6-930f-061b01e23...](https://www.ft.com/content/41644020-ee28-11e6-930f-061b01e23655)

They also take credit cards now.

<comment><USER2> They are also rolling out contactless payments as well.

My guess is they initially avoided it due to costs (swipe/interchange fees), but looked at the improvement to checkout speeds and figured likely consider it a wash now.

</comment></comment><comment><USER7> LIDL, Aldi's german competitor is also coming to the US.

[https://www.forbes.com/sites/bryanpearson/2016/10/03/lidl- is...](https://www.forbes.com/sites/bryanpearson/2016/10/03/lidl-is- coming-12-reasons-u-s-retailers-should-care/)

</comment><comment><USER8> Unless you completely refuse or are unable to comparison shop, I don't get the Aldi thing.

Aldi is a cool store, but the stock is really haphazard, especially produce, and the meat sucks.

If you're really saving a lot of money, you're eating a lot of processed crap.

<comment><USER9> In germany, the meat selection in ALDI is very small. You go to ALDI for the off-brand sweets, the basic cooking ingredients (fruits, veggies, canned goods, etc.), drinks (water, beer, off-brand softdrinks) and some misc. household items.

I think the aldi nearest to my home does not even sell pre-processed meals outside of frozen stuff (like Pizza) and canned soups.

In germany, they also have a rotating selection of cheap random stuff, ranging from laptops over garden chairs and tools to rugs and sheets. It's usually a very limited supply and most of the time not that bad for the pricetag (even the laptops).

That said, I prefer the competitor "Penny", because I know the layout better and they are a bit nicer, at least in my town.

<comment><USER8> Sounds like they are the same as in the states.

</comment></comment></comment><comment><USER10> Walmart is for Ruralites and Aldi is for Urbanites, I think this sums up why NYT would praise Aldi over Walmart.

<comment><USER11> In my city, they are currently building an Aldi across the street from the existing Walmart. They are in a relatively rich first ring suburb.

<comment><USER10> Makes a lot of sense, suburbs are both literally and culturally the border between rural and urban</comment></comment></comment>


Kickstarter meets ticketmaster - <AUTHOR> http://www.getaudienced.com/

<comment><AUTHOR> We run Audienced, a platform that helps event organizers get a guaranteed minimum attendance for an event while eliminating all upfront financial costs & risks. Is that something that could be of benefit to you?

</comment><comment><USER2> Both Kickstarter and Ticketmaster are one word.

<comment><AUTHOR> thanks auto correct</comment></comment>


GCC 7.5 - <AUTHOR> https://gcc.gnu.org/ml/gcc/2019-11/msg00099.html <comment><USER2> If anyone else was questioning their memory, thinking GCC 9.x had already been released, see this:

>This is also the last release from the GCC 7 branch which will receive no further fixes from now on. GCC continues to be maintained on the GCC 8 and GCC 9 branches and the development trunk.

<comment><USER3> GCC 9.x? I'm stuck in the time when GCC 2.95 was a (buggy) thing everyone relied on until 4.x stabilized. Never used GCC ever since.

GCC is an important piece of software, probably as important as the Linux kernel, but by now I presume it should have become a gigantic pile of outdated and barely maintainable code. Unless I'm wrong of course!

<comment><USER4> llvm really gave gcc a kick in the pants. It's now blazing forward!

<comment><USER3> It did, though I think the support of languages and HW platforms is still a bit limited compared to gcc?

</comment></comment></comment></comment><comment><USER5> How does the GCC project afford to fix "more than 215 bugs" on an old branch? Do they cherry pick commits from 8 and 9? Do they fix bugs on the 7 branch and then cherry pick them into 8 and 9?

<comment><USER6> Yes. Fixes are made on HEAD and applied/back-ported to the release branches where appropriate.

</comment></comment><comment><USER7> There seems to be a lot of confusion in the comments here. At one time, GCC had three-part version numbers: major.minor.patchlevel. Starting with gcc 5, they switched to a two-part scheme: just major.patchlevel. 7.5 adds nothing but bug fixes to 7.4, there are no new features. That's why the changes page pointed to on gcc.gnu.org only describes gcc7 changes.

[https://gcc.gnu.org/develop.html#timeline](https://gcc.gnu.org/develop.html#timeline) gives a good picture of how gcc releases work.

</comment><comment><USER8> I went looking for release notes. Doesn't look like they exist, per se.

[https://gcc.gnu.org/gcc-7/changes.html](https://gcc.gnu.org/gcc-7/changes.html) is about the entire GNU 7 series, but it's not broken down by release. The list of tickets closed is mentioned at the bottom:

> GCC 7.5: This is the [ > [https://gcc.gnu.org/bugzilla/buglist.cgi?bug_status=RESOLVED...](https://gcc.gnu.org/bugzilla/buglist.cgi?bug_status=RESOLVED&resolution=FIXED&target_milestone=7.5) > ](list of problem reports (PRs)) from GCC's bug tracking system that are > known to be fixed in the 7.5 release. This list might not be complete (that > is, it is possible that some PRs that have been fixed are not listed here).

</comment><comment><USER9> Am I the only one unable to keep pace with compiler development? For some code on x86/64 I'm still using gcc 4.7, and a cursory test with gcc 5 (or even gcc 4.9) resulted in compile-time errors I hadn't time to look into. Maybe it's just a compiler or linker flag, which tbh is something that's gone out of control. Given the intimacy gcc has with glibc and the Linux kernel, in turn, what are folks using for a stable build environment that just doesn't need to change all the time because the software does what it's supposed to do, and doesn't need, nor benefits from high-energy agility? I know there are occasionally new targets and optimizations, but wasn't C a language supposed to be easy for writing a compiler for in a couple months? Well ok the same could said about a Unix kernel I guess. I mean I appreciate the gcc project for providing backports, but the new gcc versioning scheme makes release numbering kindof arbitrary, and can be interpreted to give a finger to older software especially in C-land.

<comment><USER10> Without seeing the errors it's difficult to say. GCC 4.7 is really old (2012), and there are only a few areas I can think of off the top of my head that might trigger such problems, mostly related to C11 support and buggy feature detection or erroneous usages by older code, such as static_assert definitions or tests, possibly directly or indirectly related to older glibc headers. Interestingly, most of the C-related changes from 4.7 to 4.9 _are_ related to C11.

GCC has been less aggressive compared to clang in adding diagnostics to -Wall that trigger on correct code; -Wno-missing-field-initializers and -Wno- override-init come to mind, but those predate 4.7. -Wexpansion-to-defined is super annoying, but that came much later (GCC 8?), and strictly speaking it triggers on undefined behavior--it's just that the undefined behavior was extremely convenient, pre-dated GCC, reliably supported (at least for the most common and possibly only usages) on almost all if not all major compilers, and is the only possible approach for writing lazy feature detection macros that can be defined in library headers without include'ing possibly unused headers, polluting the namespace.</comment></comment>


Fallout: Reading Kernel Writes from User Space - <AUTHOR> https://arxiv.org/abs/1905.12701 <comment><USER2> 'Remarkably, the patent explicitly states that: "if there is a hit at operation 302 [partial match using page offsets] and the physical address of the load or the store operations is not valid, the physical address check at operation 310 [full physical address match] may be considered as a hit"'

Wait, Intel thought the insecure optimization which lead to this vulnerability was such a great idea that they _actually patented it_? Wow.

<comment><USER3> The load and store buffers are an incredibly performance sensitive part of the processor. To be maximally conservative you'd fully resolve the physical address of the load and store (including the access check) before allowing the load to bypass a non-aliasing store, or before forwarding store data to a load. But you could be waiting awhile (for example, many processors have multiple levels of TLBs), during which time you've held up a load for no good reason.

The thing to understand about the quoted portion (here is the patent: [https://patents.google.com/patent/US7603527B2/en?oq=7%2c603%...](https://patents.google.com/patent/US7603527B2/en?oq=7%2c603%2c527)), is that you don't know if "the physical address of the load or the store operations is not valid" until long before you get to operation 302. The quoted paragraph is about how you get the correct result even if you optimistically forward the store data to the load.

The gist of the optimization is as follows. Operation 302 checks if the page offset of a load matches a store. Address translation won't effect the page offset (just the page number). So if there is a match, there is a good chance that the operations alias. Now, say you do a store followed by a load. At 302, you may not have the actual physical address of the load and/or store. But if there is a match at 302 (in the page offset), there is a good chance you're just loading from a location you recently stored to. Optimistically forwarding the data from the store buffer to the load allows the load to continue to make progress. Otherwise, you'd have to wait for both physical addresses to be resolved at Operation 310 before the load could continue.

As an aside, I think de-tuning this at the microcode or architectural level is probably a fool's errand. What you need is an architectural mode that basically says "this code needs to be protected from information leakage due to timing attacks." Then you can turn off speculation or whatever in such code.

<comment><USER4> Having a mode switch isn't enough; potentially any part of the program could contain a spectre gadget, so you'd need to run the whole program in that mode.

A perhaps better approach would be to ensure that speculation aborts actually do clean up the entire microarchitectural state - ensuring that the cache state (and other persistent state) isn't affected by aborted predicted execution.

<comment><USER5> Specifically, the question is whether the current process (user or kernel) has mapped _anywhere in its address space_ data the attacker[0] should not be able to access.

0: ie, there exists a entity (other than the end user) that should not have access.

</comment></comment></comment></comment><comment><USER6> Does anyone know of a list/summary of all the different ways OoO can be exploited to create vulnerabilities? It seems there's a new attack coming out every month, it'd be nice to see a list of attacks, affected configurations, and workarounds if they exist.

<comment><USER7> I like this one: [https://arxiv.org/pdf/1811.05441](https://arxiv.org/pdf/1811.05441)

</comment></comment><comment><USER8> for those who want to know about who's impacted :

"Intel. We notified Intel about our findings, including a preliminary writeup and proof-of-concept code, on January 31st, 2019. Intel had acknowledged the issue and requested an embargo on the results in this paper, ending May 14th, 2019. Intel has further classified this issue as Microarchitectural Store Buffer Data Sampling (MSBDS), assigning it CVE-2018-12126 and a CVSS ranking of Medium. Finally, Intel had indicated that we are the first academic group to report this issue and that a similar issue was found internally as well.

AMD. We also notified AMD's security response team regarding our findings, including our writeup. AMD had investigated this issue of their architectures and indicated that AMD CPUs are not vulnerable to the attacks described in this paper.

ARM. We have also notified ARM's security response team regarding our findings. ARM had investigated this issue and found that ARM CPUs are not vulnerable to the attacks described in this paper. IBM. Finally, we also notified IBM security about the finding reported in this work. IBM had responded that none of their CPUs is affected, including System-V and PowerPC."

And here's a description (from the paper) of the mechanisms :

"The Mechanism Behind Fallout. Fallout exploits an optimization that we call Write Transient Forwarding (WTF), which incorrectly passes values from memory writes to subsequent memory reads. In a nutshell, when the program writes a value to memory, the processor needs to first translate the virtual address of the destination to a physical address and then acquire exclusive access to the location. Rather than stalling the store instruction and subsequent computation, the processor records the value and the address in the store buffer, and continues executing the program. The store buffer then resolves the address, acquires the access to the memory location and stores the data.

When a value is in the store buffer, care should be taken that subsequent loads from the same address do not read stale values from memory. To solve this, the processor matches the addresses of all load instructions against addresses in the store buffer. In the case of a match, the processor forwards the matching value from the store buffer to the load instruction. To increase efficiency, the processor uses partial address matches to rule out the need for store-to-load forwarding. WTF kicks in when a load instruction partially matches a preceding store and the processor determines that the load is bound to fail. In such cases, instead of cleaning up the state of the processor, it marks the load as faulty, and incorrectly forwards the value of the partially matched store.

Exploiting the WTF optimization. Fallout exploits this behavior to leak, through a microarchitectural channel, the value that WTF incorrectly forwards. The attacker deliberately performs a faulty load, causing the CPU to transiently forward an incorrect value from the store buffer. We subsequently leak the value using a Flush+Reload [58] side channel. As the store buffer is a shared resource used by all software running on a CPU core, the incorrectly- forwarded value might not even belong to the attacker's process."

<comment><USER9> > ... we call Write Transient Forwarding (WTF)...

WTF starts to be rather overloaded acronym. Reminds me for example of "Wobbly Transformation Format" [0]...

[0]: [https://en.wikipedia.org/wiki/UTF-8#WTF-8](https://en.wikipedia.org/wiki/UTF-8#WTF-8)

<comment><USER10> "Fallout" also seems like a particularly bad choice of name to sell this vulnerability by, since the game series is so well known.

(I would also prefer descriptive names rather than this kind of marketing, but these are the times we live in.)

<comment><USER11> The instantly recognizable connotation of "fallout" is "nuclear fallout". The game series, which is post-apocalyptically themed, almost certainly derives its name from that.

<comment><USER10> Of course it does. But in the context of computing equipment, the computer game series was the first thing that came to my mind. Nuclear fallout is in a different domain.

</comment></comment></comment></comment></comment><comment><USER12> Shared resources, especially across privilege boundaries, almost always create undesirable side channels that leak information.

Hardware designers need to come up with better isolation mechanisms.</comment>


Applying for jobs through IT recruiters is a waste of time - <AUTHOR> http://developersrant.wordpress.com/2013/07/02/applying-jobs-through-it-recruiters-is-a-waste-of-time/

<comment><USER2> Ugh. Hate recruiters, recruiters suck yada yada yada. Seems this same post keeps getting rewritten and hitting the front page of hacker news.

The lack of comments shows how uninteresting yet another recruiter bash is.

<comment><AUTHOR> This is simply a personal experience expressed in the form of blog post.</comment></comment>


Planes, Games, and Automobiles: The Nomads of Game Development - <AUTHOR> http://www.vice.com/read/the-nomads-of-game-development <comment><USER2> _" The only way to understand the world is to go to the world," says Ismail. "You have to go there, because there's too much in the way. There's too much keeping you in your safe, understandable cultural bubble because people can talk to you and you can communicate. Honestly, understanding the world requires you to go where no one can understand you."_

Isn't this lifestyle built on another form of consumerism: that of riding giant steel machines that burn tons of petrol to fly in the sky, instead of the one he's avoiding?

I also really enjoy discovering new people, cultures and languages- but I find I can do that better by staying somewhere for a few years.

To each their own though! I support Rekka and Devine on Patreon, and their boat journey has been neat to follow (and this one consumes much less fossil fuels). My father having been an avid sailor in his youth, it's something I could see myself doing with him when he retires.

<comment><USER3> I think it's worse than that -- for me it all reads, "young person with zero responsibilities chooses unsustainable lifestyle and has an awesome time". What happens to the designers when their boat needs serious maintenance, something that boats need _constantly_? We already know that one of the van guys basically only was able to function because a friend helped him sell his (broken down) van and move his crap home for him.

Game development is also a terrible platform to build a life on. What happens when you burn out? What happens if (when) no-one likes your game?

<comment><USER4> Although HN loves to hate game development, and I understand where you're coming from, at least in Rami's case he's in the top 0.1% of indie game developers and has multiple $1M+ grossing games on the market ([http://steamspy.com/dev/Vlambeer](http://steamspy.com/dev/Vlambeer)). He is in every way the exception, but I think its ok for him to live and exceptional lifestyle if he wants to without fear of burn out or "no one liking his game". The HN view of game development tends to apply more to salaried AAA developers in mid or bottom tier studios, and new, inexperienced or unsuccessful indie game developers, which might be the majority, but are certainly not Rami.

<comment><USER5> > but I think its ok for him to live and exceptional lifestyle if he wants to > without fear of burn out or "no one liking his game".

So you are ok with him living his life how he chooses? Why do you think that is something that needed to be said? That should be everyone's default disposition, so I'm not sure if you're expecting a cookie for this or what?

Also, I don't think you have a strong enough grasp of this community to begin making assertions about its opinions towards certain professionals. As it is, your comment reads like "Silly HN readers, you guys aren't as smart as me, let me explain why you should share my opinion."

<comment><USER4> I have 4153 karma and have been an active Hackernews member for over 5 years, so I feel like I have a decent amount of insight into general patterns I see in the community. I'm aware that HN is super diverse and people have different backgrounds and viewpoints, which is one reason why I like HN over other technical communities.

That said if I was to treat HN comments as a hivemind and generalize I'd say overall HN leans negative on the game industry. There is a lot of truth in these views but people tend to generalize towards "all game programmers are underpaid and over worked and should work on business CRUD apps" and not look at individual cases of motivations, success or failure. This is both stereotyping and anecdotal but I see it in game industry related threads time and time again. I wasn't trying to sound smarter than the community, I'm sorry if I came off that way.

</comment></comment></comment></comment></comment><comment><USER6> Rami Ismail gives a great talk on translation and language barriers in games:

[http://www.gdcvault.com/play/1022362/We-Suck-at- Inclusivity-...](http://www.gdcvault.com/play/1022362/We-Suck-at-Inclusivity- How)

He also does a good job of peeling back the curtain behind indie development and realities of being an independent game developer. We could use a few more people like him in our industry.

</comment><comment><USER5> What a life...

Go where you want, do what you want, buy what you want.

Get payed for making and playing video games.

Sign me up!

<comment><USER7> I don't know if he's still at it. But, a gamedev friend of mine spent at least a couple years living cheap in Bermuda with his wife, working remote four months out of the year and just chillin the rest of the time. He is a wiser man than I.</comment></comment>


Paperspace – A full computer you can access from any web browser - <AUTHOR> https://paperspace.io/

<comment><USER2> > _We care deeply about your privacy. Paperspace is a 'zero-knowledge' company > which means we do not have access to your personal information on your > Paperspace desktop._

i wonder how that works exactly if everything is "in the cloud"</comment>


Google releases Consumers Surveys, shares revenue with publishers - <AUTHOR> http://www.google.com/insights/consumersurveys/home

<comment><AUTHOR> And it's down - they didn't announced the product yet.

Pricing:

* Target a representative sample of the U.S. Internet population. $0.10 per response or $150.00 for 1500 responses (recommended for statistical significance).

* Target a subpopulation using inferred demographic data. $0.50 per response or $750.00 for 1500 responses (recommended for statistical significance).

About:

You create online surveys to gain consumer insight.

 _Which version of my new logo will people like better? How much are dog owners willing to pay for an organic cotton leash? Is my brand awareness growing over time? We all have nagging questions about our own products, companies, and industries. Now it's easy get answers and make major decisions with your consumers' behavior and preferences in mind. Write your own survey questions or customize existing templates. Target the entire US Internet population or specify a custom audience you're after, whether it's 25-34 year olds, coffee drinkers, or pet owners. Pick how many responses you want and your survey is ready to go._

People complete questions in order to access premium content.

 _People browsing the web come across your questions when they try to access premium content like news articles or videos. Opinions are valuable, so answering the question gives them near instant access to the page they want for free. They don't have to pull out a wallet or sign in and you gain insight into what people think. Why one question at a time? Many researchers are used to doing multi-question surveys in which the same respondent is asked to fill out a 10+ minute questionnaire. With Google Consumer Surveys, you can run multi-question surveys by asking people one question at a time. This results in higher response rates (~40% compared with an industry standard of 0.1 - 2%) and more accurate answers. The system will automatically look for correlations between questions and pull out hypotheses._

Publishers get paid as their visitors answer.

 _Questions run across sites in our diverse publisher network in order to get the necessary respondents. Publishers—online news sites, video creators, and app developers—make money as site visitors provide answers. Everyone wins._

(form for publishers: <https://services.google.com/fb/forms/surveysforpublishers/>)

You get nicely aggregated and analyzed data.

 _Google automatically aggregates and analyzes responses, providing the data back to you through a simple online interface. Results appear as they come in, not days or weeks later. Results & insights In addition to raw data, charts summarize responses and insights highlight interesting differences. Using the DoubleClick cookie and the respondent's IP address, Google Consumer Surveys infers demographic and geographic information for each response so you can easily segment by age, gender, location and more. See which results are statistically significant or order additional responses if your initial sample wasn't sufficient. Methodology & accuracy Unlike other online survey platforms which send questionnaires to predetermined "panels," Google Consumer Surveys takes a new approach to survey sampling, data collection and post- stratification weighting. This produces a close approximation to a random sample of the US Internet population and results that are as accurate as probability based panels._

Whitepaper comparing Google's Consumer Surveys with alernative solution: [http://www.google.com/insights/consumersurveys/static/357812...](http://www.google.com/insights/consumersurveys/static/357812512129491172/consumer_surveys_whitepaper.pdf)

</comment><comment><USER2> The website is back online <http://www.google.com/insights/consumersurveys/home>.

Here are some additional information [https://plus.google.com/102034052532213921839/posts/e7ceA3cQ...](https://plus.google.com/102034052532213921839/posts/e7ceA3cQvPD) and some examples:

\- Pop vs. Soda: [http://www.google.com/insights/consumersurveys/view?survey=c...](http://www.google.com/insights/consumersurveys/view?survey=c5x4eptj5iyxw&question=1&filter=&rw=1)

\- Physical Activity: [http://www.google.com/insights/consumersurveys/view?survey=6...](http://www.google.com/insights/consumersurveys/view?survey=6kydc2sf5il5m&question=1&filter=&rw=1)

\- Professional Photography: [http://www.google.com/insights/consumersurveys/view?survey=t...](http://www.google.com/insights/consumersurveys/view?survey=t7mym4dharmiy)</comment>


A private company is selling 2.2B photos and locations of auto license plates - <AUTHOR> http://www.theatlantic.com/politics/archive/2016/01/vigilant-solutions-surveillance/427047/?single_page=true <comment><USER2> Hmm, where have I seen that name before? Oh, yeah.

[http://www.nytimes.com/2016/01/27/nyregion/what-happened- to-...](http://www.nytimes.com/2016/01/27/nyregion/what-happened-to-jane- mayer-when-she-wrote-about-the-koch-brothers.html)

</comment><comment><USER3> maybe we just need an open data version of this. anyone can upload an image of plate with a gps tag.

<comment><USER4> Heck, we could use this: [https://github.com/openalpr/openalpr](https://github.com/openalpr/openalpr)

<comment><USER3> To make money you just pay a $5 to get off the site. Make it a non-profit raise awareness about secret license plate databases.</comment></comment></comment>


Tor calls for help as its supply of bridges falters - <AUTHOR> http://arstechnica.com/information-technology/2013/04/tor-calls-for-help-as-its-supply-of-bridges-falters/

<comment><USER2> > For those who want to donate bridges to the Tor network, the easiest route > is to use Tor Cloud, an Amazon Web Service Elastic Compute Cloud image > created by the Tor Project that allows people to leverage Amazon's free > usage tier to deploy a bridge.

What are the risks involved in doing this? Both in regards to legal responsibility for traffic, and Amazon ToS.

<comment><USER3> As long as you don't operate an exit node, there are no risks that I am aware of (as a citizen of the USA or EU, YMMV otherwise). I've been running several entry nodes with directory caching enabled on Amazon for a year or so with no problems - nor do I expect any.

If you run an exit node on Amazon, you can expect to receive a C&D at some point. I'm not sure if Amazon will intervene. There are some services elsewhere that allow you to pay for the operation of exit nodes in bitcoin.

Tor is essentially just 2 proxies chained together such that the first proxy (the entry node) doesn't know the final destination, the second one (the exit node) doesn't know the source, and the final destination only knows the exit node. If a malicious entity is operating either entry or exit nodes, it still protects your identity as long as you did not expect the exit node to send information that leaks information about yourself. If the same malicious entity or cooperating ones get lucky and operate all of your nodes, then you are unmasked. If you are afraid of this possibility, then you can configure your client to make a circuit of more than 2 nodes.

In order to find out about all of the available public nodes that you can connect to, the client queries a list of hardwired directory servers. Here, you can get this yourself: <http://126.96.36.199/tor/status- vote/current/consensus>

If you run a public server, your information will be distributed in this list. You can also run a private bridge, which is not shared publicly. You will not be used as an exit node unless you configure your server to allow exit traffic. You can specify this by port ranges, so as to only allow certain traffic. You can see this in the above consensus document in lines that start with "p". "p reject 1-65535" means it is not an exit node.

If you want to help in other ways, you can also run a directory cache that serves consensus information (as in the above link).

<comment><USER4> It's not just two proxies chained together - there is always at least one middle node between entry and exit.

<comment><USER3> The common implementation of tor does use 3, but as you can see on page 5 the basic concept functions with 2:

[https://svn.torproject.org/svn/projects/design-paper/tor- des...](https://svn.torproject.org/svn/projects/design-paper/tor-design.pdf)

It helps to simplify when explaining to others.

</comment></comment></comment></comment><comment><USER5> I know this is a little off topic but what in general are the risks of going onto the Tor network?

Like I say, I know it is off topic but I can't find anything conclusive to how risky it is?

I want to see what is on there but at the same time I hear of hackers at every corner and child porn. I don't want to encounter either ever.

Is there a safe way of browsing it whilst taking a casual look around?

<comment><USER6> In addition to what other have already noted, one other thing to keep in mind:

The Tor exit node that happens to handle any particular request gets to read the relevant traffic (since it is handling the request to the actual end server on your behalf). So, for instance, if you're logging into some bulletin board via unencrypted http, the Tor exit node handling the login request gets to see your username and password on that bulletin board in cleartext (as can all the other net infrastructure between that exit node and the bboard itself).

In one sense, this doesn't change your risk profile; if you're logging in over unencrypted http directly, you're also at risk of sniffing. However, the risk may be enhanced with Tor; there are persistent rumors of law enforcement and intelligence agencies (and others with darker-colored hats) running exit nodes which deliberately sniff the traffic they're proxying to see if something interesting comes up. It also might be possible for such a hostile exit node to mess with the content of unencrypted traffic, though I've personally heard no rumors of that.

Note that if you're using Tor to proxy _encrypted_ traffic (https), the exit node sees only the encrypted data stream, which is as secure as the encryption you're using --- and the official Tor browser bundle includes the "HTTPS everywhere" Firefox plugin to try to get you to use HTTPS where available.

Additionally, if you're using Tor, anyone monitoring the net segments between you and the entry nodes you hit may be able to tell that you're using it (though they won't be able to tell what you're using it for). If the local secret (or other) police frown on that sort of thing in principle, it could be trouble.

<comment><USER7> What worries me is that if you want to do government work (even quite innocuous stuff), your job depends on getting security clearance.

Certainly in the UK, Europe general and probably in the US, your ISP retains a list of all the IP addresses you connect to, and they supply this to the government if they have national security grounds.

At least in the UK, security clearance is grounds to pull your internet records, they say so on public government websites.

I think that in future, if your household connects to known TOR bridge nodes, that might well impact if you can do government work, from IT at your local tax office to army work.

Probably paranoia, but I'm staying away from TOR for this reason. Which is a shame because what some ad networks do is really, really creepy.

</comment></comment></comment><comment><USER8> See also: <https://news.ycombinator.com/item?id=5564358>

</comment><comment><USER9> You can also donate to <http://www.torservers.net/about.html>

And sponsor a node. They are based in SF and accept BTC.

They list their exit node servers here: <http://www.torservers.net/exits.html>

</comment><comment><USER10> also <http://crypto.stanford.edu/flashproxy/> (which seems to work - i have seen the colour of the image change on my pages; in fact one is in use right now).

</comment><comment><USER11> In case others are also having trouble finding them, the instructions on how to participate are in the email linked from the article: [https://lists.torproject.org/pipermail/tor- relays/2013-April...](https://lists.torproject.org/pipermail/tor- relays/2013-April/002089.html)

I'm interested, but still trying to learn more. In particular, are there ways to limit the traffic or prioritize other packets?

</comment><comment><USER12> Quickest road to deploy on AWS <https://cloud.torproject.org/#get_started>

(Select Obfs3)

<comment><USER13> This literally took me less than 5 minutes to setup. The defaults are perfect for those with a free AWS setup. Below are the default settings from the included torrc file.

# Start Tor as a bridge.

# Run obfsproxy

# Never send or receive more than 10GB of data per week.

# Running a bridge relay just passes data to and from the Tor network. so it shouldn't expose the operator to abuse complaints.

</comment></comment><comment><USER14> This seems like a fatal flaw for one of Tor's stated use cases: helping political dissidents living with highly repressive governments. How do you publicize a service and at the same time not allow the relevant authorities to discover it? What they're doing seems to make it _harder_ , but in principle it will always be detectable. This would make me think twice if my life depended on my connection not being detected.

</comment><comment><USER15> I tried helping out Tor. I installed their software that was supposed to make it one-click to set up a relay. Didn't work. Then I fumbled around a bit and Googled the error messages I was getting. No luck. Reboot a few times and try again. Nope.

Sorry. They need to make it way easier to get involved.

<comment><USER10> Worked fine for me, but I built from source (I guess that sounds harder; it was actually trivial if you're used to this kind of thing - just the usual ./configure; make; sudo make install) (note that this is for use only as a bridge - if you want to use it yourself, you should use a bundle with browser).

Instructions on setting up a service on OpenSuse at <http://www.acooke.org/cute/StartingTO0.html> (anyone know how to make systemd switch to a different user?)

<comment><USER15> That response is like someone on HN who said it was easy to install third- party stuff on the Windows tablets. All you have to do is simply side-load the stuff like this...

If TOR wants widespread support, they need to create a simple tutorial with a FAQ that covers all the basic problems when you do a one-click install on Windows. If malware can be written to be easily installed and configured, then it shouldn't throw errors when you do a basic install using their Windows package that you need to Google and diagnose. If a software developer gets pissed off at their installers, what will average users think?

<comment><USER10> sorry, wasn't meant as criticism, although i do think tor try quite hard to package things - it's certainly improved over the years. perhaps posting a bug report with your issue is the best way to get them to improve? <https://trac.torproject.org/projects/tor>

<comment><USER15> Didn't take it personally. I support TOR. I just don't like them complaining about network congestion/overloading when they don't do enough (IMO) to make it easy to expand the network.

</comment></comment></comment></comment></comment><comment><USER16> Has Tor tried to do anything to reduce potentially unnecessary use of their bridges? Maybe I'm being naive, but I think some teenager using Tor to pirate stacks of blue-ray movies should have to wait in line behind a Chinese civil rights blogger.

<comment><USER17> Tor is unbearably slow for large downloads, so almost anyone is discouraged from doing so. There's also no reason whatsoever for a teenage pirate to need a bridge server, since their access to the standard servers is unimpeded.

</comment></comment><comment><USER18> I'm signing up as we speak.. it's taking Amazon a while to confirm my subscription

</comment><comment><USER19> Well, the problem is that if bridge is discovered by child pornographers and used by them - police will get knocking to bridge owner's door.

No one needs that, hence the supply is low.

<comment><USER20> IIRC there was a man in (I believe) Austria who recently got arrested for this exact reason and he was trying to scrounge up enough in donations to pay for his legal fees.

Edit: [http://arstechnica.com/tech-policy/2012/11/tor-operator- char...](http://arstechnica.com/tech-policy/2012/11/tor-operator-charged-for- child-porn-transmitted-over-his-servers/)

<comment><USER21> You're missing the architecture:

      +-----+
      | You |
      +-----+
             \ <= encrypted           _
         +</comment></comment></comment><comment></comment><comment>----+            | T
         | Tor Entry Node |            | O
         +</comment><comment></comment><comment>----+            | R
                \ <= encrypted         |
           +</comment><comment></comment><comment></comment><comment></comment><comment>+  | N
           | Tor Bridge/Relay Nodes |  | E
           +</comment><comment></comment><comment></comment><comment></comment><comment>+  | T
                    \  <= encrypted    | W
                +</comment><comment></comment><comment>---+      | O
                | Tor Exit Node |      | R
                +</comment><comment></comment><comment>---+     _| K
        unencrypted => \
                     +</comment><comment>----+
                     | Internet |
                     +</comment><comment>----+

The exit node is the only one that the wider Internet sees. All other traffic within the network is encrypted.

<comment><USER22> It's that first step I don't get. How is the traffic between you and the entry node encrypted? Can't someone monitoring traffic know that you're on Tor (even if they can't know what you're doing on it)?

<comment><USER23> The box labeled "You" is not just a web browser. It should include a local (as in local to the machine running the browser) Tor proxy like Vidalia. See <https://www.torproject.org/projects/torbrowser.html.en> for an example.</comment></comment></comment>


I Have No Idea What "Hard Work" Means - <AUTHOR> https://www.currentaffairs.org/2019/09/i-have-no-idea-what-hard-work-means <comment><USER2> What he means is "I have no idea what _you_ mean when you say 'hard work'". Anyway it is probably not what he might mean, and even he means different things at different times.

The meaning he and we probably find most objectionable is that the speaker is really telling us that he or she has earned the state of privilege they enjoy, and that others who don't didn't. It is true that this is invariably BS -- they never work nearly so long or hard as any janitor you encounter. Working hard on your own behalf is nothing particularly admirable, anyway.

I have a half-sister who, with her husband, re-roofs houses in disaster areas every summer, unpaid. I would be happy to hear about how hard they worked, but they feel no need to tell me.

<comment><USER3> "they never work nearly so long or hard as any janitor you encounter"

I remember thinking when I was cleaning up construction sites that cleaning the inside of a finished office building must be a job you needed connections or experience to get.

<comment><USER2> Should I take this to mean it was light work? Lucky you.

But not the same as a janitor's.

<comment><USER3> I'm not sure what you are referring to as "it".

What is tolerable or enjoyable work depends on what you are accustomed to and other factors. Would you rather clean toilets, deal with an irate customer, be on a rooftop in the summer sun, be on an open platform ten stories in the air, or wade in some unknown sludge? Some people have issues with a different subset of those than others, so there isn't a strict ordering.

My point was that it's odd to me to hold being a janitor up as some kind of extreme end of a spectrum. It's not what you do when all else fails. It's not the thing that people with no skills and no experience are automatically assigned to.

However, if a day labor place sends someone to sweep up a warehouse, I wouldn't call that being a janitor. Maybe some would.

</comment></comment></comment></comment><comment><USER4> Realizing that diligence was a middle class status signaling value that did not apply in capital class circles was an interesting lesson. Elon Musk constantly talks about how you have to work twice as much as other people, where as Jeff Bezos is pretty open about going in for about 6-7 hours a day. Who is right? The answer is either one, they have different values, that hard work is subjective, and that you should just do what you want and what works. Hard work is the original virtue signaling in a growth economy. It says, I'm important, I'm needed, etc. But in practice it often means diligence which is often not correlated to effectiveness in most (not all) fields of knowledge work.

<comment><USER5> Surely a rational person would care about doing a solid day's work while maintaining their health, sanity, relationships etc.

<comment><USER6> Why is that rational? If I go into the office and spend a day looking at cat pictures (which certainly makes me happier than attending the endless meetings) I get paid just as much.

<comment><USER5> I can't imagine this is a good long-term career idea unless it's something you do only on occasion when you're mentally exhausted or something.

<comment><USER6> I think it's just that I believe that in any company big enough to have endless meetings it's fairly possible to put in (almost) no work and still fly under the radar.

<comment><USER3> Only if you can stay awake in the meetings. For me, meetings where I don't participate and nothing concerns me are like a soporific.

</comment></comment></comment></comment></comment></comment><comment><USER7> The author made some interesting arguments, but they lost me with their insistent jabbing at the upper management man cheating on his wife, the racist divorcee dad, the boring date, her friends cheating boyfriends, etc. The not- so-subtle political undertones distract from what I think is an interesting question.

<comment><USER6> I'm confused. How are these things related to political undertones? Prejudice maybe, but are these more common amongst certain groups?

I thought it added some wry smiles to the story anyway.

</comment></comment><comment><USER8> "I think the issue is that when people say they've "worked hard," they're implicitly suggesting superiority"

Yes, you compare it to others who do similar work. Who actually says "I worked hard... in comparison to anyone who has ever worked.". It's usually you work hard for that line of work. A software engineer can be changing links and sipping coffee or pulling 12 hours coding all day. One is doing hard work. Article seems to be arguing semantics.

<comment><USER9> When someone actually says they "worked hard" it's usually bullshit, because people who actually work hard either don't realize it or are too humble to admit it. So saying you worked hard is a justification for why you have something or why you have something and someone else doesn't.

Most people who make a lot of money don't actually work very hard by virtue of the fact that most individual wealth comes from capital, not labor. Maybe you did work hard at some point to get that capital, but that's also unlikely. After all, if capital paid out the true value then it wouldn't exist at all. So somewhere you either exploited people or natural resources to get that capital.

Hard work sucks. It almost by definition doesn't pay well, with few exceptions.

But we need to justify to ourselves and others why we value different humans different, and hard work sounds really good. So we invoke it to justify to ourselves, mostly, why we are superior to others.

<comment><USER10> >people who actually work hard either don't realize it or are too humble to admit it.

This is a completely arbitrary way to gatekeep the concept of "working hard".

<comment><USER9> It's really not. It's just how it is.

<comment><USER11> "it's just how it is" this is devilry, relative-truth. Not absolute.

<comment><USER9> "They don't think it be like it is, but it do."

</comment></comment></comment></comment></comment></comment><comment><USER6> Would it be worse to say that I don't consider myself to have ever worked particularly hard in life?

It comes easily, I'm just smart (enough) and lazy.

Then again, I'm writing this while walking home from work at 8:40pm, so there is some sort of disconnect here.

<comment><USER12> > Would it be worse to say that I don't consider myself to have ever worked > particularly hard in life?

> It comes easily, I'm just smart (enough) and lazy.

I think the key thing that people object to in the "hard work" arguments (which is also present in your "it comes easily, I'm just smart" assertion) is the (perhaps unintentional) implication that one has complete control of the outcomes one experiences. Diminishing the role of beneficial circumstances beyond one's control comes across as saying that people who are less fortunate must deserve it in some sense because there can't possibly be factors beyond their control that contributed to their misfortune.

None of this is to say that the hypothetical people who state the value of their "hard work" _didn't_ work hard, or that you're _not_ smart! It's just important to recognize that there likely were some beneficial factors along the way that if you hadn't encountered, you might be one of the less fortunate people yourself.

<comment><USER6> Oh, for sure. Had I been born somewhere else I'm not sure these traits would have resulted in the same outcome.

I guess people don't feel the need to comment on circumstances beyond their control exactly because they can't do anything about them though. Saying you worked hard is nice, appending "and I'm still only here because I started 2 leagues behind" is not.

Neither is "but I'm here mostly because I was born in the first world to a fairly well off family".

</comment></comment></comment><comment><USER13> For me "hard work" is when I'm doing something I don't enjoy for more time then I agreed to. That's an incredible privilege, but I still feel like shit when it happens.

</comment><comment><USER14> The example of Brett Kavanaugh is curious. Yes, his high school is focused on getting its students into good colleges--that's why it is Georgetown Prep, not Georgetown Something Else. It is also focused on preparing them to get through those colleges creditably, and part of that is teaching them how to manage a workload. The husband of a sometime co-worker taught science at a well- thought-of state university, and said that more kids did poorly from lack of work habits than from lack of smarts.

Does getting your algebra homework done on time count as hard work on the same level as digging ditches? No. Does it count as work of a sort? Yes.

<comment><USER15> You're not in disagreement with the author.

> Any job that requires intense physical labor is on the list of "for sure, > you worked hard," as is anything that involves great emotional and > psychological resilience, such as social work. But after that it gets > tricky. How about someone who founded a trading company or a real estate > agency in the 1980s, riding the wave of the business-friendly Reagan years? > They've worked for decades, undoubtedly with some late nights or unusual > hours. But is that the same as getting black lung? What about someone who > just worked a fairly normal 40 hour week, from age 18 to 65? Is that "hard > work?" What about someone whose job is creative and enjoyable? Is that also > "hard work?" How do we measure this, exactly? Can it be done by number of > hours? Intensity of work? Sacrifices made? Cubits of human suffering?

</comment></comment><comment><USER16> I was thinking about this topic earlier this week. I think it's really misleading to defend your accomplishments with "hard work", especially given not all "hard work" is the same. I "worked hard" but have gotten a fraction for it than many folks I know. I have also "struggled a lot less" than quite a few folks I know while getting better results. It's ultimately a meaningless dodge.

Maybe success shouldn't need to be defended. I'm not sure.

<comment><USER17> You can work hard by "banging your head against the door" every minute. That doesn't mean it was the right work. When you say you worked hard or I said I did, there is no context or absolute scale to see what we meant.

</comment></comment><comment><USER18> This article just seems like someone who is obsessed about how hard other people work. I really don't get it. Why not do a job the best you can handle? Maybe that involves "hard work" however you yourself define it or maybe it doesn't. It really doesn't matter.

</comment><comment><USER19> This is what hard work and success means to me:

To the extent that you are attempting to manifest your potential, and directing your aim such that you are successful in raising the bar through a meta-goal - i.e. attempting to orient yourself through a range set of goals with the overall meta-goal being life itself.

Once you have actually determined that you want things to be better, you can then proceed to direct your actions and judgements appropriately. Perform a regular evaluation period where you can determine whether: a) your goals are in fact what you should be aiming at, b) whether your actions are appropriately working in your favor, and c) whether you are making the correct judgements. This process of evaluation is truth telling in itself.

</comment><comment><USER20> It doesn't matter how hard you worked, what matters is the results you produced.

For example, she talks about not understanding what "hard work" meant in college, but who cares? Did you get the grades you wanted? That's all that matters.

It's the same in the "real world", it doesn't matter how hard you work at something, what matters are the results. Did you get what you wanted?

I don't really understand what is the point of pontificating over the meaning of "hard work". Is this some echo of the protestant worldview?

</comment><comment><USER21> I was agreeing with the author all the way to the end, when she wrote:

"And once we stop playing this pointless game of comparing how hard we all work, we can start asking the real question of when we're going to start being compensated fairly for it."

I don't believe there is such a thing as fair compensation. No one 'deserves' anything, and work has no intrinsic value in and of itself. It's just a market place full of people haggling.

<comment><USER22> > It's just a market place full of people haggling.

then (to me) the question becomes: is this what we really want? and is there any alternative to this that works?

</comment></comment><comment><USER23> If the author worked just a tiny bit harder, he would've written a shorter essay.

I myself am a lazy guy, didn't finish reading it.

I wonder if hard work is zero sum ...

<comment><USER24> Clearly too lazy to figure out the he's a she...

</comment></comment><comment><USER25> I for one enjoy working hard from time to time. Like it back when finishing shrink-wrapped software to ship and at startups when iterating quickly to find product market for. Even in my office job we do have feature releases which are mini-marathons and fun in perhaps a similar way.

</comment><comment><USER26> It's true that I probably don't work any harder than the subsistence farmer in India living on $2 a day. Does that imply I should start donating all my extra income to them? Until what point? Until I'm living on $2 per day as well?

</comment><comment><USER27> The author seems to be saying that people who say they have worked hard deserve their privileges (of wealth or influence or both). I think the author hasn't thought it through enough. Virtually everybody believes they deserve what they have. Usually they believe they deserve more (and often they are angry that they don't have it). It doesn't matter how much they have or what they did to get it.

The rest is just justification. It's often not hard work that's the justification. Does the offspring of a rich parent "deserve" their parent's wealth? It will be a rare potential recipient that doesn't believe this.

Maybe "hard work" is a common justification, but I think it's probably the justification that is common in the author's circle. Saying, "You don't work hard compared to X" is such a terrible argument because the very next thing the person will say is, "But my work is _more valuable_ ". And if you say, "But your work is not as valuable as X" it will be rebuffed with, "But my work is more rare and therefore deserving of greater recognition". Or... You can go on forever.

The point is that people believe that they deserve what they have, even if they have orders of magnitude more than other people. People who are relatively well off still generally believe that they are worth more than what they receive. That's just how people are. You are not ever going to convince the average person otherwise because it is important to their self image.

How many times do you hear people expressing outrage or dismay that "at the very least I should have X", when in fact it's just an arbitrary line in the sand.

I even catch myself thinking that way. I'm a senior software developer with 30 years of experience. I _should_ be able to afford a house and 2 cars and to raise a family with kids and have occasional vacations in exotic places. If I happen to go out to dinner and spend $250 on a single meal once in a while, I _shouldn 't have to_ think about whether I can afford it. _I 'm a senior developer with 30 years of experience_ for goodness sake. The world is pretty freaking broken if I can't have at least that!

Right?

And there hundreds (?)... thousands (?)... Well, I don't know, but probably more than I can count... people who will be looking to find some food tonight. Any food. It doesn't matter what. Probably they can't write computer programs. So I guess they don't deserve the food as much as me.... Is that what I'm saying?

But I'll tell you what. That line of reasoning isn't going to fool me! I refuse to feel bad for what I have, because I worked hard for it. I deserve it.

And that's the whole thing in a nutshell.

<comment><USER28> No, the author is saying the opposite.

> Virtually everybody believes they deserve what they have.

Yes, and they claim that they got there through "hard work", which is totally unrelated to how hard they worked and just short for "I deserve it more than *".

</comment></comment><comment><USER29> narcissistic millennial

<comment><USER28> Prime example of what the article is criticizing.

> I think the issue is that when people say they've "worked hard," they're > implicitly suggesting superiority. I'm deserving of reward, not like those > people who are lazy ("those" people being immigrants, poor people, liberal > arts majors, whoever it is you seek to contrast yourself against).

I think we can add "narcissistic millenial" to "those".

<comment> dawg- Is it possible to be proud of your own hard work, without necessarily making a contrast to others who you perceive as "lazy'? So goes the old saying, "The true test of character is what you do when nobody is watching".

Sure there are plenty of people who wear their "hard work" as a badge. But that isn't everbody.

<comment><USER28> Sure, the article starts with those and then focuses on the "plenty of people".

The person I was replying to however is clearly among those making a contrast.</comment></comment></comment></comment>


Being Bilignual May Boost Your Brain Power - <AUTHOR> http://www.npr.org/2011/04/04/135043787/being-bilingual-may-boost-your-brain-power

<comment><USER2> I have always wondered if knowing programming languages counts as being bilingual or whether most languages are just an obscure subset of english? Do non native english speakers find they have to know/learn english to program?</comment>


California Tests Kindle-Ized License Plates - <AUTHOR> https://spectrum.ieee.org/view-from-the-valley/transportation/infrastructure/california-tests-kindleized-license-plates <comment><USER2> Can you disable the car tracking?</comment>


Darcs 2.10.0 release - <AUTHOR> http://lists.osuosl.org/pipermail/darcs-users/2015-April/027119.html

<comment><USER2> I love Darcs. And I really miss it now that I am effectively forced to use git in order to take advantage of GitHub. Not that git is bad, it is technically very sound. Its just that usability-wise Darcs wins hands down.

<comment><USER3> Don't forget Darcs Hub ([http://hub.darcs.net](http://hub.darcs.net)).</comment></comment>


Ask HN: How do top thread headlines get retitled? - <AUTHOR> This current top headline on HN, "Ranking the Most Beloved TV Shows That Got Canceled," said something about Firefly in it's title not more than 2 hours ago, but now it's been renamed. How does that happen? <comment><USER2> A non-official answer ...

It happens when the moderators decide that the title does not accurately reflect the content, or that it is otherwise somehow better served by being something different. Sometimes the change is contentious, sometimes the change is undeniably better.

There is also a short (but variable) window during which the submitter can edit a title, but that's unlikely to be the case in this instance, because to the best of my knowledge the window is at most 2 hours.

<comment><USER3> Yes, a moderator changed the title after it was noted by a user that the submitted title was misleading [https://news.ycombinator.com/item?id=12617046](https://news.ycombinator.com/item?id=12617046)

<comment><AUTHOR> Thanks guys!</comment></comment></comment>


Alibaba's DingTalk gets bad grades from China's stuck-at-home students - <AUTHOR> https://www.reuters.com/article/uk-china-health-alibaba-idUKKBN20B14D <comment><AUTHOR> Also, DingTalk made a video on Bilibili, begging its users to give higher grades (in Chinese): [https://www.bilibili.com/video/av89441613](https://www.bilibili.com/video/av89441613)

The video has got 10M plays in five days.</comment>


Logging in large mathematical models - <AUTHOR> https://www.ahl.com/logging-in-large-mathematical-models <comment><USER2> For my master thesis, I implemented a new and fancy algorithm. The code seemed to be fine and dandy for the usual, simple test cases. After using more elaborate test cases, I found cases that didn't work well.

After contacting the author, who indicated he didn't have such problems, I literally spent months trying to debug the code. When I finally gave up and re-wrote my implementation basically from scratch, and found the same problems, I contacted the author again. He then indicated that indeed there was a problem with the method for these cases, that he understood the problem and found a way to fix it. In hindsight, the problem was not hard to understand (but still, the claims in the paper were unwarranted IMO).

Conclusion? I wish I was a math prodigy, then I would have spotted the problem instantly. Also, be wary of claims made in papers.

<comment><USER3> Most papers are wrong. Not necessarily wrong in an essential way, but most of them overstate their claims.

There's a rule of thumb in the machine learning community: if you want an algorithm that does X well, find a paper on doing X, and implement the method that the paper _compares its proposal against_. That way you're near state of the art, but you're also using a method that many people have used and tested.

<comment><USER4> This is new to me, but I think it's fantastic advice!

</comment></comment></comment><comment><USER5> I know it's a bit of a tangent, but proactive, large-scale logging of models like this (such as those used in machine learning) may become desirable to meet the requirements of GDPR. If you have to be able to explain how an algorithm made a decision, you need to be able to pull up data like this somehow.

<comment><USER6> Do you actually believe people in authority have the ability to understand what's at stake in this?

If you do, you need more life experience.

<comment><USER5> I am a pragmatist looking at ways to work within the law, not an idealist looking to change the law, so it doesn't matter what I believe about people with power. They make decisions and we work out solutions to work within or around them.

And, yes, we all need more life experience :-)

</comment></comment></comment><comment><USER7> A while back I was working on a system doing fairly complex engineering calculations and I implemented detailed logging of both the values used and the actual calculations performed.

This allowed me to be able to generate a spreadsheet (with the values and calculations in place) that could show a non-developer exactly how the outputs had been calculated (you could use the features of Excel to add visual annotations of precedents and dependencies).

I was pretty pleased with that approach.

</comment><comment><USER8> If you want IMMENSELY powerful logging, take a look at how the trace logging of Racket's Medic Debugger works, an absolutely ingenious solution:

[https://docs.racket-lang.org/medic/index.html](https://docs.racket- lang.org/medic/index.html)

Highly interestingly, albeit a bit off topic, the authors of the paper from Medic originated very recently took this technique, and cranked it to 11:

[https://conf.researchr.org/event/sle-2017/sle-2017-papers- de...](https://conf.researchr.org/event/sle-2017/sle-2017-papers-debugging- with-domain-specific-events)

…which won them a distinguished paper award!

<comment><USER9> Thanks for the link - it looks great.

</comment></comment><comment><USER10> This looks to me like a standard logging toolchain where you just have programmatic access to the logs.

This is like claiming you save and load a json object (instead of its serialization) in some hashmap / DB for fast lookup.

Am I missing something?

<comment><USER11> this might seem normal but perhaps not in mathematics? There's the story about homotopy type theory that first had to be invented including a supporting programming language to formalize and automatically error check proofs, which is still a niche topic in maths (because it seems faster to think barely, I gather. Edit: and because of the problem of bootstrapping such a system and Not Invented Here syndrome).

</comment></comment><comment><USER12> That's a great approach to logging/debugging complex models on large datasets.

I'm pretty sure this can be applied outside the math, e.g. on systems with complex business rule over large datasets.

</comment><comment><USER13> I implemented a library that does exactly this -- [https://github.com/IGITUGraz/SimRecorder](https://github.com/IGITUGraz/SimRecorder) (In case anyone finds it useful). It supports storing data in both hdf5 and redis (although I wouldn't recommend redis for storing large numpy arrays)

</comment><comment><USER14> What's the advantage of the file system/repo/bespoke diag database over storing the numpy arrays in the existing database infrastructure?

Doesn't implementing this system with HDF5 cause headaches for concurrency in either direction?

<comment><USER15> I don't know the author's use-case specifically, but that could fall over in two ways:

1) if you've got models that are re-generated periodically based on new inputs/algorithm tweaks, then you can potentially end up with quite a few of these as you scale.

2) if you want to track the details that/debug the reason your production system made a given decision, you need to log not just your model but all of the parameters that went into that decision. If that type of decision happens many times a day, then you can end up with some pretty massive logs to go through.

In either case, storing that historical data in your transactional database can be a bit of a load, so it's ideal to keep it separate if you get any kind of volume. I've actually bumped into 2) at one job.</comment></comment>