Volkswagen's U.S. head: individuals engineered emissions cheating - <AUTHOR> http://www.reuters.com/article/2015/10/08/volkswagen-emissions-congress-update-1-p-idUSL1N1281B720151008 <comment><USER5> Nope, definitely systemic within the company. The engine is a system that must meet certain requirements. The standard way of meeting those requirements is to have a urea injection system. That's what the competitors in the US do; it's also what Subaru, for example, don't do, and they don't sell into the US market.

In order to pull off this little 2-person hack that Horn alleges, those couple of software engineers would have to have a major say over the entire engine design. I call bullshit. This little hack ONLY makes sense in the absence of other major engine components.

The company culture is clearly absolutely rotten and their execs are liars. Employee morale, especially in engineering, must be through the floor.

<comment><USER6> Actually, the Passat has urea injection and is implicated as well -- though it didn't exceed emissions by as much as cars without aftertreatment.

And the "couple of engineers" was in reference to the head of engineering and another guy. It wasn't implicating rogue low-level engineers.

[http://www.businessinsider.com/this-is-the-real-cause-of- the...](http://www.businessinsider.com/this-is-the-real-cause-of-the-vw- cheating-scandal-2015-10?op=1)

<comment><USER7> The current scandal is only about the "EA189" engine family with NOx traps for exhaust treatment. The new "EA288" engine family with urea injection was designed from scratch, and, I hope, doesn't need cheating software.

Edit: the EA288 is/was -- at least in Europe -- also sold with NOx trap, but VW still claims it's free from cheating software.

<comment><USER8> Considering all we've learned in this saga about the industry in general and Volkswagen in particular, I would say it's more than likely that it had cheating software anyway.

<comment><USER6> I think this is part of the problem. The EA189 without AdBlue needed the cheating software. The Passats with AdBlue also had the EA189 and likely could have been tuned to work without the cheating software, but likely have it anyway.

It's a bit of a shame that EA288 cars are stacking up at dealers nationwide and will apparently never be certified for sale (VW said they would not ask for certification), even though they almost certainly don't need the cheat. (Although it's possible that it's still present, which is of course still illegal, need it or not).

</comment></comment></comment></comment></comment><comment><USER9> "This was a couple of software engineers who put this in for whatever reason," \- I'm not buying it. What would be their motive compared say to VW's motive? VW has a heck of a lot more motive for falsifying emissions than do "a couple of software engineers." Even if you were to argue the engineers were somehow fearful for their jobs that's still VW's problem because they're the ones who created that work environment instilling that fear.

I really expected VW to do better in handling this. Some business analysts are saying VW may be out of business within five years time. Based off this testimony I say good riddance.

<comment><USER10> Also, it implies a couple of engineers could slip through arbitrary code to be executed under selective conditions. That's terrifying.

<comment><USER11> And, supposing this is true, would you actually be surprised by this? This is a car company, not a CI and code-review obsessed software company in Silicon Valley.

<comment><USER12> I found products made by car companies to be in general more reliable than those made by software companies (either in Silicon Valley or not). I trust my life to my car (software included) every day. I wouldn't do that with any software of my smartphone, not even the OS.

<comment><USER13> Doesn't it blow your mind that there are no "standards of engineering" or whatever for software? There's no licensure body for software engineers who build software running your car, and therefore no accountability on a personal level.

When an engineer builds a bridge, she has to personally sign off on the bridge, saying it's safe, and is risking not only her professional career, but I think she can also be jailed and held criminally liable if the bridge kills people due to negligence.

It blows my mind, at least, that no such thing exists for software.

<comment><USER14> Does any one really thinks that true?

[http://www.ldra.com/en/software-quality-test- tools/group/by-...](http://www.ldra.com/en/software-quality-test- tools/group/by-industry-standard/automotive-iso-26262-misra)

[https://en.wikipedia.org/wiki/Motor_Industry_Software_Reliab...](https://en.wikipedia.org/wiki/Motor_Industry_Software_Reliability_Association)

[https://en.wikipedia.org/wiki/MISRA_C](https://en.wikipedia.org/wiki/MISRA_C)

[https://en.wikipedia.org/wiki/ISO_26262](https://en.wikipedia.org/wiki/ISO_26262)

<comment><USER13> None of those are what I described, though.

<comment><USER14> > Doesn't it blow your mind that there are no "standards of engineering" or > whatever for software?

ISO, IEC, etc.

> There's no licensure body for software engineers who build software running > your car, and therefore no accountability on a personal level.

MISRA, SCSC, etc.

<comment><USER13> Combine my individual sentences, as I did, and try again.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER15> It's an utterly brazen lie to say the company didn't know about this.

Anyone who's ever built anything as part of a team - whether the Linux kernel, a space shuttle, or a Lego set - can tell you design decisions like this cannot be secreted into the final product.

So let's say a software guy decides to make this fix. How do you do that?

\- You'll have a version control system, won't you? At the very least someone will see it. There's some interesting if statements referencing the speed of the rear wheels, which you can explain at code review.

\- How do you test it? I doubt your normal office happens to contain the entire running setup of a diesel engine, with nobody around to notice, and nobody needed to tell you about the details. Because you know, an engine test rig wouldn't have details, right? You just turn it on yourself, spin the car up to speed and debug, right? And I'm sure there's nothing complicated about gas measurement, even though it tends not to be a part of CS courses.

\- What do you tell the cluelessly naive good guy engineer about his results? Would he really only test the car under test conditions? Does any engineer with even the slightest grain of curiosity test only the one mode that he's supposed to?

<comment><USER16> I guess they are worried about significant legal action against the top-brass and are throwing off some chaff. Will the legal system go after the little guy again or land the big fish?

<comment><USER15> Well it's cases like this that cause people to win/lose respect for the system.

If they throw some engineer under the bus, I don't want to live on this planet anymore.

</comment></comment></comment><comment><USER17> How is Volkswagen's US head, who lives and works in the US, able to issue proclamations with such certainty about the non-existence of private conversations and directions taking place among the engineering staff and management in Germany, where he spends almost no time?

It is as if a US diplomat stationed in New Zealand used his authority to declare that Bush had never authorized any torture, and it was rogue individual interrogators who did it. How would he know given he was on the other side of the world when the conversations would have taken place.

<comment><USER18> I would assume he is speaking on behalf of Volkswagen, not testifying as an individual on his own experiences. Volkswagen's current story is that it was a couple rogue engineers, so he is conveying that. Congress isn't going to get the CEO in to testify, so it's reasonable to assume he is speaking as a proxy.

<comment><USER19> When speaking under oath take great care to not exceed the domain of verifiable facts.

<comment><USER20> As long as you don't enter the domain of falsifiable facts, you should be fine.

</comment></comment></comment></comment><comment><USER21> This pass the buck attitude really pisses me off. You're the head of the company, department, whatever, it's your gawd damn responsibility if your subordinates do something wrong. If you hired bad people it's still your responsibility. If you hired people that hired people it's still your responsibility. I don't care if you knew or not. You set the tone, you set the impossible deadlines or requirements. You fostered a culture where someone thought that would be okay to do. Own up!

<comment><USER22> My favorite manager ever told me when I took my first management job that "you can delegate work but not responsibility"

<comment><USER23> He was very old or simply lying. Very first thing University of Washington Center for Information Assurance and Cybersecurity (accredited by U.S. Department of Homeland Security, whatever that means) teaches you about becoming a CIO is precisely delegating responsibility :)

<comment><USER22> Thanks - I understand the healthcare website debacle now better. :)

</comment></comment></comment></comment><comment><USER24> If you think this hasn't been endemic in this industry, let me take you back to 1998: 'Car manufacturers can use modem electronic equipment to adapt the engine to any type of test cycle. They can even tell the computer of the car how to recognise when the car is being driven according to a specific test- cycle and adjust the combustion accordingly. It was this kind of software that six truck manufacturers, including two European firms (Volvo and Renault), recently used in the United States to defeat the EPA's pollution control. As a result emissions of nitrogen oxides from highway driving increased by 300 per cent. " extract from: 'Cycle Beating and the EU test cycle for cars' \- Per Kageson - European Federation for Transport and the Environment - <USER2> 1998

<comment><USER24> Which further begs the question: if I could dig this up with literally 60 seconds of Googling form vaguely recalling having heard something like this before, how come this hasn't been brought up by any of the 'journalists' on the case?

</comment></comment><comment> artumi-richard I just today listened to the BBC Bottom Line Special [1] on this, and their guests, one who used to run Jaguar, and an expert on VW, don't buy it. Their argument is that the engineers in VW could have done a proper job, but the technology would add another $500 to each vehicle, and VW must have been looking for a way not to spend the $500. And all the other VW engineers would ask "How did you do it?".

[1] [http://www.bbc.co.uk/programmes/b06jt518](http://www.bbc.co.uk/programmes/b06jt518) 9mins 15 onward

<comment><USER25> They actually did have an answer to "How did you do it?"

The idea was that only DEP and EGR was needed for the smaller TDIs simply because they burned less fuel so produced less emissions across the board. That's not enough to meet current limits, but VW claimed they had better CDI that solved it. They got many patents related to that and other companies even licensed it. But how did they get such good performance, the lines was due to the turbo and the tests did not really spool that all that much.

Turns-out it was baloney, but that's how they explained it at least.

</comment></comment><comment><USER26> "This was a couple of software engineers who put this in for whatever reason," Horn said

Except when obviously true, companies that make public statements like this will have difficulty recruiting the best engineers.

</comment><comment><USER27> Uh, right. I don't think so.

I'm sure it went like this:

"Now, I'm not telling you to do anything illegal, but if you don't pass those emissions standards on the next deadline, you're fired."

At each level down the chain.

I actually saw this in action at a big company subcontracting division to two younger employees (one male and one female): "We need that contract. Period. Your expense account on this unlimited and unaudited. Make sure you go pull enough cash that you don't have to put anything embarrassing to the company on the card. I'm not telling you to do anything illegal, but if you don't close that contract, you don't have a job Monday. Anyhow, make sure the client has a good time this weekend. See you Monday. Maybe."

They closed the contract. No one asked what it took. I left that job post haste.

</comment><comment><USER28> "This was a couple of software engineers who put this in for whatever reason"

This was a system. A complex one. No way it was done by a couple of guys at 2AM, or whatever. This has to have been inter-group collusion, and management is in CYA mode and shoving individuals under the bus.

VW has gone from being a cravenly stupid company that is going to be fined billions of dollars, to a place where I suggest it is in every engineer's best interest to resign, and the earliest opportunity they can manage.

<comment><USER29> Say the detection for the emissions testing is something like a flag in the software that says the car is on a dyno, or some other factors that the ECU has legitimate reasons to keep track of.

In that case, the defeat system is just a quick if else predicated on those state variables and swapping between a set of engine parameters that is emissions test safe and another set that provides better performance.

I don't mean to insist that this must be the case, I'm just trying to illustrate that the cheat could happen at a level where much of the complexity has already been abstracted away.

<comment><USER28> Okay, imagine you're working on the hardware of the engine. Pistons, bolts, lubrication. One Friday afternoon you leave work depressed because the emissions suck, they've sucked for months, and they won't get better without a major redesign.

On Monday morning, the emissions tests are suddenly passing.

"Ha, just one of those things," you say, "I thought we had emissions problems, but I guess we don't after all. Let's ship the engine and have a party!"

There is no way that this happened.

<comment><USER29> I'm not arguing that it was a few people working in secret, just that it would be possible for the changes to be done at a high level (in the engine system), without pulling in all the people that were involved in developing the vehicle (or even the powertrain).

</comment></comment></comment></comment><comment> a-dub That's like blaming an illegal addition to your house on rogue contractors. It's such a poorly conceived lie that it is actually insulting.

<comment><USER30> "On tonight's episode of Punked: watch as we add a second floor and backyard porch to this man's bungalow while he's away for the weekend! You won't believe his reaction!"

</comment></comment><comment><USER31> What a classic terrible management move. I worked at a ski shop in college that also sold (really bad) used ski equipment on eBay. One time the lead manager accidentally let a lot of stock go live with a $1 minimum. They realized it and went back in under different accounts and bid them all up. When eBay caught them, they blamed the employees for it (myself and 4 others), who all had nothing to do with it.

We all quit the next day. Who wants to work for someone like that?

</comment><comment><USER32> Do we think VW are the only car manufacturer doing this? I very much doubt it. They're just the first to get caught.

Where rules stand in the way of profit, they get bent or broken. Banking, automotive industry... which next? Oil? Housing? Pfff ... come on, we all know they're rotten, it's just a question of how rotten and whether it comes out.

I suspect it's the tip of the iceberg for the auto industry. Whether we give a damn, and whether we can or will do anything about it remains to be seen.

<comment><USER33> Why are so many people keen on defending VW with the claim "if VW are doing it, everyone else must be"? That dilutes the severity of this problem. I've seen several threads on HN meant to do this.

<comment><USER34> > That dilutes the severity of this problem.

No, it actually increases the severity of this problem. One car manufacturer cheating emissions vs. most car manufacturers cheating emissions.

</comment></comment></comment><comment><USER35> Mental note, never buy a VW car again. Cheating on emissions is bad, but passing the buck down to the rank and file is deplorable.

<comment><USER36> Don't forget Audi, Porche, Bentley, and others that VW owns :)

<comment><USER37> Yeah, remember to not buy that Lamborghini as well.

</comment></comment></comment><comment><USER38> Expect VW US to play dumb until something out of Germany is officially done. VW wont go out of business, but they are 10 years from growth in the US and probably most other markets. I think the real fallout will come if the prestige brands are made to look bad. Audi and Porsche have much more to lose, and those are the profit centers for VW.

</comment><comment><USER39> I heard (total hearsay) from a friend in Detroit that GM spent millions of dollars over several years trying to replicate VW's diesel technology. They couldn't make the numbers work. Now we know why.

</comment><comment><USER40> By the way Mr Horn is head of VW Group of America, a subsidiary which besides operating a factory in the U.S. is more or less a distributor of cars for VW and related brands in the U.S. As other people have said Horn doesn't know what's going on at VW AG.

Also, judging from an earlier article it appears it's common to refer to various higher-ups on the product teams as Engineers even though they are quite old and were clearly acting in a managerial or even executive capacity. When Horn says "software engineer" he's making a leap and it's the wrong leap.

Horn is making these statements after being pressed by members of Congress, he's not volunteering these misleading statements.

</comment><comment><USER41> How could some engineers cheat the entire VW corporation?

Germans are known for their ridiculously meticulous standards and testing. VW should have many in-house and external-independent labs performing tests.

The engineers who write the ECU code have a great distance from the mechanics who test every engine and car batch from their production line.

How did they never tested and engine entirely; power, emissions, consumption?

You may fool the regulators but you can't fool your own mechanics.

<comment><USER42> This is even more unreal given their ad campaign "Truth in Engineering".

<comment><USER43> The engineering was solid, it worked as intended, in contrast to the software of other car companies.

</comment></comment></comment><comment><USER44> Volkswagen just ensured software developers would avoid them like the plague.

Why write code for a company that won't protect you from the mistakes of upper management?

</comment><comment><USER45> "For whatever reason" indeed. They must have just flipped a coin on the issue. Really, it's just bad luck at work here. /s

<comment><USER46> I'll add an /s

"I'm sure there was no pressure on individual performers to make the results look favorable for VW."

<comment><USER47> Internet sarcastic comments are much better when people don't add /s at the end.

</comment></comment></comment><comment><USER19> He may come to regret testifying under oath with these particular sentences, which are very much falsifiable. I'm not sure what the penalty is for lying to congress but this could very well backfire. There is absolutely no way that anybody within VW is able to put limits on the scope of the fall-out of this whole affair at this point in time.

Not smart to still be in 'damage control' mode when the only way out is total transparency.

I'll be very surprised if there won't be a mass resignation of engineers working for VW for being thrown under the bus like this.

The mere suggestion that 'a couple of engineers and their managers' could get away with deception on this scale speaks volumes about VWs perceived internal lack of process - something they shouldn't be able to get away with - <USER3> their lack of documentation for the development track of the engine.

If they've shredded documentation or if they suddenly find their version control systems and email audit trail blank there just might be jail time for some execs after all.

The thing that bugs me is that they are still somehow in lala land about how they're going to cover this thing up, digging a deeper hole in the process. You'd say the VW brand has sustained enough damage but clearly the execs don't agree with that.

"for whatever reason"... quote of the day, as if engineers put stuff in production without reasons, specifications or instructions.

</comment><comment><USER48> _OT:_

I think VW gets a bit of an unfair beating. Yes, your car probably looses in value, but

\- the cars could be upgraded (which VW seems to be willing to do for free)<USER3>
\- no one was physically hurt (not like the General Motors ignition that actually killed drivers).

Also when seeing the top 3 bestselling cars in the US being sheer gas guzzlers (Ford F, Chevrolet Silverado and the Dodge Ram), the word 'hypocrisy' seems like a fair description to the public reaction in the US.

 _That being said:_

Companies being social constructs and thus complex in their decision making process, framing this scandal as an orchestrated scheme by VW's complete management is ridiculous. What impresses me is, that this Scandal was in the making for months and they did not do anything in advance. Somehow, to me this proves the statement, that they weren't involved in this.

VW's _old_ leadership was notorious for being hands-on on even slightest car details. Specifically Ferdinand Piech - who was nicknamed "Fugen-Ferdi" because of him insisting that VW car's "Spaltmaße" (clearance?) be the industry's best - is very unlikely to not have known about this.

IMO some did know, and choose to not inform those that didn't know - not even when it was clear to them what this would cause to the company.

(edited for formatting)

</comment><comment><USER49> In my country Audi says the cars using the engine are "technically safe to use and drive". I did report that to the consumer oversight board. I believe it's false marketing, the cars are a danger to pedestrians and cyclists outside the car.

I believe it's lies upon lies from the upper management seems like a corporate culture going wrong.

<comment><USER50> There's clear evidence that many cities throughout Europe have too high air pollution, and that it kills close to what smoking does. Cities have been ordered by the EU to clean up by the end of the year, or face harsh fines. Over 80,000 people die in the UK per year (smoking kills 100,000). Mostly from diesel engine pollution.

Now try to think how many people it kills in the rest of Europe, the USA and the world?

Cars (and buses) stink, are noisy, and worse -- they kill on a massive scale. There is alternative tech available for order now that doesn't kill as much, and provides better service.

Demand that the killing machines are replaced, and the makers, and the users are brought to justice if they don't!

[http://www.theguardian.com/environment/2015/jul/15/nearly-95...](http://www.theguardian.com/environment/2015/jul/15/nearly-9500-people- die-each-year-in-london-because-of-air-pollution-study)

<comment><USER51> That's evidence of _pollution in general_ being a problem, which isn't even in the same ballpark as establishing culpability for any amount of deaths for an extra amount of one kind of pollutant by one carmaker over a fixed time period.

<comment><USER50> See what was published just days before the VW scandal in Nature. [http://www.nature.com/nature/journal/v525/n7569/full/nature1...](http://www.nature.com/nature/journal/v525/n7569/full/nature15371.html)

<comment><USER51> Correct me if I'm wrong, but that article talks about particulate emissions, something which even the cheating VWs don't have a problem with - <USER4> datapoint that led to this whole thing coming to light were high NOx emissions, not particulates.

</comment></comment></comment></comment></comment><comment><USER52> These are worrying times for VW if the new CEO to say something like this. Many VW engineers will be considering leaving if VW is going find a couple of engineers as scapegoats, and not be back them up.

<comment><USER53> Presumably there are a couple of engineers who deserve some of the blame, but wow, what better way to say "Our culture is poison". Both because they're throwing independent contributors under the bus, and because he's saying the culture is such that the engineers would think it's OK.

To be fair, I think he might not have meant it to come out that way.

EDIT: Looking again at the article, it seems like the US CEO isn't even aware of this: "The German automaker has suspended 10 senior managers, including three top engineers, as part of its internal investigation." So VW doesn't necessarily have a poisonous culture. But they do have a US CEO who doesn't prepare when he testifies to Congress.

<comment><USER52> Unfortunately, we may never know the truth in the scenario. The way that automotive world works with the multi-tier supplier chain, anyone who is not close to the source, will never know where the real blame is.

Behind the scenes in VW's HQ in Wolfsburg, it must be chaos.

</comment></comment></comment><comment><USER54> "This was a couple of software engineers who put this in for whatever reason"

In other words, it's the programmers, the "geeks," the "code monkeys" that should take the fall for all of this and not the non-technical, MBA-holding managers to whom they answered and who almost certainly enjoy better compensation and more respect.

The perils of our profession's relatively low prestige are on display here for all to see.

<comment><USER55> _and not the non-technical, MBA-holding managers_

Not that it changes your overall point to VW/VAG is a pretty big on having engineers in senior management. Both their former and current CEO have a serious technical background (metallurgy and Computer engineering respectively). As does their chairman of the board.

</comment></comment><comment><USER56> Well, is there any surprise they've chosen a couple of fall guys to blame it on?

</comment><comment><USER57> Let's assume what he says is true. Then the managers of those engineers are incompetent and their managers also. If you hire people who do this sort of thing, and you don't even notice, then the problem is yours. There's no scapegoating here.

Still assuming this is true: If a few rogue individual engineers can do this, imagine what else could be lurking in that codebase.

</comment><comment><USER58> Meanwhile here in the US we let people mod their diesel engines and nothing is done about it. [https://en.wikipedia.org/wiki/Rolling_coal](https://en.wikipedia.org/wiki/Rolling_coal)

</comment><comment><USER59> I'm assuming the development process for embedded auto software like this would be pretty rigorous, meaning it would be fairly difficult to sneak in code like this.

I also assume that testing of emissions software like this would involve a lot of lab/road testing with the vehicles against the EPA test profile and therefore a lot of people would be involved. Suggesting this is the stealthy work of just a couple of software engineers is very hard to believe.

Can anyone who's worked on embedded software in the automotive environment comment?

<comment><USER60> The investigations regarding Toyotas(?) unexplained/unintended acceleration suggest otherwise. The code was apparently pretty bad, halfheartedly reviewed and just unmaintainable. While there the software seemed to have (a huge number of) errors put in by mistake, of course most independent reviews should also spot intentional mischief.

I don't claim that VW had comparably crappy code, but "industry practice" likely wouldn't have caught it either way.

<comment><USER61> Here, this link will help:

[http://www.safetyresearch.net/blog/articles/toyota- unintende...](http://www.safetyresearch.net/blog/articles/toyota-unintended- acceleration-and-big-bowl-%E2%80%9Cspaghetti%E2%80%9D-code)

<comment><USER60> Thanks for adding this!

</comment></comment></comment></comment><comment><USER62> I've always wondered how something like this plays out legally: Can the engineers be somehow held accountable for writing something they were forced / asked to by their managers?

<comment><USER63> Yes, they can. Most engineers (in the US) have to take an ethics course that covers this. Just because your boss says do something, doesn't give you a pass.

<comment><USER64> Do you have a source for ethics education and engineering? I've never heard of this being something that most engineers are taught (in the US).

<comment><USER65> My CS program contains a 1 credit hour course on ethics in computing. I believe this is required as part of the ABET accreditation process: [http://www.abet.org/accreditation/accreditation- criteria/cri...](http://www.abet.org/accreditation/accreditation- criteria/criteria-for-accrediting-computing-programs-2015-2016/)

</comment></comment></comment></comment><comment><USER66> Even if that was the case (which I doubt), it's the executives' fault for creating a culture which fosters cheating. It's their fault for not being aware of what's going on in their own company. They probably have a culture of "Don't ask, don't tell".

These execs get paid huge salaries/bonuses for being responsible for stuff. But when things go wrong, the first thing they do is shift the blame back down to their workers.

</comment><comment><USER67> The "engineers" that these stories refer to are kind of like saying the CTO is an engineer. These weren't rank and file employees.

</comment><comment><USER68> Related side question. A few months before this leaked out I bought a brand new Golf TDI(diesel) and I'm in California where it sounds like my car is technically now illegal?

I have no idea what I should be doing at this point. I've been significantly overpaying on the note but not sure I should now. Does anyone have an idea what this means for the owners of the vehicles in question?

<comment><USER69> Your car is still legal to drive. edit: Source is the apology letter I got from VW a couple days ago.

</comment></comment><comment><USER70> What a total fucking asshole.

<comment><USER37> Seriously, well put.

</comment></comment><comment><USER71> If what he is saying is true (which is about as likely as me getting pregnant), that would be far, far worst than cheating for a test: he is saying two people can put a chemical storage, undetected, and release it according to a complex plan, without any tester, QA, manager finding about it.

If that blatant lie even makes sense to him, he should not be let near an anything industrial.

</comment><comment><USER72> Yeah, an intern did this in his spare 20% time.

[http://www.independent.co.uk/news/world/europe/volkswagen- em...](http://www.independent.co.uk/news/world/europe/volkswagen-emissions- scandal-vw-was-warned-four-years-ago-its-emissions-test-software-was- illegal-a6669376.html)

#enron #pinto #oilspill

</comment><comment><USER73> It's quite possible that the "company" didn't actually know about it because they built a culture where people know what when to break rules and when not to ask.

It's not all that different of a culture than a government agency that doesn't need to be told to do things that benefit the party of the current regime.

</comment><comment><USER74> I actually suggested this theory a few weeks ago: [https://www.linkedin.com/pulse/volkswagen-different- theory-w...](https://www.linkedin.com/pulse/volkswagen-different-theory-what- happened-marc-hoag)

<comment><USER75> I don't understand. Why would software guys fear the results of some chemistry-based process? They don't seem to have any control over that.

The ones fired would have been the mechanical engineers that designed the engine, aren't they?

<comment><USER76> You know how those software guys are always trying to impress people with the performance of their software...

</comment></comment></comment><comment><USER77> Let me guess - was this a 2-person mistake, too?

[http://www.mirror.co.uk/news/uk-news/major-security-flaw- lea...](http://www.mirror.co.uk/news/uk-news/major-security-flaw- leaves-100-6595166)

</comment><comment><USER78> An earlier article said that the two who engineered the cheating were the chief engineer and the guy in charge of engines. So whether they engineered it or not, they (and the CEO who has already been fired) are responsible; that's their job.

</comment><comment><USER79> OK, Volkswagen AG, release the source control log for the software. Let's see exactly who put in that code, and the change log for it. I'd really like to see the unit tests for the defeat code.

</comment><comment><USER80> It's a good thing that there are stiff penalties for perjury during congressional hearings. Few examples just during this past year, when people lied under oath, got caught, and... Oh, never you mind.

</comment><comment><USER81> > _" This was a couple of software engineers who put this in for whatever > reason,"_

I'm not a lawyer, but establishing motive is like the most important and basic thing to establish, right?

</comment><comment><USER82> It is a crime that Top Gear is no longer on the air during these times.

</comment><comment><USER23> couple of software engineers, just like a couple of low ranking employes decided to do the _same_ thing at Peugeot, Renault, Citroen, Volvo and Ford!

[http://www.focus.de/auto/ratgeber/kosten/unregelmaessigkeite...](http://www.focus.de/auto/ratgeber/kosten/unregelmaessigkeiten- nicht-nur-bei-vw-abgas-rueckfuehrung-mehrere-stunden-nicht-aktiv-vorwuerfe- auch-gegen-ford-und-peugeot_id_4998888.html)

</comment><comment><USER83> Funny how this coming down to software vs "hardware" departments.

Except not so funny as scapegoats will lose their jobs and benefits.

No way software people did this in a bubble.

</comment><comment><USER84> Yes, this was the work of individuals. Hundreds, if not thousands of individuals that did it with the blessing of some reasonably high up people.

</comment><comment><USER85> "Software Engineering department will get bonuses depending on emission results!" \- legal way to push department to cheat.

</comment><comment><USER86> But who merged pull request to master?

</comment><comment><USER87> I just heard this on the radio - "it was down to a few software engineers". What nonsense.

</comment><comment><USER88> Bob: Hey guys, we didn't pass emissions tests.

Jay: I think we can adjust the ECU parameters.

Bob: Oh, nice. let's try that.

Mechanic James: By doing this you guys lowered the lifetime of a few components. We might need to re-engineer the whole thing :(

Bob: What if we detect we are undergoing tests and tweak it? This way we still can deliver the project on time.

Some manager: Yeah, do it.

Now Bob and Jay are in trouble.......

</comment><comment><USER89> Fraudvergnügen: It's what makes a car a Volkswagen.

</comment><comment><USER90> Does not matter. The people at top are responsible.

<comment><USER6> He's not at the top of the corporation, just of the US operations. He might well not have known.

</comment></comment><comment><USER91> Even if this is true, and I doubt that it is, it's not really that important. The real question is why people in positions of responsibility didn't prevent it from happening.

</comment><comment><USER92> See, this is why I can never believe anyone when they try to justify executive level compensation. They always say it's cause the executives are "taking risk" and "responsible for the company." And when things are going well, they're quick to take credit. But when stuff like this goes on, suddenly the CEO still gets his golden parachute, and none of the remaining executives know what's going on.

</comment><comment><USER42> This ridiculous story is not going to hold and they know it so watch for higher ups trying to divest. I mean Bosch told them it would be illegal to use the way they were intending. I surely hope someone is watching who at the company is selling shares.

</comment><comment><USER93> Haha!!

</comment><comment><USER94> ...and monkeys might fly out of my butt. :-D

</comment><comment><USER95> How predictable. Personally I am hesitant to buy anything ever from them.

</comment><comment><USER96> Lets throw our own people under the bus and see what kind of juicy leaks come out!</comment>


Cryptographers Urge People to Abandon IOTA After Leaked Emails - <AUTHOR> https://spectrum.ieee.org/tech-talk/computing/networks/cryptographers-urge-users-and-researchers-to-abandon-iota-after-leaked-emails <comment><USER3> The crypto space to me is really frustrating. I've seen friends become something very close to scammers, I can't have interesting technical discussions about the topic without being called doomsayer and even when I pitch an idea what to use the Blockchain for, all I get is:"let's do an ICO and raise a shitton of money!" although that doesn't really make sense towards what I'm proposing.

I feel like the nerd in class that hopes that the teacher (regulatory bodies) will pinch those bullies so we can get back to topic. I hate that feeling.

<comment><USER4> I had a conversation with a guy creating a crypto startup recently. It only took five minutes in before the thin veneer of an actual business model fell away and the real motive—free money—starting coming out.

His position, distilled: if people are stupid enough to give me money, who am I to say no?

I suggested his easy money might not be so easy if (for US investors) the SEC, FTC, CFTC, etc. crack down and claw back investor money. Maybe that's fine if you've become hugely profitable in that time, but if you haven't...

<comment><USER5> I agree with your friend. Dumb people will give away their money anyway. Might as well give it to me. What's wrong with free money as long as I'm not doing illegal stuff.

<comment><USER6> Morals, ethics?

<comment><USER7> If investors are giving money based on lies, there's a legal system in place to help them recover failed fraudulent investments. I'd wager in many cases investors know exactly what they're investing in. So I don't see any ethical/moral conflicts in a lot of these BS cryptocurrency schemes, just a lot of wasted potential and a somewhat shameful divergence from the idealistic core tech - private, trustless transactions through a decentralized blockchain (bitcoin).

<comment><USER6> Stunned by that comment. Stunned.

<comment><USER7> I almost didn't write it and knew I'd be downvoted. Many folk that hang around HN are very idealistic myself included. My main point is that you shouldn't feel sorry for cryptocurrency investors; many know exactly what they're gambling on. Easy money and BS "tech innovation" being used to grab portions of that cash pile is the reality I see. What's your opinion of the mkt?

<comment><USER8> > My main point is that you shouldn't feel sorry for _cryptocurrency > investors_ ; many know exactly what they're gambling on.

This category is broader than you imagine and now includes unsophisticated individuals who can't tell the difference between the blockchain and a bike chain. Lately, I've been hearing a lot of "invest in bitcoin" talk from people who are not well-informed, and do _not_ know what they are gambling on. What's illegal for penny stocks should be equally illegal for cryptocurrency - <USER2> Feds should crack the whip.

<comment><USER7> It's not hard to understand the high risk, high reward investment principle. There are plenty of safe index funds, blue chip stocks, and bonds available to retail investors/unsophisticated individuals. It's not difficult to understand that cryptocurrencies are _highly_ speculative. I would never advise my parents to buy bitcoin, ethereum, or any other cryptocurrency without adding the caveat that they be prepared to lose everything in an exchange hack or some other shady behavior.

I can understand the need for the SEC to protect investors when it comes to investment into traditional companies - regular people can't read a 10k and don't sit in on analyst calls that are mandated for public companies registered with the SEC. But if you're investing in an unregulated, highly risky asset, where you don't understand any of the assets or technology the company owns...come on man.

<comment><USER8> > But if you're investing in an _unregulated_ , highly risky asset,

A lot of people are about to find out that cryptocurrencies are, in fact, regulated. It's weird to see some techies agree with the idea that "pedestrian thing, _on a computer_ " patents are invalid (because the "on a computer" bit is irrelevant), but when it comes to "fraud/misrepresentation _on a computer_ ", then it suddenly needs to be legally special-cased as somehow different. Intangible/abstracted assets predate cryptocurrencies - there's nothing new about crypto in the eyes of the law.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> 58 days ago I posted a bunch of evidence about why IOTA didn't make sense: [https://news.ycombinator.com/item?id=16039802](https://news.ycombinator.com/item?id=16039802)

I think Bruce Schneier's take was particularly prescient: "In 2017, leaving your crypto algorithm vulnerable to differential cryptanalysis is a rookie mistake. It says that no one of any calibre analyzed their system, and that the odds that their fix makes the system secure is low."

The question is, though, is this going to make a difference to the people playing? Very little about the crypto space makes any sense. It's in a lot of ways just a database but harder -- so much harder -- because the community insists nobody can be trusted. In reality there are very few situations in which you really can't trust one other entity.

<comment><USER10> I'm beginning to feel that any type of gentle academic evidence of flaws in a given scheme or technology in the crypto currency space is blasted as "FUD", so

Here's a though exercise: if all the cryptocurrency communities call out research as paid shilling FUD, are we morally obliged to exploit weaknesses and attempt to demonstrate flaws with actual real collateral damage?

<comment><USER11> That's the world today - ignore facts that don't agree with your reconceived views and deny deny deny.

The problem with exploiting vulnerabilities for personal gain is that you're hurting more innocent investors, too. It's kinda like carpet bombing, in that you'll accomplish your goal of killing the enemy, but you're taking a lot of other people with them because they have a spurious link.

OTOH, maybe you can gain enough influence to make changes that would marginalize or punish the corrupt operators. A lot of these quasi-cryptos have a hidden layer to prevent this (Ripple, IOTA, etc.). Their coins are more like unregulated stock in the founder's company than a decentralized machine. I'm not sure if that makes them more or less susceptible to a single, strong outside force. They could simply delete your coins, but without knowing the exploit that's just temporary. You would have an actual target (other than 'the system'), which you could pressure (aka blackmail), but they could cash out immediately and walk away.

If I was running a scamcoin (and a terrible person) I'd probably bring the exploit finder into the inner circle to gain positive media (with the associated price bump), wait for the first major act of the new person and then cash out, step down, and disclose the vulnerability. Spin it as the new person trying to cover up problems they caused or improperly addressing the original problem and preventing a good solution.

To counter that, the exploit finder could condition their involvement on releasing the hidden layer's source as a matter of transparency, but really be working on a way to fork without the hidden layer or replace it with a democratized layer. Now you can claim the founders were greedy and didn't want to fix the exploit, and that your working solution was rejected because it took away their ability to manipulate the market. Turn it into an announcement of the new 'fixed' coin.

It'd be a fun movie plot, but probably terribly boring to watch.

</comment></comment></comment><comment><USER12> As an aside, is there anyone else extremely disappointed with the quality of discussion over at /r/cryptocurrency? This conflict over IOTA has been unfolding for a long time and the guys on that subreddit defend it fanatically, accusing DCI of "FUD"ing IOTA. I am not sure if the subscriber base is technically illiterate or users that hold a given coin have a strong incentive to dismiss any criticism.

As someone fascinated by the technology in the crypto space but very skeptical about the real-world usefulness of many of these projects, I wish I had a better forum to read beside the odd HN post.

<comment><USER13> > This conflict over IOTA has been unfolding for a long time and the guys on > that subreddit defend it fanatically, accusing DCI of "FUD"ing IOTA

It's not just IOTA. It's more or less any of the top 10 / 20 (by MarketCap) Crypto-currencies on the respective sub-reddits. It's especially bad on r/Ripple where anything critical of Ripple / XRP is instantly deleted by the mods in the name of F.U.D (Fear, Uncertainity, Doubt), and even a small +ve news is posted several times in a day, even trials of Ripple that doesn't even use XRP for the transactions.

And then there was this self-proclaimed shill who described in great detail how he made tons of money shilling various coins on reddit. [https://www.reddit.com/r/CryptoCurrency/comments/7xkm0z/i_wa...](https://www.reddit.com/r/CryptoCurrency/comments/7xkm0z/i_was_a_paid_cryptocurrency_shill_iam_here_to/)

From what I've observed on r/Cryptocurrency and related coin subs on reddit over the past month or two, Reddit is being carefully manipulated by whales and scam coin creators to attract bag holders and manipulate the markets for a quick profit. And many of the mods engage in circle-jerk postings, maybe they are in on it too. Who knows.

<comment><USER14> There may be people in on it, but I wouldn't be surprised if it's overwhelmingly just people defending their position. I've seen the exact behaviour I see at r/cryptocurrency in video games. No Man's Sky pre and post launch was exactly the same. Sensible people casting doubt on what it is you actually do with the game, being shouted down by those who are convinced it's going to be the greatest thing ever. Then lots of defensiveness against all the post-launch signals that it's a mess.

<comment><USER15> People who have bought into something will become more entrenched in their views because doing otherwise will cause a "loss of face." It's understandable really, nobody wants to admit to being duped.

<comment><USER16> The desire to remain consistent with one's previously established positions is one of the great psychological motivators:

[https://www.amazon.com/Influence-Psychology-Persuasion- Busin...](https://www.amazon.com/Influence-Psychology-Persuasion-Business- Essentials-ebook/dp/B002BD2UUC)

People will often act against their own best interests to do so.

</comment></comment></comment></comment></comment><comment><USER17> After +1 hour reading things start to be surreal.

IOTA team swapped their in-house hash algorithm "Curl" for their new in-house hash algorithm "Kerl" in [https://github.com/iotaledger/iri/commit/539e413352a77b1db20...](https://github.com/iotaledger/iri/commit/539e413352a77b1db2042f46887e41d558f575e5) , while at the same day blogging about it and claiming their new in-house hash algorithm is actually SHA-3: [https://blog.iota.org/upgrades- updates-d12145e381eb](https://blog.iota.org/upgrades-updates-d12145e381eb)

> "Therefore we have made the simple decision to temporarily switch Curl with > Keccak (SHA-3) for cryptographic signing in IOTA."

</comment><comment><USER18> The amount of brigading by IOTA community also shows how toxic it is. Just look at the comments on the ieee article. Or the some of the tweets. They were also spamming the /r/MIT subreddit about the DCI.

Just watching this all unfold is making me realize how toxic parts of the cryptocurrency community is and it's making me really sad.

<comment><USER19> It's a wild west, we need a winter of sorts to destroy most of these projects. If IOTA isn't secure it should fall sooner than later.

Hopefully what exists in several years time will be good technology, and hopefully the scams have gone away.

I'd wager than marvels like Bitcoin and Ethereum will still be here.

</comment></comment><comment><USER20> > The researchers disclosed to IOTA that the hash function they were using, > which was an in-house concotion called Curl, was broken.

Classy. Not only do they keep their cryptocoin proprietary, they did use a proprietary hash function too.

I wanted to predict the inevitable fall of the currency but realized that the enterprise might just turn into a bank. If they are the gatekeepers to every transaction, they are already sort of a bank. An unregulated bank. With their own bank notes.

<comment><USER21> Is there any (valid) reason why someone would want to use their own hash function? Secure hash functions, as far as I am aware, are pretty established and widely available and really easy to use..

<comment><USER22> No. There is no valid reason to do so.

<comment><USER23> if you understand the Math and agree that whatever you do needs to go through a proper peer review then why not. Saying "nobody may research the subject because you'll fail" is bad science. If you work in this space, know your Math, and don't rely on your _invention_ to be put into production this year then why not study the problem space and innovate by following the established best practices!?

Problems arise when someone thinks they should now turn this invention into a money-cow or label it "proprietary magic" (under the pretext of protecting IP/copyright).

<comment><USER24> _> if you understand the Math and agree that whatever you do needs to go through a proper peer review then why not._

Because crypto is really hard and you can't prove your new algorithm is secure. Only after being a popular high value target for years can you have much trust in your new hash. Even SHA2 might have flaws nobody knows about - that's the main purpose behind having SHA3 standardised - as a backup.¹

If you are not the designer of the algorithm it's even worse, because an exploit could be put in intentionally by proper choice of constants which are in some cases undetectable.

[1] [https://security.stackexchange.com/questions/152360/should-w...](https://security.stackexchange.com/questions/152360/should- we-be-using-sha3-2017)

</comment></comment></comment></comment></comment><comment><USER25> It still baffles me that the official response from the IOTA foundation is that the vulnerability was inserted intentionally as copy-protection.

And even more baffling that their flagship partner Bosch does not seem to have problem with this practice.

<comment><USER22> I doubt that any of the decision makers at Bosch have enough theoretical knowledge to be able to form any sort of opinion about it.

<comment><USER26> Well, as IOTA targets embedded devices (I guess the reason Bosch is interested) they should have enough knowledge to form an opinion about transaction sizes:

 _" transactions in IOTA are 10KB (in contrast, Bitcoin transactions are on average 600B)"_ [1]

[https://medium.com/@neha/cryptographic-vulnerabilities-in- io...](https://medium.com/@neha/cryptographic-vulnerabilities-in- iota-9a6a9ddc4367)

</comment></comment></comment><comment><USER27> I Abandoned IOTA months ago after looking at their Github issues page. [https://github.com/iotaledger/wallet/issues](https://github.com/iotaledger/wallet/issues)

Great idea, just a little too early for massive adoption.

<comment><USER28> I abandoned it after hearing about the Curl debacle, and _trying_ to abandon it (and selling all of my $100 worth of IOTA) just validated my decision. The wallet couldn't show the right balance unless I "reattached" a bunch of times, and, get this: In order to validate a monetary transaction, the wallet _transmit zero-amount "validation" transactions that refer back to the original one_, essentially validating itself. Such secure.

</comment></comment><comment><USER29> I appreciate how the example collision (page 18 in the email dump) uses the lyrics of "Push it to the limit" by David Hasselhoff. The number 9 seems to be appearing a lot both in the collision raw text and previously when they refer to the bleed. I guess it has something to do with the function being based on trinary logic - could there be another flaw there related to this?

Also kind of amusing seeing the different sides of the IOTA team: Sergey is antagonistic and difficult to communicate while David tries to smooth things out and get everyone to play along nicely.

<comment><USER30> My favorite part was the IOTA folks insisting that their 'higher level' protocol would fix all their rookie mistakes. (Yes! We intentionally built our castle on quicksand! But look how hard the walls are!)

It's a great example of why you need to start from sound foundations.

<comment><USER26> My favorite part is that they don't seem to understand the concept of hash collisions _at all!_ ( _" ONE collision, nah that's nothing"_ [1]) and when JP Aumasson explains a possible attack the response is basically _" Who would do such a thing?"_ [2]

[1] [https://twitter.com/c___f___b/status/967511451442302976](https://twitter.com/c___f___b/status/967511451442302976)

[2] [https://twitter.com/c___f___b/status/966315009071607809](https://twitter.com/c___f___b/status/966315009071607809)

</comment></comment></comment><comment><USER31> Irrespective of whether IOTA has issues or not their constant hostility is surprising to say the least. If you run something in public domain there are going to be people coming after you. And people need to learn to adapt which IOTA is sorely lacking.

<comment><USER32> unless you can frame the narrative as the 'world vs us'. it works for musk fans.

</comment></comment><comment><USER33> Interesting that after much bad news on IOTA, the market is still pricing IOTA well [https://www.coingecko.com/en/price_charts/iota/usd](https://www.coingecko.com/en/price_charts/iota/usd) market is ignorant, uninformed, or its all none total speculative

</comment><comment><USER34> I'm baffled as to why an obviously intelligent bunch of people take to Twitter to discuss this.

<comment><USER35> To have an audience. They are using Twitter as a platform for a public debate in lieu of an alternative.

</comment></comment><comment><USER36> Everything about this project is weird. I started looking into it since I do IoT stuff and there was so much hype.

Then you find out about the curl issue. The unnecessary ternary. Read the white paper and all issues with tip selection was "solved" with hand waving. Then you find out it doesn't even have a client that works on a typical IoT device... Reading the email chain made it ultimately clear: these people have no idea what they're doing. They just threw a bunch of random ultra cool sounding tech together ("post-quantum crypto", of course) and started hyping, while the code never worked. Obvious scam.

</comment><comment><USER37> Sergey appears to have attitude issues and the IOTA team have serious damage control issues, but the MIT team also, maybe wisely, appear obstructive at certain important points. In paticular the probability of attack in the wild seems a valid question that was never dealt with by the MIT team? (Disclosure: I'm not invested in IOTA one bit, nor is anyone I know)

What I witnessed after reading through all those letters is a common clash I see over and over of two types of cryptanalysts: Those whose knowledge grew in resource expansive environments (internet/cpu space) and those whose knowledge grew in resource constrictive space (e.g. satellite broadcasting). In particular they clash on the idea that there can ever be a valid use case for a less-than perfect security schema, so-as-to provide certain desired resource benefits, while still being sufficiently secure for a specific use case.

Cryptographers dance around who has perfect security at heart, never getting down to the issue: that they disagree on necessary trade-offs. By the time they do get to it, one will disengage entirely from conversation and assume the other is just stupid. That has happened here (both sides I think). It's a common outcome.

(Disclosures: I worked as a software+hardware reverse engineer, not cryptographer, in the satellite TV space. So I'm not anything close to an authority on the topic but was around these debates often.)

</comment><comment><USER38> Still holding @ 2.23

Dealing with and accepting loss is actually quite difficult....

</comment><comment><USER39> Get your pitchforks out on crypto. Wrong side of history you are on.

</comment><comment><USER40> IOTA co-founder, Sergey Ivancheglo aka Come-from-Beyond, has an open invitation for cryptographers to discuss this issue on his blog:

"Intermediate summary of Heilman et al. claims about the security of a previous version of IOTA signature scheme"

[https://medium.com/@comefrombeyond/intermediate-summary- of-h...](https://medium.com/@comefrombeyond/intermediate-summary-of-heilman- et-al-add87228efab)

<comment> TD-Linux What is there even to discuss? The disclosures and email tell me everything I wanted to know.

<comment><USER40> Where is the code which exploits the supposed vulnerability in IOTA?

After all, there are code samples available today which demonstrate the SPECTRE and MELTDOWN attacks.

<comment><USER41> There are explanations on how it works here [[https://archive.is/6imWR](https://archive.is/6imWR)] [[http://www.tangleblog.com/wp- content/uploads/2018/02/letters...](http://www.tangleblog.com/wp- content/uploads/2018/02/letters.pdf)]

You don't need code to prove that a vulnerability exists, it is sufficient, especially for crypto primitives like hash functions or cipher rounds, that there is a mathematical vulnerability that can be potentially exploited.

</comment></comment></comment></comment><comment><USER42> Somehow I doubt kind of people buying IOTA are going to abandon it just because the core team is incompetent.

And to be honest they may be right. Because crypto offers instant liquidity, unlike in venture the people who are going to make the most money aren't necessarily going to be the best at picking the winners.

</comment><comment><USER43> IOTA has seemed questionable for quite a while, and I'd also urge people to stay away from it, but is it just me or is Green's conduct here pretty out of line?

He says:

> _If you want a postcard summary of why you should avoid the Iota project — > with your brains and your money — this conversation is it._

but if you read the conversation he's referencing, he doesn't come off as reasonable in it either.

<comment><USER44> I am a researcher in cryptography who just read the full email transcript posted at [1]. I found Ethan and Neha's email responses to be patient and sincere.

In contrast, the IOTA team members Sergey and David have no idea what they are talking about. The IOTA constructions were broken, and instead of understanding that, they made bogus points and tried to attack the four DCI researchers in various ways.

In that context, I think Green's tweet represented a bit of frustration at the way public rhetoric is being misused.

[1] [http://www.tangleblog.com/wp- content/uploads/2018/02/letters...](http://www.tangleblog.com/wp- content/uploads/2018/02/letters.pdf)

<comment><USER19> It doesn't surprise me that IOTA followers are toxic considering the founders are toxic individuals. I find it interesting how IOTA founders are the complete opposite of Satoshi and Vitalik.

<comment><USER45> I am an avid follower of the current crypto space and I cannot grasp how unprofessional and immature the IOTA team handles their issues. It's almost as if 75% of the crypto space as of late is ran by children/teenagers/adults which cannot grasp the basic concepts of being civil to one another. This is all very unfortunate for the overall growth and perceptions of the crypto community as a whole

<comment><USER32> that's what happens to people when they feel like they're competing in a closing-off market. want to see a similar level of people clawing at each other's throats, being petty & antagonistic, competing for scraps? look no further than the music industry, especially the amateur scenes. more beef than an industrial cattle farm.</comment></comment></comment></comment></comment>


Some Asians College Strategy: Don't Check Asian - <AUTHOR> http://www.usatoday.com/news/education/story/2011-12-03/asian-students-college-applications/51620236/1?csp=24&kjnd=tSURpFGpLNi2TpNlfTs1DEBRSJeYGpvQ7L50hafjqV0H5j7x0hJ8u2lsGHlRrIjp-39d76dfb-63b2-449a-8f45-31c410aa4bda_2atXzGZ8X7dD1wSudvLP0vNfw+Ow1NXzDRZ2nyEJexCKymb9vz2NcayCmAClS/Ha

<comment><USER2> This wire service story has already been submitted to HN:

<http://news.ycombinator.com/item?id=3309108></comment>


ServiceStack: new REST API for .NET/Mono - <AUTHOR> https://github.com/ServiceStack/ServiceStack/wiki/New-Api

<comment><USER2> Can anyone recommend a good setup for Mono? In my tests, mono-fascgi + nginx has been extremely slow at handling requests. Wanted to give this a spin some time ago, as well as ASP.NET MVC, but I couldn't get decent performance on linux.

<comment><AUTHOR> We outline the different ServiceStack + Linux deployment options over at:

[http://stackoverflow.com/questions/12188356/what-is-the- best...](http://stackoverflow.com/questions/12188356/what-is-the-best-way-to- run-servicestack-on-linux-mono)

Some instructions on how to run ServiceStack as a self-hosted daemon (or behind an Nginx reverse proxy):

[https://github.com/ServiceStack/ServiceStack/wiki/Run- Servic...](https://github.com/ServiceStack/ServiceStack/wiki/Run-ServiceStack- as-a-daemon-on-Linux)

All of ServiceStack live demos runs on an Ubuntu vServer under Nginx/MonoFastCGI including the Razor Rockstars live demos:

<http://razor.servicestack.net> \- ASP.NET Hosted live demo powered by Linux / Nginx / MonoFastCGI

<http://razor-console.servicestack.net> \- Self-Hosted Console App hosted behind Nginx Reverse Proxy

<comment><AUTHOR> @7Digital also discuss their ServiceStack on Linux deployment strategies using Nginx / FastCGI Mono + Capistrano which they describe in detail at:

[http://blogs.7digital.com/dev/2012/09/25/atomic-mono- deploym...](http://blogs.7digital.com/dev/2012/09/25/atomic-mono-deployment- with-capistrano-and-nginx-under-debian/)

They also discuss their experiences with ServiceStack, and how they've customized it to their liking: [http://blogs.7digital.com/dev/2012/09/20/lessons-learnt- whil...](http://blogs.7digital.com/dev/2012/09/20/lessons-learnt-whilst-using- servicestack-on-mono-and-debian/)

</comment></comment></comment><comment><USER3> This looks really cool...

I'd like to add that I've used various parts of the ServiceStack suite of products before and I've always found them to be very easy to use, and reliable. Also, the maintainer was always good about responding to bug requests and following up about them.

Glad to see it's still moving forward.

</comment><comment><USER4> Can someone amend this post to include the project name?

</comment><comment><USER5> If I'm right, the ServiceStack creator is now working for StackOverflow. Do they somehow use this framework and/or play a role in supporting it?

I think the framework is good as it is, but knowing about big names backing it would give me more faith going forward.

<comment><AUTHOR> Yes I've been working for StackOverflow for over a year...

We use ServiceStack to handle all of StackOverflow Careers BackOffice web services. StackOverflow uses ServiceStack's JSON Serializer (.NET's fastest) for all its JSON serialization needs: [http://blog.stackoverflow.com/2012/02/stack-exchange-open- so...](http://blog.stackoverflow.com/2012/02/stack-exchange-open-source- projects/)

Whilst StackOverflow Careers makes uses the __entire __ServiceStack's technology suite:

      - ServiceStack Web Service Framework to handle all BackOffice services
      - ServiceStack's JSON serializer
      - ServiceStack's C# Redis Client
      - ServiceStack OrmLite (along with Linq2Sql + Dapper)
      - Bundler: Our node.exe based - for all compilation and minifiying of less, css, js assets.
      - Built-in MQ support in ServiceStack which lets us re-use existing Web Services in a MQ Host

The Redis MQ Support is heavily utilized, where all offline tasks and sync services go through, you can read more about it at: [https://github.com/ServiceStack/ServiceStack/wiki/Messaging-...](https://github.com/ServiceStack/ServiceStack/wiki/Messaging- and-redis)

Whilst I go into a bit more about our technical setup at: [http://stackoverflow.com/questions/12537521/host- application...](http://stackoverflow.com/questions/12537521/host-application- server-in-windows-service-or-iis)

StackExchange is very supportive considering any enhancements I need to do to make ServiceStack work better for our usage at work is done on Company time, i.e. The newer Redis MQ Server implementation to support parallel processing of MQ messages was mostly done at work. As well as any bug fixes, regressions we run into.

</comment></comment><comment><USER6> I've been using ServiceStack + AngularJS in a project for a couple of months now, and it's been the bees knees. I'm excited to try out the new API style.

</comment><comment><USER7> How does this compare to the ASP.NET 4 Web API?

<comment><AUTHOR> They hold different philosophies for building web services (and general approach to C# development), I go into this in more detail at:

[http://stackoverflow.com/questions/9699083/servicestack- vs-a...](http://stackoverflow.com/questions/9699083/servicestack-vs-asp-net- web-api)

<comment><USER8> As someone who has been knee deep in WCF for years and now diving into WebAPI I still don't understand the real differences between ServiceStack and WebAPI. It seems like this answer was written well before WebAPI was released or fully baked. Now that it's out is there anything new you can tell us?

So far, I've been able to achieve everything I need using just WebAPI and Attribute Routing[1]. Is there something fundamentally different you can demonstrate or describe for us? I did some research on ServiceStack and the only thing that stuck was that you also had your own serialization libraries too. (Although I still prefer JSON.Net) I guess what I'm looking for are the _Can Dos_ and the _Can't Dos_ too. Not just 'we like this approach instead'.

[1] <https://github.com/mccalltd/AttributeRouting>

Edit - Reworded.

<comment><AUTHOR> Check out the: Advantages of message-based web services [https://github.com/ServiceStack/ServiceStack/wiki/Advantages...](https://github.com/ServiceStack/ServiceStack/wiki/Advantages- of-message-based-web-services)

Which explains the natural benefits of the design ServiceStack promotes (no other MS fx promotes this). We strongly believe that message-based services promote the best practices for remote service development, something we've benefitted from for years.

ServiceStack is different in that it can support the same service running on SOAP, MQ endpoints (in addition to JSON, XML, JSV, CSV out-of-the-box) it even can be re-used as the controller for HTML web pages, see: <http://razor.servicestack.net>

<comment><USER8> Thanks! I'll do some deeper investigation today for sure.

</comment></comment></comment></comment></comment><comment><USER9> Wonder if this would work with F#

<comment><AUTHOR> Sure, some F# + ServiceStack love :)

F# Web Services on any platform in and out of a web server! \- <http://www.servicestack.net/mythz_blog/?p=785>

Async, Cached Twitter API Proxy in F# \- <http://www.servicestack.net/mythz_blog/?p=811>

C# + F# Self Hosting Examples \- [https://github.com/ServiceStack/ServiceStack/wiki/Self- hosti...](https://github.com/ServiceStack/ServiceStack/wiki/Self-hosting)</comment></comment>


Images of Saturn from Cassini - <AUTHOR> https://www.nytimes.com/interactive/2017/09/14/science/cassini-saturn-images.html <comment><USER2> Just amazing. Imagine working on this project (or any project) 20 years ago and seeing the payoff now. Wow.</comment>


Apple Stops Online Sales in Russia Over Ruble Fluctuations - <AUTHOR> http://www.bloomberg.com/news/2014-12-16/apple-stops-online-sales-in-russia-over-ruble-fluctuations.html <comment><USER2> When I was younger, I took part in one of the first post-cold war student exchanges between the U.S. and Russia. A few days after I arrived there (during a very hot summer in the Eastern Urals), the Ruble (the version they were using at the time) collapsed from something like 1500:$1 to 5500:$1 and it became impossible to convert currency.

It finally "stabilized" somewhere around 5500-6000:$1 but the exchange markets still wouldn't function. Finally, running out of currency, our hosts did the exchange with their personal funds, hoping that the dollars we were giving them would turn out to be a good investment later. I didn't really understand at the time what a huge risk they were taking, and how much money, relatively, they were converting for us.

It's one of many generous things that happened during that exchange that's given me a perpetual fondness for the Russian people, even if sometimes I share in many of the grim feelings for the national politics.

<comment><USER3> "Our hosts did the exchange with their personal funds, hoping that the dollars we were giving them would turn out to be a good investment later."

Sorry, but I got lost reading this. Mind explaining it a little more thoroughly?

Your Russian hosts gave you Rubles for your dollars right?

<comment><USER2> Yes.

In Moscow we could use Dollars more or less at most places, but deeper into the country it was harder for the Russians to use dollars so the local economy was Ruble only.

If the Ruble continued to fall, and someday the exchange markets reopened, they could buy more Rubles with the dollars they exchanged with us. This is important because while the exchange rate continued to worsen, their income stayed relatively the same. If they made (pulling a number out of thin air) $10,000 a year, or R15m at the start of our trip, they were only making $2,800 a year at the end even though the number of Rubles they made stayed the same.

If the Ruble continued to fall, say to 8,000:$1 they'd end up making less than $1,900 a year. However, if they bought USD from us at 5500:1 it meant that if it fell, they could buy back 8,000 Rubles for each dollar later, keeping them closer to $2,800 exchangeable dollars.

If the Ruble reverted back to the original rate (1500:$1) then they'd lose their shirts because they just gave us their currency at less than 1/3 value.

Just as importantly, if they were making about $10,000/year, exchanging $400 or whatever with an American kid meant that half their monthly salary was locked up in nonexchangeable USD. The USD had value, but it was non-liquid at that moment.

IIR they weren't able to exchange the dollars until later that year and there were caps on the amount that could be exchanged at once. The next year we repeated the exchange program and I believe some of those dollars we exchanged with them were brought with them to use here.

A few years later, in 1998 the Ruble was reformed on a 1,000:1 basis.

Things were devaluing so fast that people still had Kopek coinage sitting in coin jars that were unspendable and stacks of low denomination currency (1 Ruble to 500 Rubles) that while spendable, were incredibly inconvenient.

<comment><USER4> Now your hosts were nice but they were not completely altruistic either.

Despite often being illegal, Soviet Union had a history of people holding a portion of their savings in foreign currency(just look up Bulgakov's Master and Margarita or Ilf and Petrov's Golden Calf for early references).

So if your local currency is in a free fall you would be well advised to convert your local roubles into hard currency.

I rememember very well the day that my father exchanged 10,000 roubles for $1,000 in the late 1980s from an American tourist. The official rate was ~1.3 dollars for 1 rouble (rouble was officially more valuable than dollar!) at the time, but the catch was you could only exchange a few hundred at a time in very special circumstances. (I believe this is how it is in Venezuela right now too, other places with currency controls).

So your hosts were making a nice gesture but also being smart in times of uncertainty because historical precedent was that their roubles would become worthless not come up.

</comment></comment></comment></comment><comment><USER5> It is interesting to watch the effect of the Russian ruble collapse on Bitcoin markets... It is causing an influx of Bitcoin buyers on Russian exchanges: over the last 14 days, BTC is up +11% from 19000 RUB to 21000 RUB with most of the uptick happening in the last 24 hours ([https://bitcoinwisdom.com/markets/btce/btcrur](https://bitcoinwisdom.com/markets/btce/btcrur)) despite being down -13% from 380 to 330 USD on American markets ([https://bitcoinwisdom.com/markets/bitstamp/btcusd](https://bitcoinwisdom.com/markets/bitstamp/btcusd)). There is an opportunity for arbitrage! The ruble is collapsing so fast that even buying at 21000 RUB is a steal (worth 310 USD as of Dec 16 21:00 UTC). And the collapse will likely continue in the next few days and weeks, so the arbitrage opportunity will grow bigger.

Edit: @potench preev.com is an inaccurate indicator of the BTC/RUB rate because it is basing its calculation "based on the USD Bitcoin price" per its footnote, so of course it will never show a big deviation from the BTC/USD rate. For the true rate you need to look at the real-time bid/ask spread on a BTC/RUB exchange such as btc-e.com which is why I linked to [https://bitcoinwisdom.com/markets/btce/btcrur](https://bitcoinwisdom.com/markets/btce/btcrur) or you can also look at [https://btc-e.com/exchange/btc_rur](https://btc-e.com/exchange/btc_rur): bid/ask is 20855/21099 RUB as of 22:09 UTC, so the arbitrage opportunity is approximately 2000 RUB.

<comment><USER6> Bitcoin, however, was still a worse investment over the Ruble for all of 2014 (So far.)

[http://qz.com/312598/bitcoin-is-the-worst-investment- of-2014...](http://qz.com/312598/bitcoin-is-the-worst-investment-of-2014/)

<comment><USER7> Barely! The ruble is down 50% YTD and Bitcoin is down 54%. But also consider that in 2013 the Ruble went down 6% and Bitcoin went up 800%.

<comment><USER8> That's actually even more distressing. You want stability out of a currency, not wild swings.

<comment><USER9> Indeed, but keep in mind they are talking about investment, not currency. Wild swings are great for investment, as long as it swings up eventually.

<comment><USER6> Even in terms of investment, you want steady, constant growth... not unpredictable swings.

</comment></comment></comment></comment></comment></comment><comment><USER10> When a currency is devaluating but interest rates are low, the banks can borrow at low rate, sell rubles for dollars into sliding market, wait for ruble to slide more, then repay the loan with cheaper rubles and make a big profit. The trouble is they also accelerate the slide, courtesy of the low rates. The central bank rate increase was likely done to head off that particular danger. Of course there are many other factors pressing the ruble down as well, which central bank can't control, but at least they are trying to prevent worseing the situation with their own hands.

Long term high interest rates plus high cost of import supplies will likely strangle the economy though. Increased demand for domestic goods due to higher import prices is a good thing, but one needs capital to operate most businesses. When interest rates are high you can't borrow, and when political situation is flaky fewer people will want to buy equity.

<comment><USER11> at 17% the interest rate paid by the RUB shorts amounts to 0.046% per day, whereas just today trough to peak USDRUB moved 35%. In this environment unless you "do a Sweden/Slovakia" and ramp rates to 1000% by preventing offshore banks from funding onshore (a form of capital controls), a measly 600bps hike is meaningless.

<comment><USER12> USBRUB moving 3500bps today tells us volatility is high. It does not indicate that we can expect an average move of 460 bps per day going forward. Carry trades perform horribly amidst increasing or ambiguous volatility. Their leveraged structures make sharp adverse swings game overs.

<comment><USER11> sure - but we're talking RUB specs who are in it for a 5 day move. Nobody is expecting annualized vol of 566% (35% * sqrt(262)). RUB has been moving several chunky percentages daily for 2 weeks at least, and is down 50% against the dollar 3 months. 17% annualized doesn't make an iota of difference in that kind of market.

Where it _does_ make a difference is that long end bonds (OFZ 28s) are now yielding 15%. That's 15% for 13 years ;) Chunky yield in the long end. That will tempt the long term investors in (IF big if, they don't impose controls). Care to catch the falling knife?

<comment><USER13> Can you elaborate on the impact of capital controls? Are bond investors not able to trade out of their positions?

<comment><USER11> The imposition of capital controls signals a country that is willing to trap money within its borders. That very fact is scary, and scary requires compensation. While that usually does not apply to what are classed as "long term" investors (direct purchasing of real assets in the country, purchasing of equities or bonds usually for a minimum time period, but NOT derivatives), it still requires an additional risk premium such that bonds will then need to be cheaper still (yield more) to compensate.

The more relevant short term argument that I am referring to, is that if they were to "squeeze" the offshore market by disallowing or limiting the lending of RUB by local banks to offshore banks, then they could easily engineer a massive squeeze on the rouble offshore. That is all those people who are short RUB would suddenly find that "rolling" those positions (that is, borrowing RUB so you can sell it), suddenly costs a fortune. This is even true in FX forwards because ultimately these are a derivative of interest rates. If it costs you a fortune to "fund" the RUB, then the converse is that it pays you a fortune to lend RUB to the market. If it pays you handsomely simply to lend short-term RUB to the market, there is no reason to buy long term bonds, whose price must "compete" with the front end of the yield curve. In short: high short end rates will deter purchasing of any instrument with a (significantly) lower yield in the long end of the yield curve. If we were to get the above mentioned RUB squeeze to anything like 100% or higher, imagine how silly it would be to buy a long end bond at 15% (even though yes yes, long term it might still be a good investment - but that would require a long discussion about the implied path of future rates. Humans have proven themselves to be very short termist in markets and thus it is likely that long term bond prices would fall, dramatically in that scenario).

So to answer your question directly, bond investors still would be able to trade out of their positions (though in a disorderly market even that cannot be taken for granted), but at a very low price.

<comment><USER13> Thanks, super informative. I appreciate it.

</comment></comment></comment></comment></comment></comment></comment><comment><USER14> A company makes a smart decision to pause sales in order to ensure they're not selling their products at a loss due to the drop on the value of rubies....

One shouldn't expect otherwise.

<comment><USER12> Putin has an annual press conference coming up soon. Many anticipate a softening of his stance on Ukraine, in a bid for loosening sanctions. I am more fearful of him introducing ham-fisted capital controls. Apple may be similarly cautious.

<comment><USER13> Shouldn't we all be afraid of the opposite, namely war, like with Japan 1941? After all, one of the major reasons for trying to retain control of Ukraine was for the energy resources and pipelines; the rout in oil has shown how important energy is to the Russian economy and I would expect that losing control of Ukrainian energy assets would permanently impair the Russian economy.

<comment><USER15> War is the worst case scenario for everyone. I don't think even Putin would leap straight there, though if you think it's a possibility then yes it's a bigger thing to fear than capital controls.

If the Russian economy is already overdependent on oil money, they should be looking to diversify, not to acquire more energy assets.

<comment><USER16> [https://en.wikipedia.org/wiki/Russo- Georgian_War](https://en.wikipedia.org/wiki/Russo-Georgian_War)

</comment></comment></comment></comment></comment><comment><USER17> Heard a story on the radio this morning about a couple in Argentina that cooks meals for tourists in their home. They stopped processing areservations in pesos b/c they couldn't confidently predict the price even two months out. Now they only deal in dollars

These people: [http://www.npr.org/blogs/parallels/2014/12/16/370979773/arge...](http://www.npr.org/blogs/parallels/2014/12/16/370979773/argentinas- approach-to-inflation-ditch-the-peso-hoard-u-s-dollars)

<comment><USER18> Apparently bitcoin is very popular in Argentina for that exact reason: [http://www.economist.com/blogs/schumpeter/2014/06/bitcoin- ar...](http://www.economist.com/blogs/schumpeter/2014/06/bitcoin-argentina)

<comment><USER19> You know your currency is fucked when Bitcoin is seen as the more stable option.

</comment></comment></comment><comment><USER20> Another facet of this is an increase in buying in rubles to turn a cratering currency into real value before it's worthless.

<comment><USER21> Yes, this is exactly what happened today. Many Russians were buying anything they could for their cash.

[http://www.cnbc.com/id/102273163](http://www.cnbc.com/id/102273163)

</comment></comment><comment><USER22> There may be people buying iPhones for rubles, selling them online for other currencies, and then exchange it back to rubles for a profit. If that is indeed happening, I imagine apple would want to stop it before other markets are affected by side imports.

<comment><USER23> > There may be people buying iPhones for rubles, selling them online for other > currencies, and then exchange it back to rubles for a profit.

Why would you do that sort of roundabout route when you could just trade rubles for other currencies directly?

Plus selling the phone back to get rubles doesn't seem like a smart move unless you think you've timed the bottom and the ruble isn't going to slip further against the dollar. Doesn't seem like people really think that.

At the moment, people would rather hold iPhones than rubles. I don't think that's in an attempt to turn a profit, it's just because an iPhone (or a major appliance, or a computer) is perceived to be a better store of value than rubles. Which given how electronics depreciate, is a really bad sign.

<comment><USER24> If Apple is slow to update its prices online you have a clear arbitrage opportunity.

</comment></comment></comment><comment><USER25> Around 1980 the USD central bank interest rates were raised to above 20%.

In the end of the 80es when the Swedes had their big banking crisis, interbank rates were above 30% and the Swedish krona halved compared to the neighbouring Scandinavian countries.

The late 90es had the Asian currency crisis where most South East Asian currencies where more than halved against the dollar. That crisis more or less led to the unfolding of the biggest hedge fund at that time (LCTM) which had to be bailed out by the FED.

Thru the 90es the Russians defaulted and the ruble collapsed manifold.

I am curious on how Russians sees the current situation?

Is this the west/USA waging currency war against Russia?

Is this a return to "normal" with things being like they were in the 90es? Or Russia being isolated as under the Soviet union?

I met a few Russians on my travels. And I always enjoyed the company of those I got to know. My impression is that Russia has an young (age 20-30) emerging middle class who are smart, reasonable and completely unlike the image we have in the west of druken, violent crooks with bad taste.

I would hate to see Russia return to isolation.

</comment><comment><USER26> If you missed this fun (or not) trivia fact: Apple is worth more than the entire Russian stock market (as of the middle of November) [[http://www.bloomberg.com/news/2014-11-14/apple-could- swallow...](http://www.bloomberg.com/news/2014-11-14/apple-could-swallow- whole-russian-stock-market.html)]

<comment><USER27> These comparisons are almost meaningless - e.g. Apple is worth about the same as Korea's stockmarket, almost as much as China's.

<comment><USER28> Rather than meaningless, it's amazing.

Apple is not worth anywhere near as much as China's stock market however. The Shanghai Stock Exchange all by itself is worth several trillion dollars.

</comment></comment></comment><comment><USER29> Because Apple products are expensive and are known to retain value over time they are a good place for people to dump their rubles knowing that they'll be able to sell the ipads and recoup their cash. But Apple doesn't want to be left holding the bag of rubles.

<comment><USER24> They hedge their currency risk so it's unlikely they have much exposure to the Ruble.

<comment><USER30> Any hedge they have is probably based on typical sales volume, this situation is not typical.

</comment></comment></comment><comment><USER31> We changed the URL from [http://www.bloomberg.com/news/2014-12-16/apple-stops- online-...](http://www.bloomberg.com/news/2014-12-16/apple-stops-online-sales- in-russia-over-ruble- fluctuations.html?hootPostID=6d80e7f3350c6b65d4f4721218e9b939), which is about a side effect (Apple halting online sales), to the more general article about the story, via [https://news.ycombinator.com/item?id=8757165](https://news.ycombinator.com/item?id=8757165).

Edit: Ok, we'll change it back. Sorry for the mistake.

<comment><USER32> Great, now all the comments lost context. This is asinine. Don't modify the submissions, or titles.

<comment><USER33> It drives me crazy.

I know it's well intentioned, but this power in the hands of a few seems to do more harm than good. Why not just let the populous up/down vote things as they see fit?

I wouldn't have upvoted an article about the general ruble decline (it's not news to me). I would (and did) upvote an article Apple halting Russian sales.

<comment><USER34> > I know it's well intentioned, but this power in the hands of a few seems to > do more harm than good. Why not just let the populous up/down vote things as > they see fit?

Err... this isn't a democracy and has never claimed to be one.</comment></comment></comment></comment>


Die Smiling on this Hoverbike - <AUTHOR> http://www.wired.com/autopia/2011/06/die-smiling-on-this-hoverbike/

<comment><USER2> From 3 days ago: <http://news.ycombinator.com/item?id=2640972>

Much discussion there.</comment>


Reducing customer service call volume at Expedia - <AUTHOR> https://marker.medium.com/how-expedia-solved-a-100-million-customer-service-nightmare-d7aabc8d4025 <comment><USER2> Anecdotally, NTL (cable company) in the UK faced a similar issue, and took the worst way to resolve it.

Instead of looking at and understanding why customers were calling in, they decided to outsource management of support to a company that specialised in cutting down the costs.

The methods they employed had employees required to reduce the average support call duration, much like it suggests Expedia started out doing. Employees that didn't hit or beat their average call duration targets got put on a performance improvement plan, and then ultimately laid off.

That average call time was then steadily reduced in stages. It was a great success, the average call duration dropped rapidly. What was happening was employees would "accidentally" disconnect customers if they came close to the target call time. Or just randomly drop calls early on if they were worried their average was too high. Customer dissatisfaction was significantly higher. Call volume was higher (but not everyone bothered to call back). Average call duration was down though, so big success! Customer retention was getting worse, but that, of course, was tracked under a different silo by a different team who had nothing to do with the support org.

At one stage, they actually dropped the target average call duration to less time than it took to reboot a NTL router / set top box, which was the very first thing customers were advised to do by support agents. They did finally notice, then, but their only solution was to increase the time by 30 seconds or so.

Give the wrong people at the wrong level the goals, and you're going to get poor results.

<comment><USER3> One big issue is that only easier things get handled.

Some metrics are plain wrong.

<comment><USER2> Speaking of wrong metrics, another anecdote:

I used to work for an ISP. Some genius high up in the support management chain decided that tickets were taking too long to resolve, and the touch rate was too high. So they implemented a way to measure how many times a ticket communicated on before it was resolved, and somehow turned that in to what they called a "conversion percentage". Percentage of what was never expressed.

The result, naturally, is that customers started talking to a void. Tickets took longer to resolve because the support engineers would waste time not asking clarifying questions because that would hurt their conversion percentage. Customer dissatisfaction soared.

A week or two after it went live, some engineers in the NOC team figured out that if you took a ticket, resolved it, and then re-opened it, you'd hit a 100% conversion rate, regardless of how you then communicated on it. Slowly but surely that started spreading through the company and things pretty much got back on track and customer satisfaction returned.

When I left, they were still tracking that conversion percentage, even though every support engineer was now taught during on-boarding that the way to handle a ticket was to resolve it and re-open it. Firmly on its way to becoming "it's just the way things are done"

</comment></comment></comment><comment><USER4> One big reason why outsourcing support is such a bad idea is that the incentives of the outsourcing company are diametrically opposite from yours. They want billable hours, and will do the least work possible to hit your KPIs so they can keep the contract.

For example, if 58% of your customers are calling in to have their itineraries resent, that's _awesome_ for the company because that's a lotta volume (and volume is money!) _and_ it's really easy to hit KPIs for call length, satisfaction etc. They are thus doubly _negatively_ incentivized not to do any upstream resolution: not only would they get paid less, but all their KPIs would plummet, because the remaining calls are now on average harder to resolve, take longer, and have more grumpy customers leaving bad CSAT scores!

<comment><USER5> T-Mobile customers can see a real live A/B test contrasting well-trained, thorough domestic support personnel versus an offshored organization that seems incented to simply dispose of calls as quickly as possible.

I have a particularly complex set of service combinations with T-Mobile. Any changes to my account configuration or devices can result in downstream breakages, especially when one promo crowds out another.

I used towork in support, and I was the first person in the company to carry the title Advanced Support Engineer. I know the initiative and the depth of knowledge it takes to be a truly effective support engineer. I have been consistently impressed by the personnel in T-Mobile's Eugene, Oregon support operation, which I tend to encounter when calling during conventional business hours in my time zone.

in contrast, the overseas support operation that I land on if calling off hours has been worse than useless, so much so that I generally give up and live with whatever breakage has occurred until business hours if it all possible. They are just that bad. They may execute the most basic, redundant, rote solutions, generally, but they demonstrate no discomfort in flat lying about both solutions and what steps they've taken... especially account credits. One of these screwups was so severe that it took a dedicated rep in Oregon hours of research and numerous tasks spanning a week to repair (plus follow-up three weeks later to make sure the fixes "took").

It's absolutely night and day -- awesome or excruciating.

</comment></comment><comment><USER6> I had a terrible experience renting in a foreign country thru Expedia. The rental was a condo, and the person never showed up. I'm standing in front of the condo for 5 hours, they wouldn't do anything to help me. They even claimed to have called the property owner, who said I wasn't there, whilst I was standing right outside the building!

If there's ever a problem, Expedia will claim time be "just a middleman" and you are screwed. I never got my money back and had to find another hotel at 11pm.

<comment><USER7> They actually are a middleman (travel agent). I stick to dealing directly with the big hotel brands, as their policies dictate that if you aren't able to be accommodated, then they will find you an equivalent room and pay for at least the first night. And if they can't do that, I would trust them to compensate some other way.

Otherwise you can just do a chargeback and claim services not rendered and you should get your money back that way.

</comment></comment><comment><USER8> Having booked travel through Expedia several times recently, I can confidently say that Expedia has not fixed its top issue for calling them - "To get a copy of their itinerary." Their system is buggy at best. The funny thing was the customer service reps couldn't send me a copy of the itinerary either. Nothing came off of Twitter too. Finally, one heroic chat rep was able to copy-paste the itinerary and email it via some random customer service email address. Loved him but never going to use Expedia ever again. EVER.

P.S.: If you check their twitter feed, you'll know what I am talking about. It is also an engineering failure from the get-go. Not being able to retrieve your itinerary online is an engineering failure on so many levels.

</comment><comment><USER9> This is a subject that's very near and dear to my heart.

At its core, it's the alignment between customer experience and product. Support as the customer facing representatives need to be empowered to convey the shortcomings of the product as it impacts customer experience. The product team in turn should then digest that information and relay it to the people building the product, the engineers in this case.

Good alignment/synergy etc. between those teams leads to good customer experience.

My support team calls it "support deflection" (in terms of how it impacts support). "Upstream solution" (from the article) is roughly equivalent, from a different perspective.

The challenge that we face as support professionals becomes existential, once we've mastered deflection. As the article points out, it is very hard to measure that type of success.

The less reliance customers have on support, the less important support is to the company's success. The less important support is to the company's success, the less support is valued.

If fewer support requests come in for an extended period, then the support team will be downsized.

If a company is around long enough, surely undervaluing support services will likely catch up with them as new features and eventually new products are rolled out, and support professionals can only hope that executives are far- sighted enough to recognize that.

The problem that I've been stuck on for a while is how to align company interests with the support team's best interests, long term. Solving customer problems "upstream" is essentially fighting against ourselves.

</comment><comment><USER10> My experience with Expedia support is so-so as of this month.

One of my flights was cancelled by Air France and Expedia informed me about it and offered 2 alternative flights or partial refund. To communicate my choice I had to call their support in IE or UK. This is not easy when one is in Indonesia in a remote place. They refused to communicate over email.

So I got a skype subscription and called them. I told them I wished to be refunded because none of the alternative flights offered were compatible with my schedule and they were not able to offer any other flights. They said ok but were unable to tell me how much I would get refunded. They also said I would be receiving an email confirming the refund in 2 days. I received no email, I called them back and asked if my refund choice was registered in their system and if I could get a proof of our agreement for a refund eg. via email. They assured me an email would be sent shortly. It has now been two weeks and I have so far received no refund nor any email or other proof that we have agreed on a refund nor do I know when to expect this refund or the amount.

<comment><USER11> To be fair their support is probably getting hammered these days because of the Coronavirus. I was trying to reschedule my trip last week, and I alone consumed several HOURS of support time (they had to call the airline several times, where there was a waiting time, and the hotel).

Nevertheless, I was able to reschedule my non-refundable non-rescheduleable trip, and the support was great.</comment></comment>