How to Trick an Online Scammer Into Carving a Computer Out of Wood - <AUTHOR> http://www.theatlantic.com/technology/archive/2010/09/how-to-trick-an-online-scammer-into-carving-a-computer-out-of-wood/63674/ <comment><USER2> 419 scammers are pests, but scamming the scammers can go to far: [http://www.thisamericanlife.org/radio- archives/episode/363/E...](http://www.thisamericanlife.org/radio- archives/episode/363/Enforcers). This story is interesting to listen to because you start off with the predictable reaction -- those scammers are getting what they deserve. But then it takes a left turn -- they get quite a bit more than they deserve. You end up feeling sorry for the scammers. Listen to the piece and see what you think.

<comment><USER3> The sad thing is that because of the conditions they have grown up in, scamming is one of the only ways for them to improve their lives. Now this man has actually produced some very good works of art, and instead of receiving the recognition and compensation he deserves, he is being ridiculed. This would have been a great opportunity to pull someone out of his current condition and encourage him to make and sell art, improving his life and the lives of others. Instead, he's pretty much guaranteed to continue scamming.

<comment><USER4> I doubt this man spent time to produce the sculpture -- remember his full time job is scamming, there is probably not much time left for sculpture. Scamming is quite profitable and labor in Nigeria is probably fairly cheap, so if I had to guess, he probably payed someone else to do the work.

<comment><USER5> Or, as smoody pointed out below, force someone to do it at gunpoint.

</comment></comment></comment></comment><comment><USER6> Reminds me of when I used to mess with telemarketers. My goal was to see how long I could get them to stay on the phone, thinking that the more of their time I wasted the less profitable telemarketing would be (how much of an affect could one person really have though?). I was very successful with a variety of techniques and it turned into a fun game. I almost regret having changed my number, no longer getting these calls now.

One good one was when I had this lady first describe her work environment, finding out she was working with her fiance. I had her describe him and their relationship and somehow got her talking about her willingness to engage in a threesome. After almost 15 minutes she politely told me that she had to go, being at work and all. Maybe 10 minutes later her supposed fiance called me back asking why I was talking 'rubbish' to his girlfriend. I talked with him for a while and got him to tell me about their relationship, that they had been engaged for 5 years, got engaged when she was 15! He started describing her physically but finally hung up when I tried to get him to tell me how much she weighed (I was really curious after he told me she was 5'4"). That one was during work and by the end of the conversation I had the entire office sitting around listening.

Another time I was walking home when I received a free satellite tv installation offer (during the recorded intro to these calls, before pressing '1' to speak to someone, is the best time to prepare what you're going to say). When the guy came on the line I got him to describe in as much detail as he was willing the contents of each of the porn channels that was included in the service. It's fun to imagine what their going through on their end of the call (sometimes I would get them to yell into the phone claiming I could barely hear them).

I think it's really entertaining when they get mad at me for wasting their time! This usually happens when it's a get-out-of-debt scammer. I answer all their questions and they get excited when they hear my monthly credit card bill (never bothering to find out that I pay off the entire balance each month) but then get real upset when I try to convince them that I'm not in need of their 'services'. Half the time they don't believe me, trying to 'comfort' me by telling me that it's ok to be in financial distress, everyone is in debt and are experiencing tough times right now.

I tried to think of ways to promote this kind of behavior, figuring if one in ten of their calls were to people like me then telemarketing would be much less profitable, but I never tried to follow through with anything. Now that I don't get these calls anymore I kind of miss it. Over the years I became much more comfortable, confident and competent dealing with telemarketers in this way.

</comment><comment><USER7> A lot of this scam-bait stuff is going way too far, and I can't help but wonder about the ire on behalf of all those 'innocent victims' out there.

In the classic 419 scam (and yes, there are other variants that don't imply quit the same level of corruption in the scammed): the offer is clearly drawn from money that some dictator has supposedly stolen from his own country. Trying to take part in this theft doesn't make you 'innocent', it makes you a smaller-scale, dumber, would-be crook.

<comment><USER8> Again I feel reminded of this quote: you can't con an honest man.

<comment><USER9> Sure you can. While classic scams do appeal to greed, more modern ones appeal to altruism. They focus on advance fee scams, but there are plenty of fake charities and whatnot out there.

</comment></comment></comment><comment><USER10> Alternatively, the (classic) how to trick an online scammer into buying a fake laptop: <http://www.zug.com/pranks/powerbook/>

(I know it's only tangentially related, but I love that story.)

<comment><USER11> A good read, and interesting on the retaliation by the scammer. Interestingly, someone I know asked me to look at an ebay purchase they were going to make, which I immediately suspected was a scam. They pulled out (hadn't actually put on a bid) and the retaliation was a succession of viruses and phishing emails, all of which (sort of) looked like ebay dispute resolution emails.

I'm guessing the scammers have their message boards as well, and the standard retaliation response is to try and send viruses and/or phish out a credit card for fraud.

</comment></comment><comment><USER11> The original, linked article is the better read, although towards the end the author descends into a mess of racism accusations and uncertain direction. He completely misunderstands the use of the term 'owned', and somehow thinks it's a colonial throwback to slavery rather than common internet speak for a complete win over an opponent.

The writer of the original piece needs to stop worrying about the scammers for two seconds and see that this movement, ultimately, has a good chance of putting scammers out of business. Because I can see more people getting in on the anti-scam game than people falling for the scams. And once the numbers on a prospective scammer turn up to be 3:1 that it's a fake acceptance, they are going to start giving up on it.

<comment><USER12> Hmm,

The wood-carving thing actually makes me think they aren't "in business" in the conventional sense. Rather, they seem like desperately poor people grasping at whatever opportunity appears. I suspect that most of their efforts are utterly unsuccessful and it's more of a matter of doing it because they have nothing better to do.

<comment><USER11> Your points are all valid but they are still knowingly engaged in crime. It's unlikely this guy carved the thing himself, rather he would have commissioned it from a local carver.

It's sad that they are poor, to me, still not an excuse to start stealing from people. There's a lot of poor people in the world that are still honest. I'd much rather see compassion shown towards them.

<comment><USER12> My point had nothing to do with compassion or lack-of compassion.

My point was _only_ that, in regards to the original poster's idea, you aren't going to "put them out of business" by providing them with lots of frustration. They slog through an immense amount of futile activity already and are quite willing to do more.

Judge them however you want...

</comment></comment></comment></comment><comment><USER13> the danger here is, of course, that the person coaxing the wood sculptures might be doing more harm than good. who knows what circumstances led to the sculptures... could they have been done at gun point or other threat? if riches are at stake, then i believe the answer is 'yes.'

<comment><USER14> It's not organized crime, more like a national past-time in certain geographic regions. I used to frequent the same internet cafes where these guys stay up late; they look a lot like problem gamblers. Tens of men huddled over CRTs, chain smoking and no one talking to each other. You can see people bringing in "evidence", photos in USBs to send over to marks, multi-tasking in IMs and chats and romancing webcam girls on the side.

I haven't seen any money while there. Whoever comes there spends their disposable cash to surf, most of the day they just play games or facebook.

I found the scammers to be pretty polite and "brotherly". There is a lot of empty bragging and people will pretend to be big-shot mafiosi, but it's all a con aimed at you. The programmer in me wanted to inquire about bot-nets: everyone claimed to have "one", but the responses were confused at best.

In cities where bonafide scammers are present, there will also be the "runners". But just like in border towns or trafficking areas, it's best to walk out of the venue when in comes a guy followed by a crowd. A solitary quiet man, behind whom walks a group: biggest tell-tale of a made man, and if you look foreign or moneyed, it's best to get up and walk away without making eye contact, but while also not looking scared or nerved. (The one time I came very close to this I put my beers next to the computer and yelled to the attendant "I will be back for this". Never did.)

The one guy who I _know_ makes a decent living at this plays the big leagues. He is in the scrap-metal business and has a physical office. Former diplomat from a _central_ Africa country. His business, however, is strictly document forgery, insurance fraud and shipment theft. He was a real gentleman, an avid tennis and muay thai fan, but the people competing for his attention were thugs and would do anything for him.

TL;DR, the guys who pull out guns have more at stake; usually drug money, or theft of their stolen properties. 419ers are bored nerds; natural born Office jockeys who would have been employed in administrative positions if there were jobs available to them.

</comment></comment><comment><USER15> this is epic. I think you could make a business of this and sell these wood artifacts on ebay.</comment>


The Best Fix for American Drug Prices Is Already on the Books - <AUTHOR> https://www.bloomberg.com/view/articles/2018-05-11/to-lower-drug-prices-trump-should-boost-generics <comment><USER2> Crazy idea: all pharmaceutical research should be publicly funded and publicly-owned. No private ownership of drug patents, period.

"But this will disincentive research!" Maybe. Or, tax people more in order to fund the research. The tax revenue would go directly to researcher salary, equipment, expenses. Even a large tax increase can't be worse than today, where people have to pay huge $$$ anyway, of which a large cut goes to executives, shareholders, the marketing machine, etc.

I believe by and large, people that go into pharma research do it because they're interested in the science and the cause, not because of any privatized profit motivation. Which is to say, people will still go to grad school in the same fields and continue carrying on the research work (so long as the facilities and equipment is available, which we can solve). The downside is that many senior executives would be displaced or be forced to accept lower compensation.

Someday in the distant future, people will look back and be as shocked that private companies profited from medicine, as we're shocked today about the for-profit firefighters in ancient Rome.

<comment><USER3> The reality is that governments have demonstrated limited interest in effective biomedical R&D. Europe does not invest proportional to their population or economy, their governments are happy to outsource it to the US (and Asian) private sector. And there is the longstanding issue that what governments do fund is biased by political fashion and not the areas of research that will maximize benefit. There has been a huge growth in private foundations that fund biomedical research that governments should fund but do not.

Also, pharma researchers will accept massive pay cuts "for the science and the cause" as soon as Silicon Valley software engineers take massive pay cuts "to make the world a better place". I have many friends in biotech and they are as interested in maximizing their paycheck as software engineers, it would be naive to expect anything different.

<comment><USER4> Many software developers do take massive pay cuts to make the world a better place. People like Linus Torvalds could be much, much wealthier if they wanted to. Jason Scott is probably the single most important digital archivist alive right now; he's currently running a podcast to help himself get out of debt while he works for a company that doesn't offer a pension plan.

I'm not just talking about fringe developers either[0]. A huge portion of our modern tech industry is built on the backs of people who made a conscious choice to prioritize public good over their own good.

I've nothing against developers who make money, but my takeaway from the tech industry has actually been that pure profit-driven motivations are simply not enough to build sustainable, resilient technology. I really hope it's not naive to expect at least some small portion of the population to be charitable because my experience has been that it's often the charitable people who end up getting crap done.

Pharma research may be different, because that comes with equipment costs. You need access to that equipment. But if that's the case, the software industry is maybe a bad example to compare it to.

Generally agreed on government funding though; maybe I'm too pessimistic but I just can't see the US government investing serious effort into drug research.

[0]: [https://arstechnica.com/information- technology/2014/04/tech-...](https://arstechnica.com/information- technology/2014/04/tech-giants-chastened-by-heartbleed-finally-agree-to-fund- openssl/)

<comment><USER5> Don't look at anecdotes, look at the numbers. How many brilliant Stanford graduates are going to work for the advertising industry every year instead of something that'll make the world a better place? There is no correlation between brilliance and altruism.

<comment><USER4> > There is no correlation between brilliance and altruism.

That's not really relevant to what was being discussed above. The question isn't whether smart people are altruistic, it's whether there's a correlation between altruism and _effectiveness._ The question is whether charitable individuals are capable of building better products than profit-driven individuals.

And given that depending on who you ask anywhere from around 60-90% of modern businesses run Open Source software, it does appear that statistically (not just anecdotally) there are parts of the tech industry that are better served by non-commercial, charitable entities. If that wasn't the case, businesses would use closed source software instead.

When Heartbleed came out, nobody suggested "well, maybe we should just all switch to a proprietary for-profit solution instead of using SSL," because a good, proprietary, for-profit alternative doesn't exist and probably never will. It's just not the type of technology that Silicon Valley is good at building.

Note that this is something that seems obvious in hindsight, but it was up for debate early on in the software industry. Companies like Microsoft went so far as to compare Open Source to communism. There was a strong belief that people opening up licenses, giving away software for free, and even (the horror) requiring other licensees to give _their_ software away for free would remove any incentive for businesses to innovate and that in the long run it would strangle new technologies.

<comment><USER5> Most open source development is done by for profit companies. Not through altruism, because software just happens to be an input into more profitable businsss activities. If Google licenses Android to smart phone manufacturers, maybe it gets $5-10 once every two years. But by giving it away for free and creating an Android user, Google makes that much _every month._

<comment><USER4> The fact that many companies incorporate Open Source into their profit models does not diminish or remove the enormous number of unpaid, volunteer hours that have gone into those projects.

It's simplistic to the point of insulting to claim that the Free Software movement can be completely encapsulated by Capitalism. You have to ignore vast swaths of the history of the Internet, Linux, and the origins of Copyleft itself. You have to either ignore efforts by organizations like Mozilla or the Internet Archive, or claim that those efforts aren't important.

Richard Stallman's original announcement of GNU included the following message: " _I consider that the golden rule requires that if I like a program I must share it with other people who like it. I cannot in good conscience sign a nondisclosure agreement or a software license agreement. So that I can continue to use computers without violating my principles, I have decided to put together a sufficient body of free software so that I will be able to get along without any software that is not free._

 _If I get donations of money, I may be able to hire a few people full or part time. The salary won 't be high, but I'm looking for people for whom knowing they are helping humanity is as important as money._"

This is the environment that Linux came out of - people who decided they cared about something more than they cared about money. You don't have to be an absolutist like Stallman, but you also can't pretend that absolutists like Stallman haven't played important roles in building our modern computing world.

</comment></comment></comment></comment></comment></comment></comment><comment><USER6> I would suggest anyone who is interested in reducing drug prices listen to this econtalk episode with Robin Feldman[0]. They discuss her book "Drug Wars how big pharma raises prices and keeps generics off the market"[1]

In the episode they discuss all sorts of shenanigans that pharmaceutical companies pull. If you want real reform you need to look at weak patents, eliminating "reverse payments" to generic companies, and obstruction to bioequivalence testing. Drug companies routinely manipulate the current laws, specifically Hatch-Waxman in order to stifle competition from generics.

[0] [http://www.econtalk.org/archives/2017/06/robin_feldman_o.htm...](http://www.econtalk.org/archives/2017/06/robin_feldman_o.html)

[1] [https://www.amazon.com/Drug-Wars-Pharma-Raises- Generics/dp/1...](https://www.amazon.com/Drug-Wars-Pharma-Raises- Generics/dp/1107168481)

<comment><USER7> I'm going to go on a bit of a rant. I agree blocking access to drugs for bioequvalence is bullshit. Drug companies can do it by claiming that the law doesn't allow them, but it's BS.

The reverse payments I see nothing wrong with. Imagine you're a drug company holding a patent. A generic company decides to try and invalidate it. You think there is a 50% chance they'll win. The generic company agrees.

You have two options: 1) spend a ton of lawyers and still possible lose, 2) go to the generic company and agree to let them produce the generic 5 years before patent expiration.

With the 2nd option, you're basically agreeing to split up the pie evenly rather than risking it all.

In every case where reverse payments are made, the drug goes generic _earlier then if they went to court and got the patent upheld_.

Eliminate reverse payments and what you're doing is making lawyers rich. In many cases the branded drug company would win and block generics for many more years than if a reverse payment was made.

</comment></comment><comment><USER8> Here's the rub with developing something as important as a medicine. If it works, people often _need_ it to prolong their life or improve their health. What drug companies are selling... isn't like what other companies are selling. We ought not look at it through our normal understanding of goods in a free market.

Take, for example, Sovaldi, a drug which cures Hepatits C. After discovering this drug and confirming its efficacy and safety, the next question should be, IMO: how fast can we possible scale production to quickly stop liver damage and cure those suffering. Instead we have Medicaid setting absurd rules ([https://www.chlpi.org/wp-content/uploads/2013/12/Annals- of-I...](https://www.chlpi.org/wp-content/uploads/2013/12/Annals-of-Internal- Medicine-HCV_July_2015.pdf)) whereby sufferers must get sicker before they are eligible to be cured. That is seriously fucked up.

Edit: to be clear on why states must do this: Gilead Sciences is charging $84,000 for a course of treatment and there isn't enough money to treat every patient. Gilead offers the same treatment in Egypt for $900. ([https://www.reuters.com/article/us-hepatitis-egypt-gilead- sc...](https://www.reuters.com/article/us-hepatitis-egypt-gilead- sciences/gilead-offers-egypt-new-hepatitis-c-drug-at-99-percent-discount- idUSBREA2K1VF20140321))

<comment><USER9> > If it works, people often _need_ it to prolong their life or improve their > health. What drug companies are selling... isn't like what other companies > are selling.

It is very much like many other products. If you don't have food you will die of starvation, if you don't have shelter you will die of exposure, etc.

The real issue is drug patents. Nobody is complaining about the price of ibuprofen any more than they complain about the price of eggs. But the patent system is doing what it's supposed to here -- twenty years ago any patented drug didn't exist, and it nominally only exists now because the inventors judged it to be sufficiently profitable to conduct the research. If you make it less profitable, you'll get less research.

There is no drug research fairy. If you don't want the money to come from high insurance premiums that pay for high drug costs, it has to come from somewhere else. And just devising some other method (like taxes) of extracting the same money from the same general public isn't really going to change much.

<comment><USER10> But drugs are produced in other countries, too, and they manage to do so without charging what US companies do.

<comment><USER11> Those companies charge exactly the same as the US companies when selling to the US. Much like US companies charge the same as European companies when selling to Europe.

</comment></comment></comment></comment><comment><USER12> One of the biggest reasons is that the US essentially subsidizes drugs for many other countries, such as Canada, that impose price controls, and threaten to allow other manufacturers to provide the drugs in their countries if the companies don't accept the imposed price. This enables these countries to get access to new medicine without paying the cost of developing the new medicine. Its time to end the trade war on medicines.

<comment><USER13> are you sure that the cost of development is the real issue?

the last numbers I saw, 2015, showed marketing budgets for medicine to be often times twice what was being spent on R&D in the US. currently, only four countries allow this time of marketing: New Zealand, Brazil, Hong Kong, and United States.

it sounds like the US isn't necessarily subsidizing drug R&D, and instead is subsidizing marketing.

<comment><USER9> > the last numbers I saw, 2015, showed marketing budgets for medicine to be > often times twice what was being spent on R&D in the US.

The marketing expense isn't necessarily avoidable.

Suppose it costs $50M to research a drug and patients will pay $1000 for a course of treatment. If you do no marketing then you get forty thousand customers and can't recover your R&D. If you spend $100M on marketing then you get two hundred thousand customers and more than recover both your R&D and your marketing expense.

Spending more on marketing can allow them to do R&D they otherwise couldn't by providing more customers to spread the R&D costs over.

<comment><USER14> Surely though the best marketing is the effectiveness of the product, if it's a good product then doctors will prescribe it.

I don't know if it's true or not, but looking at US movies/tv shows it seems like everyones taking medication which I doubt they actually need. Here I don't know very many people who actively take regular medication, I know that people who move here complain at how doctors are stingy with prescriptions, yet life expectancy is longer than the US.

<comment><USER7> _Surely though the best marketing is the effectiveness of the product, if it 's a good product then doctors will prescribe it._

That's true for very few physicians. The ones that are leaders in their field and do research know the ins and outs of new drugs, but your average doctor knows very little.

<comment><USER14> But, they all have access to knowledge bases and information and guidelines provided by the government.

<comment><USER7> That is true, but setting aside time to do research is a much bigger hurdle than taking 10-15 min to listen to someone give you the highlights.

And in case anyone was wondering, doctors are very skeptical of drug reps and always verify anything they say before believing it. But it gives them a starting point.

<comment><USER15> "always verify". Well that's obviously not a true statement but (not singling you out) I wonder why there is such a compulsion to defend the honor, as it were, of medical doctors.

It's not exactly the same as pharma, but in the medical device field salespeople are commonly in operating theatres. Clearly the surgeons doing this do not have time to "verify" what they are being told. [https://www.washingtonpost.com/national/health- science/why-i...](https://www.washingtonpost.com/national/health-science/why- is-that-salesperson-in-the-operating-room-for-your-knee- replacement/2016/11/14/ab8172fa-78e6-11e6-beac-57a4a412e93a_story.html)

Americans would do well to stop idolizing doctors and realize that huge numbers of them are lazy, incompetent shills who routinely do a lot of preventable damage.

</comment></comment></comment></comment></comment></comment></comment></comment><comment> olog-hai There's a newer Bloomberg article with a matching title that's different from the one linked to here.

[https://www.bloomberg.com/view/articles/2018-05-13/trump- dru...](https://www.bloomberg.com/view/articles/2018-05-13/trump-drug-plan- shows-why-high-prices-persist)

<comment><AUTHOR> Your article was the content of the link I originally submitted.

Mods: Can you update thread to point to the above link? Bloomberg bamboozled my post.

</comment></comment><comment><USER16> The example of Humira is a terrible one: as a monoclonal antibody, to produce the exact molecule would require another company getting its hands on the plasma cells that produce it.

It is not as simple as copying the chemical steps to produce the drug, as in Lipitor.

Complicating this, there is evidence that not all Monoclonal antibodies that act on the same receptor are made equal - some might be antagonists whereas others are agonists.

</comment><comment><USER17> We have compulsory copyright licensing for sound recordings - if you pay the statutory license fee (9.1 cents per song or 1.75 cents per minute) to the original publisher, you can press records of their song, whether or not the original publisher approves.

Why not have compulsory patent licensing for drugs?

</comment><comment><USER18> More generics isn't really the solution. The US actually has more generic prescriptions as a percent of total prescriptions than most developed countries and US spend on drugs as % of healthcare spend is similar to other OECD countries. The cost comes from high priced innovative meds. They are high priced bc they are expensive to develop -- $2.7b to get a drug approved per most accepted estimate. Most of that is cost of failure. So the issue is if we cut those prices we stop innovation. R&d in most disease areas has already slowed dramatically bc it's too expensive / risky

</comment><comment><USER5> > Over time, Big Pharma learned how to game the law. As the patent for the > original drug approached expiration, companies added wrinkles — a new > coating for the pill, a time-release version, and so on — that they then > patented. If the patents were approved, and they usually were, the > exclusivity clock would start ticking all over again.

This is so misleading. When this happens, only the new "wrinkles" are patented. The drug itself becomes free for everyone to use.

> Hatch-Waxman worked exactly the way it was supposed to for a long time. For > many high-priced drugs, once the generics flooded the market, the price went > down by as much as 90 percent. Statins are a good example: The price of > generic Lipitor is about $19 for 90 tablets, while the price of branded > Lipitor is $165 for 30 tablets. The market share for the branded drug often > dropped by 75 percent or more. Over time, Big Pharma learned how to game the > law. ... The rheumatoid arthritis drug Humira is a perfect example of what > has happened.

Humira is not an example of "gaming" Hatch-Waxman. Hatch-Waxman assumes that the secret sauce is a drug's chemical formula. It creates a process for generics to be created when the patent on that formula expires. Humira is what's called a biologic, where the secret sauce is not only the active ingredient, but the process of manipulating bacterial cells into manufacturing the drug: [https://www.bloomberg.com/view/articles/2017-09-07/why- the-b...](https://www.bloomberg.com/view/articles/2017-09-07/why-the- biosimilar-drug-revolution-hasn-t-arrived). AbbVie has another five years left on those patents.

The Hatch Waxman equivalent for biologics ("BPCIA") was only enacted in 2010: [https://www.fda.gov/drugs/developmentapprovalprocess/howdrug...](https://www.fda.gov/drugs/developmentapprovalprocess/howdrugsaredevelopedandapproved/approvalapplications/therapeuticbiologicapplications/biosimilars/default.htm). Because the law is so new, it will take time before the generics industry figures out how to navigate the new process.

The article's list of policy recommendations is also ignorant:

> First, shorten the patent exclusivity period to 10 years. (Drug companies > say they need the longer time to recoup their research and development > costs, but the truth is most companies spend more on marketing than R&D.)

Drug companies have similar SG&A (overhead) to R&D ratios as tech companies: [https://news.ycombinator.com/item?id=16585194](https://news.ycombinator.com/item?id=16585194). The point, though oft-repeated, is completely nonsensical. Nobody _wants_ to spend money on marketing. If drug companies could recoup their investment faster by spending less on marketing, they would.

> Second, outlaw the practice of paying companies to keep generics off the > market and similar forms of gamesmanship.

It's already a violation of antitrust law: [https://www.ftc.gov/news- events/media-resources/mergers-comp...](https://www.ftc.gov/news-events/media- resources/mergers-competition/pay-delay).

> Third — and most important of all — don't allow companies to extend the > period of exclusivity beyond the original 10 years.

That incorrectly assumes that patenting improvements to a drug extends the period of exclusivity of the original drug.

<comment><USER4> > Nobody wants to spend money on marketing. If drug companies could recoup > their investment faster by spending less on marketing, they would.

This doesn't pass my sniff test. The original article's point isn't that companies are marketing just so they can flush money down the toilet. Of course it's profitable. The article's point is that _we as a society_ want them to spend more money on R&D; and our current strategy seems not to be working.

Companies would prefer to spend no money at all. If they could recoup their investment by spending no money on marketing, research, development, or production, they would. So a good company does the next best thing: they find the least costly, most efficient way to make money and prioritize that.

If companies are investing the majority of their resources into marketing, that at least suggests that they view marketing as the most efficient way to make money. Since we as a society would prefer that R&D be the most efficient way for them to make money, it may very well be reasonable to look for ways to decrease the cost of R&D and/or to _increase_ the cost of marketing.

Shortening the exclusivity period of a patent might be an effective way to decrease the value of marketing, since marketing returns likely benefit from exclusivity.

<comment><USER5> Shortening exclusivity would _increase_ the need for marketing, because companies would have to get the word out to consumers and doctors much more quickly.

<comment><USER4> Again, that _might_ be true, but it rings multiple alarm bells when I actually think about it.

First, it assumes that the profit margins of a company remain constant. Again, companies want to make money. Period. So say that you're a pharmaceutical company and you're currently getting back 250% of your costs for a drug over the exclusivity period.

If we double that exclusivity period, you are not going to decrease the price you charge or the amount you invest into marketing so that your return on investment remains 250%. At least, not if you're a good business owner. You're going to double down on the exact same strategy and make 500% return. If I later halve your exclusivity period, you won't stay at 500%. You'll go back to 250%.

The fact that a company has less time to do marketing does actually suggest to me that marketing becomes a less bulletproof, desirable strategy for making money - which is exactly what the author wants. It might still be a bad idea; maybe pricing becomes the most efficient way to generate profit, or maybe the drug industry has such tight margins that there is no efficient way to generate profit and everyone closes up shop.

But it seems at least somewhat reasonable to say that if you expect the cost of marketing to remain constant, and the return per dollar to go down severely, you invest less into the now inefficient strategy.

The other reason that I have difficulty accepting this claim at face value is because I can think of markets where it just isn't true: for example, mobile apps. Nobody smart is investing huge amounts of money into marketing a mobile app. What they're doing is churning out five apps a week on the hope that one of them goes viral.

The lack of exclusivity in that market means that constant development is basically a requirement to keep your head above water.

Now, do we want our drug industry to look like the mobile app market? The mobile market is so opposed to long-term investment that the majority of games and apps developed there are rushed-to-market crap. So we probably don't want our drug research to operate the same way.

But I don't think it's immediately unreasonable that there might be a middle ground we could hit between 20+ years of exclusivity and 4 days of exclusivity.

</comment></comment></comment></comment><comment><USER19> > Over time, Big Pharma learned how to game the law. As the patent for the > original drug approached expiration, companies added wrinkles — a new > coating for the pill, a time-release version, and so on — that they then > patented. If the patents were approved, and they usually were, the > exclusivity clock would start ticking all over again.

I don't understand how that would ever work.

Time-release version of X, now with new coating: patented

Old version of X, with no time-release and without new coating: patented expired, correct?

If so, what stops the generic manufacturer from just manufacturing the old version of X?

<comment><USER7> There is nothing to stop them. The only reason why "evergreening" works is because doctor's write scripts for the "new" drug.

Sometimes it's worth it! If you need to take a drug three times a day and the new version is only once, that can improve patient adherence and outcomes.

However, sometimes the added value is minuscule. In that case insurance companies should just say "we're not paying for it since the added cost is not worth it".

Problem solved.

<comment><USER19> > Sometimes it's worth it! If you need to take a drug three times a day and > the new version is only once, that can improve patient adherence and > outcomes.

For the moment, let's ignore that case by simply calling the improved drug a new drug.

> However, sometimes the added value is minuscule. In that case insurance > companies should just say "we're not paying for it since the added cost is > not worth it".

I'm still lost.

1\. Doctor writes prescription for "B", which is really "A" with an insignificant change and a price bump.

2\. Patient goes to pharmacy and looks at their copay for "B." Whoa! they say-- that is surprisingly high.

Wouldn't the pharmacist then have the responsibility to jump in with their expertise at this point? Say: "B" is just a slightly altered "A" to get you to pay more, so call your doctor and ask them if you can take "A."

<comment><USER7> The pharmacist should do that, but they don't.

I've done it before at a doctor. "Here is the latest drug!!". "Can we start with a generic?", "Sure".

My co-pay just went from $50 per month to $5.

I work with a lot of doctors, they don't think it's their job to save patients money. They'll do it if asked though.

<comment><USER20> I find that kind of astounding. In the UK you pay a flat fee per prescribed medication, and multiple times I've had doctors prescribe me larger amounts on my repeat prescriptions so that I have to pay the flat fee less often - completely unprompted by me.

<comment><USER7> Docs will do that in the US too sometimes, but one challenge is that the "fee" you owe with private insurance differs from plan to plan. Doctors can't track the hundreds of different plans, so they don't know if the more expensive branded drug will cost you $10 more per month or $100 more per month. If you share that info with your doctor they are more than happy to work with you.

<comment><USER19> I'm still confused.

Isn't it simply a matter of asking, "Is there a generic for this, doc?"

Both $10 and $100 are greater than $0, the patient almost certainly always wants $0, and a doctor can do that math without referring to your plan.

Or is this a case of this particular method being combined with N other methods such that the doctor has trouble getting the correct information to the patient?

Edit: By $0 I mean "$0 more than the cheapest generic." :)

</comment></comment></comment></comment></comment></comment></comment><comment><USER21> I'd go a step further... you cannot sell a drug unless there is another company prepared to sell a generic version. It's a national security issue. Dual vendor required.

Also, no blocks on re-imports from foreign nations.

<comment><USER7> So under your system, a drug for a super rare, low sales product would never get produced because no generic company wants to make it?

<comment><USER22> Under a pure market approach, if a drug is not profitable it will not be produced ans sold. If you have a rare disease, you would probably prefer a mixed approach like a mix public and private medical research.

<comment><USER7> There is a middle ground where it's profitable for one company to make a drug, but not profitable for two companies to do it.

<comment><USER22> I agree with you and in a pure profit oriented you will always have rare diseases without remedies. You may even have curable patient that are non- economical to save for for-profit health-care company.

Imagine you have only pure market that provides 80% total availability for disease remedies. If you want to cover the 20% uncovered case by bringing a non-profit player, you may threaten some of the pure market player. At the end you may have a 60% coverage by for-profit and 40% by non-profit.

The end solution may seems to be less economically efficient but it covers more of the population and I think it is a good thing.

</comment></comment></comment></comment></comment><comment><USER23> The title posted here makes no sense at all. It's not the original and it has nothing to do with the article. The moderators should really change it as it implies somehow that the high cost of drugs is the fault of the American people, not the horrible patent laws that we have no control over or the actual drug companies exploiting ill people for mass profits.</comment>


Github Issues Review Chrome Extension – collaborative peer reviews - <AUTHOR> https://github.com/micho/github_issues_review_chrome_extension

<comment><AUTHOR> Quick links here:

\- Download: [https://chrome.google.com/webstore/detail/github-pull- reques...](https://chrome.google.com/webstore/detail/github-pull-request- revie/almhmmbcihjjadclalgeejgllhboddgp)

\- Screenshot: [https://a248.e.akamai.net/camo.github.com/422f12e0eb5db07e3b...](https://a248.e.akamai.net/camo.github.com/422f12e0eb5db07e3ba8dae399cc0cf78a8ea32c/687474703a2f2f636c2e6c792f696d6167652f3335314f31333030313831302f636f6e74656e74)

</comment><comment><USER2> One thing that seemed missing from the README was a mention of what persistence method is used for this. I'm guessing it's using the Chrome extension shared storage mechanism, but people who aren't familiar with extension development probably wouldn't be able to guess that.

</comment><comment><USER3> Nice. Similar idea to this one a colleague built. [https://chrome.google.com/webstore/detail/looks-good-to- me/h...](https://chrome.google.com/webstore/detail/looks-good-to- me/hbiicpfmjknedfbhgpbckmmnbnmjknfo)

</comment><comment><USER4> Like the idea!</comment>


The world's first quantum software superstore–or so it hopes–is here - <AUTHOR> https://www.technologyreview.com/s/611139/the-worlds-first-quantum-software-superstore-or-so-it-hopes-is-here/ <comment><USER2> Can it actually be proved that such a store exists?

Or it can be proven to exist now, can its growth and revenue be plotted?</comment>


MY WEBSPOT – access to unlimited mobile internet in 4G everywhere in Europe - <AUTHOR> http://my-webspot.com

<comment><USER2> Most providers in most European countries sell pre-paid sim cards for data. In general something like Euro 10 for one Gig.

Personally I like a fee per bundle more then a fee per day. Some days I only briefly check my email, the next day something happens and I have to work extensively.

I came across this very informative website: [http://prepaid-data-sim- card.wikia.com/wiki/](http://prepaid-data-sim-card.wikia.com/wiki/)

</comment><comment><USER3> What a load of bullshit. Not only is it expensive compared to local prepaid data SIMs, they flat out lie in their marketing.

Unlimited turn out not to be unlimited, but limited to 10 GB per month.

Coverage map show Russia, but small print says does not cover Russia.

DIAF</comment>


Dragonfly: Alibaba P2P file distribution system - <AUTHOR> https://github.com/alibaba/Dragonfly <comment><USER2> Container image distribution seems to be one of the primary problems this tackles:

"DevOps ... brings a lot of challenges: the efficiency of image distribution, especially when you have a lot of applications and require image distribution at the same time. Dragonfly works extremely well with both Docker and Pouch, and actually we are compatible with any other container technologies without any modifications of container engine."

FWIW, this was a similar problem that I tackled for Golang gopher gala hackathon 2015 - a custom bittorrent based docker image registry POC.

[https://github.com/netvarun/docket](https://github.com/netvarun/docket)

Interestingly my problem statement was somewhat similar:

"Large scale deploys are going to choke your docker registry. Imagine pulling/deploying a 800mb base image (for example the official perl image) across 200 machines in one go. That's 800*200 = 160GB [EDIT: Correction thanks to kingbirdy] of data that's going to be deployed and it'll definitely choke your private docker registry (and will take a while pulling in from the public image)."

<comment><USER3> Devs need to be aware that bittorrent now(for sometime) have a DHT solution that allows to have "mutable" slots with the crypto public address tied up to it.

So if you own the private key you can write the payload, and just share your public-key address to the people you want to share the payload with. In the payload you can write a traditional immutable torrent manifest for instance, which is in essence a public-key crypto based update system.

For a lot of cases i think it's a better approach than what IPFS and DAT provides, because you dont care about it being a global address. All you want is to share with a group of people, more in the p2p social/organic way.

I was playing with it once, using the libtorrent library and the main bittorrent DHT, and it was a very nice experience.. it finds the payload pretty fast when you think its a DHT, and you are working in pure p2p fashion.

The only single point of failure here is the DHT bootstrap peer.

Im planing to use this feature to distribute binary images for clients that have my public key.

<comment><USER4> Very interesting. Forgive my ignorance, but you mean it's kind of like mutable torrents that only the uploader can modify?

Does a client just search the DHT for the public key? I thought torrent clients searched for the hash of the files.

If it's searching for the public key then how does one person upload multiple different torrents, or do they create a new public key for each torrent? How does a client know which is the latest version if it has been updated multiple times?

Are there any example projects using this?

<comment><USER5> > Are there any example projects using this?

The only project I know is gittorrent ([https://blog.printf.net/articles/2015/05/29/announcing- gitto...](https://blog.printf.net/articles/2015/05/29/announcing-gittorrent-a- decentralized-github/)), but it hasn't gone anywhere.

</comment></comment></comment></comment><comment><USER6> Ok, this is huge.

>At Alibaba, the system transfers 2 billion times and distributes 3.4PB of data every month, it has become one of the most important piece of infrastructure at Alibaba. The reliability is up to 99.9999%.

I took at look at their repo and it turns out there are surprisingly lots of good stuffs in it which never gets much spotlight or attentions.

<comment><USER7> 3.4PB/month is only 111,000 GB/day. That's not particularly huge in this day and age, except to consumers.

<comment><USER8> I imagine this traffic is fairly bursty as well.

Also keep in mind this is a data distribution system. In the case of large data pushes, new builds, etc. it's important that all peers get the new data on a timely and reliable manner.

I suspect you're underestimating the problem this project solves by focusing on a mostly irrelevant data rate stat.

</comment></comment></comment><comment><USER9> Quay has supported Docker image pulls over BitTorrent for a couple years now [https://coreos.com/blog/torrent-pulls](https://coreos.com/blog/torrent- pulls).

Docker themselves have discussed making the official registry extensible enough to support BitTorrent pulls, but I don't know if anything ever happened there.

Facebook has been using BitTorrent for deploys for something like 9 years now. They configured the tracker to prefer sharing peers with longer matching subnet prefixes, to keep bandwidth off the backbone as much as possible.

<comment><USER10> Is there a paper or talk you could link to that describes their system?

<comment><USER11> Justin linked the blog post[0] which is probably the best written description. The short of it is that when you upload layers to Quay, it stream calculates the BitTorrent pieces. Private layers are given unique swarms isolated by namespace and peer discovery is protected by a tracker[1] that contains middleware validating JWTs passed in the announce URL of the torrent. A custom client[2] can be used to simplify downloading and importing of images into the local docker CAS.

Honestly, most organizations don't have sophisticated enough networks that the benefits outweigh the complexity of p2p orchestration. This is why it's popular at Alibaba, Facebook, Twitter, but most people are still just using the OCI distribution protocol[3].

Feel free to contact me (Keybase is in my profile) if interested. I'd love to get more people on the path to p2p, but it's often a solution looking for a problem.

[0]: [https://coreos.com/blog/torrent-pulls](https://coreos.com/blog/torrent- pulls)

[1]: [http://chihaya.io](http://chihaya.io)

[2]: [https://github.com/coreos/quayctl](https://github.com/coreos/quayctl)

[3]: [https://github.com/opencontainers/distribution- spec](https://github.com/opencontainers/distribution-spec)

</comment></comment></comment><comment><USER12> From: [https://github.com/alibaba/Dragonfly/blob/master/src/README....](https://github.com/alibaba/Dragonfly/blob/master/src/README.md)

\- supernode(Java)

\- dfdaemon(GoLang)

\- getter(Python)

Interesting distribution of languages in what seems to be a somewhat self- contained project.

Oddly, [https://github.com/alibaba/Dragonfly/tree/master/src](https://github.com/alibaba/Dragonfly/tree/master/src) only contains the getter and the supernode at the moment

<comment><USER13> Looks like dfdaemon is at the root: [https://github.com/alibaba/Dragonfly/tree/master/dfdaemon](https://github.com/alibaba/Dragonfly/tree/master/dfdaemon)

</comment></comment><comment><USER14> This is very similar to what BitTorrent does or am I missing something?

</comment><comment><USER15> The amount of p2p-ness in it is not any much more than that of any DFS of previous decade.

The P2P slang though is freaking everywhere here. There are P2P bank and P2P brand sausages.

<comment><USER16> Not quite, this is not a "distributed file system" this is a "file distribution system".

This is more along the lines of [https://github.com/lg/murder](https://github.com/lg/murder)

</comment></comment><comment><USER17> How does this compare to IPFS or BitTorrent?

<comment><USER18> or Syncthing

</comment></comment><comment><USER19> Sounds like Twitter's Murder tool from 2010:

[https://blog.twitter.com/engineering/en_us/a/2010/murder- fas...](https://blog.twitter.com/engineering/en_us/a/2010/murder-fast- datacenter-code-deploys-using-bittorrent.html)

<comment><USER20> Looks like its not maintained anymore

</comment></comment><comment><USER21> Anyone knows if it does NAT hole punching? I'm interested in such a tool for deploying to remote machines.

<comment><USER22> If you only need such a tool, other comments on this submission linked ways to use bittorrent with docker, and µTP [0] seems to be reasonably good at punching through the good old style of NAT, where ports are sequential and on the same IP, with something that can coordinate accessible to both. It also enables gentle use of your bandwidth, in the sense of playing reasonably well even if you don't have fq-codel or similar in use on the router. With somewhat nice networks it can be pretty gentle on other users of the networks, without wasting any part of it. Do consider QoS though, it is preferable to send other, important traffic first, as µTP is good at backing off in these cases. The latency in backing off is just a little high to be stealthy towards concurrent TCP connections. Packet loss is rare, but lag spikes are still a nuisance.

[0]: [https://en.wikipedia.org/wiki/Micro_Transport_Protocol](https://en.wikipedia.org/wiki/Micro_Transport_Protocol)

</comment></comment><comment><USER23> Is Dragonfly a server or client? Cause it is compared with wget which is a client. Am I missing something?

<comment><USER24> It's P2P, so I would wager it is both.

<comment><USER25> It duplicates most features of libtorrent.org; but requires a server. Therefore, a comparison to wget misses the 17 year old Bittorrent protocol.

<comment><USER26> Exactly. They should be comparing their P2P protocol to the whatever is currently the best of open-source, P2P libraries.

</comment></comment></comment></comment><comment><USER27> Also a bit confusing as Dragonfly BSD is known for it's custom filesystem, HAMMER:

[https://www.dragonflybsd.org/hammer/](https://www.dragonflybsd.org/hammer/)

<comment><USER24> If you get confused between a filesystem called HAMMER, and a file distribution system called Firefly, I don't know what to tell you.

They are different things with different names.

<comment><USER28> Well, this one is called Dragonfly, not Firefly, so the naming overlap is relevant.

<comment><USER29> Do you also get confused by Dragon Naturally Speaking and Firefox and their similarity to those 2 names?

<comment><USER28> These are literally two things with the name Dragonfly. I don't particularly care about whether the shared name is confusing here but it is disingenuous to pretend that identical names are merely similar.

<comment><USER29> Dragonfly and Firefly are not the same word.

<comment><USER28> One of us is clearly missing something here. The two things I'm talking about are Dragonfly BSD and Alibaba's Dragonfly file distribution system. Neither of these are named Firefly.

</comment></comment></comment></comment></comment></comment></comment><comment><USER30> Jesus, is it so hard to google at least once before naming a project? Just recently that Sonar thing now this. What's next? A Windows editor called Linux?

<comment><USER30> To be fair, I've to admit that Dragonfly BSD doesn't show up on the first result page when you search for "dragonfly". What does show up is the malware by the same name.

Side note: this also shows how simplistic Google search really is. No way to search for "Dragonfly /computers/" opposed to "Dragonfly /nature/" with the terms in slashes denoting a concept or domain instead of a syntactic element.

<comment><USER31> eh?

dragonfly +nature

dragonfly +computer

Though i don't blame you for not knowing, out of 4-5 operator cheat sheets and guides i only see mention of the "-" to exclude terms.

<comment><USER30> "+" is for synonyms. That might lead in the right direction but it's not the same.</comment></comment></comment></comment>


Ask HN: A large community owner is trying to strong-arm us. What should I do? - <AUTHOR> I am the founder of StartGlobal https://startglobal.co/ where we help anyone incorporate a business in the US, get EIN, bank account, phone number, and address. Recently we've been experiencing delays from IRS to get the EIN. IRS was shut down and they are slowly getting back to speed.
Even though it was not our mistake, I think it was our responsibility. At first, I tried to design some beautiful logos for the client to keep them happy. By this time we already finished the incorporation process at the state and registered agent.
When the EIN delay was going beyond expectations, I agreed to do a full refund without taking any cut (after issuing a full refund, we were at a loss because we already paid for registered agent, filings, state fee, even out of pocket logo design). I then sent the following email taking the blame and issuing a full refund.
I've issued a full refund, you should have an email with details. You may keep the fully incorporated company, registered agent service paid for 1 year, and the EIN application set to IRS and the logo we did. I am really sorry you had to go through this experience, and greatly appreciate your patience until now. I truly understand your situation and I'd be no different. But please be aware that this was not the experience I hoped for, and I hope you forgive me for wasting your time.
Now after the refund, the client asked us for some edits in the logo and also to work again on the EIN followup with IRS and see if we can get it, and we did it on a good faith that client might pay after we get it. After many followups with the IRS, we were able to finally get the EIN and when I asked the client if they'd like to continue the process and pay us, he is saying he'd like the EIN for free because I "promised" to get him everything for free. He is a founder member of a large startup community and being an early-stage company I am afraid if this will affect our reputation if we don't do this. What should we do? <comment><USER2> If you issued a full refund, you are not obligated to do anything more for them. You already accepted responsibility by issuing a full refund even though the IRS delays are unprecedented and your client needs to understand that. I would say "We are happy to continue the work but we need to re-establish formal relationship again through a new contract/payment in order to ensure we are able to provide quality service to you".

If they don't like it, you can tell them to go somewhere else politely unless you are at their mercy and begging for their business. A client that takes a full refund AND then still expects free work is not a client you want to work for. They will continue to make your life worse.

<comment><AUTHOR> Thanks for the input! I am trying to see how we can solve this. I offered a $200 discount (our total price was $599) and they were still not ready to accept.

</comment></comment><comment><USER3> You need to add in a 'force majeure clause' to your contracts.

The IRS issue was not your fault. There was NOTHING you could do about it. EIN number, was probably the easiest part of setting up the company. Value ... maybe a 10% refund.

I think your taking the blame, was not a good business practice. Explain what happened, offer a discount, or offer a logo, or cancel everything (eating expenses, but not giving it away free).

Write this off as a lesson learned.

<comment><USER4> I completely agree with this. Taking 100% blame for something you had 0% control over was a bad move. Your intentions were good but as sloaken said "maybe a 10% refund".

The issue you have now is that this client has been trained to walk all over you and expect everything they ask for. Not too dissimilar to that of a dog that learns to cry for table scraps.

The bottom line is that your clients come to you because they need you as much, if not more than, you need them. You don't have to throw it their face but it's extremely useful to remind yourself of it from time to time as it will help you take up for yourself when, and hopefully before, these types of situations arise.

</comment></comment><comment><USER5> If the premise of your company is to make it easier for people to start a company, I'd say just do this one for free. If they tell others that it was a still a lot of back and forth then it people might not think that it's worth the extra money to start the company. I set up my own LLC and it was really pretty cheap and easy (the sales tax system is another story).

</comment><comment><USER6> If you promised it deliver it.

you're not getting paid because you offered to do it for free. chalk it up as a life lesson. sales 101.

going forward, you need to position your service for what it is. Dealing with government BS so the client doesn't have to. and set that expectation early there will be problems but that's what your there to deal with(for a price). You need to explain how much pain and frustration your saving them.

And if they can't appreciate what you are doing for them you need to find other customers.

<comment><USER6> I know one thing I would like a button to close a company down. I still get letters from the state from all kinds of agencies I have know idea about. quarterly reports I didn't file.

<comment><AUTHOR> Thanks for the input. We are trying to figure this out now. It should be easy to start a company, but more easier to close one down.

</comment></comment></comment><comment><USER7> I know this is obvious, but it's important to look at your online agreement or terms of service for customers to see what they've agreed to.

If you're not legally culpable, I don't see the reason for the refund. The fact is most people, especially in the US, will understand that delays caused by the government's servers are not at all unusual, and they might just side with you. Perhaps you're concerned about social dislikes before you've even gotten your startup off the ground. I find communication is really the most important thing; if there are few enough customers to contact, I'd call every one of them; it may see like a waste of time, but they're really key to your business, and having the founder contact them may bring tears to their eyes.

We are dealing with human beings, and so-called disasters could be turned around to super-power you. Remember that Intel's biggest growth was when there was a rare problem with their pentium chip, and they fixed and returned every one, and made a huge marketing campaign around it; if you have already refunded, you should make sure to let everyone know about it, and reach out to customers, perhaps via email directly. Testimonials are a really big deal in the social world, and every one of them could be your biggest advocate for lots of new customers.

Regarding what you're apparently most worried about, the bigshot late stage startup, that wouldn't be a concern to me. Some bigshot, who can't afford an EIN number; rmember what YC awards on here, it's called karma, and it goes around. If they try to injure you, they may have a lot more to worry about when you tell your side of the story, especially if they are bigger.

Let me tell you a true story that happened to me just today. I had an investor on my podcast some weeks ago. This person seems really nice, and has a huge social following, so I offered something that really goes against the way we operate, to do some free legal work for this person, so they could view our work, and if they were satisfied, to establish a regular working relationship together.

A few weeks later, this person reached out to take me on my suggestion (and was actually mad that I hadn't contacted the person earlier, but apparently the email had ended up in spam) and not only did we do the work namely fixing a really bad contract bringing a ray of sunshine to it (sorry for the braggadocio), and they kept all the changes, but I did a full video of explanation for this person and their partner.

Yesterday, they reached out to me again, this time stating that we could work together but that their partner is cheap, so watch the fees, then telling me how much we could charge, and when we mentioned our customary retainer for this sort of work, refused but did mention they needed the work done right away.

My response was to say sorry, we can't do it, and send them a polite email, and fortunately as founder like you I don't have to answer for it. I'm just letting you know about own experience, acquired over decades.

This is my hugely wordy way of saying to stay away from time wasters, big- shots or otherwise, and put your time into what counts. I've seen lots of folks like this, and all they do is waste your time, because you've got finite time, and let it be spent with the good customers, who love you and want to work with you. The 'big shot's' power is mostly a perception, and most people are fair minded. Don't give into it.

My only caveat is that I don't know the exact circumstances and I'm taking your word for it.</comment>


.gay Generic Top Level Domain(gTLD.) is now open for public - <AUTHOR> https://icannwiki.org/.gay <comment><USER2> I preordered [https://areyou.gay](https://areyou.gay) a while ago.

I don't know what to put on it, but I would like to host a FAQ or something for people who are questioning their sexuality and make it v wholesome.

<comment><USER3> I think you should transfer the domain to a gay rights org or similar because this type of domain squatting is obnoxious. I mean, you didn't even have a plan for the domain, just registered it for the novelty.

<comment><USER2> Isn't the open internet great? I can register a domain to use for my own purposes so I can also have a voice.

<comment><USER4> You don't have a voice that's why you're asking here. So someone gave you a good suggestion and you shit on it. You should let it expire or give it to a group who actually wants to do something with it.

</comment></comment></comment></comment><comment><USER5> I want to see whether Google will register google.gay

Will they sue me for registering it?

<comment><USER6> OTOH, you can probably get away with apple.gay, as long as you don't make your site too specific to apple computers.

<comment><USER7> Too late, already registered on February 19th.

</comment></comment></comment><comment><USER8> I'm not a native speaker; does gay unequivocally relates to romantic preference? Or can it also simply mean "joyful"?

<comment><USER9> There's still a distinct meani g of happy. For some reason, though, people are trying to stamp it out; I hardly hear it used in new writing.

<comment><USER7> It's not remotely as nefarious as that, it's just that language evolves over time and conflicting meanings that cause confusion tend to be weeded out.

</comment></comment></comment><comment><USER10> .gay sounds like a great way to champion gay rights for a given place or brand. I hope it will become a typical pattern and actually do some good in the world.

<comment><USER11> In reality, and like most things these days, this will only promote more division and more identity politics.

</comment></comment><comment><USER12> Just checked and yes, enola.gay is registered. That was inevitable, I suppose, if a bit regrettable. (Edited for clarity.)

</comment><comment><AUTHOR> Official Website: [https://www.ohhey.gay](https://www.ohhey.gay)

<comment><USER7> You should add some context that this is the official promotional site for the TLD by the registry operator.

</comment></comment><comment><USER13> If your name is Gay, iam.gay would be very cool. MAybe I'll change my name by deed poll.

</comment><comment><USER14> Someone should buy Haaaa.gay and just embed:

[https://www.youtube.com/watch?v=YaG5SAw1n0c](https://www.youtube.com/watch?v=YaG5SAw1n0c)

</comment><comment><USER15> At last! What I've always wanted.

</comment><comment><USER6> Before anyone gets any ideas:

>The use of .gay for anti-LGBTQ content or to malign or harm LGBTQ individuals or groups is strictly prohibited and can result in immediate server-hold. Prohibited behavior includes harassment, threats, and hate speech. For the complete policy, see: [https://toplevel.design/policy](https://toplevel.design/policy)

<comment> globular-toast Who makes these rules? Who decides what is "anti-LGBTQ"? Will they keep adding on more letters to LGBTQ as they become available?

<comment><USER16> One of these questions is not like the others

<comment> globular-toast The point of the "more letters" is probably not what you think, but rather what if the entity in charge decides to add a new one (say, X) but it turns out the L, the B and the T now feel alienated by the X? Or, conversely, what if they decide _not_ to include some new group (say, P)? It just seems rather nebulous and down to the whims of some mysterious entity with unclear motives.

<comment><USER17> What are you even on about?

There is no "entity in charge" of the LGBT acronym, no central authority approving, adding or removing letters, or any particular concern over "alienation" from adding or removing letters. It's a cultural idiom, not an ISO standard.

Regarding its "nebulous" nature and "motives," quoting from Wikipedia[0]:

        The initialism, as well as some of its common variants, have been adopted
        into the mainstream as an umbrella term for use when labeling topics
        pertaining to sexuality and gender identity.

        The initialism LGBT is intended to emphasize a diversity of sexuality and
        gender identity-based cultures.

Note that, while the quote mentions "LGBT" specifically that description also applies to the "common variants" also described, including LGBTQ, LGBTQIA, and others mentioned elsewhere in the article. I only point that out because one of your flagged comments mentions how alienated and confused you are by "the whole gay thing," so I wanted to be as clear as possible.

And if you're instead talking about the registrar, they're not a mysterious entity, and their motives are clearly spelled out on their policy page[1].

[0][https://en.wikipedia.org/wiki/LGBT](https://en.wikipedia.org/wiki/LGBT)

[1][https://toplevel.design/policy](https://toplevel.design/policy)

<comment> globular-toast Erm... I'm talking about the entity in charge of the TLD.

<comment><USER17> I still don't understand what exactly your concern is. The TLD doesn't even contain any elements of the LGBT acronym, yet you seem deeply concerned about them altering it willy nilly and this having some widespread negative effect on the gay community.

This despite going through the unnecessary effort of making an entire, completely off topic top level comment announcing how confused and alienated you were by the gay community and how you wish they would just stop being so visible so you didn't have to think about gay sex all the time.

I mean, I'm sure the gay community appreciates your concern and apologizes for the inconvenience, but it seems like you're trying very hard to start a tempest in teacup without even any tea to stir. Don't worry, the gays will be fine.

Now relax and enjoy some Scissor Sisters: [https://www.youtube.com/watch?v=HHv0jW4p_xA](https://www.youtube.com/watch?v=HHv0jW4p_xA)

<comment> globular-toast > Now relax and enjoy some Scissor Sisters: > [https://www.youtube.com/watch?v=HHv0jW4p_xA](https://www.youtube.com/watch?v=HHv0jW4p_xA)

Going out of your way to offend me. Nice. "Tolerant".

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER18> nvidia.gay is up for grabs at 2k euros. I wonder if they'd buy it off you.

<comment><USER19> They will file a trademark claim against you under ICANN UDRP[1]. These are arbitrated so they're cheap to file and are very quick to resolve (compared to courts).

[1] [https://www.icann.org/resources/pages/help/dndr/udrp- en](https://www.icann.org/resources/pages/help/dndr/udrp-en)

<comment><USER18> Thanks, although my example was none too classy, I had no idea what is actually done in such cases.

</comment></comment></comment><comment><USER20> How about .straight and .bisexual?

<comment><USER21> The community for straight people is too disparate to be meaningful. .bi is interesting though..

<comment><USER22> Two-letter codes are generally countries. In this case, Burundi.

</comment></comment></comment><comment><USER23> hxxp://ni.men.gay/wo?na-gar

</comment><comment><USER5> [https://read.if-you-are.gay](https://read.if-you-are.gay)

<comment><USER24> Well that'd be so funny... NOT!

</comment></comment><comment><USER25> Why is it .gay and not .lgbt? Seems like the latter would be more inclusive.

<comment><USER26> The company that runs it also does other domains like .art, .blog, .design, .group, .ink, .llc, .photography, .style and .wiki.

I'd say they just picked the most generic thing that they thought would make the most money.

</comment></comment><comment><USER7> Wow there's a lot of childish homophobia coming out in the comments here. I (naively?) thought that HN would be better than that for some reason?

<comment><USER27> Thinking that HN would be better than <insert any bad behavior here> is a bit too hopeful of an assumption. Everyone is prone to bad / ignorant behavior. HN users being no exception.

<comment><USER7> I disagree. Some communities are actually much better than others on a variety of issues. I mistakenly thought that this was an issue HN would have been better and more mature on too, but I'm learning I was wrong about that.

But compare your average 8chan poster to your average HN poster and you'll see that for sure not all online communities are the same.

<comment><USER27> I don't even know how to argue on this. The average HN poster is definitely better than the average 8chan poster.

But that does not mean all HN posters are free of being hateful/discriminating against certain groups of people. Be it by accident or willfully.

<comment><USER7> I for sure never thought that all HN posters were free of hate/discrimination, I just thought they were better than we're seeing here. But I was proven wrong in that.

</comment></comment></comment></comment></comment><comment><USER9> Sadly, thatsso.gay is akready taken. That would have been a really funny domain to have.

<comment><USER28> Why?

<comment><USER9> Because it's generally recognized that calling something gay is funny and a way to insult it, at least here in America. Is that not the case in other places? I've heard at least some of it in south and central america, though it tends to be taken more personally there than in

</comment></comment></comment><comment><USER11> This opens the flood gates for TLDs for every community under the sun (as long as it is PC, of course). There is gold at the end of the rainbow... But really this trend of multiplying TLDs _ad infinitum_ has been going on for years because it's free money.</comment>


Beat Detection Algorithms [pdf] - <AUTHOR> http://www.flipcode.com/misc/BeatDetectionAlgorithms.pdf

<comment><AUTHOR> Just finished up a hackathon doing a JS port of a dumb little flash images-to- beat-of-music app.

While I had fun with the WebAudio API (sorta...), the part about beat detection straight up kicked my ass.

For the curious, some other things I've seen on the topic:

[http://tech.beatport.com/2014/web-audio/beat-detection- using...](http://tech.beatport.com/2014/web-audio/beat-detection-using-web- audio/) isn't bad, but leaves out some implementation details that will cost you a little time.

[http://stackoverflow.com/questions/657073/how-to-detect- bpm-...](http://stackoverflow.com/questions/657073/how-to-detect-bpm-of-the- song-by-programming) had a few interesting links.

[https://www.clear.rice.edu/elec301/Projects01/beat_sync/beat...](https://www.clear.rice.edu/elec301/Projects01/beat_sync/beatalgo.html) has some Matlab implementations.

[http://www.ee.columbia.edu/~dpwe/papers/Laro03-beattrack.pdf](http://www.ee.columbia.edu/~dpwe/papers/Laro03-beattrack.pdf) was a decent paper, but I was too fried to make progress with it.</comment>


IBM and University of Alberta use AI to predict schizophrenia with 74% accuracy - <AUTHOR> http://betakit.com/ibm-and-university-of-alberta-use-ai-to-predict-schizophrenia-with-74-percent-accuracy/ <comment><USER2>
      The study included a total of 95 subjects — 46 patients with schizophrenia, 49 without

... and those are the only numbers given in the article.

It must be fun to be one of those falsely diagnosed as schizophrenic.</comment>


Ask HN: What's your password strategy - <AUTHOR>

I like to rotate my passwords, and I like to have a strategy to do so. I don't use the same password for every website but I like to use a derivation of a master password.  I personally would love to be able to use randomly generated strings from 1Password, but the issue is portability. Currently, if I do that, anytime I want to log on anywhere at school or another public computer, I have to log in to dropbox(provided that I can remember that password) and use the 1Password Anywhere feature which just isnt feasable. So here's what I do: pick two numbers, either randomly or meaningful, or pick a longer number and split it in half. Take two words that you'll be able to remember. Add the first two letters of the website for that password on to the end of the two words. Bookend this string with the two numbers and finish it all off with a special character.<p>Do you have any better suggestions for me? <comment><USER2> I use LastPass. I have absolutely no knowledge of what my passwords even are for 90%+ of the services I use. A few daily-use ones i've just struggled through memorizing the generated passwords. Mostly though I use the Chrome extension that contains all my passwords.

</comment><comment><USER3> I have a single password that I use to sign into a bunch of sites that don't require any real information about me associated to my throwaway/spam email acct, that I wouldn't care if it got hacked.

I also have my important passwords for which I use a combination of letters, numbers and special characters which consists of my base password. I then add another string to the end such as a word or zip code, which I rotate every 2-3 weeks depending on where I login and whether or not I feel that connection is secure.

Haven't gotten hacked, yet... we'll see.

</comment><comment><USER4> I use a master password for all the services I like to login to in to remotely (google, facebook, dropbox, etc), for the rest of the services I use 1Password. It's not the most secure, but it's the practical imo. If I happen to get hacked, I only have to change the passwords for the few services I login to remotely.

</comment><comment><USER5> The problem of logging in at public computers will decrease after you leave school. I am content with 1Password.</comment>


Just so you know: I've picked up maintenance of django-annoying. - <AUTHOR> https://github.com/skorokithakis/django-annoying

<comment><AUTHOR> Hey everyone!

django-annoying is a very useful project, as most of you Django developers will know. Unfortunately, the developer is nowhere to be seen, so it was languishing (even not working with 1.5).

I managed to pick up development/maintenance and moved it to github (I also issued the first new release in years on PyPI), so if you want to report any bugs or issue pull requests, that's its new home.</comment>


Ask HN: Is SQL a primary tool of choice for doing ETL pipelines in 2019? - <AUTHOR> Is SQL considered a primary tool of choice for ETL pipelines in 2019? Benefits/drawbacks? <comment><USER3> I've used and created numerous ETL stacks. So learn from my mistakes.

First, move the code, not the data. Batch processing is for mainframes. I know, I know, this has been impossible to realize any where I've ever worked.

Second, less is more. If command line tools work, use 'em.

Avoid IDLs, maps, schemas, visual programming, workflow engines, event sourcing, blah blah blah. It's all useless abstractions, measured by number of indirections and stack trace depth. It's all wicked hard to debug. It's all abandoned unmaintained obfuscation layers.

Data processing (ETL) is just cutting and pasting strings. Input, processing, output. Sometimes with sanity checks. Sometimes with transformations, like munging date fields or mapping terms ("yes" to "true"). Very rarely with accumulators (aggregators) where you need some local persistent state.

Third, and this is pretty rare, use better APIs for data extraction. It's all just scrapping. Don't over think it. I wish I could show the world the APIs I created for HL7 (healthcare) data. For 2.x, I created "fluent" (method chaining) data wrappers (like a DOM) which could not blowup (used Null Objects to prevent null pointer exceptions). For 3.x, I used path query thingie to drill down into those stupid XML files. This was CODE, not mappings, so it was practically a REPL, meaning fast to code, fast to debug.

Fourth, you control the execution. Be more like Postfix/Qmail, where each task has it's own executable. Be less like J2EE, BizTalk, where you ask the runtime to control the lifecycle of your code.

Good luck.

<comment><USER4> I'm really interested in learning more about the work you have done with HL7 v2/v3. I have never really thought of HL7v2 messages being document oriented. Does the data wrapper allow you to access PID 3(1) will give you the first external ID(full CX) and PID 3(1).1 will give you just the ID?

<comment><USER3> Yes.

Even better, for 2.x I had an "HL7 interface" compiler that would transform the client (partner's) specification into verbose wrappers. The resulting "fluent" code read just like the spec.

Alas, data quality, consistency was always a challenge. Data from multiple sources would come thru the same feed. So often our scrapping (data extraction) code would extract PID (for example) from 3 common places, from most specific to more general, then use null coalescing to choose the best match.

As hinted before, 3.x & XML was handled the same in spirit, but very different code, using query expressions instead of method chaining.

I'd be happy to share whatever I have.

Here's a temp (24 hours) email address if you wish to PM me: email@example.com

</comment></comment></comment><comment><USER5> There is no on size fits all tool for good ETL processes IMO.

We just built a pretty involved ETL process over the past couple of months that had two key components, a true batch ETL process and then a real-time synchronization between two Postgres databases on different servers. In our case, real-time is basically less than 2 seconds for this application so nothing insane, we routinely see it working in about 300-500ms.

We could've solved it all using just SQL to be fair, but found that it was better even for the batch ETL process to utilize SQL only for the data CRUD and leave the business logic to the application code we wrote (it is in nodejs). This let us take advantage of some caching which reduced the database server load dramatically during the processing, id lookups and other known values being the core area we reduced with cache. It also let us keep our business rules in code which is where we have them for data validation etc, so we could reuse existing code sections to maintain consistency and not try to replicate some pretty sophisticated validations and transforms in SQL.

I have also worked on projects in the past where the entire process was done using SQL and it is doable but I'd say it was less than ideal and definitely can cause significant overhead on the DB servers which is unnecessary most of the time.

I had one project where the entire ETL process was in MSSQL with stored procs and SSIS, spread across multiple servers. That system worked really well overall (integrity was super good and reasonably maintainable for SQL people), but again strained the DB servers a lot and was fairly slow. We moved most of that ETL process to C# at the time given it was a MS house, and the difference was dramatic. Load on the DB servers dropped and ETL that would take hours to run on SSIS with stored procs etc took minutes with C# and the utilization of memcached. In fact on that one we even wrote extensions for SQL Server using .NET and had SSIS call our external code which used the caching engine to reduce DB lookups amongst a bunch of other things. Pretty damn cool solution but definitely wasn't simple, lots of smart people worked on that project to make it super reliable and fast.

</comment><comment><USER6> SQL has definitely become the defacto tool for a lot of data processing. This model of working is generally referred to as ELT as opposed to ETL.

For small/medium scale environments Fivetran/Stitch with Snowflake/BigQuery using getdbt.com for modelling is an insanely productive way to build an analytics stack. I consider this the default way of building a new data stack unless there's a very good reason not to.

For larger scales Facebook has Presto, Google has Dremel/Procella/others, and a lot of data processing is done using SQL as opposed to writing code.

The only downside really is that it tends to be fairly focussed on batch pipelines (which are fine for 95% of workloads). But even that is becoming less of an issue with Beam/Spark so you can use SQL for both batch and streaming.

Source: Solution Architect at an analytics consultancy.

</comment><comment><USER7> Coming from a Big Data (Hadoop / AWS) background, I would say that SQL is a good choice for analyzing data once it is in the data warehouse, but it is not needed to get the data there. Most of the ETL code in this environment is based on Apache Spark 2.x and either Scala or Python as programming language. On a Hortonworks HDP cluster, you would use PySpark directly; on AWS, you would use AWS Glue which allows you to set up and schedule the PySpark code. Although PySpark does not stop you from using SQL, there is a preference to write the transformations in terms of native Spark functions. So, to answer the question: no, SQL is not a primary tool of choice for implementing ETL pipelines in 2019 - PySpark is.

</comment><comment><USER8> Generally yes. You can use gui tools like Matillion or AlteryX to orchestrate, or you can just run SQL as commands or view materializations. With the advent of natively scalable SQL based databases such as Snowflake, SQL is clawing back territory from the 'Big Data' databases rapidly. It is way more flexible to throw a ton of hardware at SQL as opposed to pre-conforming your data for specific usecases with non-relastional databases. Obviously if you're talking realtime or sensor data SQL may not be your best strategy, but you're question is really broad.

</comment><comment><USER9> I've been doing ETL-heavy work as (async) Python microservices.

The most important thing is to separate your infrastructure layer from your business logic, and have the infrastructure layer satisfy interfaces that are defines in business logic layer.

That way, when my datastore switches from FTP to S3 or MySQL to Postgres, I can swap out the datastore client, make sure the new one satisfies the interface, and the rest of the application works just the same.

The microservice's database to track and schedule runs is in Postgres, with SELECT FOR UPDATE providing an easy way to treat a Postgres table as a work queue.

<comment><USER10> You words reminded me that 10-15 years ago ORMs were all the hype, and one of the advertised benefits to use them was that you are abstracted from an underlying database engine, and can flawlessly switch between them.

A bit later, as usually the hype went down and there were a lot of critical articles about ORMs, where they said that well, it is not actually that easy to switch database engines in a real project even with ORMs, and more - <USER2> hide great specific powerful features of databases. And finally - how often do you actually jump between database engines, what a strange benefit after all?

I am in no way criticizing your approach, and I clearly understand the rationale and benefits. It was funny, your words reminded me that our industry progresses in an infinite spiral trajectory, and repeats itself on a next level. :)

<comment><USER9> I know what you mean! In my approach, the ORM would be in the infrastructure layer - I'm assuming I will have to switch it out, if I'm using one at all.

It's the times where I have to migrate from one implementation of an interface to another where I have learned the most, usually meaning "I let the structure of this database get reflected too much in the business logic"/

</comment></comment></comment><comment><USER11> I work at Mode, an analytics tool company. We get questions about data engineering from our customers all the time, and the solution we recommend 90% of the time is DBT

[https://www.getdbt.com/](https://www.getdbt.com/)

If you're used to software development, DBT gives you a very similar workflow for managing the T and L portions of the job.

</comment><comment><USER12> If you are doing a serious ETL job, you'll need to handle rejects - I mean, a way to identify which rows of data cannot be handled gracefukky by your ETL.

Doing this in plain SQL or Spark is complicated.

This is usually a job for commercial ETL tools like informatics, talend, data stage...

</comment><comment><USER13> Personally, I'm a fan of scripts for this type of work... Node, Python, Perl, etc. That's just me though. I know a lot of people that live and breath SSIS packages and similar.

It really depends on where you want the data and how you want to use it.

</comment><comment><USER14> As with most questions about stack choice: it depends.

</comment><comment><USER15> It depends on your sources and whether you want to do real-time or not.</comment>


Lock elision in the GNU C library - <AUTHOR> http://lwn.net/SubscriberLink/534758/bcb45583bc25268d/

<comment><USER2> I think TSX is one of the most exciting architectural advancements to come around in a long time. There are lots of situations where memory could theoretically be modified by multiple threads simultaneously, but where conflicts will almost never occur in practice. Applications tend to be coded to avoid writes to the same memory from multiple cores not just for correctness reasons but because it causes cache lines to bounce between cores. However, because of the theoretical possibility of simultaneous access, you still have to use a relatively heavyweight lock to ensure correctness.

TSX promises to lower the overhead dramatically in such cases. I don't think it's clear yet how it will be implemented, but Dave Kanter over at RWT has speculated it will leverage the cache coherence protocol: <http://www.realworldtech.com/haswell-tm>.

To support cache coherency, cores already communicate with each other about reads and writes to cache lines. Multiple cores can have a line in their cache, but if any core modifies their copy, they have to send a message to the other cores signaling that their old copies are invalid (somewhat more technical description: <http://www.di.unipi.it/~vannesch/SPA%202010-11/Silvia.pdf>).

Kanter speculates that TSX uses this existing mechanism to cheaply support transactions. During a transaction, each core keeps track of the cache lines it has read or written to during the transaction. If it gets a message that another thread has written to a line it read from, or otherwise read from a line it wrote to, it aborts the transaction. Otherwise, the transaction commits successfully. This theoretically allows transactions that don't conflict to be handled completely locally to the core.

<comment><USER3> This only seems to help the uncontended case though; under contention it sounds like things will be just as expensive as before (or even more so).

Correct me if I'm wrong but this sounds like "shared-state multithreading will be faster, as long as you don't have any parallelism."

There is no free lunch in shared-memory architectures. But there are lots of techniques for addressing the problem. One of my favorite ones these days is to have a set of states, one per CPU, that each program writes to in its fast path, that can then be periodically merged into a global state.

<comment><USER2> Rather, it's "shared-state multi-threading will be faster as long as you don't have any mutations to shared memory."

Shared state isn't the problem--mutable shared state is the problem. That's what kills you from a cache point of view. But the mere possibility of mutations to shared state shouldn't needlessly make things more expensive when those mutations are rare or non-existent in practice. That's where TSX comes in.

<comment><USER3> Ah, I think I see. This would be useful for shared state that is read much more often than it is written. That seems to position it mainly as a competitor to RCU, which is likewise designed to have cheap reads and more expensive writes. But compared to RCU, both reads and writes would be faster with TSX (and in particular, uncontended writes would be especially faster).

<comment><USER4> The idea behind TSX was originally called speculative lock elision, and comes from this MICRO 2001 paper [1]. The paper provides a more coherent motivation of SLE/TSX than the Intel marketing literature, and contains some concrete (simulated) performance results.

[1] <http://pages.cs.wisc.edu/~rajwar/papers/micro01.pdf>

<comment><USER3> I see. So if I may paraphrase, TSX can also help write work-loads by eliminating locks that weren't actually needed because no actual contention over the same data occurred. This can happen if the lock was too coarse- grained and the writes were actually to different memory, or if the dynamically-executed path didn't actually mutate shared memory.

Both of these are problems you can code around, but doing that would likely require greater effort and complexity.

It's unclear to me whether recovering from mis-speculation is more expensive than acquiring the lock up-front. In other words, is there overhead in the case that you almost always encounter actual contention?

It's also unclear to me how this can be done without instruction set support, as the paper claims. Without having the critical sections explicitly marked how can this be implemented?

<comment><USER4> > I see. So if I may paraphrase, TSX can also help write work-loads by > eliminating locks that weren't actually needed because no actual contention > over the same data occurred. This can happen if the lock was too coarse- > grained and the writes were actually to different memory, or if the > dynamically-executed path didn't actually mutate shared memory.

Yes, those two cases are what I'd consider secondary benefits, that could be achieved straightforwardly (but not easily, as you note) without TSX. The primary case -- when the lock is fine-grained, and thread A _does_ mutate a value that thread B _does_ eventually read, but B's read doesn't come until A is already out of the critical section -- is the big win for well-designed software, IMO.

> It's unclear to me whether recovering from mis-speculation is more expensive > than acquiring the lock up-front. In other words, is there overhead in the > case that you almost always encounter actual contention?

On big x86 cores, there shouldn't be much performance overhead. x86 already has mechanisms for holding tons of in-flight memory operations in the core without releasing and/or committing them to the rest of the memory system. TSX simply specifies a new reason for holding them ("we're in the middle of an SLE transaction"). If your transaction is aborted because another core touched the lock, you can flush that buffer of memory ops and rollback to a checkpointed register file almost instantly. The paper describes a hardware predictor that decides whether or not to initiate a transaction on a given store; in any smart implementation, this predictor would incorporate information like "have I incurred expensive rollbacks due to transaction aborts on this store?" In that way, a smart implementation should tune itself to yield arbitrarily low average- or worst-case overhead over the non-TSX case.

> It's also unclear to me how this can be done without instruction set > support, as the paper claims. Without having the critical sections > explicitly marked how can this be implemented?

(No pedantry intended, I'm sure you already read this) It's explained in Section 3.3. They look for this pattern:

      Initially, MEM[x] = A
      STORE MEM[x] = B
      ...
      ...
      STORE MEM[x] = A (original value)

They consider anything between these "silent store pairs" to be a potential critical section, and the predictor can choose to try to elide the two stores and form a transaction, if it wishes. It is always OK to elide these two stores, whether it's a "real" critical section or not, as long as nobody tries to read MEM[x] in the middle (in the memory model they assume, anyway).

You definitely require ISA support if:

1) Your silent stores may have other side effects (I/O?)

2) You have synchronization primitives that don't fit the silent store pattern (ticket lock?)

3) You have a strong business interest in making the behavior of existing software, even at the microarchitectural level, as predictable as possible (Intel?)

</comment></comment></comment></comment></comment></comment></comment><comment><USER5> Am I the only one with a weird enough mind to have pictured Ellison (Larry) locked in the GNU C Library? Perhaps chained in a buffer with a never ending loop of "join us now and share the software"...

Very interesting article in any case.

<comment><USER6> I did that too. Weird.

</comment></comment><comment><USER7> HLE is a pretty nifty implementation (note: I work for Intel) and an interesting use of CISC.

IA instructions can optionally contain a prefix, such as LOCK, address and operand override (use 16, 32, or 64 bits), and repeat (REP). For example, in Linux, a single "REP STOS" (repeat store string) instruction is used to zero- out the kernel's BSS region. REP can also be added to instructions that don't use it, and the CPU will simply ignore it--this is defined behavior.

For HLE, using REP in front of certain instructions on a spinlock will begin or end a single memory transaction without actually writing the spinlock value. Of course, on older CPUs, the REP will be ignored and the spinlock read/write will occur. On CPUs that support HLE: if another memory agent accesses the spinlock, the transaction will be canceled.

</comment><comment><USER8> About using the Intel "Transactional Synchronization Extension"† (TSX) from the upcoming Haswell processors to provide a "fast path" to lock code. The code would first attempt to just do what it wants, but within a memory transaction. If the memory transaction cannot commit, there is a conflict from another core or thread, then it goes to the regular slow path with the locks. If applied at the right granularity almost all of your atomic operations should succeed with the TSX path and you will rarely invoke the heavier lock based code.

There is talk of performance measurement, but no numbers in the article, probably because Haswell isn't released.

␄

† [http://software.intel.com/en- us/blogs/2012/11/06/exploring-i...](http://software.intel.com/en- us/blogs/2012/11/06/exploring-intel-transactional-synchronization-extensions- with-intel-software)

</comment><comment><USER9> Interesting. Haskell (or at least GHC) uses software transactional memory pretty heavily. They currently use llvm as the back end, though, so won't benefit from these changes in glibc. I wonder if whatever library used there by default will start to incorporate these features anytime soon?

<comment><USER4> The libc one uses is orthogonal to the compiler, modulo support for any language extensions the libc might use. LLVM supports glibc just fine, and will use it by default on Linux systems.

That said, whether GHC can make more use of glibc's TSX support (as opposed to generating TSX instructions itself) than any arbitrary C program depends on any impedance mismatch that may exist between the elided lock wrappers and GHC's own STM semantics. No idea if GHC has a way to reason about libc-managed locks.</comment></comment>