Beating the bookies – how the online sports betting market is rigged - <AUTHOR> https://arxiv.org/abs/1710.02824 <comment><USER2> "Send enquires about this paper to our newly acquired private island in the Bahamas." :-)

One of the most depressing things I realized when I learned to count cards, and confer upon myself a small but meaningful advantage in the game of Blackjack, was that the casinos simply ask you to leave if you win too much. That put an upper limit on the rate at which one could win. The folks who figure out slot machines have a much better time of it because it takes longer for the casinos to figure out they are losing money.

<comment><USER3> _the casinos simply ask you to leave if you win too much._

I was actually backed off from the first place I went to after I learned how to count cards while I was _losing_. If the pit boss or dealers know how to count themselves and identify you as a counter, they want no part of it. In most cases, they'll either "flat bet" you (tell you that your initial bet is your maximum), or they'll tell you that you cannot play blackjack there. Actual barrings usually don't occur until the second or third offense.

<comment><USER4> That's only if you vary your bet size. That's a definite tell.

<comment><USER5> How do you profit from card counting without varying your bet size?

<comment><USER6> I guess you could always count from the sidelines and bet only in advantage spots. Not very practical I suppose but card counting in general seems pretty implausible these days

<comment><USER7> One strategy I know of is to have multiple people. One person sits at the table and always bets low, the other always bets high but only sits down if the person at the table signals to them that the odds are good (They can be off doing other stuff while the low-better is doing the counting, so they're not just standing around waiting).

You're still right though, the odds aren't that great and once they get a hint you're doing something weird you'll get thrown out.

<comment><USER8> That's what the MIT blackjack team did in the 90s.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> There has to be more to this story. I would like to hear the bookmakers side of this. Perhaps the researchers were scripting or triggered some other kind of security tripwire.

Winning $900 split across several different bookmakers is absolutely nothing in the sports betting industry.

William Hill, one of the companies that the researchers claim restricted them is a multi billion dollar company. They aren't sweating small time bets like this.

EDIT: I noticed that the screenshots they used as proof their bets were restricted are for bets on very minor football leagues (Australian semi pro football), its common for betting limits to be lower for games that don't see a lot of betting action & is not proof enough to me that the bookmakers lowered their limits globally

<comment><USER10> My guess is that it's the manner by which they were winning. Thirty $50 bets per week over five months is over 500 bets. That's not the long run but winning at an 8.5% ROI over that many bets is probably enough for the house to realize they're somehow a winning player even if the stakes aren't huge.

<comment><USER9> Sportsbooks in general don't mind winning players though, especially big ones. They make money off the vig and move odds to get equal money on each side of the bet so that they'll make money either way.

<comment><USER11> They hate "sharps", winners who win through skill exploiting lines, they love those who win by chance. Sharps will continue to bleed them, the lucky fish often become whales whose luck fades over time.

<comment><USER12> In casino games the "luck" stays the same. One or two percentages (or more) lower than 50%.

<comment><USER11> You are confusing luck with expectation. Luck is the deviation from expectation. I've seen terrible poker players win a big tournament, or crush cash games for months on end, and quit their jobs because they mistakenly believe they are skilled at poker. Their positive deviation above their actual expectation (their "luck") soon evaporates, and they moan constantly because their results begin to actually match their expectation, or even worse, they get "unlucky" and their losses are greater even worse than their expectation.

<comment><USER13> > You are confusing luck with expectation. Luck is the deviation from > expectation.

you're actually thinking of the square root of luck there.

</comment></comment></comment></comment></comment></comment></comment><comment><USER14> I've been trying my luck at tennis betting ( [https://matchstat.com/profile/soniman](https://matchstat.com/profile/soniman) ) and the only strategy that seems to work consistently is finding injured players and betting against them. Good example of an injured player is a player that limps (Andy Murray at Wimbledon this year). The computers don't watch matches so injured players can be overvalued by the models. Also, any British tennis player tends to be overvalued because the books are based in the UK. For instance there is an overweight British player named Marcus Willis that last week was a 1.01 favorite to win at a tournament in Las Vegas; he lost, apparently due to dizziness aka being fat. There are some other patterns that can be exploited but the highest return strategy has to be betting against injured favorites. However, I think that would probably require more tennis watching than I want to do. Maybe I could hire somebody on Mechanical Turk to watch tennis for me and report on players who appear to be injured or take medical time outs.

<comment><USER15> Maybe you could find a sport where injuries are logged or can be inferred from other data? For instance if a racehorse performs far slower than its typical pace, its odds of losing the subsequent race might be higher.

<comment><USER16> The whole point is that computers can't use injuries as a data point. This strategy won't work in any sport that can have the injury factor calculated by a computer.

</comment></comment></comment><comment><USER17> > _A few weeks after we started trading with actual money some bookmakers > began to severely limit our accounts, forcing us to stop our betting > strategy._

Isn't this like literally one of the oldest tricks in the book? I remember reading _Reminiscences of a Stock Operator_ , which talks in part about early 1900's bucket shops, and the same stuff was there even then. Similar stuff is also mentioned in market microstructure textbooks with market makers on one side and informed traders on the other side.

Is _rigged_ even the right word here? It might be, but did the bookmakers have a responsibility to keep accepting their bets? Is it different from claiming that casinos are rigged?

<comment><USER18> Same here. If you have a strategy that wins you money overall, bookmakers either send you a nice email saying your account is closed as they do not welcome professional players or they just severely limit your stakes (think $20 a bet).

(Will Hill, Interwetten, Betway are exactly the type of bookies that will close your account as soon as they catch on)

Yes, the odds can be exploited and there is a whole bunch of services offering picks, but eventually the sportsbooks catch on and close your account. The sportsbooks that welcome professional players are few and far between and their odds are on point.

</comment></comment><comment><USER19> I always thought bookies just kept moving the lines until they were making money at the snap of the ball. Their initial line isn't as important as adjusting the line so that there is equal money on both sides of the line and therefore the bookie is guaranteed money due to the small fee they build into the bets. Their profits don't depend on accurately predicting the game's score, just moving the line strategically as more bets come in.

<comment><USER20> Easier said than done.

Suppose your book is balanced, and you have $25,000 on each side. Then a new bet comes in, size $250,000, on one side of your book -- what to do?

Or, more simply, when you set your initial line, what do you do when a known sharp immediately wants action on one side?

<comment><USER21> You open things up to select sharps earlier than the general public, with limits. They get the benefit of potentially better lines and you get the information benefit of what the pros think the fair price for any bet is, allowing you to adjust the line before allowing bets from the general public. That's how a lot of the offshore sportsbooks handle it, but it may not actually be legal to do so in Vegas. I don't actually know.

<comment><USER20> One other factor is sometimes you actually are willing to take principal risk -- like when there are hoards of people willing to place bad bets on the local team -- you're willing to run an unbalanced book.

</comment></comment></comment></comment><comment><USER22> Yes, of course you can beat the "popular" bookmakers.

Once you start beating them ( being profitable in value prices ) they will simply close/ban your account. Nowadays, it happens extremely fast ( in a day or a few hours, depending on your moves ). It's a well known tactic, and in practice, you cannot do anything about it ( other than keep opening new accounts in new names ).

Try beating a betting exchange.

<comment><USER23> They did have betfair in the list - I wonder if they placed any bets there.

</comment></comment><comment><USER24> "Retail" bookmakers are only interested in mugs. If your betting patterns indicate someone who is wise to the market, you will be limited or shown the door. Be prepared to have arbitrage positions pulled out from under you. The game is rigged insofar as the book decides if it wants to entertain your position.

If you want to make money you have to bet against, and be able to beat the books that know what they are doing - The high limit, low margin books like Pinnacle, SBO, IBC et al will happily take you on.

</comment><comment><USER25> Not mentioned in the abstract is that they also used real money:

> During that period we obtained an accuracy of 47.% [sic] and a profit of > $957.50 across 265 bets, equivalent to a 8.5% return (Table 1, Figure 3).

For some reason the "ok but how much did you ACTUALLY MAKE?" is always my favorite part of this kind of business or economics literature.

</comment><comment><USER26> > A strategy intended to beat the bookmakers at predicting the outcome of > sports games requires a more accurate model than the ones bookmakers have > developed over many years of data collection and analysis.

I disagree with this assumption and I think they have painted themselves into a corner because of it. To illustrate, imagine charting win rates against bins of price-implied-chances. $3 horses win roughly 33% of the time, $4 horses 25% for example. It resembles a noisy 1:1 linear relationship. Do the same for your selections and your line will be noisier, but crucially you're not taking bets where the price is worse than your estimate. This can leave a window of profitibility when you subtract the two, even when you are less 'accurate' as measured by win rate or KLD or other measures.

The goal is profitibility, not accuracy. The problem with including the odds you are betting against as a feature for your ensemble is that it dampens that window. If you're right about your selections, you'll bet less and win less. * If you're concerned about the volitility that comes with being less accurate, there are better ways to address that.

I've been doing this for a couple of years and in many ways it's a dream side- project. Location independent, no customers, automatable, and in some jurisdictions tax-free. It can be a little lonely at times though. I would love to chat with anyone else applying tech/math to beat the bookies. Sorry for the throwaway, I'll put a contact in my profile.

<comment><USER27> > I would love to chat with anyone else applying tech/math to beat the > bookies. Sorry for the throwaway, I'll put a contact in my profile.

How's that contact information coming along? :-)

</comment></comment><comment><USER28> What the authors are doing is "chasing steam" and most books will ban or limit you if you try this. This should come as no surprise to people with experience in the industry. The books aren't limiting/banning them for winning - it's the way in which they were winning. They are betting slow moving books' lines. Their strategy only works at poorly managed books as long as they can bet quickly. I doubt they made any bets at a sharp friendly book like pinnacle.

If anyone would like to collaborate with some model building get in touch. I already have a large db of most of the stats you'd ever need and some okay but not amazing models for most major sports.

<comment><USER29> How would I go about contacting you?

<comment><USER28> Contact info is in profile but andr3w321 at gmail

<comment><USER30> People say their contact details are in their profile, but I never see contact details in any profile I view. Am I missing something?

<comment><USER31> If you view your own profile you can see your email, but some people don't realise it doesn't show up in your public profile.

</comment></comment></comment></comment></comment><comment><USER26> A couple of suggestions to improve on this method.

The authors' regression left an intercept or 'adjustment term' of 3.4% - 5.7%. For a perfect bookmaker, this intercept term would be equal to the overround. The number calculated unfortunately averages that overround between different bookmakers and at different times (overrounds often decrease over time). It might be more effective to adjust for the actual overround of each market sampled, i.e. divide each price by the sum of the inverse of the prospects.

They appear to use a flat betting strategy, and the threshold to bet or not was selected based on profitibility. I was simplifying in another comment when I said profitibility should be the goal. In reality it's utility you should be optimizing for. Nobody wants a ultimately profitable system that reads like an EKG, they want a high sharpe ratio. The paper's results are actually very good here, but the trend could be lifted and stabilized further by betting proportionally to expectation, or by explicitly optimizing for such.

<comment><USER32> Interesting. Are you refering to kelly criterion?

<comment><USER26> There needs to be some degree of risk aversion given the uncertainties in estimating your edge. I am thinking more along the lines of a CRRA utility function - something a little less aggressive than kelly.

<comment><USER33> in your opinion, is there any practical benefit over just using half kelly?

</comment></comment></comment></comment><comment><USER34> Link to the code used from the PDF:

[https://github.com/Lisandro79/BeatTheBookie](https://github.com/Lisandro79/BeatTheBookie)

</comment><comment><USER35> They claim the market is inefficient. I worked for a company that to some extend fixed that. Can't remember all the details (I worked on a different part), but something like this:

Most big betting companies were customers. They all continuously sent their updated odds to us, and we would broadcast to the other companies. They would react to the change based on certain rules and send new updated odds back to us. This would then converge.

The inefficacy comes from promotions, company X always wanting to have odds .1 better than company Y etc.

Edit: Not sure how it works now, but: [https://www.betradar.com/](https://www.betradar.com/) and [https://mts.betradar.com/](https://mts.betradar.com/)

<comment><USER36> Haha, this exactly describes arbitrage. My old company was "sent ... updated odds" (well, we scraped their sites often...) and "we would broadcast to the other companies" by betting on those other companies' sites when the lines crossed, and "This would then converge" because one of the bookies in the arb would move the odds after we hit them (or after we hit them a second or third time...)

<comment><USER35> Yeah, but with the difference that they pay a flat fee for the arbitrage through a third party, not leaving money on the table for others.

</comment></comment></comment><comment><USER37> Just like the rake or the edge the house advantage of making the rules has always been the reality of betting. Online or offline. Forever. Get too successful and you're no longer invited to bet.

Asymmetry of information has never been the bookmaker's most powerful weapon. The book is.

Those who are successful at it accept this reality. They grumble and make peace with it - paying the super taxes and liquidising markets where they're asked to.

Ultimately however, while it's interesting to see how they do some of this (and there are plenty of practices not covered in the paper, I assure you) it's a bit like complaining the DM won't let you do something in dungeons and dragons - you're dicing with the god of your domain so the rules can change at any minute.

</comment><comment><USER38> I used to work for a popular UK online bookmaker. The thing that a lot of these comments are missing is that bookies aren't going to sell a product at a loss. They're also not even selling the product you think they are (something akin to an investment).

Bookmakers sell excitement / entertainment - the thrill of the potential win is the product, and costs approximately 10% of what you can afford to stake.

</comment><comment><USER39> The most interesting thing was left out: how did they find this data? Both historical and realtime is pretty hard to find. Ten years of odds from a dozen bookies looks like a massive task.

Next: how do you mask this behavior to not be obvious. Once you have a betting stratetgy the real difficulty is turning it into one that isn't obvious.

</comment><comment><USER40> I don't see why we cannot have completely distributed betting platform using para-mutual betting strategy that collects and distributes bets and winnings on the basis of publicly reported sports results with no take-out.

I prefer para-mutual rather than a house deciding the odds. It is a more free- market approach. It has been used in horse racing, but the takeout has been too large which makes it hard to be profitable.

<comment><USER20> Yes -- customers don't like it because it's too hard to win.

Said another way, customers will choose to place bets with market makers, instead of some paramutual operation.

<comment><USER41> Why is it harder to win?

<comment><USER26> In paramutuel betting the prices are not fixed, so you cannot leverage differences between the odds on offer and your estimate of the prospect's chances. The final odds you receive are equal to the total stakes placed on that prospect divided by (sum of all stakes less the operator's take). You can't formulate an optimal staking plan with unknown odds, and you're forced to compete on win rates rather than price ineffeciencies.

<comment><USER40> I don't think your reasoning is correct.

A bookmaker setting the odds presumably would hire the most accurate handicapper analytics team to set the odds appropriately, then take their cut. In paramutual betting, you need only be better than the average bet..and there are a lot of stupid betters. You know the odds pretty well by posttime.

<comment><USER26> My reasoning is that the 'invisible hand' of many less informed bettors leads to market efficiency in aggregate, particularly in highly liquid markets. Greater liquidity in parimutuel markets makes for odds that are less of a moving target, but it leaves fewer mispriced prospects to capitalise on. So not only do you have to beat a fairly efficient market, but you have to beat it on average by a margin equal to the parimutuel operator's take. In less liquid markets the unknown odds are more of a problem. That's not to say it profits can't be made, just that it's harder imho. With fixed pricing, even though the numbers don't 'add up to one' there either, you can lock in a positive expectation long before price consensus is reached.

<comment><USER40> I can see what you are saying. I would say however that most $$ players in the stock market have analytic teams, etc. If instead, a substantial portion of the players in the stock market placed their bets on randomly chosen stocks, the aggregate would be much less efficient and that could be exploited.

I've sat and watched people betting on horses for a long time. The majority choose based on the name or color of the horse (sentimentalist), the going favorite (risk averse), the longest odd (big paydayists). Many others play on weaker signals (owner, jockey). A few bet on the advice of experts in the daily form, and these probably do make the market more efficient. In aggregate, from my own experience at Golden Gate Fields where the take-out is 14%, my average ROI was around -6 to -10%. This suggests that I was beating the market, but the takeout was killing me.

So I've imagined that in a decentralized paramutual pool with minuscule or zero takeout, and given a common population of betters, I'd make a steady profit.

I should add that another advantage of paramutual over bookmaker odds is the pool maintainers do not care if you are a winner or a loser, and won't freeze your accounts on you.

<comment><USER42> It's closer to 22.5% in total. [1] But it also depends on specifics like race, location, type of bet [2]. Looks like a conventional win, place, or show bet has the least taken out of it

[1] [http://www.calfairs.net/files/publications/14.pdf](http://www.calfairs.net/files/publications/14.pdf) \-- check slide 5

[2] [https://www.scribd.com/document/84887435/CA-Authority-of- Rac...](https://www.scribd.com/document/84887435/CA-Authority-of-Racing-Fairs- Ten-year-Satellite-Report-2000-2009-Final-03-11-11-Compressed-Version) \-- An Example for Northern CA breeds on page 10

<comment><USER40> I always focused on Win Place or Show betting becsuse if the lower take.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER32> They could have reduced a lot of the work (calculating the mean across 32 bookmakers and applying a constant for the margin) by just taking the price from a highly liquid exchange like betfair, which is pretty close to a 100% efficient de-marginated line.

Although it still wouldn't have prevented their accounts from being limited.

</comment><comment><USER43> Shouldn't the goal of a bookie simply be to balance his book and have no position on the actual odds of the match? Of course it makes sense to deny action to known sharps, this should come as no surprise.

</comment><comment><USER44> I don't totally understand why the bookmakers would limit their accounts.

The bookmaker wants to balance his book for each game to make sure he makes a profit no matter what the outcome is. To balance their books they might give better odds for an outcome than what a statistical model might suggest.

But what difference does it make if the bettor who helps them balance their books is a consistent winner or not?

Do they prefer to give these "good" odds to people who are losing money long term?

<comment><USER45> > The bookmaker wants to balance his book for each game to make sure he makes > a profit no matter what the outcome is.

This part is not really true. Bookies will very often have an unbalanced book and will be happy to keep taking action on the side that increases their exposure, if the price is right.

<comment><USER44> That's fine.

Although the strategy works here because the bookie moves the line to make it more attractive to bet on it. If he does not care about balancing the book, why move the line? He could just keep increasing his exposure

</comment></comment></comment><comment><USER46> For anyone interested in running data analysis I have probably the biggest and most comprehensive dataset in the world, bigger that what this team had available.

</comment><comment><USER47> Personally I've come to find poker is the only reliable way to "beat the house". But that's because you're not beating the house, but other players - the house gets their rake.

If you play a tight-aggressive game in venues the pros avoid (anyplace with less than $2/hour comps in Vegas) you can do decently. Not get rich, but make a few hundred in an afternoon.

</comment><comment><AUTHOR> Courtesy of slashdot:

[https://science.slashdot.org/story/17/10/21/1744218/data- sci...](https://science.slashdot.org/story/17/10/21/1744218/data-science- meets-sports-gambling-how-researchers-beat-the-bookies)

</comment><comment><USER48> TL;DR: Researchers made $950 (8.5%) but say it wasn't worth the effort [1].

[1]: [https://github.com/Lisandro79/BeatTheBookie](https://github.com/Lisandro79/BeatTheBookie)

<comment><USER23> They really should have automated the bet placement from the outset, which would have made it worth the effort. Could probably make a few thousand $ before getting cut off, per person. Definitely a worthwhile venture, considering you could probably then sell off the software after using it for some more profits.

Sites like [https://www.oddsmonkey.com/](https://www.oddsmonkey.com/) are kind of on this track already.

<comment><USER48> I've qualified my comment - thanks.

</comment></comment></comment><comment><USER49> Wanted to post this a few days ago, but life got away on me.

I worked for a monopoly bookmaker and spent a fair amount of time looking at how they work. Our turnover was $2.6bil/yr with $150mil profit in a country with less than 10 million people.

So, Some things to give you guys a bit more context. 1\. There are two types of bets. Fixed Odds and Tote. \- Tote is a pool based betting system where the odds can change after you have placed your bet. The odds are calculated automatically based on the distribution of bets on the options available. Typically the house will keep 50%+ of the total pool as profit and distribute the remainder among the winning punters. This is a very high profit betting system that the book keepers are trying to keep alive. It's dying off at a pretty rapid rate though. \- Fixed Odds Betting (FOB) is where you get payment on the odds you lock in at the time of placing your bet. Most betting now is FOB.

For the sake of responding to various points other respondents have made I will focus only on Fixed Odds; especially as Tote is only used for horse/dog racing.

2\. How do the odds work? For us, we had university students who'd manage the books. They had software that showed them how much risk/leverage they had and what the guaranteed profit was. They can set "bet limits" and manually approve (or deny) any bet that was greater than the bet limit. Most of the time they would have open websites from other bookies and copy the odds from theirs as they change. It's quite popular for bookies to just copy each other manually.

For Live/In-Play betting the book keepers will watch the event and manipulate the odds as things occurred. Either using their own knowledge or copying from other gambling sites. Again, the process is completely manual at the back end.

There is a move for organisations around the world to consolidate on their sources of odds (e.g. using a common back-end odds distribution platform); but ultimately there is still a large manual component to changing the odds, especially during live play.

3\. How do they make money? On Tote, they take 50%+ of the total pool before creating dividends.

For Fixed Odds, they balance the books. They change the odds to always ensure it's in the houses favour. We always aimed for 10-15% profit on events with fixed odds bets. Home players/athletes will always have much lower odds because of people's tendency to bet with the heart.

They deny bets. The bookie doesn't have to take your bet. For large bets they will often push back an offer to you at a lower rate than advertised to ensure their books stay balanced. For live/in-play bets they'll delay your bet until that odd is no longer available ensuring your bet is not accepted.

They have A LOT of different betting options where only a few will actually win. People tend to bet with their hearts and the number of options are setup to basically ensuring the bookie is profitable.

If you win too much, they shut down your account. They have no obligation to deal with you. Their goal is to make money and they see your gambling as a way you "enhance your enjoyment of the event", not an attempt to make money. So there isn't a large tolerance for people who do make money.

4\. How do I (the punter) make money? Surprisingly, you can consistently make money gambling.

Don't bet on Dogs/Horses. Even the top 1% of punters barely break even. They're profitable because of the kick-backs the bookies give them for having high turnover (>$1mil/yr).

Find a sport you know a lot about that supports in-play betting. It's going to be you vs a person. So if you have indepth knowledge of the sport you'll be able to see changes in flow and make winning bets before the bookie notices.

FWIW, I bet on League of Legends. During the LOL Worlds I can make 2500% with >90% win rate. Now, I'm only winning a few $k total so nothing significant.

That's all I can think of at the moment. Happy to answer any questions you have.

</comment><comment><USER50> How long do you think the authors would have waited to publish this paper if their accounts were not restricted and they would continue to profit? My guess would be a long time.

</comment><comment><USER51> "Of course the game is rigged. Don't let that stop you—if you don't play, you can't win." \-- Robert A. Heinlein, "Time Enough for Love"

</comment><comment><USER52> A bookmaker's tissue prices (the initial market prices they offer up), are not intended to be a measure of probability. They are meant to be a measure of expected Weight of Money (WoM).

A bookmaker's job is not to accurately reflect odds of occurrence, but to ensure a balanced book of liabilities. There are many books covering this going back hundreds of years and is the principle discovery of those who gathered at Tattersalls coffee shop on the Strand and invented modern bookmaking (via horse racing), and for whom there is named an enclosure on all 55 British racecourses to this day.

The tissue has to be "over-round", (i.e. the probabilities they represent have to add up to over 1.0, or 100%) because sometimes a market will look at the prices, see that the odds are very much in the favour of a selection and act accordingly. As such, the WoM causes a market to move.

This is not news. If you have a reasonable idea of true odds and you are being offered different odds, Kelly (who worked with Shannon - the creator of Information Theory), established the optimal stake to bet at each stage. A substantial amount of research has been done on Kelly Criterion and its application because it underpins many a hedge fund strategy: it works for fixed odds games, fiscal markets and bookmaker markets just as well.

There are trading opportunities here, and there is a wide community of people who look to exploit inevitable market moves using exactly this technique: establish average industry odds, look to where betting exchanges are and bet accordingly, moving out of the market when a profit comes to you.

The bookmakers don't care - you've helping get turnover up, and they know increasing turnover through the market is the best way to get balanced liabilities.

On a horse race, they might offer prices that give up an over-round of 110%-130% most days, but on Premier League Football which has a much, much higher turnover, and WoM is far more predictable (due to team loyalties coming into play), over-rounds can be as low as 102%.

These markets are therefore more likely to provide value to the authors' methods, however I note they are making the majority of their bets between 5 and 1 hours before a game, and therefore to some extent are able to factor in team selection and some of their results might be the result of a market inefficiency: team line-ups aren't announced until 60 minutes before kick-off.

Now, here's two major downsides:

1\. Bookmakers will eventually end up closing their accounts, because winners are never welcome long-term.

2\. There is a reason why successful gamblers don't publish. Even Thorpe who invented card counting and blackjack basic strategy realised publishing (which was his academic need) ultimately caused him to need to do something else: he ended up privately and quietly running a hedge fund.

These guys are probably finished within 2 weeks of this paper going around, and what's more because now a whole ton of people will go to implement this method, bookmakers will adapt and simply move from tissue to industry average as quickly as possible, whilst limiting even more players to reduce liability exposure (as has been the style in recent years).

EDIT: I only skim-read the paper when I wrote the above. Now I've read it a little more closely I am even more convinced there is nothing to note here, and also, their accounts have already been limited or closed.

<comment><USER44> Interesting.

You say: "The bookmakers don't care - you've helping get turnover up, and they know increasing turnover through the market is the best way to get balanced liabilities."

And then: "1\. Bookmakers will eventually end up closing their accounts, because winners are never welcome long-term."

Is there any rational behind bookmakers not welcoming long-term winners?

It seems like a bookie is very similar to a market maker on the stock exchange. Why would a market maker care if an investor makes money as long as he can flatten his positions every hour or so and make profit with the spread.

Isn't it accurate to say that bettors are competing against each other and the bookie is just taking a fee for making the market?

<comment><USER52> Over the short-term people adding WoM and bringing the book into line with other bookmakers is welcome, because it means the market is becoming more efficient.

At the micro level then, turnover is welcome. But as the old adage goes "turnover is vanity, profit is sanity"

At the macro level somebody has to say "how do we maximise profits or at least minimise losses?" and picking off accounts that are costing you money is an easy step to take.

The ideal client for a bookmaker is an idiot with a strong view. They don't want people who will consistently win, because it's taking up WoM for other customers who could be invited with those more generous odds and who _don't_ consistently win.

</comment></comment></comment><comment><USER53> I am confused. If the betting is rigged, then how were the authors able to beat it consistently? What did I miss?

<comment><USER3> "Rigged" is the wrong word. What it essentially says is that all of the data that a professional could reasonably analyze is already baked into the odds that bookies initially set, and if those odds stayed the same until bets were no longer being taken on the event, sports betting would be consistently unprofitable because of the 10% "juice" that most bookmakers charge. However, the odds do not stay the same. Once they are posted and bets begin to come in, the market takes hold and begins swaying the odds one way or another, because bookies change the odds in order to try to balance their books such that they have little to no actual risk. It is in the swinging of the odds by the market, which has many unsophisticated participants - people betting on a team simply because they live in that city etc. - where money can be made, because there is an incredibly large amount of "dumb money" in the sports betting market.

<comment><USER54> No, rigged is exactly the right word. From the paper:

> Our strategy proved profitable in a 10-year historical simulation using > closing odds, a 6-month historical simulation using minute to minute odds,

> _and a 5-month period during which we staked real money with the > bookmakers_.

>

> Our results demonstrate that the football betting market is inefficient ‒ > bookmakers can be consistently beaten across thousands of games in both > simulated environments and real-life betting.

> _We provide a detailed description of our betting experience to illustrate > how the sports gambling industry compensates these market inefficiencies > with discriminatory practices against successful clients_.

<comment><USER3> The fact that they limit the accounts of successful bettors doesn't mean that it is "rigged". It simply means that they reserve the right to do business with whom they choose.

<comment><USER55> They are artificially manipulating the market. That's about as textbook rigging as putting three masts on a ship.

<comment><USER56> If you consistently lose money whenever you transact with someone then you're going to stop doing business with or try to limit how much you lose. They're not rigging the actual game in any way. But if they're not throttling you it's a pretty good indication that you're not coming out ahead.

If there was an obscure way to play craps that made the house advantage negative they would either change the rules or ban the practice. You could call it rigging but by that token every game is rigged, that's the whole point, you're playing in the hope of beating the odds. The rules and odds are completely transparent, 'rigging' would be like using weighted dice.</comment></comment></comment></comment></comment></comment>


Show HN: Information Extraction and Knowledge Graphs - <AUTHOR> https://programmerbackpack.com/python-nlp-tutorial-information-extraction-and-knowledge-graphs/ <comment><USER2> An interesting blog post. Because there is nothing "to play with or try out," blog posts aren't really in the spirit of Show HN. They are better as just regular submissions even when posted by the author. But to be clear, I learned something. Thanks.

<comment><AUTHOR> Thank you very much for mentioning that to me, I didn't realize that. I'll make sure I'll respect this in the future</comment></comment>


A Peek into Einstein's Zurich Notebook - <AUTHOR> http://www.pitt.edu/~jdnorton/Goodies/Zurich_Notebook/index.html

<comment><USER2> This should be the same notebook:

[http://echo.mpiwg- berlin.mpg.de/ECHOdocuView?url=/permanent/...](http://echo.mpiwg- berlin.mpg.de/ECHOdocuView?url=/permanent/echo/einstein/Einstein_ZN/index.meta)

</comment><comment><USER3> Mirror: [http://web.archive.org/web/20121112213332/http://www.pitt.ed...](http://web.archive.org/web/20121112213332/http://www.pitt.edu/~jdnorton/Goodies/Zurich_Notebook/index.html)

</comment><comment><USER4> Looks like it has been removed, I get a 403 error. :-/

<comment><USER5> Same problem here. Because I am outside US, maybe?

</comment></comment><comment><USER6> Fascinating despite being way over my head. Amazing to see math of such complexity done without a computer - we're so spoiled today.

<comment><USER3> I am not a physicist, maybe one would elaborate on this, but it seems to be rather straightforward mathematics for a physicist to use (fancied-up calculus), and I don't know how a computer would help here at the level of development computers are today, my laymans impression is that the art here lied in choosing what to derive, not in the derivations themselves.

</comment></comment><comment><USER7> Incredible. You can almost feel the mind thinking freely in that workflow. Pen and paper in the right hands...

</comment><comment><USER8> In one of his letters to her he writes down a list of demands:

<http://www.giagia.co.uk/2010/03/23/ada-lovelace-day-2010/></comment>


The FBI files on D. B. Cooper - <AUTHOR> https://vault.fbi.gov/D-B-Cooper%20/ <comment><USER2> Since I had no idea:

[https://en.wikipedia.org/wiki/D._B._Cooper](https://en.wikipedia.org/wiki/D._B._Cooper)

      D. B. Cooper is a media epithet popularly used to refer to an unidentified man who hijacked a Boeing 727 aircraft in the airspace between Portland, Oregon, and Seattle, Washington, on November 24, 1971, extorted $200,000 in ransom (equivalent to $1,170,000 in 2015), and parachuted to an uncertain fate.
      Despite an extensive manhunt and an ongoing FBI investigation, the perpetrator has never been located or positively identified.
      The case remains the only unsolved air piracy in American aviation history.

<comment><USER3> Also Numb3rs s6e10 has a plot related to D. B Cooper.

<comment><USER4> Prison Break tied a storyline into the D. B. Cooper story as well. A pretty popular story it seems :)

<comment> J-dawg Another pop culture reference: The comedy 'Without a Paddle' is about a group of friends searching for D.B. Cooper's treasure

<comment><USER5> The "... in Popular Culture" section is large enough to have been spun off into it's own article on Wikipedia: [https://en.wikipedia.org/wiki/D._B._Cooper_in_popular_cultur...](https://en.wikipedia.org/wiki/D._B._Cooper_in_popular_culture)

</comment></comment></comment></comment></comment><comment><USER6> I am a different DB Cooper.

[https://en.wikipedia.org/wiki/Dale_Cooper](https://en.wikipedia.org/wiki/Dale_Cooper)

</comment><comment><USER7>
      I got this girl and she wants me to duke her
      I told her I'd come scoop her around 8, she said "Super!"
      That sounds great, shorty girl's a trooper
      No matter what I need her to do, she be like "Super!"
      Own his own throne, the boss like King Koopa
      On the microphone he flossed the ring "Super!"
      Average emcees is like a TV blooper
      MF DOOM, he's like D.B. Cooper

</comment><comment><USER8> I was really hoping the end to Mad Men was Don Draper being DB Cooper. It would've made for a much better ending.

<comment><USER9> I found that theory before I got into the show, it would've been really perfect. Totally fit with his appearance, manner, and history too.

<comment><USER10> Except most (all?) of the show takes place before 1971, right?

<comment><USER11> Correct. The last season ends in November 1970.

</comment></comment></comment></comment><comment><USER12> The two greatest unsolved FBI cases of all time are D.B. Cooper and the disappearance of Jimmy Hoffa. They're still trying to dig him up in Michigan and elsewhere. I was by the Machus Red Fox building in the past two weeks with my Dad who had his own run in with Jimmy over 75 years ago.

[http://www.freep.com/story/news/local/michigan/oakland/2015/...](http://www.freep.com/story/news/local/michigan/oakland/2015/07/29/hoffa- disappearance-anniversary-teamsters/30862419/)

<comment><USER13> Nice segue.

</comment></comment><comment><USER14> The interesting thing about the case is that as recent as 5-7 years ago people have been finding money with serial numbers matching the ransom payment. More over, the FBI confirmed that the ransom money never entered the circulation again. Honestly, after doing some random research and listening a few radio shows on this subject, it almost seems like Cooper accidentally dropped the money bag but survived the actual parachute landing.

<comment><USER15> According to [http://www.onedollarbill.org/decoding.html](http://www.onedollarbill.org/decoding.html) , serial numbers are not globally-unique identifiers:

 _" The letter which precedes the numbers must be the same number that you saw identifying the Federal Reserve Bank. The last letter of the serial number or suffix letter identifies the number of times that the Bureau of Engraving and Printing used the sequence of serial numbers: A is the first time, B is the second time, C is the third time and so on. With one run for each letter of the alphabet (26) and 32 bill per run, there are a total of 832 bills per serial number."_

<comment><USER16> The site you quoted seems to be something that someone copied & pasted and slapped together. When the author says "32 bill per run", he seems to imply that the whole sheet always has the same serial number. Well, it certainly is not always true as you can see in this high-res photo of an uncut sheet of 1985 bills:

[http://ultrafree.org/graphics/1985-1uncut.jpeg](http://ultrafree.org/graphics/1985-1uncut.jpeg)

A much more informative site which seems to be someone's labor of love says there is an exception with old Nationals "in which all notes on a sheet had the same serial number and had to be distinguished by their plate position letters." Although he's not explicit, my reading of that site suggests that modern bills (including the time of the D. B. Cooper incident) should be unique.

[1] [http://www.uspapermoney.info/general/number.html](http://www.uspapermoney.info/general/number.html)

</comment></comment></comment><comment><USER17> Odd that the text (presumably OCR?) version is pretty much garbage. What happened?</comment>


Introducing Redis To Go - <AUTHOR> http://blog.redistogo.com/

<comment><USER2> Er...a Linode 512, at $20/month, is $5/month cheaper than their smallest plan, and $90 cheaper than the comparable plan. Unless you really, really cannot do a simple

        tar xf redis.tar.gz
        cd redis
        make

then I don't understand why on Earth you'd do this.

<comment><AUTHOR> It is cheaper, but I don't think you are running an OS that will only use 12 MB of memory. This also doesn't include backups or system monitoring. And if you are EC2, well this just isn't an option if you are concerned about network latency.

<comment><USER3> Network latency is not that much of an issue when you have your application in the cloud as well.

And I'll only believe claims about backups if the contract contains an enforcable clause that I'm entitled to a pound of the CEO's flesh if/when they lose my data for whatever reason.

(Amazon S3 occasionally loses your data when two/three servers fail at once. If they had such a clause, their executive board would probably be short of a leg or maybe two).

<comment><USER4> _Network latency is not that much of an issue when you have your application in the cloud as well._

What?

</comment></comment></comment></comment><comment><USER5> Have to disagree with this. Redis is dead simple to install and configure out of the box.

This seems like a solution to a nonexistent problem.

<comment><USER6> I would add a seemingly expensive solution to this non-existant problem. It may kind of makes sense for heroku because afaik they don't offer redis support out of the box.

<comment><USER7> Yeah, I would definitely consider using the $25 plan with Heroku. The only other real option is to run Redis on its own EC2 instance, which costs $62 minimum (well, once you're sure the thing's sticking around, you could make it a reserved instance I guess). Now, that's a lot more powerful, but if I'm using something that doesn't even need the 100MB of RAM, I might as well save some money and use Redis to Go.

Plus you get the added bonus of still not having to worry about any piece of your infrastructure.

</comment></comment></comment><comment><USER8> Wouldn't the network latency kill the performance benefits? Redis is also dead simple to setup. The front page is not clear what the $200 time frame is. Per month Per year ... per hour?

<comment><AUTHOR> Good point. The site has been updated and the pricing is per month.

<comment><USER9> Since you appear to be the developer of this product, this is as good a spot as any to point out that the link to your website in the footer of the blog is broken.

</comment></comment></comment><comment><USER10> Nice job, I guess plans and getting into the Heroku alpha program nets you more upvotes on HN. (than <http://news.ycombinator.com/item?id=1456431>)

The game is afoot!

<comment><USER7> Oh cool, I missed that even though I'm on HN a lot.

You do awesome stuff, so I'm excited to see this when it launches.

</comment></comment><comment><USER11> Are the prices monthly or annual or one-off or...?

And who said Redis was hard to install?

</comment><comment><USER12> So my server, which is "here", is going to be talking to a redis which is where exactly? "over there"? How do I prevent someone from "somewhere else" from talking to it?

<comment><AUTHOR> The servers are located in EC2, and there is authentication.</comment></comment>


Ask HN: How do I get the word out on my data mining report - <AUTHOR>

Hi HN,
I did data-mining on my site and did some charting and such and BELIEVE my data to be interesting and blog worthy.  I created it hoping to get it out there and drive traffic to my web app.
I don't have much of a contact list or presence on the web, so was looking for advice on how to proceed beyond posting to forums and reddit and such.
also i'd like to know whether the data is really blog worthy at all.  If you run a blog/site in the domain how could one convince u to write something about it?
thanks much. <comment><USER3> Shoooooooot - that there is some tech-nickel words there. I have a D90 - <USER2> to see it seemed to do well but whoa... Well over my head what all those charts and such mean/do/tell me.

I'm with ig1 - you need to write an article (maybe multiple) and submit to others. I would suggest you really work to make it accessible to people like me. I have quality gear, took a photography class in college, and like to shop for gear.

</comment><comment><USER4> Write an article about about it and submit it to photography magazines, techcrunch, etc.

</comment><comment><AUTHOR> thanks guys i kinda gave up after seeing no comments at first, but im glad i checked back. I will write several versions of it as articles and hopefully it will be worthy of showing up somewhere.

</comment><comment><AUTHOR> link to the data mining report: <http://wiki.elograde.com/full:data-findings></comment>


GNOME 3.8 released with classic mode similar to GNOME 2 - <AUTHOR> https://help.gnome.org/misc/release-notes/3.8/

<comment><USER2> You're a bit late, gnome 3.8 was released back in March. HN story at the time: [https://news.ycombinator.com/item?id=5451646](https://news.ycombinator.com/item?id=5451646) .

(FWIW, I've been running it for a couple of weeks, and rather like it. A couple of the 'classic mode' extensions make gnome-shell a lot more usable (especially TopIcons extension which lets you have a useful taskbar), and it feels a lot more responsive than gnome-shell 3.6).

</comment><comment><USER3> Can anyone verify if "focus follows mouse" and "raise on focus" are both supported?

<comment><USER2> Yup. dconf-editor* -> org.gnome.desktop.wm.preferences has focus-mode (click,sloppy,mouse), auto-raise (true,false), and auto-raise-delay (0-10000) keys, all seem to work fine. (focus-mode is also settable from gnome-tweak- tool's 'window' settings).

* n.b. not to be confused with gconf-editor, which has an equivalent set of settings which won't do anything (it's being deprecated)

<comment><USER3> Thanks.</comment></comment></comment>


Things Every Kid Should Master - <AUTHOR> http://www.bostonglobe.com/magazine/2015/02/26/things-every-kid-should-master/uM72LGr63zeaStOp9zGyrJ/story.html <comment><USER3> My son is six and I take him to school each day (its about 25 minutes). He started asking me to tell him stories on the way to school, so I told him stories about how I was a spy and the missions I went on.

Eventually I put him in my stories too -- he was member of the Omega Force. At some point he didn't like how his character was acting in the story, so he began to tell me what his character should do. Well over the course of about a month we now do a full on role-playing game on the way to school. The level of role playing matches nearly anything I did when I played D&D. He now leads the Omega Force and I lead the Alpha Force. His squad is larger, but mine is more specialized.

That 25 minutes of rather sophisticated verbal interaction is pretty incredible. I can weave in interesting scenarios and discussions, and the level of engagement is great. I have no data to support it, but I feel like its something that almost every kindergartner could benefit from.

<comment><USER4> You should have him write down his thoughts as well - it sounds like his story development and verbal communication skills are getting a bunch of work, would be awesome to have his written communication match

<comment><USER3> That's a good idea! Thanks.

</comment></comment></comment><comment><USER5> > Conversation

I know barely any adults (late 20s, early 30s) who know how to do this. The concept of "conversationalist" is lost in modern society, you have to be watching whatever junk is on TV in order to communicate with people - <USER2> forbid you want to talk about something genuinely interesting.

<comment><USER6> Why else do you think "conversationalist" is lost in modern society? I know countless people who are wonderful conversationalists.

Something to consider... the first step to good conversation is being genuinely interested in other people's interests. If they're talking about something you don't know, take it as an opportunity to learn. Sneering at the values of others is poor attitude, which leads to poor conversation. If you really believe people only want to discuss "junk on TV", and that it's not "genuinely interesting", you're forgetting that it's interesting _to them_. Declaring an objective view that makes you superior and them inferior is, well, rude.

A little more about tv, weather, and other "uninteresting" subjects here - conversation is about finding common ground, something to talk about. We talk about the weather, or tv, because it's a place to start. It gets the conversation rolling, to help us discover bits about each other, and eventually find that "genuinely interesting" common ground.

<comment><USER7> Can't say I've experienced that. It always seems to decline into the size of a celebrity's arse. Even one of my friends, an ex-physicist and current private jet pilot has nothing to say past the size of an arse.

I just go and read a book now.

<comment><USER6> Celebrities are an important part of generic conversation... they give us someone to gossip about that neither of us actually know. Public shaming is an important (if offensive) social ritual.

That said, just about anyone can talk about something other than Kim Kardashian. No one actually cares about her. Talking about something like a celebrity's arse is a way of _avoiding_ talking about stuff we actually _care about_ \- as someone else noted in this thread, the fear of being seen as uninteresting, or weird, or an outsider, if you talk about stuff that matters to you.

The way to get people to talk about what they care about is to _ask them_. Your friend the physicist/pilot? Ask him about physics. Ask him about airplanes. Ask him about his clients as a pilot. Pretty soon, he'll be talking about much more interesting things - interesting to him, and probably interesting to you.

And all you have to do is listen, and he'll think you're awesome.

<comment><USER8> With some people that's really hard. They don't see your interest or disinterest in a topic at all, even if you make it really really obvious (not looking at you, one word answers and even descending into rudeness vs making eye contact and participating in the conversation). Some people just want to complain about trivial things, even if they could tell you about antarctic expeditions instead.

</comment></comment></comment></comment></comment><comment><USER9> I find it encouraging that we're starting to discuss skills that allow young people to teach themselves rather than remaining reliant on guidance of others.

Some of the skills discussed in this article are difficult to instruct in an educational setting (though they can certainly be structurally reinforced) - but I see far too many young people who see 'education' as a task to be completed/survived rather than a series of personal skills they will need/want to apply to life at large.

The comments about testing aren't particularly interesting (as other commenters have pointed out) - but we should certainly be aware of the distorting effects of high stakes testing on the values/goals of our schools.

<comment><USER10> "Dumbing us Down" by John Gatto is a great eye opener about schooling and how schools are great at _killing_ creativity.

</comment></comment><comment><USER11> I would give anything right now to have had someone actually have taught me collaboration and conversation when I was a kid. I remember crying multiple nights in elementary school because I didn't have friends and didn't know how to work with people. At the very least, I wish I'd known about How to Win Friends and Influence People before high school.

<comment><USER6> Nice to see a shout out to the greatest book ever written on being a good conversationalist! "How to Win Friends and Influence People" ought to be required reading in high school.

My wife struggles with introversion. She hates talking to strangers and non- friends, calls networking "schmoozing", and generally sees only the negative in meeting new people. I've tried to get her to read this book for years, but she fights me on it.

<comment><USER12> I read that book and I've gotta say fans of it have really oversold it. It has some good advice but I found some tactics questionable due to appearing as insincere to me.

I also find it weird that you characterize your wife's introversion as a struggle like introversion is some sort of abnormal condition. Some people just don't enjoy random conversations about random topics because randomness is usually not a great way to select for quality.

<comment><USER6> Really? I thought sincerity in conversation was the entire point of the book! He says quite explicitly that if you're insincere, people will sense it and distrust you.

As for my spouse... yes, it hurts her, and she's aware it hurts her. It limits her effectiveness as an artist, and it limits her career options. You can't "cure" introversion (it's not a disease, it's a personality), but you can compensate for it. And part of that is just cultivating internal attitudes - for example, not thinking of networking as something deceptive and selfish ("schmoozing"), but rather as something sincere and generous.

<comment><USER13> As an artist, introversion has its benefits too. Extraversion reduces time spent pondering, employing imagination and intuition.

</comment></comment></comment></comment></comment><comment><USER14> I feel like the writer doesn't fully understand how deep some of the flaws in our educational system can run. Take, for example, the section on collaboration which hints only at educator behavior influencing students to exclude others. There are so many other factors that would cause a child to segregate and organize themselves into cliques, and as much as our educators may impact the social structure of the students, the behaviors of parents, guardians, caretakers, and other adults in a student's life play just as important a role in their development, especially if students spend more time with them than their teachers. This issue extends beyond the scope of our education system and is not easily remedied through changes in the institution. Some of these habits may not be fixable.

This isn't to say the article doesn't raise good points, especially that elementary and early school years should focus more on developing social skills than academic ones. However it fails to assess how varied children can be in terms of background, home life, opportunity, and development. Would it be fair to assess a student to whom English is a second language on the same scale as one that grew up in an English speaking country? Can we compare a student on the autistic spectrum with a socially and verbally mature student? The author makes these topics seem simpler than they actually are, and that hurts the message that they're trying to get across.

<comment><USER15> _good points, especially that elementary and early school years should focus more on developing social skills than academic ones._

I hold that United States schools are stultifyingly boring for a great many pupils, because the lessons are too easy,[1] and as a consequence many pupils languish in their social development because they are not allowed the opportunity to pursue development of knowledge and skill while engaged in group work at school. One reason I think this is that I know a whole country, where my wife grew up, where most school pupils were instructed in school in a second language that they didn't speak at home[2] and where the school lessons were much more challenging. That seemed to produce better social results than have ever been found in United States schools.

[1] [https://www.americanprogress.org/issues/education/report/201...](https://www.americanprogress.org/issues/education/report/2012/07/10/11913/do- schools-challenge-our-students/)

[2] My wife is part of the majority population of Taiwan in her generation that grew up speaking Taiwanese at home but was only permitted to speak Mandarin at school. (Mandarin is approximately as different from Taiwanese as English is from German, or as Spanish is from French.) The school lessons in Taiwan were, and are, much more ambitious in promoting deep thinking than United States lessons are, and include a mandatory requirement to begin studying high-school-level algebra and geometry by seventh grade--even for below-average pupils. Oh, and of course school pupils all over the world begin study of modern foreign languages--usually English--by much earlier ages than most pupils in United States schools, even if they are going to school in a second language, as many school pupils do in many countries. United States schools are too wimpy academically to fully develop young people socially.

<comment><USER14> Yes, in my experience elementary school was hellishly boring, but I believe that comes from them being focused on teaching mechanical processes, such as the times table, rather than thought processes. In that way it'd be just as beneficial to a student's social faculties to teach critical thinking as critical thinking would benefit from the teaching of social skills. Having to simultaneously master another language provides students with a direct method of developing their communication skills and requires them to better rationalize their thoughts, so naturally it would foster better social habits.

To also touch on one of your notes, I also think that the United States is slow in teaching foreign language, and exposure to other cultures and languages would help to dispel the myth in American culture that the U.S. is the center of the universe. Perhaps American schools should start teaching only in Mandarin as well. :p

</comment></comment></comment><comment><USER16> How about empathy and self control? How about humility? These 3 skills are sorely needed.

<comment><USER17> I very much agree with the Buddhist thought process that compassion, empathy are pivotal to well-being. I was a bit disappointed that these were absent from the list. In my opinion they are foundational in regards to points 4 through 7.

</comment></comment><comment><USER18> Misleading title. Should have been "Things every child educator should master."

Children _should_ be children, it's the adults that are messing up.

</comment><comment><USER19> This article seems to be flame-bait for the standardized test world, rather than pointers on how to direct children.

e.g. "If only our standardized tests measured this amorphous thing!" rather than "Here are tips for promoting [inquiry] in children".

Looking at BG's comments show the article's content is ignored and instead is people blasting their opinion of the current state of educational reform.

<comment><USER14> I agree with this. By opening with a diatribe on standardized tests, and focusing on assessing skills that are inherently difficult to assess, Engel moves the discussion from helping children to condemning our education system (it's bad, but this article shouldn't be about testing or schools at all).

<comment><USER20> _By opening with a diatribe on standardized tests, and focusing on assessing skills that are inherently difficult to assess, Engel moves the discussion from helping children to condemning our education system_

Exactly. The article just comes across as so silly. Take this part:

 _There are several ways we might measure a child's disposition to inquire. We can easily record the number of questions the child asks during a given stretch of time._

It's been a while since I've been in a classroom but I would probably jump out the window if every class period each student had to get their average 1.4 questions in to maintain their grade.

It's like the article writer lives in a fantasy world.

</comment></comment></comment><comment><USER21> "Testing these concrete things is bad, let's instead attempt to test these poorly-defined subjective things."

</comment><comment><USER22> Nothing about being a creator rather than a pure consumer? Too many are caught up in the cycle of chasing grades and completely forget what the _point_ of an education is supposed to be.

<comment><USER6> What _is_ the point of education? Not the point of learning, but the point of schools and degrees and such?

One could argue that it's mostly about learning obedience, how to worship the clock, and how to tolerate doing things that are arbitrary and stupid. School isn't to make us good people. It's to make us good employees.

<comment><USER23> Is it learning obedience and worshipping the clock, or is it patience and self-control?

I went to lunch with some coworkers the other day. The restaurant we went to told us there would be about a 5 minute wait because some machine or other was just repaired. One of my coworkers had a little freak-out about how long it was taking and stormed out to go to another restaurant. (And this was a "fast food" restaurant we were already at.) I'm pretty sure he hadn't finished walking to the next restaurant by the time we got our food, but waiting an extra few minutes was too much for him.

The rest of us had a minimal amount of patience and it was no big deal. He wasn't a slave to some clock and he didn't have to obey anyone telling him to wait. But he also didn't get what he claimed he wanted (to be served faster).

<comment><USER6> Why can't it be both?

</comment></comment></comment></comment><comment><USER24> Was pleasantly surprised by the article. Turns out there's some excellent guidance for the new parent there, given that very few of us can expect these lifeskills to be taught at school; folks can at least take a stab at teaching these at home.

</comment><comment><USER25> Boom, I was expecting to find "Programming, every kid should know how to program" at any point in the article. Glad I didn't.

</comment><comment><USER26> This is a great start and very well intentioned. I worry that all of the assessment methods, though, are very geared towards extroverts.

A shy kid just won't perform well on most of those metrics. If a kid isn't comfortable with the tester, they'll be unlikely to pepper them with questions.

Now, we should _definitely_ be trying to give that shy kid the tools they need to deal with that situation. In-person collaboration and meeting new people is a vital skill. We can't let our entire testing regime be blocked behind that skill, though.

</comment><comment><USER27> "Why not test the things we value, and test them in a way that provides us with an accurate picture of what children really do, not what they can do under the most constrained circumstances after the most constrained test preparation?"

Because the purpose of testing is to use it as a weapon to punish political opponents and brag about your own accomplishments. Also it can be used as a weapon to reduce funding, if you want to reduce funding and need to provide an acceptable theoretically 3rd party evaluated reason, ditto if you see firing people as an inherent good. Along the lines of security theater logic, the more painful and obnoxious and offensive the testing is, the better it must be and the more the school must care about kids, for a similar example see the logic behind corporal punishment in schools from decades past (we only beat the kids because we love them or somesuch nonsense). Finally if you carefully design racially biased tests or socioeconomic biased tests, then even a proud liberal can select the lilly white rich subdivision for their kids "because of the test scores" in an abstract public sense, not the highly politically incorrect race and income data.

The analogy with security theater is the truly critical point from above. School testing exists to improve schools in exactly the same way TSA jackboots molesting women at airports improves security.

<comment><USER28> I strongly disagree.

The latest round of standardized testing, Common Core, is quite good. However, _some_ schools and teachers and administrators will always act like:

> TSA jackboots

This doesn't mean that the testing or the test material is a bad idea.

</comment></comment><comment><USER29> This is an excellent thought experiment I genuinely wish were reality. That said, the root problem with standardized tests is that they're inherently hackable. Standardized tests are like desktop software: you write once and ship everywhere, and once you do people start reverse engineering it.

Any sufficiently hackable test will appear to work in the short-term if you just place it in front of admission to prestige, because of the large subset of high-achieving people whose lives' missions are to game any test placed in front of them, which drastically skews how well the test appears to predict achievement.

The things mentioned in the article are excellent things to test, but if you start measuring e.g. the number and type of of questions a kid asks within x minutes, cram schools will just adapt, and in a few years you'll hit the end of another cycle.

</comment><comment><USER30> > The educational philosopher Harry Brighouse has suggested that the ability > to think about something for 20 minutes at a time (sustained focus) may be > one of the most powerful cognitive skills we acquire in school.

What school did this guy attend? The one I was sent to as a child was little more than a zoo. If anyone left there able to concentrate for 20 minutes (myself included), it had nothing to do with the curriculum, the teaching methodology, or even the values espoused by the teachers and administration.

</comment><comment><USER31> I am surprised about the lack of math/algebra emphasis.

</comment><comment><USER32> The traits that we should be testing for in this article are great but what do you think it would take to adopt this type of testing?

It seems like public school are stuck continuing with the current tests because these tests are used to procure federal/state funding. For schools to change, the country's consciousness needs to change about how society defines a "well rounded" individual

</comment><comment><USER33> Where does "who invented the cotton gin" fit into all this?

</comment><comment><USER34> #7 is arguably even the most important.

If only administrators and those above them would be made to understand that there is more to child development than standardized test scores.

<comment><USER35> My reaction was the opposite. Of course well-being is important, but it's a state isn't it? How do you learn it, or teach it? It sounds perilously close to the whole "self-esteem" thing that became prominent in elementary schools a few years back. As if self-esteem was not something developed through accomplishment and the subsequent increase in self-confidence, but rather a piece of information to be conveyed by an instructor. "Shazam! You now esteem yourself."

<comment><USER36> Lots of people with impressive accomplishments don't have self-esteem. Impostor Syndrome, Dunning-Kruger etc. If you don't have self-esteem, you'll let yourself get pushed around because you think you're not really worth anything. But with it, you'll start to see your own potential, which is a good motivator to start working toward it.

<comment><USER37> Self esteem is a double edged sword as it does not consider empathy and may lead to narcissistic values. That's the flaw with the self only aspect.

</comment></comment></comment></comment><comment><USER38> Empathy is probably the most important thing kids should learn.

</comment><comment><USER39> play</comment>


The Social Network (2010) - <AUTHOR> http://www.rottentomatoes.com/m/the-social-network/

<comment><USER2> From... six critics. That's like saying you got a 100% increase in conversions... from two trials.

</comment><comment><USER3> I think the better indicator that this sample may be reflective of the final meter is the vocabulary in the reviews - highly positive - one claims "movie of the year".

It's possible to get a tomato and still have some happenstance language in the critique - the fact that the language, here, is immensely positive is a strong indicator that such a strong meter will hang around.

That, and it's directed by David Fincher.

<comment><USER4> And written by Aaron ("A Few Good Men", "The West Wing", the _hugely_ underrated "Sports Night") Sorkin.

<comment><USER5> If I could upvote you 100 times, I would. Completely agree about Sports Night, I still re-watch both seasons of that at least twice a year.

Also a huge fan of Studio 60 - not sure I could call that one "underrated", as it did have some problems, but... still absolutely loved the writing/characters in it.

Sorkin's the reason I can't wait to see this :)

(And luckily for me, I happen to be in NYC early October, so don't have to wait the extra couple of weeks for it to arrive in the UK!)

</comment></comment></comment><comment><USER6> Here's my rant on RT: Their 0-100% score tells you very little about how good the movie is. It simply tells you how agreeable to critics it is.

Because often critics just contribute a binary yes/no review, movies that can get a lot of critics to somewhat approve of them can get very close to 100% (art house movies often score too high because critics would lose face giving a thumbs down). Movies that half of the critics love and half of them dislike can get 50%.

I'd really like to see a site that matches up my movie tastes with a few movie critics that predict my tastes well, and then keeps me updated with their reviews.

<comment><USER7> Something like Netflix's personalization, but for movies out in theaters? I'd pay for that.

</comment></comment><comment><USER7> It will be interesting to see if this film translates to mainstream audiences. Even though there are half a billion users on Facebook, I'm not sure all of them are interested in a story about the company's history, no matter how good it might be.

<comment><USER8> From the trailers it looks like a drama about money, sex, betrayal, etc. Seemed pretty mainstream to me. I almost get the idea that Facebook users might find it a little over the top. I don't know anyone who takes facebook all that seriously, even though they use it everyday.

</comment></comment><comment><USER9> _sigh_

THR and Variety are notoriously tough reviewers. Besides, if you think Sorkin's going to fuck up the dialogue in a movie about _communication_ , you're on crack. Sure, a 100% is just a leading indicator of quality, but it's a supremely high quality leading indicator.

<comment><USER10> And it's not just Rotten Tomatoes. It seems like whenever I read something about this movie written by film critics in the last few weeks it was full of praise. Screenplay and editing must be excellent.

</comment></comment><comment><USER11> Brilliant. When I opened Rotten Tomatoes I saw a picture of myself, because the site is using the Facebook personalization API... Very eerie actually- not that I'm against the technology- but just noticing it for the first time on this particular movie entry...

</comment><comment><USER12> Out of 6 reviews... Which could indicate a great movie, or dumb chance, or preferential treatment of reviewers willing to commit to being positive.

</comment><comment><USER13> Excited to see it. I just hope the real life facts dont get in the way of my enjoyment. At this point, if they try to make Eduardo out to be the good guy, I'm probably going to walk out of it. In every report I've read about the early stages of that company, that guy sounds like a real douche-bag.

Ben Mezrich's Bringing Down the House was a good read, but it sounded very embellished / made up. I didn't trust it as a read of non-fiction. And the movie 21 was just a complete piece of crap.

However this movie is directed by the guy who did Fight Club, so I'm sure it'll be turn out (and be reviewed) pretty well

</comment><comment><USER14> Peter Travers (Rolling Stone)' only 4 star rating of the year, 100 on metacritic, which takes a mean-average of critic ratings implying perfect reviews, although sample size is 4 on metacritic, 2 of which are also on rottentomatoes. To note: Inception had 100 on several reviews on big names prior to wide-release, but settled at 74.

It is pretty safe to say that the reviews are going to be strong as the big critics have their reviews out and they lead the way. Being released in September also puts it favourable for Oscar nominations.

</comment><comment><USER15> I much prefer IMDB to check on movie ratings (they're much more user-based and accurate to me, at least): <http://www.imdb.com/title/tt1285016/>

User reviews here: <http://www.imdb.com/title/tt1285016/usercomments>

So far it seems positive (though that could easily be affiliated reviewers that got to watch the movie early).

<comment><USER16> Roger Ebert recently wrote a bit about the gulf between fan opinion and professional reviewers. He also touches on review aggregators and what they bring to the mix.

[http://blogs.suntimes.com/ebert/2010/07/the_myth_of_a_perfec...](http://blogs.suntimes.com/ebert/2010/07/the_myth_of_a_perfect_film.html)

</comment></comment><comment><USER17> As owner of a consultancy that makes custom web applications for clients, I'm taking the expected mainstream success of this movie as a very bullish signal about growth in the overall ecosystem of web startups.

<comment><USER18> As a random guy on the internet I'm going to go and replace my bullshit meter because it just exploded while I was reading your post.

<comment><USER19> Obie owns/runs Hashrocket which I recognize as a RoR shop, so that part is true.

As for the correlation between the movie's success and the growth of the web startup world, I disagree. I would guess that people will see this movie because it's about a website that's a part of their daily lives.

Even if I didn't know who Obie was, I don't think his comment warranted getting downvotes or being called out as bullshit.</comment></comment></comment>


Distributed Filesystems for Deep Learning - <AUTHOR> https://www.logicalclocks.com/why-you-need-a-distributed-filesystem-for-deep-learning/ <comment><USER2> What does NVMe have do to with any of this? That's hardware storage technology, and it's being compared to Python versions of deep learning frameworks?

The whole point of having a file system is that programs are abstracted from storage. What does native support even mean unless the file system somehow hooks into the running code?

This makes no sense. It seems like a post trying to push HopsFS as more performant, but comes across as inaccurate and misleading.

<comment><USER3> I agree, it appears as though NVMe was simply added in there to look impressive and give another tick when in actual fact it's not relevant to this layer of tech. This should be abstracted beyond hardware storage and be based solely on the filesystem itself.

</comment></comment><comment><USER4> I am baffled why they have a matrix of what data science libraries they support. They support reading and writing CSVs and Parquet (and, I assume, arbitrary files as well). How is this specific HopsFS support for Pandas? Pandas is dumping csv format to a file object which happens to write to HopsFS.

This is a pretty weak sales pitch.

Edit: I'm not actually baffled. It's clearly SEO to catch people googling around for "pandas support" or the like.

<comment><AUTHOR> Author of the post here. Part of the point of the matrix was to show that not all distributed filesystems have native support for Python frameworks commonly used for Data Science - Pandas/PySpark/TensorFlow. The other part is support for NVMe hardware. Having been a professor for 17 years, i barely know what SEO is.

<comment><USER5> Can you explain what it means to have native support? Are you saying you hook into Python's file io to implement remote IO? Or is the filesystem mounted via FUSE or another technology? I Don't think that Pandas has any HopFS specific code in it.

<comment><AUTHOR> "In Pandas, the only change we need to make to our code, compared to a local filesystem, is to replace open_file(..) with h.open_file(..), where h is a file handle to HDFS/HopsFS."

HopsFS is a drop-in replacement for HDFS. Here's some more details on native HDFS connectors in Python by Wes McKinney -[http://wesmckinney.com/blog/python-hdfs- interfaces/](http://wesmckinney.com/blog/python-hdfs-interfaces/)

<comment><USER5> I think what you are saying is that you're wire-compatible with HDFS, and Pandas has access libraries (special file objects) that support HDFS.

</comment></comment></comment></comment></comment><comment><USER6> Gluster/Ceph support NVMe. That matrix isn't right

</comment><comment><USER7> Not by that graph, you don't. Pick a number in the middle: 2^24x10^6 words. That's 130TB, give or take. Sounds a lot, right? Not when you remember 16TB drives and 15 bay RAID enclosures exist...

</comment><comment><USER8> Can anyone working on this project comment on how HopsFS compares to similar alternatives? The post and the paper compare it to HDFS, but that's not really fair - HDFS was not designed to handle huge quantities of small files, and if you use it for that, you will of course not get good performance. Competitive offerings would be things like Qumulo, Amazon EFS, Objective FS, perhaps BeeGFS & Lustre, and so on.

<comment><AUTHOR> HopsFS is a derivative work (a fork) of HDFS - it is still 100% wire- compatible with HDFS. It has distributed metadata (not metadata stored on the heap of a single JVM). It also supports putting small files on NVMe storage - [https://www.logicalclocks.com/millions-and-millions-of- files...](https://www.logicalclocks.com/millions-and-millions-of-files-deep- learning-at-scale-with-hopsfs/) It's also open-source. So, it works in places HDFS works. HopsFS is, to the best of our knowledge, the only distributed filesystem with distributed metadata (apart from Google Collosus - the hidden jewel in their crown).

Lustre, BeeGFS are HPC filesystems, i believe. No data locality. Amazon EFS - feels like a MapR clone, but they expose some tunings - higher latency for higher throughput and larger clusters. We don't make that trade-off. I don't know enough about Qumulo and Objective FS to comment on them.

<comment><USER9> > HopsFS is, to the best of our knowledge, the only distributed filesystem > with distributed metadata

Then you haven't done your homework, since Gluster has had that architecture for a decade. And it supports real POSIX semantics, too - not just the HDFS subset.

<comment><USER10> Indeed, and (as notacoward will know) the HPC filesystems OrangeFS/PVFS2, Lustre, BeeGFS, and GPFS all have -- or can have -- distributed metadata, and POSIX semantics (or POSIX modulo locking).

<comment><USER9> Correct. I didn't mean to exclude those, but they weren't part of the original comparison. I didn't mention Ceph because somebody else already had, plus I'm not a maintainer for that as I am for Gluster. ;)

It might also be useful to point out some semi-useful distinctions regarding _levels_ of distribution. The PVFS2 lineage has had _fully_ distributed metadata for even longer than Gluster. OTOH, both Lustre and Ceph have separate metadata servers, more than one but still fewer than there are data servers. That doesn't necessarily limit scalability, but it does increase operational complexity. I'm honestly not sure about BeeGFS or GPFS; I don't pay much attention to them since one is only fake open source and the other doesn't even try. If we want to include proprietary offerings we could also add EFS, ObjectiveFS, OneFS, PanFS, etc.

BTW, for those here who know me, at $dayjob I've switched from working on Gluster to working on a more HDFS-like data store. (They chose not to call it a filesystem because it isn't one, which I consider a refreshing bit of honesty in a field full of pretenders.) It scales way beyond any of the things mentioned here, but it's proprietary so I can't compare notes as much as I was able to with Gluster. :(

<comment><AUTHOR> We have different interpretations of what we mean by "distributed system". I mean a group of servers that provide a single service. Or more precisely, metadata is a single distributed shared memory layer, where file systems semantics are same regardless on which partition/shard of the metadata a metadata operation is performed.

What Ceph and many others do is support partitioned metadata. One volume == one metadata partition. But cross partition metadata operations do not have the same semantics as intra-partition metadata operations. So, moving a file/dir between volumes (partitions) is not atomic - they do not solve the _hard_ consensus problem. We do. Gluster is also not solving the consensus problem. The end result is leaky filesystem abstractions. If you move a file between two directories, it may go fast. But between two different directories (across volumes), it can go very slow. Renaming a file in Gluster can cause it to move between hosts.

<comment><USER9> > Renaming a file in Gluster can cause it to move between hosts.

No. It does not. Within a volume, files will _always_ be renamed in place on the replicas where they currently reside. There are other problems with rename in Gluster, having to do with how the locations of files are tracked when they're not where they're "supposed" to be according to the hashing scheme, but movement of data as part of rename is not one of those problems.

"Across volumes" is not an interesting or relevant case, because the whole point of separate volumes in Gluster (and most other distributed filesystems) is to provide _complete_ isolation. You seem to have this idea that users would want to have lots of volumes and still move data around between them. Is there something weird about _your_ architecture that forces them to do that? Small volumes are operationally complex and stand resources. Generally, you'd have just one volume. Access control, quota, etc. could still be applied at the namespace or directory level, below volumes. It worked rather well at the world's largest Gluster installation, which I helped run for a year and a half, and I believe most other installations are similar.

Please try to learn about other systems before you make wild claims about them, lest your audience perceive them as deliberate lies to benefit your business. I don't think your claims about Ceph are anywhere near accurate either, but I'll leave those for one of my Ceph friends to address.

<comment><AUTHOR> My claim is exactly about what contributes a distributed namespace - i claim it is a namespace that spans multiple hosts and requires (distributed) consensus algorithms so that nodes that provide the metadata service (operations like rename) can execute operations only in metadata. Volumes are not just for isolation. They also partition the distributed namespace, and make it easier to scale - but at a cost. Operations that cross volumes do not solve the consensus problem, so they do not execute operations like move atomically. Whether that is a copy, then delete or something else is not the issue. They still do not provide a single distributed shared namespace abstraction with atomic operations.

<comment><USER9> "i claim it is a namespace that spans multiple hosts and requires (distributed) consensus algorithms..."

That's a very unique definition of "distributed". To many in the field, "spans multiple hosts" alone is sufficient. Some might refine it to preclude single nodes with special roles or by adding requirements such as a single view of the data etc., but it's a joke to add atomicity or implementation details such as how a consensus algorithm is used. There are plenty of distributed data stores that don't provide atomicity for all operations, and users are happy with that. There are plenty that use leader election instead of consensus to deal with consistency issues, and users are happy with that too. Inventing a novel and highly specific definition to exclude them seems more than a bit disingenuous. If we want to go that route, I'll point out that HopsFS is misnamed because it's _not a filesystem_ according to commonly used definitions. There are many hard problems that it wimps out on, that a real filesystem must address.

"Operations that cross volumes do not solve the consensus problem, so they do not execute operations like move atomically."

Again, you seem to be using "volume" in a very unique way. _Your architecture_ might misuse "volume" to mean some sort of internal convenience that's practically invisible to users, but others don't. To most, a volume is a self- contained universe, much like a virtual machine. Users are well aware that moving data between either will not have the same semantics as moving data within.

<comment><AUTHOR> My definition of distributed metadata for a FS is the same as you would have for a distributed database. If I shard a database across many servers and do not define semantics for transactions that cross shards, that is what you are talking about. I am talking about a system that supports cross partition transactions.

<comment><USER9> Let's recap. You've gone from claiming that Gluster doesn't have distributed metadata (untrue) to claiming that it copies data on rename (untrue) to claiming that renames aren't atomic across partitions (untrue because such partitions aren't even part of Gluster's architecture). Those are some very mobile goalposts. You've made similarly untrue claims about Ceph, which does have a sort of metadata partitioning but not the way you seem to think. Along the way, you've tried to redefine terms like "volume" which have well established domain specific meanings.

Such desperate attempts to disparage systems you obviously haven't even tried to understand are not helpful to users, and are insulting to people who have spent years solving hard problems to address those users' _actual_ requirements. Believe it or not, systems exist which don't work like yours, and which prioritize different features or guarantees, and there's demonstrated demand for those differentiators. If you want to learn about those differences, to compare and contrast in ways that actually advance the state of the art, please begin. I say "begin" because I see no evidence that you've attempted such a thing. Does your academic employer know that you're misinforming students?

<comment><AUTHOR> I disagree. I tried to define a distributed namespace for you, and you won't accept my definition. And you don't accept that one of the reasons volumes exist is to partition namespaces to make them easier to scale. Luckily for at me, peer-reviewed conferences and journals do accept my definition.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> What's the relevance of NVMe? Is that I/O latency still relevant in the presence of networking and scheduling overheads?

<comment><AUTHOR> If your V-100 GPU can process 150 images/sec, can S3 keep up? What if you now have 10 GPUs or 100 GPUs? Now, HDFS can't keep up. But HopsFS can....

<comment><USER5> Many people on these forums train directly from S3 and know how to get that kind of throughput. Like any blobject store, the key is to use multithreaded readers and high concurrency and set up your storage properly.

</comment></comment></comment><comment><USER12> I'm wondering if Kudu + Impala + RAPIDS + YARN +TonY might do instead... maybe more support?

<comment><AUTHOR> What's RAPIDS?

How long will Impala last, post Hortonworks merger? With Hive at position 15, and Impala at position 35, who do you think the merged entity will bet on? [https://db-engines.com/en/ranking](https://db-engines.com/en/ranking)

<comment><USER12> [https://news.ycombinator.com/item?id=18186392](https://news.ycombinator.com/item?id=18186392) for RAPIDS

Impala was the wrong thing to posit in the list above - maybe SPARK SQL is better as you can run that under YARN as well.

</comment></comment></comment><comment><USER13> What about Elastic File System?

<comment><AUTHOR> It's the same as s3 - no support for NVMe.

<comment><USER5> That's a non sequitur- EFS is a way to mount filesystems via NFS. What EFS does internally is kind of irrelevant- Amazon could implement it with NVMe, or whatever technology that they build internally, and the client would see the performance.

</comment></comment></comment><comment><USER14> we're using Spectrum Scale with its HDFS connector. while we're talking about options, I thought I'd mention it as well.

<comment><USER10> For what it's worth, you can do the same freely with at least OrangeFS, Gluster, and Lustre if you must do things a Hadoop-ish way; I assume also with Ceph.

</comment></comment><comment><USER15> Problem of both HDFS and HopFS - they written in Java, which makes them slower and eating much more memory than native filesystems like Ceph.</comment>


Jesse Schell on the Future of Gaming - <AUTHOR> http://gigaom.com/2010/02/22/video-reality-tv-iphone-the-future-of-technology-why-its-all-a-game/

<comment><USER2> Watch this. It has much, much, much wider application than the game industry.

</comment><comment><USER3> Large thread on this two days ago: <http://news.ycombinator.com/item?id=1142424></comment>


Writing software is hard - <AUTHOR> https://m.signalvnoise.com/writing-software-is-hard-388d5e982ad9#.nm9ek5de1 <comment><USER2> Posted previously: [https://news.ycombinator.com/item?id=13266506](https://news.ycombinator.com/item?id=13266506)

</comment><comment><USER3> I'd amend the title to read "writing _maintainable_ software is hard."

I currently work on a codebase, primarily written in C++ targeting embedded. There's an overwhelming amount of technical debt. There are literally comments in certain critical methods saying "//DO NOT MODIFY", because nobody knows what the method _actually_ does. Don't get me started on the maintainability headaches involved with the massive amount of inline ASM.

I have extremely intelligent people on my team. PhDs and patent holders. It becomes problematic when someone who's an expert in MATLAB attempts to translate her code into C++.

Keep in mind, this codebase is used by millions of people across the globe.

/rant

</comment><comment><USER4> If it was easy, we probably wouldn't be making so much money :)</comment>


Can anyone explain why TechCrunch decided to delete this comment? - <AUTHOR>

The post was in reference to the author not being impressed with many of the startups he was hearing about which he said was due to a lack of ambition.
I left a comment, but, to my surprise, I revisit the page to find that TC had deleted it.
Here was word for word the comment I left:
"But this article was 2/3's of the way written. I suppose I will finish it for you, since the subliminal reasoning for this, should you attempt to elucidate further yourself, may leave you at unease with your colleagues.
You've identified the real problem: the lack of ambition, but, you didn't diagnose why this came to be. Is it possible that this was an inadvertent result of the culture created through the consumption of tech media-- which is primarily click driven?
When the ecosystem is click driven we inevitably have a much greater frequency of startups being covered-- which means even if the startup is doing nothing new and/or the technology is rather unimpressive[1] they will still get coverage as long as the on-demand hype machine abides by the brands/names that are attached. It's become to easy and what we are witnessing is simply complacency.
The bills need to be paid, this is a reality that unfortunately you can't avert-- it's just unfortunate that when this done at such a frequency, potency is diluted and the inadvertent result is a greater the likelihood the companies that are doing more dynamic things fall through the cracks, creating a subpar standard to aspire to that isn't producing the optimal results of what the space is capable of.
I don't know man-- sometimes I just wish that monetary prevalence didn't always take precedence. Now I'm digressing to utopia..."
[1] edited- original post read impressive-- but was should have read unimpressive.
So, question is-- why was this comment deleted? <comment><AUTHOR> here is link to article: [http://techcrunch.com/2012/09/14/how-to-disrupt- petty-inconv...](http://techcrunch.com/2012/09/14/how-to-disrupt-petty- inconveniences/)

and here is the link to the comment on facebook which is still viewable, but as you will see not viewable on the post itself: <https://www.facebook.com/pages/Technito/258151947609961>

</comment><comment><USER2> Because it's TechCrunch.</comment>