Introducing the new Google Search Console - <AUTHOR> https://webmasters.googleblog.com/2018/01/introducing-new-search-console.html <comment><USER2> Instead of making us follow these Amp non-standards, Google should give an AMP logo in Google search results and preference (the same as an amp site) if the load time of the page is under 100ms. So now you have a choice, either DIY or use Google's tech to make your site fast.

This way it will create dozens of tech companies competing with AMP focused on making the web faster - the end result that Google supposedly wants and this way everyone wins!

<comment><USER3> And how would you do that? Would the browser load every single page on a Google search and then badge?

You can't test a page's load speed from Google's servers and then expect it to load for the user in the same way.

Especially because one of the key points behind AMP is the use of CDNs, especially global ones (like Google's, Microsoft's or Akamai), which greatly speed up load times while also reducing traffic for the ISPs.

<comment><USER4> This. The web host is likely going to try to spoof a low load time by sending a fake page to Google's web crawler. Then it'll be an endless cat and mouse. Also, there are more than a few websites on the internet. It sounds fairly unreasonable to keep fresh numbers on page load times.

<comment><USER5> They might be able to crossreference the result from their crawler with metrics from Chrome users that opt-in.

</comment></comment></comment></comment><comment><USER6> It will be like the new adwords. Looks pretty, but is missing half the features and eventually..once it comes out of its "beta" everyone who's been using this tool daily, will regret not staying on the older version.

<comment><USER7> Webmaster tools never had many features, so I'd be surprised if they reduced the featureset significantly.

</comment></comment><comment><USER8> Here is the link to the new search console:

[https://search.google.com/search-console](https://search.google.com/search- console)

My sites aren't available yet but looking forward to trying it

<comment><USER9> ...and if you bookmark that URL for later, it'll simply be titled, "Status". Thanks Google.

<comment><USER10> Haha, the search engine gigant doesn't have to care about such :-)

</comment></comment></comment><comment><USER11> Doesn't work well in Firefox. WTF google?

Hangouts doesn't work. Now search console too?

<comment><USER12> But it's Safari that's the new IE, right? Killing the 'open web'?

</comment></comment><comment><USER13> Perhaps I'm missing something but why do we need a "console"?

Why can't we just post these search-engine related settings in a special file on our servers?

That way, we don't have to manage 10 consoles for 10 different search engines.

<comment><USER14> The Google console is quite helpful in tracking errors on your page. You can obviously do that yourself but for small projects, using Google's console is easier.

Search analytics is another topic that obviously needs a page on their side.

Overall I'm happy with Google's console, especially compared to Bing.

<comment><USER13> > The Google console is quite helpful in tracking errors on your page.

Google could publish an open-source tool that you can run to check your page for errors.

</comment></comment></comment><comment><USER15> Bit of a weird introduction, as it does not contain a URL to the actual new console (apparently it has a different URL compared to the current 'webmaster tools').

Also, once you do find the URL, nothing seems to work (at least not for me on FF/chrome/safari). Can't see, select or add my websites to the console.

</comment><comment><USER16> A link to the Search Console would have been great to have in the blog post.

<comment><USER16> As posted in another thread here the link seems to be [https://search.google.com/search-console](https://search.google.com/search- console)

<comment><USER17> Shows zero of my properties and property selection menu appears to be broken.

Available only to select few users, maybe?

</comment></comment></comment><comment><USER18> Search console is the most powerful tool available for SEO. It's weird that it was neglected for so many years.

Nice to see they still work on it _sometimes_. 16 months of data is massive. Now you have to export everything every 3 months to keep historical data.

<comment><USER19> It's not available to me yet.

Besides the old 3-month limit, the other annoyance was the search terms they provided would cut in and out. So sometimes they'd give you SERP info on certain keywords and then a few months later, they'd eliminate reporting on those keywords.

</comment></comment><comment><USER20> That was about time. I was always flabbergasted by how rough and confusing the search console was. I really like how Google are revamping their marketing/admin tools. The Google Optimize [0] tool is pretty nice and the only A/B testing tool I know that is already pretty powerful in the free tier. 0: [https://www.google.com/analytics/optimize/features/](https://www.google.com/analytics/optimize/features/)

<comment><USER21> Relevant: "Conversion rates are much lower than they should be.".... "For Google, conversion rate is based on visits instead of visitors." Source: [https://www.abtasty.com/blog/data-scientist-google- optimize/](https://www.abtasty.com/blog/data-scientist-google-optimize/)

</comment></comment><comment><USER22> dang when i saw console i was like awesome! then i went to the website :( our definitions of console are different :(

</comment><comment><USER23> How does it compare, feature wise to the old one?

<comment><USER24> Wondering the same thing! These new releases tend to get a lot prettier but they almost always lose important, lesser-known features

<comment><USER25> My favorite is still Google Voice.

Google: "Ok, lets finally update the UX from a decade ago so we can thread messages... oh yeah but you can't actually configure your voicemail account with your carrier, or even view what your voicemail number is until you revert to legacy google voice. Actually just use legacy if you want to be able to reliably delete messages too"

</comment></comment></comment><comment><USER26> This is definitely a different era of google

<comment><USER27> In what regard?</comment></comment>


Audiophilia Forever: An Expensive New Year's Shopping Guide - <AUTHOR> https://www.newyorker.com/culture/culture-desk/audiophilia-forever-an-expensive-new-years-shopping-guide?mbid=nl_Daily%20011318%20Nonsubs&CNDID=32548661&spMailingID=12738773&spUserID=MTM1NzUzOTQwODk1S0&spJobID=1321152218&spReportId=MTMyMTE1MjIxOAS2 <comment><USER2> Not a word about investing into the acoustics of his living room. So probably just another clueless cargo-cult follower writing to make a buck advertising the new tech that needs to be sold.</comment>


Tesla's dream: Wirelessly powered devices that get their energy out of thin air - <AUTHOR> http://americablog.com/2013/08/teslas-dream-the-amazing-possiblities-of-wireless-electrical-power.html

<comment><USER2> Yet another reason for why electric cars have so much potential in the long run compared to gasoline-powered cars, and that's besides the fact that batteries themselves can double the storage capacity every 10 years or so, and therefore the range (or same range for lower price), or that electric cars can be charged for free with energy from solar panels.</comment>


Italy, America, and the iPhone - <AUTHOR> http://mattrichman.net/post/137631916338/italy-america-and-the-iphone <comment><USER2> The "distribution" thing is a red herring.

Most Italians just get their phones from their carriers anyway (like Vodafone, etc), same as with Americans, and they have equal reach that Walmart or AT&T/Verizon/T-Mobile etc would have in the US.

It's all about the price (which, besides Italy being a poorer country, there's also a substantially larger asking price for an iPhone compared to the US -- like a 20-30% price hike).

Plus Apple support and presence has never been as big in Europe as in the US anyway. They may be chasing the Chinese and Japanese markets, but they have a ho hum network (for both sales and support) outside of maybe London and Paris.

The status symbol priorities also play a role, it's true. But more would buy the iPhone is those two things (price, support) were in order.</comment>


White South Carolina Police Officer Shoots Fleeing Black Man in Back - <AUTHOR> http://daringfireball.net/linked/2015/04/07/south-carolina <comment><AUTHOR> Wait until every cop in America has a video camera running at all times, and lack of video footage implies they have something to hide.

<comment><USER2> Wait until it still doesn't make a single bit of difference.

<comment><AUTHOR> A cop has been charged with murder, while yesterday he was being lauded for a good shoot. Not "a single bit of difference"?

<comment><USER2> They charged the guy who killed Eric Garner, and look what happened there.

<comment><AUTHOR> No, they did not charge the cop that killed Eric Garner. So this case is already different.</comment></comment></comment></comment></comment>


Algorithm Finds Thousands of Unknown Drug Interaction Side Effects - <AUTHOR> http://www.nature.com/news/drug-data-reveal-sneaky-side-effects-1.10220 <comment><USER2> This should come as no surprise as our effort at better living through chemistry is barely in its infancy. Millions of people take drugs prescribed to them by their physicians with the implicit understanding that they are safe because the USDA says they are when in fact we have precious little idea if they really are.

The potential revenue from new pharmaceuticals is huge and the pressure to get new drugs out into the market is immense. It's initially comforting to see that people are working on this problem of drug interactions since as the article mentions, we couldn't do it in the clinical trials. Still, the cynic in me thinks this is just another step in getting everyone taking more and more prescription medicines. If the pharmaceutical industry can show that two drugs have no interactive side effects, it's much easier to convince a patient to begin taking a new drug. Western medicine is already heavily dependent on fixing the symptoms instead of the cause.

It's clear that in many cases we have almost no clue why a particular drug works. It's thrilling to see risks removed from the system by finding drug interactions previously unknown. However, it's frightening how much more likely it is that physicians will rely on prescribing a pill because it's easier than convincing the patient to change.

<comment><USER3> You mean the FDA? No offense, but just because you are well versed in one subject does not mean you really understand the idiosyncracies of another. Every time I see a drug / medicine related thread on HN I cringe. Much in the same way you would if you read a technology related post on a forum geared towards doctors and pharmacists.

<comment><USER4> As a pharmacist, I approve this comment!

</comment></comment></comment><comment><USER5> Just to be fair, the original title of the research article is "Data-Driven _Prediction_ of Drug Effects and Interactions" ( <http://stm.sciencemag.org/content/4/125/125ra31> ). Emphasis on 'Prediction' mine.

47 are then supported by retrospective analysis of an EMR database.

</comment><comment><USER6> Obviously this algorithm has lots of false positives.

<comment><USER7> Why is that obvious?

<comment><USER8> A false positive costs much lower than a false negative in this instance so it should follow that FP should be more common. However, without the data it is still speculation.

<comment><USER5> Whether false positives or false negatives are more harmful is a difficult question. A false positive might stop millions from taking a life-saving drug. A false negative might cause millions to take a life-threatening drug.

<comment><USER8> By nature a false positive interaction could be re-tested more easily than a false negative though, correct? Also, that fits in with medical "first, do no harm" framework more readily.

<comment><USER5> Regarding your second point, I agree. But it must be balanced against the other tenets of medical ethics. At any rate, this seems to be an important study. I look forward to seeing the same study carried out across different EMR databases and then meta-analyzed. That will be extremely powerful.

</comment></comment></comment></comment></comment></comment><comment><USER9> Anyone know if that dataset is publicly available?

<comment><USER10> Yes, its public.

I think the below link should have some of the data.

[http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInforma...](http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Surveillance/ucm082193)</comment></comment>


Ask HN: Anybody looking to hire a noob developer with weak skills? - <AUTHOR> I just graduated and I am looking for a position as a front end developer. I only have very basic PHP and Javascript skills. I'm now going through the difficult task of looking for a job or an internship but it seems like everyone wants more expert level in that area.<p>Can someone offer up any advice? <comment><USER3> Getting your first job in a field can be a pain, if I had a dollar for every time I saw an ad for a graduate position that wanted experience that you could only get by already having a similar job, I'd have been able to retire early...

Companies look for experience since it demonstrates your ability, do you have a design portfolio, even if it's just mock up sites you've done in your own time to learn?

I've done a fair bit of hiring for the company I work at (unfortunately we're not hiring at the moment) and if a candidate with little experience can show us the kind of things they can build we regard it more highly than someone who has had a job for a year but hasn't shown much initiative beyond being given a task list and doing it.

</comment><comment><USER4> Find a place where you can talk to someone in power (i.e., not just HR) and demonstrate your passion for what the organization does (it helps if you actually have some).

I wiggled my way into my first programming job because I really liked what they did, and made it clear to them. My boss later told me that the code samples I submitted weren't very good, but anyone can learn to write code--He hired me because I really demonstrated a desire to do the work.

</comment><comment><USER5> You could sell yourself a bit better. "noob developer with weak skills" doesn't scream "hire me."

Embrace the positives. Say that you're a recent graduate with a good basic grounding in programming in X, Y, Z and willing to learn whatever you need. You're willing to relocate and are flexible about hiring. Say that. List any good examples of teamwork and/or projects that you have done. Believe in yourself.

Good luck.

</comment><comment><USER6> It may be hard for you to find a job directly. A lot of lower level positions are filled by temp agencies. I would recommend looking into that. Think of a temp agency like an internship where you actually get paid decently - <USER2> frequently turns into a full time position.

Another way to go is to freelance. There are always small businesses that need little help here and there with their website. The key to that is to build relationships.

For developers who have more experience and also looking for jobs, going to events and speaking with recruiters can work well. Doesn't work for less experienced though, because it's not worth going through a recruiter for that ($$).

</comment><comment><USER7> Elance

</comment><comment><USER8> I looked on your website and it love the design - its beautiful.Movetampabay looks great as well. I think you should advertise yourself as a user interface designer. You said you are not much of a designer, but I think otherwise. If you really hate doing web design,then you should switch,but I think you have the talent to do well at it.

I also think you should put a picture of youself on your website.

Good Luck!

</comment><comment><USER9> You should say where you are.

<comment><AUTHOR> Sorry, I didn't mention it because I am willing to go any where. I am in Florida now.

</comment></comment><comment><USER10> Whats "weak?"

Where'd you go to school and do you have links to some stuff you've done? Were you a CS Major? Do you do design?

I'm an amateur too but I've gotten far enough to hack something together. Could use some help (specifically with JS and AJAX on this mobile webapp im working on. If that sounds up your alley, comment with your twitter handle, we're funded (angel) and would love to have another hand on deck.

Cant pay a ton of course while we're trying to keep burn low but if you've got hustle and are willing to learn, I'd love to chat w/ you.

<comment><AUTHOR> Weak meaning my skills are on the novice level for javascript and PHP which is what is needed for a front end job.

I went to school at the Art Institute for Web Design but I realized I am not much of a designer and love to code.

I don't have a twitter but my email and portfolio with a couple stuff is up in my profile.

</comment></comment><comment><USER11> Similar question, north SF Bay Area (Sonoma, Marin). I got a BS in CS last year and all of the entry-level jobs require 3+ years experience. Even the fast-food jobs require prior experience, no joke. I am currently back in school to refresh my IT skills, but I will be in the job market again in two months so I may as well start looking now. Is anyone in my area hiring inexperienced developers?

<comment><USER12> Well, you have to get creative with what you call experience. Don't lie, just "get creative". If you spent three years in school coding and working on programming projects, you have three years of development experience.</comment></comment>


Scrambled Eggs and Serifs - <AUTHOR> http://www.frerejones.com/blog/scrambled-eggs-and-serifs/

<comment><USER2> This problem isn't unique to fonts.

I used to joke to my students that the hardest part of programming is coming up with names -- for variables, functions, classes, modules, packages, etc. I still tell them that, but now I'm relatively serious about it. ("Relatively", because I don't know that it's the _hardest_ part, but it certainly is hard.)

<comment><USER3> "there are two hard things in computer science: cache invalidation, naming things, and off-by-one errors"

But you're totally right. I was watching my friend code last night, we both have the same # of years experience, and I was like "what does that function name even mean? How about 'FindNeighbors' instead of 'CheckNeighbors' ". I'm sure I do the same thing in my own code.

If you still teach, do you ever have your students do code review on eachother's code?

<comment><USER2> > If you still teach, do you ever have your students do code review on > eachother's code?

Yes, I teach. No I haven't done that. But it's a great idea. I'll think about it.

<ggchappell proceeds to think about it>

<comment><USER3> In the more hardcore-about-code-review job I had, the code reviewer was as responsible for bugs / broken stuff that came out as the person who wrote the code. And in that job we didn't just read the code briefly, you'd load up the branch and test their code. I'm not sure that it was the best practice, it slowed everything down and led to a more formal culture.

If you do this, in order to incentivize the code reviewer you might actually make 10% of the grade based on the code submitted by the person they reviewed. (Granted that could be kind of harsh, and I always hated group projects in school.)

<comment><USER4> Probably best to make code review an extra credit exercise because you don't want to ever penalize a good student by tying their rewards to the actions of someone who might be slacking.</comment></comment></comment></comment></comment>


Is the U.S. stock market rigged? [video] - <AUTHOR> http://www.cbsnews.com/news/is-the-us-stock-market-rigged/ <comment><USER2> I thought this was going to be interesting but it turns out it was just an advertisement for IEX. They brought out a) guy selling a book about IEX (Michael Lewis) b) CEO of IEX (Brad Katsuyama) c) Chief Strategy Officer of IEX (Ronan Ryan) and d) IEX investor (David Einhorn). No shit they are going to make it sound like the markets are rigged and IEX is the answer.

<comment><USER3> As far as I know, IEX is of no use for the average joe investor. We aren't buying and selling in blocks of 1000 or 10,000.

Believe what you want, but there is a reason companies pay big money for super low latency.

</comment></comment><comment><USER4> Instead of being angry about hyperbole and ignorance, my new policy for HN story's touching HFT is to just answer questions. I build HFT systems for a living. My experience is only in commodities futures but I can try to explain equities markets as well. Ask away instead of passing on bad info.

<comment><USER5> What language do you write your trading platform in? What FIX engine, if any do you use?

Do you colo with a broker or go directly to the exchanges?

Do you arbitrage the roll of ETFs that track commodity futures (e.g., VXX, USO)?

Do you roll your own backtester or use an open-source or off the shelf one?

In general, what is the win/loss and Kelly's expected payoff of your various strategies? Do you aim for out-of-the-ball-park home run's or consistent singles? Thanks.

<comment><USER4> My experiences have been in C++, JVM languages and FPGA. I've used proprietary FIX solutions, Onixs, and QuickFIX.

I've always colo'd with the exchanges I've traded on.

I've never traded a US equity.

I've used both off the shelf analytics solutions and built my own. (I don't design trades typically, rather I write systems for folks that do).

I don't usually go in for deals that involve expected payouts of PnL. I build technical solutions for a fixed price for people that have a particular technical requirements set. I'm not sure I even know what a Kelly's expected payoff is.

<comment><USER5> Thanks, kasey_junk for your detailed answers. Greatly appreciate it. Not trying to get any inside or competitive info as I do more options algorithmic trading on a time frame of weeks and months. But I'm always curious how the other half live.

Are you using Scala specifically as your JVM language, CUDA for your FPGA? Also, have you dealt with Complex Event Processing (CEP) as way to combat stale quotes and feeds that require long time to process? These are area's that I haven't explored, so curious about your $0.02 if you have any experience/opinions, thanks.

<comment><USER4> I prefer to work in Scala on the JVM if I can, but Java is still very prominent. CUDA is most common on FPGA though I haven't worked in it for awhile. Have not dealt with CPE in trading.

</comment></comment></comment></comment></comment><comment><USER6> Two big problems with the market:

\- Shutting down overnight, lots of schemes happen after hours or pre-market, why don't we have an always on market?

\- HFT has run amok and they aren't that smart yet or they are too smart: [http://buzz.money.cnn.com/2014/03/26/whoops-shares-named- ocu...](http://buzz.money.cnn.com/2014/03/26/whoops-shares-named-oculus- spike/) \+ [http://kotaku.com/people-are-accidentally-buying-stocks- with...](http://kotaku.com/people-are-accidentally-buying-stocks-with-the- name-oc-1552319497) (people aren't buying these, machines did)

It is no longer a market a small investor can make money in except for buying after bad news and long term. Investors need HFT to compete now.

<comment><USER7> >It is no longer a market a small investor can make money in except for buying after bad news and long term. Investors need HFT to compete now.

How is this a bad thing?

A person is still quite able to invest, in the sense of buying an index fund. Why does it matter if a small investor can't profit by trading on news? Isn't this a sign that the market is efficient?

<comment><USER6> Yes, but most of the fat is trimmed by the HFT hedge funds, even long term now, the last vestige of growing wealth.

It is a very good thing that there is buying on the dip as a natural force of the market. The good news market is almost totally owned by HFT and already pre-baked.

But the speed of the algos trims much of what would go to smaller investors that really are needed just like consumers. More volume and movement is always good but being so efficient it takes most of the gains from the support structure of the market harms confidence.

Lots of the gaming happens pre-market and post-market where small investors can't always play and where the HFTs own by scraping 80-90% off the top, over time the delta is big. HFT does have benefits being so efficient though, may even help prevent bubbles because it can trade without emotion.

</comment></comment></comment><comment><USER8> Hahaha. Yes. When big brokers can peek at trade data before the public and do HFT before you can even look at a chart and click "buy," it's pretty damn rigged.

<comment><USER9> Ok, but this shouldn't really affect anyone but other HFTs, right? Maybe you get a very slightly different price but that shouldn't matter if you are holding investments long enough. If you are trying to outtrade HFTs and losing, I'm not sure that shows the stock market is rigged.

<comment><USER10> Are you saying you're okay with the price at which you're purchasing a security being driven up (if infinitesimally), _at your expense_ , so that someone else can pocket the delta?

<comment><USER9> I guess specifically I didn't like the way he phrased it, as if they are stealing data somehow. In fact they bought the access. Other than a clock rate (or 60km limiting box, as shown in the video) I'm not sure how you would create rules against HFT. I'm also not convinced if HFT is a net win or loss for economy, I'm not even sure how to consider it.

<comment><USER8> How does buying access matter? It's like giving access to see other player's cards in poker (because you're a high roller) and then making bets based on it. The rest of the players only see their cards and the river. That's not a fair game.

</comment></comment></comment></comment></comment><comment><USER11> Their solution at IEX was to coil a bunch of fiber so that high speed traders' orders get slowed down...I'd have to know more about why they thought that was necessary, but on its face this seems like complete overkill. Obviously there are probably reasons that I don't understand but it seems like this problem would be really easy to solve in software. For instance, I don't see why they don't just trade on a step interval. Basically everyone who arrives within 1 second of each other should have their trades executed simultaneously. Thus you have step of 1 second - nobody can get ahead of you because the travelling time between any two exchanges for even the slowest person is much, much less than 1 second. Does this make the market too slow? Just make the step interval whatever the slowest latency is for any given round trip at any given moment. That would still be on the order of milliseconds but nobody would be able to front-run you because any trade you make will be ahead of theirs or at the very worst at the same time as theirs.

<comment><USER4> Discrete time matching always forgets about the case when there are more buyers than sellers (or vice versa). If 2 people are buying at price X and 3 people are selling at price X, which 2 sellers get "filled"?

In most markets currently that is done in FIFO order. Discrete time values do not eliminate the race into/out of a level.

<comment><USER12> A fair lottery would solve that.

<comment><USER4> There is a pretty serious bias in the markets against non-repeatable matching algorithms. Explaining to the rest of the market participants why a given trader (or group) had better than average luck is not something that most exchanges are interested in doing.

<comment><USER11> I suppose by repeatable you mean that if you replayed the sequence of orders - both buy and sell - the matching algorithm would match them the exact same way as it did originally?

I understand the desire to have deterministic trading algorithms, but they're already in the unenviable position of having to explain why some traders get information faster than others, which seems like it's even harder to explain.

Plus, the algorithm could be repeatable if you just replay the sequence of random numbers that led to that order - i.e. if the sequence of numbers were made public along with the executed transactions.

You can measure the randomness of a random number generator, so it would be easy to demonstrate the the orders were picked in a perfectly or near- perfectly random way. You could even hook the random generator up to a quantum event like a decaying atom to get even more evenly distributed randomness. I'm sure for something as important as fair market conditions the institutions in question would spare no expense - neither to have the trades executed randomly but to prove to the public that they are generated randomly.

<comment><USER4> So, A) you are making assumptions about fairness that no trader would ever make. There is literally no argument, mathematical or otherwise that you can create that would explain to them why the idiot in the desk next to them gets a bigger bonus than they do based on trade execution.

B) You are making assumptions about market information "fairness" that no trader would ever make. The exchanges have no problem explaining why some folks have faster information than others because they always have.

Pit traders had an advantage over phone brokers. Phone brokers had an advantage over which ever poor slobs they could get on the hook. No one on the exchange side of things thinks people all have the same information at the same time, if that was a requirement how would you time coffee breaks?

<comment><USER12> _if that was a requirement how would you time coffee breaks?_

A fair lottery would solve that.

You're using an interesting sense of bias, where a fair lottery is in fact perfectly _unbiased_.

There are lots of things which can cause an arbitrary assignment of winnings to a given trader. Go back and re-read _A Random Walk Down Wall Street_ if you've forgotten. The _problem_ isn't the randomness, it's the bias, particularly to those with an inside advantage and edge.

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> Question for kasey_junk, seeing as s/he is taking questions :D

Lewis: "This form of front running is legal. It's legalized front running. It is crazy that it's legal for some people to get advance news on prices and other --[information on] what other investors are doing. It's just nuts. It shouldn't happen."

Do you agree or disagree with Lewis' assessment of the state of HFT?

<comment><USER4> I completely disagree. First as a caveat, I've never traded US equities. That said, the idea that all participants in the markets have the exact same access to information that every other participant has is ludicrous. That has never been the case, and can never be the case.

Trading costs as a whole are lower now than they have ever been. Is that entirely due to HFT? No, of course not, but any "fix" you could put in place would most likely have huge negative unintended consequences. Let's all take our cheap investments to the bank and be happy that we don't have to pay 6'8" gorilla's to make our trades.

</comment></comment><comment><USER14> tl;dr If you are buying stocks as an individual for less than 10 years, don't: you are going to get killed.

see also: buy index funds

<comment><USER15> don't write programs if you have less than 10 years of programming experience; your horrible code will make programmers all look bad. BTW. no one's gonna hire you if you have less than 10 years of programming experience in a specific language.

I always enjoy people's weird logic.

<comment><USER9> No, I think this is more "don't show up at NFL games expecting the coach to throw you on the field because you play in a weekend league."

</comment></comment></comment><comment><USER16> the thing i like about this story is that they found a problem (front running) and are trying to create a better company. rather than trying to get the government involved.

<comment><USER17> They are not front running. Look up what it actually means.

<comment><USER16> Yeh you're right. It's similar to front running, but not illegal.

</comment></comment></comment><comment><USER18> Please add "[Autoplaying video w/ audio]" to the submission title.

<comment><USER19> We usually just add "[video]" when a video autoplays, even when there's an accompanying text article as well.

Do I detect an annoyance with autoplaying videos with accompanying audio in your suggestion? If so, I share it.

<comment><USER18> _We usually just add "[video]" when a video autoplays_

Fair enough, I couldn't remember if the usual tag for this was "video", "audio warning" or something else.

 _Do I detect an annoyance with autoplaying videos with accompanying audio in your suggestion?_

Indeed. I don't care about silent video, but when I've scrolled halfway down the page and then the video at the top of the page starts making noise...

</comment></comment></comment><comment><USER20> Why is it that the intermediate exchange in NJ is able to see the desired trade? Shouldn't that information be encrypted?

</comment><comment><USER21> I always thought that the digital stock market should take a page from the cryptocurrencies and add a digital proof-of-work that will cap the speed of a transition. A "temporal tax" would keep the system from wild swings and reactions, similar to what we saw a few months back.

<comment><USER22> What's the point? The first person to get the transaction still makes money, so there is still an incentive to get the information first. Everything will just be on an arbitrary delay. And what's wrong with swings and reactions? That's supposed to happen as new information comes in which changes things. If the people reacting are wrong, they lose money and the people who see it make money.

</comment></comment><comment><USER23> Don't senators and congressperson's portfolios do a whole lot better than the S&P 500?

</comment><comment><USER24> I understand how HFTs are making their money; however, how does this drive up market? Would they not be able to make the same money in a downmarket (a delta is a delta, whether you are going up or down)? All I can see this affecting is the billions (trillions?) of dollars in HFTs amplifying whatever the market is doing.

<comment><USER25> That was an oversimplification done for the audience's benefit. If the video were a math class, he would have said "without loss of generality, assume that the market is going up."

</comment></comment><comment><USER26> Katsuyama says: "When I'm trying to place a large order, the price goes up!" So, what's wrong with that? Have we cancelled the laws of supply and demand now? Let's beat up Adam Smith now!

<comment><USER12> The explicit problem is that when Katsuyama changed its strategy to time arrival of trades to all exchanges simultaneously, the effect vanished.

This isn't a supply and demand issue, it's HFT being able to detect an impending large movement and explicitly front-run it, at other exchanges.

</comment></comment><comment><USER27> Is the US Stock market rigged? Were you born yesterday? Of course it's rigged. The entire world is rigged.

<comment><USER19> This is a vacuous statement of indignation. We don't want those on Hacker News. Please don't post them, even when the sentiment itself is justified.

Re-read what you've posted and, if it mostly just expresses indignation, delete it. Then feel happy: you just put the signal/noise ratio up a tick for all of us.

Not picking on one commenter—this is for everybody.</comment></comment>


Computer Architecture for Network Engineers - <AUTHOR> https://github.com/lukego/blog/issues/18 <comment><USER2> A tiny nitpick, and I don't mean to be pedantic, but "in-flight" usually is used to describe an instruction in the functional units, not "in the pipeline".

Other than that it's a pretty good introductory article! If you'd like something a little heartier, but still not quite Hennessy and Patterson's QCA (although I highly recommend it), try this article: [http://www.lighterra.com/papers/modernmicroprocessors/](http://www.lighterra.com/papers/modernmicroprocessors/)

</comment><comment><USER3> As a sometimes CPU architect/logic designer and also occasional protocol engineer I think that this is stretching an analogy a bit too far, individually some of the TCP A is like CPU B is probably fair, but the totality? nah

</comment><comment><USER4> [https://fgiesen.wordpress.com/2014/03/23/networks-all-the- wa...](https://fgiesen.wordpress.com/2014/03/23/networks-all-the-way-down/)

[https://fgiesen.wordpress.com/2014/03/25/networks-all-the- wa...](https://fgiesen.wordpress.com/2014/03/25/networks-all-the-way-down- part-2/)

</comment><comment><USER5> It is indeed a good way of looking at architecture and networking. But to me it is more of a hindsight tool or validation of things you already know, than a learning resource for someone who doesn't know details of architecture.

</comment><comment><USER6> I really like the analogy. If you do make the talk/tutorial with some pretty graphics to show the comparison, then I'd like to watch it!

</comment><comment><USER7> I know a _little_ about networking to appreciate the simulation table. I can't vouch for the accuracy. The author does invite critique.</comment>


A Garden Variety of Bloom Filters - <AUTHOR> http://matthias.vallentin.net/blog/2011/06/a-garden-variety-of-bloom-filters/

<comment><USER2> For those of you who don't know what a Bloom Filter is or the concept isn't quite clear, you should check out this site: <http://www.jasondavies.com/bloomfilter/>

He has a nice interactive demo with an explanation on how it works.

<comment><USER3> Unless I'm misunderstanding how a Bloom Filter works, I don't think the interactive demo is working correctly.

If you add letters a - z as keys, then do a search for some numbers, such as the number 6, you get "Probably there". I was under the impression that a Bloom Filter could determine if something was definitely NOT in the data set, in which case this the example isn't working.

Please correct me if I'm wrong :)

EDIT: After playing around with it some more I'm questioning if I do fully understand how it is supposed to work. It seems there are many cases of "Probably there" when the key definitely is not. So I'm guessing in those cases you'd want to search the set to make sure?

<comment><USER4> You are slightly confused. False positives do indeed happen, but false negatives can never happen (if one of the bits aren't set for the corresponding hash, then it is guaranteed to have never been added before).

Collisions in hash tables are related to false positives in Bloom Filters.

Also, regarding your last statement: You don't always need to look at the actual set in some cases, but for a lot you do. Bloom Filters give you a fast, compact way to prevent a lot of unnecessary searching. Spell Checkers, for example, you could reduce the amount of times you had to search through the dictionary, or for finding files or something on disk, you could eliminate a percentage of searches for the files. How much these can be reduced is related to the size of your Bloom filter.

<comment><USER3> Ok that makes sense, thanks for taking the time to explain. Seems obvious now! :)</comment></comment></comment></comment>


Things you can't do in Rust: tail recursion - andrew-lucker https://medium.com/@andrew_subarctic/things-you-cant-do-in-rust-tail-recursion-4eab052cc56e <comment><AUTHOR> Do people often rely on tail-recursion for real-world programs? Before anyone wonders if I know what it is, yes, I do, and yes, I've used it appropriately, but it feels so... academic.

<comment> andrew-lucker There is a whole style of programming dedicated to tail-recursion called Continuation Passing Style. It is not usually useful for programmers directly to use, because it is so complicated to read/write, but it is very useful for compilers to generate code in CPS.

If you wanted to, for example, write a new experimental functional language and reuse part of the Rust compiler to sanity check and generate the executable, then tail-recursion would be really important.

Similarly, if you create something like a parser generator and don't have tail recursion optimization, then you are going to run out of stack space before being able to parse stuff. So, there are lots of practical applications that depend on this feature too.

<comment><AUTHOR> Ah, thanks, makes good sense. I think of "write a new experimental functional language and reuse part of the Rust compiler" as somewhat "academic", but matter of taste. But for production parsers, I think I'd try to avoid using the stack or limit the stack, anyway.

</comment></comment></comment><comment><USER2> This example is just infinite recursion. You will get a similar result in, say, Python.

Maybe the author wanted to say that Rust (currently) doesn't perform an optimization called tail call elimination that would make this program consume zero stack, which would make it hang instead of erroring out.

Some languages (like Scheme) are specified to optimize tail calls, so you can rely on this behaviour there.

<comment><USER3> To be clear about this, LLVM will sometimes do TCO. But Rust doesn't guarantee it. We have reserved a keyword to make this happen sometimes in the future, possibly.</comment></comment>


YouTube-mp3 Situation update - <AUTHOR> http://www.youtube-mp3.org/situation-update? Poor guy really thinks he can win against Google and the RIAA.. <comment><USER2> Google closes down, without apparent (and often without real) reason, many adsense and adwords. This is probably because of the sheer size of users of these services. I know 100s of people personally who have lost 10.000s to 100.000s of $ in revenue because of it while they still have no idea what they did wrong. In this case it's a bit more clear; whether I agree or not, I can see why they won't want you to offer that service.

I don't think any petition will get them to return on their stance here, but Google+ with a lot of influential people (from Google if possible) would be your best chance IMHO.

</comment><comment><USER3> why doesn't youtube just wipo that domain?</comment>


Startup/Bootstrapped Marketing Recap | Tactical Steps - <AUTHOR> http://www.craigkerstiens.com/2011/03/07/startupbootstrapped-marketing-recap/ <comment><USER2> Great reference</comment>


Mob Wars, the million-dollar-a-month independent Facebook app, may legally belong to SGN - <AUTHOR> http://venturebeat.com/2008/08/26/mob-wars-the-million-dollar-a-month-independent-facebook-app-may-legally-belong-to-sgn/ <comment><USER2> Something similar happened.

I initiated, built and launched this application over a weekend. I evangelized it within the company (so that I get time and resources) and grew this application on facebook to a strong user base ~.5 M in 2 months. The company even changed track from their original business.

The founders refused to spare even ~half percent stake in the company. I quit. David Maestri did otherwise.

The application got acquired for couple of million dollars. And I made $1000. The size of deal was definitely because of the strong founding team. But I am confident I would have driven it to a self sustaining business or sell it for may be a million or close.

The company definitely deserves a lump sum. I used their resources. But I was the one who really bet big and convinced everyone when people actually opposed on time being spent on a facebook application. I largely contributed to the vision - which they eventually are getting to. All I expected was some gratification for myself.

How does one deal with such a situation?

<comment><USER3> You do what you did: you quit.

You will never get rich working for someone else. You may make a few million over the course of years, but without ownership, you'll never build true wealth.

Chalk it up to learning, start your own thing, don't look back. Or come and work for me ;-)

<comment><USER2> :) Thanks Ryan.

I never regretted that decision of mine. I believe the dots will connect going forward. If things go well I should be bootstrapping my firm by Oct-Nov.

But this incident disturbed the faith in me about work, people and trust. It has been hard to get over it completely.

<comment><USER4> I sometimes get very angry at people that have screwed me over. You don't need to get over it completely - learn from the lessons.

But don't let it negatively impact your future decisions, the best revenge is living well.

<comment><USER5> I concur. Holding a grudge hurts no one but yourself. Anger damages the angry person far more than the target of the anger. It merely distracts you from your goals.

</comment></comment></comment></comment></comment><comment><USER3> That completely sucks. I can't believe that some company thinks they're entitled to what a developer built in his spare time, using no company resources (assuming that's the case, of course). Fuck being owned by some piece of shit corporate machine.

I feel like I read somewhere that these types of agreements are usually not enforceable in California, but Freewebs looks like they're headquartered in Maryland (though SGN is in CA), so it might depend on where he was working when this occurred. Regardless, it sounds like bullshit. Hope he fights and wins.

<comment><USER6> I predict SGN doesn't want the application but is instead looking for an out of court settlement to grab a piece of the pie.

<comment><USER3> It comes down to petty greed. But it's even worse than greed. It's unbridled greed with no integrity whatsoever. Why build something solid that adds value when you can steal it from someone else who toiled and risked to do so?

</comment></comment></comment><comment><USER7> "The tragic part for Maestri, at least is that if he'd just waited until he'd left Freewebs to begin development, his former employer would have a far harder time making the charges stick."

So legally, the advice that I've heard (even here at hacker news) of keeping your day job, and work on your project on the side is usually wrong. And even more so if you work at a tech company already.

<comment><USER8> That's a great question!

It depends on your employment contract and the law in your state/country. It also depends greatly on the area your startup's in and whether your startup is even remotely similar to what your employer is doing and whether you might become their competition down the line.

A chance of getting entangled in a lawsuit increases exponentially if you're going to take on your employer. Also, most emp contracts usually also state that you cannot contact your employer's customers or people you met through the company's business engagements for at least a year and usually more. Also, don't try to poach current employees or you'll get sued. Never use employer's computers or any other equipment that belongs to them for working on your startup (that includes take-home laptop that they might have given you). Don't even use their Word or Outlook for your own stuff. Delineate your startup as much as possible and don't share your ideas/prototypes with anyone at work since they might testify against you when called in front of a judge.

I know of one entrepreneur who got sued by a former employer and the argument they used was that even though he quit before he started the company, the IDEA was conceived while he was working for them and that he could not have come up with the idea had he not worked for them. Needless to say, the charge was ludicrous but it diverted his focus off startup and it also scared some investors and the startup was stillborn.

PS: IANAL but I know a lot about law and have been advised by many lawyers so take my advice with a grain of salt.

</comment></comment><comment><USER9> I think the legality of this depends on the exact details of the contract he signed. If they are a gaming shop and he releases a game, they may have some kind of a claim here. A few months ago I decided I wanted to complete a web research application I had started back in graduate school (and continued after that). This is a completely different area than my company's, but I still had to go through a long series of discussions maintaining this job (for paying the rent while finishing...) and working on this in my spare time. This lead me to my currently insane 70-80 hour weeks which will maintain for uhm... more months than I'd like.

As I understand it, if you do not use any of the company's trade secrets or intellectual property and you develop on your own time with zero company resources, you should be OK. By releasing this game under an alias and in what appears to be a related area, it seems like this guy actually may have been violating some kind of a contract.

I think the moral of the story is, document everything you do as outside of work hours and if you can pull it off, get your company to send you an email stating they know about your work and know it is not related.

<comment><USER10> Depends on the employment contract. Some companies specifically state that they own any idea you have while working for them. Be especially careful with working for large companies that seem to have a product in any market you can think of, and are planning to enter the others.

<comment><USER9> The applicable statute:

2870\. (a) Any provision in an employment agreement which provides that an employee shall assign, or offer to assign, any of his or her rights in an invention to his or her employer shall not apply to an invention that the employee developed entirely on his or her own time without using the employer's equipment, supplies, facilities, or trade secret information except for those inventions that either:

       1. Relate at the time of conception or reduction to practice of the invention to the employer's business, or actual or demonstrably anticipated research or development of the employer; or

       2. Result from any work performed by the employee for the employer.

(b) To the extent a provision in an employment agreement purports to require an employee to assign an invention otherwise excluded from being required to be assigned under subdivision (a), the provision is against the public policy of this state and is unenforceable.

</comment></comment></comment><comment><USER11> Let's also consider an employer's perspective here. OK, you are a Y Combinator start-up, and you have hired your 18th employee (there are at least 10 that have 20+ employees, I think). If that employee were doing something like this on the side, would that sit well with you? How would you draw the line?

Forget the ownership aspects of whatever the employee is creating on the side. Would you be happy as his/her manager to realize his/her mental efforts are going somewhere else?

Think hard before you answer.

<comment><USER12> I would draw the line whenever he stops doing his job right. No difference from someone who watches TV on his free time.

</comment></comment><comment><USER13> _Almost any tech company makes its employees sign agreements guaranteeing the company the rights to any intellectual property they produce while employed._

That sort of nonsense should be outlawed. It runs COMPLETELY against the grain of free enterprise, as a person who works and toils over a product has to suffer losses when their employer wants to take their property, claim it as business owned and grab the difference? BULLSHIT

<comment><USER14> I certainly don't know what happened in this situation, but there are also quite a few scenarios where the company may be justified.

Would it be fair to companies if any employee could create anything they wanted to, on company time, and screw the company over? Would it be fair to the company if the employee could use all of the trade secrets and intimate knowledge of the company's product to build a competitor on his/her own time, and screw the company over?

It's obviously not a completely black and white situation.

</comment></comment><comment><USER15> thats why I'm a proponent of doing your startup full time, it eliminates this type of BS.

<comment><USER16> You created the initial version of this game in a weekend, but I assume a lot of development has been done since then. How much venture capital would have been required to create the final version of this game? (firstname.lastname@example.com)</comment></comment>


Can Re-Training Programs create Jobs in US? - <AUTHOR> http://mdshannan1.blogspot.com/2011/09/why-us-need-re-training-programs-to.html

<comment><USER2> Sure it can, you have to hire someone to teach all those useless classes.</comment>


Snowden's temporary asylum status expires in Russia - <AUTHOR> http://www.bbc.com/news/world-europe-28586840

<comment><USER2> How does Snowden spend his days In limbo? Where does he live and who pays for his living expenses?

<comment><USER3> If only there were some website where you could type these questions in and find out...

He has a job doing tech-support which I assume pays for some of his expenses, he also has people who have donated money to his support.

<comment><USER4> Idk how I would respond if I called tech support and was connected to Edward Snowden.

<comment><USER5> I know exactly how I'd respond: I would thank him for doing a great service to humanity, and then I'd ask him whatever tech support question I had. And I'd be thinking to myself that I won the tech support lottery, because I'm pretty sure Ed Snowden knows more about tech than your average support person.

<comment><USER4> I mean sure, but just the very concept is kind of strange and somewhat entertaining (and also sad).

</comment></comment></comment></comment></comment><comment><USER6> I wonder why this submission is falling so fast from the front page yet still getting upvotes?

<comment><USER7> AFAIK some keywords, such as NSA and perhaps Snowden cause this.

<comment><USER8> My trust in HN has dropped significantly since this censorship became so comprehensive.

<comment><USER6> since I posted this comment it gained about 16 points and dropped from page 2 to page 4

</comment></comment></comment></comment><comment><USER9> There are many in the German government that would love to give him asylum, but they dare not risk it. Not a fear of the voting public, but the wrath of the US.

</comment><comment><USER10> Given current US/Russia relations I don't feel he has much concern about extending this.

</comment><comment><USER11> Is there any way we can help ?

<comment><USER12> I got a spare room.

</comment></comment><comment> at-fates-hands Considering there's still a treasure trove of documents he has yet to release, I'm sure the Russians will be keen to keep him on the payroll until all the documents he has are released.

<comment><USER13> Iirc he doesnt anymore, their in the hands of the newspapers.

<comment> at-fates-hands Considering no one can confirm how many documents he took, there's no way to say if he still has any documents in his possession.</comment></comment></comment>


Show HN: Scraping the NYT Most Emailed list with Kimono - <AUTHOR> https://jaan.io/scraping-most-popular-feeds

<comment><USER2> Curious why you would not directly use the RSS feed from NYTimes or the New Yorker? Where is the need to scrap?

<comment><AUTHOR> I haven't found an RSS feed for the most emailed articles. Do you have a link? That would be awesome!</comment></comment>


Rules for Remote Work - <AUTHOR> https://joecortopassi.com/articles/rules-for-remote-work/ <comment><USER2> Interesting rules, thanks for sharing. We've just made an online conference on remote work and team management with 12 industry experts, you can check out the recordings as well: [https://6nomads.com/remote- conf](https://6nomads.com/remote-conf)</comment>


Ask HN: Does anyone else hate April Fools' Day? - <AUTHOR> <comment><USER2> Yes, the pranks tend to be lame and boring, it's especially depressing when you think probably people spent months working on these things.

<comment><USER3> And the pranks can have adverse consequences, like people losing their jobs:

[https://news.ycombinator.com/item?id=11404115](https://news.ycombinator.com/item?id=11404115)</comment></comment>


Ask HN: Who are some of the most inspiring founders on the planet? - <AUTHOR> Who do you consider legends of the startup world and why? <comment><USER2> I know he is not the most liked person but I have to say that Peter Thiel for me is number 1. After him comes Elon Musk.

<comment><USER3> I like Thiel because he:

1\. As far as I know, mostly finances things without marketing hype.

2\. Takes immortality research seriously. (Yo, get Countess Bathory on the phone). This is more important to me than space. Though I feel like any true advances will be kept under wraps.

3\. Doesn't seek attention or hype and is reasonable about things. Frankly, the fact that he doesn't draw attention to the fact that he is gay is endearing - it's just irrelevant. Anderson Cooper type of deal.

I do question his bitter lawsuits, but I am too lazy to research the details.</comment></comment>


Jepsen Disputes MongoDB's Data Consistency Claims - <AUTHOR> https://www.infoq.com/news/2020/05/Jepsen-MongoDB-4-2-6/ <comment><USER3> In the circles I run in, MongoDB is regarded as a joke and the company behind it as basically duplicitous. For example, they still list Facebook as their first user of MongoDB on their website, for example, but there is no MongoDB use in Facebook hasn't been for years (it came in only via a startup acquisition).

I had the misfortune to use MongoDB at a previous job. The replication protocol wasn't atomic. You would find partial records that were never fixed in replicas. They claimed they fixed that in several releases, but never did. The right answer turned out to be to abandon MongoDB.

<comment><USER4> I was floored by this comment yesterday from one of their Developer Relations people:

> _Did any of you actually read the article? We are passing the Jepsen test > suite and it was back in 2017 already. So, no, MongoDB is not losing > anything if you know what you are doing._

[https://twitter.com/MBeugnet/status/1253622755049734150?s=20](https://twitter.com/MBeugnet/status/1253622755049734150?s=20)

Can you imagine saying the phrase "if you know what you are doing," in public, to your users, as a DevRel person? Unbelievable.

<comment><USER5> Firstly let me point out that this response is neither intended as a defence of MongoDB defaults which are atrocious, or of the company, who are arguably duplicitous.

However I can _quite easily_ see how a non-native English speaker could use the phrase "if you know what you are doing" to mean "if you are careful".

<comment><USER4> Perhaps, but the point is, if you're working in DevRel and you think your role is to defend the product from criticism, and you do it by placing the onus on the developer to figure out how to use your product safely, you've totally lost the plot.

<comment><USER6> Also, if you're a DevRel, maybe you should have strong verbal and written communication skills in the language of your users? Just an idea

</comment></comment></comment></comment></comment><comment> naked-ferret From the jepsen report:

"""

Curiously, MongoDB omitted any mention of these findings in their MongoDB and Jepsen page. Instead, that page discusses only passing results, makes no mention of read or write concern, buries the actual report in a footnote, and goes on to claim:

> MongoDB offers among the strongest data consistency, correctness, and safety > guarantees of any database available today.

We encourage MongoDB to report Jepsen findings in context: while MongoDB did appear to offer per-document linearizability and causal consistency with the strongest settings, it also failed to offer those properties in most configurations.

"""

This is a really professional to tell someone to stop their nonsense.

<comment><USER7> MySQL and PG are not truly consistent per default, they don't fsync every writes.

MongoDB explains that pretty well: [https://www.mongodb.com/faq](https://www.mongodb.com/faq) and [https://docs.mongodb.com/manual/core/causal-consistency- read...](https://docs.mongodb.com/manual/core/causal-consistency-read-write- concerns/)

<comment><USER8> > MySQL and PG are not truly consistent per default, they don't fsync every > writes.

Postgres most certainly does fsync by default.

It's tru, you can disable it, but there is a big warning about "may corrupt your database" in the config file.

<comment><USER7> No PG does not fsync every writes, more details here: [https://dba.stackexchange.com/questions/254069/how-often- doe...](https://dba.stackexchange.com/questions/254069/how-often-does- postgres-or-mysql-make-the-fsync-call)

My point is people complain about MongoDB are the one not using it most likely, MongoDB is very different from 10 years ago.

I like to remind people that PG did not have an official replication system 10years ago and as of today is still behind MySQL. No DB is perfect, it's about tradeof.

<comment><USER9> > It writes out and syncs the accumulated WAL records at each transaction > commit, unless the committed transaction touched only UNLOGGED or TEMP > tables, or synchronous_commit is turned off.

So wal is synced before commit returns, and if you power cycle immediately after, the wal is played back and your transaction is not lost? So it's fine?

It does not need to sync all writes, only the records needed to play back the transaction after restart. This is what all real databases do.

</comment></comment></comment></comment></comment><comment><USER10> You can tell a lot about a developer by their preferred database.

* Mongo: I like things easy, even if easy is dangerous. I probably write Javascript exclusively

* MySQL: I don't like to rock the boat, and MySQL is available everywhere

* PostgreSQL: I'm not afraid of the command line

* H2: My company can't afford a database admin, so I embedded the database in our application (I have actually done this)

* SQLite: I'm either using SQLite as my app's file format, writing a smartphone app, or about to realize the difference between load-in-test and load-in-production

* RabbitMQ: I don't know what a database is

* Redis: I got tired of optimizing SQL queries

* Oracle: I'm being paid to sell you Oracle

<comment><USER11> And you can tell a lot about a developer when they post comments like this.

Almost none of is remotely accurate e.g. RabbitMQ isn't even a database.

<comment><USER12> I can't believe the one item that was so obviously added as a joke went right over head.

It may be good idea to take a break from the computer and find something less stressful to do.

<comment><USER13> Perhaps that's because some other message brokers are now being touted as databases[0][1], I remember seeing a thread about it on HN couple of days ago.

[0] [https://www.confluent.io/blog/okay-store-data-apache- kafka/](https://www.confluent.io/blog/okay-store-data-apache-kafka/)

[1] [https://dzone.com/articles/is-apache-kafka-a-database- the-20...](https://dzone.com/articles/is-apache-kafka-a-database- the-2020-update)

<comment><USER14> Kafka is a very different beast from RabbitMQ.

</comment></comment></comment></comment></comment><comment><USER15> All: we've changed the submitted URL from [https://www.infoq.com/news/2020/05/Jepsen- MongoDB-4-2-6](https://www.infoq.com/news/2020/05/Jepsen-MongoDB-4-2-6) to the work it is reporting on. You might want to read both, since the infoq.com article does give a bit of background.

Edit: never mind, I think the other URL - [http://jepsen.io/analyses/mongodb-4.2.6](http://jepsen.io/analyses/mongodb-4.2.6) \- deserves a more technical thread, so will invite aphyr to repost it instead. It had a thread already ([https://news.ycombinator.com/item?id=23191439](https://news.ycombinator.com/item?id=23191439)) but despite getting a lot of upvotes, failed to make the front page ([http://hnrankings.info/23191439/](http://hnrankings.info/23191439/)). I have no idea why—there were no moderation or other penalties on it. Sometimes HN's software produces weird effects as the firehose of content tries to make it through the tiny aperture of the frontpage.

</comment><comment><USER16> Lying about your test results from Jepsen is like going onto a reality show with Chef Ramsey, being thrown off for incompetence, then putting his name on your restautant's ads "Chef Ramsey ate here!"

I'd pay to watch Kyle screaming at people in the MongoDB offices, not that he screams or anything. Just a spectacular mental image: "IT'S NOT ATOMIC! IT COULDN'T SERIALIZE A DOG'S DINNER!"

<comment><USER17> I would watch a tech version of Ramsey's show.. oh boy!

<comment><USER18> Yep, always thought shame there isn't one but too small of a niche I guess. Also, almost everyone telling online that they apply best practices at their company is maybe lying and wishful thinking; that would come out so no-one would apply for the show. So maybe more of a startup show where 'a Ramsey' comes in when a (bootstrapped or angel invested; VC funded is not saveable that way imho) company is in distress for tech reasons put in by the founders. Relevant pet peeve for this thread; let us (tiny, cash strapped startup company with founders who know just not enough about prod envs to do a lot of damage) do everything autoscale in the cloud and now we have a burnrate of $28k/mo on AWS bills with 5 users.

</comment></comment></comment><comment><USER19> MongoDB's big problem is that their present user base _does not want_ the problems fixed, particularly at default settings, because it would mean going slower. Their users are self-selected as not caring much about integrity and durability. There are lots of applications where those qualities are just not very important, but speed is. People with such applications do need help with data management, and have money to spend on it.

The stock market wants to see the product as a competitor with Oracle, so demands all the certifications that say so. MongoDB marketing wants to be able to collect money as if the product were competitive. Many of the customers have management that would be embarrassed to spend that kind of money on a database that is not. And, ultimately, many of the applications do have durability requirements for _some_ of the data.

So, MongoDB's engineers are pulled in one direction by actual (paying) users, and the opposite direction by the money people. It's not a good place to be. They have very competent engineers, but they have set themselves a problem that might not be solvable under their constraints, and that they might not be able to prove they have solved, if they did. Time spent on it does not address what most customers want to see progress on.

<comment><USER11> If they only cared about performance then they would've left the write concern defaults to not acknowledge writes either locally or within a replica set. Or just read from the nearest replica and don't worry about potential consistency issues.

Also this isn't 2011. MongoDB is not a competitor to Oracle and never really has been by people that knew that a DocumentDB was not usable as a SQL one. It's other SQL databases that are the real competitors e.g. Snowflake, Redshift are.

<comment><USER19> You know it, I know it, MDB knows it, and most of their customers know it, but that doesn't matter: the stock market doesn't. MDB wants to be valued like a durable-database company, and to be able to charge durable-database prices. They need a plausible durable-database story to get those, regardless of what actual current users want.

It is possible there are still potential users not buying until they get that story. MDB wants those users.

</comment></comment></comment><comment><USER20> MongoDB started life as a database designed for speed and ease of use over durability. That's not a good look for a database.

People have told me that they have since changed, but the evidence is overwhelmingly and repeatedly against them.

They seem to have been successful on marketing alone. Or people care more about speed and ease of use than durability, and my assumptions about what people want in a database are just wrong.

<comment><USER21> > MongoDB started life as a database designed for speed and ease of use over > durability. That's not a good look for a database.

I think it depends. One could say the same about Redis, but it's wildly successful and people love it.

The difference is now they are advertised. Redis makes no claims to be anything other than what it is - a fast in-memory database that has some persistence capability but isn't meant to be a long-term data store. MongoDB, on the other hand, made (and continues to make) claims about being comparable in atomicity and durability to traditional SQL databases (but magically much faster!) that haven't withstood scrutiny.

Keep in mind, too, that most data ain't worth much. It's one thing to entrust data of low value in MongoDB; another to store mission-critical data in it. I would look askew at leadership who didn't ask hard questions about storing data worth millions or billions of dollars in MongoDB without frequent snapshots -- and even then, the value mustn't be contingent on the 100% accuracy of said data.

<comment><USER22> When I'm thinking about data stores in large systems I like to break them down depending on how they are used on two main axes: is it fast/slow moving and durability from "we don't care" and "we must never lose data".

It's easier to reason about systems if there's fewer things that require durability guarantees, ideally you want to be able to draw data flows that look like a tree instead of a graph.

I find that Redis fits great because it's perfect for a whole bunch of different temporal shared state needs, everything from sessions to partial results. I've also deployed things like Ehcache, MongoDB, and Memcached to fit these needs and found other tools such as Kafka or RabbitMQ to be great "glue".

Having the root of your important data be something "boring" like Postgres or MySQL (or even Oracle!) is just good risk management to me. I wouldn't want to trust Redis or MongoDB for important data because it adds to the things I have to worry about. It's "keeping your eggs in one basket" while making sure that basket is really well looked after.

</comment></comment></comment><comment><USER23> The Jepsen analysis : [https://jepsen.io/analyses/mongodb-4.2.6](https://jepsen.io/analyses/mongodb-4.2.6)

</comment><comment><USER24> I wonder if I'm the only sysadmin in the world who doesn't hate MongoDB. Yes, I wouldn't use it for new projects, and yes, I wish RethinkDB had taken its place, but it's not as horrible as people seem to think. Default configuration... If it weren't for RDS' doing PG-bouncer-style connection management, 95% of production postgres instances would probably fail. It innodb_buffer_pool_size wasn't set properly, plenty of data-centers would light on fire. If no one setup a firewall or AOF for redis, it's data-loss and data-exposure waiting to happen. If no one adds auth to an HTTP route, it's open to the world, etc etc etc. If tech-stacks were legos, software engineers would earn a heck of a lot less.

I absolutely agree it's been used by people who just don't want to write SQL queries, or being used as a text-search-engine in place of something like more appropriate like ElasticSearch, but to mock successful projects who were based on it seems silly. It reminds me of interviewing candidates at a startup who primarily used PHP/MySQL. Most of them openly laughed and called it all horrible. I voted "no" on them, and sometimes injected a somewhat toxic "ah, you're right - we should close up shop. Someone call Facebook - tell them their tech stack is horrible - shut it all down!".

You can learn a lot about a developer by asking "What do you think about Mongo, JavaScript, or PHP", and if their response isn't a shrug, they're probably more concerned with what editor is correct than if the product they're building is useful. It's an exceptional filter to reject zealots and find pragmatists.

All that said, MariaDB with MyRocks is _awesome_, but certainly not with the default settings :)

<comment><USER25> RethinkDB is a better solution to every problem that MongoDB claims to solve. I wouldn't use it for everything. But once my need for a document store outgrows what's convenient and easy in Postgres with JSONB, I reach for Rethink. It's great. There's a Jepsen analysis of it a while back too that is quite positive.

It's a shame that Rethink did so many things right and failed as a company while Mongo continues to do almost everything wrong as a company and still gets business.

<comment><USER26> > It's a shame that Rethink did so many things right and failed as a company > while Mongo continues to do almost everything wrong as a company and still > gets business.

This seems to be more the rule than the exception, doesn't it?

It's even not that hard to come up with explanations for this, main one certainly being that popularity depends essentially upon simplicity.

And simplicity might not even be economically as inept as we would like it to be. Indeed, since only a small minority of all the systems that are designed reach production and stay there for long then it can make sense to use the quickest piece of junk available, at least until proven it will stick.

</comment></comment></comment><comment><USER27> There is much amusement to be obtained from reading Jepsen's report:

"MongoDB's default level of write concern was (and remains) acknowledgement by a single node, which means MongoDB may lose data by default.

...Similarly, MongoDB's default level of read concern allows aborted reads: readers can observe state that is not fully committed, and could be discarded in the future. As the read isolation consistency docs note, "Read uncommitted is the default isolation level".

We found that due to these weak defaults, MongoDB's causal sessions did not preserve causal consistency by default: users needed to specify both write and read concern majority (or higher) to actually get causal consistency. MongoDB closed the issue, saying it was working as designed"

[http://jepsen.io/analyses/mongodb-4.2.6](http://jepsen.io/analyses/mongodb-4.2.6)

</comment><comment><USER28> MongoDB is horrible, I get it.

What do I use in this situation:

1) I need to store 100,000,000+ json files in a database

2) query the data in these json files

3) json files come from thousands upon thousands of different sources, each with their own drastically different "schema"

4) constantly adding more json files from constantly new sources

5) no time to figure out the schema prior to adding into the database

6) don't care if a json file is lost once in awhile

7) only 1 table, no relational tables needed

8) easy replication and sharding across servers sought after

9) don't actually require json, so long as data can be easily mapped from json to database format and back

10) can self host, no cloud only lock-in

Recommendations?

<comment><USER29> Elasticsearch? [http://smnh.me/indexing-and-searching-arbitrary-json-data- us...](http://smnh.me/indexing-and-searching-arbitrary-json-data-using- elasticsearch/)

Depends on what your queries look like, I guess.

<comment><USER30> Just adding that I have used elasticsearch for a use case under the above constraints several times in the past and it worked well.

Ironically once because mongo was such a pain to work with I dumped the data from it into ES to get the better API, usability and Kibana.

</comment></comment></comment><comment><USER31> I think it's remarkable this report has been out for a week now and no one at MongoDB has commented on it. At least, not that I have seen.

<comment><USER32> Maybe they're too busy spending their MDB money.

[https://www.google.com/search?q=NASDAQ:+MDB](https://www.google.com/search?q=NASDAQ:+MDB)

<comment><USER11> I genuinely am confused by comments like this.

Are companies not supposed to invest money into their product, sales, people etc ?

And why does being listed on the NASDAQ imply being flush with money ?

<comment><USER32> > Are companies not supposed to invest money into their product, sales, people > etc ?

> And why does being listed on the NASDAQ imply being flush with money ?

It was intended to be a playful reference to MDB's stock price being on a tear right now, not simply being listed on NASDAQ.

Expand the timeline on the graph to "Max", it's at an all time high.

</comment></comment></comment></comment><comment><USER33> "We found that due to these weak defaults, MongoDB's causal sessions did not preserve causal consistency by default: users needed to specify both write and read concern majority (or higher) to actually get causal consistency. MongoDB closed the issue, saying it was working as designed, and updated their isolation documentation to note that even though MongoDB offers "causal consistency in client sessions", that guarantee does not hold unless users take care to use both read and write concern majority. A detailed table now shows the properties offered by weaker read and write concerns."

That sounds like a valid redress, or am I missing something ?

<comment><USER34> Kyle's point is that it's arguably valid but certainly unhelpful: the _default settings_ are liable to lead to data loss. Moreover, he draws attention specifically to transactions as something which you would expect to make things safer, but in fact there's a rather arcane part of the documentation that notes that you need to manually specify both read and write concerns on every transaction individually if you want transactions to behave consistently, regardless of the concerns specified at the database level.

Basically, there are a large number of pitfalls that it's very easy to fall into unless you have an encyclopaedic knowledge of the documentation, and you need to ignore some of the words that are used (like "transaction" or "ACID") because they carry connotations that either do not apply or only apply if you do extra work to make it so.

<comment><USER35> How is this any different than DynamoDB where you specify that you want either eventual consistency vs strong consistency? DDB also does eventual consistent reads by default.

Is the argument that Mongo's documentation isn't clear?

<comment><USER34> I trust Kyle when he tells me that the behaviour he observes is surprising. From the analysis ([https://jepsen.io/analyses/mongodb-4.2.6](https://jepsen.io/analyses/mongodb-4.2.6)):

"In order to obtain snapshot isolation, users must be careful not only to set the read concern to snapshot for each transaction, but also to set write concern for each transaction to majority. Astonishingly, this applies even to read-only transactions."

"This behavior might be surprising, but to MongoDB's credit, most of this behavior is clearly laid out in the transactions documentation… MongoDB offers database and collection-level safety settings precisely so users can assume all operations interacting with those databases or collections use those settings; ignoring read and write concern settings when users perform (presumably) safety-critical operations is surprising!"

<comment><USER35> There is difference between "Mongo's documentation sucks" and "Mongo is technically deficient". The former can be corrected by updating the documentation.

Yes, I agree as far as the end user is concerned, they are losing data either way.

<comment><USER36> I think the implication here is that "Mongo's documentation is deliberately bad in order to hide their technical deficiencies," i.e. they're hoping people will use the defaults, be impressed by the speed, and never realize until it's too late that they're not getting the consistency they were promised.

</comment></comment></comment></comment></comment></comment><comment><USER37> Oh, Jepsen and MongoDB again? Somebody get the popcorn!

<comment><USER38> Unfortunately, not an entertaining showdown - too one-sided.

<comment><USER39> Because MongoDB is web scale?

<comment><USER40> Some readers might not be familiar with that particular meme: [https://m.youtube.com/watch?v=b2F-DItXtZs](https://m.youtube.com/watch?v=b2F-DItXtZs)

IMHO it perfectly describes the hype-reality disconnect at the early days of MongoDB. Yeah it was that bad.

Mongo has improved since, the hype has toned down and the NoSQL space is more crowded these days.

<comment><USER41> i remember diaspora chanting about using mongodb.

then a year or two later they admitted that their data model mostly fitted the relational model, and that they spent a lot of time basically reimplementing relational integrity in application code, in ruby.

yeah, diaspora has never been fast. I'm not sure they can blame it on mongodb though.

<comment><USER42> I remember the Mongo hype when it came out and I really couldn't understand it. You are just throwing away a lot of useful features of a relational database because "schemaless" and "big data". The majority of people using it were on single server setups.

</comment></comment></comment></comment></comment></comment><comment><USER43> I still remember when MongoDB was the new kid on the block and it was lauded as the only thing you should be using here on HN.

I'm glad my gut instinct was correct and that it really wasn't worth the hype. It reminds me of Ruby on Rails.

<comment><USER44> I've never used RoR but I know people that still swear by it. It's outdated by today's "standards," but ActiveRecord was and is still a gem (heh) and a lot of RoR's foundational principles have been adopted by the existing major frameworks.

Regardless of technical acumen, I believe RoR doesn't deserve to be compared to Mongo for one reason: the RoR developers never tried to gaslight their users into thinking _they 're_ the reason everything broke; they never said only "if you know what you're doing" can you avoid these hidden pitfalls.

</comment></comment><comment><USER45> Every time I see a post about Mongo it makes me wonder what could have been if RethinkDB was managed differently.

</comment><comment><USER46> I worked at one company where the network traffic just on the MongoDB master was around 2gb/s. We had machines with terrabytes of memory, and Mongo worked fine - until we had some replica set nightmares. Mongo support is amazing, but when replication breaks it's very hard to diagnose (usually it was our fault, but it felt very fragile).

</comment><comment><USER47> I used mongodb for 1 year for a milti million user app. I abondened it. The reliability and stability is just not good. I wanted it to be good, but it turned out to be a different

</comment><comment><USER48> Ok, so defaults suck, marketing is misleading, documentation and error messages are not exactly obvious. Assuming you are already stuck in the soup, putting those issues aside and getting practical instead instead of throwing more fire on the discussion:

If you set w: majority and r: linearizable/snapshot, both on collection, client and on transactions. Plus assuming you accept snapshot over Isolation. How bad are those remaining cases in reality and how do these issues compare to other databases? The final "read your future writes" error looks quite scary and does not seem to be caused by configuration error, same with "duplicate effects".

<comment><USER49> "Informally, I would summarize the CAP theorem as: If the network is broken, your database won't work."

\- Dwight Merriman, former CEO, and "one of the original authors of MongoDB" [1]

A word to the wise suffices. Sometimes the word in question is implied by other words.

For those who get this oblique post, note that throwing the above _bon mot_ in an interview session for a "distributed systems engineer" and asking for an opinion is a excellent way to differentiate between Peter Principle and Principal Engineer.

[1]: [https://web.archive.org/web/20100903213540/http://blog.mongo...](https://web.archive.org/web/20100903213540/http://blog.mongodb.org/post/475279604/on- distributed-consistency-part-1)

</comment></comment><comment><USER50> Discussed previously:

[https://news.ycombinator.com/item?id=23191439](https://news.ycombinator.com/item?id=23191439)

<comment><USER15> Surprisingly, it seems not to have made the front page: [http://hnrankings.info/23191439/](http://hnrankings.info/23191439/). There's clearly community appetite to discuss this, so we won't treat the current submission as a dupe.

<comment><USER51> " Did HN's antispam measures get a lot more aggressive recently? The last handful of Jepsen reports have really struggled to make it to frontpage, despite significantly higher vote-to-age ratios than comparable posts. Once they're on FP, they reliably hit top 10, but Dgraph's (1/2) " [https://twitter.com/jepsen_io/status/1261640852666855426](https://twitter.com/jepsen_io/status/1261640852666855426)

<comment><USER15> Funnily enough I emailed aphyr earlier this afternoon to let him know that the current submission was at #1 on HN and that [https://news.ycombinator.com/item?id=23285249](https://news.ycombinator.com/item?id=23285249) had strangely failed to make the front page despite all the upvotes it got. There wasn't any moderation in either case. Nothing has changed recently. There's just a lot more randomness than people assume...2000 submissions a day competing for the same 30 slots creates a lot of weird high-pressure effects.

</comment></comment></comment></comment><comment><USER52> Our company migrated away from MongoDB, here's a talk about how we did it, in case you're thinking about what is involved and how to do it safely: [https://www.youtube.com/watch?v=Knd3m2qh0o8](https://www.youtube.com/watch?v=Knd3m2qh0o8)

</comment><comment><USER53> Ubiquity used MongoDB for their CloudKey Gen1 series. When there was an unexpected shutdown, there's a random chance it would lose its configuration [1]. If your SD backup didn't work, you'd lose configuration for all WiFi hotspots. If you did client installs like I did, this was a total nightmare. How did they solve it? Release new, more expensive hardware with a battery backup acting like a UPS. Never solved Gen1 issues. Imagine your phone corrupting after a hard reset. Thanks Ubiquity & MongoDB

[1] [https://community.ui.com/questions/MongoDB-corrupt-after- eve...](https://community.ui.com/questions/MongoDB-corrupt-after-every- powercycle-Cloud-Key/6be0a9ad-0049-4a1a-8f73-35cac8b531f9)

</comment><comment><USER54> If you want to be "that guy" on parties, ask people what MongoDB is trying so solve. If they bring up the typical "noSQL document store" stuff, aks them why you'd want to use MongoDB for that.

</comment><comment><USER55> MongoDB uninstalled our cloud hosted cluster once and the site was down and we needed to setup a large database from backups. Their response was very unhelpful. I would never touch MongoDB again.

</comment><comment><USER56> Regardless of tech, MDB is a weird stock that go up steadily every time.

<comment><USER57> I have never understood the stock price. I tried shorting at one point, that was a mistake.

</comment></comment><comment><USER58> It looks like relatively few people clicked through to read the analysis itself, so @dang's kindly offered to repost it. You can find the analysis here:

[https://jepsen.io/analyses/mongodb-4.2.6](https://jepsen.io/analyses/mongodb-4.2.6)

... and the corresponding HN thread here:

[https://news.ycombinator.com/item?id=23290844](https://news.ycombinator.com/item?id=23290844)

</comment><comment><USER59> If you're looking for MongoDB done right, it does exist and it's called RethinkDB. For some reason it didn't catch on and become popular — but it's nicer, and most importantly, it doesn't lose your data.

Data point: I have been running my production system (a fairly complex SaaS) on RethinkDB for the last 4 years.

<comment><USER60> RethinkDB is no longer supported, its major caveat.

<comment><USER59> Yes. Although the degree of "support" always depends on how much you pay for it :-) I doubt MongoDB is "supported" in the way most people understand that word.

From my point of view, RethinkDB is not regularly developed and improved. There is progress, but it's slow. Which is a pity, because it's a really good database, and one that tries really hard to be correct above all else.

The only other correct distributed database with strict serializable guarantees that I know of is FoundationDB, which nowhere near as easy to use as RethinkDB is (but it's somewhat easier with their document layer, which pretends to be MongoDB, just done right).

</comment></comment></comment><comment><USER61> Main argument for using document-oriented databases: [https://martinfowler.com/bliki/AggregateOrientedDatabase.htm...](https://martinfowler.com/bliki/AggregateOrientedDatabase.html)

</comment><comment><USER62> Anyone has a recommendation for a NoSQL database?

[https://news.ycombinator.com/item?id=23253870](https://news.ycombinator.com/item?id=23253870)

 _(not Mongo obviously)_

<comment><USER38> This question sounded familiar - turns out I replied to it in another thread: [https://news.ycombinator.com/item?id=23286054](https://news.ycombinator.com/item?id=23286054)

To repeat my (non)answer:

There is no way to recommend NoSQL database without knowing what you need it for because NoSQL databases are highly specialized systems. If you need general-purpose database use an SQL one.

It's kind of a weird question, now that I think about it. Why would anyone seek out a database based on what it doesn't have?

<comment><USER63> I'd actually say the reverse. SQL databases are highly specialised datastores: they make sense if you need one particular transaction model and one particular query language and are prepared to coerce your data into one particular model to do so.

If you're starting from just "I need to store some data" I'd look to e.g. Riak or Cassandra before looking to an SQL database.

<comment><USER64> SQL DBs are not specialized.... they're incredibly general...

You are never starting from "I need to store some data" you're always going to start from "I need to store and read some data" otherwise /dev/null would work if you are not going to read the data back.

the problem with cassandra and riak is precisely the read aspect of the problem which quickly degrades the performance of those systems.

I've used both cassandra and postgresql at scales most companies never reach. cassandra I'd only touch for immutable time series data and only if that information was large enough to not fix on a single server and i didn't care about consistency. everything else is a SQL rdbms.

<comment><USER63> For simple reads, the SQL model forces significantly worse performance: MySQL benchmarks found that 75% of the time for a pkey lookup was spent on parsing the SQL. For more complex querying, SQL databases _can_ be fast... and they can also be extremely slow, and you can't tell for any given query just by looking at it.

The much-vaunted consistency comes at a significant cost: index updates block writes, and more insidiously, it's very easy to be surprised by a deadlock or a stale transaction with a long-running query. I've seen an SQL database stop committing any new writes because someone ran a seemingly innocuous query 23 days ago. And a lot of the time - including every web use case I've seen - <USER2> can't actually make any real use of those consistency guarantees.

Writing either a transformation pipeline that serves the same function as a secondary index, or a deliberate map-reduce style aggregation, takes more up- front effort. But it means you understand what's actually going on a lot more clearly and are much less likely to hit that kind of unpleasant surprise.

</comment></comment></comment></comment></comment><comment><USER65> [repost - asking for help] I am disappointed with the direction that MongoDB took this past few years. Going ACID shows in benchmarks [1] and it's not advisable if you are using MongoDB for stats and queue. (No one uses MongoDB for financial transactions despite the changes.)

And the recent change to a restrictive license is worrisome as well. I have been thinking of forking 3.4 and make it back to "true" open source and awesome performance. (If any C++ devs want to help out, reach out to me! username @gmail.com)

[1] [https://link.medium.com/PXIeZfhhH6](https://link.medium.com/PXIeZfhhH6)

<comment><USER66> Why not use PostgreSQL instead? It supports a JSON document data type natively. It also has exceptional stewardship as an open source project.

Mongo should never be a first choice, but a last choice for edge cases.

<comment><USER67> Postgres has terrible indexing with json. It doesn't keep statistics so simple queries sometimes take much longer than expected due to query planner not knowing much about the data.

<comment><USER68> DB noob question: if you know that you should be indexing on a json attribute, can't you put it into a «proper column» and index there?

<comment><USER69> There are a number of ways to do this:

* Extract the attributes you're interested in into their own columns, index these. With the extraction happening outside the database, this is the most flexible option.

* Similar to above, use a trigger to automatically extract these attributes.

* Also similar to above, used a generated column[0] to automatically extract these attributes.

* Create an index on the expression[1] you use to extract the attributes.

My use a JSON in PostgreSQL tends towards the first option. This works well enough for cases where documents are ingested and queried, but not updated. The last three options are automatic - add/change the JSON document and the extracted/indexed values are automatically updated.

[0] [https://www.postgresql.org/docs/12/ddl-generated- columns.htm...](https://www.postgresql.org/docs/12/ddl-generated-columns.html)

[1] [https://www.postgresql.org/docs/12/indexes- expressional.html](https://www.postgresql.org/docs/12/indexes- expressional.html)

</comment></comment></comment></comment></comment><comment><USER70> <rant>

This corruption is brought on by the stock market.

Have a look also at Shopify. They go and tack on 2% fees when customers use Google Pay or Apple Pay to checkout with. They recently announced that FB would be pulling ecom sales within app, and yet Shopify plans to charge 2% on top of FB fees. That's what I could gather despite the pricing being rather opaque.

Is this a step forward or backwards? Charging 2% / transaction for modern Internet protocols running on cheap hardware across a public network?

</rant>

</comment><comment><USER71> Obligatory [https://www.youtube.com/watch?v=b2F-DItXtZs](https://www.youtube.com/watch?v=b2F-DItXtZs)

</comment><comment><USER72> Can anyone share any positive experiences with MongoDB? I wouldn't think MongoDB as perfect like any other piece of tech, but the unanimous hatred for it seems a little overblown. Not trying to discredit the bad experiences people have had with it. Just curious to know where people are using it successfully

</comment><comment><USER73> This has been a known issue for a while:

[https://hackingdistributed.com/2013/01/29/mongo- ft/](https://hackingdistributed.com/2013/01/29/mongo-ft/)

MongoDB: Broken By Design

<comment><USER11> Might want to read up as this involves a completely different set of issues.

And most of those listed in the blog were fixed many years before 2013.

<comment><USER73> Actually, I read both articles. In fact, the author of the first article was my very own distributed systems professor in school. The persisting issue in both articles is a non rigorous specification of when a write actually completed. Both articles point out that a fault tolerant database should be ACID compliant, which does not live up to MongoDB's claims.

</comment></comment></comment><comment><USER74> MongoDB is the /dev/null of databases

</comment><comment><USER75> How is Cassandra as an alternative to MongoDB?

<comment><USER76> I mean, they are completely different. MongoDB is more-or-less a traditional RDBMS with automated failover and trying to staple on more advanced features. Cassandra is a masterless DynamoDB-ish database with features like hinted handoffs. You really need to know how consistency and distributed systems work if you're looking to pick Cassandra. It's a great implementation, you just can't compare it to MySQL/Postgres/etc like you can with Mongo

<comment><USER25> wat.

I hope this is a joke.

</comment></comment></comment><comment><USER77> Dan Luu suggested on Twitter that MongoDB trolled Kyle into testing Jepsen again. I think they've made a mistake though. :-)

</comment><comment><USER78> It seems that the only tangible benefit remaining for DocumentDBs over SQL platforms (PostgreSQL, SQL Server, etc.) is scalability. Jr. devs thinking they can have a career in software dev without learning SQL is not a benefit.

</comment><comment><USER79> Typical HN posts of late hating on Javascript and MongoDB from database elitists -- the thing is there's a tool for a job and as engineers we need to figure out what tool best suits our use cases. It could very well be a NoSQL database such as Mongo or a relational one like Postgres or MySQL.

<comment><USER80> > the thing is there's a tool for a job

Really? Which job do you belive needs a _" maybe store some of this data, sometimes"_ kind of database?

<comment><USER81> I'm not defending mongodb in and sense and had stern talks with some of my junior developers who were too eager to try out this new hot mongo thingy on a new website, but there are plenty such jobs.

For example, climate data gathered from hundreds of thousands of devices every minute can very much survive some data to be lost. Or some astronomical observations data.

I wouldn't choose mongoDB for it, though.

<comment><USER64> your example is a perfect use case for postgresql via the timescaledb extension.

<comment><USER81> I actually agree. I love postgresql and we've been using it for all our projects since our company was founded (well, except mobile apps, obviously), and it never failed us.</comment></comment></comment></comment></comment>