FTC Weighs Seeking Injunction Against Facebook over How Its Apps Interact - <AUTHOR> https://www.wsj.com/articles/ftc-weighs-seeking-injunction-against-facebook-over-how-its-apps-interact-11576178055 <comment><USER2> Not sure I quite understand the argument against Facebook here, though I might just be missing something.

The market for messaging apps is pretty saturated with competitors at the moment; if I didn't like their WhatsApp and Facebook accounts merging, for example, I could just bounce to iMessage / Kik / Snap / Telegram.

Whether or not you agree / disagree with the security & user experience argument from FB (I personally can see both sides,) the low switching costs will always mean the user has quite a bit of power here.

<comment><USER3> "The users can just switch" doesn't actually work when the switching costs are high, and the switching costs for a social network are a lot higher than you are making it seem.

If all of my friends are using WhatsApp, either I have to use it too or I have to convince everyone to switch to my preferred app just for my sake. And in the case of iMessage, I have to ask them to switch their phone too.

If these systems could interoperate with each other it'd be different, but as- is, the switching costs are astronomical.

<comment><USER4> So what should the government do? Tell all your friends not to use WhatsApp?

How do you propose that you get interoperability and security?

iMessage already interoperates with the standard phone messaging protocol - sms.

<comment><USER5> The benefit of iMessage is the encryption. Which SMS doesnt compare to that level of encryption used by Apple. Would rather iMessage be a paid for Android app. I woulda bought it but I might just switch phones altogether. Sick of Googles spying and political regime insanity (see Project Veritas).

</comment></comment></comment></comment><comment><USER6> Over the last 2 months, I've noticed Facebook has almost completely stopped responding to any vulnerability finds or bug bounty tickets submitted. It's gotten really bad.

For instance, right now all 70,000,000 of the Facebook users in Vietnam have their information posted online on an open web server. I am not going to post it just yet in the last hope that someone from FB will reach out to me, but the info is their along with their IMEI number, cell phone number, information on what ads they've seen and clicked on, etc. For some users there are absolutely private messages, it appears to be only for Muslims in Vietnam though (so not all nor am I saying all).

Until yesterday, the S3 bucket [https://whatsapp- messages.s3.amazonaws.com](https://whatsapp-messages.s3.amazonaws.com) was open. You can search CNAME records if you have a SecurityTrails subscription, and this bucket belongs to a certain FB contractor. In fairness it could have been something known as "Domain Shadowing" whereby you secretly hack a groups subdomain records for evading firewall purposes, but in that case the argument still stands.

It had over 13,000 pictures from various Latin American police departments that change every few days while I tracked it.

NOTE: FB Security ppl, my email is in my profile.

EDIT: This is a very good paper from Tsinghua University in 2016 on Domain Shadowing, which more people should be aware of. Check your subdomains, and make sure you use the free option in SecurityTrails to do it. Passive DNS checks aren't enough here: [https://faculty.sites.uci.edu/zhouli/files/2018/09/ccs17.pdf](https://faculty.sites.uci.edu/zhouli/files/2018/09/ccs17.pdf)

<comment><USER7> I'm sorry, rshnotsecure but until you come up with those 5k words you promised about ProtonMail, CreditKarma and a huge list of others being fronts for spy agencies[1] your posts will only ever be titillating scuttlebutt for me.

Last we heard you had the lounge room covered in butcher's paper, plotting it all out and _gave your literal word_ that you'd post that "report" within 96 hours. That was 42 days ago.

I have no idea if what you say about FB's server in Vietnam is true and I'm not here to argue with you. The pity is that you post about interesting things.

But if making big claims with unshakeable evidence "to come" which never actually arrives is something that we deplore in our politicians and mainstream media, why let it creep in here?

[1] [https://news.ycombinator.com/item?id=21412052](https://news.ycombinator.com/item?id=21412052)

<comment><USER6> Here's the Vietnam server: [http://126.96.36.199:9200/_cat/indices](http://220.127.116.11:9200/_cat/indices)

This is the indice you are looking for. 71,000,000 records.

yellow open fb_vietnam ChkUhOlHQpO_RozynrZdXA 1 1 71839979 11829285 10.1gb 10.1gb

You are right on the 96 hours vs 42 days thing though. It's a lot of work man. No one has volunteered to help. There are a ton of ASNs what can I say lol. I will post about CreditKarma today. Not ProtonMail.

Type in "openvpn.creditkarma.com" to Spyse.com please. What do you see? It says it points to a server in Tunisia right? And that server has 10,000 other host names on it almost all from...hnagroup.com? China's really really big private conglomerate correct?

Anyway again you are right, but I'm tired, I posted the Vietnam server (at least enough data to confirm amounts although we can debate purpose of cluster which is fascinating).

EDIT: Damn it I just realized your account was created new a few hours ago. Can't you use your real name like I do? Maybe email me?

NOTE: To your credit I finally posted. Like 1250 words but we'll get there. My wife wants me to move the butcher paper after all :) [https://blog.12security.com/](https://blog.12security.com/)

<comment><USER7> > This is the indice you are looking for.

I'm not looking for any indices. This is another classic fake news ploy - piling one outrageous claim on top of another without ever actually substantiating anything.

> Can't you use your real name like I do? Maybe email me?

Why? I don't want to chat. I want you to back up your claims the way you made them - publicly.

> I finally posted. Like 1250 words.

832 to be precise, and a whole bunch of those making even bigger assertions with even less proof offered. And a whole lot of stuff "tbd"

> we'll get there

I hope so, because we're not even close yet.

<comment><USER6> Lmao this is not a real person.

You should triple down and write another angry response. Other than this one which is a little snarky (I apologize but it had to be) I promise I will respond to that one as nicely and patiently as I have the others.

Big Hug []

</comment></comment></comment></comment></comment><comment><USER8> In my opinion, the ship has already sailed for the FTC to have any say in how Facebook interoperates its own applications. If they were truly concerned about a monopoly on social media, _the sale of those apps should have been blocked in the first place_.

What is the point of telling Facebook not to make their apps interact with each other? Are they thinking Facebook employees _can 't_ see the databases of their other apps? Is the FTC going to pretend they know anything about how the databases are _supposed_ to operate? What about one big database with an App field; would they know if Facebook did that? What about multiple databases on the same server; is that an antitrust violation?

The entire rationale for looking at Facebook _now_ just seems like going after an easy target. Either evaluate the antitrust arguments on their own merit and break those apps off Facebook, or leave it alone.

</comment><comment><USER9> The timing of some of this feels disingenuous... it wasn't that long ago Facebook completed their graph API migration... functionally their is not a "platform separation" any more their are just different labels on Facebook APIs this change is now heavily reflected in dev tools... Facebook will simply argue that any disentanglement would have high impact to business and move on with their day. The people who work for the government and do this stuff know enough about how platforms work to know that the time to do this was over a year ago before this huge shift was locked in.

</comment><comment><USER10> [http://archive.is/FCE1L](http://archive.is/FCE1L)

</comment><comment><USER11> If it got to that and it's not some sort of WSJ attempt to advocate for this sort of thing, facebook must fight it tooth and nail, and stop these attempts at governmental micromanagement once and for all.

It's suspect that they are going after tech companies which are national treasures but are enemies to corporate media, it's strange that this sort of action isn't being considered against actual malicious monopolists that the press isn't constantly attacking. No one is suggesting going after disney for instance even though they have that market cornered.

<comment><USER4> What market exactly does Disney have "cornered"?

But it should give anyone pause when the government takes upon itself the power to decide how a private company architects its solutions.

<comment><USER5> Entertainment. I guarantee anybody in your average household is a fan of something Disney outright bought the rights to. Whether its Marvel (if I hear about another reboot of X-Men or another Avengers movie imma pull my hair, OTOH I did like Guardians... Yikes even I am falling for it!) then theres Star Wars which has a huge following of people from all ages. The typical Disney public domain rip offs as well. The insane amount of music they own the rights to.

Here in Florida its scary how much power Disney has. They also own the media to a point and you find it hard to see anybody speak badly of them within the media. The amount of things I see covered up that never winds up in the media genuinely scares me. How is it possible the media keeps specific things quiet for Disney.

<comment><USER4> There are still five major movie studios in the US and the minor studios like Blumhouse and Tyler Perry Studios routinely put out movies that have a higher ROI than the major ones.

As far as TV, there are plenty of TV studios and streaming services.

<comment><USER5> > and streaming services.

And now they're removing their content from those. They also own several of those streaming services (ESPN, D+, and significant enough parts of Hulu). By contrast their streaming competitors own one single serve (with maybe the exception of Google and Apple who own music streaming services and video streaming).

If nobody sets up a roadblock I fear where it ends, if it does.

I forgot to add a disclaimer: long before I was a developer I worked for Disney. I'm somewhat biased, but I also attempt to not be blind. It's a monopoly on your children to a significant scale. If they stop buying out companies, it still feels like they've bought out some of the more major fandom franchises.

I just went to check, and after their last major buyout they own all of the Narnia movie rights.

<comment><USER4> The entire idea of the internet was suppose to be "disintermediation". Why are we now wanting middle men?</comment></comment></comment></comment></comment></comment>


Think You've Got Your Credit Freezes Covered? Think Again - <AUTHOR> https://krebsonsecurity.com/2018/05/think-youve-got-your-credit-freezes-covered-think-again/ <comment><USER2> Since I was having a hard time finding the link in the article. Here's the page to add a NCTUE security freeze:

[https://www.nctue.com/consumers](https://www.nctue.com/consumers)

Here are the pages for adding security freezes to the other big four agencies:

[https://www.freeze.equifax.com/Freeze/jsp/SFF_PersonalIDInfo...](https://www.freeze.equifax.com/Freeze/jsp/SFF_PersonalIDInfo.jsp)

[https://freeze.transunion.com/sf/securityFreeze/landingPage....](https://freeze.transunion.com/sf/securityFreeze/landingPage.jsp)

[https://www.experian.com/freeze/center.html](https://www.experian.com/freeze/center.html)

[https://www.innovis.com/securityFreeze/index](https://www.innovis.com/securityFreeze/index)

Most people don't know about Innovis, which was mentioned in the article. That said, I've never run into a company where I've needed to lift an Innovis freeze. Anyway, I agree that it's tiring to constantly track which data brokers allow (are required to) adding a security freeze. In my opinion, better legislation is needed to curb this activity and data brokers in general.

<comment><USER3> I tried to fill out the NCTUE freeze page several times, it keeps claiming there is a reCAPTCHA error despite me clicking it out correctly and getting the green check.

Seems to be intentionally broken to avoid people freezing.

<comment><USER4> I've seen the reCAPTCHA issue a few times this month on other sites. My guess this is a result of Google shutting down the reCAPTCHA v1 API.

</comment></comment></comment><comment><USER5> It's all so tiring. It's seriously going to be decades until we get politicians in power that understand the Internet and start holding companies accountable that don't give a shit about security (namely, all of them).

<comment><USER6> Will it ever happen? When the current generation of children reaches office- holding age, a majority of them may understand internet _culture_ , but I don't think a significant fraction will ever understand the internet in a technical sense.

<comment><USER7> What is your reasoning behind that? This is seriously one of the dumbest "kids these days" statements I've ever read. Kids will become technologically sophisticated, many kids already are. Many kids will attain higher technical proficiency than _you_ ever had, for example. Why would it be otherwise?

<comment><USER8> There are 100 reasons - the innards of technology boxes are less accessible. Programming languages are more abstract, and few dig down to the lower layers (which used to be all there was). Its changing faster and hard to get a mindshare before it all changes again.

I don't think its a 'kids these days' statement at all. More a 'technology these days'

<comment><USER9> Right. Take 2 10-year-olds. Give one a C=64 and a paper manual, and one an iPad and an internet connection. Which one is more likely to learn to program?

</comment></comment></comment></comment></comment><comment><USER10> Bottom line: Whenever possible, wherever possible, pay cash.

I used to laugh at my father-in-law whom I once considered to be a conspiracy theory whackjob with his off-the-grid lifestyle. As more and more of his "crazy" insights turn into New York Times headlines, I'm starting to think he was ahead of the curve.

<comment><USER11> Am I missing something here? Paying cash doesn't prevent the opening of fraudulent lines of credit in your name.

Equifax et al don't particularly care if you pay cash, they have some form of record on you either way.

<comment><USER12> I guess the idea is that if one were "off the grid" enough not to have a phone, one's details wouldn't be in their DB in the first place. That would make one's identity harder to steal, which would cause the thieves to move on to the next victim.

<comment><USER13> The SSN of someone "off the grid" would actually be perfect for doing what is called synthetic identity fraud. If a given SSN has no credit history, you can make up a fictitious person and apply for a credit card with it. The application is probably declined, but a record of this SSN associated to a completely fictitious identity now exists in the credit reporting agencies databases, which makes it more likely that a future credit application will be approved.

If there is no other identity using this SSN, then the likelihood of the fraudulent activity being dectected is low.

This is why SSNs belonging to elderly people, children/teenagers, and the recently deceased are of relatively higher value for a lot of people out there doing identity fraud.

<comment><USER14> Stealing someone else's unused SSN is not synthetic identity creation, it's just fraud. There is a living person associated with that SSN, even if they are off the grid.

Synthetic identity fraud involves you making a bunch of inquiries with completely fake SSNs and squatting on them for a few years, maybe bringing them out of cold storage every so often to make a few more inquiries to keep them on the credit-building radar before you ultimately qualify for actual lines of credit and do a bust-out. There is no oblivious meatbag being used as a patsy-- the identities are completely fake, which is what makes them synthetic.

<comment><USER13> There is more than one way to construct a synthetic identity, but in most cases that I've been aware of, a stolen SSN is used to construct the totally fake identity.

Using a completely fake SSN instead of a stolen SSN is what distinguishes "synthetic identity theft" from "synthetic identity fraud".

The identities in both cases are completely fake, but only in the identity theft case is the synthetic identity backstopped by a legitimate SSN that belongs to a real person somewhere. If that real person is a child or "off the grid", then the CRAs have no idea who the real person is and their SSN is ripe for creating a synthetic identity that will probably go undetected for years.

</comment></comment></comment></comment></comment></comment><comment><USER15> I am tired of people calling me a "victim" of identity theft. I'm not the victim. The companies and government agencies that allowed themselves to be defrauded by someone posing as me are the victims. If you didn't do your due diligence in properly verifying the identity of the person you handed out cash, goods or services too it's on you. That being said, the entire system is broken. I've had 5 cellphones taken out in my name, I've had multiple credit cards opened using my social security number. I've been interviewed by a special agent with the State Department because someone tried to get a passport using my data. I'm sick of dealing with other peoples fraud problems. Fix it already!

<comment><USER16> > I am tired of people calling me a "victim" of identity theft. I'm not the > victim.

Usually, the person whose identity is stolen is the victin.

> The companies and government agencies that allowed themselves to be > defrauded by someone posing as me are the victims.

They are also victims, but until you—often at considerable personal effort—have gotten all the debts and other bad marks by the fraudster dissociated from your identity, you are disadvantaged by it, and if you are disadvantaged or have had to pay a cost to escape that disadvantage, you are a victim, too.

<comment><USER17> > They are also victims, but until you—often at considerable personal > effort—have gotten all the debts and other bad marks by the fraudster > dissociated from your identity, you are disadvantaged by it, and if you are > disadvantaged or have had to pay a cost to escape that disadvantage, you are > a victim, too.

You seem to be misunderstanding the point badly--the situation you have described is nonsense.

Firstly, what do we mean by "having your identity stolen"? I submit that this is not an attack on me as a person, it is an attack on a fictional representation of me held by a conglomerate of corporations.

This is the point of OP: I am not involved in these transactions! I haven't committed a crime, purchased anything, or interacted with either the party who has defrauded this conglomerate nor this conglomerate. In fact, due to the way this information is collected and stored, I may not have _ever_ had any direct relationship to either party.

So what's happened here is that a conglomerate of companies have been defrauded and now insist that I am to make amends for it. It's nonsensical, and no amount of repeating the facts of the current situation will make it anything but ridiculous.

</comment></comment></comment><comment><USER18> I'd pay money for a service that monitors who all the services are in the credit reporting space and freezes my credit at them for me.

<comment><USER19> I would rather anyone opening a line of credit be required to obtain proof of acceptance by the person named on the account and until doing so they are 100% liable for all debts incurred. How this is defined will require regulation. The burden is increased at any time a request is made outside of the state and city of residence

<comment><USER20> This is already the law. You are not responsible to pay debts incurred by another person. The problem is when the bank or company that gave credit to someone else that used your identity information is allowed to lie to a credit bureau about the fact that it was you who defaulted on the debt. Otherwise you could just tell the bank, "Pound sand stranger, I never borrowed any money from you. Stop bugging me or I'll get a restraining order.", and never have worry about how someone has pretended to be you to a bank.

Large fines for banks and companies that libel people by telling information brokers lies is the way to stop this problem. The idea that when this happens it is "identity theft" and the person whose identity was stolen is responsible for the problem is ridiculous. This problem should be called libel and the perpetrators (the entity reporting false info that hurts you) punished.

<comment><USER12> These proposed "large fines" would still require regular people to deal with putative creditors and occasionally to defend themselves in court. Structural changes are required to make the system as it actually functions more equitable to humans. What are structural changes? For instance, prior to enforcing a debt the law could require the production of an authenticated video recording of the debtor clearly stating, "I am John Doe and I agree that I owe EvilCo $1,000 on May 9, 2018." This requirement would make it more difficult to lend money, so it would be opposed by some very deep pockets. I suspect that any actual solution to the inequity in this area would make it more difficult to lend money, so progress is likely to be slow.

<comment><USER20> I agree. First, it could be made a criminal offense for a company or bank to lie to credit agencies and then public prosecutors could pursue these companies for big PR wins.

Second, making giving and getting credit harder to do is probably a good thing for society. Do people really need to take out a half million dollar home loan from their phone a la Rocket Mortgage? Many cultures had/have very strict rules about charging interest on loans. Debt jubilees were common in past civilizations and seem necessary to prevent the masses from creating their own via revolution. Making personal loans illegal and bringing back the loan shark is not the way to go, but making going into debt to buy consumptive goods much harder would be big positive for most people in the US.

Like you said, there is a huge amount of money and power behind easy lending, so change seems unlikely. On the other hand, change seems to be in the air these days, so maybe one can afford a bit of hope.

</comment></comment></comment></comment></comment><comment><USER21> I absolutely hate our entire credit system, the lazy way it handles our sensitive info, and the eagerness to extend credit to anyone without scrutiny of their identity.

Big incoherent rant incoming:

In 2016 someone stole my identity and in about a 1 month period took out tens of credit cards and new bank accounts and phone account which they immediately over drafted. They somehow managed to change the address in my credit file, and so I wasn't getting any failure to pay notices. At the same time it was quite soon I found out because at least one creditor managed to find my real address asking about a 18,000 loan approval for furniture from The Brick. The police were easy enough to deal with and took a report and I had to use this to prove to creditors it was fraud.

I had Equifax do their own investigation, and they agreed it was fraud, but they only removed the creditors who happened to appear on file during their investigation.

A year past, and I check my credit and it was 799, all clear. Then last fall I get a call from a creditor saying I owe them money, they required I go to their bank to fill out the paperwork. One more cleared. Then in February I'm hearing from yet another bank about an overdraft account. Once again I proved to them it was fraud by providing the police report number. "Ok, well it will take us 30-45 days to remove this from your credit file". Its coming up to 45 days and its still there. I called them to clarify about when it will be removed, and get the same bs. I do not believe their word so I have also sent another dispute to equifax. MY MORTGAGE RENEWAL IS DUE JUNE 1 AND THIS SHIT HAS STOPPED ME FROM MOVING MORTGAGE PROVIDERS TO GET A BETTER RATE.

Now during this latest fiasco, I do another credit check and discover ANOTHER phone company on there says I have a bad debt with them and a note that a collections agency is after me. I succeed in having this removed from my credit file after dealing with Equifax, but for reasons beyond me they did not remove the one from the bank I am disputing in the previous paragraph, so I have to hope it will clear when they way it will (30-45 days from early April), or hope my second dispute works. Note that these latest bad debts are still originating from 2016, its just taken the banks that long to pin the blame on me and attack my credit file.

So this has been a lot of stress on me, I have failed to secure a new mortgage lender which will cost me in interest over the next 3 years on mortgage payments. In addition, I just discovered my TransUnion file is still a giant mess with shit from last fall on it. Apparently Trans Union and Equifax don't synchronize.

Long story short, there isn't shit you can do to speed up anyone helping you. The police are competent and will have your back but they can't stop creditors from attacking your file. Equifax is slow and not thorough in their investigations and make themselves hard to reach. The banks can pin a bad debt on anyone they chose, never mind that the onus should be on them to prove I took out the loans they claim. Like, show me a signature, show me the ID used to get the loan... Oh its fake? Then fuck off. Nope, they will make you jump hoops to prove it isn't you, and then the cherry on top is they will be very very slow to do anything to remove it off your credit file once you have in fact proven its not you.

Good luck to all of you, you are all vulnerable to this shit.

<comment><USER22> The answer is registered mail, return receipt requested, and a complaint with the CFPB, although how on-the-ball that agency will be since the new administrator was appointed is YMMV.

<comment><USER23> I'd be more inclined to write a pro forma letter, then sue for statutory damages for technical violations under the FCRA and/or FDCPA the instant the established deadlines pass. It puts the burden of proving the validity of the debts on them, and if they ignore you or drag their heels, you win by default.

</comment></comment></comment><comment><USER24> Anyone else getting an unsecured link warning from the freeze link on the NCTUE website?

[https://www.exchangeservicecenter.com/Freeze/jsp/SFF_Persona...](https://www.exchangeservicecenter.com/Freeze/jsp/SFF_PersonalIDInfo.jsp)

</comment><comment><USER25> Brian Krebs - can you please change to a responsive theme?

Thanks.</comment>


Hot pepper Xcode tips - <AUTHOR> http://www.raizlabs.com/dev/2015/03/spicing-up-xcode/

<comment><USER2> What is the point of the pepper. This is a serious question. Is it to go quickly through the presentation? to just keep the presenter from being boring? or something else?

<comment><USER3> I loved the idea of Hot Pepper Gaming (the YouTube channel that inspired the talk), and I think it's funny, so I wanted to try it myself. I had a ton of fun doing it.

</comment></comment><comment><AUTHOR> One of our developers here at Raizlabs followed hot pepper gaming's example while providing tips for using Xcode

<comment><USER4> Fantastic. Evil. I love how this idea forces the speaker to be brief with the milk within reach.</comment></comment>


YouTube Unveils Plan to Vet Content, Channel Partners - <AUTHOR> https://www.hollywoodreporter.com/news/youtube-unveils-plan-vet-content-channel-partners-1075155 <comment><USER2> Youtube really needs a 2-tier advertising system.

Pay the discounted rate if you're okay with your ads being posted against unpopular and unvetted content.

Pay the full price if you only want ads against vetted and popular content.

You democratized video, and now you're hammering against the little publishers.</comment>


Sidecar Puts Passengers Aside, Pivots to a Mostly-Deliveries Company - <AUTHOR> http://www.forbes.com/sites/ellenhuet/2015/08/05/sidecar-pivots-to-mostly-deliveries-company/ <comment><USER2> Sidecar is an excellent illustration of the first mover disadvantage. When Sidecar started, they looked carefully at the existing laws and crafted their product to stay, to their best judgement, on the exact right side of the law. That's why, in early Sidecar, for example, drivers negotiated directly with passengers for the fare and why sidecar cars carried no visible exterior markings.

Lyft and Uber decided to play the much higher risk/higher reward game of antagonizing regulators in exchange for better product. In the early days of the rideshare war, the Lyft pink mustaches were everywhere and everyone was made aware that Lyft existed due to them.

It's an excellent example of how "playing clean" is not always in an entrepreneur's best interest and how luck and a willingness to take risk can alter fortunes on a profound scale.

<comment><USER3> According to Wikipedia, Uber was founded almost 3 years before Sidecar (March 2009 versus January 2012)... so Sidecar may have suffered a "timid-mover disadvantage", but not really a "first-mover disadvantage".

<comment><USER4> Uber Black was 2010 - 2012. Than after Lyft properly launched their service Uber followed with the now more popular UberX.

<comment><USER5> UberX launched in early 2012, before Lyft. But Lyft, Sidecar and Uber were all likely developing their 'low cost uber' versions at around the same time. After how much success Uber had already had with just Black cars, the idea for a lower cost alternative was obvious enough that lots of companies jumped in.

<comment><USER4> Sure? IIRC Uber was hesitant to launch a 'ride sharing' product until Lyft did in SF. Any links to guide me?

</comment></comment></comment></comment></comment><comment><USER6> _"Customer acquisition and driver acquisition in this category are very expensive, and it does make a big difference when you offer promotions — $500 for a driver, or $20 credit — versus our more typical $5 offerings of credit."_

So that's where Uber's capital is going. They're buying market share with actual money.

</comment><comment><USER7> I recently put together a collection showing the evolution over time of the top ridesharing/taxi hailing startups (using content from the Internet Archive) like Lyft, Shuddle, Uber, Sidecar, and BlaBlaCar [1].

One thing that surprised me the most when putting it together - Zipcar was founded way back in June 2000, well before Uber/Sidecar/Lyft, and is still around alive and well today [2]. I often feel like it does not get enough credit for helping to improve the transportation space, especially compared to attention given to Uber/this new class of transportation-focused startups.

[1] You can check out the collection here: [http://www.startuptimelines.org/collections/uber_lyft_taxi_s...](http://www.startuptimelines.org/collections/uber_lyft_taxi_startup_app_competitors/)

[2] [http://www.startuptimelines.org/startup- timelines/zipcar/](http://www.startuptimelines.org/startup-timelines/zipcar/)

<comment><USER8> Maybe I missed this (it's hard to peruse the milestones/headlines from the timeline layout), but there's no mention on the timeline of Zipcar being bought out by Avis in 2013. While Zipcar is still alive in name today, I don't think everyone would call it an unmitigated success:

[http://www.theverge.com/2014/4/1/5553910/driven-how- zipcars-...](http://www.theverge.com/2014/4/1/5553910/driven-how-zipcars- founders-built-and-lost-a-car-sharing-empire)

[http://blogs.reuters.com/felix-salmon/2013/01/02/aviss- smart...](http://blogs.reuters.com/felix-salmon/2013/01/02/aviss-smart-zipcar- buy/)

> _As with many high-flying IPOs, however, Zipcar never fulfilled its promise, > and its stock never again saw those heady first-day levels. By the end of > 2012, its market capitalization had fallen to $330 million, while Avis > Budget's market cap was $2.1 billion — making an acquisition both easy and > obvious. In the past eight months alone, Zipcar stock fell by 40% while Avis > stock rose by 60%_

</comment></comment><comment><USER9> I tried to use sidecar once but ran into ui bugs that didn't let me complete a purchase. Not sure if my isolated experience is representative, but I read "customer acquisition blah blah blah" \- making the app work better could have helped.

</comment><comment><USER10> Typical example of Next Feature Fallacy (including their pivot): [http://andrewchen.co/the-next-feature-fallacy-the-fallacy- th...](http://andrewchen.co/the-next-feature-fallacy-the-fallacy-that-the- next-new-feature-will-suddenly-make-people-use-your-product/)

</comment><comment><USER11> Separately, continuing after funding to use a domain hack side.cr is a non- starter. Worse yet SEO miss if you google "sidecar" they don't even come up. [1]

[1] Except on the right panel as "company" but not in the actual results unless you click that link.

</comment><comment><USER12> I'm curious what patents Sidecar holds, if they truly were the innovators, and if they can use that as some sort of advantage against Uber and Lyft.

</comment><comment><USER13> I suspect Uber will eventually move into deliveries too. It's a natural extension.

<comment><USER14> [http://newsroom.uber.com/bringing-the-neighborhood-shop- to-y...](http://newsroom.uber.com/bringing-the-neighborhood-shop-to-your- doorstep/)</comment></comment>


Ask HN: Anyone Using CockroachDB in Production? - <AUTHOR> What do you like about it's killer features?
How interchangeable with Postgres was it?
How is performance? <comment><USER2> Tried it before. Compatibility depends on how complicated your SQL is. There's a detailed list of supported syntax which is getting better every release but it's best for simpler OLTP scenarios. [1] Horizontal scaling and reliability are great. Security setup and certificate management is horrible, and if you don't use certificates then you also can't set passwords for any user so its all or nothing for some reason. [2]

Performance for OLTP queries with high-selectivity using primary keys/indexes is pretty good. Will not be as fast as PostgreSQL because there's consensus required to get the latest data, although it now supports timestamped queries so you can get stale data from replicas for faster reads. Complex queries can slow down or stall the server. Deletes are handled by tombstones so heavy update/delete workloads will cause increasing amounts of slowness in queries until compaction happens so it needs tuning for your workload. [3]

Treats all the servers as one giant global cluster instead of multiple sub- clusters for each regional location so there will be very high latency for writes if you're running in multiple regions. I recommend sticking to a single region with multiple zones instead, or very close regions like US east + central.

Overall its a good solid database choice for core operational data that you want safe. Skip it if you need fast low-latency key/value (use redis or scylla) or large analytical queries (use numerous data warehouse options).

1\. [https://www.cockroachlabs.com/docs/stable/detailed-sql- suppo...](https://www.cockroachlabs.com/docs/stable/detailed-sql-support.html)

2\. [https://github.com/cockroachdb/cockroach/issues/32448](https://github.com/cockroachdb/cockroach/issues/32448)

3\. [https://github.com/cockroachdb/cockroach/issues/32522](https://github.com/cockroachdb/cockroach/issues/32522)

</comment><comment><USER3> > Anyone Using Cockroach DB in Production?

Yes, used it in 2017 for a fairly small dataset (order of GBs) and 5-7 nodes running in Kubernetes.

> What do you like about its killer features?

Easy admin and good at scaling horizontally.

> How interchangeable with Postgres was it?

Wire compatibility helps but there were many minor differences at the syntax level. I tried for the longest time possible to maintain compatibility with Postgres syntax and it was painful.

> How is performance?

Back then it was garbage, probably 10-100x slower than Postgres. This was pre-2.0, mind you, I heard that things improved noticeably since then. And most of the problems can be circumvented by not doing stupid things and optimizing your queries/data model. You might want to throw a cache in front of it anyway.

<comment><USER4> This is what I love about HN - a simple question about a somewhat rare technology and a detailed answer.

I tried running Cockroach DB a few years ago too, also on Kubernetes. It worked but today we just use Postgres. Turned out we didn't really have the problem Cockroach DB is designed to solve. We just needed a reliable performant SQL database.

</comment></comment><comment><USER5> Using it in production since early 2018 as our data backbone. Other systems are feed by using CDC.

Killer features: Change data capture (to feed elk for full text search & as a event bus for microservice), follower reads, horizontal scaling, easy to manage/operate, compliance by letting user decide where should their data reside (row-level partitioning)

Before choosing crdb we made a deep investigation about the ppl involved, the activity on github, the quality of the process/PR & contributions to the go ecosystems. It's incredible how high is the quality bar for these folks.

The first version wasn't that good but since 2.0 things changed A LOT.

Really enjoying it

</comment><comment><USER6> We used it in production for a few months with around 4k QPS and a P99 of ~100ms. We specifically were using it because it has a "follow the workload" feature where it moves ranges of data to the server/region where they're most used.

We ran into a few panics (crashes) but support was great and they managed to fix the issues within a few days. Ultimately we stopped using it after multiple cascading failures that ended up bringing down the whole cluster. Granted we were running 4-core nodes with high CPU utilization so I'm sure it would've performed better with more cores but it wasn't in our budget.

</comment><comment><USER7> We have used it in production for over a year in a highly threaded environment, with multiple terabytes of data, and have had virtually no issues. The support is excellent, but it practically takes care of itself. The Postgres compatibility is very good, and we rarely run into compatibility issues. If you have a highly concurrent OLTP workload, I would suggest using CockroachDB.

</comment><comment><USER8> I wanted to try it before, but the clock synchronicity thing scares me too much to dare. How do you even keep them in line? What happens if I'm monitoring and see an elevated clock-skew? Can I even do anything or do things just start to fail mysteriously?

I think CockroachDB will be amazing when cloud providers start offering APIs for High-Resolution Clocks as a service, of even better, if I got to opt-in to the system clock being synced to highres, much like in GCP you can upgrade your instance's network to premium tier on the fly.

<comment><USER2> NTP works fine. This is a very minor risk. VMs on cloud providers are already preconfigured to use NTP with their own time servers so they are very reliable [1][2] and you can always add your own monitoring [3]. Also CRDB is tested with clock-drift and you can read the jepsen report [4].

1\. [https://aws.amazon.com/blogs/aws/keeping-time-with-amazon- ti...](https://aws.amazon.com/blogs/aws/keeping-time-with-amazon-time-sync- service/)

2\. [https://developers.google.com/time/](https://developers.google.com/time/)

3\. [https://docs.datadoghq.com/integrations/ntp/](https://docs.datadoghq.com/integrations/ntp/)

4\. [https://jepsen.io/analyses/cockroachdb- beta-20160829](https://jepsen.io/analyses/cockroachdb-beta-20160829)

<comment><USER9> >NTP works fine. This is a very minor risk.

Until it doesn't. Low probability large impact doesn't directly translate to minor risk imo

<comment><USER10> I'm terrified of meteors too!

</comment></comment></comment></comment><comment><USER11> Can we expand this question to include other so-called NewSQL database technologies like TiDB, YugaByte, NuoDB, Citus, etc.?

<comment><USER2> TiDB is mysql-compatible, written in Go, and runs on top of TiKV which is a custom key/value store written in rust. It's rapidly developing and serves well for a horizontally scaled simpler OLTP workload. It can handle OLAP queries better than CRDB but neither are great at it. It also has more moving pieces with the placement driver, TiKV and TiDB layers so install is harder.

Yugabyte is a proprietary C++ document-store engine called DocDB running on top of RocksDB. It offers access in multiple APIs with Redis, Cassandra CQL and now PostgreSQL. Dataset can only use a single API so you pick when creating each one. The PG part is getting the most focus right now and is very capable since it takes the planning part from PG itself. Supports more advanced queries than CRDB. More moving pieces again with a separate Master and Tablet layers. Great UI with cloud integration and multi-regional management built in. Also recently made all enterprise features free for open source too.

Citus is an extension, basically an automatic sharding system for PG that works across servers. It's bottlenecked on a single master node working with multiple workers but there's an experimental branch called Citus MX to make every node master-capable. Since all servers are just running PG, you have all the expected functionality and extensibility. All normal tables in a database live on the master (which can be limiting) and only "distributed" tables will be stretched across worker nodes. If you have a good scheme for sharding (like TenantID) then all data for tables sharing the same shard key can live on the same server and get 100% query functionality. Distributed SQL queries are not as good and can have strange failures. There was big recent update to improve this but you will have issues with very complex queries and joins. Interesting use-case for OLAP workloads that need OLTP like updates with indexes and constant writes. Microsoft uses this for a project and now owns the company so its available on Azure.

No experience with NuoDB but another mention is MemSQL for a distributed relational data warehouse that has a unique in-memory rowstore and on-disk columnstore model for fast OLTP and OLAP query support. The best support for complex SQL out of all of these and will probably run anything you send it. MySQL compatible and one of the most polished database products. It's expensive and proprietary although now has a free limited community edition, but extremely fast for analytics while maintaining usability.

<comment><USER2> Adding on: there's also Vitess for anyone looking for automatic sharding for MySQL. It's not an extension but works as as middleware that operates mysql instances for you and has great support on Kubernetes. Lots of big companies as users and originated from Youtube. The system is written in Go and has unique features like table materialization and resharding ability.

<comment><USER12> Vitess is far too complex to run imo unless you want to become a vitess-admin. We tried it for good 6-weeks to get a working PoC and moved to CRDB and had one working in hours.

</comment></comment></comment></comment><comment><USER13> I tried to use CockroachDB for a side project. Gave up after a month on and off of trying to migrate a x00 GB MySQL database to it.

If you are starting a new project and willing to build on it, it looks pretty awesome. But there is zero migration capabilities, which in my mind would be the core focus of a product that pitches drop in scalability.

<comment><USER2> It's Postgres-compatible so MySQL will be more difficult to migrate seamlessly, but they do documentation detailing the process and functionality to restore a database dump from object storage: [https://www.cockroachlabs.com/docs/stable/migrate-from- mysql...](https://www.cockroachlabs.com/docs/stable/migrate-from-mysql.html)

Did this not work?

<comment><USER13> Sure - but the majority of customers with cash who want to migrate to something better are not going to be coming from Postgres.

The MySQL import does not work for a lot of edge cases. They are aware of this and have open bugs that don't seem to be moving quickly. :/

</comment></comment></comment><comment><USER14> Used in DC/OS for the Identity and Access Manager [https://docs.mesosphere.com/1.13/overview/architecture/compo...](https://docs.mesosphere.com/1.13/overview/architecture/components/)

</comment><comment><USER15> Not directly related.

But you can definitely find some case studies from here - [https://www.cockroachlabs.com/community/tech- talks/](https://www.cockroachlabs.com/community/tech-talks/)

</comment><comment><USER16> Does anyone know how much the support costs?

<comment><USER17> Yeah, without at least a few lower-level support pricing examples, I just assume it's too much and move on when I've looked at it... the backup limitations for community installs is a no go for me.

</comment></comment><comment><USER12> We are using it in production on Kubernetes. As a former MS SQLServer DBA here are some thoughts:

Firstly, if you're using SQLAlchemy as anything more than a query builder rethink your approach and act accordingly. That being said, maybe rethink the use of an ORM at all -- but that's my pet peeve it might not be yours. SQLAlchemy in particular at least in versions 2.x just doesn't like the retryable transaction approach that CRDB takes. We've had to move to modifying SQLAlchemy in our application to get it to co-operate. Had we been using stored procedures or raw sql this would not have been a problem.

We started with v2.1.x and are now on v19.1 with a commercial license and overall are happy with the support: we have direct access to the engineers via slack, a ticket escalation process with quick turnaround times, and monthly meetings -- all that help us with the edge cases one is likely to hit with a new database.

Even the smallest requests get heard: I had been using one of the betas to check out what would be coming in the admin UI and didn't like the SQL query plan at the time and mentioned it in passing and a product manager reached out and picked my brain and my input along with others made it into the shipping release. The turnaround and the professionalism was impressive.

Our ops team loves CRDB. We set it up in K8s and it just works. It heals itself if a node goes down and replication ( a huge pain the ass with MySQL) is seamless. That said, they have a managed CRDB option, which if we didn't insist on running our own infrastructure I would have signed our company up for instead of rolling it ourselves.

I'd say in our use case properly setup schemas with good indexes doing queries that are not OLAP like, but OLTP in nature, perform very well.

We went with CRDB about 6 months ago because it had the best Kubernetes support at the time and was much easier to get started with.

We recently did a Mysql 5.7 to CRDB migration (actually a number of them for our partners) and it went smoothly. We did many trial runs, but I wrote some MySQL dump and import scripts in python and we just waited for it to complete. The documentation helped there a lot.

It's not perfect, nothing is. We had hoped for a simple drop in replacement for a single node Postgres instance but one that could scale and be "cloud native". It is sort of like that. With more nodes you have the overhead of consensus but it scales with more nodes almost linearly in performance, and it's robust -- we can kill a node or two (depending on your replication factor you can determine how many nodes you want to allow loss of) and bring it back up again and it's fine. There's novel approaches with interleaving that help performance, the documentation is great, the support is as well.

Best thing to do is try it out -- you can play around with a paid license for free -- and try it on your data and workloads.

</comment><comment><USER18> What other solutions are you evaluating?

</comment><comment><USER19> As a DBA, I can say that it takes 5 to 10 years for a new database or file system to be ready (reliable, few global mutexes, third-party tools, programming language support, etc.) for production with paid customers.

So you should be using MySQL or Postgres for the Source of Truth (SoT) and things like Cassandra or CockroachDB for specialized use cases (caches, end- user metrics with TTL, etc.) But I would never put important data in a distributed database since there's too many moving parts and it's impossible to verify if the input matches the output over a decade.

If you need multi-master for failover today, I would pick Percona XtraDB Cluster (I've used it in production for 5 years) or MySQL 8.

[https://dev.mysql.com/doc/refman/8.0/en/group-replication- mu...](https://dev.mysql.com/doc/refman/8.0/en/group-replication-multi- primary-mode.html)

If you don't have paying clients, then it really doesn't matter what tools you choose.

An example of the above is MongoDB, irrationally beloved by HN. Until recently it had a global write mutex and a POS (myisam-class) storage engine. After most companies migrated off of it for production, there are two Innodb-class storage engines now (native and WiredTiger) that add credibility ... after a decade of data loss and poor write performance. Those new engines will be ready for heavy-duty use in ... about 5 years.

<comment><USER20> Exactly. I also migrated companies off from MongoDB after a total data-loss that even the vendor could not recover or explain.

Postgres is surprisingly capable, rock solid service that is always my first choice. I also used Riak in production for several years and the biggest problem was with it that the operations team forgot the root password because they did not need to login for a long time. :) I have heard great things about Percona XtraDB but I never used it. RocksDB is also interesting but it is just a data store not a full fledged service, based on it there is one service called TiDB that I have heard is very reliable.

<comment><USER2> TiDB is not based on RocksDB, it runs on top of TiKV which is a custom distributed key/value store written in rust.

<comment><USER20> I thought it uses RocksDB and the Rust interface they wrote.

[https://github.com/pingcap/rust-rocksdb](https://github.com/pingcap/rust- rocksdb)

[https://github.com/tikv/tikv/blob/master/src/storage/kv/rock...](https://github.com/tikv/tikv/blob/master/src/storage/kv/rocksdb_engine.rs)

<comment><USER21> TiKV is built on top of RocksDB, see our chief engineer's speech of how we used RocksDB in TiKV: [https://pingcap.com/blog/2017-09-15-rocksdbintikv/](https://pingcap.com/blog/2017-09-15-rocksdbintikv/)

Because of the write amplification issue of RocksDB, we build a RocksDB plugin, TiTan. Here is its design and implementation detail: [https://pingcap.com/blog/titan-storage-engine-design-and- imp...](https://pingcap.com/blog/titan-storage-engine-design-and- implementation/)

<comment><USER20> Excellent thank you.

</comment></comment></comment></comment></comment></comment><comment><USER22> Also, how "exciting" is it?

<comment><USER2> Meaning what exactly?

<comment><USER23> Something like I'd like to pretend people don't make technology choices out boredom, but I'm tired of pretending.

<comment><USER24> Of course not. They make tech choices for their CV.</comment></comment></comment></comment>


The Achilles heel of the lean startup - <AUTHOR> http://www.markevanstech.com/2012/03/14/the-achilles-heel-of-the-lean-startup/

<comment><AUTHOR> Will all the talk of Lean Startup, something about it just doesn't sit right with me. Maybe it's the obsession with metrics, or maybe it's about people's obsession with the "recipe for success".

I've had this conversation with some close friends who were hellbent on build, measure, learn like it was some math formula that would make your startup work. I work on feeling. There are a lot of times the numbers don't line up right away but I know it's not time to give up. (In all honesty, I am not well versed in the art of giving up at the right time)

Bottom line is, a lot of times new theories appear to be a panacea but they're far from it. Even good ones. If startups really were that formulaic, then you could replace the people with bots at some point, which I hope won't be happening anytime soon. (or maybe there's an analytics startup there, which sends you a text message to "Pivot Now")</comment>


The Worm and the Wiretap: Dangers of crypto backdoors - <AUTHOR> http://www.cs.columbia.edu/~smb/blog//2010-10/2010-10-16.html

<comment><USER2> "It may very well be that more serious harm will be averted if the government has more ability to monitor encrypted traffic."

It may very well be the case that once given a tool, they use it for whatever comes to mind. In fact they might even use it for harm avoidance, but 'may very well be' isn't strong enough to support the idea. Hasn't much of the results of crypto laws been applied to domestic non-terror computer crimes? Is there any safe way to be transparent about backdoor usage?</comment>


What to look for in data science teams - <AUTHOR> http://hingeloss.com/post/2017/12/29/choosing-between-data-science-teams/ <comment><USER2> as with software development, people will benefit the most by working with the most experienced, capable, and intelligent teams that they can join. junior and senior is fairly meaningless, it is all about what knowledge, technique, and resources are available for long-term exposure as a motivated 'student' will be able to gain the most from that and achieve greater success for themselves and their organization. I recommend also looking for clues as to how much friction (impediments) there is in how people go about their work on a daily basis.</comment>


PostgreSQL tips and tricks - <AUTHOR> http://www.gabrielweinberg.com/blog/2011/05/postgresql.html

<comment><USER2> A lot of this advice is not very good, or at best is misleading.

1) Turning "enable_seqscan" off is almost never a good idea. If Postgres isn't using an index that you think it should, you should figure out the underlying reason why. This would be akin to killing a fly with a sledgehammer.

2) The main thing that is going to impact whether a given index is in memory is how often that index is used, not it's size. The advice about reducing the size of indexes is true (in that a smaller index uses less memory, which is always good), but if you have 10 indexes and you only actively use 1 of them, then those other 9 indexes aren't going be taking up much memory. The biggest issue is they are going to slow down your inserts.

3) Manually vacuuming isn't really necessary for newer versions of Postgres (>8.3 I believe, where autovaccum is enabled by default). There are times when you do need to VACCUM, but probably not that often in standard usage.

Here is some simple practice advice for postgres users:

1) If you are using an older version of Postgres, upgrade to Postres 9. It has a lot of huge improvements over previous versions.

2) Purchase & read Greg Smith's PostgreSQL 9.0 High Performance (as mentioned previously by another poster). This book is phenomenal.

<comment><USER3> Agreed! Most of this advice isn't that useful and in all honesty can be bad advice for many sites. Really "fast websites" use external caching to not hit the database at all.

<comment><USER4> The post wasn't about anything but the DB layer. I cache everything, but still when you hit the DB layer, you want it to return instantly if at all possible. I said specifically at the very top this wasn't a tuning or scaling guide.

</comment></comment></comment><comment><USER5> As an aside, I'm reading Greg Smith's PG 9.0 High Performance, and it's an excellent in depth study of database optimization. The first 90 pages are bottom up: storage technologies and how to tune and benchmark them, memory, cpu, file system choices, and operating system tuning parameters. Once the fundamentals are in order, it covers the internals of PG in even more interesting detail.

Trivia: A fsync call on any file on ext3 will force a sync of not just that file but the whole volume; ext4 fixes that. If you buy large SATA drives, then partition them to "short stroke" using only the outside of the disk (discarding the slower spinning inner region) you get competitive performance to SAS disks for many workloads.

One of the best low level books I've found in a long time.

<comment><USER6> Is that book full of these trivias? I would love to read more trivias like that.

<comment><USER6> Why the down vote? And why is that blog so interesting again?

It's a shame the book doesn't have a Kindle edition.

<comment><USER7> It could be your use of the word 'trivia.' At least in the US, it sometimes carries a negative connotation. It might read to some like you are being sarcastic.

<comment><USER6> ohh... sorry, English is not my native language that's why I didn't knew this. I wasn't being sarcastic, I actually liked the comment.

Thanks for the explanation astine :)

<comment><USER8> Also, trivia is a mass noun. "Lots of trivia like this", not "lots of trivia _s_ like this".

"Details" or "specifics" sounds less ... trivial. "Nitty gritty details" ("let's get down to the nitty gritty") is also OK.

<comment><USER9> Would one of a body of trivia be a "trivium?"

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> After many years using PostgreSQL too (8.2), I have other kind of advices. We didn't tweek much the configuration. Instead, we use date indexing, denormalization triggers and different kinds of materialization to have a good performance overall.

Date indexing: This was tricky in the beginning as a timestamp with timezone column can not be indexed (because the value is not fixed). So we use timestamp without timezone columns and handle zones another way. Then, for a big table with events that you want to keep for a long time, you can CREATE INDEX i_date ON events(ts::date). Then in all queries fetching for this table, make sure to add a WHERE ts::date = 'YYYY-MM-DD' clause, this way you will hit the date index and get fast queries even with billions of rows.

Denormalization triggers: Suppose you have a Russian puppets structure like usergroups, users, events, items. You need to access to items, and also most of the time filter by a 'type' value that is properly stored at usergroups level. The straight way is to do a quadruple join, but this can be slow. So we add a denormalized column 'denorm_type' in items, with two triggers: one trigger forbids direct update of 'denorm_type', another trigger reflects any change done at usergroups level in the 'denorm_type' column. This helps a lot.

Materialization is a higher level of denormalization than the one above. It is necessary to keep the 'real' data highly normalized, but most often one need to access to the data in a more user-oriented form, specially for reporting purposes. Views are excellent at that task, but they are computed in real- time. Materialization is the process to write some of these views on the disk, and index them, for faster access. With proper triggers, it is feasible to keep these materialized views safe (ie. read-only) and always up-to-date (any change in the base data triggers an update or insert in the materialized view).

</comment><comment><USER11> Be very, very careful following the advice about disabling table scans ("enable_seqscan = false") — especially globally. In performance tuning terms, that's often the equivalent of swatting flies with a howitzer. (You can also set that on a query-by-query basis, which, depending on the query, may be more sensible.)

The fact is, though, that often-times, a table-scan _is_ the most efficient query plan. Yes, indexes can speed things up, but they do so by increasing random IO (leaving completely aside the write-time penalty you pay for indexing). When your storage medium is spinning rust, increasing random IO _will_ eventually — and much sooner than you think, at that — cost more than buffering an entire table into RAM and sorting/filtering there. Moreover, all that additional random IO will degrade the performance of the rest of your queries.

</comment><comment><USER12> While I agree with the second tip ("Replace a live table with DROP TABLE/ALTER TABLE") for ad-hoc stuff, its big disadvantage is that it requires knowledge of the indexes on the table. If you have one or more scripts to maintain that use this method on a particular table, and the indexes are changed in an unrelated update to your table structure, you have to make sure those changes are also reflected in your scripts.

Oracle allowed you to disable the indexes on a table (something like 'ALTER INDEX foo DISABLE'), which allowed them to become stale when data in the table was modified. You could do this right before a big import operation for example, and you could then rebuild (a different 'ALTER INDEX' syntax) them when the operation completed.

PostgreSQL doesn't appear to have an equivalent at the moment.

edit: wording

<comment><USER13> Doesn't PostgreSQL support something like:

        RENAME TABLE current TO current_old,
                     current_new TO current;

Which is the common pattern of atomically swapping out tables in MySQL with zero downtime.

<comment><USER14> Almost all DDL is transactional in PostgreSQL, so the usual equivalent to the above would be:

      begin;
      alter table current rename to current_old;
      alter table current_new rename to current;
      commit;

Transaction-safe DDL is a very handy thing indeed.

</comment></comment></comment><comment><USER15> One thing that helped us a lot was setting

checkpoint_completion_target = 0.9 # up from 0.5 default

So the checkpoints get written out more evenly and cause less and shorter "stop the world" events. This was especially painful on Amazon EBS (which is a relatively bad idea by itself btw) with large shared memory size. Decreasing this actually improved performance for us.

</comment><comment><USER16> Increasing shared buffers should not be so far down the list. It is the first, single biggest impact on performance that you can do. I think everyone knows that it's something that should be tuned right away, but I think it should very first on the list, since it's so easily overlooked by someone just starting out.

</comment><comment><USER17> Almost always, a lightweight connection pooler like pgbouncer can do wonders. It is so easy to configure and have it up and running that there is really no reason not to use it.

<comment><USER16> Just make sure your developers don't use temporary tables in very strange ways. (like between different sessions. wtf.)

</comment></comment><comment><USER18> Good stuff! I just permanently archived a searchable copy.

Off topic, but some PostgreSQL love: after using DB2 on a customer job for the last 5 months I much more appreciate PostgreSQL which is the RDMS I usually use. It seems like PostgreSQL is developed and documented in such a way to make it easy to use while DB2 is designed and documented to optimize IBM consulting services revenue.

</comment><comment><USER19> A few of these tips apply to mysql also.

Copy table from a tab delimited file. In mysql you can use load data infile which will do the exact same thing.

Indexes in memory obviously very important in mysql land too. Also when using joins make sure the columns being joined on are indexed.

Using innodb o_direct will prevent linux from swapping out the mysql process.

Mysql's default in memory table tables are very small which usually results in creating on disk tables, to prevent this increase tmp_table_size and max_heap_table_size. Alternatively you can specify mysql engine memory for in memory tables if you know how big they are going to be.

<comment><USER20> _Using innodb o_direct will prevent linux from swapping out the mysql process._

This is close, but not exactly true. Linux is loathe to discard buffer cache (even with swappiness at zero) it will often choose to swap out MySQL code/data instead when faced with downward memory pressure.

Using O_DIRECT helps to not swap out MySQL because it doesn't load up the buffer pool with a bunch of already-cached-by-mysql data, but it doesn't _prevent_ it. A big rsync job for example will still load every copied file into the buffer cache and potentially cause swapping.

If you want prevent the InnoDB buffer pool (or Oracle SGA) from ever being swapped out use huge pages; they can't be swapped out.

I've read that the split-LRU kernel patch (kernels >= 2.6.28) fixes a number of these bad swapping behaviors, but I haven't tried it myself.

</comment></comment><comment><USER21> I've been meaning to get around to playing with Python/PL and Postgres for a little while now</comment>


Keep Ruby Weird - <AUTHOR> http://keeprubyweird.com/?utm_source=hackernews&utm_campaign=announcement <comment><USER2> Seems like a good idea. How do they plan to do it?

<comment><AUTHOR> Can you be more specific? We'll be hosting downtown at the Alamo Drafthouse Ritz, a movie theater known for its quality experiences and great food. We're holding a CFP for speakers.

Does that answer your question?

<comment><USER3> how is that weird?

<comment><USER4> It's a reference to this: [http://en.wikipedia.org/wiki/Keep_Austin_Weird](http://en.wikipedia.org/wiki/Keep_Austin_Weird)</comment></comment></comment></comment>


Show HN: LinkedIn for Chatbots - <AUTHOR> https://presbot.com/ <comment><USER3> I really, really need to be able to interact with one of these to evaluate it. like, demo right on the homepage without having to sign up or anything.

Right now I have no idea what it is, and no incentive to invest any amount of time finding out. I did not wake up this morning thinking that I need a chat bot for my resume, so they really need to make it clear wtf this thing is and how it will help me accomplish a goal

<comment><AUTHOR> No signup needed to chat with one. Chatbots are accessible publicly once created.

Check this out - [https://presbot.com/network/](https://presbot.com/network/)

or here's one - [https://presbot.com/3-mohapsat/](https://presbot.com/3-mohapsat/)

<comment><USER4> I too struggled to understand what it did. Everything on the site is so vague it doesn't really mean anything or a simile that doesn't land.

If I understand correctly from trying out the examples, it's supposed to be a chatbot that a potential employer can ask questions about you? If that's the case, what is the improvement over "the competition" of a CV/résumé?

I tried 3-mohapsat and I don't think it's working at all:

Me: Where have you worked in the past?

Presbot: What are you looking for in the next opportunity

Me: Where are you working at the moment?

Presbot: What are your areas of expertise

Me: What is your biggest achievement?

Presbot: Ahh, I don't know. I'll bring this up in my meeting with Alexa in a couple hours.

Presbot: I pulled some results from Wikipedia for you:

Presbot: [Jann Arden] [List of Billboard Hot 100 chart achievements and milestones] [The Highwaymen (country supergroup)] [The Matrix] [Your Name]

Me: What kind of job are you looking for?

Presbot: system overload.. beep bop!

[Edit: Also, possessive "its" has no apostrophe – I see that mistake three times on the homepage alone.]

<comment><USER5> The possessive 'its' should not have an apostrophe, similar to his or hers. [https://www.dictionary.com/e/its-vs-its/](https://www.dictionary.com/e/its- vs-its/)

<comment><USER4> Yes, I know… that's why I said possessive "its" has no apostrophe.

</comment></comment></comment></comment></comment><comment><USER6> For some reason, I read the title as a social network for chatbots to look for jobs. Honestly, a little disappointed to see it that it wasn't that.

<comment><USER7> Yes, I think TS need to edit the title because it tends to give misunderstanding

<comment><AUTHOR> Presbot is a platform to build your professional chatbot identity and engage with your professional network. Drawing inspiration from Linkedin, Presbot has real people behind the chatbots who can certainly network with each other but the chatbots can't on their own free will. Also, Linkedin is for people to make meaningful professional connections that can help you land your dream job, Presbot has the same tenet.

</comment></comment></comment><comment><USER8> I gave it a try. On the profile I tried, the blurb was of the form "As <position> at <company>, I'm focused on using <technology>...", so I asked:

\- What kind of <technology> experience do you have?

\- How long have you been <position>?

\- When did you join <company>?

\- What does <company> do?

Result: 0/4\. "I have no clue" or similar meaningless answers every time.

<comment><USER9> > what was your last job?

> Yes, I am available. Please email me more details or a link to the job > requisition at ...

Tbh a classical ELIZA style chatbot would have gotten this right. And it could have run in the browser without a server or language model.

</comment></comment><comment><USER10> I do like the idea, but I can't seem to import my LinkedIn profile. That for me is an essential feature. Forcing me to fill out a lot of data without an immediate return seems to be a bit of a big hurdle. Also in the advanced training page the user can enter custom questions. I feel like this page could be improved with sample questions, otherwise it is hard to deduct what to write there.

<comment><AUTHOR> Agreed. Importing LinkedIn profile is in the works. I like the idea of providing sample questions to pick from in advanced training, added to the roadmap. Cheers.

</comment></comment><comment><USER11> It looks like "Debug Mode" is enabled in this website [1].

That may contribute to some security vulnerabilities in the future.

I encourage you to consider turning debug mode off in production.

[1] [https://presbot.com/1-<>/](https://presbot.com/1-<>/)

<comment><AUTHOR> Thanks, was testing some bugs that were reported. Turned it off, will bounce the app once the traffic dies down a bit.

</comment></comment><comment><USER12> This is an interesting idea! Wouldn't the title make more sense if it was Chatbots for LinkedIn?

It is quite funny in its responses but not quite coherent. Tried questioning this bot - [https://presbot.com/57-minkowsky/](https://presbot.com/57-minkowsky/)

Me: Are you an engineer?

Presbot: Ahh, I don't know. I'll bring this up in my meeting with Alexa in a couple hours.

</comment><comment><USER13> I love the idea. At this point, I wouldn't put it on my profile, but can see the value in doing so once it's progressed.

Right now, much of the interaction is with the bot itself, and not the person it's representing. As a questioner, I don't care how the bot is doing - <USER2> would expect the answer to come on behalf of the bot owner. The humor around "asking Alexa in a few hours" was unexpected, and the links provided didn't seem related to my original question.

Also, please support submitting the question when hitting Return, rather than forcing a click on the submit button every time.

Again, I think natural language chatbots can be a big deal, acting as interactive agents that represent their owner for all sorts of situations. But until that interaction is smoother and slightly more believable - not "out of the uncanny valley" perfect or passing a Turing test, but closer to an actual dynamic conversation - it's tough and provides less value, more slowly, than simply scanning profile text.

<comment><USER13> Let me add that it might be better if the chatbot WERE acting as an agent, referring to its owner by name, instead of pretending to BE them. For example, when asked "What are your areas of expertise" it might be better if the chatbot responded with something like, "diggum has 15 years experiencing designing creative tools for designers..." Right now, it's a bit split personality responding as the bot for some things and as the owner for others.

<comment><AUTHOR> Thanks for the great feedback

\- The humor does seem out of place for a professional chatbot experience, will get that sorted out \- Return was interfering with the private messages, will enable that as soon as I figure that out \- Also, the part about acting as an agent and not split personality (1st person vs 3rd) is what had me confused as well. I'll update the response in the 3rd person format.

</comment></comment></comment><comment><USER14> Something like this would be a pipedream in 2012. Really amazing, but at the same time a bit concerning - there is this AI engine that acts on my behalf and expresses my personality. That's pretty damn creepy.

<comment><USER15> Automate all the boring stuff away, like networking and job hunting! \s

Sarcasm aside, I wouldn't mind if there were digital cookies or agents which represented us in mundane stuff. Imagine if it knew you so well it could schedule dates, or negotiate a salary on your behalf.

</comment></comment><comment><USER16> What if the chatbot asked questions and created a resume for me?

What if the bot emails me to ask the question "What are you working on?"

What's new?

What have you worked on these past few days?

And use the answers to create a constantly updated resume.

</comment><comment><USER17> How can I see one of these in action?

<comment><USER10> I'm not associated with them at all, so YMMV. I guess the only way is to create one yourself, which is a bit of a pain as I wrote above. Here is my bot [https://imgur.com/a/GRMUlTB](https://imgur.com/a/GRMUlTB) I'm sure it be possible to get a better result than me though.

</comment></comment><comment><USER18> Looks fun, I'll have a try later today.

BTW : Create a free personal chatbot in a breeze, and share it's link on your resume! -> its link?

<comment><AUTHOR> fixed, thank you.

<comment><USER18> And someone downvoted me for helping fix a typo. Hacker news, much love.

<comment><AUTHOR> Repped

</comment></comment></comment></comment><comment><USER19> > hi

Looks like I have no information on K.'s' professional achievements

> oh really?

Presbot 1:47 AM

Hmm… no clue. I'll try to ask my bot buddy Google Assistant to see if it knows.

I pulled some results from Wikipedia for you:

\- 70/20/10 model (learning and development)

\- List of songs and yells of the University of Trinity College

\- O RLY?

\- Really Really Free Market

\- Really Really Happy

<comment><AUTHOR> Yeah, it's not a great conversationalist just yet, but you'll see it'll get better with time. Conversations, State and Dialogue management are on my roadmap. Thanks for sharing.</comment></comment>


Mapping the Segregation of Metro Atlanta's Amenities - <AUTHOR> https://www.citylab.com/equity/2018/05/mapping-the-segregation-of-metro-atlantas-amenities/560912/ <comment><USER2> This article completely fails to address the reality of visiting South DeKalb. Go drive around in Google Street view and you'll see abandoned commercial property after abandoned commercial property. Businesses struggle to exist there.

The problems there cannot be fixed by zoning more commercial real estate and the comments about healthy restaurants are laughable: look at the restaurants that thrive in the area in question and look at the areas with the abundance of healthy restaurants. There's zero overlap.

If a salad bar chain could do well in South DeKalb, they'd have no shortage of venues to consider. Atlanta is self-segregated in a multitude of dimensions that are much deeper than race.

<comment><USER3> you're missing the big picture.

[https://www.atlantastudies.org/2017/09/07/jason-rhodes- geogr...](https://www.atlantastudies.org/2017/09/07/jason-rhodes-geographies- of-privilege-and-exclusion-the-1938-home-owners-loan-corporation-residential- security-map-of-atlanta/)

<comment><USER4> That article you posted uses a set of maps as its source, but if you go to the maps they refer to, a lot of cities have changed drastically where there was redlining. Atlanta may not have changed much from the redlined map, but Los Angeles and San Francisco definitely have, and those were the only ones I looked at. Many of the areas that were redlined are full of multimillion dollar SFRs and lots of great shops and commercial properties.

</comment></comment></comment><comment><USER5> One thing that is missing from the maps is the interstates, 20/75/85/285\. Atlanta is very much defined by its interstates and it contributes to our overly divided mature. Leaving them out clouds the analysis of the food part, though the others seem to mostly stand fine without it, but it's still be nice to see.

</comment><comment><USER6> I own a home in Atlanta and I've noticed this too, but I think this article misses a bigger point: Crime. If you're a corporation, or business owner, it seems obvious you'd want your business in a lower crime area. The city wants you to build in neglected areas because the hope is commerce brings higher incomes, which may result in safer streets, etc. Which begs the question: Who should take the lead in this, the businesses, or the government? I think it's asking a lot to ask/demand businesses take the risk first, before the government has done a better job of reducing crime, poverty, and improving infrastructure.

<comment><USER7> That's what the subsidies and other incentives that usually accompany that are nominally for

</comment></comment><comment><USER8> Segregation is common in US cities, and not just in the South. A friend in civic tech likes to say, "there is only one map of Chicago". Over time I've found it to be true: whether you're looking at these types of amenities, census data, crimes, health metrics. If you look at mapping apps like Runkeeper, you'll be able to point out U.Chicago as an island of activity within the South Side.

Over time, ideas of how our cities are and ought-to-be get cemented in and polarized. Housing prices reflect that pretty well and businesses listen to that before opening a new location. When economies do grow, instead of gradual betterment of everyone living there, you see a neighborhood flip/gentrify so quickly; look at Bushwick or Williamsburg in Brooklyn.

</comment><comment><USER9> > persistent segregation which was caused by government and private market > policies and practices.

It could also be related to crime. The researchers should overlay their data with a map of crime.

In Seattle, bank branches in some areas have the tellers protected by bulletproof glass. In others, there is no protection at all. These are branches of the same bank. My neighborhood bank closed and left because of regular armed robberies.

<comment><USER10> It's the policy of redlining, outlined in this extensive history: [https://www.theatlantic.com/magazine/archive/2014/06/the- cas...](https://www.theatlantic.com/magazine/archive/2014/06/the-case-for- reparations/361631/)

The resulting economic disparity may sustain the segregation without anyone doing it on purpose anymore, but the source is clear.

</comment></comment><comment><USER11> An interesting article

But a very bad GDPR opt-in experience. You can't say "no thanks", only "set preferences". From that popup I couldn't spot how to continue without opting into to anything. Very "dark pattern" feeling, which is silly for a company that has put any effort at all into GDPR compliance. If a company puts in the hours making sure you don't opt in by default, why risk the wrath of regulators by trying to funnel opt-in? It's like they read the rules without groking the spirit of them.

Luckily I was on an iPhone so I just hit the reader view without accepting anything.</comment>


How Robert Louis Stevenson came to live, die, and be buried in Samoa (2018) - <AUTHOR> https://www.weeklystandard.com/micah-mattix/wide-and-starry-sky <comment><USER2> When living there, Stevenson wrote the beach of Falesa, a thriller about the rivalry between the white protagonist, a newcomer copra trader, and a colonial of the old guard who has become somewhat of a cult leader on the island.

It's a short read, but exciting, and still feels very modern. There's a touching love story in the background between the trader and his native wife, that feels fresh and open-minded and gives the story more color than for example the oppressive sickness and despair of the more famous Heart of Darkness, which is also a novella that essentially develops the same themes.

I'd recommend Stevenson. Dr Jekyll and Mr Hide is still a great mystery, ideal for that lazy afternoon on the beach.

<comment><USER3> I also enjoy his books, especially Treasure Island is a book I go back and read again once in a while.

Funny enough, I just realized I hardly know anything about the author, reading up about his life now.

<comment><USER4> For those who don't know Treasure Island is public domain and you can read it right now for free on web!

1\. [https://bubblin.io/cover/treasure-island-by-robert-louis- ste...](https://bubblin.io/cover/treasure-island-by-robert-louis- stevenson#frontmatter)

2\. [https://www.gutenberg.org/ebooks/120](https://www.gutenberg.org/ebooks/120)

</comment></comment></comment><comment><USER5> Required trivia: In Samoa, Stevenson eventually donated his birthday to someone with better use for it:

[http://www.lettersofnote.com/2012/06/i-have-now-no- further-u...](http://www.lettersofnote.com/2012/06/i-have-now-no-further-use- for-birthday.html)

</comment><comment><USER6> I found it interesting that many early to mid XX century American English and Continental authors of renown admired Stevenson (who enjoyed some fame while alive) but was not a favorite of literary critics by a long shot, so much so he was basically "delisted" in the 70s from the Oxford Anthology of English Literature, but has since enjoyed a rebound.

Very interesting life given his bouts with poor health. Made the most of it.

<comment><USER7> He sure did have an interesting life. He came out to California to marry and ended up staying a while. There's a state park just north of San Francisco named after him.

[https://en.m.wikipedia.org/wiki/Robert_Louis_Stevenson_State...](https://en.m.wikipedia.org/wiki/Robert_Louis_Stevenson_State_Park)

</comment></comment><comment><USER8> Kind of depressing that the book only has one reader review and ranks 1.6 million on the U.S. Amazon store, which means it's barely selling (I'm a publisher; that number translates to just a few copies per month at most). That's despite a boatload of editorial reviews like the featured article.

It could be because the retail hardcover and Kindle prices are over $30 (!) but I am also wondering if the market for biographies is completely saturated, especially by big books by marquee authors. I've recently read the Da Vinci and Einstein bios by Walter Isaacson, both of which took ages to get through. There's just not much time to read other stuff, like the Carreyou bio of Theranos I've been eyeing at the library.

Incidentally, the Kindle edition of the Stevenson biography is currently on sale for $3.99 on the U.S. Amazon store. I bought it based on this review.

</comment><comment><USER9> My Dad named our first nice sailboat Vailima, named after Stevenson's house. I means, if I remember correctly, 'house on 5 rivers.'

</comment><comment><USER10> The weekly standard is an 'interesting' source. Did you also notice the featured article, [https://www.weeklystandard.com/holmes-lybrand/fact-check- was...](https://www.weeklystandard.com/holmes-lybrand/fact-check-was-the- recent-california-fire-started-by-the-u-s-government-using-space-lasers).

</comment><comment><USER11> One of the most fascinating things I've read by Stevenson was his description of his creative process, the success of which he attributed to the "little people" in his dreams, or what he called his "Brownies".

Referring to himself in the third person, Stevenson writes:

 _" This honest fellow had long been in the custom of setting himself to sleep with tales, and so had his father before him; but these were irresponsible inventions, told for the teller's pleasure, with no eye to the crass public or the thwart reviewer: tales where a thread might be dropped, or one adventure quitted for another, on fancy's least suggestion. So that the little people who manage man's internal theatre had not as yet received a very rigorous training; and played upon their stage like children who should have slipped into the house and found it empty, rather than like drilled actors performing a set piece to a huge hall of faces._

 _" But presently my dreamer began to turn his former amusement of story- telling to (what is called) account; by which I mean that he began to write and sell his tales. Here was he, and here were the little people who did that part of his business, in quite new conditions._

 _" The stories must now be trimmed and pared and set upon all fours, they must run from a beginning to an end and fit (after a manner) with the laws of life; the pleasure, in one word, had become a business; and that not only for the dreamer, but for the little people of his theatre. These understood the change as well as he. When he lay down to prepare himself for sleep, he no longer sought amusement, but printable and profitable tales; and after he had dozed off in his box-seat, his little people continued their evolutions with the same mercantile designs._

 _" All other forms of dream deserted him but two: he still occasionally reads the most delightful books, he still visits at times the most delightful places; and it is perhaps worthy of note that to these same places, and to one in particular, he returns at intervals of months and years, finding new field- paths, visiting new neighbours, beholding that happy valley under new effects of noon and dawn and sunset. But all the rest of the family of visions is quite lost to him: the common, mangled version of yesterday's affairs, the raw-head-and-bloody-bones nightmare, rumoured to be the child of toasted cheese -- these and their like are gone; and, for the most part, whether awake or asleep, he is simply occupied -- he or his little people -- in consciously making stories for the market._

 _" This dreamer (like many other persons) has encountered some trifling vicissitudes of fortune. When the bank begins to send letters and the butcher to linger at the back gate, he sets to belabouring his brains after a story, for that is his readiest money-winner; and, behold! at once the little people begin to bestir themselves in the same quest, and labour all night long, and all night long set before him truncheons of tales upon their lighted theatre._

 _" No fear of his being frightened now; the flying heart and the frozen scalp are things by-gone; applause, growing applause, growing interest, growing exultation in his own cleverness (for he takes all the credit), and at last a jubilant leap to wakefulness, with the cry, "I have it, that'll do!" upon his lips: with such and similar emotions he sits at these nocturnal dramas, with such outbreaks, like Claudius in the play, he scatters the performance in the midst._

 _" Often enough the waking is a disappointment: he has been too deep asleep, as I explain the thing; drowsiness has gained his little people, they have gone stumbling and maundering through their parts; and the play, to the awakened mind, is seen to be a tissue of absurdities. And yet how often have these sleepless Brownies done him honest service, and given him, as he sat idly taking his pleasure in the boxes, better tales than he could fashion for himself."_

...

 _" Who are they, then? and who is the dreamer?_

 _" Well, as regards the dreamer, I can answer that, for he is no less a person than myself; -- as I might have told you from the beginning, only that the critics murmur over my consistent egotism; -- and as I am positively forced to tell you now, or I could advance but little farther with my story._

 _" And for the Little People, what shall I say they are but just my Brownies, God bless them! who do one-half my work for me while I am fast asleep, and in all human likelihood, do the rest for me as well, when I am wide awake and fondly suppose I do it for myself. That part which is done while I am sleeping is the Brownies' part beyond contention; but that which is done when I am up and about is by no means necessarily mine, since all goes to show the Brownies have a hand in it even then."_

[http://www.worlddreambank.org/S/STEVBROW.HTM](http://www.worlddreambank.org/S/STEVBROW.HTM)

<comment><USER12> Although "Brownies" is rather cringe-worthy, I routinely write out problems just before going to sleep, and often awake with solutions.

<comment><USER11> Why do you think it's cringeworthy?

Stevenson was likely referring to this: [https://en.wikipedia.org/wiki/Brownie_%28folklore%29](https://en.wikipedia.org/wiki/Brownie_%28folklore%29)

 _" A brownie or broonie (Scots), also known as a brùnaidh or gruagach (Scottish Gaelic), is a household spirit from British folklore that is said to come out at night while the owners of the house are asleep and perform various chores and farming tasks."_

That appears to be pretty unobjectionable to me.

<comment><USER12> Thanks. Although I still wonder about the origin.

</comment></comment></comment></comment><comment><USER12> FYI: Moore (1943) Defoe, Stevenson, and the Pirates. [https://sci- hub.tw/https://www.jstor.org/stable/2871539](https://sci- hub.tw/https://www.jstor.org/stable/2871539)

</comment><comment><USER13> I just happen to be reading Treasure Island for the first time. Well worth a read!</comment>