An Amateur Porno Set Off a Federal Witch Hunt - <AUTHOR> https://www.thedailybeast.com/deep-sleep-how-an-amateur-porn-film-set-off-a-massive-federal-witch-hunt <comment><USER2> Summary: Young man has a passion & dream to make movies. He has a thriving business but it's not enough to fund the movie. His friends agree to fund a movie but with a clause - it must be a porno movie.

He researches existing literature and it sucks. He won't make porn. He'll make a real movie that also has explicit sex.

He shoots normal scenes first, and sex scenes later. Sex scenes has some issues like the actors unable to get erection and him not knowing how to direct the scenes.

Movie is finished. He edits, distribution says he must include at least one lesbian scene, some cum shots... things he'd cut out.

Movie is a success in theaters and gets sold out every single day and then all hell breaks loose. The police comes after him. Lots of complaints were sent to officials - some by people who hadn't actually watched the movie.

The film maker had used a property owned by the church without permission and the church was the force behind the investigations.

The court case was a sham. The investigators invokes anti-fornication laws 100s of years old. And the cops (probably) sets the film maker up.

One of the lead actors commits suicide just as the case becomes a federal one. The government takes all the earnings of the film, fines him, bans him from film making for a few years and gives him a suspended jail sentence.

<comment><USER3> Are you implying laws should be ignored after a few years go by?

That's not how laws work. Many of the laws that our important to daily life are hundreds of years old.

<comment><USER4> I think that'd be a wonderful thing if all laws had a natural sunset clause and were forced to be renewed to see if they confirm to the standards of the current society.

Private lewdness was one of the charges. Even the prosecutor said that it was all a bit over the top but he had these laws as tools so he used them.

<comment><USER5> In some jurisdictions (especially civil law, I think) the DA has no discretion and must follow up on any accusation that could lead to criminal charges. There is no possibility for selective enforcement.

This way the legislative branch has pressure to amend the criminal code as it becomes obsolete---in this case, imagine if people had started accusing others of fornication and private lewdness left and right, and DAs through the state were forced to follow up on those accusations...

<comment><USER6> > There is no possibility for selective enforcement.

There are huge holes in that. What if, hypothetically, the DA looks at a situation and says 'there is no chance of criminal charges', but some 40-60% of the population thinks there should be?

There is even a fairly topical case of that where a substantial number of Democrat lawmakers are upset with how Barr is handling the Muller report. Imagine squaring a similar case with a law that says prosecution must proceed - I think Barr would be quite happy to claim there is no chance that an Obstruction case would lead to criminal charges.

In the confrontational Western-style justice system the prosecutor always has discretion. Not to acknowledge that just muddies the water in difficult cases. It is much safer to accept that selective enforcement happens, that sometimes the justice system fails to deliver justice and that there need to be release valves in how people are appointed so the abuses don't get out of hand.

<comment><USER7> Note that this wide prosecutorial description is not universal among western countries (maybe it is a common law thing?)

Germany has compulsory prosecution ([https://en.m.wikipedia.org/wiki/Compulsory_prosecution](https://en.m.wikipedia.org/wiki/Compulsory_prosecution), the German article Google Translated has mich more detail) and it's subject to review by the courts ([https://de.m.wikipedia.org/wiki/Klageerzwingungsverfahren](https://de.m.wikipedia.org/wiki/Klageerzwingungsverfahren) and [https://de.m.wikipedia.org/wiki/Ermittlungserzwingungsverfah...](https://de.m.wikipedia.org/wiki/Ermittlungserzwingungsverfahren))

Note that as a result of this plea bargaining are a rare thing.

</comment></comment></comment></comment></comment></comment><comment><USER8> How unapologetic the prosecutor still is offends me. Trying to use anti fornication laws from 1796 and knowing it's a miscarriage of justice? Yet still thinks it's the right decision.

<comment><USER9> The laws were on the books, and the public's opposition to what the movie- maker was doing seems to fit in with the intent and scope of the laws. Maybe you can blame him for selective enforcement, but I don't think that qualifies as a miscarriage of justice. (That is, of course, assuming we take at face value that his denial that he didn't steal and ship a copy of the movie to Oklahoma.)

The prosecutor took measures to avoid normalizing what he and the community saw as deviant behaviors. That's one of the key purposes of law: it allows society to express its condemnation of certain conduct.

<comment><USER10> Selective enforcement is absolutely a miscarriage of justice. Justice is the character of law that makes it fair. If you disagree with this, I would like to hear your defense.

<comment><USER11> The world is an inherently unfair place so it's dubious that laws that attempt to regulate it should be fair.

<comment><USER6> What you have there is an argument that is true but also unacceptable. It is true that the world is unfair and that pushing it into a fair state is impossible.

However we can't formally accept that as a society without kicking out one of the main pillars supporting the justice system. If we expect and accept that the justice system to produce unfair outcomes it becomes challenging to justify having court appearances. We could get unfair outcomes much more efficiently by just letting the police and DA arrest whoever with regular internal purges against corruption.

</comment></comment></comment></comment></comment><comment><USER12> A friend of mine looked after his parents' house while they were on sabbatical. He rented it out for porn shoots. His mother was furious when she found out -- because he'd kept all the money himself.

Now they use airBNB.

Also a porn film was shot at the Stanford AI lab (the DC Powers building) in the 1970s. The first image I ever saw transmitted over the arpanet (around 1978) was a (very grainy) frame from this film.

<comment><USER13> There are a million jokes, memes, fake stats, and even an Avenue Q song about how the Internet is for Porn. I just can't believe it has roots that far back!

</comment></comment><comment><USER14> > So he shot all the non-sex scenes first, then went back to film the required > sex.

This sounds a lot like the film Caligula, from 1979.

They got Malcolm McDowell, Teresa Ann Savoy, Helen Mirren, Peter O'Toole, John Steiner, John Gielgud etc - but re-shot a bunch of pornographic scenes long after principle photography had wrapped, and then edited the whole thing into what's now a bit of a cult classic, but at the time was banned by most countries censorship boards, disavowed by many who were involved in the production, and the subject of a swathe of legal attacks.

[1] [https://en.wikipedia.org/wiki/Caligula_(film)](https://en.wikipedia.org/wiki/Caligula_\(film\))

</comment><comment><USER15> The '70s seem to have been an interesting time: [https://longreads.com/2019/04/29/what-i-learned-from- doing-a...](https://longreads.com/2019/04/29/what-i-learned-from-doing- amateur-porn/) . Link is all text and presumably SFW, but caveat emptor.

<comment><USER16> I remember in an interview, Henry Miller contended that in the time before there was lot of media portrayals of sex that people actually engaged in sex more.

It seems like the 70s were something of the time when the media began honestly representing the social undercurrent of sex. But that representation was perhaps part of it going away.

<comment><USER17> There was a sexual comedy that came out some years ago that said the openness of sex led to an increase in social acceptance of more things out the norm. The main characters however complain that they missed out because they grew up in the 80s when HIV hit which led to a massive backlash and obvious public health scare. The characters are all friends and they eventually plan a party based on what they think they missed out on. The theory makes a little more sense to me.

</comment></comment></comment><comment><USER9> The more I learn about the 1970s the more the 1980s makes sense to me.

<comment><USER18> I was born in the 80s so I have zero context for either decade.

Recently watched the CNN documentaries on each decade and it was truly eye opening how nuts things were essentially every decade in one way or another. But above all of then, the 70s seemed like the WEIRDEST time to me.

Highly recommend the documentaries - super informative. Was fun to re-live the 90s and 00s a bit too.

</comment></comment><comment><USER19> Seems like maybe the writers of Boogie Nights were aware of this fiasco...

</comment><comment><USER20> What's "amateur" about that? It seems that the word appears only in the title.

Edit: he had a $25k budget at least (over $150k adjusted for inflation) and while it was his first movie he hired professional performers and the intention was the commercial exhibition of the film in movie theaters (as it did happen).

<comment><USER21> He was clearly an amateur since he had never made any kind of movie.

But his intent was to make a professional/commercial film.

They are not necessarily disjoint concepts.

To top that off, at the end of the interview he gives himself a humorous critique suggesting he found his own work failed to make the professional grade.

<comment><USER20> I wouldn't classify as "amateur" a commercial movie where the lead roles are played by professional actors, but that may be just me. The downvotes I got agree with you. (Being a bad movie may make it look "amateurish", but that's something different.)

<comment><USER22> > a commercial movie where the lead roles are played by professional actors,

I'm not sure you've gotten anywhere, describing what professional means, when you make it dependent on other entities as an explanation. There are plenty of bad SAG actors. What about just saying "union production"? Although I think that's the minimal bar, there have been non-union successes...which don't look professional to me. eg [https://www.reddit.com/r/acting/comments/a59tum/what_are_the...](https://www.reddit.com/r/acting/comments/a59tum/what_are_the_most_successful_nonunion_films_if_any/)

<comment><USER20> Are you suggesting that films can be classified as either "union" or "amateur"? Star Wars is an amateur film then. And saying "amateur porno" is redundant.

<comment><USER22> > Star Wars is an amateur film then

Based on what? There are many unions and they have changed and stabilized over time. SAG-AFTRA [https://www.sagaftra.org/about/our- history](https://www.sagaftra.org/about/our-history) is the one that is distinguishing. The Director's Guild (which George Lucas and others, had spats with) or a writer's guild, etc have traditionally been little more than elitist extortion committees.

<comment><USER20> Based on the SAG boycott on non-DGA directors.

</comment></comment></comment></comment></comment></comment></comment><comment><USER23> I hate our legal system. I don't have any faith that justice is found there.

</comment><comment><USER24> The beginning talks about how likable the guy is.

Greatly discusses the legal trouble his film caused.

The ends with him agreeing how the film sucks and maybe the prosecutor was right! (EDIT: he says this as a joke, apparently this needs to be pointed out?)

...

I get it. That's a good story and he absolutely does come off as a likable guy.

<comment><USER25> For people who haven't read the article this is extremely misleading:

> The ends with him agreeing how the film sucks and maybe the prosecutor was > right!

He says that, only as a joke that it's so bad it should've been illegal. While in reality a great mis-justice was done to the man, and he was railroaded into accepting a plea-deal due to the gravity of the unfair charges.

<comment><USER24> Yea, that's the point. I don't remember saying "don't read the article". He makes a joke about his previous work and that's a likable thing to do.

</comment></comment></comment><comment><USER26> tl;dr</comment>


Ask HN: How Much Alcohol/Coffee Do You Drink? - <AUTHOR> I have recently started drinking up to 6 coffees per day. I don't feel any less healthy but I feel it doesn't seem healthy. Alcohol Every Friday after work.<p>How much do you drink and does it affect your Work/Family and social life? <comment><USER2> 5-6 years ago, I was drinking about a pot of coffee a day between buying it from the coffee shop down the street or just making it at the office. Now I'm down to 2 cups during a work day and maybe 1 at night if I'm feeling tired. Usually just 1 on a weekend day. I can go without it and not get a headache which is nice.

Alcohol wise; I try not to drink during the week unless it's at dinner for business or a special event (birthday, anniversary, etc...) - on the weekend, either a glass of wine or a couple whiskeys on the rocks casually with the wife while watching a series or movie.

The only thing it's ever effected was money. Buying coffee at Starbucks or Tim Hortons on a regular basis is a waste IMO. Same with alcohol - cheaper to buy a bottle at home and drink it slowly then at a restaurant. Conversely, as you get older, it becomes (or has become for us) more important what you drink then the quantity - good stuff is expensive so you don't really want to blow through it quickly.

</comment><comment><USER3> No coffee. 2 cups of tea per day. No more caffeine than that, and none after lunch, or I won't be able sleep easily or soundly. Caffeine crashes and the exhaustion from caffeine-related sleep deprivation are both so much worse than just not consuming caffeine at all. It's not worth it.

Alcohol: up to two beers at a date, happy hour, or other event. Three if it lasts more than 4 hours. A glass of wine if eating dinner with someone else. Probably averages 2-3 nights per week with any alcohol consumption. No negative effect on social life (though I do enjoy social events more when self-medicating for social anxiety with those beers). More than one drink does affect my sleep that night and energy levels for a couple days, but I haven't been hung over in a decade except for after that one wedding...

<comment><USER4> I can drink coffee up until I go to bed and have no issues sleeping. However, if I drink more than a few beers I wake up after a couple hours and can't get back to sleep.

</comment></comment><comment><USER5> 1 - 3 cups of tea per day. About to make my second one now. I worry about potential health effects, but I am striking a happy balance here in contrast to when I was drinking twice as much tea.

In terms of drinking, we usually have a bottle of wine or prosecco 1 - 2 times a week. I get one or two drinks whenever we go out for dinner so that is once a week. I don't think I see an impact, but I don't ever get drunk. I just get sleepy and I usually give in to the call.

</comment><comment><USER6> I just had to quit drinking coffee cold turkey due to stomach issues.

I was drinking around 2-6 lattes on any given day for maybe the past 2 years. Never had a problem. About 1 month ago I started having horrible acid reflux and general stomach pains, would wake up vomiting, etc. I went to the ER, discussed my habits, did some scans - and they told me to stop drinking coffee.

First week was hard. Second week I had no cravings and was feeling great. By week 3 (last week) I had lost weight (this was my only dietary change), I was feeling sharper mentally and I was no longer still half-asleep in the morning before that first cup. In short - felt a worlds difference.

I'm not against coffee now, but cutting it out of my life, even if it's temporary, definitely was the right decision for me.

(As for alcohol, I don't drink.)

</comment><comment><USER7> 5 to 7 cups of black coffee a day sometimes with a shot of espresso. Research on health and coffee is mixed though I think the overall current conclusion is that the benefits outweigh the risks. However, I admit to being biased because I really enjoy coffee.

Almost no alcohol - 1 every few weeks or months depending on circumstances. Party/reception or business dinner at high-end restaurant. And, then it is just one drink.

I am not against alcohol but very cautious around it. I have lost a friend to a drunk driver and had family members lose custody of their kids because abuse of alcohol ruined their ability to drive, work and function as responsible adults.

On the other hand I know plenty of people who drink alcohol much more regularly than I without any notable effect.

</comment><comment><USER8> Large bowl-o-latte in the morning. 10 oz of strong French Press coffee when I get to work, and another 10 oz after lunch ~3pm. Then one 12-16 oz beer/ale/porter at home to bring my plane in for a landing.

How does this effect others? Heck with them, this is the minimum functioning me.

</comment><comment><USER9> I'm in tech management,so my days tend to be pretty stressful. I also may or or may not fit the profile of a typical dysfunctional veteran.

The easiest way to answer your question is like this: If drinking alcohol was an Olympic sport I'd have a few gold medals, and I drink enough caffeine to keep the entire Chinese Red Army awake for a week.

</comment><comment><USER10> 2 double shot espressos, one early morning, and one late morning. No coffee after noon.

1 glass of watered red wine with dinner (my only meal each day). When going out, I will have 2-3 glasses of wine, but only once every two weeks or so. The rest of the time I'll drink Perrier or soda water.

3-4 glasses of water a day. More on workout days, but I find on a ketogenic diet my water requirements are less. I don't add salt to my food, and I cook all my meals. I don't use condiments either.

I don't drink tea, energy drinks, juice, smoothies, or pop. What I enumerated above is what I drink.

</comment><comment><USER11> Anywhere from 3-8+ coffees a day depending on how I'm feeling that day and what my workload looks like.

I always have a coffee first thing in the morning, another in my travel mug on my drive to work, and usually make another when I get to the office.

Some mornings/extremely busy days I'll sub in a caffeine pill for that extra boost.

I usually cycle down my caffeine use a few times a year, going from my usual 3-8 down to about 2 coffees a day. I find this helps me from building too much of a tolerance and needing to constantly increase the dosage.

I really only drink alcohol on the weekends and even then not very much.

</comment><comment><USER12> I generally don't drink coffee, it makes me jittery and anxious.

Alcohol...once a month but in large quantity. Usually go on a bender while barhopping/at a party with friends. Definitely need to shake that habit.

</comment><comment><USER13> Almost no coffee, but a fair bit of caffeine - maybe 3 or 4 cups of tea with an occasional soda mixed in. I'm working on night classes for a graduate degree as well, so towards the end of the semester mix in some energy drinks too - definitely not healthy but the only way I can make it through.

Very little alcohol, 1-2 drinks once every other week or so, often with stretches of a month or more with none at all. I just don't seem to enjoy it as much as many people do, although it is nice sometimes.

</comment><comment><USER14> 3 - 5 cups of coffee per day. And an average of 3 beers per night.

Once or twice per year I tend to take 1 - 4 weeks off from alcohol and taper down on caffeine significantly. .. sometimes down to no caffeine. It's rough going for a few days but afterwards I feel that I can't tell the difference between not consuming and consuming

</comment><comment><USER15> 2 cups of coffee a day. One after breakfast, but the coffee making process is a morning ritual. I buy specialty coffee (Verve, Four Barrel, etc), grind the beans myself, and brew them using a pour over method on the Chemex device. I'll have whatever coffee we have at work in the afternoon for another energy boost.

</comment><comment><USER16> I used to drink coffee occasionally but if I take a break or drink too much I get really jittery and sick. So I'm not longer drinking it. Though I might have a half cup before the gym as a boost.

As for alcohol, it's always present at work and it's hard to not drink. I can't say no to free beers ;)

</comment><comment><USER17> French press (~35g of beans) or 2 consecutive double espresso shots in the morning. Another double espresso shot in after lunch.

I drink a fair bit of alcohol; probably 3-10 servings during the week and 10-20 servings most weekends. _Not_ drinking would affect my social life.

</comment><comment><USER18> 2-3 Cups of coffee per day and 3-5 pints of beer (5-7% alcohol) per week.

This is considered low for myself and times in my life I have had about 2x this amount of alcohol/coffee. I am not saying my current consumption is healthy, that is just what it is.

</comment><comment><USER19> I drink 2 or 3 cups of coffee per day. I stopped drinking alcohol about 12 years ago. I'm sure it has affected my social life but it's only one of many factors.

Edited to change the wording of the third sentence.

</comment><comment><USER20> 0-3 cups of coffee per day (with 3 being rare) 3-16 drinks/2 weeks. I just graduated so that number was even higher cause of celebrations, but it was usually pretty low.

I wish there were more comments here, would be a cool survey for HN

</comment><comment><USER21> I try to limit my coffee to around 3 cups per day, or else it starts to affect my sleep.

I drink, on average, 6 standard drinks per week (all beer). Usually after work and, in the last 3 three years, no more than 2 at once.

</comment><comment><USER22> Usually 24 cups if I'm drinking coffee that day. But it's sporadic: I'll keep that rate for a month or two, and then nothing for 2-3 months.

I'll have a beer or wine or something once every 6 months or so.

</comment><comment><USER23> 2-3 double shot flat whites. Typically before 2pm - most of the time with MCT or Brain Octane oil and a bit of unsalted butter mixed into the milk while steaming it.

Another 2-3 cups of black tea. No alcohol.

</comment><comment><USER24> I go decaf for coffee and avoid caffeine. I think I feel better for it. Typically 4 or so beers a week, more if there is a night out but that's rare nowadays

</comment><comment><USER25> coffee free for the last 30 days (used to drink about 2 cups per day) and alcohol free for the past year (used to drink about 3 glasses of wine every evening and more on the weekend) Feel great - work/family and social life definitely massively improved.

</comment><comment><USER26> One coffee a day after breakfast. Have gone alcohol free for a good 5 years now! Feels better.

</comment><comment><USER27> 1-2 cups of coffee. No alcohol. Trying to stay youthful into old age.

</comment><comment><USER28> No alcohol. 1 or 2 drip coffees each day I make myself at home.

</comment><comment><USER29> None, except for the very occasional glass of wine.

</comment><comment><USER30> No alcohol. 1-2 coffee a day.

</comment><comment><USER31> none

</comment><comment><USER32> Shot of espresso in the morning. Probably like 10-12 beers a night?</comment>


Apple will let you emulate old apps and run iOS apps on ARM Macs - <AUTHOR> https://techcrunch.com/2020/06/22/apple-will-let-you-emulate-old-apps-and-run-ios-apps-on-arm-macs/ <comment><USER2> Any idea if this means you'll be able to run 32-bit macOS apps that won't run under Catalina?</comment>


Microsoft on GitHub - <AUTHOR> http://microsoft.github.io/

<comment><USER2> The page is broken on HTTPS: it tries to load [http://angular- ui.github.io/bootstrap/ui-bootstrap-tpls-0.2....](http://angular- ui.github.io/bootstrap/ui-bootstrap-tpls-0.2.0.js) (does Github even allow hotlinking these days?) and Chrome doesn't let it.

<comment><USER3> They should be using protocol-less URLs and then this wouldn't happen:

[http://benpowell.org/https-and-http-the-protocol-less-or- pro...](http://benpowell.org/https-and-http-the-protocol-less-or-protocol- relative-urls/)

</comment></comment><comment><USER4> This is really awesome to see. One unimportant nitpick: a lot of your recent Microsoft websites specify only the "Segoe" font, which means they render in Times for those of us on non- or old versions of Windows. Maybe load Open Sans or use a websafe sans-serif for us?

<comment><USER5> Yesterday I would have said they don't care about other platforms and compatibility issues. Today I'm not so sure. Anyway, looks like at least the GitHub has alternative font values:

        wf_segoe-ui_normal,"Segoe UI",Segoe,"Segoe WP",Tahoma,Verdana,Arial,sans-serif

</comment></comment><comment><USER6> Looks sort of unfinished on my end.

[https://i.imgur.com/CFN8bsq.png](https://i.imgur.com/CFN8bsq.png)

<comment><USER7> You may need to tell your browser to load scripts over HTTP.

<comment><USER8> Which is a really bad idea.

<comment><USER9> I see you're still using Netscape Navigator Gold...

<comment><USER10> Not true.

Sending javascript over HTTP does technically allow for MitM attacks. Not to mention we know how great even Chrome's loudly bragged about sandbox is (it isn't is my point).

If you are loading the patch in https, all connections should be in https.

<comment><USER9> You're right and I missed the point!

</comment></comment></comment></comment></comment></comment><comment><USER11> It's more like a code dump for now (one "initial commit" commited by dotnet- bot - [https://github.com/dotnet-bot](https://github.com/dotnet-bot))

I wonder if they're gonna let their employees push atomic commits to this repository, or if it will be more like one big monolithic commit dump from time to time.

</comment><comment><USER12> So "tile thought" is now applied to all things Microsoft? I personally prefer lists with expandable details ( an accordion menu ) Also would like to point out that the page is entirely useless without scripts enabled. I use NoScript by default and the page does not degrade smoothly at all.

<comment><USER13> Does it degrade smoothly if you disable CSS? At this point Javascript is a vital part of the web experience and it seems pointless to say that a site doesn't degrade smoothly with it disabled.

<comment><USER12> Depending on the number and variety of sites that you visit, enabling javascript universally is hazardous. You can easily observe this to be true by clicking 30-50 links randomly as fast as you can in your browser. You will find that inevitably your system will become infected with malware in the process of doing this, regardless of protection.

Generally for accessibility purposes websites should be designed so that basic text and links appear and are functional with JS disabled, and even CSS disabled. If the website is some sort of dynamic application with moving widgets than I can understand that JS might be needed, but not for a basic list of project Microsoft has put on Github.

Even if JS is used for templates, it is preferable to use semantic HTML that can be enhanced rather than using a template for the entire page and show "{tags}" all over the page when JS is disabled.

TLDR: A page with a basic list of projects shouldn't need JS. Web applications: yes, basic lists: no.

<comment><USER14> > clicking 30-50 links randomly as fast as you can in your browser

This will not give you malware. It might temporarily bog down your system though.

<comment><USER12> This is not a naive statement by me caused by assumptions. I actually did exactly this with a patched modern browser and good antivirus and my system became infected in the process.

Mind you I indiscriminately clicked on known "bad" advertisements in the process of doing this. Don't believe me? Try it for yourself. It is actually very easy to get malware if you click around foolishly.

Also, I have written multiple web crawlers, and have collected a large variety of JS based malware that can and does break modern browser security just in the process of fetching domain homepages. ( JS code embedded directly in index.html on domains )

<comment><USER15> If you're correct, you've uncovered some 0days in the wild. Go figure out which site, capture what happens, and turn it in to the respective browser developers for a nice bug bounty.

Or, you know, maybe they aren't wasting a 0day on "obviously bad advertisements".

<comment><USER12> That's actually a really good idea. Thanks. :)

</comment></comment></comment></comment></comment></comment></comment><comment><USER16> It's heartwarming to see that Microsoft is using AngularJS

</comment><comment><USER17> They really need to start serving Segoe UI to non-Windows users.

</comment><comment><USER18> I wonder about the future of CodePlex...

<comment><USER19> Codeplex needs to die... It was shit when it came into existence, it's still shit now.</comment></comment>


How to make perfect UX a priority in Agile - <AUTHOR> http://www.uxmatters.com/mt/archives/2014/05/the-secret-to-making-ux-a-top-priority-in-agile.php

<comment><USER2> Kudos for giving Blizzard due credit.</comment>


Authentication using a picture as a password - <AUTHOR> http://www.youtube.com/watch?v=dF5s-_6r0x4&feature=youtu.be

<comment><AUTHOR> I would be interested to see the mathematics behind their claim that a picture password is more complex than (what I assume is) a typical android log in password. I would imagine it would be a little more tricky for any malware to capture a password. Capturing and determining which mouse clicks (for the PC case) are significant is considerably more difficult that analysing the output of a keylogger.

</comment><comment><USER2> Looks interesting, i think there is a lot of room for innovation around how the picture is displayed so you can reduce the risk associated with people looking over your shoulder and snapping a picture of what you did. I think you could possibly even get into patterns where the picture always changes, but has a common theme. I.e. you need to click the woman, man and pet.

<comment><AUTHOR> I think you are right - I would also imagine that you could have a collage of images with meaning to you e.g. places you've worked and you pick the order your worked at them - thus the same set of pictures could be presented in different configurations changing your tap-pattern each time.</comment></comment>


Oregon Engineer Makes History with New Traffic Light Timing Formula - <AUTHOR> https://ij.org/press-release/oregon-engineer-makes-history-with-new-traffic-light-timing-formula/ <comment><AUTHOR> "Mats's work was generally met with interest, but when he e-mailed the Oregon State Board of Examiners for Engineering and Land Surveying, things took an abrupt illegal U-turn. The Board told Mats it had no interest in hearing about his ideas. Fair enough. But the Board didn't stop there. After a two-year investigation, it fined him $500 for publicly criticizing the timing of traffic lights without having a Professional Engineer license. The Board also forbid him from continuing to discuss his research."

</comment><comment><USER2> currently on the front page: [https://news.ycombinator.com/item?id=22456867](https://news.ycombinator.com/item?id=22456867)</comment>


GitLab 7.11: 2FA, publicly viewable Enterprise Edition - <AUTHOR> https://about.gitlab.com/2015/05/22/gitlab-7-11-released/

<comment><USER2> GitLab CEO here, we're very exited that we're able to open up the Enterprise Edition code and the 2 factor authentication.

<comment><USER3> I've implemented 2FA for SVN and Mercurial (not yet for git) for Indefero. I highly recommend showing users the key for their storage - I've had to extract the keys from FreeOTP and Google Authenticator a number of times.

How do you leverage 2FA with LDAP/AD accounts? Do you store/check the key in gitlab and then auth the users against LDAP/AD - or store the key in LDAP/AD?

<comment><USER4> GitLab developer here! Thank you, Sytse, for answering already, I'm happy to go into a little more depth.

> I highly recommend showing users the key for their storage - I've had to > extract the keys from FreeOTP and Google Authenticator a number of times.

I'm curious, in what situation would you need to extract the key while you still have access to it in one of your apps? We have recovery codes for the situation where you've lost the key in your app, but that doesn't seem to be what you're describing. If you're moving from one app or phone to another, you can just turn off 2FA on GitLab and then turn it on again—you'll get a new key.

> How do you leverage 2FA with LDAP/AD accounts? Do you store/check the key in > gitlab and then auth the users against LDAP/AD - or store the key in > LDAP/AD?

The 2FA flow is the same for regular GitLab users and those backed by LDAP. After the initial username/password auth step, they are presented with the 2FA form. In both cases, the key is only in GitLab.

<comment><USER3> > in what situation would you need to extract the key while you still have > access to it in one of your apps?

I use FreeOTP on Android to store and generated OTPs. Many years ago I had an HTC One (old version). I was listening to music one day and it just died - wouldn't turn on. Thankfully I extracted most of my OTP keys and was able to setup FreeOTP from scratch. If I didn't - I would be in a world of hurt for the ~20 services that may or may not provide recovery codes (I know you do - but just keep in mind phone dying or theft).

Like I mentioned in the previous post - to me a recovery key isn't to be used lightly, in my opinion it should only be used for "oh crap I need to login right now and I don't have my phone".

I'm not saying I don't trust you and recovery codes - I already got burned once and I don't want to be in that position again. My solution is to squirrel away the OTP keys. Besides - I can already get it by using a barcode scanner on the QR code you generate so I'm not sure what we are arguing about.

</comment></comment></comment></comment><comment><USER5> I love that GitLab has their code publically viewable. It provides another example of a large Rails application which is always useful as a way to "see how other people approach challenges" as well as my favorite "why the heck doesn't this work" problems that surface when implementing CI/RSpec, etc.

<comment><USER2> Thanks! Apart from code we're starting to make almost all of our internal procedures public too, we started with [https://gitlab.com/gitlab-org/gitlab- ce/blob/master/doc/rele...](https://gitlab.com/gitlab-org/gitlab- ce/blob/master/doc/release/monthly.md) did [https://about.gitlab.com/handbook/](https://about.gitlab.com/handbook/) a month ago and are now working on support, sales, onboarding and other processes.

</comment></comment><comment><USER6> I recently switched from Github to Gitlab (although only for private projects). In the past I'd used Bitbucket but I disliked the interface. Gitlab, on the other hand, is much better. I think overall I still prefer Github, but it's nice to have unlimited private repos and Gitlab is approaching being as good. It's a remarkably polished project.

<comment><USER2> Thanks for your kind words Tim! We are happy to offer everyone unlimited private repos on GitLab.com [https://about.gitlab.com/gitlab- com/](https://about.gitlab.com/gitlab-com/) What can we improve so you would prefer GitLab over GitHub?

<comment><USER6> Wait, really? Oh. I host it myself, I figured it'd cost for private repos at GitLab.com. Probably should have checked that :-)

Regarding improvements, I don't really have any specific examples. Just keep at it and I'm sure that, over time, you'll refine it until it's perfect. It's really great as it is.

<comment><USER2> OK, thanks for your response!

</comment></comment></comment></comment><comment><USER7> Another positive development with Gitlab EE is that its client-side JavaScript and all code that generates or is compiled into it is Expat-licensed:

      https://about.gitlab.com/2015/05/20/gitlab-gitorious-free-software/

<comment><USER2> Thanks to you Mike! <3

</comment></comment><comment><USER8> If someone forks CE and adds features missing in CE but present in EE, will GitLab accuse them of looking at the EE code and violating the license? Will that accusation stick?

It will be interesting to see.

BTW I hate the open core model. [http://en.wikipedia.org/wiki/Open_core](http://en.wikipedia.org/wiki/Open_core)

<comment><USER2> Open core is a double edge sword, but we try to enlarge the advantages (resources to do release management, fix bugs, upgrade dependencies, investigate security reports) and minimize the downsides (no artificial limits in CE, no crippleware).

Looking at the EE code is fine but you can't use the code. So merge requests that add EE code into CE will not be accepted.

<comment><USER9> The point being made above seems to be "Releasing the source for EE poisons dev efforts", because if I fork CE and add code that does something EE adds, GitLab might claim that I "stole" code from EE, or looked at EE and was tainted with knowledge from EE code.

Even if I hadn't copied code from EE, the claim could be made and would be destructive to the ability of community members to fork and improve the open source codebase

<comment><USER2> I expect that in most cases we'll be able to tell from the code. In cases where we can't tell we assume that people made it themselves. We are releasing the code to make EE easier to work with (having private repos and docker images is hard). We did not want to make it harder to contribute EE features to CE. If people are worried about poisoning please let me know. Merging EE functions into CE has always been at the discretion of the core team, we don't need any excuses.

<comment><USER10> That is super cool. Thank you for demonstrating that it's possible to be a CEO of a successful, community-friendly proprietary version-control company without turning into Larry McVoy.

<comment><USER2> Thanks Geoffrey, I appreciate the kind words.

</comment></comment></comment></comment></comment></comment><comment><USER11> While I am not against rapid UI evolution, as seems to be the case with GitLab, I think you may want to revise the choice of background vs text colour on the sidebar. It does look pretty, but the WCAG guidelines describe a minimum level of contrast for people with vision imparement to be able to read well. This side bar has a very low level of contrast, and putting the URL of a GitLab project into various WCAG evaluatioin tools will tell you the same.

Having said that, thank you for GitLab. Even though this post is about the Enterprise Edition, it is good to see some high quality open source alternatives to GitHub (i.e. the Community Version). We moved F-Droid there and have had mostly a very positive experience.

<comment><USER2> Thanks for moving F-Droid to GitLab.com, we appreciate it. You can change the colors of the sidebar on /profile/design I'm not sure if any other choices improve the contrast. Feel free to submit a design that meets WCAG guidelines.

</comment></comment><comment><USER12> Awesome! I really like using Gitlab CE. Are there any plans for more code review type enhancements? We typically do lots of discussion on merge requests and often times, certain code features or plans will get scheduled for a refactor sometime in the future. Though, all those comments kinda get hidden away after the branch is merged. Maybe what I should be doing is using the Wiki, but it would be nice to be able to leave comments on the HEAD (i.e. comments on current state of a project).

<comment><USER2> Good to hear you like GitLab. We don't have any plans for code review improvements at the moment. Your proposal sounds interesting. We normally try to refactor the code before merging the merge request. If there are any loose ends we make an issue out of it.

</comment></comment><comment><USER13> Something I consider for quite a while: Would it be possible to use GitLab as a 'frontend' only? With remote repositories?

I ask, because our company recently killed of a Gitorious instance that worked fine for ~5 years, to 'upgrade' us to TFS and there are quite some .. issues and limitations with that idea. Looking for a way to make IT happy ('yes, code is stored in TFS') and keep a usable/productive environment.

<comment><USER2> I'm sorry but that is not possible. You would have to use scripts to mirror the repos [https://github.com/samrocketman/gitlab- mirrors](https://github.com/samrocketman/gitlab-mirrors) but we don't recommend it.

<comment><USER13> Thanks a lot for the 'official' answer. Appreciated, not unexpected, but sad.

Actually that link DOES look quite interesting..

<comment><USER2> Don't do it! :)

<comment><USER13> Or else? ;-)

Frankly, I'm torn and might spin up a test instance on Monday (it's a day off here) to give it a try. Can't do jack about TFS hosting the sources, but anything that gives me a saner/better interface and usable issues etc. on top would be .. awesome.

<comment><USER2> OK, go for it :)

</comment></comment></comment></comment></comment></comment><comment><USER14> Lovely! Congrats Sytse, Dimitry and co! Will be upgrading next week.

As a long time user (nearly 3 years now), it really is amazing that you guys manage to release dead on the 22nd EVERY month and always with new features! Amazing.

<comment><AUTHOR> Thanks, that's great to hear.

Actually, our release cycle is open source as well[1], so you can see EXACTLY how we do it every month ;-)

[1] [https://gitlab.com/gitlab-org/gitlab- ce/blob/master/doc/rele...](https://gitlab.com/gitlab-org/gitlab- ce/blob/master/doc/release/monthly.md)

</comment></comment><comment><USER15> Curious: Does gitlab support really large files?

<comment><USER2> Yes, via Git Annex [https://about.gitlab.com/2015/02/17/gitlab-annex-solves- the-...](https://about.gitlab.com/2015/02/17/gitlab-annex-solves-the-problem- of-versioning-large-binaries-with-git/) (we'll soon release a patch release to fix a problem with it in 7.10.x)

<comment><USER16> But that's only in EE, right?

<comment><USER2> Yes, but feel free to contribute Git LFS to CE :)</comment></comment></comment></comment>


Why your 'A' grade SSL is 'outdated cryptography' on Chrome - <AUTHOR> https://certsimple.com/blog/chrome-outdated-cryptography <comment><USER2> (Author of SSL Labs here.) From Chrome's perspective, it's easy to complain about obsolete cryptography because it takes only its connections into account. When you take a wider view and include other browsers, all sites today effectively must use obsolete cryptography because client-side support for authenticated suites is not yet strong. Even Google would have a "uses obsolete crypto" warning because it negotiates CBC suites with, say, Safari.

Non-obsolete crypto is relatively easy to achieve with Chrome and Firefox, but not as much with IE and Safari, and many other clients.

There's no doubt that sites with authenticated crypto should get a better score, but it's not straightforward to establish fair criteria. For example, should SSL Labs penalise sites that don't use authenticated encryption with IE? Because, to do that, you must either accept the DHE key exchange or deploy with dual (RSA and ECDSA) keys. The former is slow, while the latter requires more effort.

You could say that SSL Labs should focus only on the cases where it is possible to use authenticated encryption, but that creates another problem: should SSL Labs report the reality, or award best effort? It's not easy to decide, but so far I've been leaning toward the reality.

Finally, there's the case of using ChaCha20/Poly1305; should SSL Labs encourage the use of cipher suites that are not yet standardised? Should Google get a pass because we "know" they know what they're doing?

<comment><USER3> I don't understand why AE ciphersuites can't just own the 'A' grade, and everything else be a 'B' and below. You are currently awarding an 'A' "for effort". You should stop, now. We're all adults here.

And yes: you should be encouraging Salsa/Poly1305. They're _de facto_ standards, and will very soon be _de jure_ standards as well. Plenty of other "nonstandard" behavior is encoded into TLS. It's also not fair to call Salsa/Poly1305 a "Google" initiative.

<comment><USER4> There are still many clients out there that do not support AE ciphersuites (e.g. literally every version of Safari). If SSL Labs capped the grade at B for supporting these clients, then the top grade at SSL Labs would effectively become a B, as no serious site would get an A grade anymore. It wouldn't actually accelerate the transition away from these clients/ciphers.

Making the effective top grade a B might also reduce the physiological motivation for improving a seriously bad TLS config. At least in the US, anything less than an A is considered very close to failing these days (it's silly, but it's the reality). Making changes just to get a B isn't nearly as motivational as making changes to get an A.

<comment><USER3> Those older clients are actually vulnerable to real cryptographic flaws in TLS.

<comment><USER4> Which flaw, specifically, would the latest version of Safari be vulnerable to?

<comment><USER3> I guess it depends on your confidence in the Lucky 13 patch, right?

<comment><USER4> Indeed.

And to be clear, I'm not saying we shouldn't be moving away from the CBC ciphers as fast as possible - just that dishing out a B grade for supporting a large number of users is not a good way of accomplishing that.

</comment></comment></comment></comment></comment></comment></comment><comment><USER5> I don't know if it's planned with the lets-encrypt tool but these things should be really be done automatically. It feels a bit like downloading manually .tar.gz files to do system updates instead of a standard update tool. A system update should update the list of recommended ciphers for OpenSSL and the webserver should just grab the list from OpenSSL directly, there should not be any need to modify any configuration file. It's really exhausting to keep-up with the current state of ciphers all the time. People are pointing out the cost of certificates but in reality, the complexity of all of this is the real barrier, it's far more complex than it should be.

<comment><USER6> Another way to think about it: if there's a use-case where the API OpenSSL provides (requiring that the consumer specify a list of cipher-suites, etc.) is wrong, then that consumer should not be directly consuming OpenSSL; that level of abstraction is the wrong fit for their project.

What we need is a "library" that just _hardcodes_ agreed-upon conventions on top of OpenSSL. Import that _instead of_ OpenSSL, use its API (which would be mostly transparent calls through to OpenSSL using the current best practices), and everything gets taken care of. New best practices? Updated library package.

<comment><AUTHOR> Mozilla Server Side TLS is that set of agreed upon conventions. The Moz logic is great, since the conventions produce 3 sets of cipher suites with different compatibility / security tradeoffs.

The idea of a library to implement that logic is solid: we made [https://www.npmjs.com/package/ssl-rsa- strength](https://www.npmjs.com/package/ssl-rsa-strength) as a library implementing Mozilla's logic a little while ago.

</comment></comment></comment><comment><USER7> I appreciate Google being aggressive in this front, but they've trained people, like my mom, to ignore the little green lock in the last 6 months.

<comment><USER8> It's worse than that. They've trained users to ignore the RED "X" on the little lock and the RED strikethrough on the "https" that Chrome shows for sites with SHA1-signed certs that expire after 2016[1]. Reasonable or not, some sites can't or won't update their certs and have no choice but to tell Chrome users to ignore those warnings.

[1] [http://googleonlinesecurity.blogspot.com/2014/09/gradually-s...](http://googleonlinesecurity.blogspot.com/2014/09/gradually- sunsetting-sha-1.html)

<comment><USER9> You don't have to ignore the warning. It means that what you see on the site _might_ have been recorded or modified in transit, and what you type in _might_ be recorded or modified on the way back to the server. You might decide that it's fine for browsing but decline to put in personal information on those sites. It's up to you.

<comment><USER10> The problem is that outside of our niche of people who really care about tech, most people just care about the contents of the webpages they visit. They aren't reading/understanding the warning and have no context for making that assessment of safety and responding appropriately. They just know that they want to go to their bank's site or play the latest Mafiaville or whatever.

These changes in Chrome are teaching them that the red marks on the location bar are just a normal part of those interactions.

</comment></comment></comment></comment><comment><USER2> For completeness, also be aware that it's possible to get (yellow and red) warnings in Chrome even if you're serving a full SHA2 certificate chain and have the best possible configuration otherwise. Apparently, this is because Chrome outsources certificate chain construction and validation (different library depending on the underlying operating system), and they sometimes create SHA1 paths even when better paths are valid.

My understanding is that this happens because of cross-signing, where some intermediates are using SHA1. Just in the last couple of days I had users complaining about two different sites with certificates from two CAs.

<comment><USER4> Indeed. Debian is one OS which is affected by this, because they shipped an outdated version of NSS (the crypto library used by Chrome) which does the suboptimal path generation.

Yesterday, the latest version of NSS was finally uploaded to Debian Unstable, fixing the problem there, but Debian Stable is still affected, and will be until it's updated either through a security update or a stable point release. I plan to agitate for this if necessary.

Here's the relevant Debian bug report: [https://bugs.debian.org/cgi- bin/bugreport.cgi?bug=774195](https://bugs.debian.org/cgi- bin/bugreport.cgi?bug=774195)

</comment></comment><comment><USER3> It really _shouldn 't_ be possible to get an SSL Labs 'A' with anything but GCM or Poly1305.

<comment><USER11> Doesn't that mean giving up old browsers though?

<comment><USER3> Are you the kind of person who sees a 'B' as a failing grade?

<comment><USER11> If getting an A requires things that are, for a large portion, not feasable then I consider that a failing grade, but it is the test that fails, not me.

Anyway I asked a factual question, I would like an answer.

<comment><USER3> The answer is "no".

</comment></comment></comment></comment></comment><comment><USER12> Our "C" rated SSL is "modern cryptography" on Chrome - because we offer ECDHE_RSA/AES_128_GCM - but because we also offer RC4, because eCommerce, and bloody winXP users still existing, and still being a source of revenue. They do win a "UPGRADE YOUR BROWSER, SUCKER, YOU ARE INSECURE!" banner, but we will have to keep RC4 available for as long as Windows XP continues to provide >1% of our clients' revenue.

<comment><USER13> You don't need to use RC4 to support windows xp with IE8 - you can even get away with using TLS1.0 and above. If you need to support IE6 though, then yeah, you're stuck :/

The cipher list below (for apache) should work just fine with XP/IE8 and later, but still net you an A+ in ssl labs, with a bit of other work.

SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA128:DHE- RSA-AES128-GCM-SHA384:DHE-RSA-AES128-GCM-SHA128:ECDHE-RSA-AES128-SHA384:ECDHE- RSA-AES128-SHA128:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA- AES128-SHA128:DHE-RSA-AES128-SHA128:DHE-RSA-AES128-SHA:DHE-RSA- AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM- SHA384:AES128-GCM- SHA128:AES128-SHA128:AES128-SHA128:AES128-SHA:AES128-SHA:DES- CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 SSLProtocol All -SSLv2 -SSLv3 SSLHonorCipherOrder On

</comment></comment><comment><USER14> Seriously, SSL is such a mess. I dutifully switched my product's website to SSL a year or so ago, and I can't count the amount of time I've spent trying to optimise my SSL settings, and trying to figure out what optimising them even means. I'm pretty technical, but like trying to run a correctly configured mail server, this is just too hard. So I end up not worrying about my grade and probably end up using old, obsolete encryption because of it.

We're going to lose this fight unless we can make this easier. I mean, I care and I should be capable of working this out, but I'm no security expert - it's just too much work and most of the information is totally incomprehensible to me so I end up not bothering. I'm sure I'm not alone.

<comment><USER15> I've had a lot of good experiences lately using SSLMate ([https://sslmate.com/](https://sslmate.com/)) to automatically buy, download, generate chains, and renew certificates.

Then, I use Mozilla's SSL configuration generator ([https://mozilla.github.io/server-side-tls/ssl-config- generat...](https://mozilla.github.io/server-side-tls/ssl-config-generator/)) to set up the cypher suite.

Those two together will have get you running very quickly. It's certainly a good idea to know what you're dealing with, but there are smart people behind both of these services that really simplify things.

</comment></comment><comment><USER16> This area is a mess, there is now a difference between all major browsers and even between Firefox versions (With the Developer version complaining where regular FF does not).

For example firefox mentions SHA-1 insecurity when visting the google jquery CDN. [1] Chrome doesn't seem to care however and there are other sites where it is vice-versa.

[1] [https://ajax.googleapis.com/ajax/libs/jquery/2.1.3/jquery.mi...](https://ajax.googleapis.com/ajax/libs/jquery/2.1.3/jquery.min.js)

<comment><USER17> It's frustrating that Chrome and Firefox don't give any details about why the connection to an insecure site fails. You just get a "connection reset" page in Chrome, and a generic SSL security error in Firefox.

I was deploying a virtual appliance and was baffled as to why I was unable to load the configuration web page from both Chrome and Firefox, until I visited the vendors web page and downloaded a hotfix that removed the SHA-1 certificate. This was an appliance released in 2015. It's a shame that the software vendor is releasing insecure virtual appliances, but it's also a shame that Google and Mozilla can't get their act together enough to display an informative error message.

How about "The site you are attempting to visit uses weak cryptographic algorithms, so the connection has been blocked."

And, please give advanced users a way to click through anyway. If this is a virtual appliance I'm deploying in an isolated network and I'm trying to access the administrative page, I should be allowed to get there, without some nanny state browser manufacturer determining that my crypto isn't safe enough.

<comment><USER18> > _And, please give advanced users a way to click through anyway._

This already exists, and you can find it by reading through the source of Chromium.

</comment></comment></comment><comment><AUTHOR> Author here. This actually came from a real case: it was fairly surprising the SSL Labs test wasn't flagging the ciphers that cause the 'outdated cryptography' warning.

<comment><USER19> Thanks for this blog post. I was attempting to fix up a server last week and couldn't find or figure out a fix for it at the time.

<comment><USER20> I think this is the experience of a lot of people and good security folks are taking notice to try and help: [https://blog.digicert.com/understanding-the- google-chrome-co...](https://blog.digicert.com/understanding-the-google- chrome-connection-tab/)

</comment></comment></comment><comment><USER13> I'd recommend checking out [https://cipherli.st](https://cipherli.st) for practical information on ciphers. It's quite easy to achieve an A+ by following their recommendations, once you understand the impact some of the changes might have (eg, using HSTS).

There's even a set on config for poor sysadmins who are forced to support legacy clients on XP.

Happy to answer questions as well - I do this for work.

</comment><comment><USER21> Reminds me of a trick used by creationist high school textbooks.

They cover (in little detail, and with some errors) evolutionary theory, but they say it's obsolete and then give you the flavor-of-the-month creationist justification as "what's current".

</comment><comment><USER22> While we are on the topic of A-grade security: The article might want to point out that using standard DHE is preferable to ECDHE if you can handle the additional CPU load. The problem with ECDHE is that all currently TLS-approved curves need to be considered unreliable. This will change if 'draft-irtf-cfrg- curves-02' is accepted and Curve25519 and Ed448-Goldilocks become part of the standard.

<comment><USER10> Can you clarify why the TLS-approved curves should be considered unreliable?

<comment><USER23> Mumble mumble NIST mumble.

<comment><USER10> That's the problem with tinfoil; no matter how much I wrap around my computer, I can never be sure if the government has suborned my local grocery store.

</comment></comment></comment></comment><comment><USER24> Very informative but not very educational. I had much more pleasure reading what they referenced: [https://wiki.mozilla.org/Security/Server_Side_TLS](https://wiki.mozilla.org/Security/Server_Side_TLS)

</comment><comment><USER25> Reading this article and the comments here I'm stuck by how complicated (at least non trivial) the whole thing is.

At the same time we are trying to push every website (including blogs, little- league clubs, joke sites) to https-only.

</comment><comment><USER26> Secure connection: fatal error (40) from server.

ECDHE or gtfo :(

</comment><comment><USER27> I recommend <[https://bettercrypto.org/>](https://bettercrypto.org/>).

</comment><comment><USER28> Chrome changing the rules again? Unpossible! Still, the good news is that people are leaving chrome for ff, safari, new ie, so soon, it will be less of a problem. Hurrah!

<comment><USER29> You don't think forward secrecy is critical to secure communication? I'd rank it as more important than the SHA1 weakness problems.</comment></comment>


The AWS and MongoDB Infrastructure of Parse - <AUTHOR> https://medium.com/baqend-blog/parse-is-gone-a-few-secrets-about-their-infrastructure-91b3ab2fcf71#.ctvj6t498 <comment><USER2> "our idea is that developers should know about details such as schemas and indexes (the Parse engineers strongly agreed in hindsight)."

Many engineers I have worked with like to throw around terms like: "CQRS", "Event sourcing", "no schema's", "document-based storage", "denormalize everything" and more. However, when pushed, I often see that they lack a basic understanding of DBMSes, and fill up this gap by basically running away from it. For 95% of the jobs, a simple, non-replicated (but backed-up) DBMS will do just fine. The remaining 3% might benefit from some sharding and the final 2% are for specialized databases, such as column-oriented DBMSes, document- oriented DBMSes or graph-based databases.

At this very moment, I'm working on a classical DBMS, searching through around 200 million tuples within the order of 2 seconds (partially stored columnar using psql array types and GIN indexes). Not for a second did I consider using any kind of specialized database, nor optimized algorithms, except for some application-level caching. Some problems are caused by DBMS to backend and backend-to-frontend network utilization, which is generally not something a specialized DBMS can solve.

<comment><USER3> You are assuming that 95% of the jobs will remain the same throughout their entire lifetimes. Which is not true and I'm sure many would prefer for a database not to become a liability in case they happen to attract a lot of users, which requires more availability than a classical DBMSes can provide.

The problem is more about constrains required to scale and achieve high- availabily and low-latency, rather than a database choice. But classical DBMSes lack those constrains and you may and probably will end up with a design that just cannot be scaled without removing and redesigning a lot of things that users rely on, and also with engineers that never learned how to do that, so you may end up in an urgent need of a distributed systems specialist too. Scary. You can avoid that by using systems designed for high- availability from the very beginning, like Riak. Same goes for blobs and filesystems, using POSIX API instead of an object storage is going to haunt you later on, because in your designs you may end up relying on things that ultimately cannot scale well and guaranty latency, as gitlab learned the hard way.

<comment><USER4> I'd say YAGNI applies here since 95% of systems will never need to scale beyond the capabilities of a properly-configured RDBMS. Ensuring consistency and durability with a non-relational, distributed datastore generally requires a lot more detailed understanding of its internals than with a RDBMS, so choosing one of those rather than a RDBMS is going to impose costs that should be deferred until they're absolutely necessary.

<comment><USER5> Plus it's waaay more likely that bugs in application code will haunt you even more. Especially the lack of things such as foreign keys and other constraints.

Schemas can be straight jackets, but in many situations they're actually closer to life jackets.

<comment><USER6> Totally agree that schemas are lifejackets. Carfully modelled data can outlive the application it was originally written for.

Poorly structured data, on the other hand, where the schema and consistency is enforced in code is much harder to migrate.

Maybe our problem is that we write systems with a disposable point of view? We don't even think the system will be around long enough to require migration.

</comment></comment></comment></comment></comment><comment><USER7> As an Urban Airship employee from 2010-2013 this is eerily familiar stuff. I feel like we should all have a big "we tried and failed to use mongodb as a distributed oltp database" support group.

One point of disagreement:

> However, they had an unspoken value system of not trusting their users to > deal with complex database and architectural problems.

At least for enterprise customers (who actually pay the bills) this is the correct value system. It's not that enterprise developers are bad. It's that enterprise development moves incredibly slowly, so the more freedom and options and implementation details you give them to work with the longer it will take them to be successful.

What you as a startup engineer may feel like is a powerful and expressive API just added 6-12 months to the development cycle of every enterprise customer... and thus many will give up on the integration.

</comment><comment><USER8> Parse's MongoDB deployment was definitely not the largest by a longshot. Not even top 10.

They had the largest number of databases on a single replica set for sure, but largest in terms of cluster size or data size? No way.

<comment><AUTHOR> Do you have a source for that? It is what the Parse engineers told me, but that kind of information is notoriously hard to corroborate. Any time I asked this question to MongoDB engineers they were evading concrete numbers for deployment sizes.

<comment><USER8> I can assure you that you are incorrect. I am the primary source and very aware of Parse's scale.

I worked on building a feature that allowed Parse to list that many databases (it had response that does not fit in the document size limit of 16mb -- requiring multiple cursors, command cursors).

I am a collaborator on the parse-server project, love the Parse team, respect what they built, and have no incentive to deceive you.

I worked at MongoDB for 4 years in various roles, consulting, engineering, etc and saw all sorts of deployments of various shapes and sizes.

<comment><AUTHOR> I do believe you, but can your share some more hard facts?

I've changed the passage to "one of the greatest". As in the disclaimer above the article, I'm mainly presenting assertions by Parse engineers, who were very sure about being the largest MongoDB user in terms of cluster size.

</comment></comment></comment></comment><comment><USER9> I would like to hear more about the introduction of LSM-based RocksDB storage into the Parse architecture. Specifically how long before they felt MMAP was no longer meeting their needs?

I was also curious about this? >"Frequent (daily) master reelections on AWS EC2. Rollback files were discarded and let to data loss."

Was the velocity of rollbacks sufficiently high enough that they were't able to process them?

I thought Parse was a great company with a great product that served an important market. I'm still not sure why FB acquired them only to retire it.

<comment><USER10> We had issues with the MMAPv1 storage engine from the beginning, mostly due to it's lack of document level locking and storage bloat.

One of the biggest benefits from being acquired by Facebook was working with the RocksDB team to come up with MongoDB +RocksDB, but this was also happening around the time when Parse was starting to wind down.

Being the only company running this new storage engine freaked us the fuck out. So we tried to blog and give talks as much as possible to get other folks interested and willing to test it out.

Parse now running MongoDB on RocksDB: [http://blog.parse.com/announcements/mongodb-rocksdb- parse/](http://blog.parse.com/announcements/mongodb-rocksdb-parse/)

Strata: Open Source Library for Efficient MongoDB Backup: [http://blog.parse.com/learn/engineering/strata-open- source-l...](http://blog.parse.com/learn/engineering/strata-open-source- library-for-efficient-mongodb-backups/)

MongoDB + RocksDB: Writing so Fast it Makes Your Head Spin: [http://blog.parse.com/learn/engineering/mongodb-rocksdb- writ...](http://blog.parse.com/learn/engineering/mongodb-rocksdb-writing-so- fast-it-makes-your-head-spin/)

MongoDB + RocksDB: Benchmark Setup & Compression: [http://blog.parse.com/learn/engineering/mongodb-rocksdb- benc...](http://blog.parse.com/learn/engineering/mongodb-rocksdb-benchmark- setup-compression/)

<comment><USER9> I thought you might say that. If you have an app that does a lot of deletes that bloat becomes noticeable quickly. I think they might actually have document level finally. Rocks is interesting as it deals with the write amplification problem and its tuneable. TokuMX was another interesting storage and had good compression. I would curious if you ever evaluated that.

Are you with Baqend now? Is this your medium post?

<comment><USER10> Before even looking into RocksDB we tried TokuMX and even got pretty well aquatinted with their dev team. We ran into the same issues when testing Wired Tiger. Neither could handle millions of mongo collections. Since we were involved in the RockDB storage engine from the beginning, we made sure the implementation could handle that many collections.

This isn't my post and I don't work at Baqend, but I will say that the author comes off as a presumptuous asshat.

<comment><USER9> Yeah this reads like this person at Baqend is going over "lessons learned" while at Parse. That's why I thought it might be someone from Parse. It seems like bad form to write a blog post on what other people should have done differently and use that to publicize your own company. Parse was a success.

<comment><AUTHOR> > So here are some facts and trivia that are not so well-known or published > that I collected by talking to Parse engineers that now work at Facebook. As > I am unsure about whether they were allowed to share this information, I > will not mention them by name.

It's both lessons learned by Parse engineers and us, so I think the intended ambiguity is okay.

<comment><USER9> >"It's both lessons learned by Parse engineers and us"

All of those bullet points are from Parse's history. Where is Baquend's contribution to the "lessons learned" in this blog post then?

</comment></comment></comment></comment></comment></comment></comment><comment><AUTHOR> Author of the post here. If you have additional key insights about the Parse infrastructure, please post them here and I will directly add them to the article.

<comment><USER11> What was the throughput (per server) after the Go rewrite? I'd imagine an async rewrite would be a lot more efficent than the 15-30 req/s that the Rails version was getting.

How often (if ever) did you experience data loss due to the MongoDB write concern = 1?

<comment><USER12> I was the lead for Parse Push. I reverse engineered Resque on Go so we could let any async stage be switched to Go cleanly. I added a few features like multiple workers per Redis connection and soft acks that let the next job be pulled by the framework but still kept the server alive during a soft shutdown until the job completed.

After moving the APNs (v2) server to Go I was able to map the wonky networking to Resque much better. My show & tell that week was literally "this MacBook is now running higher throughout than our cluster of >100 push servers". For APNs, the rewrite was a boon of over 500x per server.

As a great fringe benefit, fewer Resque servers meant less Redis polling. CPU dropped from 97% to <10 IIRC, which helped save us the next Black Friday where Redis falling over from Push load often caused problems.

I don't recall how often data loss was a problem. One of the points that hadn't gone into a lot of detail was when certain ops tricks happened. For example, I recall that reading from secondaries was reserved for larger push customers only. These queries could be wildly inefficient, have results in the 10s of millions, and due to mongo bugs would lock up an entire database if they had a geo component in the query. In later versions of Mongo where the bugs were fixed and after another generation of the auto indexer we were able to send all traffic back to primaries.

<comment><USER12> Oh. And our ResqueGo outperformed Ruby so badly that we couldn't do a reasonable X% server transition from Ruby to Go by rebalancing the number of workers. It had to be done my making a new Go queue and doing % rollout of which queue work was sent to. We learned that the hard way once (though luckily there was no regression IIRC).

</comment></comment></comment></comment><comment><USER13> I'm surprised vendor lock-in was not mentioned as a problem. I suppose it's a fundamental problem with offering "BaaS" and since the author is pushing his own BaaS offering, any critique of vendor lock-in may be unlikely.

I first used parse for a mobile app that grew to 600k users. I was totally against adopting parse, for a number of reasons including the unpredictable billing, failing background jobs, and the need for retry logic in every single write query. But my biggest issue was with vendor lock-in. When your app succeeds, and the backend is dependent on parse being in business, parse becomes a liability. Eventually you will need to take on a project to "migrate off parse." And you know what? That sucks for parse, because they were counting on power users to pay the big bills. But in reality, once a customer became a "power user," they started the process of migrating off parse.

When parse shutdown, I initially felt vindicated - ha! Vendor lock-in. But they handled the shut down and transition to open source _extremely_ well. As a result, parse-server is now an open sourced, self hosted product with all the benefits of Parse.com but without the vendor lock-in!

I've been using parse-server exclusively for new projects. I'm very happy with it and it is always improving (though backwards compatibility has been hit or miss... but that's the price of rapid releases). It's very easy to setup, and does a lot of crud for you (user registration, email confirmation, login, forgot password, sessions). You can call the API from an open source SDK in almost any language on any platform (note: I'm the maintainer of ParsePy). Also, because you have direct access to mongo, you can optimize query performance outside of parse. For example you can create unique indexes, where with parse you had to query for duplicate objects at application level. There's even initial support for Postgres instead of mongo. Also, the dashboard is nice for non-technical people on your team to understand your data structures and ask more informed questions.

I'm not sure I would ever use another BaaS. It just seems like such a dumb decision to offload the entire backend of your app to a proprietary service. If the service was totally open source from the beginning, with a self hosted option, then I would consider it. At least that eliminates the threat of vendor lock-in. I get the feeling that the excellent handling of parse.com shutdown was an exception to the rule. I don't want to take unnecessary risks with the most important code of a project.

<comment><USER14> I agree, the parse shutdown was organized extremely well. The open source parse server, one year of migration time and a ton of new vendors that now offer to host your parse app, all made it much easier to handle the shutdown. It's also great to see the community still working on the open source server.

That said, there are a lot of upsides to having a company work full-time on your proprietary cloud solution and ensure its quality and availability. If an open source project dies or becomes poorly maintained you are in trouble too. Your team might not have the capacity to maintain this complex project on top of their actual tasks.

Also open sourcing your platform is a big risk for a company. Take RethinkDB for example: Great database, outstanding team but without a working business model and most recently without a team working full time, it is doomed to die eventually.

Nevertheless, we try to make migrating from and to Baqend as smooth as possible. You can import and export all your data and schemas, your custom business logic is written in Node.js and can be executed everywhere. You can also download a community server edition (single server setup) to host it by yourself.

Still a lot of users even require proprietary solutions and the maintenance and support that comes with it. And often they have good reasons, from requiring a maintenance free platform to to warranties or license issues. After all, a lot of people are happy to lock into AWS even though solutions based on OpenStack, Eucalyptus etc. are available.

</comment></comment><comment><USER15> "applications were often inconsistent" \- I've heared this about parse before. Always thought this is due to using MongoDB. If you use the same database how can you enforce more consistency?

<comment><USER14> Although MongoDB has its limits regarding consistency, there are things that we do differently from parse to ensure consistency:

\- The first thing is that we do not read from slaves. Replicas are only used for fault tolerance as it's the default in MongoDB. This means you always get the newest object version from the server.

\- Our default update operation compares object versions and rejects writes if the object was updated concurrently. This ensures consistency for single object read-modify-write use cases. There is also an operation called "optimisticSave" the retries your updates until no concurrent modification comes in the way. This approach is called optimistic concurrency control. With forced updates, however, you can override whatever version is in the database, in this case, the last writer wins.

\- We also expose MongoDBs partial update operators to our clients ([https://docs.mongodb.com/manual/reference/operator/update/](https://docs.mongodb.com/manual/reference/operator/update/)). With this, one can increase counters, push items into arrays, add elements into sets and let MongoDB handle concurrent updates. With these operations, we do not have to rely on optimistic retries.

\- The last and most powerful tool we are currently working on is a mechanism for full ACID transactions on top of MongoDB. I've been working on this at Baqend for the last two years and also wrote my master thesis on it. It works roughly like this:

       1. The client starts the transaction, reads objects from the server (or even from the cache using our Bloom filter strategy) and buffers all writes locally.

       2. On transaction commit all read version and updated objects are sent to the server to be validated.

       3. The server validates the transaction and ensures the isolation using optimistic concurrency control. In essence, if there were concurrent updates, the transaction is aborted.

       4. Once the transaction is successfully validated, updates are persisted in MongoDB.

There is a lot more in the details to ensure isolation, recovery as well as scalability and also to make it work with our caching infrastructure. The implementation is currently in our testing stage. If you are interested in the technical details, this is my master thesis: [https://vsis-www.informatik.uni- hamburg.de/getDoc.php/thesis...](https://vsis-www.informatik.uni- hamburg.de/getDoc.php/thesis/868/Thesis_Erik_Witt.pdf)

</comment></comment><comment><USER16> would be intresting understing more about "Static pricing model measured in guaranteed requests per second did not work well." . what happened and what would have been a better solution afterwards.

<comment><AUTHOR> The core problem was that the sustained number of requests did not really cause any bottlenecks. Actual problems were:

\- At the Parse side: expensive database queries on the shared cluster that could not be optimized since developers had no control over indexing and sharding.

\- At the customer side: any load peaks (e.g. a front page story on hacker news) caused the Parse API to run into rate limits and drop your traffic.

<comment><USER17> I feel like for indexing it could be done dynamically, based on on performance analysis from performance and query profiling. This way you can also do it without really understanding the application logic at all.

<comment><USER14> That's actually exactly what parse did. They used a slow query log to automatically create up to 5 indexes per collection. Unfortunately this did not work that well especially for larger apps.

I guess 5 indexes might be a little short for some apps. On the other hand too many or too large indexes can get a bottleneck too. In essence, you want to be quite careful when choosing indexes for large applications.

Also some queries tend to get complicated and choosing the best indexes to speed up these queries can be extremely difficult especially if you want your algorithms to choose it automatically.

<comment><USER18> We created more than 5 indices per collection if necessary. But fundamentally some queries can't be indexed, and if you allow your customers to make unindexable queries, they'll run them. Think of queries with inequality as the primary predicate, or queries where an index can only satisfy one of the constraints like SELECT * FROM Foo WHERE x > ? ORDER BY y DESC LIMIT 100, etc.

<comment><USER14> That is absolutely right. You can easily write queries that can never be executed efficiently even with great indexing. Especially in MongoDB if you think about what people can do with the $where operator.

What would in retrospect be your preferred approach to prevent users from executing inefficient queries?

We are currently investigating whether deep reinforcement learning is a good approach for detecting slow queries and making them more efficient by trying different combinations of indices.

<comment><USER12> Fwiw, the Google model is to just cut unindexable queries from the feature set. You can only have one sort or range field in your query IIRC in DataStore

<comment><AUTHOR> The Google Datastore is built on Megastore. Megastore's data model is based on entity groups, that represent fine-grained, application-defined partitions (e.g. a user's message inbox). Transactions are supported per co-located entity group, each of which is mapped to a single row in BigTable that offers row-level atomicity. Transactions spanning multiple entity groups are not encouraged, as they require expensive two-phase commits. Megastore uses synchronous wide area replication. The replication protocol is based on Paxos consensus over positions in a shared write-ahead log.

The reason for the Datastore only allowing very limited queries is that they seek to target each query to an entity group in order to be efficient. Queries using the entity group are fast, auto-indexed and consistent. Global indexes, on the other hand, are explicitly defined and only eventually consistent (similar to DynamoDB). Any query on unindexed properties simply returns empty results and each query can only have one inequality condition [1].

[1] [https://cloud.google.com/datastore/docs/concepts/queries#ine...](https://cloud.google.com/datastore/docs/concepts/queries#inequality_filters_are_limited_to_at_most_one_property)

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER19> What is the reasoning behind not using sharding and homing each customer to a replica set?

How did you deal with customers growing large or spiking in usage? How did you manage such hot spots?

Sharding is one of the key benefits of MongoDB (and frankly most NoSQL solutions). Of course, you have to pick a good shard key.

One reason I can think of is that IMO sharding on MongoDB before version 2.4 had too many issues to be production reliable. If Parse started with MongoDB 2.2 or earlier, then I can see how they would avoid sharding.

</comment><comment><USER20> Interesting new take on baas with Baqend. Looks like gomix?

<comment><AUTHOR> There are definitely some similarities. Like in Gomix, the idea of Baqend is to let developers easily build websites, bots and APIs. However, Baqend has a stronger focus on how data is stored and queried, while Gomix is more focused on a rapid prototyping experience inside the browser.

In my opinion, the whole movement about BaaS is all about making things as smooth as possible for developers and shorten the time-to-market to its minimum. What some providers like Parse lost on the way, are the capabilities for small prototypes to grow to large systems. That requires being scalable at both the level of API servers, user-submitted backend code and the database system. And at some point, it also requires letting the user tune things like indices, shard keys and execution timeouts. This is the route we took at Baqend. We do not want to be the simplest solution out there, but we aim to provide the fastest (we use new web caching tricks) and most scalable one (we expose how the database is distributed).

</comment></comment><comment><USER21> Can I suggest you edit your title? I thought someone on our team had written this but, in this context, "lessons learned" is kind of misleading.

<comment><AUTHOR> Could you share what your lessons learned were? That would be a very interesting perspective. Do you disagree with some of the points?

</comment></comment><comment><USER18> Clash of Kings used Parse, not Clash of Clans.

<comment><AUTHOR> You're right of course, I just changed that.

</comment></comment><comment><USER22> If the database is the bottleneck, why are you using the same (MongoDB) database as Parse (and Firebase) were?

The problems described in the article are quite literally are my pitch deck, which I have successfully raised from billionaires Tim Draper and Marc Benioff of Salesforce, for [http://gun.js.org/](http://gun.js.org/) . So why did you decide to stick with MongoDB when many other databases have tackled solving those problems?

<comment><USER23> _var gun = Gun( '[https://gunjs.herokuapp.com/gun');](https://gunjs.herokuapp.com/gun'\);) _

_gun.put({hello: "world"}).key('random/WVNqluP4q'); _

and

 _Databases can be scary_

Who exactly are your target market?

<comment><USER22> People who are fed up about databases not working. That is primarily developers, but also happens to include some government clients as well.

<comment><USER23> If you are in this business professionally as some form of engineer, thinking that "databases are scary" is like your car mechanic thinking "engines are scary"</comment></comment></comment></comment>


Imagine a world without hunger, then make it happen with systems thinking - <AUTHOR> https://www.nature.com/articles/d41586-020-00086-5 <comment><AUTHOR> "According to systems thinking, changing the food system — or any other network — requires three things to happen. First, researchers need to identify all the players in that system; second, they must work out how they relate to each other; and third, they need to understand and quantify the impact of those relationships on each other and on those outside the system."</comment>


Are rich people more unethical? - <AUTHOR> http://www.cnn.com/2012/02/27/health/rich-more-unethical/index.html?hpt=hp_c2

<comment><USER2> Rich people that have never been poor can be, but not all of them.

I have a relation who was incredibly right-wing until the crisis hit him and his family real hard - financially speaking.

He wasn't bitter about it, most of the bad things happened because he was rich enough to be lazy about taking care of the important stuff - until it mattered, but of course that was too late. But he came to realise that Banks and the like are very friendly when you have money, and eager to kick you when your down, if you don't have money.</comment>


Dear Facebook: here's how to not screw up Instagram. - <AUTHOR> http://techland.time.com/2012/04/10/dear-facebook-heres-how-to-not-screw-up-instagram/

<comment><USER2> I definitely agree with keeping them separate and not confusing people into thinking they may/may not be connected. It's fine to maybe mesh a few features together from Facebook to Instagram or vise versa, but keeping them separate is a must.</comment>