Results for: Pick a number from 1 to 10 - <AUTHOR> http://nfrom1to10.appspot.com/results/ <comment><USER2> I just updated the results page with a breakdown per-interface. There are more metrics involved, but it'll be a little while before those get pulled together. (The volume of data was unexpected, to say the least, so there weren't any stats helpers in place prior to "launch".)

As for "revealing" the answers ahead of time, the whole question of "Pick a number from 1 to 10" and the common result being 7 is well known (ish), so I'm not too worried. Plus, the results aren't visible until you choose. PLUS, there were over 80k votes before that first graph was available, and over 140k by the time I put the per-interface breakdown up.

Hopefully, the volume will outweigh the people who try to manipulate it. (There's also IP tracking, which will eventually be used to help filter out anomalies, in addition to geolocating.)

I know, it's not exactly precise science, but it's a fun experiment that was simple and kept the friction to a minimum. I'm not exactly trying to write a paper. :)

Thanks to everyone who voted and spread the word, especially jedberg who posted it to Reddit.

</comment><comment><USER3> Can you do it again for 1 to 100? I'd like to improve my success probability for this <http://xkcd.com/628/>

<comment><USER4> I guess 42 will be a spike in the results - but it would be interesting to verify ;)

<comment><USER3> Are you implying than the average HNer or Redditer has a different bias wrt the average girl?

</comment></comment></comment><comment><USER5> It seems like publicizing some preliminary results of the study before the study is complete would put the legitimacy of the final results very much so at risk.

constructive criticism ... for science

<comment><USER2> After 80k votes, it seemed prudent to give those asking "so what?" a response.

<comment><USER5> If (as I'm sure you do) you have the times along with the votes it would be cool to see if there was actually a bias when comparing the before response and after response datasets.

</comment></comment></comment><comment><USER6> There was something about "for SCIENCE!" that was irresistible.

<comment><USER2> That bit of "flair" was added almost on a whim. I'm glad I did add it. I think that helped the experiment get the attention it did, especially when it got onto Reddit. It's funny how a simple factor can (probably) have such a big impact.

Also, the unintentional "mystery" of not including any information about the who or why on the site itself may have helped. I noticed many comments on HN and Reddit that speculated about each, which no doubt increased the activity profile. A little bit of personality goes a long way.

</comment></comment><comment><USER7> Was this only posted to HN? If it was posted on other sites it would neat to check the numbers against refers to see if HN is better at picking random numbers than, say, Reddit.

<comment><USER2> It ended up getting posted to Reddit, and has been making the rounds elsewhere. (Over 100,000 uniques, holy cow!)

Unfortunately, the referrer wasn't tracked. I wasn't expecting to get enough hits for it to matter. The expectation was for maybe 100 over a week. IP addresses are tracked, so I'll eventually do a location breakdown.

<comment><USER8> Assuming you don't delete them, why not use your webserver logs to link datestamp+IP address to referrer-url?

</comment></comment></comment><comment><USER9> The positioning of the dropdown clearly has a lot of influence on the results. People are horrible random number generators! In this case, people are unlikely to choose the ends (1/9) or the center (5) leading to a bimodal distribution with two modes at 4 and 7.

<comment><USER10> Note however that different types of input systems were used. Refreshes show the following different types:

      * A slider that need to be dragged to choose a value.
      * A list of numbers shown with highlighting on hover.
      * An input cursor that on hover changes to a list of numbers (within a curved border this time).
      * A text field (enclosed in a circular border), allowing manual input.

I'm quite interested in how results varied depending upon input type (possibly more interesting than seeing how they vary by referrer).

<comment><USER11> Well, why don't you have a look? :) <http://nfrom1to10.appspot.com/results/>

</comment></comment></comment><comment><USER12> I picked 3 because it's in the easiest number for me to strike (text box version).

My mouse is typically in the middle of the screen, closest to the higher- scoring numbers in the graphical version.

I'd like to see the results from the same test where you have to type the number as a word for the textbox (e.g., three), and where the numbers aren't laid out in the same numerical order for each user in the graphical version.

</comment><comment><USER13> Seems like people are so number centric and go for the universal lucky number. Not sure why. How did this help? Were you able to do any behavior analysis based on this small experiment?

I would suggest, go on a weekly result and keep tabs on result variation.

Try posting it on various/diverse communities. This might help to understand alot about visitor behavior on various websites as you can track referring websites too.

If possible graph the picks depending upon the referring websites. Or even a graph based on region, tracking via IP should be easy.

For those who missed the Parent post, check here <http://news.ycombinator.com/item?id=2375149>

</comment><comment><USER14> I picked 9. I feel good about doing my part to even out the distribution I predicted :)

</comment><comment><USER15> I thought people would think 10 was the least random number, so I picked it, :-)

I won! Yay.

</comment><comment><USER16> can you do this again but with an arbitrary range. I'm curious what happens when you don't have the lucky number effect (i.e. 7)

</comment><comment><USER17> You didn't count my 11 :(

<comment><USER14> Did you actually use firebug and submit an 11 to the server? That would be pretty funny.

<comment><USER17> Yep, sure did.

</comment></comment></comment><comment><USER18> The problem is that it didn't ask us to choose the numbers uniformly. Perhaps I picked from a Gaussian distribution with mean 5 and a strict cut-off at 1 and 10.

I don't think anything can be concluded unless it asked for "choose uniformly at random among the 10 numbers," if we want results robust against question biasing.

<comment><USER19> "Choose randomly" is generally understood to mean "choose randomly from a probability distribution containing as much entropy as possible given the problem." I mean, I could choose from the probability distribution {0.001, 0.001, 0.998} but it's commonly understood that this isn't what's meant by the word "random".

PS: I chose 2. Eat that, seveners!

</comment></comment><comment><USER20> Yay, I tried to guess and pick the number which I thought would probably be picked the least amount of people.

10 was a good guess.

</comment><comment><USER21> The results for 'select' seem strange. That's the only place where random specified/not makes much difference. It also indicates that a lot more people picked numbers under 'select/specified' than any other condition.

Have you checked your logs for possible vote stuffing?

<comment><USER2> Not yet, though IP addresses are one of the metrics for each vote. I'm still working on efficient ways of going through everything.

</comment></comment><comment><USER22> Is it wise to publish updated results while the data is still being gathered? Random selection should be blind and independent and its quite possible people may change their vote to favor an 'unpopular' number or vice versa.

</comment><comment><USER23> I think that the reason that 7 is so high is because it is commonly believed that 3 will is the first one you think of. As a result people choose to pick something higher, hence 4 and 7 being the top two.

<comment><USER24> I thought it was commonly believed that 7 is the most often picked - I've seen a "magic" trick based around that before.

Sort of looks to me like people avoid the extremes and also 5 (possibly because it's "right in the middle" and so perceived as "less random"?).

</comment></comment><comment><USER25> I'm surprised the number 1 wasn't higher up because of the slider (since it defaults to 1). I guess people aren't as lazy as I thought they were and actually took the effort to move the slider :P

</comment><comment><USER26> I don't get it. How is the selector relevant to what people choose? You might as well have put a different colored rabbit somewhere on the page and measured responses that way.

</comment><comment><USER27> I'm getting nothing but "1800".

<comment><USER2> Momentary stupidity on my part. I told it to cache the expire time instead of the actual value. Oops. Go figure, it was the _one time_ I think "Eh, this is a small change. I don't need to go to a testing version first."

</comment></comment><comment><USER28> Science has error bars.

<comment><USER2> Science takes time ;)

</comment></comment><comment><USER29> contrarian pride for 1 or 10.

</comment><comment><USER30> Glorious #2 Master Race.</comment>


Show HN: Public repo for data and code to build Malaysia Haze Watch - <AUTHOR> https://github.com/HazeWatchApp <comment><USER2> I'm not a developer in any way, but to what extent could any geographic pollution monitoring system be tied to a land-ownership registry? I would think openly publishing the names of the biggest offenders in semi-real-time could be at least minimally effective... and given the scale of the problem, that could still be quite helpful. Am I totally off here?</comment>


What is Technical Harassment?  - <AUTHOR>

In our complex technical environment there are many opportunities for a competent technical individual to be the subject of technical harassment. Sometimes it can be so subtle that you may not even be aware you are being harassed. Worse yet, you may inadvertently technically harass another person by accident.
Following are some guidelines to help you determine if you are being technically harassed.
If you are repeatedly asked the same technical question you may be the victim of technical harassment. While it is most common to be asked the question repeatedly within the same conversation, some instances have been identified of habitual technical harassment. Habitual technical harassment is not uncommon and has been known to exhibit group tendencies where members of a group may ask the same question repeatedly. Untreated, these instances of group technical harassment can continue for years.
If you are asked a technical question by a non-technical person and they do not write your answer down it is likely the question is frivolous. Most non-technical people are not capable of remembering a true technical answer for more than 30 seconds.
If you are forced into a discussion where a person uses more than three (3) buzzwords in one sentence the person is most likely a fake and you are the unwitting victim of technical harassment. One note of caution, competent technical people have been known to inadvertently use buzzwords after reading mindless drivel like PC Week or LAN Times. If the person has been known to use more common technical terms in the past such as "stuff" and "things", they are most likely victim of computer magazine brainwashing.
If during a troubleshooting session a person uses the term "trick". For example "maybe we could trick the database into thinking it has been updated". This is a sure sign of technical harassment.
If a person explains that a needed feature will be provided by a vendor and that person is nontechnical then you are at risk of being technically harassed. If you believe that person, you have definitely been technically harassed, if you don't believe them you have only been technically annoyed.
If when trying to resolve a technical problem with a product from a vendor and you are instructed to call the salesman that sold us the product you are being set up for technical harassment. It is a common reaction for a non-technical person when they have purchased technical equipment to call another non-technical person. The dialogue between two nontechnical people usually provides some sense of comfort that they aren't the only ones who are confused.

<comment><USER2> My favorite is...

I send an email out explaining that the staging server is staging.foo and the IP is 192.168.1.20 and that people should probably add this to their HOSTS file.

Then everyday when it comes time to test on the staging server, everyone is like "what's that IP again?" !?!

</comment><comment><USER3> I've had a great time reading this.</comment>


What are the Kyrgyz and Uzbeks fighting about, anyway? - <AUTHOR> http://www.slate.com/id/2256919/

<comment><USER2> Answer: the article's author has absolutely no clue, so he's going to incorrectly repeat some irrelevant, half-understood anecdotes he got from people in his Rolodex, and pull the reliable old hand-waving trick of blaming it on poverty without any concrete explaining why _these specific_ poor people are rioting at _this specific_ time. Gross nonsense like _The Uzbeks are phenotypically more similar to East Asians, while Kyrgyz tend to look more like Russians or Persians_ could have been fact-checked in about 30 seconds by looking up images on Google or Wikimedia Commons, but clearly no one bothered.

If you're curious about Central Asia, the two blogs below are good starting points:

<http://www.neweurasia.net/>

<http://www.registan.net/>

Neweurasia is written mostly in Russian or local languages, and then retranslated into English. They've been quite active on Twitter in circulating images and translating reports/tweets about what's going on in Osh. Registan is written in English by American grad students with academic interests in the region (and also tends to have more "meta" posts, fisking/bemoaning godawful mainstream media coverage of their region like this Slate article).</comment>


How Startups and a Government Agency Were Fooled by a Convicted Thief - <AUTHOR> http://muckraker.me/2012/01/03/the-masked-vc-how-ni-startups-and-a-government-agency-were-fooled-by-a-conman/ <comment><USER2> It's odd the author and the James written about both misuse "weary" for "wary", and the place James supposedly misuses it is in quotes. That makes me question the author's understanding of quotes.

It's more interesting the dispute was about alleged failure to disclose when the relevant paragraph of legalese seems to assess the termination fee for any reason at all:

"5.4 Should for any reason either party withdraw from the negotiations due to information that would hinder an investment being discovered by the Investors _or should the Investors withdraw for any reason after the date of this letter_ , the Investors shall be entitled to recover..."

"We decided we don't like the deal" is a reason. That phrase "for any reason" is the proverbial hole you can drive a truck through.

<comment><USER3> My heart does go out to this guy, but this is yet another case of "know what you're signing, or hire a lawyer who will."

The perpetrator in this case happened to be an actual fraudster-in- sheep's-clothing, but even among friendly business partners, contracts are drafted for the maximum benefit of the author. If you have any doubt whatsoever about whether a clause puts you in jeopardy, strike or rewrite it until you're 100% sure you're not getting fucked.

</comment></comment><comment><USER4> Looks like he trying to clean up incase his parole officer finds out (given the sentence reduction), anyone know what court originally convicted him in 2007?

</comment><comment><USER5> Post-YC, startup founders are pretty young. Would you care if a founder had gotten into legal trouble for a business started before their 18th birthday? I don't know -- given that YC looks for 'naughtiness' that might even be a positive.

Just to be clear: I am not talking about myself, or anyone that I personally know. I know of a case kind of like this, though.

<comment><USER6> Depending on your jurisdiction, it is generally held to be impossible for persons under the age of majority (usually 18) to enter into contracts for non-essential items (food, clothes, rent etc).

That's why most contracts are actually made with a _parent_ on behalf of the child.

<http://en.wikipedia.org/wiki/Age_of_majority> <http://en.wikipedia.org/wiki/Capacity_(law)>

(IANAL, TINLA).</comment></comment>


Ask HN: What was your biggest failure in 2019? - <AUTHOR> Let's stop listing our incredible achievements from the year that has passed.<p>Instead, let's discuss our biggest failures, what we didn't do, what we launched that flopped, and so on. <comment><USER2> Not being able to overcome the fact that a lot of people are technically better than me, after 10 years in the field.

You know how bad people feel after they go on Facebook and look at how beautiful their friends' lifestyle is, without realizing it's all fake? Well, that's how I routinely feel when I browse HN, except that the work on HN is not fake, is way too real.

I see people posting brilliant work, work that is very much related to my field and that I should be able to come up with myself, and instead I barely comprehend.

It then gets worse: I navigate to their personal blog, and find hundreds of incredibly well-crafted articles about a wide range of topics. For example, just today from a front HN link I jumped to a personal blog of an engineer who was writing impressively deep thoughts about unit testing, dependency injection, Go, Java, hardware, architecture, backend, frontend, performance, ... Very, very thoughtful posts. I would not be able to reflect so deeply on those topics, even if I am relatively familiar with them. I have no other choice than believing I am stupid, at least in comparison to such person.

It then gets worse one more time: I look them up on LinkedIn, infer their age and see that they are younger than me, and that they were holding a demanding full time job all along, while writing those incredible articles and producing that amazing work on the side.

And it's not even that I could work harder. I literally cut 90% of my recreational activities: I don't have a TV, I don't watch Netflix, Youtube, games, I completely ignore news, I don't read novels, I don't have kids to take care of, ...

Every day after work I read technical books/articles until I go to bed. The only things I do non-related to my professional development are going to the gym regularly, and spending quality time with my girlfriend a few hours on weekends. But, I would say that on a given week I routinely put 70-80 hours on my professional development (including my full time job), so I couldn't do more than this without it affecting my mental and physical health.

<comment><USER3> Is it imposter syndrome? Or just being unwilling to come to terms with the fact that other people will inevitably be better than you?

<comment><USER2> Fair point, I changed the wording. Thanks.

<comment><USER3> You should try to make peace with it. It's no way to live always being upset someone is younger and smarter than you. Keep working hard, but do it for yourself.

<comment><AUTHOR> I am, myself, still struggling with how un-smart I am. Not stupid, but just so much to learn from people I meet (in real life, or online) yet so little spare time.

<comment><USER4> Look at it this way. Your abilities/worth/knowledge/whatever are multidimensional and can be mapped on a "Radar Chart"([https://en.wikipedia.org/wiki/Radar_chart](https://en.wikipedia.org/wiki/Radar_chart)). They can also be classified as "Inherent" or "Incidental/Accidental". Thus there can be no single "smartness" index defining a person.

Thus for example, i am "un-smart" on both Web Dev and Statistics/Probability axes. The former is Incidental and easily fixed in a couple of months of work but the later is Inherent and needs a lot more time for assimilation. Thus the same word "un-smart" actually conveys two different meanings in the two different axes and one should not conflate the two.

</comment></comment></comment></comment></comment></comment><comment><USER5> My biggest failure in 2019 was getting utterly trapped in analysis paralysis.

Throughout the year, I explored many different side project ideas but I rarely took any action on most of them.

I didn't ask people for feedback on the ideas. I didn't cold email potential customers to see what they thought. I didn't build much.

Instead, I got stuck in a "research" mindset and consumed endless information on new industries or business models, often eventually talking myself out of most of my ideas -- without having any direct data on whether or not something would have worked out.

I think a more productive approach to 2020 will be defining specific actions or experiments I can run on new ideas, rather than remaining purely in the theoretical realm.

It's moderately depressing to look back at how little I _really_ learned. Few of my theories ever made contact with reality.

<comment><USER6> Been there. I realized doing something was better than planning most of the time in the initial stage. Most ideas or things you will come across have already been done, so trying to find something original isn't a solid strategy. If it's entirely new/original, then it will come to you naturally. You will discover it because you are the consumer facing the problem.

Otherwise, focusing on getting feedback and people to talk to you, get them to send their email (even though I personally hate the practice of collecting private info without any product and sending 'spam') is important to maintain their investment along with a simple sketchy prototype to show off. It's easier to give feedback on something that you can see or interact with than written details. Showing vision is better than explaining it even if it is completely basic. Your users will flesh out the product for you, they will tell you what features you can add, how it should work and how you can actually do it.

All of that wouldn't happen without a basic structure. Without seeing a shitty painting, I wouldn't comment on it.

I wonder how depressed marketers and other similar people feel for manipulating others and hogging their personal life with irrelevant things.....such as getting someone's dad addicted to a gambling or social platform and that resulting in negligence.

</comment></comment><comment><USER7> Getting let go from FAANG basically because I didn't ignore huge red flags while getting hired (thought bc it was a FAANG there wouldn't be atrocious mgmt policies etc).

The requirements for the position as the recruiter described was Go / Elixir experience, did my code screen / take home in elixir and nailed it, principle engineer reviewed my code and green-lit my application mostly bc he was also a devout elixir fan. My manager had a language barrier and seemed pushy when I did my on-site and told her I wanted to use the normal allotment of time for the take-home (they fast tracked my app which would've only given me two days for a take-home that is usually give 5 days). Learned the team wasn't doing any Go / Elixir (was "maybe going to transition at some point" and never did) and was basically just servicing a shitty monolith built in Ruby.

Lots of $$$ so I just decided to take the offer, since my prior situation was a startup that was trying to avoid paying me my full salary and boss didn't "believe" in benefits.

Team at FAANG was also socially inept enough to always send my comments up to my boss instead of team member meant to onboard me - first task was also horribly scoped and made me look like an idiot for double checking my work (small fix with HUGE implications and side-effects if done wrong, with flaky tests galore). Boss berated me for being the only person who would tell her I wasn't going to be done at the end of the day, but never caused a critical problem in prod. Basically got fucked when I admitted a mistake that I promptly fixed and re-deployed, proving their BS "nobody gets blamed for fuckups" _policy_ was definitely bullshit.

I got paid $20k to leave as my team was let go for unrelated reasons (you can guess which FAANG) and did some work with a friend on a startup for a few months. The transition out / lack of luck searching for new jobs really kicked me in the stomach and confidence - and on top of that I'm still looking for work. But hey, hopefully 2020 goes better.

<comment> T3OU-736 Dogpiling onto the well-wishing. Having recently been through a bit of interviews at various places, can't help but wonder - how does (and is it even possible) a candidate defect this toxicity or the interview stage? With the 20/20 hindsight, were there any tell-tale signs?

<comment><USER7> I have very limited experience, but the few that jump out to me emanate from a few conversations I had with employees during my on-site. For context, the role was for a "Golang / Elixir Engineer" during my onsite all anyone talked about was relatively new code written in Ruby and only mentioned Golang or Elixir in a "maybe 8 months down the road" way. Language barrier with your manager seems innocuous, but the problems I ran into were deeper and seemingly cultural. (she expected to only be told things were going right, nothing was ever her fault, asking for help is indicative of weakness or incompetence etc. - I'm trying to not sound racist but other employees had similar comments)

Also, it took me quite a bit of time to work through the mental gymnastics as to whether I was just bad at my job or if I was in a toxic environment. In hindsight it was definitely toxic - as an example senior engineers on my teams were being put in a position where they had to make decisions TPM's should be making (but still hung out when they made the wrong decisions).

<comment> T3OU-736 So, nothing _really_ jumped out, though there were weak clues/hints. I am not a coder, so it isn't obvious to me that a mention of a technology being "8 months down the road" is unreasonable. But now it would be.

Language (and, it appears, also a _culture_ ) barrier with management is... tricky since we don't want to come off as racist/*-ist. Not sure how to evaluate it, aside from asking scenario questions if the manager. Maybe if there manager, during their interview, didn't care about finding the candidate's behavior when they reached their knowledge limits is another sign?

</comment></comment></comment></comment><comment><USER8> Ignoring my personal relationships because I put too much time and focus on reaching the goals for my side business. I did reach the goals I set out to do in the beginning of 2019, it gave me hope to move forward with this project and potentially work on it full time. But I often canceled weekend plans with my boyfriend, ignored things he wanted to do, because frankly the idea of being able to build my own brand is an idea I'm so excited for and I just can't wait to prove that I can reach my goal.

I learn that relationships, just like businesses or work, the more commitment you put in, the more result will show. Also, it's important to balance work and life and make expectations clear.

</comment><comment><USER9> Like another comenter, I'd say mine was "not starting". More specifically writing. I often get ideas about articles I'd like to write but I just end up talking myself out of it.

This past Sunday morning - like countless others - I woke up with an idea comparing JavaScript ES5/ES6 to vanilla ice of various quality and decided to just post it in all its shitty glory on Twitter. It didn't go viral and I didn't gain any fame from it... but I wasn't the only one that found it funny (my biggest fear crushed) so it's a start, I guess.

</comment><comment><USER10> Took over responsibility for some shell scripting glue. Task: migrate to new servers. "How hard can it be?" 2 months later I am still completely lost. I have random input/output files going everywhere, complete flustercuck of perl dependencies, and ftp processes I can't test lest I bork prod.

Edit: and even if I "succeed" I'm pretty sure it needs a total rewrite. I think I'm performing a cross-org table join using ftp and text files with undocumented time dependencies.

</comment><comment><USER11> My biggest failure for 2019 is not starting. Overanalyzing ideas to the point that I forget about them and then doing that same process again on another idea. So my new year resolution is going to be 12 startup mvp's in 12 months!

<comment><AUTHOR> Oh yes, that's a tough one. Procrastination is a lethal mind weapon.

</comment></comment><comment><USER12> Rewrote a non-trivial app from Vue to svelte/sapper. It was a success until we discovered that it turns out Svelte handles errors by freezing instead of letting you show an error page and/or report it to Sentry.

</comment><comment><USER13> I have difficulty in letting go of engineering problems. My mind is always racing for a fix.. I have been in danger of burnout several times because of this. 2020 I need to learn to chill a bit more.

</comment><comment><USER14> I would suggest that rather than just listing things that didn't work out, people also discuss what they learned from the experience and how they would act differently if they had a chance to do it all over.

</comment><comment><USER15> Accepting a job without doing more research on the company and the actual job but of impatience. 11 months down the drain.

</comment><comment><USER16> Lack of movement forward.

Looks good on paper (got a promotion) but I can feel I just drifted with the current.

</comment><comment><USER17> Flunked out of a really good sales job I wasn't qualified for, and proceeded to get fired from an Amazon warehouse because my sleep problem still eludes medicine/psychiatry/homeopathy/jungian therapy. Still unemployed, won't be able to make 15/hr again.

Still dwelling on past relationship, failing at current.

Still stuck at home.

2020 is the year, though. "This time" I'll get around to the language learning and applications.</comment>


Show HN: Open-Source Business Intelligence for BigQuery – Looker Alternative - <AUTHOR> https://mprove.io <comment><USER2> @akalitenya are you even in the clear with this? Some of the Old LookML syntax is an exact copy.

But more importantly, the challenge for any such tool is to go beyond use by 2-3 people. At 2-3 people anything will work. Where BI tools (open source and close source) struggle is scale: having all the right features for, essentially, a group of users who actually don't know how to work with data (did I just say that aloud?). Chartio caps at 20 people. RJ capped at 50-100 (and later became Stitch for that reason). We haven't seen where Metabase caps, but I bet it is in a similar range. Very few BI products have actually surpassed 100 users at target installations. And beyond 1,000 is a real challenge that only few, and even then with a lot of assistance, can support: Tableau, Looker, Microstrategy, maybe Birst, maybe Domo.

Also, a combination of BI with LookML is a complicated product. During my days at Looker, we were handling 50+ bugs / week, and filing 1,000+ tickets. Every day we were filing over a 100 new features.

So with all that, the question is, is it really worth the struggle? What's the end vision for supporting this? Why should someone who implements BI for a living bet on this product?

<comment><AUTHOR> > Some of the Old LookML syntax is an exact copy.

I met the online Looker demo a few years ago when I was looking for a business intelligence tool for another project.

Looker has a closed source code, so I did not see what algorithm is used to build queries.

I kept those LookML features that I understood and liked. However, in some places LookML is confusing (for example - references and aliases). I made them differently.

Later Looker quit using YAML.

> Very few BI products have actually surpassed 100 users at target > installations. And beyond 1,000 is a real challenge that only few, and even > then with a lot of assistance, can support: Tableau, Looker, Microstrategy, > maybe Birst, maybe Domo.

Scaling that size is not a top priority right now.

> Also, a combination of BI with LookML is a complicated product.

You should have told me this a couple of years ago. It seemed pretty simple.

> So with all that, the question is, is it really worth the struggle?

Yes, If people will use it.

> What's the end vision for supporting this?

In future - maybe "skinny" or "thin" option mentioned here - [https://medium.com/open-consensus/2-open-core-definition- exa...](https://medium.com/open-consensus/2-open-core-definition-examples- tradeoffs-e4d0c044da7c)

>Why should someone who implements BI for a living bet on this product?

If you can not afford Looker (like me) and want to use similar product.

<comment><USER3> I used Looker for years and have always wanted a product priced less for the enterprise and more for developers. I even prototyped a similar tool myself.

Congratulations on the launch. I will be using mprove and will give you feedback along the way.

EDIT: I would love to get in touch with you and learn more about what your plans are. Possibly collaborate if you'd be open to that. I'm firstname.lastname@example.com -- I couldn't find your contact info anywhere!

<comment><AUTHOR> Thank you, I sent you an email. I am open to any suggestions - firstname.lastname@example.com.

</comment></comment></comment></comment><comment><USER4> The headlining feature seems to be that it has a dark and a light theme. This isn't very encouraging for the rest of the product. I'm sure there's a lot of good stuff here, but themes aren't important enough to be the first feature mentioned.

<comment><USER5> +1 it should not be the first feature listed, bump that down

</comment></comment><comment><USER6> this is super cool and i would pay for this. Bigquery is a cheap alternative to a lot of the mobile analytics tools.

Quick point however - why do you need a new database ? You can use a table inside bigquery itself. It seriously reduces the dependencies required.

<comment><AUTHOR> Mprove creates permanent derived tables in BigQuery if the user wants it.

But you can't use BigQuery for OLTP.

<comment><USER6> No - I'm talking about MySQL being needed as a dependency.

[https://github.com/mprove- io/mprove/blob/master/deploy/docke...](https://github.com/mprove- io/mprove/blob/master/deploy/docker/ce-prod/docker-compose.yml)

Can you not use biquery database itself. Create tables for your internal use instead of MySQL ?

<comment><AUTHOR> The MySQL image specified in the docker-compose file you mentioned. It is used for the internal data of the Mprove application (users, projects, members, etc.). Each user action in the web client (angular) can initiate several queries to this database through an backend request. Delays here are crucial. network latency - so you need to keep the database as close as possible to your server side, read / write delays - for queries such as finding a user / setting a username / creating a member, etc.

<comment><USER6> Hmm..I would prefer to not have it. In production, managing database persistence is very hard. Especially when you go down the kubernetes road.

I would take higher latency, but avoid pulling in a whole database infrastructure.

Plus a huge number of us use postgresql..so that becomes another set of a mess. I would strongly urge you to do this on the same bigquery database that you would connect to anyways.

<comment><AUTHOR> Bigquery exists only in Google Cloud. This is a columnar database like Redshift. It is not designed for fast processing small queries that are necessary to support the operation of any application. Its main feature is that it can scale the execution of heavy analytical queries between 10,000 nodes transparently for the end user. Users have no control over Bigquery instances. This is super cost effective cloud analytical database as a service to be used as centralized data warehouse for company of any size.

</comment></comment></comment></comment></comment></comment><comment><USER7> BigQuery PM here. Nice work!

Here's a recently built Graphite connector as well:

[https://twitter.com/vadimska/status/1112816503055843330](https://twitter.com/vadimska/status/1112816503055843330)

</comment><comment><USER8> Will this handle (repeatable) "record" (struct/array) data types natively?

/disclaimer: work in google cloud

<comment><AUTHOR> Yes it should, there is special "unnest" parameter for fields in BlockML reference - [https://mprove.io/docs/blockml/fields/dimension](https://mprove.io/docs/blockml/fields/dimension)

<comment><USER2> wow, impressive! how about symmetric aggregates (e.g. being able to do correct summation/aggregation on numeric values despite a one_to_many join)?

<comment><AUTHOR> Yes, look at this page - [https://mprove.io/docs/blockml/fields/measure](https://mprove.io/docs/blockml/fields/measure). You need to specify measure "type" and "sql_key" that will be used to avoid counting duplicates.

<comment><USER9> How do the underlying queries look for symmetric aggregates? Sadly SQL never supported the ability to compute an aggregate based on the unique values of another column.

<comment><AUTHOR> Mprove does it the way the Looker did it before:

       CREATE TEMPORARY FUNCTION mprove_array_sum(ar ARRAY<STRING>) AS
      ((SELECT SUM(CAST(REGEXP_EXTRACT(val, '\\|\\|(\\-?\\d+(?:.\\d+)?)$') AS FLOAT64)) FROM UNNEST(ar) as val));

...

       SELECT COALESCE(mprove_array_sum(ARRAY_AGG(DISTINCT CONCAT(CONCAT(CAST(a.id AS STRING), '||'), CAST(a.population AS STRING)))), 0) as a_cohort_size

Recently, Looker began to do it differently, most likely to improve bigquery performance:

      COALESCE(ROUND(COALESCE(CAST( ( SUM(DISTINCT (CAST(ROUND(COALESCE(lesson_5_cohorts.population ,0)*(1/1000*1.0), 9) AS NUMERIC) + (cast(cast(concat('0x', substr(to_hex(md5(CAST(lesson_5_cohorts.id  AS STRING))), 1, 15)) as int64) as numeric) * 4294967296 + cast(cast(concat('0x', substr(to_hex(md5(CAST(lesson_5_cohorts.id  AS STRING))), 16, 8)) as int64) as numeric)) * 0.000000001 )) - SUM(DISTINCT (cast(cast(concat('0x', substr(to_hex(md5(CAST(lesson_5_cohorts.id  AS STRING))), 1, 15)) as int64) as numeric) * 4294967296 + cast(cast(concat('0x', substr(to_hex(md5(CAST(lesson_5_cohorts.id  AS STRING))), 16, 8)) as int64) as numeric)) * 0.000000001) )  / (1/1000*1.0) AS FLOAT64), 0), 6), 0) AS lesson_5_cohorts_m_sum_distinct

</comment></comment></comment></comment></comment></comment><comment><USER10> Link to the open source part?

I can't seem to find any on the site...

<comment><USER11> There's a github link at the top of the page:

[https://github.com/mprove-io/mprove](https://github.com/mprove-io/mprove)

<comment><USER10> Oh, had to rotate my phone, does not show up in the menu on a smaller screen layout.

<comment><USER12> Wow, very different menus on mobile based on orientation. Vertical has "Pricing" and no open source mention. Horizontal gives a GitHub link and no mention of pricing.

I assume there was a time when this was meant to be a non-open source project.

<comment><AUTHOR> thanks, i just fixed it

</comment></comment></comment></comment></comment><comment><USER13> Do you support on-prem?

<comment><AUTHOR> Source code is open. Anyone can deploy Mprove to his server and use it for free.</comment></comment>


How to create an anonymous website? - <AUTHOR>

Hi,
This is a throwaway account for obvious reasons.
I'm looking to create an anonymous site, which will need hosting and domain registration. The site will only need static files served from it. All ideally with no trace back to me. And that won't be pulled by an ISP at the first letter from a lawyer. Does anyone know an easy and inexpensive way to do this?
Why?
I'm in the UK and some months ago signed up to a course in a non tech related field. Fees were required in advance so I did the usual searches and visited the premises before paying a sum of around £10,000.
Following the "start" of the course it very quickly became very clear there was literally nothing backing them up and no chance of actually learning anything or gaining a qualification. Through a series of calls, emails and letters I tried to resolve this and am currently undergoing legal proceedings to try and recoup some of the fees.
I have recently discovered there are at least several dozen people who feel they have been duped by the same company. Two years ago, through legal channels, they had them shut down and the owner blacklisted from running a company in the UK. They resurfaced a year later with the previous owner's son as the new director and have since begun re-advertising under a different name.
As they are a small course (with high fees) it would be very easy to get high in the search results for anyone searching for their name(s).
>>
I do not want to post anything slanderous or untrue online, just a warning to other prospective students that there is a very high dissatisfaction rate with previous students (possibly over 50%), with examples of the problems faced by some.
I would prefer to remain anonymous for the time being at least until other legal action has gone its course.
Thanks <comment><USER2> just remember that when you register a domain, to get private registration. Also make sure its a new domain, if you already registered it with your real info, changing it to private won't help, your old info can still be found.

Also there are a few extra things you need to do, to protect yourself:

a) dont' use the same Google Analytics and Google Adsense account on that website as on your other sites.

b) When you answer emails for that site, always do so after accessing the email service through a proxy.

c) Make up a fake name for your persona for that site. Make it be completely irrelevant to any of your other identities. Make sure the name is very common, but something less less obvious than John Smith.

</comment><comment><USER3> Have you looked at NearlyFreeSpeech: <https://www.nearlyfreespeech.net/about/>

<comment><AUTHOR> Thankyou - that is exactly the service I'm looking for - my google-fu is low today :)

I'll leave the post open rather than delete, just in case anyone else happens upon it.

Thanks again

</comment></comment><comment><USER4> Would it be possible to use something like <http://www.wikileaks.com/> to expose the fraud?

wikileaks has held up against some tough opponents.

</comment><comment><USER5> TOR hidden service - then publish the onion address with tor2web.com

A bit involved, but about as secure as it gets these days.</comment>


[pdf] Ninth Circuit Decision on AT&T Throttling - <AUTHOR> http://www.commlawmonitor.com/wp-content/uploads/sites/512/2016/08/ATT-Ninth-Circuit-Decision-Throttling.pdf <comment><AUTHOR> Brian Fung's piece covering the decision [https://www.washingtonpost.com/news/the- switch/wp/2016/08/31...](https://www.washingtonpost.com/news/the- switch/wp/2016/08/31/how-the-worlds-biggest-tech-companies-could-wriggle-out- of-all-privacy-regulations/)</comment>


Sometimes, Iowa Democrats award caucus delegates with a coin flip - <AUTHOR> http://www.desmoinesregister.com/story/news/elections/presidential/caucus/2016/02/02/sometimes-iowa-democrats-award-caucus-delegates-coin-flip/79680342/ <comment><USER2> The Democrats qsmmin a party favor for themselves years ago to ensure a constant inner city voters, republicans and independents alike know that this is due to the democrats handing out billions of welfare recipients, government assistance programs. I'm a republican and I'm sick of seeing single mothers all over the inner cities of this great country, abusing the help people get should me going just to the needy. know how to work the system for free money. chose to have 4 or 5 in D.C, handing out more money than checking to see</comment>


Queer [video] - mkr-hn https://www.youtube.com/watch?v=5Hi6j2UXEZM <comment> mkr-hn The video has a star emoji in the title for a reason discussed in the video, but HN stripped it out. Just to make it clear I didn't editorialize the title.</comment>


Ask HN: Y Combinator 2020 summer – who else has applied? - 100-xyz Go ahead and introduce your company and any other details you feel comfortable with such as how many times applied before, stage of company ... <comment><AUTHOR> [http://www.AppearMe.com](http://www.AppearMe.com) \- flat fee on-demand legal services for startups.

<comment> 100-xyz I tried to open the site and its very slow. Took about 20 seconds to open.

</comment></comment><comment> 100-xyz We are Toonclip.com It lets users create animations in minutes. Our first time applying. We also participated in Startupschool.</comment>


Declan McCullagh on Pressure-CookerGate - <AUTHOR> https://plus.google.com/112961607570158342254/posts/FWAVRVaN64h <comment><USER2> An M-66 is a perfectly legal firecracker sold in all states where firecrackers are legal, and purchased by what is probably millions of people over the years.

[http://www.fireworksforever.com/products- page/firecracker/m-...](http://www.fireworksforever.com/products- page/firecracker/m-66/)

Only $11.59 for a box of 36.

The author of this inane hit piece Declan McCullagh, gives his title as "Chief Political Correspondent" at CNET, and an employee of CBS. He posts that Ms. Catalano posted "photos of explosives."

Are M-66 explosives? Yep. Firecrackers are in fact explosives. They are also legal to purchase, possess, and light.

How is posting a stock photo of these firecrackers to your Facebook feed on July 4th relevant to anything at all? CNET Chief Political Correspondent Declan McCullagh certainly is implying that posting such a stock photo gives the JTTF, whose members are generally coordinated under and paid by the FBI, the right to search your house and ask questions about pressure cookers which you happen to have recently done a google search for. Why does CNET Chief Political Correspondent Declan McCullagh believe this is justified or that anyone would consider posting a stock photo of firecrackers on the 4th of July to be something anyone would reasonably expect should lead to an armed house search and interrogation by JTTF members?

What does this stock firecracker photo posted on July 4th have to do with JTTF agents searching this couple's house and asking them specific questions about whether they owned a pressure cooker?

<comment><AUTHOR> Jesus. He's not saying the visit was a good idea. In fact, he's said the opposite. Character assassination is bad enough, but to do it for absolutely no reason at all?

The fact is that you have absolutely no idea what happened, or even what some of the words in your comment (like "JTTF") actually mean. But you've got strong opinions and a lot of outrage. Because you're an outrage tourist. I wouldn't be surprised if you'd be angrier to find out that the FBI HADN'T been dragnetting Google.

<comment><USER3> He's also saying the fireworks were the reason the police visited. Which seems unlikely.

<comment><USER4> Seems significantly more likely than what was claimed in the original story:

1\. The NSA using monitoring technology against civilians in the country, against their charter. 2\. County sheriffs deputies who are members of the Joint Terrorism Task Force obtaining this classified information from the NSA 3\. Aforementioned county sheriffs accidentally divulging said classified data

I'd listen to an argument that points 1 or 3 occurred. Point 2? Way more likely that somebody read facebook than interagency communication of classified data from a classified program to a local law enforcement officer with no security clearance.

<comment><AUTHOR> We don't even know that the police looked at the page; for all we know, someone called this in to the police.

<comment><AUTHOR> Hey, guess what? Someone called this into the police. I'm shocked.

[https://news.ycombinator.com/item?id=6144198](https://news.ycombinator.com/item?id=6144198)

<comment><USER4> Absolutely stunning that Occam's Razor won out.

<comment><AUTHOR> Occam's Razor is pre-Snowden thinking. Like 9/11, Snowden changed everything.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER5> I knew Michele a long time ago when she blogged regularly about politics on A Small Victory (asmallvictory.net). Back then she was very sensationalistic, and would go out of her way to artificially inflate or dramatize stories. I'm not surprised she drew the conclusion that the visit was due to Google searches; I also wouldn't be surprised if she intentionally inflated the story, she has always had a talent for getting attention.

Also worth noting. She was a pretty loud blogger post 9/11 and during the Iraq invasion. She ran a very popular Iraq invasion blog, lauded for updating about the invasion faster than the mainstream media. It would not surprise me in the least if the Feds have her on _some_ list or another (not saying that's ok mind you). The Washington Post article about this talks about whether she'd be on a watch list, and Michele talks about searching for how to cook lentils and ending up on a watch list ("This is where we are at," Catalono wrote. "Where you have no expectation of privacy. Where trying to learn how to cook some lentils could possibly land you on a watch list."). That part is Michele knowing exactly how to craft a sound bite for media consumption. I suspect this story is demonstrating a real abuse by the terrorism obsessed police / feds, and the rest is Michele throwing fuel on the fire for the headlines.

</comment><comment><USER6> It's unlikely that Google searches led to this. Most searches are done over HTTPS by default these days. Plus it was local law enforcement, and while the feds may share leads, it's unlikely info gleaned from an illegal top secret NSA program targeting citizens would be exposed to those citizens in this manner 100 times a week.

A county police "task force" monitoring local citizen Facebook posts is a MUCH more plausible explanation. Still disturbing, but the reporting on this so far has been terrible.

</comment><comment> grey-area If this was the cause, why were they questioned about pressure cookers and not fireworks?

There's now a guardian article up about this with contradictory responses from the FBI and local police.

[http://www.theguardian.com/world/2013/aug/01/new-york- police...](http://www.theguardian.com/world/2013/aug/01/new-york-police- terrorism-pressure-cooker)

I find it unlikely that the JTTF don't have access to prism and other tools, so that seems a more likely avenue to me; searching for pressure cookers could lead to all sorts of non-https pages on amazon for example. After all that's exactly what these tools are designed to do - dig whatever the operators consider flags for terrorism this week out of the haystack of Internet traffic.

<comment><USER7> Alas, this only shows you don't understand what PRISM is. (Hint: it's not used by local cops! Sigh.)

<comment><USER8> >this only shows you don't understand what PRISM is. (Hint: it's not used by local cops

You're making two big assumptions here:

1\. PRISM is the only secret program with access to web browsing data.

2\. People with access to PRISM and similar programs can't pass low-priority leads to local cops.

<comment><USER7> I'm not making those assumptions.

</comment></comment></comment></comment><comment><USER9> In case you were wondering, that stock picture of fireworks was posted on July 4th.

<comment><USER10> Ahh but it is exactly when we are celebrating our superior freedoms and democracy (which the whole world is naturally jealous of) that the terrorists will attempt to strike a cowardly blow at those very same things we are celebrating. Therefore any and all expressions of said celebration are veeery suspect indeed.

</comment></comment><comment><USER11> I have a feeling most people who are up in arms about this haven't been visited by the police because they live in a bad neighborhood and "there were reports of X" happening in their apartment. When their neighbor was wrong, or the dispatcher had the wrong number, or whatever.

<comment><USER12> Or because they got tired on a long drive and pulled over on the side of the road to sleep. Waking up to a spotlight, then a gun and flashlight in the hands of a bored cop on the night shift is a tense situation.

</comment></comment><comment><USER13> TL/DR: Lazy and irresponsible journalist makes wild and unsubstantiated claim in the process of pointing out wild and unsubstantiated claims by other lazy and irresponsible journalists.

</comment><comment><USER14> First of all, it looks like a stock photo. And they are clearly labled with "Consumer Fireworks" and "Made in China"

<comment><AUTHOR> It's an entirely stupid reason for a police visit. It could have been a film of someone setting an actual bomb, and it would still be an entirely stupid reason for a police visit.

</comment></comment><comment><USER15> "Instead of drawing the most likely conclusion, she instead blames this on local Long Island cops MONITORING HER GOOGLE SEARCHES."

Ah...I get it...that would never happen in the US.

</comment><comment><USER16> Did neither Declan nor tptacek read the original article? This post doesn't explain anything.

<comment><USER17> Exactly. This article doesn't explain why they questioned her specifically about pressure cookers.

<comment><USER7> As I said in the G+ comments: "I would be surprised if cops on routine visits about explosives did not ask about pressure cookers. The Boston bombing wasn't that long ago."

</comment></comment></comment><comment><USER18> Nice, now it makes the whole thing even worse, we have police scaring people because their 4th July fireworks.

</comment><comment><USER19> _Instead of drawing the most likely conclusion, she instead blames this on local Long Island cops MONITORING HER GOOGLE SEARCHES._

Any hard sources on this besides this guy blabbing off and making more drama?

<comment><USER10> This M. L. Hunt person in his comments writes a very reasonable response:

"+Declan McCullagh In her account she mentions the cops specifically asking her husband if they have a pressure cooker. Assuming that is true, where do you think they got that idea from? Besides, how many millions of people do you think posted pictures of fireworks, even relatively powerful ones like M-66s, around 04July? What do you think made Ms. Catalano's photo(s?) particularly worthy of follow-up?

It seems to me you are bending over backwards to make this story seem less worrisome than it is. But like others who have replied to your post, I don't find a whole lot of comfort in your alternate scenario even if it turns out to be more in line with what actually happened.

But anyway, you're a journalist, aren't you? Do you have any plans to try to get to the bottom of what actually happened here? Or do you just plan to stand on the sidelines and try to pooh-pooh the whole thing?"

To which good ol' Declan responds: "Alas, I'm working on an unrelated story today."

Well that settles that.

<comment><USER6> Asking about pressure cookers is probably a standard question these days, for obvious reasons. It doesn't mean they were monitoring Google searches.

<comment><USER17> Bullshit. A pressure cooker is probably one of the most common kitchen accessories ever. Anyone who planned to use one for malicious intent would just deny that they have one.

<comment><USER20> cops also ask you if you have drugs in your car.

<comment><USER10> I'm sorry I can't tell if your response is written in a joking manner or if you are serious but while reading it I got this funny visual of a cop responding to a routine noise disturbance by asking the offending homeowner "any pressure cookers in the house?"

<comment><USER20> i am not joking

</comment></comment></comment></comment></comment></comment></comment><comment><USER21> This is in reference to HN item - [https://news.ycombinator.com/item?id=6140545](https://news.ycombinator.com/item?id=6140545) right? So is the question why are police visiting her house about fireworks, or why she thought they connected her pressure cooker and backpack Google searches? Or is the question about why are news establishments doing little to no fact checking before they publish something?

</comment><comment><USER22> It's sounding more and more like the cops were just following up on a "see something, say something" report from the public.

</comment><comment><USER23> A picture of consumer-grade fireworks posted on July 4??

<comment><USER24> Celebrates July 4? must be a terrorist.

</comment></comment><comment><USER25> That's quite disturbing. Ok, it was posted on a social media, which may not be perceived as needing as much quality work than a press paper, etc.

But how could a journalist be such on the affirmative without presenting any source ? We can't even see the facebook post he's talking about. The guy reacts to a shitstorm by providing unsourced opinion with a well established media name to link a cop operation with a stock firework picture supposed to be posted in USA on 4th.

That's at least lacking rigorous work.

<comment><USER7> Yes, it's lacking "rigorous" work. It's a G+ post, which is almost by definition not "rigorous." Sheesh. Would you like a refund on your subscription fees?

<comment><USER25> Thanks, but I'm not subscribed. Actually, I wouldn't mind if it was not making HN home.

And yes, I think with your reputation, you're expected not to have weak public opinion. That's a responsibility which come with social weight, sorry about that (and to be clear : I'm ok with you having unsourced opinions, just not when it implies influencing people).

<comment><USER7> Um, Google+ doesn't have subscription fees. It was a joke.

<comment><USER25> But it has subscription via circles. It was disarming the joke.

</comment></comment></comment></comment></comment><comment><USER7> Thanks, tptacek, for posting this. Though now I'm participating here and it's taking time away from my surveillance article for tomorrow! :(

</comment><comment><USER26> Hmm, not a bad hypothesis as far as completely baseless speculation is concerned.</comment>


When I got fired from my own company - <AUTHOR> http://marcbarros.com/when-i-got-fired-from-my-own-company

<comment><USER2> It takes a lot of courage to share something like this publicly. The social dynamics in our community ensure that most of these stories never get told. Huge props to Marc for being so transparent and open.</comment>


How Did Amazon Become So Much More Valuable Than eBay? - <AUTHOR> https://capitalandgrowth.org/questions/1676/how-did-amazon-become-so-much-more-valuable-than-e.html?childToView=1677#answer-1677 <comment><USER2> Amazon directly competes with 3rd-party sellers. I used to sell various products on Amazon about 10 years ago and as soon as I started seeing a decent amount of sales, A listing would appear by Amazon (not a 3rd-party seller) and undercut me. My sales would soon disappear after that.

They clearly use your sales data to make money and directly compete with you. This combined with their atrocious support for sellers has turned me off to their marketplace completely.

Ebay, on the other hand, doesn't do this and I think this is why Amazon is able to make so much more money.</comment>


TieSight - Mac App uses iSight to teach you how to tie a tie - <AUTHOR> http://tiesight.com/EN/TieSight_EN/Welcome.html <comment><USER2> Definitely something I could use. Having to visit Youtube for that once or twice a year where I have to wear a tie? Yeah, it's a pain.</comment>


Dark Algorithms and UX in Health Insurance - <AUTHOR> https://jb-rubinovitz.ghost.io/dark-ux-and-health-insurance/ <comment><USER2> This is a classic case of attributing the results of stupidity to malice. Insurance companies are _heavily_ regulated and subject to massive fines and public embarrassment for things like deliberately preventing the insured from filing claims. And this sort of petty detail in the UI, being directed from C-level executive management? It doesn't pass the smell test. They're managers, not supervillains.

No, this is just garden-variety incompetence. This is the result of incredibly complex specs being implemented by teams of dozens or hundreds, including offshore work and highly distributed teams. The kind of evil plots suggested here would be passing through not just executives, but architects, product owners, scrum masters, lead engineers, and the delightfully unpredictable hands of the cheapest programmers money can buy. _Dozens_ of opportunities for sabotage of something so overtly evil.

I've worked in insurance IT quite a bit. They're just not smart enough to do this.

<comment><USER3> >This is the result of incredibly complex specs being implemented by teams of dozens or hundreds, including offshore work and highly distributed teams. The kind of evil plots suggested here would be passing through not just executives, but architects, product owners, scrum masters, lead engineers, and the delightfully unpredictable hands of the cheapest programmers money can buy.

Perhaps that is the evil plan working as intended

<comment><USER4> Yeah I mean the fact is that revenue comes from people not collecting benefits.

<comment><USER2> Nope. The vast majority of benefits are processed without incident or complexity, between the medical provider and the insurer, with no action required by the insured. Rigging some UI for hand-entry of claims in a way that suppresses only some, but not all? That's not going to make an appreciable difference in the bottom line.

<comment><USER4> I'm not sure what the "nope" is in response to: Insurance companies profit when people don't collect on their benefits. If I pay and don't use it, they win. This is why companies that self-insure take out further insurance on their pool in case someone does actually get very sick and it drains the pool, which does happen more often than you'd think.

</comment></comment></comment></comment></comment><comment><USER5> This post lists three examples of dark algs/UX.

However, the second and third examples are speculative. They might be a result of an algorithm, or they might be a result of stupid policies, or they might be a result of operational errors. Who knows? Alleging that it's a "dark algorithm" sounds cool, but it isn't really damning without evidence.

So we're down to one example of actual dark UX, which doesn't make for a particularly strong allegation. I wish the author had taken just a little more time before publishing to collect a few other authentic examples.

That said, I know from my own experience that health insurers' online portals tend to have horrible UX, because it is not in their best interest to have great UX. Just yesterday, I had to fill out a form indicating that none of the five members of my household had other insurance or were Medicaid-eligible. There were a total of 10 checkboxes I had to check. After checking each one, I got a loading indicator and then the page refreshed, so I had to scroll the page back down to the next checkbox. It didn't prevent me from filling in the information, but it definitely made it needlessly tedious.

<comment><USER6> It's further concerning that he appears to have written his own example code for the speculative dark patterns, presenting it alongside real code from insurer websites, which I found misleading at best and downright deceptive at worst...

<comment><USER7> And then, like, criticize the code quality it as if it came from them for design choices that _the author_ made?

> So, my laugh cry scenario here is that they statistically set/just have > variables in the code base that say claims spanning a certain amount of time > are denied. E.g.

WTF, you wrote that code!

<comment><USER8> He is pointing out the arbitrary nature of the algorithm the workings of which was determined through research. The psuedo code is obviously not the point. The reader would have to either be ignorant of the discussion to make this mistake or intent on blurring the discussion.

</comment></comment></comment></comment><comment><USER9> Im disappointed to confirm these are quite real. I've seen a particular case where a benefits enrollment page has a check box list where your selected enrollments at the top of the page _silently change as you edit fields near the bottom_. And you can bet which direction those values change. It doesnt hide a submission button like the author's example but it DOES prey on those who neglect to review their submission in full before continuing (which I guarantee is a significant and quantifiable benefit to the payer)

It's a big stretch of the imgaination to dismiss this behavior as 'designer error'. This is a great example of where class action lawsuits can and should be employed.

</comment><comment><USER10> Insurance companies give me 90 days to submit a claim, then they do not have to pay it. I guess their rationale is that if I did not bother to submit an invoice, then why should they pay? And by the way, a lot of them will not accept paper claims any more. They want all the claims done electronically through their ancient systems.

And by the way, this push for electronic medical records(EMR) was a way for them to further decrease their overhead (thank you insurance lobbies). EMR that is CCHIT certified (= Certification Commission for Health Information Technology, pronounounced 's-h-i-t', for real) can interface with payment clearing houses directly (I as a physician have to pay to use a clearing house, btw). Then the claims are scrubbed by their scripts and rejected for various reasons, (for example, no referral from PCP, invalid gender for treatment code,etc). No humans involved. This allows them to lay off vast swaths of auditors. And effectively outsources their work onto the physician's office. And increases their earnings per share.

Now you know why the only new physicians you will meet are not homegrown anymore. It costs 400k to go to medical school. Then you have 3-7 years of postgraduate training at 35k a year. Then your debt has ballooned to 800k with deferred interest payments. BTW, new physicians get paid 90-100k.

<comment><USER11> The new physicians making $90k a year after residency are not the ones with $400,000 in student loans, and $800k in total student interest load is... atypical, to say the least. $300-400k total load post-residency is much more - I refuse to say "reasonable," but much more realistic. And the folks who have $400k total load are on a fast track to make $250-300k/yr just a few years after residency. A huge part of why doctors are paid as well as they are is because of the risk and the massive opportunity cost of getting an MD and going into practice.

Yes, it takes planning and dedication to set yourself up on solid financial footing post-medical school, and it's entirely conceivable to be in your early thirties before you're debt free. But it's pretty damn hard to be struggling financially as a physician in any field.

<comment><USER10> there is no 'reason' as to why physicians are paid what they are paid. It's simply supply and demand. A typical physician has to care for 5000 patients a year (that is the size of a full panel, btw) and gets 150k a year. Do the math - 20 patients a day, 5 days a week for 50 weeks a year is 5000 new visits. BTW, phone calls are not billable hourz. Oh, and I forgot 'emergencies' like aunt jemima who has severe burning when she pees, etc and needs to be seen ASAP or else she's going to the ER. Now add in the follow ups and emergencies and you're up to 35 patients a day. If you work for 8 hours (no lunch or breaks) that's 4 patients an hour. But wait, you have to enter patient info into an emr while you're supposed to be talking to the patient. Maybe you need to sip some water and go to the bathroom too. So that's 10 min a patient. for 150k a year.

Instead, most MDs are looking to become 'boutique' docs. Each patient pays $2k a year for as much primary care as he/she wants. The boutique doc limits his panel to 500 patients. He makes 6x the money for 1/10 the work. now the doctor can spend an hour with each patient and treat him like a human being instead of a slab of meat. And cut the expensive insurance middleman out of the picture.

So right, who needs the stress and bs that comes with a corporate existence. Why should the people who do the work (programmers and coders in your case perhaps) support the wasted busy work of people who are corporate drones and never actually interface with customers/patients/clients/aka the people who pay the money.

</comment></comment></comment><comment><USER12> Not a dark UX, but another move I've seen that is line with the stall-for-time method that the author mentions is the requirement for a prior authorization, even for very common medicines. I have seen first-hand the amount of work this tactic creates in medical offices. It definitely allows insurance companies to stall in their approval/payment for medicines and, I'm sure in a % of cases, not pay at all (ie, the paperwork doesn't get completed, patient gives up trying to navigate the complexity of the system).

<comment><USER13> My most hated is "The thing your doctor did was not medically necessary, therefore we won't cover it."

</comment></comment><comment><USER14> Never attribute to maliciousness that which can be explained with incompetence.

Since two of the three are complete speculation, without any evidence, I'll address the first example.

"suppress empty div" could mean that it is hiding an element _until a condition in the form is met_. You need to see the accompanying JS to understand any intent. It's entirely possible that bad programming is keeping the div both empty and suppressed.

<comment><USER15> But developer incompetence and poor product design can be a 'dark' strategy as well, just a passive one instead of an active one. Good UX doesn't matter because users are not customers to health insurers, they are just cost centers. Their employers are the customers, and those guys care about how cheaply they can buy insurance and retain the quality level of their desired employee base. If the goal is to sell lots of insurance and pay little in claims, the strategy makes sense. So you spend as little as possible on UX.

NB: I think this is also how enterprise tech works.

<comment><USER14> Insurance companies don't think of "members" as just cost centers. They're a lot more picky around the concept of "hiring and firing" customers.

Put it in the perspective of a freelance web developer. If someone is going to pay you x amount of dollars to build a website, but cost you more than x in support time and change requests, you either have to up the price or not take on the customer.

</comment></comment></comment><comment><USER16> Let's forget ethics for a moment, is there any decent way to calculate the difference in bottom line between creating and providing a great service (presumably leading to a good reputation, and maybe better, more long-lasting relationships with customers), and squeezing and weaseling every penny out of your customers and doing everything in your power to keep them from using the service they bought (presumably leading to bad reputation, but more cash upfront)?

In some industries/companies/communities/what-have-you, there seems to be this consensus that exploitation, scamming and backstabbing is the obvious way to make the big bucks. Is this objectively true, or is it just some kind of oral history and tradition handed down from 1800s industrialists?

<comment><USER17> I think for most insurers it's more profitable to provide less service and preventative care for two specific reasons. One, once a person is 65 they are covered by Medicare and are the government's problem. Two, if someone under 65 does get seriously sick or injured, chances are they're going to have to leave their job or get fired. With most people getting insurance through work, the insurance company isn't on the hook anymore either.

<comment><USER18> The ACA required most health plans to provide free preventive health services.

[https://www.healthcare.gov/coverage/preventive-care- benefits...](https://www.healthcare.gov/coverage/preventive-care-benefits/)

</comment></comment></comment><comment><USER19> Meanwhile, these insurance companies are making billions of dollars of profit ([https://www.axios.com/profits-are-booming-at-health- insuranc...](https://www.axios.com/profits-are-booming-at-health-insurance- companies-1513302495-18f3710a-c0b4-4ce3-8b7f-894a755e6679.html)). There's no reason insurance companies need to be involved in health care at all.

<comment><USER20> > There's no reason insurance companies need to be involved in health care at > all.

As a Brit I sort of agree, because I think free healthcare at point of delivery is the right solution. However the more continental model works perfectly well and many countries use private insurers for that sort of model. Insurance companies have a lot of expertise in risk management, analysis and costing so to claim that they've no place in healthcare is odd.

<comment><USER2> Getting far afield here, but... there's something uniquely broken about the financial model of the US system. Pretty much every industrialized nation in the world pays about half as much per capita as the US does for its health care, and pretty much all of them provide universal coverage. And this is _everyone_ , from purely socialist systems to mostly private systems.

Which is why I have issues with both the right-wing insistence that nothing is wrong with ours that a little more free market can't cure, and the left-wing insistence that only a purely socialist single payer system will provide financial responsibility and social justice.

</comment></comment></comment><comment><USER21> My spidey sense in law has to ask myself, "I wonder if that's HIPAA compliant" lol. Should just call it the Dark HIPAA requirements and un-UX in Health Insurance.

Laughing aside, the real data pinch in HI is the meta data of the meta data. Once you understand the abstracted layer there, the rest is following EDI compliance contracts and creating views appropriate to the end user.

It's a slippery slope.

</comment><comment><USER22> I worked with health insurance as a software developer (in Brazil so maybe it's all different, but still) and here are my observations.

1\. Making claim submission impossible

He said he tried to submit a claim at the last minute. I believe the hidden page was just lazy work, because the page has to be visible again soon. If they hid the page at an appropiate time I don't know.

2\. Routinely denying claim submissions

As far as I can tell, in Brazil a regulated health insurance plan can't deny you service based on how many claims of a certain kind you submited. They could before.

But the industry will find other ways to not lose money. Readjust prices based on number claims (from all users), prevent you from entering the plan and making a claim right away, pay for a quota of every procedure you take.

All these mechanisms already existed, but if they can't deny claims anymore they will tweak the other points.

3\. Saying the customer has other insurance

This is fucked up. I can't find a explanation that isn't malicious.

Here is one similar case that happens in Brazil though. If you have health insurance and use the free health insurance (SUS). The Union will charge back your plan, not you. It's their job to sort it out not yours.

<comment><USER23> comparing brazil with the US on health insurance and worker rigths is a joke, you know that, rigth?

even with the latest illegitimate corrupt government killing most of the work laws last couple months, its still way ahead of the US.

<comment><USER22> I know but I still think the points 1 and 2 are pretty much the same for both countries.

</comment></comment></comment><comment><USER24> United Healthcare... no surprised!

</comment><comment><USER25> beat's post is correct. This is straight up incompetence and just another symptom of how bad things are in Health Care Software right now. To give more background I'll tell you that I work in a company that builds software for major state-wide US insurance companies. Our software touches hundreds of thousands of lives on a daily basis.

The problems you're seeing may not even be a part of our software even though our software will do the heavy lifting. Each Health Care customer hosts their own product, although in recent times this is changing thank god. This means they maintain their infrastructure apply patches, manage their network, do their own provisioning, etc, etc.

This also means if a manager has the bright idea to $save money$ and build their own in house customer facing claim submission portal we don't know about it. How can we open our software up to work with their shitty portal if we don't even know it exists?

It may be harsh to assume the portal is shitty and has bugs like you experienced in item 1 but it really is a safe guess. Most of these US based companies are staffed by workers in India. These workers in India have a long list of obstructions preventing them for doing their job properly: a) 8am on the US Eastern seaboard is 6:30pm in Noida, India. So they're all basically working the night shift.

b) Lack of strong labor laws or fair compensation. High turnover in a critical IT field is bad, bad, bad. I've gotten calls from people in India who are trying to find out who their manager is in the US. I don't know bro I'm just a vendor :(.

c) A handful of non-technical, non-healthcare business folks managing a team of people on the other side of the world is a recipe for pain and failure. Here's a great example of the mysterious cause behind "Rising Medical Care Costs".

d) Lack of training and proper communication leads to horrible awful undocumented software abominations. I am borderline willing to blame my balding on these troubleshooting calls.

What I'm trying to make clear is that the failure may be between some shitty interface Insurance Co. #1234 built to save money. If that's the case it'll never really get fixed as the people managing it may not even be aware of the bug. If all goes well and the interface to our Insurance software is actually correct then "Hooray!" your customer facing bug gets to someone like me!

Now we've entered the realm of a US based software company. But our service that processes this information is 12 years old, the ONE guy who built it in the 90s is long gone, and there's no documentation. So good luck getting someone to take ownership of that. That's assuming I can even reproduce the bug. Reproducing the bug may be an odyssey wholly unto itself. It might take 3 developers and 1 QA engineer that has 20 years of seniority to figure out why logging isn't working for this 12 year old service.

So that's your dark pattern. It is more of business pattern but its a pattern that has so far caught every god damn piece of Health Care software in its net.</comment>


Ask HN: How do you invest? - <AUTHOR> We invest a lot of time in learning new skills and building new things. But what do hackers do for financial investments?
I know we all want our next product to make it big. How do we stay financially responsible in the long run?
Do you seek out a financial advisor? Stash cash in savings accounts? Buy GOOG and AMZN stocks? Save money for a house? Eat ramen? <comment><USER2> Personally, I invest in individual stocks like Starbucks (companies you think won't be going away anytime soon). I've tried to think like Warren Buffet. If you're investing in a company like Gillette, you're asking yourself, "Are men going to stop shaving?"

That being said, I also put money in more conservative investments like mutual funds. You can usually set it and forget it. With the markets doing pretty good the last two years, I think I was able to grow at 10-15%.

I also try not to spend too much of my time reading up on this myself. I pay Motley Fool for investment advice and follow their thesis. They have a pretty good track record and they've gained my trust.

It's also important to have no fear. I've known people who lost out on the recent market gains because they were too scared to buy in at the low point. Now, they put themselves in the market when it's booming.

I've also played around with Bitcoin, but that was more for fun and I definitely wasn't planning for the long-term there.

<comment><AUTHOR> Thank you for your insights. I also subscribe to Motley Fool as well and benefited from it. I tried reading in the past, and you are absolutely correct, it just either became too consuming and the financial jargons just became less interesting.

<comment><USER2> No problem. It's important to take your time and build it up slowly. After awhile, you'll realize you've put away good money and have made some substantial gains. If you're lucky, you'll collect a few 10 baggers and maybe even a 100 bagger in your lifetime. Couple that with a good product and/or a good job and you'll be pretty secure.

</comment></comment></comment><comment><USER3> I believe in the 80% solution. Put your money to work rather than stuffing it in a mattress, but favor the 80%-of-optimal solution that's easy to set and forget, instead of the 100%-optimal solution that requires micromanagement, expertise, and luck.

To that end, keep only a little in savings (emergency fund), and put the rest in a 401k or IRA, invested in index funds. Index funds are a great 80% approach, because:

\- They track an index (e.g. S&P 500) which is likely to go up and up in the long term

\- They are very low cost since they are not actively managed

\- You can watch them every day, or forget about them for two years

\- Warren Buffet recommends them, and he seems to be doing okay

<comment><USER4> what is a good amount for the emergency fund? Is it lie 4 times the monthly income or even more?

<comment><USER5> Depends on your situation. If you have kids and fixed costs, 6-12 times monthly spend (not income). If you're alone and have more flexibility, 3 times monthly spend.

</comment></comment></comment><comment><USER6> I actively traded stocks for a period. I can say that any gains you make represent work like any other, except you can lose all your money. If you do not want to focus on trading, do not trade. It can be a horrible way to live.

Warren Buffett's #1 rule is: Do not lose money. #2: Never forget rule number one.

What you really learn is that the amount of risk you should take on actually is a lot less than you think it is. Too much volatility will kill your returns (Kelly Criterion).

Second is that you shouldn't make an investment unless you just about have domain level knowledge. You need an edge. Over-trading increases volatility, which is bad because you lose more than you gain just by laws of percentages.

If I could tell my younger self what to do, I'd tell him to just sit tight. Hold the money and don't worry. You don't need to invest it all. At some point there will be a market crash and it will be a no-brainer to buy. Buy then and hold for long term. And you can hold because you have a cash cushion, and a price that makes financial sense no matter what (like a good dividend or rent percentage to invested capital).

<comment><USER7> I would agree for the most part with the exception of "At some point there will be a market crash and it will be a no-brainer to buy."

Timing the market is incredibly difficult. I'd just suggest setting up an automatic dollar cost averaging scheme into a fund mix of your choice and ignoring it from that point on. As long as you have a decent enough time horizon, you'll wind up well ahead.

<comment><USER6> I agree and disagree. I think legging into positions is a great way to ensure lower volatility and is a good for psychological discipline. However market timing works in the Buffett sense, when you make a purchase that immediately makes sense no matter (almost) what the asset price does after that. If you could hold forever, you'd simply make money off the dividends. Like buying a house - you can't lose due to price fluctuation (barring a ghost town event) if you planned to make money from rent. But if you bought hoping for capital appreciation, you could get really hurt.

</comment></comment></comment><comment><USER8> If you're interested in actively managing your own portfolio, for some reason this isn't mentioned much places, but you need to have a deep understanding of financial accounting. That will allow you to read and make sense of financial statements. There is no other way, if you're an active, vs. passive, investor. Then one easy strategy is to do what Charlie Munger calls, "sit on your ass investing". That's where you wait for bargains in strong companies to come along and you swing real big. Diversification is not as important if you know what you're doing. And you don't have to sit around if the market is fairly priced, just when it's high. During that time, when you're not doing much investing, you should be reading, studying, and aspire to get into more special situations (arbitrage) and look for Ben Graham style net nets with micro to small cap companies that aren't as popular. Even OTC market value investing with your own scuttlebutt. There aren't much of those so you have to see what ratios people substitute for those nowadays. For example, you're not going to find decent insurance companies selling at less than their working capital minus total debt. Those used to exist after the Great Depression. Now you have to find companies say, selling at some higher ratio than what would have made a decent margin of safety in those days. It can take a long time and a lot of watching the market, looking at different stats to know what you should be looking for.

</comment><comment><USER9> I was very interested in trading stocks for a while, and would read up a lot about the markets. I then met a stock broker and he told me that it wouldn't be very different from gambling if I were to invest based on the news without much understanding of how the markets work.

One thing he told me really struck a chord: I spend 8 hours a day at work, and read about the markets in the night or on weekends. But he lives/breathes/eats the stock markets all day and it's been his full time job for the past several years. So my money would be safer if he invested it for me instead of doing myself (unless I had : [http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect](http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect))

So now every month or so he calls me to recommend a stock and explains why I should invest in it. This let's me use my spare time to develop my skills (since I'm somewhat early in my career) and I've gotten good returns on my investments in exchange for a small commission.

<comment><USER7> Overall, this is pretty bad advice IMO. Investing in individual stocks, even at the recommendation of a "professional" is a fools game. You will be hard pressed to find a "professional" who can routinely and repeatedly beat an index fund.

Also, "professionals" charge high expense ratios when compared to funds through Vanguard, Fidelity, etc. A lot of financial advisors stress the importance of expense ratio as one of the most important facets of your portfolio.

For the record, I invest in index funds through Vanguard with expense ratios ranging from .05% to .22%.

</comment></comment><comment><USER10> I had tried few different ways to invest and found the "coffeehouse investor" book to be pretty good guide on how to invest by spending really little time.

Book is really short. But you don't need to buy the book. The website lists the index funds, the returns over last 20 years and portfolio %. I am mostly invested in Vangaurd index funds and it has worked out well for me.

[http://www.coffeehouseinvestor.com/coffeehouse- beans/coffeeh...](http://www.coffeehouseinvestor.com/coffeehouse- beans/coffeehouse-returns/)

</comment><comment><USER11> This is probably, like, a horrible way to do things. But I try to find a company (1-10 billion market cap) that I personally believe in, but saw-tooths a lot. I usually check-in once a day and watch it go up and down. I then guess if I think it is sort of on a crest up or down, then buy or sell accordingly. And never accept a loss on a trade. Instead just sit on the shares until they are in the black. And example would be CAVM. I figure it's sort of like gambling. I don't really know what I'm doing, but it works (so far).

<comment><AUTHOR> I did that before too! Honestly, it did feel like gambling a bit. It worked until the company one day went bankrupt.

After that I started looking at newsletter service like Motley Fool, but curious to see if there are other services that people recommend. I have looked at Personal Capital (iPhone app), but haven't booked a session with them yet.

Personally, I have been a turned off by financial advisors.

<comment><USER11> Bankrupt, ouch.

Yeah, most people don't seem to think very much of this strategy (I'm even down-voted?), and no one talks about it. But I'm not sure why. I hear about the crappy 5% a year or whatever returns people get, which just never seems very good to me. But, idk, maybe I'll lose all my money one day. That's a real possibility as I spurn most advice I get.

Anyway, Good luck!

</comment></comment></comment><comment><USER12> The bulk of my money is in index funds. Given my age, it's mostly in equities.

I started off with advice from this article by transferring a bit of my savings weekly into a target date retirement fund.

[http://www.iwillteachyoutoberich.com/blog/asset- allocation-i...](http://www.iwillteachyoutoberich.com/blog/asset-allocation- investor-psychology/)

Today I have other index funds in other sectors, but the asset allocation is pretty much the same ratio. Hope that helps!

</comment><comment><USER13> 1/3 in a domestic (U.S.) index fund, 1/3 in an international index fund, 1/3 in CDs, money market funds, savings bonds, or T-notes. Rebalance once a year. Spend no additional time thinking about any of this.

<comment><USER7> This advice should be adjusted given your age / risk tolerance. 1/3 in cash equivalents is a bit conservative for a 20 or 30 something, for example.

</comment></comment><comment><USER14> Buy rental properties across the nation, try to earn 10-15% annual return on rents after expenses, plus appreciation in the long run.

</comment><comment><USER15> The bulk of my money is in Bitcoin.

<comment><USER16> In case you're not joking: This is a bad idea.

Though I admit it might work out well. If you're lucky.</comment></comment>


Who's requiring uneeded fluff in their passwords? - <AUTHOR>

Perhaps some public attention on "Please use a symbol, greek unicode character, and capital letter in your password" type rules on sites will get them to use modern password strength testing that doesn't make people write down passwords.
Please say "Site name:Bad part of password policy"
github:requires numerals even if entropy is very high without <comment><USER2> I'd love to use greek characters in passwords, if only I could be sure sites weren't silently folding them to question marks, or deleting them altogether.</comment>


Online physics learnware - <AUTHOR> http://www.worldscienceu.com/ <comment><USER2> Site doesn't work for me with Firefox; works OK with Chromium.</comment>