More than 1/3 of American adults take prescription drugs linked to depression - <AUTHOR> https://www.washingtonpost.com/news/wonk/wp/2018/06/12/more-than-a-third-of-american-adults-take-prescription-drugs-that-may-increase-risk-of-depression-study-says/?noredirect=on&utm_term=.ebe3473f95f7 <comment><USER2> The headline is a bit confusing. At first glance it might seem to say that 1/3 of Americans take anti-depressants. What it's actually trying to say is that 1/3 of Americans take prescription drugs that have depression as a possible side effect.

Even that seems bizarre to me. What is the percentage of Americans who take prescription drugs... period? I'm a middle-aged man, and (knock on wood!) still haven't come down with any chronic conditions requiring medication. So I would probably assume that no more than 1/3 of Americans take ongoing prescription drugs at all.

Which means I'd have to presume that pretty much all drugs have depression as a possible side effect.

<comment><USER3> As a middle aged man I assume you're not on birth control, a prescription drug whose side effects can include depression. I would not be surprised to find that prescription birth control makes up the lion's share of the third of the adults counted in the article.

Based on my family's anecdotes it seems even discounting even the contraceptive angle, hormonal BC seems to be the crude implement to treating a host of female health issues we don't under stand very well.

<comment><USER4> No doubt contraceptives are a part, but they are actually taken by a small minority of people. They are only applicable to relatively young females, those capable of having children. Younger children and older women aren't much of a market. Depending on what ages you want to pick, the applicable population is probably less than 1/4 the total, with a smaller subset of that actually taking.

A quick googling says that 28% of women of "reproductive age" use the pill, which by my math is about 14% of women overall and 7% of the total population.

<comment><USER5> What math are you using? By my understanding, women are able to reproduce for around or close to half their lives.

First result[1] I found while googling defines women of reproductive age as 15-49, and claims that group makes up 24% of the overall population, which would line up with my intuition.

[1] PDF [http://www.who.int/management/UsingDataToImproveServiceDeliv...](http://www.who.int/management/UsingDataToImproveServiceDeliveryAnnexes.pdf)

EDIT: Ah, reread your comment, looks like your math lines up with mine, just misunderstood. Thought you meant 7% of the population were women of reproductive age, not that 7% of the population were women of reproductive age on the pill.

Although it also makes me wonder if a sizable percentage of women outside reproductive age are on the pill. Either because they are fertile longer than average, or as treatment for other conditions.

<comment><USER6> Hormone Replacement Therapy is similar.

</comment></comment></comment></comment></comment><comment><USER7> Slightly confusing title, but this is about drugs for other conditions that can include causing depression as a side-effect. Not the percentage that are taking anti-depressants.

A little bit of a vicious circle... 1.) You have symptoms of conditions that are exacerbated by depression. 2.) Buy drugs to treat those symptoms 3.) Drugs exacerbate depression 4.) Get anti-depressants to combat depression 4.) Anti- depressants have deleterious side-effects in other areas 5.) Goto 1

<comment><USER8> There are also a lot of cases where things like mineral deficiencies cause depression symptoms, and then SSRI antidepressants make them much worse.

</comment></comment><comment><USER9> And that doesn't include self-medication with alcohol or marijuana and so forth.

<comment><USER10> Admittedly I 'self-medicate' with marijuana. It has completely transformed me. I used to play video games and basically do nothing, but with marijuana I'm insanely productive.

I'm talking do the 9-5 on coffee only, then go home and worth 4-6 more hours on programming projects.

I have a fanbase for my website, I have transformed my programming abilities, and I've grown a non-profit.

I'm not sure I can work this many hours daily completely sober. My bigger concern is how many years I can work like this. This is year 3.

<comment><USER11> Dammit, yet another weirdo ruining marijuana's carefully cultivated slacker image. ;)

I have a friend who reacts to marijuana similarly to what you describe and it still boggles my mind that two people can take the same drug and react in nearly opposite ways.

<comment><USER12> Anecdote: My mom has never done any drugs etc but first time she caught me smoking she said: "some people do drugs, drugs do most people". I'll never forget and it's true in my experience.

<comment><USER13> Inanity masquerading as profundity.

</comment></comment></comment></comment></comment><comment><USER4> >> "The study analyzed a detailed survey of thousands of American adults taken every two years between 2005 and 2014, in which people opened their medicine cabinets and showed researchers all the prescription drugs they had taken in the last month."

So the methodology and therefore tittle is a little misleading. This is about people who have been prescribed something and have taken it in the last month. Those taking prescription drugs without a prescription probably don't admit this in surveys. All the off-prescription opioid addicts are not represented. The title should be that 1/3 of Americans have been prescribed drugs.

I'd rather see data from pharmaceutical manufacturers. I'd be interested in how many tons of depression-linked medication is being consumed, from which we can estimate the total number of 'prescriptions' that are active. That would cover the off-prescription consumption too.

</comment><comment><USER14> Do 1/3rd of Americans need to take prescription drugs on a regular basis? If that's the case then alarm bells should be going off even if they're not depression related.

</comment><comment><USER15> Clickbait headline. Not article refers drugs which may have some correlative relationship with depression (e.g. side effects may include), not drugs associated with treatment of depression.

<comment><USER16> I also mis-read the headline, but that doesn't make the headline incorrect.

<comment><USER15> Not incorrect, just clickbait.

<comment><USER17> I'm not so convinced; "linked to" is absolutely bog-standard for correlation. The interesting thing is how much more we're primed to read it the other way even with relatively unambiguous language.

</comment></comment></comment></comment><comment><USER18> fix the goddamn headline

</comment><comment><USER19> how about: people are already depressed and having sketchy, addictive drugs pushed on them by a large profit-driven medical industry will not fix the real issue?

<comment><USER20> The headline doesn't seem to be about antidepressants.

</comment></comment><comment><USER10> They didnt include caffeine? I've read that caffeine is one of the best drugs for reducing depression.

Note: I have a love/hate relationship with caffeine.

<comment><USER21> This is just "prescription drugs" which excludes caffeine.

</comment></comment><comment><USER22> This is a curious issue, clearly this is a problem. Having 1/3 of your population depressed is IMO near a crises. However, how would a government or a body of people act to fix this? Aiding people with tax breaks may not even be touching the issue due to the nature of depression.

What can a governing people do to make it's people not miserable except for handing out happy pills?

<comment><USER16> Reading the article reveals that you have fundamentally misunderstood the issue here. The first paragraph of the article states:

> More than a third of American adults are taking prescription drugs, > including hormones for contraception, blood pressure medications and > medicines for heartburn, that carry a potential risk of depression, > according to a study published in the Journal of the American Medical > Association.

Not on "happy pills," but on medicines that may lead to depression as a side effect.</comment></comment>


Most published research results are false - <AUTHOR> http://www.johndcook.com/blog/2008/02/07/most-published-research-results-are-false/

<comment><USER2> Note well. This is for clinical research.

This says nothing about C.S., Math, Physics, etc...

It's harder to publish a big result in Physics, for example, and have it stand for long without being questioned. Just ask Jan Hendrik Schön.

<comment><USER3> From what I've seen, I wouldn't say CS is immune to this at all. CS seems to only be accepting papers that illustrate some big win.

In my short PhD career, I've an article in the drawer where the result was inconclusive. This result, in and of itself, was fairly valuable. It spoke much about all the ways things went wrong. However, wrong was not acceptable for publication, it had to be right. Much hand-wringing was done to identify the bits that could be pulled out from it, polished, and then submitted (I was not especially pleased about this). The paper was, rightly, rejected, but all the valuable findings are still in my drawer. The pressure to produce Good findings rather than Bad findings does not meet my expectation of scientific pursuit.

I took great joy in seeing a history of science exhibition recently, and reading the notebooks of people like Newton and Darwin, who they themselves spend a lot of time writing out all the things that went wrong, all the concerns they have about their "results", and general humbleness. It speaks volumes that the greatest scientific minds that have ever lived have put more caveats and concerns in their published work than 98% of what is published today does. To get published, it seems you have to be 110% sure that your work is 120% amazing. I don't think this is a good state of affairs.

</comment></comment><comment><USER4> Given that this proclamation was itself a published research study, does it mean that it is also probably wrong? The barber also publishes papers?

</comment><comment><USER5> I always found statistics deeply disturbing when I was studying it at school.

You can establish a null hypothesis and then test at some confidence interval (say, 99%) and find that your null hypothesis still holds.

But, being somewhat ingenious, you may decide to lower the confidence interval (to, say, 98%) and find that your experimental evidence is now significant enough for you to reject your null hypothesis and accept your alternative hypothesis.

Lies, damned lies, and statistics, indeed.

<comment><USER6> This is why different communities accept p-values at certain levels (typically <0.05). You have to live with some amount of uncertainty when the data generating mechanism is random... that is unfortunately the nature of the beast.

<comment><USER5> I understand this. But let's say I ran some experiments and collected some data in an attempt to disprove theory X. What does it mean for me to say that at the 99% confidence level X is still true, but at the 98% confidence level it is not true? I just find it a bit spooky, is all.

</comment></comment></comment><comment><USER7> Linke to the PLoS article that this post summarizes: [http://www.plosmedicine.org/article/info:doi/10.1371/journal...](http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124)

</comment><comment><USER8> Just out of curiosity, what is the alternative to using p-values? What should researchers really be doing?

<comment><USER9> Most studies use raw p-values. This, coupled with "exploratory analyses" which are not mentioned in the research, leads to a high false discovery rate.

Alternatives include using simulations to estimate false discovery rates (this can be done analytically for some problems). Bayesian frameworks can also be applied, depending on the amount and accuracy of prior knowledge of the system under study.

That said, I see nothing wrong with publishing results with nominal p-values, as long as the researchers indicate the weakness in their results. Meta- analyses can always come back later and use the results they publish.

What annoys me is seeing 50 or so statistical tests done, and then researchers stating they have found something when 1 of those tests shows a p-value of 0.05. Just using a Bonferroni correction, the simplest of all corrections, would demonstrate that findings like this are not significant.

</comment></comment><comment><USER10> I enjoyed this related article a few months ago on the problem: [http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_...](http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer)</comment>


Real-time behavioral experiments on the web w/ Node.js - <AUTHOR> http://link.springer.com/article/10.3758/s13428-014-0515-6 <comment><AUTHOR> Paywall-free link: [http://www.researchgate.net/profile/Robert_Hawkins6/publicat...](http://www.researchgate.net/profile/Robert_Hawkins6/publication/266381125_Conducting_real- time_multiplayer_experiments_on_the_web/links/545beca70cf2f1dbcbcb083d.pdf)

<comment><USER2> Thanks, just saved me a few minutes :)

Springer (and Elsevier) are so freaking ANNOYING sometimes :(

And links to them instead of something actually readable... perhaps even more so.

<comment><AUTHOR> No problem. Pay walls suck. Enjoy!</comment></comment></comment>


Adobe Brackets: An open-source code editor for HTML/CSS/JavaScript - <AUTHOR> https://github.com/adobe/brackets

<comment><USER2> It doesn't run in the browser at the moment, the actual app can be downloaded here: <https://github.com/adobe/brackets-app>

(you'll have to download the brackets repo too and put it inside the `brackets` folder)

</comment><comment><USER3> While it's obviously early days, what I find most interesting about this is that it's (a) Adobe, (b) Open Source and (c) developing in the open.

Does anyone know more about how this came to be/what the plans are for the future?

<comment><USER4> The history of the project is quite simple. We wanted to see if it was possible to build a high quality code editor for web development in JavaScript, HTML and CSS. The idea being two fold. The engineers working on Brackets would experience the day-to-day development pains of building for the web so they would be in a better position to innovate new solutions. The other benefit is that most open source tools are out of the reach of their target audiences. If you have the skills to use Brackets, you have the skills to build Brackets.

Our original prototype ran completely in the browser using node.js for file i/o. So why did we take it out of the browser and focus on the desktop first? While we believe the future of development is heading towards the cloud, we found that many people are still looking for traditional desktop tools. By starting on he desktop and then supplementing with a cloud-based version, we hope to help transition developers to this new model, without asking them to give up what know and love.

Of course, it's not just about the cloud. Because we're developing on the open web platform, we can go wherever that platforms goes. While it's not our immediate focus we hope to build a version of Brackets optimized for tablets as well as an embedded version that could run inside existing web application.

Adobe has committed some of its best engineering talent to the project for the foreseeable future. If you check the activity on the repo, you'll see it's not a part-time side project. We have a long way to go before we would declare 1.0, but we think we can get there for the desktop version within the year.

I invite everyone to join our mailing list (<http://groups.google.com/group/brackets-dev>) or pop in to Freenode IRC channel #brackets to chat w/ the development team.

\---

Adam Lehman Brackets Product Manager

<comment><USER3> Thanks for your thoughtful and helpful reply. :)

</comment></comment></comment><comment><USER5> It doesn't work on Linux. The brackets-app only has code for windows and mac.

<comment><USER6> It's HTML, CSS, and JavaScript running on top of CEF. CEF works on Linux, so it should be pretty easy for someone to get Bracket running on Linux.

<comment><USER7> For those who don't know, CEF is apparently this: <http://code.google.com/p/chromiumembedded/>

I had just wondered last week if something nice like this existed.

</comment></comment></comment><comment><USER8> This is Win/Mac only. There does not seem to be a Linux version.

</comment><comment><USER9> I don't think the world needs another desktop editor, but it'll be interesting to see where it goes with web-based editing with Adobe behind it.

Ace seems to be a better alternative for now and is entirely JS-based:

<http://ace.ajax.org/>

</comment><comment><USER10> "Although Brackets is built in HTML/CSS/JS, it currently runs as a desktop application in a thin native shell, so that it can access your local files. (If you just try to open the index.html file in a browser, it won't work yet.)"

</comment><comment><USER11> Was impressed until I had to download 125(!) MB of Desktop app to be able to run it.

</comment><comment><USER12> Would be interested to see a demo or screenshots also!

<comment><USER13> Certainly agree.

Not even a link to a static project description page (as far as I could tell)

I couldn't find a single picture with a quick browse through. (I know this is superficial) but no screenshot/screencast/videos is bad marketing and doesn't get me as excited about a project

<comment><USER14> Here is a quick screenshot[1] from the main window. It's still far away from being really useful, but as an experiment, it looks decent.

I personally recommend Eclipse Orion[2], it's not as stylish as Brackets but is fully functional and you can even import repositories from github.

[1] <http://i.imgur.com/7Wa6p.png>

[2] <http://www.eclipse.org/orion/getstarted.php> (choose "Download and Install Orion on your localhost")

<comment><USER7> I tried to compare CodeMirror, Ace and Orion a while back. Orion... well, it certainly feels like it's made by the Eclipse team. The splash page is flashy, but Orion itself looks and feels terribly clunky.

I think Ace (part of Cloud9) looks nicer and as far as I could tell, did everything Orion does. It also will import Git repos. Did you find anything that Orion does particularly well or better than besides Git repos?

<comment><USER14> I agree that Cloud9 looks very stylish compared to Orion. One thing i like about Orion is comment and code folding features. I also liked how easy it was to just download and run Orion, as opposed to "building" cloud9 with all the dependancies and such.

But yeah, if you appreciate beauty, certainly going with cloud9 is the obvious choice.

<comment><USER7> Cloud9's repo includes pre-built binaries, I've never built cloud9 personally. And both Ace and CodeMirror support code-folding.

I don't mean to nitpick, I'm working on something similar to these (with some important differences) and I would like to pick the "best" core editor. Like I said, is there anything else that Orion does better? I don't care about appearance, they're basically all customizable to an extent that they can be made it look identical. And for the most part, they all seem to support the same features (obviously CodeMirror is missing things like Git repos because it's only the browser-side of things)

</comment></comment></comment></comment></comment></comment><comment><USER15> bleh, another web thingy editor.

</comment><comment><USER16> Demo?

</comment><comment><USER17> "built in HTML, CSS and JavaScript."

Why is that good? Eclipse is built in Java, and it's janky.

<comment><USER18> What does Java have to do with anything?

</comment></comment><comment><USER7> Looks like the editor is based on CodeMirror from a brief peak at the source.</comment>


Create awesome mobile websites quickly and easily - <AUTHOR> http://mobilizer.me

<comment><USER2> No tour, no about, no demo, no samples?

<comment><AUTHOR> FIXED. Thanks.</comment></comment>


The Philosopher of Surveillance - <AUTHOR> https://firstlook.org/theintercept/2015/08/11/surveillance-philosopher-nsa/ <comment><USER2> The "Socrates" of this article believes he's safer if his entire life is monitored rather than just part. So much wrong with this.

1 - What about no surveillance until there is a reasonable suspicion confirmed by an impartial Judge (see 4th Amendment to the Constitution for details)?

2 - Ok, he feels safer with Big Brother looking after him ... but apparently that justifies in his mind that everyone should be watched since he prefers it for himself. Is that narcissism or just sloppy thinking?

3 - One of the many dangers of this sort of surveillance is the bizarre notion that we are the data collected. There is NO level of surveillance that equates to the actual knowledge of the self (sorry Google, NSA, etc.). Think about it for yourself - is there any data set large enough that it would capture without misunderstanding who you are? Now remember that the rest of the 7 billion people on this planet have just as rich lives (inner and outer) as you.

On that last point, keep in mind the so called "targeted" ads that Google uses. Or the movie/book suggestions from sites that use them. Now imagine such bizarre misunderstandings leading our NSA big brothers to act. Not a pretty picture.

Larger and more haystacks aren't the solution. Realizing that humans aren't stacks of data is.

<comment><USER3> Just to play Devil's Advocate, real total surveillance might not be bad. If you were hooked into a system that actually monitored all minds in real time and connected you directly to whatever other mind you wanted to reach out to with no misunderstanding, you'd have a hive mind that was no longer human but might be capable of more complicated thoughts.

It's been done plenty of times in science fiction, but now we're getting to the point where we can actually transmit some information directly from mind to mind. I think there's room for voluntary connectivity without being invasive. There should always be a way to have total privacy whenever it's desired, and that should be true for the near future without question.

But there's a chance that at some point, people won't even want it anymore. If it happens voluntarily as people become more comfortable with each other, I'm not sure if it will even be bad. Forced is no good of course, and using surveillance to impose oligarchical control is terrible, but if we move gradually, I don't think anyone can really predict how it might turn out.

<comment><USER4> I know this idea has oft been explored in science fiction, but if you haven't seen it, I recommend that you check out the series _Psycho-Pass_. It has an interesting and intellectually hefty take on the subject.

</comment></comment></comment><comment><USER5> "A target that has no ill will to the U.S., but which is being monitored, needs better and more monitoring, not less. So if we're in for a penny, we need to be in for a pound."

This is profoundly sick.

<comment><USER6> He has a childish idea that the purpose of surveilance is to detect the bad people, the enemies.

From his first SIGINT Philosopher column: > I guess if we were a corporation, we could make our mission statement (or "corporate philosophy") this: "building informed decision makers -- so that targets do not suffer our nation's wrath unless they really deserve it -- by exercising deity-like monitoring of the target." Now that's philosophy.

</comment></comment><comment><USER7> I've said it before and I'll keep saying it:

Does anyone else find it deeply suspicious that Wall Street (and investment bankers in general) remain mum about this? We're talking about a group which is one of the most powerful in the world, has tremendous influence over government, and has the most to lose by invasive NSA surveillance.

<comment><USER8> No, because I think Wall St. is fairly safe from the NSA in at least three ways. First, they can hire the best defensive programmers (and probably do). Second, even if they get breached, the NSA can hire the best lawyers (including plenty of ex-NSA, ex-Justice Department folk) - so it can't be used against them legally. Third, the NSA and Wall St. are fundamentally on the same team, at a very deep level.

<comment><USER9> Your third point is dead on. Otherwise, why do surveillance on private (foreign) corporations?

</comment></comment></comment><comment><USER10> So how long until someone deanonymizes "Socrates"?

<comment><USER10> Found his blog: [http://workshopheretic.blogspot.com/](http://workshopheretic.blogspot.com/). Easily google-able from phrases in the article.

<comment><USER11> [http://baltimorereview.org/index.php/fall_2014/contributor/j...](http://baltimorereview.org/index.php/fall_2014/contributor/jacob- weber) if you'd like to put a face to it

<comment><USER12> Relevant section quoted below:

 _THE INTERCEPT HAS A POLICY of not publishing the names of non-public intelligence officials unless there is a compelling reason, as with our naming of Alfreda Bikowsky, who oversaw key aspects of the CIA's torture program. Withholding Socrates' identity presents certain problems in the age of Google, however. If I quote from his blog, or give its name, or provide other search- enhancing morsels, like the name or location of his graduate writing program or where he was born, I might provide the sort of data that could instantly reveal his name with a few keystrokes._

 _So I am more or less trying to do what the NSA and a large number of agencies and corporations do with the personal data they possess — stripping away names and other identifying information to "anonymize" the data before sharing it. The beauty of anonymizing data, according to the (very many) entities that do it, is that nobody can be identified — citizens and consumers do not have to worry that their privacy is violated when petabytes of data are collected about what they do, where they go, what they read, where they eat and what they buy, because their names are not attached to it. The conceit is that our data does not betray us._

</comment></comment></comment></comment><comment><USER13> Thought this would be about Palantir, the surveillance company with a philosophy PhD CEO.

<comment><USER9> Me too</comment></comment>


How To Find Unadvertised Jobs - <AUTHOR> http://www.glassdoor.com/blog/find-unadvertised-jobs/ <comment><USER2> Uh, so the answer to 'how to find the unadvertised job' is 'look at company websites' or 'do google searches'?

I haven't been in the job market for years, so I don't have a great answer, but these seem pretty weak ways to find jobs that aren't advertised, because, if they are on company websites or available via Taleo, they _are_ advertised.

How about:

      * research a company, find out their biggest problem (competitive research), find the manager who should be tackling the problem (linked in), and email him an outline of a solution.  I don't manage a ton of folks, but if someone did this, we'd definitely have a conversation.
      * go to tech meetups and meeting interesting people
      * comment on a corporate blog regularly
      * offer to work for free (if you can afford it)
      * ABL (always be looking)
      * never eat lunch along
      * blog about the industry you want to get into
      * work on an open source project
      * instagram (or whatever the kids are doing these days)

This is an oldie instance that one of my friends did to get hired back before the dotcom bubble: [http://donwrege.com/XOR/home.html](http://donwrege.com/XOR/home.html)

<comment><USER3> "* offer to work for free (if you can afford it)"

I know there are stories where working for free is a good strategy, but they are vastly outnumbered by stories where not working for free was the best option.

I'd argue against it unless you have a really, really, really good reason to work for a specific organization for free.

<comment><USER2> I think you are correct. One reason I worked for free for an organization was that it was in an entirely different space than my previous experience, and I wanted to know more about it. Other good reasons to work for free: * gain skills * meet people

But you definitely want to be careful and limit it, otherwise you'll get taken advantage of. It's very similar to offering a free software service--often you don't know if what you have is valuable until people start paying for it.

</comment></comment></comment><comment><USER4> _" When you see a job posted on (insert leading job board name here) somebody paid to have it advertised there. You know where it's free for a company to post all of their jobs? Come on, guess… If you suspected a company's own careers website, you would be correct!"_

If somebody paid for it recently, it is a strong signal they really have an opening. Jobs on companies' own sites might only be there to:

a) show to customers and investors the company is in good shape.

b) keep their recruitment buffers filled just in case.

I've been through jobs in companies with "no recruitment" enforced due to budget cuts and guess what, the job ads on their sites were still there.

<comment><USER5> My last job we were advertising and interviewing for a position that did not exist on the belief that somebody wanting job X could be talked into job Y instead.

I don't work there anymore.

</comment></comment><comment><USER6> Well this is unfortunate: [http://i.imgur.com/41Yn4ye.png](http://i.imgur.com/41Yn4ye.png)

<comment><USER7> You might have more luck getting the developer to fix it by providing some information about your browser. It works fine in the latest stable Chrome on Mac OS X, for example.

<comment><USER8> Broken on Firefox for me too.

</comment></comment></comment><comment><USER9> At least within the area of the UK which I lived in until recently the idea that there are lots of unadvertised jobs is a myth. When I was unemployed I did the experiment of applying both via "the usual channels" and also speculatively based upon fairly elaborate analysis to try to identify suitable organisations. The number of "conversions" \- which means inquiries turning into job interviews - was vastly higher for the usual channels, with very few companies responding to speculative inquiries.

<comment><USER10> What makes you think your methodology was a good one for finding the unadvertised jobs ?

Plenty of unadvertised jobs will go through networks, word-of-mouth and personal recommendation - obviously cold-calling wouldn't give you good results in such cases.

Generally it's estimated that roughly 40% [for 2009-2010] of jobs are unadvertised, and that's calculated using US BLS data ([http://www.bls.gov/news.release/jolts.htm](http://www.bls.gov/news.release/jolts.htm)) comparing number of job openings companies state they have versus actual hiring numbers.

While there might be variation between countries, I doubt that there's significant difference between the UK and the US.

<comment><USER9> I think my methods were good. They were based on data mining company information for all companies registered with Companies House, such as changes in numbers of employees, net worth, main products and the like. I tried to apply scientific method to the task of finding employment.

If the unadvertised jobs cannot be detected by any empirical evidence then I would suggest that they be treated like other mythological phenomenon.

<comment><USER10> Did you miss my link to the BLS data showing the difference between hiring number and job openings ?

There's plenty of empirical ways to show that unadvertised hiring occurs. Another alternative way would be simply to survey companies and employees and ask them.

The fact that unadvertised recruitment happens doesn't mean that the jobs will be available to anyone who attempts to apply via cold-calling.

A company that receives a strong recommendation for a candidate via someone they trust may well create a new role especially for that person, but obviously such a role wouldn't exist if someone unknown tried to apply speculatively.

</comment></comment></comment></comment><comment><USER11> Isn't it better to let a recruitment company do the hard work for you?

In a technical job market there are recruitment consultants that know all of the companies, what might work for you and where there might be a need. In my experience they do provide a very good service. However, we tend to dislike them when they are not needed so we might overlook them as our first point of call.

<comment><USER12> _Isn 't it better to let a recruitment company do the hard work for you?_

If you want to entrust your professional future to someone who doesn't understand the work you do and whose interests aren't necessarily aligned with yours, sure.

Recruiters aren't all bad - they can be one prong of your job-search strategy. (I just found a great startup job through a recruiter.) But don't put them in charge.

And read _Ask The Headhunter_.

<comment><USER13> "Recruiters aren't all bad"

I keep hearing it, but I haven't really ever seen it.

<comment><USER12> Recruiting is a huge industry. They are obviously not all bad or useless.

Most small companies simply cannot afford to employ a full-time in-house recruiter. If you were running a startup, with a thousand things to do daily, you too would outsource the basic search and screening function.

</comment></comment></comment></comment><comment><USER14> It could be a waste of your resources, because many such listings are actually not designed to lure good qualified prospects, they have a completely opposite intent - lure idiots and unqualified people. Even if you apply, they'll talk to you, then talk again, then disqualify you on some ridiculous grounds.

I'm talking about Green Card PERM process in these companies. The law mandates them to advertise, but their intent is to prove to the DOL that there's no adequate candidates around. They _already have_ someone for this position, moreover, this person (most likely) already occupies this position, they just started his/her GC process.

</comment><comment><USER15> Taleo makes a great job making it as painful as possible to search and apply for jobs. I find it pretty funny a "hack" of how to look through taleo listings is in an article about unadvertised jobs.

<comment><USER12> Yup. On the one hand, recruiting is hardly the rocket science of software engineering, and on the other hand, it's hard to imagine a more user- unfriendly system than Taleo, yet somehow they seem to dominate the market.

<comment><USER16> Silk Road and SuccessFactors are just as terrible in my opinion but seem to also have a healthy percentage of the market. I believe all these HR software systems follow a similar pattern of starting out solving a fairly simple problem, then as Sales brings in more and more disparate customers, the developers keep tacking on stuff until it is a hodgepodge mess.

</comment></comment></comment><comment> vezzy-fnord I was expecting a post that told you to "get off your lazy ass and look for some in your area". Instead, I got a primer on Google search operators.

</comment><comment><USER17> In my experience 'Option one' is not so bad. You need different industry directories (not just "yellow pages") to find most relevant companies and then contact them. Laborious but better than being one in a hundred applicants.

</comment><comment><USER18> Can't you just used Indeed.com? I thought every major ATS allows you to submit your job listings to Indeed.com and similar sites.</comment>


Ask HN: Where to Learn Database Design? - <AUTHOR> What are the best resources for learning database design?<p>In interviews for back-end software engineering jobs, it's common to write out models and relationships as an exercise. Yet I struggle to find resources to practice this further and learn more. <comment><USER2> Have a look this one [http://www.databaseanswers.org/data_models/index.htm](http://www.databaseanswers.org/data_models/index.htm)?

You can learn the database design by example.

</comment><comment><USER3> Data Model Resource book by Len Silverstone. Would advise to read and understand all three volumes though.

</comment><comment><USER4> I'm going to advocate strongly for getting your hands in the dirt immediately and leaning on that more than traditional database design books (they have their place, however I think they're a relatively weak benefit when starting out, and often more of an added intellectual burden that you don't need yet).

I learned everything I know about database design from building a lot of Web services over a long time. Starting from very simple concepts and gradually increasing complexity as my competency increased. I think you can simulate that process and skip some of the time to an extent however (ultimately design & theory must meet real-world use to become great at it). The only books I read were on the SQL language. I added to that by researching optimizations online. After learning SQL and a standard relational database approach well I found everything else was comparably easy to pick up. Good books on SQL will often cover (to an extent) connected topics, including limiting the duplication of data and indexes. If you already know SQL, then even better.

To compress your learning time, I would set up a relational database (using whatever you prefer, MySQL or Postgres for example). I would come up with a fake application concept, something with size to it (not just a blog application or note keeper; unless you need to start at that level of simplicity, then by all means). Maybe lay out your own version of a Stack Overflow Q&A type site (ideal because of its structure and semi-obviousness), strictly only on paper so to speak (do not write a single line of application code, do not worry about scaling, backups, etc. etc.). If you happen to get stuck, in the case of Stack Overflow you can reference their actual database design online.

To start I would probably map out the relationships on paper in combination with the actual database you're working on putting together. Later on you'll generally find you can keep that in your head without any effort, it'll become second nature.

Once you feel like you have a good schema, I would stuff your tables full of large amounts of fake data and stress the system with manual queries, to begin simulating performance and learning the benefits of proper indexes and how queries are impacted variously.

Do this over and over and over again. You should be able to do one of these per week in your spare time (or more, depending on how much time you want to throw at it and how much you want to accelerate your knowledge). Every time you do this, you'll get better at it. It'll also be helpful to redo old designs, to see how you'd improve upon them as your skill advances. And obviously improvise where you think it makes sense.

<comment><USER5> I'd say the OP is not a complete beginner. Anyhow, people should provide more context when asking technical questions. Your approach will get him to pass a standard interview for a DB dev role. However, if one wants to understand how to map a domain model to a DB model and gain knowledge transferable between roles and teams - learning how to model data properly is the only way to go. In the end, not every data storage is a SQL DB. When it clicks, everything else will be much easier.

To be more to the point: OP, if you just want to pass the interview take the advice from 0x54MUR41

</comment></comment><comment><USER5> If you want to learn more and morph into a more architectural role: Data Modeling Essentials, Graeme Simsion, third edition, 517 pages.</comment>


Euphony - A MIDI visualizer in WebGL - <AUTHOR> http://qiao.github.com/euphony/

<comment><USER2> Some quality issues, but fun nonetheless. Love watching this piece by Rachmaninov. It gives you an idea how freaking huge his hands were (and thus how difficult it is to play).

<http://qiao.github.com/euphony/#158>

</comment><comment><USER3> Audio is terrible and notes are wrong.

</comment><comment><USER4> The audio crackles and distorts like crazy here, painful to listen to.</comment>


Ask HN: Why do you want to be an entrepreneur? - <AUTHOR>

When I saw this question, my first thoughts were as follows.  Looking at these, not sure if they're exactly on point, but they sure do reveal a bit about my thinking.<p>* When I see a problem, I can't help but fix it.<p>* Inefficient processes piss me off.  I instinctively start thinking of better solutions.<p>* I love shortcuts and using system to my advantage.<p>* I get a kick out of being resourceful and coming up with duck-tape solutions.<p>* I enjoy doing a different thing each day and look forward to challenges and crises.<p>* I want to make the calls – more than that, and I want to be responsible for any bad decisions.<p>* I need to make a difference in the world.<p>[See also: http://euwyn.com/2009/11/12/why-do-you-want-to-be-an-entrepreneur/] <comment><USER2> Originally, simply to get food on the table.

Now after having done this for 20+ years simply because I can't imagine living life without this freedom.

It's addictive. There were years when I probably could have made more money by taking the jobs that were offered, but none of them was ever even close to tempting.

</comment><comment><USER3> I suck at being a cog in the machine and I don't have any ambition to stop sucking at it.

Where I have had responsibility for something and some freedom for how I handle it, the rubrics I use get generally better results than most of what I see around me. Having had very positive experiences with that, it's hard to accept the mediocrity of most other environments.

I would also be healthier.

I also think I would make a lot more money. My current income is inadequate to my needs so that matters.</comment>


Ask HN: I can't get a job and it is bringing me down, what is going on here? - <AUTHOR>

I am a software developer who moved to the USA recently from Ireland. I have a job at the minute, so I am not desperate yet. However, I have interviewed for various companies all over the country recently. I have yet to be asked a single technical question, and have instead been asked to constantly jump through hoops regarding riddles - which are often philosophical in nature.
Also, I am in New York - I have had issues with over five potential employers just not turning up to the damn interviews.
I also had one company tell me that my greencard isn't enough, and that they only hire citizens.
Maybe I should just go back home at this stage, I am never going to pass an interview that is 100% riddles, and I am never going to pass an interview where the employer doesn't turn up.
Does anyone have any input? <comment><AUTHOR> I should also add - I have had companies contact me to ask if I am interested, I say "Yes, lets talk more." and then I never hear from them again.

<comment><USER2> What kind of sw dev are you? Feel free to drop me an email -- firstname.lastname@example.com

<comment><AUTHOR> Hey Ryan,

Mainly, I code in Java

But I also have a ton of experience in the following : C, C++, C#, JavaScript, PHP

Some frameworks and technologies I am familiar with are : Spring, DropWizard, maven, CodeIgniter, Laravel Angular,Node,

Databases that I have experience with are : SQL, Mongo

Experience lies heavily in the Big Data, performance, high availability world (due to a financial past)

Other : FIX Protocol, Sharepoint development, Continuous integration & Unit testing, Environment automation

</comment></comment></comment><comment><USER3> Leave. Come to New Zealand.

We love the Irish, you'll fit right in and we have an abundance of Dev. jobs.

<comment><AUTHOR> I wish I could - but my wife is a American to the very core, refuses to leave! :(

<comment><USER3> Next best thing then. Work for a New Zealand company, but in the US.

[https://www.xero.com/nz/about/careers/](https://www.xero.com/nz/about/careers/)

Scroll down and you will see job listings and a whole section for the United States. Looks like there are even positions in New York.

<comment><AUTHOR> Hey! Thanks! I will hit these guys up tomorrow - Do you work there? If so, let me know incase they offer a referral scheme where you would get money if I get a job.

</comment></comment></comment></comment><comment><USER4> what languages and frameworks do you know?

<comment><AUTHOR> Thanks for getting in touch.

Languages :

Java (main language), C, C++, C#, JavaScript, PHP

Technologies :

Spring, DropWizard, maven, CodeIgniter, Laravel Angular, Node

Databases SQL, Mongo

Experience lies heavily in the Big Data, performance, high availability world.

Other : FIX Protocol Sharepoint development Continuous integration & Unit testing Environment automation</comment></comment>


Show HN: XgeneCloud – Instant REST and GraphQL APIs on Any SQL Database - <AUTHOR> https://github.com/xgenecloud/xgenecloud <comment><USER3> Wow this looks insanely feature-rich and with lots of docs and videos. Amazing!

How does it compare to Hasura (haven't used it but seems similar?)

<comment><AUTHOR> Thank you so much.

Hasura supports generating GraphQL on Postgres.

XgeneCloud supports generating both REST APIs and GraphQL APIs on all the following databases PostgreSQL, MySQL, MSSQL, SQLite and MariaDB.

The key difference with generated APIs are - with Hasura you would need an extra service running on your backend for model-validations & business logic. This can be a container/microservice or serverless functions. With XgeneCloud, the APIs are not in a blackbox but generated express.js API source code. This would mean adding business logic is as seamless as changing that CRUD function. Since we use express.js (Thank you TJ Holowaychuk) - it is easily extendable as per your needs with npm being a great ecosystem to add middleware, rate-limiting, logging and what not. Also to note, Hasura is on Haskell language (although powerful as a language it is not mainstream language to build your backends on)

In essence,

Backend with Hasura = database + hasura (for crud) + microservice/serverless functions (for business logic)

Backend with XgeneCloud = database + xgene's API framework (regular run of the mill MVCish code with familiar controller->service->model). This means it is quite easy to comprehend, debug and scale.

The webhook pattern employed by say Firebase/Hasura/Any-API-in-a-box is good for only really simple projects or prototyping. But essentially, we believe webhook pattern for even simple CRUD operations is a major anti-pattern plaguing backend frameworks these days. Not to mention, simple things like debugging, SQL transactions become really hard on databases. Having said above, hasura has realtime capabilities which we do not have at the moment.

Serverless Support : Our APIs can be deployed as serverless functions on any cloud platform (we use express.js - which amazingly works as Serverless App). This is a big leap - so there is no more Cloud Platform specific serverless code. We support AWS Lambda, GCP Cloud Function, Azure Function App, Alibaba Function COmpute, Vercel Now.

Database Client :

We ship a fully featured GUI Database based client (much like MySQL Workbench / PgAdmin) that works on all databases - this supports autogenerating up/down SQL schema migrations automatically. It has features to bookmark queries and very intuitive :)

We also ship a full featured API Client (much like postman/insomnia) to maintain your API collections and we also support performance test of APIs via GUI interface.

edit : sentence corrections

</comment></comment><comment><USER4> Interesting, [https://xgenecloud.com/demos](https://xgenecloud.com/demos) was useful.

will be adopting in few usecases soon

</comment><comment><USER5> Wow this looks really impressive. Seems like it's the author of another mysql to rest api [1]. The name is a bit clunky though, but that's the best worst problem to have.

[1] [https://github.com/o1lab/xmysql](https://github.com/o1lab/xmysql)

<comment><USER6> Thank you.

Yes, XgeneCloud started as a simple project (Xmysql) that was first showed on HN[1].

[1] Xmysql on HN : [https://news.ycombinator.com/item?id=15580914](https://news.ycombinator.com/item?id=15580914)

</comment></comment><comment><USER7> This looks very interesting; my understanding of the product from looking is at the website is that it provides a database client/editor, it autogenerates a serverless API from the database schema.

I have a few questions: how is authentication handled? And does the serverless model mean that a new database connection has to be established every time an API request is made (because connection pools can't be used)?

I can see from the docs that it has signup and signin endpoints, and that it provides some email verification as well. But do those tables get added to the DB? And where does the email get sent from?

<comment><AUTHOR> >> my understanding from looking is at the website is that it provides a database client/editor

Yes, XgeneCloud comes with a fully featured GUI Database Client for schema design and migrations. No more database design from cli or code.

>> how is authentication handled?

We use session based authentication.

>> does serverless model mean that a new database connection has to be established every time an API

Yes, if there are any improvements please suggest it. We were inspired by a commit[1] made to our old solution in Xmysql - which turned REST APIs to Lambda functions.

So, we went onto extend this Serverless function support for all Cloud Platforms (AWS / GCP / Azure / Alibaba / Vercel)

>> But do those tables get added to the DB?

Yes, they gets added to Database.

>> And where does the email get sent from?

we are using nodemailer for mailing and the smtp config can be updated in the corresponding config file for the environment within server/config/ folder.

[1] : [https://github.com/o1lab/xmysql/commit/3b279855a2562765975f7...](https://github.com/o1lab/xmysql/commit/3b279855a2562765975f7271490e4483254d7c96)

</comment></comment><comment><USER8> This is awesome!

Is there any way to exclude certain tables or customize what is generated in any way? It seems like it just creates a model for every single table in the database, no matter what. I would love to be able to customize the config to exclude certain tables.

As a side note: I was able to get the GraphQL server working for a small mysql database, but when I tried to hook it up to a larger database, it gave me this error: Error: Field "UagFbDataAggregate.min" can only be defined once.

<comment><AUTHOR> >> Is there any way to exclude certain tables or customize what is generated in any way?

Yes, for this you can create the project from our GUI Desktop tool to your existing database.

Then from CLI you can just create database models with below

      xc gen.apis --model

However, I see there is an issue with our production application and workflow.

I 've created an issue to track this.

[https://github.com/xgenecloud/xgenecloud/issues/3](https://github.com/xgenecloud/xgenecloud/issues/3)

>> Field "UagFbDataAggregate.min" can only be defined once

May be this happens when there is clashing column name with an existing column in table.

[https://github.com/xgenecloud/xgenecloud/issues/4](https://github.com/xgenecloud/xgenecloud/issues/4)

</comment></comment><comment><USER9> This seems really useful and even though I may never use it, the price is low enough that I could justify maintaining a Premium membership "just in case"

<comment><USER6> Yes, Thank you. By using this you will never have to write CRUD APIs ever. That's about 60-80% of work in backend.

Also it is absolutely free to use with all features.

</comment></comment><comment><USER10> Interesting! How would you compare this to something like Strapi (strapi.io)? I've been using them at some point last year and I remember the product was pretty good.

<comment><AUTHOR> Strapi, I guess, can create APIs when you create models via Strapi's own UI Modeller itself (much inline with traditional API frameworks) and it does not generate APIs automatically on existing database schemas.

With XgeneCloud, you can point to 'ANY' existing or pre-built database schemas and your APIs are ready in seconds. So XgeneCloud introspects the underlying database schema automatically and then generates models, controllers, relational controllers, auth & ACL!

And to stretch the fun factor a bit - you can also point multiple databases as input - we will still manage to generate APIs for all databases together :) - happens within seconds.

Strapi ships an UI to create-edit models, but then I am guessing it is not a complete Database Client. They are targeting headless CMS space - so their needs might be much geared to provide richer datatypes in content management rather than to provide a fully featured database client. Also, I am not sure on how they manage SQL schema migrations (if somebody can help answer that). With XgeneCloud, your schema migrations are automatically generated as and when you make schema changes - this makes the process of backend database design a really seamless and enjoyable experience. What this means is - <USER2> desktop tool can be used by any database admins to manage schema migrations easily.

And finally, XgeneCloud's APIs are also Hybrid Serverless functions : which means without changing a single line of code - generated APIs can be deployed as Serverless functions across any Cloud Platform (AWS Lambda, GCP Cloud Functions, Microsoft Azure Function App, Alibaba Function Compute and Vercel Now)!

Two major-major benefits from this :

1\. Your APIs also work as Serverless Functions.

2\. Then those Serverless Functions are not cloud specific.

So the same code will work on any Serverless Cloud Infrastructure. Your code is not locked in with a Cloud provider.

Plus, we have a built in API client - where you can debug and perform test generated APIs (much like postman)

So with XgeneCloud, you can cover all aspects of API lifecycle management (Database, REST API Server, GraphQL API Server, API Client and Serverless)

</comment></comment><comment><AUTHOR> We support following Databases :

• MySQL

• PostgreSQL

• MSSQL

• SQL Server

• SQLite

• MariaDB

API Framework also comes with a built in GUI Database Client!

• Point & Click Schema design & migrations

• Works on MacOS, Windows & Linux

Happy to answer any questions and take your feedback.

edit : formatting

<comment><USER11> Congrats on the release!

Quick heads up: The link on graphql schema generation and stitching seems to be broken. It points to a localhost URL.

<comment><USER6> Thank you! We are fixing this.

</comment></comment></comment><comment><USER12> Excellent, I'll be testing this tonight. FYI Looks like a little typo on your pricing page: "You can get DARK Theme for FREE by just getting your friends to VISIT your referral page. They don't to sign up!" Probably should read: "They don't even have to sign up!"

<comment><USER6> Excellent! Would be lovely to hear some feedback. :)

Thank you for catching the typo in pricing - we will be fixing this.

</comment></comment><comment><USER13> One thing I'm not finding in this or similar tools is how to do row level authorization. For example: imagine a "forms" table. Users can submit forms and see and edit "their" forms, but not those of other users. Users need GET/POST/PUT /forms based on a UserId field in forms.

A more complex scenario: employees can submit forms. Managers can see forms for their employees, but not those of other managers. Managers need GET/POST/PUT on /forms based on forms.userid AND a relationship defined in manager_employees table. Managerid is not stored in forms because it could change.

Almost every api I've ever written has something along those lines. How would that be accomplished in a tool like this?

<comment><AUTHOR> We provide simple role based ACL mechanism (toggleable from CLI/GUI) [https://docs.xgenecloud.com/en/v0.5/rest-apis/rest- acl](https://docs.xgenecloud.com/en/v0.5/rest-apis/rest-acl)

However, this simple ACL is not intended to encompass all business rules or hierarchies of every application. Hence we decided to provide each router (or controller) with its own middleware handler where these rules can be coded easily.

Explained in below diagram : [https://docs.xgenecloud.com/en/v0.5/rest- apis/rest-acl#workf...](https://docs.xgenecloud.com/en/v0.5/rest-apis/rest- acl#workflow)

</comment></comment><comment><USER14> I'm working on the same kind of thing but that gives you a typesafe React UI and REST API through code generation based in your existing database.

Notable features including filtering, pagination, and jwt-based authentication.

[https://eatonphil.github.io/dbcore/](https://eatonphil.github.io/dbcore/)

</comment><comment><USER15> Is support for NoSQL on roadmap?

<comment><USER6> NoSQL is on our roadmap.

Which NoSQL you had in mind ?

<comment><USER16> How about ElasticSearch? Kafka? Neo4J? Btw, I'm your first Twitter follower! ;-D

<comment><AUTHOR> Hey vladsanchez, thanks much for following.. you deserve a yes for all the above ;) (this is after checking our number of followers on twitter! haha.. folks please follow us at [https://twitter.com/XgeneCloud](https://twitter.com/XgeneCloud))

I kind of understand Elastic and Neo4j usecase.

Can you please give an usecase for Kafka - you can raise an issue detailing so in github.

However, I will have to admit that before agreeing new databases - we will be working to make existing databases concrete.

</comment></comment></comment></comment><comment><USER17> Related project: [https://sciencedb.github.io/](https://sciencedb.github.io/)

</comment><comment><USER18> Great work. We will definitely explore this to reduce our dependency on Firebase. This is definitely needed.

</comment><comment><USER19> Nice idea. GraphQL on SQL is a kind of need of the hour. I will check if we can use this in our next service.

<comment><USER6> Which SQL database are you using ?

It will take about two minutes to try

</comment></comment><comment><USER20> Really! Where do we check it?

<comment><USER6> Please click on the link below :)

Github : [https://github.com/xgenecloud/xgenecloud](https://github.com/xgenecloud/xgenecloud)

Website : [https://xgenecloud.com](https://xgenecloud.com)

Usage: takes just two minutes

       npm install -g xc-cli
       xc app.install
       xc app.open

       xc new <project_name>

</comment></comment><comment><USER21> hmm, nice idea. I could use it in my next project, may be for all hackathons!

<comment><AUTHOR> Yes, this will be perfect.

</comment></comment><comment><USER22> This seems useful, is it possible to do relational queries?

<comment><AUTHOR> Absolutely.

Example: consider `country` and `city` are two tables and are related. City has a foreign key pointing to Country.

In REST API framework : We automatically generate /api/v1/country/:id/city for REST APIs.

In GraphQL API framework :

{Country {CityList} } graphql schema will be automatically generated with resolvers as well.

So they work out of the box

<comment><USER23> Very impressive work, but I am not familiar with auto-API generators, so I have some basic questions:

What if the foreign key reference was not an parent-child relationship, instead was a one-to-many relationship between 2 first class entities? For example, Company (1) related to Contacts (many), but they should be accessed as /v1/companies/:id and /v1/contacts/:id.

What if there was a unique constraint on the foreign key reference, so that the relationship becomes a one-to-one relationship? What does the auto- generated REST endpoint look like?

How does it handle join-tables which implement many-to-many relationships? Does it generate the REST end point for the join-table instead?

How does it handle different subcollection semantics? In other words, sometimes we want to treat the entire subcollection as an atomic entity. Other times, we want to create, delete and update individual elements of the subcollection.

How does it support SQL query optimizations?

<comment><AUTHOR> Thank you for your input.

Good & insightful questions.

>> instead was a one-to-many relationship between 2 first class entities?

This can be treated as virtual relationship between 2 entities and can be linked together in our models. This has to be done manually for now. May be we can do better with our UI tool. I've raised an issue to track this - [https://github.com/xgenecloud/xgenecloud/issues/6](https://github.com/xgenecloud/xgenecloud/issues/6)

one-to-one relationship returns a list instead of an object as of now.

>> How does it handle join-tables which implement many-to-many relationships? Does it generate the REST end point for the join-table instead? In GraphQL : the schema is like a graph traversal so a join query is possible. In REST : no - this will have to be created manually.

>> How does it handle different subcollection semantics? Subcollections can be handled individually i.e in their own routes or in nested routes in REST framework. CUD operations for subcollections in GraphQL i.e nested CUD operations (mutations) are not possible.

>> SQL Query optimisation: N+1 query optimisations are done with data loaders.

Please let us know of any other questions.

<comment><USER23> Thanks for taking the time to respond. This is great information that points me in the right direction regarding the capabilities of the framework. I will probably need to actually use it to have more intelligent questions. I normally write my API endpoints by hand, and I am used to having complete control over the data shape and performance characteristics of the API, but I will keep this in mind in the future.

</comment></comment></comment></comment></comment><comment><USER19> Interesting one. I will check out for our project.

</comment><comment><USER24> deleted

<comment><USER24> >Mike Amundsen > @mamund > · > Aug 21, 2016 > remember, when designing your #WebAPI, your data model > is not your object model is not your resource model is n > not your message model

[https://twitter.com/mamund/status/767212233759657984?lang=en](https://twitter.com/mamund/status/767212233759657984?lang=en)

<comment><USER25> I was interested enough to dive deeper into this quote, but I could only find the slides related to the talk. I then found this blog post which deep dives into Mike's talk and provides some great examples, including the prior art on which his talk is based.

[https://medium.com/good-api/api-maturity- fb25560151a3](https://medium.com/good-api/api-maturity-fb25560151a3)</comment></comment></comment>


Ask HN: Which distributed database solutions would you suggest? - <AUTHOR> Hi guys,
One of my project, started in 2013 has grown beyond the point of self-healing. It was cool active-active sync on the webapp (behind HAProxy) and one high-spec'd DB Server (both web servers would connect to this db server). Current DB size is ~145GB (Based on backup, actual size might be more) and its postgresql (upgraded to 10).
Now the problem is that the database has grown beyond the point of ease. The constant need to start services, stop long executing queries (easily takes upwards of 20s for simple search). Higher ups have asked to look for Distributed solutions ( I am strongly against NOSQL while management is strongly against paid solution but they might give in after I tinker with some sort of trails / open source versions). I am looking for guidance from you guys as are you using any distributed database in production (with the library available in php for connecting and stuff obviously).
I have tried installing cassandra (after which I am strongly against NOSQL) and citusDB which has some very serious limitations on the community edition (their open source offering) and also in general. Some more searching results in Postgres BDR, Cockroach DB, TimescaleDB (but this doesnt partition on NON-TIME columns).
My requirement is:
* to have a distributed database solution in place which is horizontally scalable.
* Easy to add nodes, remove nodes without any downtime (sure I can accomodate some write-locks for setup)
* Have the ability to tweat replication factor. Ideally I would love to have replication = number of nodes i.e. each node has complete database. So that when there are simple queries, it doesnt have to do distributed queries (which make system slow) and when the query is a bit complex, it does distributed since data is available in each node.
* Best case: Some GIS based plugin / extension on the solution would be icing on the cake.
* SQL compatible, so that least of the application rewrite is required.
Please help .. <comment><USER2> There is no in-place replacement.

Most likely your app relies on database performance for a lot of queries, that can be fast only on local databases. This makes a serious rewrite necessary if you want to go with distributed solutions and requires knowledge and experience in distributed systems. Which takes a lot of time and that's why choosing distributed databases from the beginning instead of MySQL/PosgreSQL is so important.

Judging by your description, your best bet is probably pre-sharding. It's not a proper distributed solution, but it doesn't sound like you need one. It works like this: you partition your data into N shards, where N is two orders of magnitude higher, than the number of server you plan to have, then split each table into N tables belonging to each shard and use a static mapping of shards to servers in the application. Queries won't be able to cross shards. Some work on the app will be necessary and on the infrastructure tooling to manage servers, shards, replication, monitoring. As you would need to be able to move shards between servers, restart them, backup them and so on.

<comment><AUTHOR> Say suppose we go ahead with keeping the system as is and re-write it parallelly (essentially beginning) what distributed database would best help our use case. (Fast, distributed, ACID, transactional, no single point of failure, should have some GIS functionality)

</comment></comment><comment><USER3> You need more information on _why_ you need to constantly start services, and why there are long running queries.

The latter is actually one of the easiest problems to find a solution for. If your problem was too many concurrent transactions/high load, then that would involve distributed solutions. But slow queries? That is solvable with less work.

<comment><AUTHOR> The queries are slow because the resultant dataset is of high volume and then we are acting on the data. While this can be remedied (its being worked on) - I saw some demos on the web for distributed systems and think sooner or later I need to do that. Makes senses to do now

</comment></comment><comment><USER4> Try old school tricks first:

\- Explain your queries: you will be surprised to know a lot if stuff madre with ORMs is not well indexed.

\- Take a look at the slow query log (and run that queries with Explain before).

\- Exploit recency: I bet most reads are from recent data.. Or data that was written near other data you ask... If that is the case xonsidera partitioning the tables.

\- Thing about the best index type form your tables. Some are better than others.

\- If some data is not going to be written again, a columnar data store would help...

Don't go distributed at first.

Disributed means when something fails you have to know what abd where is failing... Instead of knowing just what.

If you want to go distributed.. Try sharding right now. If it works for you, add a cache and use the shards.

But try old school, boring stuff first because you might need it later.

<comment><AUTHOR> Will check what me/team can do on old school lines. Thanks

</comment></comment><comment><USER5> First make sure you can't solve your problem by moving to a more powerful server. 145GB is tiny. 256GB of DDR3 is less than $600 on eBay, and can fit your whole DB.

Also, are you on a SSD?

<comment><AUTHOR> I certainly could. Yes I am on SSD. But then moving to a bigger server is essentially just delaying the problem I am going to have to solve sometime in future. Also right now I can afford some time window of bad (slow) performance while I essentially migrate.

<comment><USER6> You'd be surprised how far you can push the problem into the future. OVH will sell you a server with 512GB of RAM for under a K a month. You can easily go into the terabytes.

<comment><USER5> Hetzner PX92 with 256GB DDR4 is just $219/month.

</comment></comment></comment></comment><comment><USER7> I'd actually be really interested to hear more opinions, especially with regard to scaling Postgres.

It seems like a few companies have actively been working on the horizontal scaling part of Postgres, along with the RDS version.

Would this be something where he could move his data to AWS RDS, change no code, and have everything 'just work' due to AWS dealing with the scaling of Postgres?

That seems to be their selling point, though I wonder if anyone has actually used it with this much data.

I'm really hesitant of using a NoSQl solution, too, for any projects just because most don't have transactions, don't have _real_ SQL APIs, etc.

</comment><comment><USER8> Redesign and fix your database. 9 times out of 10 it's poorly designed DB and queries. 145GB means nothing How many rows? Can you partition? Can you use smaller datatypes? Are you read heavy or write heavy? Are your indexes designed accordingly? Have you split your DB into OLTP and OLTA DBs? Do you have lots of triggers? Are you having the database doing lots of work which you can outsource to the CPU? Did you actually tune your DB parameters? Is the hardware scaled out max vertically? 145Gb is nothing.

</comment><comment><USER9> Stick to Postgres and investigate how you can improve things.

I guarantee you have not reached "max postgres capability", far from it.

You will 100% regret changing the database as a solution to your problem and you will buy yourself lots of other problems.

</comment><comment><USER10> bitcoin

<comment><AUTHOR> you mean some kind of blockchain type solution ?

<comment><USER11> With a pinch of AI

<comment><AUTHOR> any documents / example?

<comment><USER11> I was just adding buzzwords.

To be honest, if you haven't tried to optimize your hw/code/schema/indexes. I think I would start with hardware, then schema and queries.

I solved a problem with a database for a finance product, but the problem wasn't the database, it was abusing the ORM that made the resource usage go nuts. It made sense at the time of writing the software because the deadlines were tight, but should've been rewritten before the system started failing.</comment></comment></comment></comment></comment>


Michelle Carter guilty of manslaughter in texting suicide case - <AUTHOR> http://www.cnn.com/2017/06/16/us/michelle-carter-texting-case/index.html <comment><USER2> The charge of involuntary manslaughter seems odd; it was deliberate action, sustained over an extended period of time so not an act in the heat of the moment, with the clear and unmistakable intent of causing death which did lead to death.

If suicide doesn't break the legal chain of causation (which was a necessary legal theory for even a involuntary manslaughter charge and conviction), then it seems to be clear-cut first-degree murder.

</comment><comment><USER3> She was on suicide pills (so-called antidepressants) herself, but they didn't mention this until the last paragraph:

> Earlier in the trial, a psychiatrist testified that Carter was delusional > after becoming "involuntarily intoxicated" by antidepressants. She was > "unable to form intent" after switching to a new prescription drug months > before Roy's suicide, and she even texted his phone for weeks after he died, > the psychiatrist testified.

Other sites say that she was switched to celexa. The safest ant-depressants are the MAOIs. Last time I looked at Wikipedia's article, it said this class had been unfairly maligned.

</comment><comment><USER4> Interesting verdict.

How does this apply to a large group of people posting on someone's FB wall, or DMing them on Twitter, telling them to kill themselves?

Is everyone that sends a message or a post guilty?

</comment><comment><AUTHOR> What I find disturbing about the verdict is this: If the person found guilty of manslaughter then goes and commits suicide citing, in a suicide letter, that the words of the judge made her do it, would the judge be guilty?

Now, I know that lawyers protect their own and the judge might make the defence that the above argument is too far of a leap, but one could have argued the same for the initial verdict that just by the mere suggestion and lack of action one should be responsive for someone else's action.

<comment><USER5> It wasn't "mere suggestion".

She told this guy to kill himself repeatedly. She encouraged him to do it. She badgered him about it. After he did it, she didn't call the police to ask them to try to save him. She wanted him dead.

The judge did not tell this woman to kill herself, so your hypothetical situation isn't the same thing. There is no slippery slope here. Freedom of speech does not extend to harmful speech. It is similarly not legal to tell people to go out and commit acts of violence.</comment></comment>


Ask HN: what about this idea for a startup? - <AUTHOR>

I have suddenly come up with a silly idea in the middle of the afternoon. What if I create a startup based on this idea. Lets see how people react to this.
What if we create a site for restaurant deliveries. Scenario is like this: 1. I am going home from work 2. There is a restaurant X along the way from work  to home where it has food that needs to be delivered to customer Y. 3. It seems that customer Y's house is also along my route going home.
4. What if I can be the delivery man for this restaurant and earn $ for delivering it? 5. Benefit: restaurant pays less for a delivery service/labor.
Again, I just came up with this idea a couple hours ago. Some of you will think, what kind of moron I am to come up with ideas like this and such, but I think it seems interesting. There are still loop holes, such as what if I just go and run away with the food. There are ways to solve this, such as verifying so that only "good people" can participate as a delivery man. What do you guys think?
I am going to work on this idea for the Summer. If there are any hackers or UI/UX designer interested in joining me. Shoot me an email. You need to be awesome though! <comment><USER2> Interesting idea. I think of your idea as a general p2p delivery system comprised of a network of pick-up points and drop-off points, whereby a person is compensated for delivering an item from a pick-up point to a drop-off point.

p2p restaurant deliveries seems like a bad idea because the deliveries are too time sensitive (food gets cold relatively quickly).

Why not partner with UPS/Fedex as a way to reduce some of their costs? So driving home from work, you stop by a local UPS/Fedex pick-up point where you are given a package to deliver to Fred, who lives a mile away from your home. (The local pick-up points can established such that all package have associated drop-off points within a X-mile radius.) Or for deliveries between airpots, people can be compensated (i.e. offered a discounted ticket) if they carry-on a package to be dropped off at their destination airport.

With the rising gas prices, I suspect many people would _love_ to be compensated for going a few minutes out of their way during their regular commutes. Not to mention, the publicity this would generate is immense.

<comment><AUTHOR> Thanks for the input, will definitely think about it and incorporate it into the system

</comment></comment><comment><USER3> Interesting idea, but a few points:

-Who are you targeting? People who bike? Teenagers?

-The $ one receives is essentially the tip right, so is that enough to justify the slight inconvenience of picking up package, routing to destination, spending time at destination, then returning back home?

-Would you consider expanding this to businesses outside of restaurants that normally don't have delivery services (e.g. hardware stores, video game stores, etc.)

-How do you verify "good" delivery man? Track record? How do you establish good track record?

I'd be super interested to see how far you can run with this crazy idea lol.

<comment><AUTHOR> Yea...it's kind of a crazy idea I know, asking people to be delivery man.

-I am targeting mostly people with cars, maybe bike. As of age group, I am thinking of 18 - 40.

-Yes the $ is the tip..I think it will vary depending on what the restaurant/business is willing to offer. People can either take it or not. The system could give the business a recommendation at what rate people usually take the deal.

\- I am actually planning to expand it even further not just beyond businesses, but as a replacement of USPS, Fedex, UPS. Imagine about it, the market of people using this is basically people delivering/shipping things. I bet if shipping was made more social, it would be more effective than USPS. It just needs a system that coordinate all of this and that's what I am trying to do here.

\- To verify the delivery man, I was thinking of having the user to put a security deposit first (to cover the lost cost if someone ran away with the item) when initially registering. It will be hold after a user gets X amount of positive feedback and after that it's just reputation from the business/merchant.

One of my inspiration of this idea is based of my experience when going back from school. Almost every time I went back to my apartment from school, I always show this same pizza delivery guy all the time, I know his pizza store is located along my route of school - home. Then I had a thought that I would want to deliver it, won't cost me anything too much, it lies along my usual route, cost me probably 5 minute, and I might be getting $5++ each time. Not bad as an extra money for college kids right.

</comment></comment><comment><USER4> Honest feedback alert: I would never use the service because I don't know what the other person might put in my food or mess with it, and I would never provide this service because the margin is too low and/or I don't want to knock on a stranger's door. That bing said there a lot of crazy things that became wildly successful so if you truly believe in it and you can actually see yourself getting the service and providing it happily then do it.

<comment><AUTHOR> Thanks for the honest feedback. I agree with that, you even don't know what they put in your food when you order food at a restaurant right? Not until someone told you what's going on back at the kitchen. But again, the incentive is there. Might still need to think about it more throughly for any loopholes.

<comment><USER4> You're right, we have no idea what is being put our food in the kitchen but the restaurant has for many years possibly tried to establish its brand and reputation as a trustworthy establishment, one idiot with nothing to lose could ruin it. Why would they be willing to take the risk if it does not affect their bottom line or save the customer scrilla? But please do not be discouraged by my comments, I am just thinking it through with you...

</comment></comment></comment><comment><USER5> cool idea. you could have some 'ebay rating' style thing to rate the delivery people that could probably result in something reasonable happening. Maybe let the delivery person report paid/didn't pay and nothing else to prevent the reciprocal feedback problem ebay had... that, or just pay through the website, then there is no question of the delivery person getting paid.

You'd need to make sure you had a solid 'independent contractor' deal with the delivery people; I don't know how hard that would be to do.

I don't think you'd be stepping on the taxi people, so I don't see any entrenched party who would want to kill the idea (except maybe pizza places that have already solved the problem, but only for themselves?)

I eat a lot more pizza than I would otherwise, just 'cause it's the only thing that delivers, so if you can figure it out, I think it's a great idea.

<comment><AUTHOR> Yea and to think about it, it can be expanded to other stuff as well. Not just restaurants. Imagine this:

1\. I am located at LA, have a friend at New York who has something that he needs to give me.

2\. A person named Adam is on his way back to New York to LA.

3\. Why not let Adam take that something from my friend in New York and bring it back to LA with him?

4\. Waiting/paying for USPS, UPS, Fedex, is too long and costly. This is better

<comment><USER6> I can just imagine trying to explain to the TSA agent at the airport that I was carrying someone else's package. I'm sure something could be worked out.

<comment><AUTHOR> Yea, that's also one prevention that needs to taken care off in the system. The user should make it clear what package is being carried and have the permission to open the package, if necessary. I don't want this to be a drug smuggling facility in the end.

</comment></comment></comment></comment><comment><USER7> Interesting idea. There is something similar to this in India although only available for one city at the moment. Check it out may be you can get some useful insights <http://www.deliverychef.in/>

</comment><comment><USER8> Most takeaway orders happen at a different time of day from when most people are going home from work.

Unless the restaurant can use this system for a substantial proportion of their deliveries, it will be more hassle than it's worth.

<comment><AUTHOR> I am not only considering it just for restaurants, as explained on my reply below.

<comment><USER8> The "substantial proportion" bit still holds for other businesses.

For general package delivery, what happens if the recipient is not in and the package cannot be delivered? The courier will not want to turn round and take the package back to the depot. If they take it home with them, they have a package they don't want taking up space in their home.

How much do individual couriers earn, anyway? How much money are you trying to save for customers?

<comment><AUTHOR> That can be solved by having some rule:

1\. Package not picked up after X amount of days are discarded, unless a communication between the courier and receiver are made

I am trying to save them at least not getting a higher price than the regular USPS but faster delivery time. That is the goal

</comment></comment></comment></comment><comment><USER9> Irrelevant: are you indonesian by any chance? Cause I am :)

<comment><AUTHOR> yes sir! student in the US! and you?

<comment><USER9> Me too, we should really chat and talk, I would love to exchange ideas, being a one man band gets really tough sometimes :P anyway i can contact you?

</comment></comment></comment><comment><USER10> Cool idea in that it's more efficient than the status quo.

<comment><AUTHOR> it's cool, but not sure if the traction is enough.. this kind of worries me</comment></comment>