Show HN: We structured and compared hospital chargemaster prices - <AUTHOR> http://pricemed.org/procedures/transfusion-of-blood-or-blood-products <comment><USER2> This is the kind of data that is needed for true transparency, and true medical reform. Prices will be gouged so long as you can't compare; Realistically, this still has a long way to go (Because I'm certain that a lot of these differences are in part due to one procedure being billed as many different codes under the current US regulatory mess), but it's a good step up from what we've had available in the past.

<comment><USER3> The only thing that's going to matter in effectively reforming medical cost structures in the U.S. is getting a critical mass of people who are shut out from medicine due to the insurance scam. While there are presently _some_ people who are in a bad position, it's not enough people to really mount a meaningful united front. Sure, people are upset, but not enough people are upset in the right way to really make policymakers fear for their careers and/or lives.

Huge swaths of "money" in the medical industry are the result of paper pushing, pure and simple. No one is spending their own money, or even really "real" money: the doctor's bill will say something outrageous for the cash price, because they know the insurers will demand a large reduction, and even though they write "$40,000" for something, they don't actually expect anyone to pay that directly. They expect you to either call and get a "cash payment discount", i.e., something close to what they charge insurance companies (which is still really high, because insurance companies aren't spending "real" money either), or to use your insurer to "pay" the bill.

Remove the lecherous middle-men who do nothing but shuffle numbers around and we can start making progress on medical pricing. This is not the ONLY cause of high costs, but it is easily the largest and most perverse problem point, because this type of billing completely destroys all conventional rules of supply and demand.

Interestingly, the Affordable Care Act does the opposite of this, and strengthens the lecherous middle-men to the point that it will be illegal not to give them money (the government will forcibly take the money from you in the form of a "tax penalty" if you do not voluntary sign up for "medical insurance").

<comment><USER4> This is a generalized problem with insurance, tho. Ie, if you had a likely expense, you would plan for it (house, college, etc). You buy insurance precisely because you don't think you'll need it.

So, the people that need to use it are always going to be a small group. The people that don't have it, another group. The people getting screwed (on the sharp end) are the subset of those where there is overlap. This will always be a small number (by math).

That being said, for everyone is paying into the system but its more death by thousand cuts. As long as pricing is opaque and uncorrleated with out-of- pocket marginal payments, there will be a problem. The cleaner solution is simply to remove the facade of pricing a-la-carte services.

The problem there, is two fold: (1) how to control the inevitable buracracy with a huge budget making life or death decisions; and (2) how to incentivise research and science/product development without the cash cow of monopoly pricing (patents are useless if selling is illegal).

While these problems seem to be solvable, they are daunting and not simple. In a world where the NSA is spying on every electronic communication over the internet, perhaps even more so. Ie, a world where privacy and trust is being undermined by those seeking to entrench their own power. Its harder poltically to delegate more responsibility to the "government".

<comment><USER3> I agree that this is how insurance works. Normal insurance is a hedge against anamolous disaster. Medical care is something that practically everyone will need at one point or another, i.e., it is not an anamolous disaster, and therefore insurance is a problematic billing model from the get-go.

We should start by saying that insurance is often considered a perverse industry because their incentives are inverted. Normal businesses make money by providing a product or service to clients, and the incentives are therefore aligned. Insurance makes money by explicitly NOT providing a product or service in exchange for your money; if they have to give you the thing you've paid for, they are less profitable. The more frequent claims against a particular insurer are, the more incentive they have to make things difficult, because some people will ultimately give up and that'll be one less (or lessened) payout that the company must supply. As medical insurance policies will likely have several claims per policy per year, they have an incentive to be the worst type of insurance.

Most people with home insurance will never use it, because their house is not likely to get robbed, or catch on fire, or encounter even more esoteric occurrences. Considering the multi-hundred-thousand dollar investment most people have in their homes, a nominal fee of a few hundred bucks per year is entirely reasonable to protect that asset.

The opposite is true of medicine. Most people will get medicine sometime, and it's not uncommon to need access to medicine several times per year (even excluding the chronically ill). Even car insurance will often go several years without being used, but not medical, especially not if you have a policy that covers multiple people (policy holder + spouse + dependents).

Insurance simply _does not work_ for services that are commonly required, and that's what it boils down. There is a major feeding frenzy operating on the backs of the nation's health, and everyone is pointing the finger at the other entity, and again, there's more to it than _just_ insurance, but insurance is now and always will be the most major cause of problems in this industry. The only way to bring it back to sanity is to expose it to true market forces; the clientele must spend their own money to receive medical services, and therefore medical providers must set prices within an affordable range or go out of business. The leeches who do nothing but sit there and push paper must be removed, plain and simple. No amount of whining about how it will put some people out of work, etc., can be tolerated on this. The fact is that insurance is a very, very bad thing to have so deeply ingrained in our medical system, and we won't see major improvement until those people go away.

Other countries have band-aided it by saying "OK, I see meidicine 'costs' inordinate amounts of money, we'll just write what is effectively a blank check to the industry", but that's not how we should do it in America. We must accept that market forces must dictate the prices, that hospitals the size of universities are probably not plausible, and that the whole industry must be brought back down to earth. The industry is obviously not going to like that, but it must be done. We need some ballsy politicians who can make it happen.

<comment><USER4> _the clientele must spend their own money to receive medical services_

The flaw / issue with this, is that sick people are in no position to negotiate. People that are sick are desperate for the drugs. People that are not sick dont really need them. that is ill suited to the simplification required of econ 101. In econ 101 you want the price to converge to the marginal cost of supply, not converging to the marginal amount of residual equity / net worth of the patient's bank account (that's the hold up problem: give me all your money or you die). Imagine if you had to negotiate the price of a fire-department visit with your house burnging? Its just not a service that is ever going to fit the framework (in particular, the behaviour assumption of non-opportunim) of the typical econ 101 course.

That being said, I agree with your analysis of the insurance model when applied to "scheduled maintenance" type services. Opaque pricing and a culture of witholding service is particulary innappropriate here. Assuming that chronic and life-threatening services are another matter, that still leaves the issue of everything in-between. The issue there is that there are life- style type elective surgeries and treatments that some may want access too (the rich) but not be willing to subsidze for others (the poor). So there is a basic triage of the types of health care, and they all need to be thought through from a sociological perspective as well as just an economic and medical/technical one.

</comment></comment></comment></comment></comment><comment><USER5> This is a fascinating data problem. I noticed a lot of the price distributions are non-normally distributed (specifically with a strong positive skew for the least consistently priced procedures). You guys should have fun with this.

Who will be the primary users of this information? It seems that insurers already are aware of chargemaster prices and ignore them since they're going to be bargained away.

Employers seem not to care to dig into how much their insurers are paying (employers just care what premium they'll have to pay).

Finally, it seems like consumers can't really use this data since they're pretty much locked into their plans anyways and will go wherever is most convenient.

Now given changing consumer behavior, I could see consumers maybe using this so they have to pay less of a deductible, I could also see insurers working with consumers to steer them to the best prices - saving both the insurer and the consumer money. However, this will require overcoming consumer attachments to preferred providers (a challenge for older adults, not so much so for younger ones who often don't have a relationship with a doctor). It can be how surprising how quickly people can overcome sentimental attachments when their money is on the line though, so who knows.

</comment><comment><USER6> Background on the chargemaster story that broke in Time Magazine. (So glad it was covered by The Daily Show or I would have missed it): [http://healthland.time.com/2013/02/20/bitter-pill-inside- tim...](http://healthland.time.com/2013/02/20/bitter-pill-inside-times-cover- story-on-medical-bills/)

<comment><AUTHOR> Here's the Jon Stewart segment: [http://www.thedailyshow.com/watch/mon- may-13-2013/bi-annual-...](http://www.thedailyshow.com/watch/mon- may-13-2013/bi-annual-competency-round-up)

Some other links to more info are at the footer of the page. There was an interesting US Senate hearing with Steven Brill, the author of the Time article.

</comment></comment><comment><USER7> Love this!

If there's some unstructured data that you're having trouble taming, contact me. I have a ton of experience with Mechanhcal Turk and can give you some advice on how best to structure your HITs to extract the data that you need.

email@example.com

</comment><comment><USER8> One suggestion: The dropdown for "Find a Hospital" needs to show the location too. I tried searching for "St. Anthony's" near my town and multiples ones from all over the country showed up. I put in my zip code but it did not filter the results. I have no idea how to look for St. Anthony's near 33708 or not. I don't know if it is St. Anthony's Medical Center or Health Center or Hospital. But there's only one St. Anthony's here in St. Petersburg, FL.

<comment><USER9> When I entered a zip code, it showed a map w/ nearby hospitals. Perhaps they have no data from yours?

</comment></comment><comment><USER10> Some tweaks to the design would help!

At the top of the page, the search boxes are mis-aligned to the text above them. Fix that. They could also take up more horizontal space, or simply be aligned better.

Add an extra ~3-10px of horizontal padding to table cells (td's) Table headings should be text-align: center except for the longest field (hospital). Also, vertical-align: middle on table headings.

Link color on the tables could also be darker.

Add left-margin/padding on the (?) question mark icon at the top right of fields.

Font size on the search drop downs could be bigger.

May want to try the font "Open Sans", too, on the tables.

Add a hover effect for all links and buttons. It's kind of nuts that you could overlook that.

Add a focus effect for all inputs. Also make input font color brighter, and on focus, make font color darker.

On the "worst hospitals" table, the percentages could use a + sign before them like "+97%".

</comment><comment><USER11> It looks like the sorting is off. It's saying that Cedars-Sinai is the worst priced hospital for the linked procedures, but at "only" a 99% marked over the average $1,300 even though there are many in the $3-5.2k range.

</comment><comment><USER12> This is all great, but cost isn't the only metric, there is also such things as readmission rate &c. Two weeks ago or three a fellow, a physician, made the front page of HN who listed prominently listed _prices_ for his hospital, but you had to go digging for readmission rates.

That guy was on a mission, and his mission wasn't to improve public health. If his mission was that he would have listed the other important metrics. He knows all about them, he is a physician, after all. This sort of dishonesty I cannot stand.

<comment><USER13> It seems you're implying malevolence and unless the other metrics substantially were worse than the other hospitals.

<comment><USER12> No one knows about his motives except that person himself, but it is indisputable that a public figure such as he has responsibilities to society, and he does not fulfill them.

</comment></comment></comment><comment><USER14> No one pays chargemaster prices. The reason hospitals set the prices so high is because they are the starting point for negotiations with networks and insurance companies. Even consumers without insurance typically don't pay chargemaster prices as hospitals normally have cash pay discounts. As payers begin to contract with ACO's, I think we'll likely see a shift to a more standard shape bell curve if reform is working.

</comment><comment><USER15> Great idea! There seems to be a minor flaw in the data though, the Healdsburg District Hospital appears do be doing many procedures for 1$. [http://pricemed.org/hospitals/healdsburg-district- hospital](http://pricemed.org/hospitals/healdsburg-district-hospital)

</comment><comment><USER16> Really cool!

One thing - the google map always sets California as the state, even when I look at hospitals in other states.

</comment><comment><USER17> Isn't the chargemaster price mostly unrelated to anything? [http://runningahospital.blogspot.com/2013/05/useless- noise-f...](http://runningahospital.blogspot.com/2013/05/useless-noise-from- cms.html)

</comment><comment><USER18> I'm confused... This is assuming a procedure in facility A by Doctor B is a commodity and is no different than the same procedure in Facility C by Doctor D. Am I missing something?

<comment><USER19> For many procedures, that's true so long as they're competently executed, which we should expect so long as the doctor is licensed to practice.

To really make buying decisions for your own health care procedures, you would want user reviews in addition to prices, just like you see on every online storefront.

</comment></comment><comment><USER20> I love it, thanks for sharing. Where did you get the original data from? (I thought chargemasters were relatively guarded secrets?)

<comment><USER21> raw California chargemaster data was in excel files at this gov't website: [http://www.oshpd.ca.gov/chargemaster](http://www.oshpd.ca.gov/chargemaster)

outside of California, the data from a release by Medicare: [http://www.cms.gov/Research-Statistics-Data-and- Systems/Stat...](http://www.cms.gov/Research-Statistics-Data-and- Systems/Statistics-Trends-and-Reports/Medicare-Provider-Charge- Data/Inpatient.html)

</comment></comment><comment><USER22> Very cool. I wished this would be done for hospitals in .NL as well.

</comment><comment><USER23> Very cool. Does anyone pay the chargemaster price though?

<comment><USER24> No, no one pays that.

And I don't think the chargemaster price is even correlated with the final price, so it's not even useful as a relative measure.

I wish it was a real price though - maybe this site could push hospitals that way. But I doubt it because the reason hospitals are able to afford Medicare prices is only because they overcharge the rest, and the prices are made opaque specifically so that that people do not complain about that.

The only way to make transparent healthcare prices is to raise medicare prices so that everyone pays the same, but that'll never happen. So what happens is that insured people pay an extra premium to help out the rest.

Airlines do the same thing, and they too have very convoluted hard to understand prices.

<comment><USER25> Yeah, I get the feeling one could fix healthcare overnight by allowing any private person to pay cash at Medicare prices, or a low multiple (1.5x?). I guess there would still be prescription drugs and devices to deal with, though.

<comment><USER26> Another idea: make it illegal for anybody but the patient to pay for their healthcare. Insurance pays the patient, who then pays the hospital. If the patient doesn't like the service, or feels they're getting ripped off? They probably won't pay as much.

<comment><USER27> Bush or Clinton, I forgot who basically was pushing that. Create high deductible plans, add tax benefits attached to them (tax sheltered health saving accounts) so that people are more directly involved in shopping for prices and it was being sold as that would drive the prices and competition to get better.

<comment><USER25> Yeah, I think the HSA was one of G W Bush's best accomplishments (and, probably, one of the only net-positive accomplishments other than helping in Africa somewhat), which is why I'm sad the new health reforms are basically incompatible with the HSA concept.

As I see it, there are basically two separate ways to make healthcare work -- expose consumers to prices AND give them the tools to make decisions (with subsidies, etc. if we want to let poor people consume more health care than other things, as a matter of policy; I don't think that's an economic argument), or have an external entity with the correct incentives acting on the consumer's behalf. Medicare seems to mostly work on the latter model.

It's clearly possible to screw up both ways, but an even surer way to screw up is to do neither/both-partially.

Ending employer deduction for healthcare (and/or making it deductible for individuals) would be a huge step, too (in either model). I see no actual upside to employers paying for health care vs. individuals or the state at all.

</comment></comment></comment></comment></comment></comment><comment><USER28> Lung transplant: 0 results

<comment><USER29> [http://pricemed.org/procedures/transplant-of- lung](http://pricemed.org/procedures/transplant-of-lung)

<comment><USER28> Didn't see that earlier - and there's only one listing. $413?? More like $143K.</comment></comment></comment>


Revealing Errors by Benjamin Mako Hill and Aaron Swartz - <AUTHOR> http://revealingerrors.com/

<comment><USER2> Oh yay, yet _another_ brand new Swartz project. Bets on how long this one lasts?</comment>


Free marketing KPI dashboard for early stage companies (g-doc and XLS) - <AUTHOR> http://www.signed.vc/2014/01/e-commerce-marketing-kpi-tracking-an-example-dashboard/

<comment><USER2> Very cool, thanks for sharing!

<comment><AUTHOR> sure, glad u like it.

</comment></comment><comment><USER3> very uselful!

<comment><AUTHOR> thanks</comment></comment>


What Happened When We Took the SCiO Food Analyzer Grocery Shopping - <AUTHOR> http://spectrum.ieee.org/view-from-the-valley/at-work/start-ups/israeli-startup-consumer-physics-says-its-scio-food-analyzer-is-finally-ready-for-prime-timeso-we-took-it-grocery-shopping <comment> xkcd-sucks A few problems with scio:

\- It uses machine learning models on 10-datapoint IR reflectance spectra, meaning it's only useful in a trained regime. It doesn't give information about composition, but instead classifies a sample as a member of a pretty constrained population. So a mystery substance that can't be roughly identified ('vegetable', 'pill' etc) can't be scanned, etc.

\- So, evaluating drugs ("we can distinguish fake from real viagra") is done by looking at the surface coating which typically has no active ingredients (and even if there were, the signal would be washed out by inactive ingredients). The model is basically trained on 10-100 scans of a presumed good viagra pill, or maybe 10-100 different good viagra pills if they felt like it

\- Building models requires purchase of a $250 license in addition to the $250 hardware, which is just ridiculous. Of course they're doing the calculations on their servers, but it still seems really scammy, hostile to developers, and counterproductive to launching an ecosystem of scanning models. The useless 10-point IR "spectra" notwithstanding, I would totally buy one of these if you could use open data and open models supported by a public community</comment>


The Definitive Guide To Website Authentication - <AUTHOR> http://stackoverflow.com/questions/549/the-definitive-guide-to-website-authentication/ <comment><USER2> From the first answer

> The only (currently practical) way to protect against login interception > (packet sniffing) during login is by using a certificate-based encryption > scheme (e.g. SSL) or a proven & tested challenge-response scheme (e.g. the > Diffie-Hellman-based SRP).

I'm no security expert but if you take into account man in the middle attack where the attacker can actually change the page/javascript you're not safe even with DH. Attacker can just drop all your fancy code and send the password in the clear and intercept it later...

<comment><USER3> I'm a security professional focusing on exactly this issue and you are exactly correct.

It is, _notably_ , not enough to safeguard the JS code with HTTPS/TLS (which, if you do that, SRP is of virtually no utility anyways). To protect the Javascript runtime, you'd also need to ensure that no element of the page was ever cached from a non-HTTPS connection, that no element of the page is currently fed from a non-HTTPS connection, and that the page is free from any and all DOM or CSS corruption issues.

Obviously, you'd also need to implement SRP safely, which very few people do; more than half of the SRP implementations I've ever tested have been trivially bypassed due to math flaws.

SRP is a terrible suggestion for generalist devs and for authentication in a web setting. If you want to do something advanced, go two-factor; I like Duo Security, which will be easier for you to integrate than SRP anyways:

<http://www.duosecurity.com/>

<comment><USER4> Why would I want to use this instead of using Google Authenticator with RFC4226 or TOTP (<http://tools.ietf.org/id/draft-mraihi-totp- timebased-06.txt>) ?

Looks like Duo has more options for receiving the code as well as the ability to enable manual approve/deny for another user role. It also looks like a private service I have to rely on though.

<comment><USER3> It's not tied to a Google account, it has a better end-user experience, a better developer experience, and an extremely credible team behind it. That said, if you were using Google Auth and I assessed your application, I wouldn't ding you for using it.

<comment><USER5> Google Authenticator isn't tied to a Google account, and developing for it involves implementing a single algorithm.

</comment></comment></comment></comment></comment><comment><USER3> I'm just a few paragraphs in and already this definitive guide is getting you flunked in audits. It's not a _question_ whether you should disable autocompletes on password fields; it's an audit point.

(It is, as an aside, also a terrible idea to ask users to store their passwords in browser autocomplete dictionaries.)

<comment><USER6> I'm confused. Aren't password fields treated differently than say, my zip code, when stored by browsers? Firefox and Chrome have pretty sophisticated credential storage facilities with syncing and master password protection.

Can you really even call browser password storage "autocomplete"? Doesn't the browser see a password field and use the credential storage mechanism?

<comment><USER7> I suppose in OSX passwords can be stored to the keychain which is encrypted, but really, how many people just allow their browser access to their keychain without ever prompting them after the first time ?

It's possible to simply "show password" for any of the stored sites using your browser's settings menu. At least for Chrome on Windows this is the case.

</comment></comment></comment><comment><USER8> I'd be interested in hearing how other people on HN implement persistent logins. I'm currently working on it, following "Charles Miller's Persistent Login Cookie Best Practice"[1], but hearing some other opinions/techniques would be great.

[1] [http://fishbowl.pastiche.org/2004/01/19/persistent_login_coo...](http://fishbowl.pastiche.org/2004/01/19/persistent_login_cookie_best_practice/)

</comment><comment><USER9> It would be nice to see something about federated security using something like SAML - if you are a large enterprise and you have multiple SaaS providers you really want external services to be using your own internal directory (which is likely to be MS Active Directory) for authentication.

Actually, it would be pretty nice if someone could offer this as a easy to use service for people building enterprise oriented SaaS apps. Is anyone already doing this?

<comment><USER10> As far as I know, Nordic Edge (<http://nordicedge.com/>) provides solutions to quite easily make your SaaS app SAML enabled to login via a remote AD/IDP, but all their products are proprietary.

</comment></comment><comment><USER11> I posted this on the StackOverflow page. I'll put it here too. Apologies if it is too long.

I think BrowserID[1] (or perhaps more precisely, the Verified Email Protocol[2]), the recent effort by Mozilla's Identity Team[3], deserves a mention here. I'll summarize it this way:

1\. Mozilla is a nonprofit with values[4] that align well with finding good solutions to this problem.

2\. The reality today is that most websites use form-based authentication

3\. Form-based authentication has a big drawback, which is increased risk of phishing. Users are asked to enter sensitive information into an area controlled by a remote entity, rather than an area controlled by their User Agent (browser).

4\. Since browsers are implicitly trusted (the whole idea of a User Agent is to act on behalf of the User), they can help improve this situation.

5\. The primary force holding back progress here is deployment deadlock[5]. Solutions must be decomposed into steps which provide some incremental benefit on their own.

6\. The simplest decentralized method for expressing identity that is built into the internet infrastructure is the domain name.

7\. As a second level of expressing identity, each domain manages its own set of accounts.

8\. The form account@domain is concise and supported by a wide range of protocols and uri schemes. Such an identifier is, of course, most universally recognized as an email address.

9\. Email providers are already the de-facto primary identity providers online. Current password reset flows usually let you take control of an account if you can prove that you control that account's associated email address.

10\. The Verified Email Protocol was proposed to provide a secure method, based on public key cryptography, for streamlining the process of proving to domain B that you have an account on domain A.

11\. For browsers that don't support the Verified Email Protocol (currently all of them), mozilla provides a shim which implements the protocol in client- side javascript.

12\. For email services that don't support the Verified Email Protocol, the protocol allows third parties to act as a trusted intermediary, asserting that they've verified a user's ownership of an account. It is not desirable to have a large number of such third parties; this capability is intended only to allow an upgrade path, and it is much preferred that email services provide these assertions themselves.

13\. Mozilla offers their own service to act as such a trusted third party. Service Providers (ie Relying Parties) implementing the Verified Email Protocol may choose to trust Mozilla's assertions or not. Mozilla's service verifies users' account ownership using the conventional means of sending an email with a confirmation link.

14\. Service Providers may, of course, offer this protocol as an option in addition to any other method(s) of authentication they might wish to offer.

15\. A big user interface benefit being sought here is the "identity selector". When a user visits a site and chooses to authenticate, their browser shows them a selection of email addresses ("personal", "work", "political activism" etc) they may use to identify themselves to the site.

16\. Another big user interface benefit being sought as part of this effort is helping the browser know more about the user's session[6] -- who they're signed in as currently, primarily -- so it may display that in the browser chrome.

17\. Because of the distributed nature of this system, it avoids lock-in to major sites like facebook, twitter, google etc. Any individual can own their own domain and therefore act as their own identity provider.

This is not strictly "form based authentication for websites". But it is an effort to transition from the current norm of form based authentication to something more secure: browser supported authentication.

      [1]: https://browserid.org/

      [2]: https://wiki.mozilla.org/Identity/Verified_Email_Protocol/Latest

      [3]: http://identity.mozilla.com/

      [4]: http://www.mozilla.org/about/manifesto.en.html

      [5]: http://www.w3.org/2011/identity-ws/papers/idbrowser2011_submission_10.pdf

      [6]: https://wiki.mozilla.org/Identity/Verified_Email_Protocol/Latest-Session

<comment><USER3> It is a little frustrating that, when the Stack Overflow community attempts to come up with a "definitive guide to website authentication", people inside and outside the community see the attempt more as a coatrack for their own blue sky proposals than as an opportunity to codify best practices.

The fact is that BrowserID, like SRP (which the SO community itself inexplicably waded into), is _not_ an authentication best practice. It's a proposal that doesn't even have widespread support in the deployed software of its sponsor. Even without getting into the pro's and con's of BrowserID, the simple fact is that if absolutely everything broke Mozilla's way, it would still be many years before a significant number of users would be using it securely.

I really don't want to debate BrowserID here, most particularly because I don't have a strong opinion about it. But isn't it weird how even the Stack Overflow people themselves can't look at a challenge like this without spinning off into deep space on sky-pie crypto answers?

<comment><USER11> I've been frustrated for a long time by the lack of browser support for strong authentication with a good user experience. Form-based authentication always felt like a terrible hack. That's the only dog I have in this fight.

I certainly didn't want to frustrate anybody by posting what I did. If you like, I will delete what I posted on the StackOverflow page. Just say the word.

<comment><USER3> No, I'm just ranting, because HN is where I rant. It's good that you care about this stuff. If anything, it's the crazy SRP stuff in the original proposal that really sets me off. It is, in the end, frustrating that nobody (myself included!) wants to buckle down and write up all the boring details of how to do secure authentication _today_.

<comment><USER11> SRP is cool stuff, but if we're going to do SRP I would be more interested in doing it right.

I think that would mean having browsers support a standard SRP protocol binding (eg <http://www.ietf.org/rfc/rfc5054.txt>) and then really nailing the browser user interface.

I could see using SRP as an alternative to private keys, or as a strong "something you know" factor in addition to private keys (a "something you have" factor) in order to establish ownership of an account.

However, this doesn't strike me as a credible first step in getting through the aforementioned "deployment deadlock" issue.

</comment></comment></comment></comment></comment><comment><USER12> You should look at two-factor authentication solutions to solve many of these issues. For example <https://www.mepin.com> is a two-factor authentication service free to integrate and use for the websites. It supports standard OpenID interfaces, so no proprietary integrations needed.

</comment><comment><AUTHOR> What about "Keep me logged in for x days"... any opinions on it? It certainly has its own fair share of security issues, if implemented incorrectly. Given that more sites now allow logins via FB, G+, Twitter and what have you, is this still relevant?

</comment><comment><USER13> This is an admirable hack of SO's intended use. It would be great if there was a site that actually afforded this particular use case. Wikis are too open- ended. Perhaps something in the middle.

</comment><comment><USER14> There is also authentication using RSA keypairs, ALA glbse.com, they require all requests to the server to be signed by the private key (there is a JavaScript and Python client).

<comment><USER3> Javascript RSA is not a good thing to suggest to developers looking to improve the security of their login system.

<comment><USER14> Because?

<comment><USER15> I'm pretty sure Thomas is tired of explaining why again and again, but short version is that the javascript environment (in a browser) is such that one can quite reasonably claim that it is impossible to implement crypto securely in it.

<comment><USER14> This argument again.

</comment></comment></comment></comment></comment><comment><USER16> 8 out of the 12 issues posed are about passwords. Clearly we need a better alternative there that could be incorporated into the browser. I think mozilla's browserid is a very pragmatic idea that deserves some attention.

<comment><USER17> How many username/password forms are there on the web? Millions? They function basically identically, but they all had to be written (or at least configured) independently. It's outrageous when you think about it, as if every language required you to write your own string library before you start coding.

The system smells; the problem is being solved in the wrong place.

The sad thing is that 15 years ago this could have been solved easily. Now any solution has an uphill struggle.

<comment><USER3> How could this have been solved easily 15 years ago?

<comment><USER17> A decent UI on top of HTTP authentication would have been an improvement, and it's something that more sophisticated systems can be built on top of while maintaining compatibility.

<comment><USER3> No, a better UI HTTP authentication would have been a setback.

HTTP auth doesn't make backend authentication code easier to write. In virtually all apps, it doesn't obviate the need to do session management, or solve the user demand for "remember me" cookies.

What it does do is shoehorn applications into a single inadequate login UX, one that (almost at the protocol level) is not amenable to basic security measures like gracefully logging out sessions.

Clearly there's quite a bit more wrong with HTTP auth than that (for instance, it does a terrible job of handling errors, it doesn't handle "forgot password", &c &c) but I'm not particularly interested in the counterfactual argument of what could have been had browser vendors only built a "good" version of it.

The fact remains that even the hypothetically "good" version of HTTP auth would at best be only marginally better than what we have now, and would leave all the hard problems this "definitive guide" is trying to address still on the table.

<comment><USER17> > HTTP auth doesn't make backend authentication code easier to write.

It makes it easier to shove into a library or middleware.

Most of the issues you list are UI problems that could have been solved with a non-modal login prompt, a "remember this login" option, and a "stop sending the Authentication header" button.

My point isn't that existing HTTP auth schemes are great, because they're not (even if they weren't saddled with a terrible UI). It's that migrating from those mediocre schemes would have been easier than migrating from the mediocre schemes we're using now will be.

<comment><USER3> HTTP auth has literally nothing to do with where the backend code for authentication lives. In our forms-auth world, there already are quite a few authentication libraries.

I just disagree flat-out that HTTP authentication would have done anything to make the problems this guide is trying to grapple with easier. And in the meantime, HTTP auth as it exists in the standards today _adds_ problems.</comment></comment></comment></comment></comment></comment></comment>


Why R Doesn't Suck - <AUTHOR> http://paulbutler.org/archives/why-r-doesnt-suck/ <comment><USER2> i've used R for several large projects. it sucks in several ways for the projects i worked on:

1\. it is extremely slow for numerics. slower than MATLAB. slower than python (with numpy). after talking to several stats people, it seems pretty much everyone ends up writing most of their code in C when using R. (in contrast, i didn't find this necessary in python or MATLAB for similar projects.)

2\. its syntax is quite clunky. want to concatenate two strings? paste(string_a, string_b, sep=''). radford neal has a series of blog posts on R's design flaws: [http://radfordneal.wordpress.com/2008/09/21/design-flaws- in-...](http://radfordneal.wordpress.com/2008/09/21/design-flaws- in-r-3-%E2%80%94-zero-subscripts/)

3\. uninformative error reporting. by default, the stack trace isn't printed. even if it is, often the errors don't really tell you what went wrong.

i don't see any advantage to R over python. yield makes a nice replacement for the lazy evaluation of R.

<comment><USER3> I agree with all of your points, yet I still find myself using R on a day to day basis instead of python. My reason for doing so is the large number of statistical packages that exist for R. I know that scipy exists, but it does not seem to have nearly the same coverage as CRAN + base libraries... Are there other python libraries I should look at?

<comment><USER2> it's true that CRAN+Bioconductor have a lot of coverage of statistics. if you need to fit a particular kind of model and it's not using MCMC then perhaps it's the best you can do if someone has already written the code.

we use sage (<http://www.sagemath.org/>) which includes a lot of the common python numerics: numpy, scipy, matplotlib, networkx. it provides nice interfaces to R and gp/pari. i also use mayavi2 for 3d plots (something i could never get R to do well under linux...) enthought have a lot of nice things for python and scientific computing. there's also pymc which i've not used (i just write the MCMC code directly).

</comment></comment></comment><comment><USER4> Are useful and powerful the opposite of sucky to most people?

Not to me. Consider a parallel example:

Visual Basic (pre-.NET, think ~3.0) opened up programming for GUIs to a much much broader audience (because Hypercard was made for a far less popular system). VBmade embedding hugely powerful applications within your own, remote object calls simple, etc. It was the most popular programming language.

It was incredibly successful, useful, and powerful. None of that could change the fact that it fundamentally sucked though.

It sucked so much though that Microsoft killed it. They killed their most popular programming product ever. They killed the product that was used to make most of the applications for their platform, their office suite, their web server, their database server, etc. In the transition to .NET they apparently they felt its foundations were so fundamentally flawed that they had to redesign the language.

<comment><USER5> I agree with your thesis but I wouldn't classify classic VB as particularly powerful either. Usually when people say a language is powerful, they seem to mean that a small number of primitives can be combined to produce a large variety of structures or that brief code can accomplish a lot of work.

Maybe a better term for powerful would be "power to weight ratio".

<comment><USER4> You're right: it's primitives sucked. However, VB let people do a huge amount of work _without_ code: Think the GUI designer, VBX/OCX controls, and OLE embedding. That's the power i was talking about, not a way to implement advanced algorithms. (People used freaking SQL databases to get passable data structures... Ugh!) Most code I've seen people writing was more related to that gluey crap than advanced topics.

Side-note: Most of my work in the past year has been done in R.

<comment><USER6> Those weren't really parts of the _language_ , though; they were parts of the _platform_. We inconsistently compare languages (i.e. Java) to platforms (i.e. the JVM), which is, I think, a lot of the reason we're so bad at comparing both.

</comment></comment></comment></comment><comment><USER7> > As in Haskell and O'Caml, operators are just syntactic sugar for ordinary > functions. Enclosing any operator in backticks lets you use it as if it were > an ordinary function. For example, calling `+`(2, 3) returns 5.

This is awesome. This is probably one of those things that I should have known but didn't. It strikes me as being very useful in combination with the 'apply' functions.

<comment><USER8> > It strikes me as being very useful in combination with the 'apply' > functions.

It's very useful with higher-order functions in general. Even more so because most of the languages with operators simply being (binary) infix function calls also default to curried functions[1], which you can easily partially apply.

Haskell also has the reverse operation (MLs probably have it as well) of being able to use a binary function as an operator: "a `foo` b" is equivalent to "foo a b", but sometimes reads much better.

[1] <http://en.wikipedia.org/wiki/Currying>

</comment></comment><comment><USER9> To me it's a statistical calculator with beautiful graphing capabilities. Once the question becomes difficult enough to consider it programming, it's time to pull up numpy.

</comment><comment><USER10> Anyone who thinks R sucks obviously hasn't used the ggplot2 package for it: <http://had.co.nz/ggplot2/> !

The neat thing about R is that it supports the functional paradigm, but it doesn't bash you on the head with it. My fellow programmers who are not familiar with lazy evaluation, continuations, list iterators (is that the right word? such as map / filter / fold) can still use it without feeling like they're missing an arm.

<comment><USER11> > is that the right word?

Higher-order functions, or "combinators" if you want to sound all math-y. They're not really _list_ iterators, because it makes just as much sense to map or fold over trees, arrays, matrices, etc. Whether you need structure- specific versions like maplist, maptree, etc. is just an implementation detail.

</comment></comment><comment><USER12> I have never used R, but why (in the beginning) would you generate data in python and then plot in R, rather than just plot in python (which looks much nicer IMO)?

<comment><USER3> Plots in R are far superior to those generated with matplotlib or other packages. Take a look at ggplot2.

<comment><USER2> what about 3D or 3D with time in R? this is what python has: <http://code.enthought.com/projects/mayavi/>

i'm not really sure about far superior for 2D: <http://matplotlib.sourceforge.net/users/screenshots.html> vs. <http://had.co.nz/ggplot2/>

<comment><USER13> I remember looking at that matplotlib page for the first time and coming away thinking those graphs look a lot better and sharper than the default R ones. However, I found the reason for that is the images on that page are quite high resolution PNGs. If you make graphs of 200dpi or more in R, they look equally good. The annoying thing though is that you have to adjust all the margins and character size settings in R to plot at a higher resolution.

</comment></comment></comment></comment><comment><USER14> Given the existence of Incanter+Clojure plus the Leiningen project (similar to R's inbuilt package management) - how does R compete? Especially considering primitive types support in Clojure - [http://groups.google.com/group/clojure/browse_thread/thread/...](http://groups.google.com/group/clojure/browse_thread/thread/c8c850595c91cc11)

<comment><USER15> R's libraray support is far more complete. R has been the defacto standard for statistics research for over a decade. This means that someone has already written an R library to handle basically any type of statstical or probability application you care to imagine. Also, for much the same reasons, the documentation for R is much much better.

<comment><USER16> _someone has already written an R library to handle basically any type of statstical or probability application_

Also, frequently the person writing the library is the guy who invented the estimator in the first place. To me this counts for something.

</comment></comment></comment><comment><USER17> My programming background is fairly extensive in non-functional languages. I work in finance, and the company I joined uses R for a considerable amount of model prototyping.

I'd really like to switch to Python+numpy/scipy, but I haven't been able to find an equivalent of a data.frame, or some numeric+string data structure that allows for easy slicing on both.

Does anybody have any suggestions?

<comment><USER18> Does numpy's record array do what you want?

<http://www.scipy.org/RecordArrays>

It doesn't quite have first-class status in the library the way data.frame does in R, but it does let you index an array using strings.

<comment><USER17> Thanks, I'll check it out!</comment></comment></comment>


Facebook Has Become The New Yahoo, And It's Obvious Mark Zuckerberg Knows It - <AUTHOR> http://www.businessinsider.com/facebook-has-become-the-new-yahoo-2014-3

<comment><USER2> Is it wrong that I just won't read anything from Business Insider?

</comment><comment><USER3> It seems like Marissa Mayer is trying to do something similar with Yahoo!'s acquisition binge. Perhaps a lot of it is for talent, but she did make one big play with Tumblr. Only given Yahoo's market cap (vs. Facebook & Google), she can't make bolder plays in the range that Facebook/Google can.

I think Mark & Marissa are similar in that respect that they see the past pattern of BigCos ignoring the start-up's potential to grow and haggle on price and want to make a bet anyway because the technology/users/growth is there to pan out in the future.

In this example Yahoo (under Terry Semel) missed out on buying Facebook. In 2002 (or 2003) Yahoo missed out on buying Google [0] also under Terry Semel's leadership. Obviously it is easier to point this out in retrospect and Yahoo did make acquisitions that didn't pan out on the scale of GOOG/FB, but in the case of missing out on these breakout companies, it was a result of haggling on price. Of course in an alternate universe Yahoo could have made the bold plays with Facebook + Google and could have been one of the most valuable companies in the world (Hindsight is 20/20).

Mark Zuckerberg has been there and seen first hand of what the other side of the table looks like and what founders of young super-growth companies want. He perhaps also has some of the smartest people advising him. And I think that is why Zuckerberg is going to leave his acquisitions alone (Oculus, WhatsApp, Instagram) like the press releases mentioned because he realizes that these "subsidiaries" of Facebook might have a better chance of succeeding as stand- alone teams than as a merger with other facebook staff teams.

Also, I'm not sure how much of it has to do with the fact that Google & Facebook are run by founders vs. Semel who came in from Warner Brothers and wanted to personally enrich himself as much as he could (He made north of 450 million from his 6ish year tenure at Yahoo). If I'm making so much money at a company that I didn't help start and I'm already in my late 50s, the last thing I would care about is what the future of technology might look like and how I can capture that value by paying billions of dollars to young companies. Perhaps the case of misaligned incentives?

[0] [http://archive.wired.com/wired/archive/15.02/yahoo.html](http://archive.wired.com/wired/archive/15.02/yahoo.html)</comment>