New Orleans man locked up nearly 8 years awaiting trial, then case gets tossed - <AUTHOR> http://www.theadvocate.com/new_orleans/news/courts/article_d76e9496-c995-11e7-bd8b-4bb8d14cacbf.html?utm_content=buffera1a9c&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer <comment><USER2> There's an inmate in LA County jail who has been there 9 years awaiting trial on attempted murder. He is completely and totally institutionalized. A very nice guy but intimidating.

Cops can arrest you for any reason, and if you can't make bail, then they place you in jails with convicted violent felons, some on murder trials.

In LA jails, you have to be racist. Depending on your pod and program, you have special requirements.

You often have to fight, especially if you are black or Hispanic. If there's a brawl inside the pod, you have to fight other races or you will be beaten by your own.

I sat in a tiny room for 11 hours with 50 inmates, half of which had face tattoos and were gang members. A room where inmates had been beaten to death and the others just cover up the cameras.

Multiple times I wasn't called to court, they were "dry runs". Mental torture and real risk required just for due process.

These types of systems only create more criminals.

The deputies act like psychopaths, taunting and roughing up inmates. But not as bad since now the FBI has cameras watching them.

The person who interceded in my misdemeanor case and put me in jail was the former head of California Homeland Security.

His lawyer, also a former federal prosecutor, was in every court appearance lobbying the prosecutor. He also represented the Chief of Police, the City Council and the Sheriff.

Their boss is a secretive billionaire whose former lawyer became the new Supreme Court Justice this year after he lobbied He is a very powerful person who hires former federal officials as political mercenaries.

This all happened because they wanted to protect a wealthy sex predator. In 2-weeks, the Homeland official is actually testifying for the sex predator for his civil trial. He is presumably testifying against my credibility.

<comment><USER3> Sincere question -- how do we spot things like this?

Let's pretend for a moment that I'm a relatively competent programmer with small amounts of disposable income to spend on hobbies, such as running algorithms over government datasets. (Okay, that's a weird one -- but I think it's good to apply my craft towards politics.) I know a lot about making computers scan through large sets of data looking for patterns, and some about making websites and whatnot.

I normally don't think technology does much for political problems, but this one legitimately sounds like we should be applying fraud-detection style algorithms to the names that appear in court records, looking for abnormalities. (Heck, we should probably look for all kinds of irregularities, not just in the pattern of names.)

Identifying the problem doesn't magically solve it, but it sounds like being able to assemble subtle cases out of thousands of documents across hundreds of jurisdictions actually _would_ provide utility towards addressing this.

It doesn't stop the corruption directly, but it shines a pretty bright light on it. And it allows groups looking to tackle it head on to focus their efforts on litigating, instead of document finding or figuring out who the culprits are.

<comment><USER4> The main problem is that most people in American society don't care what happens to prisoners. Don't be surprised if you employ these techniques only to discover nobody lifts a finger.

That said, this would be a project I'd be interested in contributing to. If you set it up, shoot me an email.

<comment><USER3> I'll be honest: I suspect that this is actually quite common and I have absolutely no idea how to fix it.

But I suspect that much like homelessness, step 0 is gathering an accurate, systemic level picture so we can even understand _what_ we're trying to fix. Targeted treatments and good intentions aren't working -- we need a system level mechanism.

Ed:

I replied to a cached copy; leaving in case anyone raises the same point.

<comment><USER5> Local elections. Local police puts local folks into local jails. County jail is run by local people.

So the question becomes, what is the gamut of these jails? Are there facilities that are run well, on time, safely, etc..? What separates them from these hellholes? What's the context? What's the difference between those counties?

</comment></comment></comment></comment></comment><comment><USER6> There are many cases of defendants' right to a speedy trial being trampled. I'm not completely convinced this is a particularly great example of that.

He was arrested while on parole, which is why he didn't get out on bail. Many of the delays came from his side. If it had gone to trial, the prosecution would have only had to prove he was guilty by a preponderance of the evidence. So the long time in detention could have just been a strategy.

<comment><USER7> > Many of the delays came from his side. If it had gone to trial, the > prosecution would have only had to prove he was guilty by a preponderance of > the evidence

No, that would be true if it was just revoking parole on the prior charge (it's kind of odd that that _didn 't_ happen, but it's possible that the original sentence ended to soon after the arrest to bother), but this was a new criminal charge which would have to be proven beyond a reasonable doubt.

<comment><USER6> You're right

</comment></comment></comment><comment><USER8> While I agree it's unconstitutional to park someone in jail for eight years without bail or a trial, this particular guy is probably ecstatic at how it all worked out:

>As a parolee with a previous drug charge to his name, Smith was looking at a sentence of 20 years to life if convicted as a habitual offender, according to one of his lawyers.

<comment><USER9> Previous drug _charge_? Does that mean that he wasn't convicted the first time, either? The only thing "habitual" there, then, would be the police's desire to arrest him.

<comment><USER10> it's pretty standard in the US for the district attourney to offer a plea bargian of parole in exchange for pleading guilty. This results in high win rate for the attourney and they look at it as most of these parolees end up making a mistake during parole and going in for longer sentances.

<comment><USER7> Unless some jurisdiction uses "parole" to include what "probation" usually means, that's incorrect, and even if that's true in some jurisdiction it's not in Louisiana, where (as is generally the case) you can be sentenced to _probation_ , or released from a prison sentence on _parole_.

So, as a parolee, he had previously been sentenced to prison and released under supervision.

<comment><USER11> If you're out on parole. you're effectively on probation though.

<comment><USER7> Yes, but you don't get parole as a plea bargain, as suggested in GP.

</comment></comment></comment></comment></comment></comment><comment><USER12> I would hope that these days no rational individual could support jail time for _any_ non-violent drug crime, with the possible exception of cross-border smuggling, which treads on some different areas of the law.

<comment><USER13> Hmm, I'm not so sure we should allow people to sell fentanyl pops or cocaine to kids without going to prison. Not arguing against your point broadly, but that's exactly the type of response that would sink an effort to remove imprisonment for non violent drug crimes.

As I see it, we need both a propaganda campaign pointing out how nuts the drug laws are in addition to some recognizable way of incrementally easing the laws and adjusting to it. That's the rational thing an individual would do anyway; i am more aware than ever you need to parcel these things into 2 year steps for american power transfers between parties.

To the person who downvoted me, this is distinctly less useful than contributing the conversation in a meaningul manner.

<comment><USER14> One of the big problems in America is simply not effectively treating drug use and addiction. Putting people in jail doesn't stop drug use.

You're talking about big distributors, not individual users. Portugal has probably had the best response to drug use, evaluation each case individually and helping people quit drugs when it interferes with their lives.

<comment><USER15> Portugal still jails non-violent drug dealers, defined only by the amount you're carrying.

</comment></comment></comment></comment><comment><USER16> The concept of bail negatively affects the poor. Washington DC has a new system where they got rid of the bail system.

They have 2 options: RoR (release on your own recognizance) and hold until trial.

Although not perfect, this is much better than a bail system.

<comment><USER17> > The concept of bail negatively affects the poor.

I thought that was intentional :) afaik bail isn't used much outside the US, Canada and UK.

The US for-profit bail industry is the usual case of Americans taking a bad idea to the extreme in the search of a little profit.

<comment><USER18> It makes sense as a deposit to make sure you show up to trial. But in practice, with bail bonds, it's a fee. The financial amounts are rarely in line with what it would take to get someone to show up trial.

<comment><USER19> If bails were relative to income, then this would maybe not be such a huge problem.

</comment></comment></comment></comment><comment><USER20> I live in New Orleans. The judge, Flemings-Davillier, is well known as a tool of the district attorney, Canizzaro, who employs dirty tactics like fake subpoenas.

For more on Orleans Parish justice, google "Give me a lawyer dog"

<comment><USER21> Oh man that was a crazy case...

</comment></comment><comment><USER22> Sadly, this is common in India. In India there are people in jail for decades, without even a trial[1].

There are just 15 judges per million people in India as compared to around 110 judges per million in US.

\----

[1] [http://www.hindustantimes.com/india/38-years-in-jail- without...](http://www.hindustantimes.com/india/38-years-in-jail-without- trial/story-mKpSuk0mtGjuii6dDQ3DsK.html)

</comment><comment><USER23> The court system claims it doesn't have enough money while over half the criminals/defendants are in for drugs.

Theres a very simple solution here.

<comment><USER24> Am I to assume you mean to legalize drugs?

<comment><USER25> I'm pretty confident he's not suggesting a rapid death-penalty for drug- offenders

<comment><USER26> Hey it works for Singapore, Indonesia, and probably the Philippines.

I think people are easier to control, but less productive when they have drugs. So it's really a question of what your nation is optimizing for at the time.

Of course having a sub-class of previously convicted minor offenders, who as a result struggle with work/life, can also be fairly effective. It's not exactly slavery - that was mostly their ancestors.

[https://ramio1983.files.wordpress.com/2013/01/image1df.jpg](https://ramio1983.files.wordpress.com/2013/01/image1df.jpg)

<comment><USER27> People in the US have a lot of drugs and are also very productive. Your assertion makes no sense. I don't know much about Indonesia, but Singapore is an authoritarian state, and Duerte is trying to turn the Phillipines into a dictatorship. Tyrants like drug laws, and especially death penalty for drugs because it gives them an easy way to keep their thugs in practice, remind the populace that they are ready to kill, and gives them an easy way to assassinate troublemakers.

<comment><USER23> Thanks. This is 100% accurate. I think OP was either a troll or we have Duterte on HN.

</comment></comment></comment></comment></comment></comment><comment><USER28> If the relevant prosecutor goes to jail for 1-2 years, they'll learn not to do that the next time.

</comment><comment><USER29> It's pretty obvious what happened here. Everyone involved basically assumed he was or would be found guilty and didn't see an issue with him sitting in a cell for "just a few more months"

</comment><comment><USER30> The government should pay back-salary to anyone with a case dismissed in this manner. This would cause queue back-pressure that would focus the system on high-value targets at the front of the queue.

<comment><USER31> You're arguing that higher income poeple should get speedier trials than lower income people?

<comment><USER32> If jury duty is anything to go on, they'd pay minimum wage regardless of the suspect's income.

</comment></comment></comment><comment><USER33> If you have so many people in jail awaiting trial that judges have multi-year backlogs to try cases, that says a lot about the justice system... but more importantly, what does it say about the laws themselves?

</comment><comment><USER34> Stop being poor. That's the only solution.

Because you can go back in time 30 years and find articles like this or fast forward 30 years and find articles like this. Its not going to change.

</comment><comment><USER35> This is outright extorsion for bail money.

</comment><comment><USER36> I think we have to be careful given the article only looks at one side. While in this case it took an absurd turn and is a clear case of "lock this dude behind bars: we don't like him", it is not always that simple.

I would give as an example the patent trolls. They abuse of the court and fill motions after motions (or dismiss early) for the sole purpose of making their case expansive to defend against. A knowledgeable inmate or greedy public defendant could use the same tactics and delay the case only for the deadline to pass. If it can turn a life sentence into a 24 months one, then procrastination and delays are the quickest way out of jail.

That being said, for recurrent inmates, the failure of the system is definitely in the rehab process. In some US states, it seems like the legal system exists only to sustain itself, its elected officials and the industries (lawyers, penitentiary, investigators, forensic) that runs it. This case is apparently a blatant example of bureaucracy indifference. If the fact were shown in their face, they would blame the weather or the defendant own motions to have his rights taken into account. I am not a socialist, far from it. However when it comes to absurdities such as this, it is a clear sign that a proper social net is necessary to avoid a slippery slopes where the incarcerated population will grow and grow and grow. So do the expanses associated with it. Better solve the problem instead of the consequences.</comment>


Ask HN: Should I mention my security clearance when applying to Google? - <AUTHOR>

I currently work at a defense contractor programming computers that [REDACTED], but want to get a job at either Netflix or Google when I get my MS CS in December. Is having a security clearance something worth mentioning to them and if so how prominent should it be on my resume if I also want to mention my personal projects, my github, a programming blog, education, technical skills, and programming work experience? <comment><USER2> Yes, you should list your security clearance. It shouldn't hurt. If Google is involved in any projects that need a security clearance, that will help you because it means you saved them money on needing to pay, and time in getting you cleared to work on a project.

It doesn't need to be prominent. Google should be experts at searching for keywords.</comment>


In Defense of the MBA - <AUTHOR> http://www.businessweek.com/bschools/content/aug2009/bs20090824_769691.htm?campaign_id=rss_topStories

<comment><USER2> As an MBA myself, I have never understood the mass criticism and stereotyping the degree receives from those whom simply have never experienced business school. Why? In my experience, I have learned how to think and design my ideas to the point where others can best understand their positive potentcy. I have applied what Ive learned in various aspects of business, into a functioning web startup as well. The most negativity that I have ever seen from my having an MBA, is being told by a potential employer, that I am overqualified for a position. And I honestly cannot complain about that. Would much rather be overqualified than underqualified. And thats just what my MBA has done for me. Made me qualified. Qualified to work in the position that I want, qualified to run my own startup, and qualified to write this post. And its not just the piece of paper I received that qualifies me. Its what I learned and experienced during my education.</comment>


PyPy 1.5 Released: Catching Up - <AUTHOR> http://morepypy.blogspot.com/2011/04/pypy-15-released-catching-up.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+PyPyStatusBlog+%28PyPy+Status+Blog%29 <comment><USER2> I did a benchmark of Unladed Swallow 1.5 years ago and at this time PyPy looked like a very academic project. It could not really run any interesting code. At this time Unladed Swallow looked like a great project since improving cPython rather than starting from scratch seemed much easier, especially since Unladed Swallow based their JIT optimization on LLVM's JIT engine.

I also recall reading PyPy's blog, especially a blog post about their Prolog based JIT prototype and I thought "wow, this sure seems like a complicated way to implement a JIT engine, I wonder if they will ever implement something that can run real code".

Fast forward to today and Unladed Swallow is dead and PyPy has implemented a Python implementation that's compatible with Python 2.7 and beats cPython 2.6 on various benchmarks. Pretty impressive and kudos to the PyPy team.

<comment><USER3> Many people seem to ignore that pypy is not a new project. It's been on the works for more than 8 years now.

<comment><USER4> It's also had a lot of funding, and more full time workers than on CPython (who has zero full time coders).

</comment></comment></comment><comment><USER5> I'm impressed that they support the features of CPython 2.7 already. PyPy seems to be developing much faster than other alternate python implementations.

<comment><USER6> IronPython actually beat us to 2.7, however I will note that it was pretty fast. We went from 2.5 to 2.7 support in under 12 months.

<comment><USER7> I will love it if in a couple of years the mainstream implementation of Python is PyPy. People are going to be upgrading their Python installations anyway when they move to Python 3. If y'all can get Python 3 support in, maybe people will switch to PyPy at the same time.

I just wish Google would throw some support your way.

<comment><USER8> Indeed, what are your current plans for Python 3.x?

<comment><USER6> Future cloudy, ask again later :) Because of the way our JIT is written it would be entirely possible for us to branch for py3k and put JIT improvements on both with relatively little effort, it's likely that at least our next release will still be 2.x and just feature optimizations though.

<comment><USER8> Here's one vote for focussing on py3k sooner rather than later, fwiw :). I understand you're anxious to reap some rewards and see (more) production use at this point, but as a big fan of much of what was done to the language in py3k and having made the switch already, PyPy vs. CPython 3.x is an unhappy dilemma for me :).

<comment><USER9> For me (and I suspect quite a lot of other people), PyPy and CPython 3 both suffer from the same problem: lack of support for third-party libraries. Moving to Py3k would only exacerbate that problem for PyPy, removing their current support for the likes of Django and Twisted.

<comment><USER8> I'd assume they'd very likely keep up support for Python 2.7.x for quite some time past the first py3k-compatible release, similar to what CPython is doing.

Although personally I actually wouldn't mind giving up one for the other, as that would allow PyPy to act as an extra incentive to drive py3k adoption in third-party libraries. Of course I understand if the PyPy developers don't just want to use their hard work as a gaming piece to drive the py3k adoption agenda, though :).

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> I wonder how hard is the integration of Stackless and the JIT features. That would be killer.

<comment><USER6> There's actually a sprint going on right now (where this release was done), and that was a topic of discussion. I don't think anyone has written up the conclusions yet, but estimate I heard was that it'd be about 1-month of person-work.

<comment><USER10> Fantastic... Christian worked hard on Stackless and I think it`s a great feature. On Pypy it would be easier to maintain and add functionality, unfortunately, I struggle on Stackless internals and it`s low level C gimmicks so I can't help on it. It would be great to have something like a good framework in the future to make it`s adoption easier. We have the Stackless Examples project to address this initial learning curve thing. <http://code.google.com/p/stacklessexamples/>

<comment><USER10> Nice, the CCP guys are great maintaining Stackless... Kristjan and Richard do a great job on it. If there are any clues or tips on how can I help out, it would be a pleasure.

<comment><USER11> Have you asked in the Stackless mailing list?

</comment></comment></comment></comment></comment><comment><USER12> Any chance you could update your Ubuntu PPAs soon?

<comment><USER13> My sentiments exactly: I would love to have a reasonably up-to-date PyPy around to try things in, but I'm not yet so invested in the project that I'm willing to watch for release announcements and download source-tarballs (or binary tarballs). Having a reliably updated PPA is pretty much exactly what I'm looking for.

</comment></comment><comment><USER14> Maybe this is a dumb question but why not just work on making CPython faster?

Edit: I guess this provides insights:

<http://codespeak.net/pypy/dist/pypy/doc/architecture.html>

<comment><USER6> I wish I could find the video the talk where we explained this in great detail, but basically: it's hard, it's been tried and failed (notably by Armin Rigo, one of the leads of PyPy). Adding a new GC to CPython is insanely hard, and manually writing a JIT for Python even moreso.

</comment></comment><comment><USER15> I'm not really intressted in the Python interpreter that pypy is but the other part pypy (the toolchain) is maybe the best thing since sliced bread.

I have the plan to start working on the scheme implmentation.

The thing I don't really like about the project is that it has one name for two things. I can understand how that came about historicly but I think the should make two names out of it.

<comment><USER6> We've started referring to it as the "RPython translation toolchain" which will hopefully reduce confusion.

<comment><USER15> You should think of a snappier name. Other than that keep up the good work. I'm atm reading all the pypy papers, very cool.

<comment><USER16> you should have a look at scheme interpreter that was written already maybe: <https://bitbucket.org/pypy/lang-scheme/overview>

</comment></comment></comment></comment><comment><USER17> I'm really interested in this project. The abundance of C as "the new machine language" for language implementation, as well as the JVM-based languages, makes me wonder what it means for the programming language landscape. I would be very interested in learning whether this un-C implementation made any difference to Python, and what can be learned from it in way of other language implementations

</comment><comment><USER18> The benchmark here <http://attractivechaos.github.com/plb/> suggests they are doing much better on performance than they were a while back. Its a straight numeric benchmark so not relevant for everything but its better than V8 which is good work.

<comment><USER6> 1.5 should be a lot better on these, the Loop Invariant Code Motion really helps on very tight loops, as these benchmarks often have.

<comment><USER18> Thats great. Dynamic languages at say Java like speeds is a real game changer over the PHP/Rubylevel speeds. At the moment we only have LuaJIT proving its possible, but PyPy is the next contender.

<comment><USER19> While this is great news for Python, there are quite a few dynamic languages now beyond LuaJIT showing that it's possible to get stellar perf - JavaScript, Racket, Clojure.

<comment><USER20> Last I heard, no JS engine was close to LuaJIT's performance; they were all worse by a factor of five or so. Clojure being in the LuaJIT ballpark would surprise me (do you have benchmarks?) but Racket wouldn't. (But I wouldn't call Scheme a dynamic language!)

<comment><USER19> [http://shootout.alioth.debian.org/u64/which-language-is- best...](http://shootout.alioth.debian.org/u64/which-language-is- best.php?gpp=on&java=on&sbcl=on&ghc=on&csharp=on&clojure=on&racket=on&v8=on&hipe=on&lua=on&python3=on&vw=on&yarv=on&xfullcpu=1&xmem=0&xloc=0&nbody=1&fannkuchredux=1&meteor=0&fasta=1&spectralnorm=1&revcomp=1&mandelbrot=1&knucleotide=1&regexdna=1&pidigits=1&chameneosredux=0&threadring=0&binarytrees=1&calc=chart)

Clojure 1.2.0 is already faster than Racket. Version 1.3.0 will probably bring it close to Go territory on those benchmarks.

For my own work I've found that getting Clojure in the Java ballpark is certainly possible.

<comment><USER21> [http://shootout.alioth.debian.org/u64/benchmark.php?test=all...](http://shootout.alioth.debian.org/u64/benchmark.php?test=all&lang=racket&lang2=clojure)

<comment><USER20> Yeah, I was wrong about Racket: [http://shootout.alioth.debian.org/u64/benchmark.php?test=all...](http://shootout.alioth.debian.org/u64/benchmark.php?test=all&lang=racket&lang2=gcc)

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER22> Can numpy be used with pypy? Or are there plans for that?

<comment><USER6> There are plans, but nothing yet, this reddit thread has some decent information: [http://www.reddit.com/r/Python/comments/h0uuv/pypy_15_releas...](http://www.reddit.com/r/Python/comments/h0uuv/pypy_15_released_catching_up/)

<comment><USER23> I'd like to second the request for a very detailed blog post written by one of the PyPy devs on how interested parties can contribute to the development of Numpy on PyPy. Lack of Numpy support is the showstopper for me and a lot of other people in terms of switching over -- almost everybody who wants faster Python is using Numpy in some way or another :-) This is something I would be really interested in working on.

</comment></comment></comment><comment><USER24> Why did they decide on a name that can easily be (mis)pronounced as a synonym for urine? That just seems like a poor branding decision, up there with "blur- ray".

<comment><USER25> What language uses "ee' sound for "y"?

And who pronounces blue, blur?

<comment><USER11> Spanish does.

Not agreeing with grandparent, though. Lots of people admit to playing with their Wiis for hours a day.</comment></comment></comment>


5 Things Microsoft's Cortana Needs To Do To Take On Siri - <AUTHOR> http://www.forbes.com/sites/kellyclay/2014/03/03/5-things-microsofts-cortana-needs-to-do-to-take-on-siri/

<comment><AUTHOR> Interestingly, there is a Windows Phone App that does all of these things....

[http://www.windowsphone.com/en- us/store/app/plexivoice/9c09f...](http://www.windowsphone.com/en- us/store/app/plexivoice/9c09f697-657d-4613-ae67-7e787121d0a4)

Microsoft definitely has an advantage over third party apps. They make it nearly impossible to edit a calendar item, and their web API's won't give you a contacts phone number.</comment>


Sci-Hub: Removing barriers in the way of science - <AUTHOR> http://sci-hub.io/ <comment><USER2> Previous HN discussion (article has neat details of how sci-hub works): [https://news.ycombinator.com/item?id=11074638](https://news.ycombinator.com/item?id=11074638)

<comment><USER3> There was another shortly before that: [https://news.ycombinator.com/item?id=11070192](https://news.ycombinator.com/item?id=11070192).

We're bending the rules not to treat the current post as a duplicate, but the community interest seems stronger than usual.

</comment></comment><comment><USER4> I am happy she is doing this. The price that paywalls are charging for accessing research papers is unacceptable: the money does not flow back to the researchers, nor to the institutions. Most of the paywalled research has been sponsored by taxpayer money and hence should be publicly accessible or at a very low "maintenance fee".

<comment><USER5> I work at a small academic database as a developer. We charge a pittance compared to what a lot of the larger players in this market do. And I'm conflicted.

We are a registered non profit, and are actively losing money. Developer costs, of which there are only two. Building costs--I also shovel snow!, marketing costs, hosting costs, and much more. We have the workload where we would benefit from a staff 3 times the size. Currently we are at 12. Years ago we had a fully functional publishing operation too.

But here is the thing. These open access places often are pay to publish. This is built into many grants today. Publishing costs. But not so much in the field I'm in. The social sciences. So sure, the result of that work should be open access. I just cannot help thinking this takes away from what you could be paying one more lab or research assistant.

a lot of these pay walled sites are struggling to survive as library budgets dwindle. Maybe there isn't much room left for the little players. Many are bought up or have folded over the last 20 years.

<comment><USER6> This is a social sciences problem, not a large player vs small player problem. At Sage, which is one of the largest social science publishers, we also lose money on social science OA publishing. We've tried for years experimenting with different APC prices and have struggled to figure out how author pays will work in the social sciences. Shoot me an email (details in profile) if you want to chat about this stuff more. It's rare to find someone with experience doing social science OA publishing on HN!

</comment></comment></comment><comment><USER7> I've used sci-hub a few times. It's a little buggy and not every article can be accessed, but it works well enough to try when I'm not on campus.

$30 to read a single article is ridiculous anyway and presents a barrier to scientists who don't have, can't afford, or don't want to pay for access. I hope sci-hub stays up and improves for some time.

<comment><USER8> If you're a student, your university likely has a proxy service that allows you to access journals from anywhere using your credentials.

My previous uni gives access to alumni for life, so I can access journals for free from wherever.

<comment><USER9> And what are people outside of the U.S. supposed to do?

<comment><USER10> Agreed. I studied in Germany, and none of the academic institutions I know of make papers accessible to alumnis. For students, sure, but for alumnis? Never heard of it.

<comment> semi-extrinsic The local hack: get a lifetime membership at the uni computer club. This gets you a shell account on a computer inside the uni network, and Rob's your uncle. Lifetime cost is about $100 here, and that money goes to a good cause.

<comment><USER11> My uni didn't have any of that. YMMV

</comment></comment></comment></comment></comment></comment><comment><USER12> Elsevier made more than 3.5 billion dollars in revenue last year. They are trying everything possible to destroy open research. They were behind three bills in the US congress to prevent universities from providing access to pre- publication research. This is research that's been paid for by taxpayer dollars.

Companies with attitudes like Elsevier need to be buried.

</comment><comment><USER13> It paints a bulls-eye on her back and its use of the term "piracy" to "free knowledge" doesn't flow well in western sentiments. While I get that the "booty" she is stealing are the fees that the journals would like you to charge, the acts she are creating are more like a librarian letting people check out books without a library card because she has an infinite supply of said books.

I am hoping that the rent-seeking behavior of the science journals can be used as the canonical example of how copyright can harm the common good.

By endorsing and upholding this egregious use of copyright, our elected officials are clearly causing more harm than good, and the perversion of the spirit of copyright, that an _author_ is granted a temporary monopoly so that they might recover some of their investment, portrays this use as indentured servitude at best, and outright theft at its worst.

So while I don't think anyone is really "harmed" because Disney won't release the original Cinderella or employs measures to keep it from being copied. It is very much the case that by creating this barrier to scientific research, a person or group who might change the world in a positive way if they had access, is perhaps even unaware that there is relevant work that they cannot get access to. That is definitely a harm in my opinion.

So I hope that the narrative here, which has been dominated by big media for so long, might get some interjection of a more nuanced understanding of why copyright exists, and how to craft laws that embrace that spirit, rather then the rent-seeking interests of the people who live off the work of others.

<comment><USER14> The narrative on copyright has been dominated by big media because _every_ political narrative has been dominated by big media and big money.

Those of us who get information from relatively unfiltered and uncontrolled sources via the Internet have long had a different perspective on copyright than those who don't.

I too hope that big media's control of the conversation is coming to an end - but they won't lose that control without a fight.

</comment></comment><comment><USER15> A _long_ time ago (even before Aaron Swartz), when I was still familiar with the active and rapidly growing filesharing community of the time, I vaguely remember reading about an effort by some of the "ebookers" to plant proxies in various universities' networks that would perform much the same function. I wonder what eventually became of it besides the large paper torrents that appeared, but I wouldn't be surprised if SciHub was related to that in some way. Back then, systems were far more open (as opposed to secured), and something like that was easier than it is today.

<comment><USER16> > an effort by some of the "ebookers" to plant proxies in various > universities' networks that would perform much the same function

Yep they did that, it's called ezproxy.

[https://github.com/kanzure/ezproxy- urls/blob/master/urls.txt](https://github.com/kanzure/ezproxy- urls/blob/master/urls.txt)

scihub uses this AFAIK.

</comment></comment><comment><USER2> Good for them. For the last article that I published, the publisher "value added" consisted of highlighting all the all-caps names in my document and asking me to define them as acronyms. Literally the only thing they did, and it wasn't even right.

<comment><USER17> Oh, I get so much more out of the publishers: 1.) Long waits during which I worry that I'm going to be scooped. 2.) Lots of typos because Elsevier outsources printing to people who don't speak proper English. 3.) PDFs that do not render correctly in some PDF viewers.

<comment><USER18> Got a source for #2?

Funnily enough, I was just reading an article from the journal SYSTEM (Sciencedirect/Elsevier) yesterday. Very prestigious journal in my field, but it was littered with typos and mistakes - I figured surely it couldn't be the work of the two authors.

</comment></comment></comment><comment><USER19> Welp, I used to get books, papers and software from non-legal sources when I was in undegrad, because I just couldn't afford it, now that I make some money I buy most of this stuff. The thing is, without all those resources in the past I couldn't have made it to where I am now. Just my 2 cents.

<comment><USER20> Kudos to you, there is little respect these days for personal decisions in the area of information access. Pirating information is wrong, doesn't matter how you twist it. There is big difference between creating and promoting free content (great) and trying to break the law to access content that already exists and is subject to copyright law. After all, requesting payment is a deal between publishers and writers. By pirating copyrighted works you're not just breaking the right of publishers, but also the rights of millions of small content creators.

<comment><USER21> What do you do of researchers that pirate their own books, and tell you never ever to buy their books, but go to libgen? I have met at least 5 world-class scientists say that "because I don't do any money on it anyway", "it's a scam", and "I wrote this book to be read". And tell "it's wonderful to imagine that this poor student read my book, and she was afraid to say she downloaded it illegally"...

"Piracy" is a very nuanced subject, depending on what/who you are talking about.

<comment><USER20> There is no nuance there, these people had a personal choice of publishing their books through a traditional publisher and decided for that. It is just a tradeoff that you need to honor. Maybe next time they will just self-publish and have a book that is truly free without the need of pirating.

<comment> Al-Khwarizmi We have no real choice. Self-publishing means that the book doesn't count at all in the CV for grant applications, tenure applications, etc. Only the very top scientists that no longer have to fight for all these things can afford to make that "personal choice". For the rest, it's suicidal.

I also have a book at a major publisher, with an outrageous price, and when I saw it "pirated" I only felt joy at the fact that more people will get to read it and thus my work is more meaningful. And I also downloaded it myself, because I actually didn't have it in PDF, only in physical form.

<comment><USER20> I have published academic books like you, and I never had a problem with this once I understood the consequences of traditional publishing. These books are cataloged in good libraries and available in Amazon. If people don't have money they can go to a library and get a free copy. The day I want a book freely available in the web I will just write one and post it on in my web page. I like the idea that authors have the option to go one route or another. Pirating books doesn't enter into this equation.

<comment><USER22> Going to a library and getting a free copy is _exactly_ what people are doing. The library is online, freely accessible without discrimination.

Since when have we asked authors permission to add their book to a library? In many places (including the US) if you publish a book it is _mandatory_ to submit it to a library.

<comment><USER20> I have never seen an illegal library, have you? Don't try to confuse a respectable institution with pirate web sites that didn't ask permission to anyone to do their illegal thing.

<comment><USER23> What is the difference, asside from someone deeming copying 'illegal'?

<comment><USER20> There is nothing illegal about copying per se, as long as there are no provisions against it. This is a basic principle of human society. For example, there is nothing illegal about walking without shoes, as long as there is no regulation preventing it as it's the case at some government offices. Your reasoning is just trying to throw away our society principles to justify your behavior. My main contention is not that we shouldn't have free information, but that it is unethical to disregard existing laws just because you don't like them.

<comment><USER23> There is nothing unethical in braking the law. It is illegal to disregard existing laws, ethics is a different matter. There were many racist, sexist, oppressing laws in the past that we consider unethical today, and may even celebrate people who broke those unethical laws in protest to authorities and 'the society'.

<comment><USER20> True, there are situations where breaking the law is the ethical thing to do. However, you are trying to put access to a copyrighted book at the same level as fighting against sexism and racial oppression. Unless you can show that these situations are closely comparable (little clue: they're not), you're just creating an excuse to avoid following laws that don't benefit yourself.

<comment><USER23> In the case of copyrighted books or research papers, it could be a matter of life and death for the user, if we are talking about access to various types of medical research, for example.

<comment><USER20> Research papers in medicine are written for specialists, who already have access to them by means of employment. I don't know how access to that literature can save lives otherwise. Even if that was the case, it is a very far fetched way to prove that you need generalized civil disobedience with regard to copyright law.

<comment><USER23> In a lot of countries in the world, medical institutions or individuals don't have enough money to pay for access to research and books (lib-gen, the sister project of sci-hub also serves pirated books). And I agree, that might not be a proof we need civil disobedience in general (with regard to copyright law), but I think it does show that copyright law doesn't work well for medical research. I think similar could be proved for other areas covered by copyright law, but that would be a long discussion.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER24> Is there a tarball of the data somewhere that one could download, redistribute and host somewhere (on the darknet, assumedly)?

<comment><USER25> There are occasional torrents, AFAIK. Hope you have 10tb of free space handy.

<comment><USER26> It is absolutely mind blowing to me that you can purchase that amount of storage with 20% to spare for around $500 these days. 12TB, $498 : [http://www.amazon.com/Red-Desktop-Hard-Disk- Drive/dp/B00LO3K...](http://www.amazon.com/Red-Desktop-Hard-Disk- Drive/dp/B00LO3KR96)

<comment><USER15> Or around $120 if you just want to archive the data and use tape:

[http://www.tapeandmedia.com/quantum-lto-6-tape-ultrium- tapes...](http://www.tapeandmedia.com/quantum-lto-6-tape-ultrium-tapes.asp)

The drive isn't cheap though, but tape is still the cheapest media for long- term archival storage.

</comment></comment></comment></comment><comment><USER27> Great to see a massive middle finger to the journal system. It's a disgrace and has to stop. Unfortunately I fear sites like this might entice more stringent protections for future journal published articles. The war continues.

</comment><comment><USER28> In Norway there is free access to NEJM, JAMA, BMJ, Annals of Internal Medicine and the Lancet (2 month delay). UpToDate, BMJ Best Practice and McMaster Plus is also free. See [http://www.helsebiblioteket.no/om- oss/english](http://www.helsebiblioteket.no/om-oss/english) for information about all included resources. You need to access these resources from a Norwegian IP to get access. From abroad, this can f.ex. be done through Tor if you define only exit through a Norwegian exit node.

</comment><comment><USER29> [http://www.freefullpdf.com/#gsc.tab=0](http://www.freefullpdf.com/#gsc.tab=0) is another useful site if you're a lone researcher who doesn't have taxpayers' money funding your literature search and can't afford (in some cases) $30-$40 to look at a published paper.

<comment><USER30> Surely that's the same corpus?

</comment></comment><comment><USER31> This is what Aaron Swartz was trying to do, right?

<comment><USER32> Allegedly. All we actually know is that he downloaded a large number of articles.

</comment></comment><comment><USER33> Speaking of moral courage, how does one contribute institutional login credentials to sci-hub?

<comment><USER34> Only guessing here - you don't provide login credentials, you provide a proxy host within the university IP range as IP-based authentication is still the most commonly used method for licensed databases.

</comment></comment><comment><USER35> May it continue. Perhaps science can go the way music has where in practice you can see most stuff for free.

<comment><USER36> At least some of the time that does happen (not nearly as much as it should IMHO). I moved from one scientific institute to another, and would often get requests from friends in the first place asking if I was able to download a paper for them.

</comment></comment><comment><USER37> As a student who is studying to become a Theoretical Mathematician, I hope Sci-hub stays open for many years ahead. In Finland, we have pretty good access but only if one is a student. Mathematics is so interconnected, that removing paywalls and any obstacles could help uncover breakthroughs by combining ideas from other fellow Mathematicians. I hope UN exercises Article 27 of Human Rights and aligns itself on the right side of history in order to better Science and to encourage curiosity in today's minds and definitely tomorrow's! Pardon my English. Thank you.

</comment><comment><USER38> There is another service in the pipeline. I came across it very recently; it is in a public-beta phase. It appears to focus on providing access to all digital libraries and specifically serving the third-world or developing countries, mostly in Africa. It has got a different (business) model and uses some advanced technologies for provisions of the articles. Given that they intend subscribing to the publishers, there is no doubt that they will remain in business for as long as the publishers themselves exist.

The projects like sci-hub.io, library.no and libgen are highly commendable. It is no news that the third-world countries are destabilized by war, economic sanctions, e.t.c. perpetuated by the world powers thereby making them re- prioritize (access to) their resources. And it is not surprising that webrtc/p2p related services are often times blocked in the first world institutions with access to articles from those digital libraries. Such technologies/protocols/tools are defined/shaped (at standardization meetings - IETF, W3C, e.t.c.) by big corporations in order to preserve their own product offerings.

</comment><comment><USER39> Sometimes we have no option but to break the law until what is considered _illegal_ is made legal.

Storing research papers behind paywalls is absolutely ridiculous. The law literally prevents the development of science.

Having personally seen people benefit directly(for purposes of research) from this initiative solidifies my whole hearted support for sci-hub.

</comment><comment><USER40> How is this diffrent(better) from [http://arxiv.org](http://arxiv.org) ?

<comment><USER41> Arxiv is by its nature open access; Arxiv does not charge for access.

On the other hand, the articles that this site hosts require payment to access. The journals typically charge $30 / article, or roughly $2,000 / year subscription.

Note that for both the open access or paid journals, researchers do NOT receive any compensation when users download articles. That is, despite the research being mostly paid by taxpayers, a PRIVATE company receives compensation for the work done by the researchers. Not only that, but the researchers have to PAY a publication fee, and that fee is higher if they want to allow open-access.

<comment><USER40> How come all papers i have come across at the arXiv are readily downloadable as PDF?

<comment><USER42> Because the arxiv is a place that hosts free pre-prints. By contrast, sci-hub is a way to get normally expensive articles for free. Arxiv stuff is already free.

</comment></comment></comment></comment><comment><USER43> Hope widespread knowledge about its existence does not kill it.

<comment><USER44> [http://libgen.io/](http://libgen.io/) which archives the sci-hub's newly accessed papers distributes the copies through BitTorrent. Libgen itself is also mirrored in multiple locations. Even if Sci-hub is taken down, it shouldn't take long for another to pop up.

<comment><USER45> They encrypt their torrents, so it's pretty useless.

<comment><USER46> What makes you think that? If the filenames confused you, those are checksums. The metadata is available as database.

</comment></comment></comment></comment><comment><USER47> So ordinary people may finally read my papers with reasonable effort? Sounds like an improvement. I am not in the academic content distribution industry though.

</comment><comment><USER48> Looks like it is piping the queries over to scholar.google.com - getting only timeouts right now though.

<comment><USER49> It uses scholar as a search engine, but then it replaces the links in the results. e.g.

        http://link.springer.com/link-to-paper

becomes:

        http://link.springer.com.sci-hub.io/link-to-paper

As far as I can tell, it works for some articles.

<comment><USER50> Just use DOI of the article, then it's instant and bypasses Google search

</comment></comment></comment><comment><USER51> So what is the strategy once Springer starts getting their domains taken down? With torrents this was never a big deal once there was the DHT - it didn't matter which search engines where taken down or which trackers, the torrents lived to see another day.

This website at this stage seems particularly easy to take down as it is a centralized weak link.

Centralized services work great when they are legitimate: Netflix, Spotify, but decentralized work best when they are not legal.

<comment><USER42> Sci-hub isn't going anywhere:

1\. The data is stored via lib Gen as torrents (for the PDFs), and a metadata database that is mirrored by hundreds of people.

2\. In case of a domain takedown the site can be resurrected at a new domain very quickly - as recently happened when the sci-hub.org domain was taken down after Elsevier sued the Sci-Hub founder.

3\. There's an onion site, which can't be disrupted by a centralised domain service.

</comment></comment><comment><USER52> There is very little innovation from academic publishers. Most don't even offer a single download that includes the paper and supplementary materials. E-book files are non-existent.

Very expensive publications, like Nature Biotechnology, should at the very least provide a single download (preferably epub) of each issue.

</comment><comment><USER53> Wonder what the size of the data set is so far?

Likely TB's?

<comment><USER54> The total size of the torrents are around 38-40 TB.

<comment><USER53> Wow, that's larger than I was expecting.

Um, is there a source for that info? Didn't see it in the referenced article. :)

<comment><USER44> I don't know where vortico's getting his/her stats, but you can calculate it yourself from the [http://libgen.io/](http://libgen.io/) , where sci-hub.io stores the newly accessed papers [1]. They have torrent archive of all the materials since 2011.

[1]: [https://en.wikipedia.org/wiki/Sci- Hub#Website](https://en.wikipedia.org/wiki/Sci-Hub#Website)

</comment></comment></comment></comment><comment><USER55> Nice. Does anyone know if there is something equivalent to this much like the SSRN (Social Science Research Network) [http://www.ssrn.com/en/index.cfm/mjensen-20th/](http://www.ssrn.com/en/index.cfm/mjensen-20th/)

Where you can perform full text search on all the papers?

</comment><comment><USER56> Server seems down (?)

Also, isn't this better done over bittorrent?

</comment><comment><USER57> in memoriam: Aaron Swartz

</comment><comment><USER58> Brain Aggregates: An Effective In Vitro Cell Culture System Modeling Neurodegenerative Diseases.

</comment><comment><USER59> Can we download only single articles, or can we download the whole 40M+ collection?

</comment><comment><USER60> While searching it goes to Google Scholar. Am I missing something?

<comment><USER61> They replace the links with their own, giving access to papers a plain Google Scholar search wouldn't.

</comment></comment><comment><USER62> (1) Isnt this illegal?

(2) How do they get access to those papers?

<comment><USER2> (1) Yes. In USA anyway. Creator is not in USA, however. (2) Academics donate their institutional login credentials to the site.

<comment><USER63> Do you know how I would go about donating my student access to the site? Do I need to give them my login, or can I run a program on my computer that logs in and downloads without giving anyone else the password?

</comment></comment></comment><comment><USER64> I didn't know that you could raid ships using a website(!).

In all seriousness, the act of "sharing the collective knowledge of mankind publicly" isn't morally equivalent to attacking ships and killing people. We should stop using terms that are clearly propaganda created by the film and music industry to try to muddy the waters.

<comment><USER65> I agree completely. We should also start owning the term and framing the debate around it. Cast it in positive anti-establishment terms. Like pirate radio, Robin Hood, V4vendetta. Lone, beleaguered hero(s) fighting for the peoples against tyrannical and oppressive regimes.

<comment><USER66> What might help is if someone started a file sharing site and named it after a place where pirates hung out and incorporated a pirate ship and the word "pirate" into their logo.

Then maybe if it became popular these so-called "pirates" could cast off the shame of being tarnished by an image they clearly don't embrace at all.

<comment><USER67> I heard about a certain site named "The Pirate Bay". They sort if got some publicity, but a positive reframing did not occur for some reason.

</comment></comment></comment></comment><comment><USER68> Umm, "pirate website" that uses a secure.sci-hub.io, but not actually a secure connection. Really, Let's Encrypt has made this a no-brainer. _Anyone_ making a site should be expected to be using SSL. _Especially_ those making anything related to anything "pirate" or "secure."

<comment><USER69> Because if there's one thing pirates are known for, it's their commitment to your security.

<comment><USER70> I just checked, the two most popular public torrent sites have COMODO CA default-on https.

These sites do want to maintain security, so their users can keep coming back rather than getting copyright strikes or worse fines and getting scared off. That doesn't stop them from putting viruses in the ad banners, but that is not getting their users arrested.

<comment><USER71> Except that torrent sites are not necessarily completely used for pirating.

<comment><USER72> Torrent sites, especially the popular ones mentioned above, are almost entirely used for pirating. Bittorrent, on the other hand, is not exclusively used for piracy.

</comment></comment></comment></comment></comment><comment><USER3> We changed the title from "Sci-Hub – Pirate website providing public access to millions of research papers" to what the site itself says.

I don't think this was an egregious title rewrite, but the word "pirate" was becoming the subject of discussion, which a title shouldn't be (and that goes double for extraneous ones).

</comment><comment><USER73> It's all fun and games, until someone in the open source community wants the same copyright protections from a commercial entity using GNU code without releasing the source.

</comment><comment><USER74> Silicon Valley and YC don't exactly have a stellar reputation for ethical behavior. Having a "pirate website" at the top of the news page doesn't exactly change that perception.

I totally get that journals are evil, and charging money for research generated with public funds is questionable. It's very frustrating as a small entity needing to view articles, and being asked to cough up $25-50. That said, there are legitimate alternatives (like emailing the corresponding author, or professional society memberships, or alumni library access, or DeepDyve). The linked website is flagrantly violating copyright and that should be cause for concern; not breaking the law is part of every engineering (and professional) ethical code.

<comment><USER75> I too am disappointed to see this here, though the young hotheads are obviously out in force today and relishing sticking it to the man.

Surely we are better than this.

One of the earliest lessons I was taught, and I taught my kids, is that if somebody else has something we want and doesn't want to share it, it's not OK to just take it.

<comment><AUTHOR> Firstly, copying isn't the same as taking.

More importantly most of the scientists want their research to be read and studied as widely as possible but have their careers to worry about. The journal system is being widely criticised but academics are not in the best position to take action against it.

The dissemination of knowledge, with it's potential for reducing inequality and increasing social mobility, is much more important than the profitability of journal publishers and outweighs the risk of hurt feelings due to a sense of ownership of knowledge (which seems like a fallacy in itself) that anyone involved could possibly have.

<comment><USER74> So why not email the corresponding author? I have yet to not get (or give) a manuscript that way. From my own perspective, each time I respond I'm possibly getting another citation. It's also a great form of networking.

<comment><USER61> What's the practical difference between getting any paper you need from the authors and getting any paper you need from such a website? Except much more work for anyone in the former case without any benefit.

<comment><USER74> Practically speaking, none I can think of. But I completely disagree that the networking aspect has no benefit.

</comment></comment></comment></comment></comment></comment><comment><USER76> I think people haven't totally thought this open access thing through.

First, publishing costs money, even in the digital age. It costs money to comb through submissions and decide which ones are worth pursuing. It costs money to hassle scientists into reviewing the submissions. It costs money to convert every submission into the same format. It costs money to develop and host a website to disseminate the articles. All of these things cost money.

Now, who is going to pay for it? Traditionally these costs were put onto the research institutions in the form of library subscription fees. Open access shifts this burden onto the author, and ideally grants would include that into the budget.

Even if grants include that in their budget (and many don't yet), there's a finite amount of money available for research. Shifting the cost of publishing onto grants will make funding available for actual research even smaller than it is now. In some fields publishing costs are entirely negligible compared to the cost of research, but in others it's not.

Also, open access would mean that you have to have funding in order to publish a paper. As it stands right now you don't actually need funding to do research in certain fields. A math professor at a university can devote some of his spare time to a project over several years and publish a paper on it with no costs at all. This happens all the time, not every paper has funding behind it.

I'm not necessarily arguing against open access, I just think people haven't fully explored the downsides of moving away from our current system.

<comment><USER77> They've been plenty explored, discussed, chewed over, and more, including the scenarios your presented. The costs have been analyzed in dozens of different ways, with many business models proposed and some executed upon.

(Note that your mathematician gets indirect funding by having access to the university library. As a non-academic, I can use the local college library but must pay access fees for some services that are free to staff and students. At a somewhat further away university library, as a visitor I can read journals online but am not permitted to make copies.)

Nor is our "current system", concentrated as it is in the hands of Elsevier (and its 37% profit on revenue), all that old. Most people outside the big publishing companies didn't fully explore the downsides of moving away from the system we had before the 1980s - or at the least, nothing like the ongoing discussions concerning open access.</comment></comment>


Nuitka Progress in 2015 – Python Compiler - <AUTHOR> http://nuitka.net/posts/nuitka-progress-winter-2015.html <comment><USER2> IMHO, this is a _very_ important project as it makes up for one of the biggest shortcomings in the Python ecosystem - distribution of software. Distribution is clearly one of the reasons that Go is so popular. It would be great if we could take advantage of Python as a language and Python as an ecosystem while still being able to deploy as if it was a walk in the park.

<comment><USER3> I find the way software is packaged in Go to be a terrible regression, but I'm more interested in why you think pip and venv's don't solve the problem for Python already?

Edit: I see from your reply that we're talking at cross purposes. I thought you meant source distribution, but you mean binary distribution to end users or deployment to production systems, and in that case I agree with you.

<comment><USER2> Being able to ship a self contained binary of your application is a very powerful concept, on which many seem to agree.

The way I see it, pip and virtualenv are not practical for deployment or distribution. You shouldn't have to download and install things during a production deployment. I even created a tool ([https://github.com/objectified/vdist](https://github.com/objectified/vdist)) to mitigate this problem, but it will always be a hack when doing it this way.

<comment><USER4> Remember that critical Go security update?

Usual procedure - update the shared library, restart affected services. Go - recompile everything.

<comment><USER5> Which only needs to happen once.

<comment><USER4> Yes but still. If you have a procedure in place to recompile and redeploy everything, you could just deploy the libraries as well.

<comment><USER6> Plus, the system admin doesn't just have to wait on the new security-patched library to be ready, they have to wait for everyone who used Go to recompile and distribute their programs.

</comment></comment></comment></comment></comment></comment></comment><comment><USER4> > The stable release has full support for Python 3.5, including the new async > and await functions. So recent releases can pronounce it as fully supported > which was quite a feat.

Wow. That is amazing! Pyston only supports Python 2 and PyPy is working on it.

Nuitka looks very promising.

</comment><comment><USER7> This looks very similar to Cython, which can also compile .py files without modification and is more mature at the moment. It seems that Nuitka wants to do a few things differently, though. The biggest difference is probably that Nuitka wants to use type inference + hints instead of explicit declarations, which make Cython code incompatible with CPython but give you more control and C interoperability.

Edit: Thanks, no more interference!

<comment><USER8> I've never heard about type _inTERference_ , but I'm sure that if something like that existed, it would have little to do with type _inference_ which you probably meant :)

interference - obstruction, collision

inference - deduction, derivation

<comment><USER9> I imagine that type interference might be what an advocate of dynamically typed languages sees in statically typed languages.

</comment></comment></comment><comment><USER10> It's been great watching Nuitka progress over the past couple of years.

I suggested the tl;dr section to Kay in the Overview ;)

I've also donated to the guy. I'd help out in the development, but to be honest a lot of what he's doing goes way over my head and I don't have the time to sit down and try to understand it all.

I encourage others better than me to help in Nuitka's development :)

</comment><comment><USER11> > SSA (Single State Assignment Form)

Is this something different from static single assignment? I'm assuming not, but I've never heard SSA expanded in this form.

<comment><USER12> You are right, this should be static single assignment.

</comment></comment><comment><USER13> This looks pretty cool, know of any production-grade software using it? I remember reading about something like this being used for Docker Compose, can't recollect what exactly that was.

</comment><comment><USER14> It's fantastic to see someone taking on the hard problems, things that take years to solve, especially if you're on your own. This is the essence of Open Source - take your time to do something right. Great stuff, good luck with it, we need more grand undertakings like this!

(This comment is mainly based on this talk: [https://www.youtube.com/watch?v=a8RRbT4BTEw](https://www.youtube.com/watch?v=a8RRbT4BTEw))

</comment><comment><USER15> I believe this project is filling the single largest gap that Python has for development of widely-distributed desktop applications. This is pure awesomeness.

</comment><comment><USER16> Looking at their future plan, can we say they are aiming to build something like RPython, but instead of a subset of Python support full Python language?

</comment><comment><USER17> Trying to understand this project. Is this more than a python compiler?

Edit: Ok, now I see that the "Overview" button is hidden on mobile.

<comment><USER18> So, from my reading of the "Overview" page, it compiles Python to C++ and then use native types as much as possible to make things run really fast.

[http://nuitka.net/pages/overview.html](http://nuitka.net/pages/overview.html)

</comment></comment><comment><USER19> The Requirements section of the user manual says that in addition to a C++ compiler, you need Python2 at compile time even if you're using Python3, due to a requirement of the Scons tool.

</comment><comment><USER20> I wonder how the performance of this compares to ZiPy.

</comment><comment><USER21> Does anyone know if this works with numpy?

<comment><USER22> It works with numpy. (I tried it and it seems to be 100% compatible, I haven't noticed any differences)</comment></comment>


Groupcache: an alternative to memcached, written in Go - <AUTHOR> https://github.com/golang/groupcache#readme <comment><USER2> For those that didn't catch it, this is from Brad Fitzpatrick, the same guy who made memcache

<comment><USER3> ASAIK, he wrote the original memcached in perl but what everyone is using now is a rewrite of memcached in C by Anatoly Vorobey [1]. Not that it diminishes his efforts or anything, just pointing it out.

[1] [http://en.wikipedia.org/wiki/Memcached#History](http://en.wikipedia.org/wiki/Memcached#History)

<comment><USER4> And then I rewrote it in C++ for App Engine and other users within Google (Blogger, Plus, hundreds others).

I also then ported the C++ memcached (called memcacheg) to Go, for profiling Go vs C++, and named that one memcachego. So I've written it about 4 times.

<comment><USER5> "I've written it 4 times, and I've gotten exceedingly good at it" :)

Kidding aside, anything you learned during the rewrites that caused noticeable changes to the design, or it's still the same thing, just transliterated?

</comment></comment></comment></comment><comment><USER6> I'd be curious to hear performance numbers (assuming a reasonable front-end server to this library). I get it that the replicated in-memory caching part is valuable. But (from painful experiences with Java) I also fear that a GC based memory management system is anti-optimal for an in-memory cache of small objects, especially as the size of heap grows to beyond a couple of GB( * ).

(* ) [http://cdn.parleys.com/p/5148922a0364bc17fc56c60f/GarbageCol...](http://cdn.parleys.com/p/5148922a0364bc17fc56c60f/GarbageCollection.pdf)

<comment><USER7> Go's garbage collector sucks pretty hard for large heaps. A 16GB heap that's mostly live objects will take tens of seconds per round, during which no goroutines will run (it's a stop-the-world collector). Basically it's 15 years behind Java in this regard.

If you're worried about this, stick with C++.

<comment><USER8> I couldn't find in the code where the objects are actually stored, but theoretically there's nothing stopping you in Go from allocating yourself up an array of X bytes (for X in the mega or gigabytes) and completely managing it yourself, leaving the GC merely for incidental activities, which themselves can be minimized through various design patterns and some care.

It is true that you should describe Go as a garbage-collected language, but unlike a lot of other such languages, Go has C-like mutable arrays readily available. In practice it's more like a sort of hybrid, in that even though it's garbage collected you have a lot of opportunities to write code that still doesn't really use it, without having to "drop down to" C or something. I still hope to see some improvements in it before I could make a big commitment to it, but I've certainly got my eye on it.

<comment><USER9> > _I couldn 't find in the code where the objects are actually stored, but > theoretically there's nothing stopping you in Go from allocating yourself up > an array of X bytes (for X in the mega or gigabytes) and completely managing > it yourself,_

Nothing stops you, except: \- The determination NOT to do something the computer can do \- Previous exposure to a proper, modern GC \- The fact that you used Go to get away from this kind of shit in the first place

<comment><USER8> I did not mean this is a technique you'd use in general. I meant specifically in the context of groupcache, where an array of bytes that you manage yourself is a reasonable approach to the problem.

If you don't want to do that, don't. I'd never start the prototype with that functionality, for instance. But when you discover that you need it, Go permits it in a way that Python or Perl do not. Go is GC'ed and you are free to use that to the extent you want, but it's easier to escape from the GC than it is in many GC'ed languages. It may not be obvious from a casual reading of the spec, but there's a lot of ways around garbage in Go. Not like, say, Rust, and certainly not the same level of Raw Unstoppable Power as C++ (at a corresponding huge complexity cost), but it's a very interesting middle ground.

Note that if you were writing groupcache in Java, you'd probably end up doing the same thing and just allocating an expanse of bytes which you manage yourself.

</comment></comment></comment></comment></comment><comment><USER10> I really wouldn't call this an alternative. If you are running memcached, it's very unlikely you can switch to Groupcache.

Parts of your application may rely on the expiration feature. But the biggest change is the inability to overwrite a current cache key. Every application I've used does this constantly (object updates).

Groupcache in its current form is useful for a very narrow set of applications.

<comment><USER11> I would think that the real limiting factor right now is that it is only available for Go applications (its a Go library, not a standalone server). There are comparably very few Go applications in the wild that could use this... where a memcached server can interface with just about anything. Or am I missing something?

<comment><USER12> Groupcache being a library is what made me so excited about it in the first place.

I'm writing a torrent tracker in Go that could make great use of a distributed cache. This would enable you to put an arbitrary number of "tracker nodes" behind a load-balancer.

One of the goals of my project is that its easy to configure: everything except the RDBMS is hosted inside a single binary and configured with a single file. In the simplest case: that binary is responsible for two web-servers, a process-local cache, and a DB connection pool. (In more complex cases, each binary simply acts as another node behind a load-balancer; and they attempt to cooperate together.)

It's useful to avoid hitting the disk by having the tracker cache torrents that are active. When you start adding more tracker nodes, this is where having a distributed cache would be nice.

Without groupcache, that means I've just forced my users to install and configure memcached. With groupcache, the configuration and server are simply part of my application. (As an added bonus, since Go is statically linked, I haven't even added any external dependencies on shared libraries.)

\---

I agree that the use-cases for groupcache are far more limited in scope, but I was still glad to hear that this was a library, and not a standalone server. I think we need _more_ distributed caches implemented as libraries, not less.

There are many use-cases where it may not be preferable to have a separate caching server. In my career: getting approval to install memcached on a customer's server could add _weeks_ to our deploy time simply because of red- tape; in my personal programming endeavors: setting up memcached is just an added layer of complexity that could easily be avoided by using libraries instead of servers.

I'd argue that we should leave memcached [more specifically: standalone distributed caches] for the more complicated scenarios where a dedicated solution is called for. Bring on the distributed-cache libraries where a much simpler, ad-hoc solution would be more beneficial to end-users.

</comment></comment></comment><comment><USER13> I want to use this, but since the keys are immutable, how can I store data like sessions which can change and would sometimes have to be invalidated from the server side (i.e. you can't simply change the session ID in the cookie and use a new cache entry, because bad-guy could still be holding on to an old stolen session ID)?

In general, how can one learn to think in an immutable fashion to effectively exploit this?

<comment><USER14> The simple solution is to always version everything. There's no such thing as an update, merely a new version of a thing.

<comment><USER15> So without any mutable storage, how do you verify that the version that the client requested is the latest version?

<comment><USER16> If the client has an old link serve an old version. But have links to what will be future versions and make the client walk them. It is doable but different. Lots of stuff is static anyway, like CSS, so you can use for this and have a different process for stuff that varies.

<comment><USER15> But as the grandparent said, serving the old version may result in a security vulnerability. There are cases where you MUST serve the latest version, and the latest version only.

<comment><USER17> Then don't use groupcache.

</comment></comment></comment></comment></comment></comment><comment><USER18> Am I reading this as a distributed, immutable, weak hash table rather than what one would consider a 'cache'?

Mind you, doing so avoids the hardest parts of caching (and especially distributed caching, which otherwise begins to underperform around ≥ 5-7 nodes), so I can see significant upside. No surprise stales, distribution update clogging, etc.

</comment><comment><USER19> I noticed this when he talked about speeding up the Google download servers. Very interesting :)

Its an alternative to memcache but not a direct replacement. I hope he adds CAS etc.

I hope they start using the kernel's buffer cache as the backing store, or explain why its not a good idea: [http://williamedwardscoder.tumblr.com/post/13363076806/buffc...](http://williamedwardscoder.tumblr.com/post/13363076806/buffcacher)

<comment><USER20> Apparently there is no CAS by design, this allows groupcache to replicate hot keys on multiple hosts.

<comment><USER19> Yes, but if it was opt-in...

Its nice if you can use groupcache as a memcache replacement even if some use- cases tie the hands of the implementation e.g. replication of hot items.

<comment><USER13> I've been thinking about something similar. I don't see how timed expiration would conflict with the two most important features - the filling mechanism and the replication of hot items. Am I missing something that would make timed expiration impossible?

<comment><USER19> Yeah on the first pass of the problem you seem right.

The CAS must have an authoritative node (my mind wanders thinking about replication and failover) but the key it protects - with the version baked in - can be replicated surely?

</comment></comment></comment></comment></comment><comment><USER21> [http://talks.golang.org/2013/oscon- dl.slide#46](http://talks.golang.org/2013/oscon-dl.slide#46)

> 64 MB max per-node memory usage

So this is best used as a LRU cache of hot items.

It doesn't compete/replace memcache comprehensively, but it does attack the use of memcache as a relief for hot items.

I can see me mixing my Go programs with both groupcache and memcache.

Edit: I have glanced through the code and cannot see where the 64 MB per-node limit comes in. Anyone see that?

<comment><USER22> It's a configurable limit. 64MB is just what he set the limit to for that particular group, that's what the "64<<20" in the NewGroup call is for. You could configure it to use many gigabytes.

<comment><USER21> Ah, that's what I was not seeing when I was glancing through the source.

In that case... for me it fully replaces memcache for how I'm using memcache today.

</comment></comment></comment><comment><USER23> How does its sharding by key algorithm work and how does it handle adding new peers? I was looking for at in the source, but couldn't find anything related to it.

<comment><USER24> You need to implement that part yourself. Look in peers.go. It'll be dependent on how you discover peers in general.

</comment></comment><comment><USER25> [https://github.com/golang/groupcache/blob/master/sinks.go#L5...](https://github.com/golang/groupcache/blob/master/sinks.go#L59)

Uh oh. The dreaded cast we see here [http://how- bazaar.blogspot.co.nz/2013/07/stunned-by-go.html](http://how- bazaar.blogspot.co.nz/2013/07/stunned-by-go.html)

</comment><comment><USER26> I'm a bit confused about how you use this system with immutable keys. At face value it's a great idea, but I need a simple example of how's it is used to say retrieve a piece of data, then later update that to a new value.

Is this anything like how vector clocks are used, where the client uses the clocks to figure out which is the right state in a distributed system?

<comment><USER27> The use-case for which this was originally designed was a download server: the values were chunks of files, each of which gets its own unique identifier. If you want to update a file, you simply create a new file for your new version, and point clients at it instead of the old version.

</comment></comment><comment><USER28> I like the idea, but it seems like it would make deployment a pain. How do I spin up a new server without rebalancing and/or restarting the world? Not to mention that now when I _do_ need to restart the world, I can't do so without also clearing my cache.

<comment><USER29> [http://en.wikipedia.org/wiki/Consistent_hashing](http://en.wikipedia.org/wiki/Consistent_hashing)

</comment></comment><comment><USER30> How does this compare to Hazelcast[1]? Seems like the same idea, but far less features?

[1] [http://www.hazelcast.com/](http://www.hazelcast.com/)

<comment><USER31> Hazelcast is incredibly buggy. Avoid at all costs.

<comment><USER32> Got any links\talks\papers on this as a dev group at my work just started using it.

<comment><USER31> Just personal experience, sorry.

* it deadlocks. Often.

* it often doesn't recover from network partitions. Symptoms include both sides of the partition never merging, and deadlocks. Recovering requires rebooting the entire cluster at once. Due to their locking strategy, a rolling reboot of the cluster isn't enough.

*some of the developers didn't seem to demonstrate understanding of races, or distributed systems. When I reported racing bugs, they asked if I could attach reliable unit tests.

</comment></comment></comment></comment><comment><USER33> It'd be interesting to compare this alternative to other solutions like Redis.

Has anyone used both/either?

<comment><USER34> If I were to speculate, I would say that Redis is probably faster.

<comment><USER35> TIL that waiting for a network roundtrip is faster than accessing an in- process hash map directly.

</comment></comment></comment><comment><USER36> How do you set an item's TTL? How do you set the maximum size of the process's cache?

<comment><USER37> > How do you set an item's TTL?

Values stored are immutable, and expire when they are no longer used. There is no concept of a TTL.

> How do you set the maximum size of the process's cache?

The cache is broken into a set of named groups, each with their own separate cache and cache-filling method. The size of each group's cache is set when created via NewGroup[1].

\--

[1] [http://godoc.org/github.com/golang/groupcache#NewGroup](http://godoc.org/github.com/golang/groupcache#NewGroup)</comment></comment>


Evidence that the NSA Is Storing Voice Content, Not Just Metadata - <AUTHOR> http://www.schneier.com/blog/archives/2013/06/evidence_that_t.html <comment><USER2> The focus on content over meta-data is a red-herring. What you say during the phone call is just that...something you say. The meta-data shows what you do.

A person browses the web at a doctor's office. Forty-five minutes later, that person calls their spouse, then an oncologist. A few days later, their spouse checks email from the oncologists office. An hour later a call is made to a diagnostic center by the first person. The next day, they call their mother and a surgeon. [see interview with Susan Landau: [http://www.democracynow.org/2013/6/12/more_intrusive_than_ea...](http://www.democracynow.org/2013/6/12/more_intrusive_than_eavesdropping_nsa_collection#) ]

The conclusions which can be drawn from that meta-data are far more solid than can be drawn from listening in to any or all of the phone calls.

Your phone calls don't show what time you get up in the morning, or when you get to work or walk the dog to the park. The meta-data does.

The focus on the content of calls is 50 years out of date. The meta-data show the best time to burgle your house or fake your identity online. They show where you go and who you associate with. There's no need to know what you said.

<comment><USER3> I disagree. You give a situation which is suggestive of the person learning they have cancer, and is a perfect example of how mass tracking can expose personal but information about our lives that we would rather keep private (particularly medical information, which I believe is the most vital kind of privacy threatened by these mass tracking schemes). But I wouldn't say that conclusion that you suggested is solid. It is equally plausible that the diagnostic center told the patient that results were negative, and the call to the mother was to reassure her and to the surgeon to cancel a preemptive appointment made by the doctor.

> The conclusions which can be drawn from that meta-data are far more solid > than can be drawn from listening in to any or all of the phone calls.

This is just false. If you were a lawyer trying to draw a portrait of intentions with the metadata in court, obviously it would help your case if you could play the tapes and show what was said in each of the calls, so the other side can't propose equally valid situations like I did and create reasonable doubt.

I think we can be so fascinated by the power of having data, that much like the other powerful forensic tools used in the courtroom, we are eager to connect all the dots like we see TV shows like CSI doing. It can be difficult to remind people that more data (particularly meta-data and not content) produces more plausible hypotheses as often as it narrows them down.

<comment><USER4> But the problem is that there is no court, and they (NSA or whomever) are left to make their own conclusions and act on them before we have a chance to defend ourselves. In the example, you can draw (at least) two conclusions, the problem is: the NSA gets to pick the one that suits their needs and defend it with misinterpreted "meta data"

<comment><USER5> ... in secret court.

</comment></comment></comment></comment><comment><USER6> I wouldn't be shocked to learn that their interpretation of "not listening in" does not include reading speech-to-text transcripts of conversations.

<comment><USER7> I suspect it means "We don't have a guy listening to your line in real time."

<comment><USER8> Or even better: "We said we aren't listening to _your_ phone calls. We didn't say we aren't listening to a _lot_ of phones, just (probably) not _yours_.

Their word manipulations are sickening. I just wish all of this went to a trial already and to the Supreme Court. Let's see them lie through their teeth to the judges then, who will actually understand everything they're saying or avoiding saying (unlike most of the press, or people out there).

<comment><USER9> But even if they lost at the Supreme Court, wouldn't they just appeal to the Secret Supreme Court?

</comment></comment></comment></comment><comment><USER10> Anyone else getting this..?

Technical Details

            www.schneier.com uses an invalid security certificate.

The certificate expired on 18/06/13 11:55. The current time is 18/06/13 14:02.

(Error code: sec_error_expired_certificate)

<comment><USER11> Yup, its expired.

<comment><USER12> another reminder of false security of certificates, because no one pays attention to them. Here's a security blog, so I will speculate that many of its readers are security conscious users, yet most probably went to the site anyway despite the security warning of an expired certificate.

<comment><USER13> What's the threat model for this? Is the MITM going to subtly change Schneier's essay? Perhaps they're going to find out the lame password I use for stupid comment forms, which password (literally: it's a word) I've been using continuously for that purpose since 1994?

<comment><USER14> Or inject an exploit into the stream and compromise the system being used to view it.

<comment><USER15> I would not think that is much of a threat here in this scenario. The threat is really that Schneier is no longer who he said he is, since it has not been validated recently. That is, his certificate, purchased and verified through an authority, has reached the age where that authority no longer guarantees that he is the one holding it. As such, someone else could have taken over his person and began acting maliciously.

Right?

Edit: So, my question "Right?" was a legitimate question. If I am wrong, I'd like to know how. Note that this is an expired, non revoked certificate scenario we are talking about. Meaning the identity was established before, and to nobody's knowledge has it been stolen. Simply now that identity has not been established for a long time.

<comment><USER14> If you put any faith into the CA systems verification process then you'd be correct.

<comment><USER15> But if you don't have faith in the CA system, then what is the additional concern over an expired cert?

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER16> Why would the NSA not be able to store all phone calls as audio? If we assume 30 minutes per person per day to store at compressed 20kbps at $0.12 per gigabyte per year (this is what you pay at Amazon), that costs $60 million to store for the data of a year's worth of phone calls of the entire population of the US. The budget of the NSA is more than $10 billion, so that's less than 0.6% of its budget. That is entirely doable. Note that I have grossly overestimated the cost here by assuming that price per gigabyteyear is what you pay Amazon, and 30 minutes per person per day is probably an overestimate, and the compression could be better too. The actual cost is probably closer to $10 million; less than one thousandth of the NSA's budget.

<comment><USER17> Also, Amazon Glacier is only $0.01 per GB.

</comment></comment><comment><USER18> In 1998 a colleague of mine received a DARPA request to develop a tape-based recording device that -- it was obvious from some quick calculations -- would be capable of archiving all voice communications going in and out of the country. We could think of no other application for that kind of technology, and assumed that the real client was the NSA.

At the time, storing such a quantity of data was completely infeasible within the physical space & budgetary constraints of the proposed program was completely infeasible. We told them so, and the project went away. However, given that:

1.) They were trying to do this 15 years ago, and

2.) Both their budget and the technological state of the art has improved substantially since then, and

3.) The government these days seems happy to treat citizens' rights with the same general contempt as non-citizens' rights, given some creative re- definitions of terms and rubber-stamp lawyering...

...I would not be at all surprised to learn that a program like this was in fact well-established by this point.

</comment><comment><USER19> The more I read about the subject the more I believe all this data is primary used for commercial reasons.

What else is to gain from monitoring loads of targets without obvious security reasons?

<comment><USER20> The biggest component of espionage is always industrial espionage.

<comment><USER21> Exactly. Personally, I'd be playing the currency markets if I had access to all this data. I'm curious how many people who are monitored are politicians and financiers (bankers/investors). The potential to abuse other economic markets relative to your own is enormous.

</comment></comment></comment><comment><USER22> Major media reported that the Boston bomber's wife's phone calls she made prior to the bombing were listened to, after the bombing.

</comment><comment><USER23> NSA is testifying right now in front of congress BTW

Oh wait no, it's the Deputy Attorney General

He says the 4th amendment doesn't apply to phone records.

Footer says "NSA director to reveal terror plots stopped by surveillance"

<comment><USER24> [http://www.c-span.org/flvPop.aspx?id=10737440108](http://www.c-span.org/flvPop.aspx?id=10737440108)

</comment></comment><comment><USER25> > And, by the way, I hate the term "metadata." What's wrong with "traffic > analysis," which is what we've always called that sort of thing?

This has also been bugging me. Metadata is a very general term, and it doesn't explain what the NSA claims its doing (whether they're doing anything else is beside this particular point). Moreover, the use of such a general term seems like it's part of the propaganda, to make us less scared: "We're not collecting data, we're collecting meta-data." Well, it turns out that they are one and the same anyway.

</comment><comment><USER26> This isn't evidence the NSA is storing voice content, this is Schneier saying, "One reason I used to discount the idea that the NSA was storing all phone conversations was that it'd be too much data but now I don't think it is."

Not new evidence.

</comment><comment><USER27> Obama says [1] that if you're a "US person" (whatever that means), "The NSA is not listening to your phone calls."

Does storing them for later use count as listening?

[1] [http://www.cbsnews.com/8301-250_162-57589732/obama-on-nsa- pr...](http://www.cbsnews.com/8301-250_162-57589732/obama-on-nsa-programs- americans-not-getting-the-complete-story/)

<comment><USER28> Not according to Clapper. I'm sure they're all on message with their phrasing. They are no doubt asserting that when you pick up a phone, someone is probably not listening to your phone call at that moment, as if we need assurance of _that_.

</comment></comment><comment><USER14> Lossy compression turns any data into metadata.

</comment><comment><USER29> To query the database, you need to know the phone number's area code is inside the united states (...? like that's fucking difficult?), then you must get "a further review" to see if they are "just expressing their first amendment rights", and then one of 20 analysts and 2 managers must approve it.

Go check out all the things that are _not_ protected by the first amendment. [http://en.wikipedia.org/wiki/United_States_free_speech_excep...](http://en.wikipedia.org/wiki/United_States_free_speech_exceptions)

If we held people accountable for all the false statements of fact they make, FOX News would have been off the air years ago and all their newscasters thrown in jail. Basically, millions of people could be subject to database queries, considering how many loopholes there are.

One example they cite using the 215 was the NSA provided a phone number to the FBI, the FBI served notice to the court to find out who the number belonged to, and they then arrested and convicted the guy for giving money to a foreign organization that the USA labels a terrorist organization. So don't do business with anyone who might know a group of freedom fighters.

</comment><comment><USER30> Lots of debate here on what is more powerful, the data or the metadata. I think in the case of voice content, a case can be made that the metadata is more powerful, simply because it's already parsed into a quantitative, objective format that's relatively easy to analyze en masse and find "suspicious" patterns within the public. With voice content, I don't think our natural language processing chops are up to par to deal with the massive amount of data and connect the dots in meaningful ways.

Of course, if you're already targeting a specific individual, and you can get a human to listen to the voice content, the debate is academic - both kinds of data compliment each other and are equally powerful. Metadata still acts as a better "gateway drug" for narrowing down individuals though.

</comment><comment><USER31> CNN already broke this story in early May. [http://www.youtube.com/watch?v=vt9kRLrmrjc&feature=share](http://www.youtube.com/watch?v=vt9kRLrmrjc&feature=share)

<comment><USER32> It is considerably more credible when an accepted industry expert commentates, over the mainstream media.

<comment><USER31> Did you watch the interview? They have an FBI agent talking about it. Do you think he's not really an FBI agent?

</comment></comment></comment><comment><USER33> Well given that youtube can store such vast amounts of _video_ the suggestion that the NSA is storing lots of voice doesn't seem like a massive stretch.

</comment><comment><USER34> I think what this points out clearly, and Schneier has been preaching this for years, is that we need to encrypt _everything_ going forward.

<comment><USER35> Speaking for myself, I'd like to encrypt the content of my communications as much as possible. But I can't control or encrypt the meta-data, such as numbers I dial, how long I talk, etc.

</comment></comment><comment><USER36> My experience has been that commercially available speech-to-text technology doesn't work very well on non-US-accented English. Maybe fixing that tech will be one benefit of PRISM. You know, just like we thank the Apollo space program for Velcro and pens that can write upside down.

</comment><comment><USER37> Yep! We've actually known this for years. [https://www.eff.org/deeplinks/2010/03/wiring-big-brother- mac...](https://www.eff.org/deeplinks/2010/03/wiring-big-brother-machine)

</comment><comment><USER38> So it turns out we should have actually listened to Shia LaBeouf back then:

[https://www.youtube.com/watch?v=dNRgP4FVDzA](https://www.youtube.com/watch?v=dNRgP4FVDzA)

</comment><comment><USER39> How about the simple fact that you don't need multiple data centers with capacities measured in exabytes to store "metadata"?

</comment><comment><USER40> Surely a compressed call recording is also 'metadata'?

</comment><comment><USER41> security certificate expired!

</comment><comment><USER42> Warning: this article does not present any evidence.

Schneier is a great cryptologist, I've read his books, I've carried them around, I'm a big fan of his work.

BUT.

He's seriously lagging behind in his coverage of this scandal, it's like he's just reposting what others have already said and often stating the obvious.

So the NSA has the capability to store voice? No, really? Like since 1940????

Sorry, Bruce, but this article is shit.</comment>


Reliable MacBook Pro replacement battery? - <AUTHOR>

Pardon me if this is a gauche question to post here, but I couldn't think of a more suitable online community to ask.
The battery on my early 2008 MBP is almost completely toast, and unfortunately I haven't found a replacement battery by anyone that got enough good ratings over the bad ones to make me feel good about a purchase.  I realize that YMMV with these kinds of things, but I trust you folks more than Amazon with matters of this kind.  Any pointers?  Thanks, all.
John <comment><USER2> I replaced mine w/ a battery from fast mac and it worked out well for me.

<http://fastmac.com/></comment>


Targeting Activity Against State Board of Election Systems [pdf] - <AUTHOR> https://s.yimg.com/dh/ap/politics/images/boe_flash_aug_2016_final.pdf <comment><USER2> some boilerplate fbi report that someone hit some board of election hosts that was vulnerable to sql injection. it ask to look at logs instead of upgrading anything, so my bet is that someone on fbi puts job security above public safety?

also this domain is Yahoo's cdn, so it was probably mentioned in some new item but posted here as a loose pdf?

<comment><USER3> Yeah, seems to be this news story:

[https://www.yahoo.com/news/fbi-says-foreign-hackers- penetrat...](https://www.yahoo.com/news/fbi-says-foreign-hackers- penetrated-000000175.html)

HN discussion (162 comments):

[https://news.ycombinator.com/item?id=12381337](https://news.ycombinator.com/item?id=12381337)</comment></comment>