Why is chicken so cheap? [video] - open-source-ux https://www.youtube.com/watch?v=JiYVoHEV5hs <comment><AUTHOR> Maybe Farmers should be required to print a photo of the actual living bird and their habitat on the packaging. When consumers would see those scruffy chicken for $2 compared to the healthy $7 they might choose the organic ones more often.

Killed and unfeathered they all look the same.

<comment><USER2> This reminded me of the "Colin the Chicken" skit on Portlandia where a couple ordering chicken gets to see its birth certificate, diet, emotional profile, heritage and family...

[https://www.youtube.com/watch?v=G__PVLB8Nm4](https://www.youtube.com/watch?v=G__PVLB8Nm4)

<comment><AUTHOR> Unavailable in my country :-/

Probably the same as this one?

[https://www.youtube.com/watch?v=WAlWrT5P2VI](https://www.youtube.com/watch?v=WAlWrT5P2VI)

<comment><USER3> yeah

</comment></comment></comment></comment><comment><USER4> Wow, 39 is much shorter than I expected.

I guess it makes sense, because how else could you keep up with demand, but my view of animal lifecycles only has stuff like bugs fully maturing so quickly.

Interesting stuff.

Now my question is why are bananas so cheap? They come all the way from like Brazil, but they're still cheaper than the apple that is grown just a couple hours from me.

<comment><USER5> Apples are also more expensive than oranges, which I too find strange.

<comment><USER6> Cheaper by what metric? Also, it depends on the variety. FWIW, my experience is that oranges are usually more expensive than apples, except for the few weeks that oranges are in season.

</comment></comment></comment><comment><USER7> Breeding a "product" with a small gene pool, churning out monoculture: [https://en.wikipedia.org/wiki/Gros_Michel_banana](https://en.wikipedia.org/wiki/Gros_Michel_banana) or a more imaginative corrective: [https://en.wikipedia.org/wiki/Influenza_A_virus_subtype_H7N9](https://en.wikipedia.org/wiki/Influenza_A_virus_subtype_H7N9) makes a plant based diet a very good hack indeed.

<comment><USER8> You don't know what the word monoculture means until you stand in the middle of a soybean field in the midwest and see nothing but more soybeans (from the same small handful of varieties provided by giant seed companies) as far as the eye can see. Please do not imagine for one second that a plant-based diet increases biodiversity.

<comment><USER9> The fact that we have so much monoculture soybean fields is that they make up the feed for the animals in Factory farming.

Here is one link I just found that point out to the fact '70-75% of the world's soy ends up as feed for chickens, pigs, cows, and farmed fish.': [https://www.ucsusa.org/global-warming/stop- deforestation/dri...](https://www.ucsusa.org/global-warming/stop- deforestation/drivers-of-deforestation-2016-soybeans)

<comment><USER8> You could pick any plant humans consume and it is inevitable that the vast bulk of it would be produced in large, efficient, monoculture operations. The delusion of small farms producing 'heirloom' vegetables in copious variety is a bourgeoisie fantasy of the western middle-class who has come no closer to a real farm than the local Whole Foods. The soy in your tempeh and tofu comes from those same farms, the same operations produce the feedstock of your kale chips and organic lentil puffs.

We have monoculture farming because it is efficient and people (not) surprisingly have decided that they have no interest in paying more for alternative production processes.

<comment><USER9> We can't conclude that this is the only way, especially with advanced automation or subsidies for ecological and sustainable agriculture. Furthermore animal farming must be the least efficient way to provide calories. So I don't understand how you can talk down solid alternatives like tempeh or tofu.

(BTW in the link i posted above it says human products take up 6% of soybean production.)

<comment><USER8> You can imagine any fantasy scenarios you want if it makes you feel any better, but nothing in the past century of agricultural progress has had a significant impact on the economic factors motivating the industry.

Animal farming is not the most efficient way to provide calories, but humanity does not lack for calories. Livestock (excepting ruminants that graze on land that is unsuitable for farming) is a caloric aggregator, turning a large chunk of calories into a smaller volume that is easier to transport and more desired by the market. If you want that to change you need to meet the market's demands (c.f. the unicorn valuations of a few new fake meat companies...)

<comment><USER10> "Animal farming is not the most efficient way to provide calories, but humanity does not lack for calories."

Actually, a lot of humans do go hungry. The inefficiency of using land for crops to feed to livestock means that the price of staples is higher than it otherwise would be.

While meeting market demands is a large part of what will change things I think you're missing a piece on how this will come about. If there were no vegans, the chance of vegan unicorns succeeding would be much smaller. If some people did not harp on about the inhumanity of industrial farming, others would be less motivated to try alternatives and the chance of vegan unicorns succeeding would be reduced.

<comment><USER11> > Actually, a lot of humans do go hungry. The inefficiency of using land for > crops to feed to livestock means that the price of staples is higher than it > otherwise would be.

I'll posit that even if you made grain free at the docks in the US, there would _still_ be people in the world going hungry. (Fewer people, certainly) Transportation and distribution is just as large a problem as production at this point in history.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> Chickens are one the most successful animals on earth! By evolving to become fatter, more succulent, and delicious, they have convinced humans to feed and house them, and in doing so have multiplied to vast numbers and spread their genes almost everywhere across the globe.

<comment><USER13> black people, too! we were so strong and resilient and good at labor that humans paid for our travel across the seas and gave us the opportunity to help build America! we were fed and housed and everything, oh my! gosh

> Shadow with me while I'm steppin' on my own resentment

> Life scatter in all directions, I was overzealous

> Overcurious, Momma worry her days get better

> I know she heard me, a timid voice in her stormy weather

> And Poppa taught me our ancestors were tarred and feathered And brought > across the sea, bodies swinging from poplar trees

> I wore a modernesque version, my burden haunted me I cautiously approach the > rather daunting sea

> By nightfall I face the man I'm 'sposed to be All this grief, been eatin' > away my stomach lining

> It's hard to eat when my poppa image stuck inside me

> I wore his death mask,

> smilin' through the trauma

> In his honor,

> I'm expounding

<comment><USER6> Animals are not people and they do not occupy the same moral category.

<comment><USER13> blacks weren't people then, either.

back then, blacks didn't occupy the same moral category as the humans who bred and traded them.

afterwards, blacks were upgraded to 2/5ths human.

now, we still struggle for acceptance and parity.

morality is a process.

<comment><USER6> Animals are categorically not humans. This is a scientific fact which you can quantify and measure. Blacks are humans. That is another scientific fact which you can quantify and measure.

<comment><USER13> blacks being humans didn't stop us from slotting them into an inferior moral category. and back then, we asserted as scientific fact that they were inferior. the apparatus of justification is something which changes over time. it is interesting to me to observe your invocation of our present day apparatus of justification. :)

<comment><USER6> Yes, but we came to see blacks as equal _because_ they are, in fact, humans, just like whites. Animals will never be humans, no matter what. They will always remain a separate moral category. Here's a simple thought experiment that should illustrate this:

Imagine a scenario in which you can save one of two lives: a human or a dog. You can't pick both. Just one. The question is not, "Which would you save?", but instead, "Would you final moral fault in a person who chose to save the dog?". Almost everybody would hold a person who chose to save the dog morally culpable for the death of the human. But they would not hold them morally culpable for the death of the dog if they chose to save the human.

That will never change. Because animals are not humans.

<comment><USER13> that's post-hoc rationalization.

we came to see blacks as humans because lots of people shed blood and fought for their rights. nothing was handed to anyone for free.

<comment><USER6> So you'd find no moral fault in a person who chose to save a dog instead of a human?

<comment><USER13> It is disappointing to observe you argue so strongly from a position of deep and profound ignorance on this topic. have you no doubt?

Recommended reading: [https://plato.stanford.edu/entries/moral- animal/](https://plato.stanford.edu/entries/moral-animal/)

<comment><USER6> Why would I be persuaded to change my mind when you A) won't answer a simple question, and B) think it's reasonable to take the position that I am wrong because I won't spend several hours digesting some link you've provided, which you can't be bothered to spend three sentences summarizing?

My premise is simple: almost every single person would find a person who chose to save the life of an animal over a human morally culpable for the death of the human. Whereas they would not find a person morally culpable for the death of the animal if they chose to save the person instead. That asymmetry directly speaks to the different moral stations that animals and humans occupy.

You have yet to level a single actual argument against that premise.

<comment><USER13> I am not responsible for your education. Take responsibility for yourself and have a nice life.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER14> I always found it a bit tacky to call it an "achievement" for the animal/plant that humans farm it. Same with "evolutionary" changes due to selective breeding. Both said in the video.

Always feels like someone is trying to spin reality on me. "What? The birds love it! Instead of living in nature in a part of Africa, they are suffering worldwide to the tune of billions!" Their hard work paid off!

<comment><USER15> What sort of lives do you think birds living "in nature" in Africa have?

Instead of being painlessly gassed to death like farmed chickens they'll be eaten alive by some predator, or instead of being provided with antibiotics as hatchlings perhaps worms will eat most of their siblings alive before they leave the nest.

Romanticizing factory farming is ridiculous, but so is romanticizing the lives of wild animals.

From a species interaction perspective the farming of animals could be described as mutualism, but humans tend to break all the rules in traditional species interaction models: [https://en.wikipedia.org/wiki/Biological_interaction#Mutuali...](https://en.wikipedia.org/wiki/Biological_interaction#Mutualism)

We're not unique though: [https://en.wikipedia.org/wiki/Ant%E2%80%93fungus_mutualism](https://en.wikipedia.org/wiki/Ant%E2%80%93fungus_mutualism)

<comment><USER16> Can't say that comparing the appalling conditions in bird factories with a life lived in the wild really compares here.

At least wild birds get to make a go of it, the factory birds never had a chance. Some of them are too fat to even walk and never even get to leave their cage.

If I was given the choice I'd choose the life of the wild bird over the caged one any day.

<comment><USER6> Just playing devils advocate here, not actually arguing either way, but with the following you have actually described the life that some people choose to live (by some definition of the word "choose"):

> Some of them are too fat to even walk and never even get to leave their > cage.

</comment></comment></comment></comment><comment><USER17> Whenever I travel outside the US, I'm always struck by how different the chickens look. Chickens in the US have a pale appearance, with flabby, almost mushy meat. They look as if they were grown on a tree, never alive and running. While chickens in say, Italy or Mexico tend to look like they were alive at some point. There's color in the meat, often a yellow tint. Their skin and muscle look used and not atrophied.

I wonder how these differences extend to nutritional value?

<comment><USER18> I've noticed the same thing with tomatoes and apricots. I've got a good friend whose family have an apricot farm in the Kurdish part of Turkey, and their apricots are amazing, but the same variety grown in Western Washington tastes like plastic. There's something seriously pathological with the "terrior" in the western part of north America. (Luckily it doesn't extend to wine.)

<comment><USER19> I'll push back on that a bit.

(A) The _terroir_ for wine is probably the pickiest of all mass produced fruit, at least according to White's _Understanding Vineyard Soils_.

(B) The soil of the Western US is extremely varied but it's hardly the only factor involved in the taste and cultivation of fruit. The most obvious wildly differing factor is climate. The high deserts of E. Oregon would produce very different fruit compared to Washington's coastal region even if you used identical soil, due to rainfall, mean/median temps, diurnal temp variation, etc.

</comment></comment></comment><comment><USER20> Land usage per chicken, difference between free-range vs. intensively reared:

1/12 m^2 vs. 1/17 m^2

Consumer cost, difference between free range vs. intensively reared:

4 euros/lb vs. 2 euros/lb.

Does the difference in production costs explain the drastic consumer cost difference between these two methods? Why can't free-range farming be automated like it is in the intensive rearing model?

<comment><USER21> Chickens do not eat in the dark. That's why the light in stables for intensively reared chicken is on way longer than the sun is up (I'm not up to date on what's considered optimal today, but I think 24 hours a day was abandoned because it caused too much stress (and stress harms growth, or even kills chicken) for weird things such as "40 minutes on, 20 minutes off")

Some of the large factors that affect production costs are feeding the chicken and e.g. capital costs of the stable, but optimizing production per dollar more or less boils down to getting the "feed conversion ratio" down.

It is about 1,6 for intensively reared chicken in the USA, meaning they have to eat 1,6 kg of food to gain a kilogram of weight ([https://en.wikipedia.org/wiki/Feed_conversion_ratio#Poultry](https://en.wikipedia.org/wiki/Feed_conversion_ratio#Poultry))

For comparison, you need two kg of feed to get a kg of eggs, so eggs are about 20% more expensive per kg than chicken meat.

You can't keep chicken that grow that fast to an age of 80 days, as they would break their legs from excessive weight.

Free range chicken are different breeds that have a higher FCR.

<comment><USER20> Thank you for an interesting comment.

</comment></comment></comment><comment><USER22> because chickens are cheap to feed and breed

you don't need automation to make it cheap

My family used to breed any sort of livestock, from chickens to lambs, from cows to pigs, from rabbits to pigeons

High quality meat, just for the family

chickens are by far the cheapest and easiest

(rabbits get sick easily, pigs eat a lot, cows need a lot of work to keep them clean etc. etc.)

<comment><USER6> Right, but _why_ are they the cheapest form of meat? There's probably some biochemical or energy utilization reason. Perhaps it has to do with chickens being fairly lean meat. I don't know, but have often wondered.

<comment><USER23> Contrasting against Pigs and Cows:

1\. Small animal which requires less bone structure to support (bone scales non-linearly with mass for land animals).

2\. Shorter lifespan and use of eggs allows simpler and quicker sexual selection.

3\. Less intelligent animal so less energy and nutrition spent on growing the brain and maintaining its function.

Its a smaller animal, basically. Fish and insects are more efficient than chickens.

[https://ib.bioninja.com.au/_Media/feed- conversion_med.jpeg](https://ib.bioninja.com.au/_Media/feed- conversion_med.jpeg)

<comment><USER15> The main reason has little to do with any of that, but just historical inertia.

There's a lot of species that could have potentially replaced the chicken if we unwind human history and started anew, but it's not a competition between wild species of birds today, but chickens with at least 5000 years of human- guided domestication v.s. most other species.

The same can be said about pretty much anything people eat and farm. Most modern fruit and vegetables were pretty much inedible in their primordial pre- domesticated forms, it's only through thousands of years of effort that we have what we've got today.

In terms innate qualities making chickens suitable for domestication over other birds, some of those are probably:

1\. They're pretty much flightless[1], even in the wild. This excludes most bird species right off the bat. Hard to domesticate something that'll just fly away.

2\. They're comfortable in groups, and have a group hierarchy. This tends to be common among species humans have domesticated. Good luck domesticating e.g. birds of pray productively, they'll constantly be fighting each other for territory.

3\. They're not picky about food, and don't require humans to actively feed them in the context of pre-industrial village life. Some birds only eat say insects, chickens can eat pretty much anything, even other chicken.

1\. [https://en.wikipedia.org/wiki/Red_junglefowl](https://en.wikipedia.org/wiki/Red_junglefowl)

<comment><USER6> I think your #2 is a pretty good insight.

Separately, I'm more interested in why chickens are more economically efficient than, say, pigs or cows than why they're more efficient than other birds. Any thoughts there?

<comment><USER22> #2 is pretty obvious, birds of prey are predators, you don't put predators together in the same space

But the same happens if you put two roosters in the same hen

chickens are not much more efficient than many other birds though

just more common

</comment></comment></comment></comment></comment></comment><comment><AUTHOR> This reminds me of this 'In a Nutshell' video:

[https://youtu.be/NxvQPzrg2Wg?t=163](https://youtu.be/NxvQPzrg2Wg?t=163)

</comment><comment><USER24> I feed my dogs almost exclusively chicken leg quarters and can get them for as low as 39 cents per pound (USD).

Chicken is insanely cheap.

<comment><USER25> I'm curious: do you remove the bones or are the dogs able to eat the bones with no ill effect?

<comment><USER24> They eat bones. Raw bones are safe, cooked bones are not.

</comment></comment></comment><comment><USER26> When I went to America, I couldn't believe how cheap chicken was. I once found boneless chicken breast for $1.99/pound. Eggs were dirt cheap as well, sometimes less than $1.00/dozen.

I don't remember ever seeing prices like these in Canada.

<comment><USER27> My record for a dozen eggs was $0.37/dozen in summer of 2017 at an Aldi in a town go about it 30,000 in Iowa. It was roughly that price the whole summer.

From my understanding, that was below the price of production. 2017 was a very tough year for chicken farmers, with a huge oversupply. The chicken rendering (killing) operations were backlogged by many months. Farmers often had no choice but to raise the chickens they already bought, knowing full well they're going to be sold unprofitably.

<comment><USER28> My record for chicken is $.35/lb for chicken drumsticks. It was however from a wholesale store and came in a 40lb box. To me that's more frightening than anything. Like how is that chicken so cheap...

</comment></comment></comment><comment><USER29> That man and his automatic shed. How depressing. I'd much rather pay 3x as much, eat chicken 1/3 as often and actually enjoy it instead of eating it every day and hardly noticing it.

All those cheap chickens may as well have stickers on saying "1/3 the enjoyment"

<comment><USER14> > eating it every day and hardly noticing it

Our distant relationship with where meat comes from is one of the most bizarre things about humans. Especially next to our psychopathic relationship with our own pets.

I've seen people freak out with disgust because there's a tiny feather or some dirt in their carton of eggs. Or disgusted by the hanging, swinging corpses behind the butcher when they go to order something that's sliced off the corpse.

<comment><USER5> There are quite a few (relatively) famous pigs on Instagram.

Try suggesting to someone that maybe they should be turned into bacon this Christmas. I'm not talking about vegans, but regular meat-eating people. Study reaction.

<comment><USER30> That's a entirely reasonable position; those pigs are more useful as entertainment than food, by a rather large margin too.

<comment><USER31> As someone with a rack of ribs currently slow cooking in the oven, I beg to differ!

<comment><USER30> Unless you mean that you don't find the pigs on Instagram entertaining, I don't see your point. Clearly there is no grave shortage of food pigs if you're eating one, so the marginal benifit of having a few extra food pigs is much less than the benefit of using those few pigs for entertainment. Kind of like how it's better to use marble for statues or architecture than for landfill, because there's plenty of landfill rock already available.

</comment></comment></comment></comment></comment></comment><comment><USER32> Related: [https://news.ycombinator.com/item?id=19743006](https://news.ycombinator.com/item?id=19743006)</comment>


"Hi, my name's Sketchfab, and I casually just did Z-brush for your browserz" - <AUTHOR> https://labs.sketchfab.com/sculptfab/

<comment><AUTHOR> Hi guys. I realize now this piece of magic was not built by Sketchfab but Stéphane Ginier. I was so "wow" that I didn't check and just posted it here. I'm a Sketchfab-enthusiastic but this was not a promotional message. My bad.

</comment><comment><USER2> This is absolutely amazing! However, it reminds me a lot more of the early versions of - [http://pixologic.com/sculptris/](http://pixologic.com/sculptris/) \- than it does of ZBrush.

</comment><comment><USER3> Nota bene: Sculptfab is an experiment made by Sketchfab on top of the amazing SculptGL tool developed by Stéphane Ginier, in order to make the workflow of publishing a model to Sketchfab easier :)

</comment><comment><USER4> Since when do we congratulate companies for slapping their logo on someone else's open source project? And this after Stephane added a Sketchfab link in the original. Seriously guys, lame.

<comment><AUTHOR> For the record, copy-paste of my comment on the root.

"Hi guys. I realize now this piece of magic was not built by Sketchfab but Stéphane Ginier. I was so "wow" that I didn't check and just posted it here. I'm a Sketchfab-enthusiastic but this was not a promotional message. My bad."

Just a simple mistake. Really.

</comment></comment><comment><USER5> Awesome! Thanks for a such wonderful resource !

</comment><comment><USER6> Wow, what incredible stuff. Congratulations.

</comment><comment><USER7> Beautifully done! Nice work!</comment>


Free Fonts for 2014 - <AUTHOR> http://www.awwwards.com/the-100-greatest-free-fonts-for-2014.html

<comment><USER2> I gave up on this list of "free" fonts after the first four were:

* Non-commercial use only

* Not able to be shared

* Actually costs money

* No-derivatives, which would seem to prevent actually using the font at all.

<comment><USER3> High quality free fonts:

\- [http://www.studiopress.com/design/google-font- combinations.h...](http://www.studiopress.com/design/google-font- combinations.htm)

\- [http://www.smashingmagazine.com/2014/03/12/taking-a- second-l...](http://www.smashingmagazine.com/2014/03/12/taking-a-second-look- at-free-fonts/)

\- [http://www.exljbris.com/](http://www.exljbris.com/)

\- [http://jamiewilson.io/norwester/](http://jamiewilson.io/norwester/)

<comment><USER4> [https://www.theleagueofmoveabletype.com/](https://www.theleagueofmoveabletype.com/) also open source

</comment></comment></comment><comment><USER5> So, to download the few of these that I like, I either have to login with FB and allow access to all my friends, or use Twitter and allow the font owner to post tweets to my account. Sorry, but that's not exactly what most people expect when they read "free fonts."

<comment><USER6> There is so much crap like this on the internet that I use Multifox to simultaneously stay logged in to my personal fb/twitter and my fake fb/twitter for crap like this.

[http://br.mozdev.org/multifox/](http://br.mozdev.org/multifox/)

It's only 1 additional step of 'open in new identity profile' then these people can access my precious fake emails and non-existent friend lists :)

<comment><USER5> Brilliant!

</comment></comment></comment><comment><USER7> Why is it so hard to provide a nice sample sheet for fonts? Just a paragraph (or just a sentence for display fonts) in the various weights would be a huge step forward in most cases.

Just compare the samples of Hallo Sans (the first font in this list): [https://www.behance.net/gallery/Hallo-Sans-Free- typeface/132...](https://www.behance.net/gallery/Hallo-Sans-Free- typeface/13212059)

To that of The Enschede Font Foundry's Trinité: [http://www.teff.nl/fonts/trinite/pdf/trinite_a3_print-600.pd...](http://www.teff.nl/fonts/trinite/pdf/trinite_a3_print-600.pdf)

</comment><comment><USER8> None of these fonts are SIL. So why is this on the HN front page?

<comment><USER9> SIL is just one of many open source licenses, and they said 'free', not 'open- source'.

I think it's on the front page because many people may be interested in modern type trends and at the very least this is an excellent collection of ideas to look at. If you want to download one for free you can too!</comment></comment>


The dirty world of Facebook EdgeRank Optimization - <AUTHOR> http://www.elezea.com/2013/02/facebook-edgerank-optimization/

<comment><USER2> Facebook seems to be making it more and more difficult for a user to see just "a feed" with all their friends' activity; instead they optimize things like this and show users only what they think they want to see, and hide everything else.

</comment><comment><USER3> Meanwhile on Google+ <https://plus.google.com/+ChrisHadfield/posts/U7UXaTGxVsz>

</comment><comment><USER4> Not sure why this particular activity is noteworthy. The 'dirty world' described here is much, much larger. 'Affiliate Marketing' is a very broad activity that involves doing the least possible work to generate a lead or sale for an affiliate for which you will get paid, and this is just an example of that.

</comment><comment><USER5> We're assuming here that EdgeRank takes outliers into account in a positive way. It's entirely plausible that they work with medians for example.</comment>


Oregon Man Fined $500 for Study Showing Yellow Stoplights Are Too Short - <AUTHOR> https://heatst.com/life/man-fined-500-for-criticizing-traffic-lights-system/ <comment><USER2> I'd recommend a more neutral site than this, HeatSt.com is almost entirely far right op-ed publication.

[https://motherboard.vice.com/en_us/article/man-fined- dollar5...](https://motherboard.vice.com/en_us/article/man-fined- dollar500-for-crime-of-writing-i-am-an-engineer-in-an-email-to-the-government)

[http://www.nbcnews.com/news/us-news/oregon-man-claims- state-...](http://www.nbcnews.com/news/us-news/oregon-man-claims-state- muzzles-red-light-camera-critique-n751371)

</comment><comment><USER3> discussion yesterday: [https://news.ycombinator.com/item?id=14197512](https://news.ycombinator.com/item?id=14197512)</comment>


Show HN: Http://realdjnamegenerator.com - <AUTHOR> http://realdjnamegenerator.com <comment><USER2> So it's just echoing the input values?

<comment><AUTHOR> that is correct. just a goof.</comment></comment>


Boston VC vs. New York VC: It's not even close - <AUTHOR> http://finance.fortune.cnn.com/2010/12/14/boston-vc-vs-new-york-vc-its-not-even-close/

<comment><USER2> "Is there a relevant metric I'm missing?"

Yes: What the numbers look like if you don't include biotech startups.</comment>


Sci-Hub as necessary, effective civil disobedience - <AUTHOR> http://bjoern.brembs.net/2016/02/sci-hub-as-necessary-effective-civil-disobedience/ <comment><USER3> By and large, the academic publishing industry operates in such a way that it ignores the internet ever happened. The internet says, "Hey, you don't have 'Kant's Third Critique of Judgement in your inner-city school library. Here, let me tell you what it says" the academic publishing industry says "whoa- you're not at an institution that pays us money - you don't deserve to know what this says."

They try to make it seem like they do so much to push knowledge forward - <USER2> they take the paper you wrote in MSWord, take credit for the peer-review, then make it into a PDF and ask for your money.

The internet is going to destroy them and they are running scared. How long can you sustain a business that relies on paywalling PDF files?

<comment><USER4> Asking for your money here refers both to a potential reader, and the the author of the article.

<comment><USER5> The latter is a point that's often forgotten, and is very true. I used to do research a long time ago, prior to deciding I'm not quite academia material. If I ever get nostalgic, I literally have to pay between 30 USD and 100 USD just to get PDFs of articles that _I_ wrote.

Of course, this is upsetting only on a personal level, but the entire process is extremely toxic to learning.

<comment><USER6> Disclaimers: I live in another country, and I research in digital forensics -- that has quite an open community.

In my situation, I've never reached anywhere near that point. All my works are available for donwload as PDFs, and even if they weren't, my research group keeps a repository where we can re-download/point/email anyone who asks for them.

We haven't published in IEEE journals (yet), but as of our countrys law, we are the authors, and we hold rights to our work (for example, distribution) as we see fit. As a matter of fact, per a "new" law (it's a few years old, but it's becoming effective now), we _have_ to keep that institutional repository I mentioned earlier.

My question is: how is your situation, and how were you forced into this ridicule thing of having to pay for your own work?!

<comment><USER5> Mine was in electromagnetism. Far less open. I don't live in the US, either, so perhaps the legal situation here would allow me some recourse. I just didn't care enough to investigate, academia is something I've long put behind me :-).

As for your questions:

1\. There are publishers which require you to relinquish all your rights to distribution, copying etc.. In other words, they basically own every right to the final work. This is usually circumvented by just distributing the latest draft for free. IIRC, one of our papers was published in such a journal. We did receive _a_ copy of the journal, which I obviously don't have, it's in our lab's library.

2\. For the others, there's no way for me to say "oh, hi, I'm this paper's author, can I please have a copy?", and in at least one case, we didn't receive a copy of the journal it was published in. Some publishers are nice and give you a pre-print PDF of the article, so that you can actually substantiate your claim that you published it if your institution requires more detailed bookkeeping. But many skip straight to giving you the middle finger and telling you how you can buy the journal if you want it.

I'm hesitant to give publishers' names because this was long enough ago that a) my memory is a little foggy and b) things I say may no longer apply.

So basically, at this point, neither me, nor my former colleagues have physical copies of the journals themselves. I didn't care enough to save the PDFs (nor am I likely to care soon -- that's why I only find it funny in a moderately annoying way), and my former colleagues aren't allowed to send me the final articles in PDF format, either, at least under the paper equivalent of EULA.

<comment><USER7> Genuinely asking (not snark, and I am not a physicist): what does it mean to conduct research in "electromagnetism" more than a century after Maxwell? Did you study superconductivity? Electrical power beaming? Magnetic properties of weird alloys?

I ask only because I thought people who study these things are more likely to say they research "electrical engineering" (if they want to be general and applied), or "condensed matter physics" or maybe "materials science".

I'm really just curious about the relevant terminology / sociology of academia.

<comment><USER5> Uh :-). I said electromagnetism because it sounds waaay less pretentious than what it was generally termed -- computational electromagnetism -- and because claiming it was microelectronics, while technically correct, was somewhat frowned upon at our university because we were in the wrong department for that. Politics and whatnot.

We were doing research on simulation and model extraction techniques for passive structures in very high-frequency (think 30 GHz) integrated circuits. At these frequencies, the lumped-element approximations break down, so you can't describe the functioning of passive devices in the familiar manner, based on Kirchhof's equations et co.. That means SPICE-family tools can't give you proper results anymore, and you kind of need them to design the circuit.

The "proper" way to do it is full-wave simulation; unfortunately -- especially for very wide-band devices -- that's computationally very costly (simulating the behaviour of the device at a specific frequency isn't very cheap on its own, but we're talking about simulating it for a whole range of frequencies, which could span tens of GHz). You could plug those results into SPICE, of course, but running these simulations took hours, sometimes even days, for a single run, which is completely impractical. It's also not the level you want to work at when designing circuits -- where you're solving "circuit problems", not "field problems".

So what we were trying to do was:

a) Figure out how to make the simulations faster and more accurate, and

b) Figure out how to extract a model from them, i.e. an equivalent circuit, which could be readily used by the design engineers.

We did that through a combination of things like running as few simulations as possible and extrapolating, running simulations in parallel and so on. It wasn't some major scientific breakthrough, but then again, I worked there during my 3rd and 4th year of university; it was the most difficult kind of work I'd done until that point.

<comment><USER6> Sounds very interesting, thanks for sharing :)

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> Civil disobedience is an important form of democracy. Maybe in the future, when all of our calls, messages and identities are monitored in full by our benevolent overlords, we won't be able to practice it anymore. Right or wrong, faraway anti-Western countries have served as counterweights against the West for Snowden, Assange and Elbakyan.

<comment><USER9> > Maybe [...] we won't be able to practice it anymore.

Do you mean by suppressing the action before it occurs? I thought that one of the points of civil disobedience was to get caught, so as to increase visibility of the problem and become a catalyst for change.

<comment><USER10> The basic problem with the "original formulation" of civil disobedience is that power structures have evolved to thwart it.

Things that should by any right be a civil case or misdemeanor are instead charged as multiple overlapping felonies in order to coerce a plea bargain, which is effective to deter all but the most dedicated from demanding their day in court. Then the court won't allow you to make the case for jury nullification or any other argument that the law is wrong to the jury, and prosecutors will be allowed to exclude anyone from the jury who is sympathetic to the idea that juries shouldn't convict when they don't agree with the law.

You can only make the case that the law is wrong to the judge, and no matter how stupid the law is the judge can only agree with you if it violates the constitution. Which plenty of very, very stupid laws clearly don't, so you become a convicted felon without having had an opportunity to make your case to anyone with the power to do anything about it.

In theory the outrageous nature of that process should inspire the public to demand change, but have we passed Aaron's Law yet? [1]

[1] [https://www.eff.org/deeplinks/2015/04/aarons-law- reintroduce...](https://www.eff.org/deeplinks/2015/04/aarons-law-reintroduced- cfaa-didnt-fix-itself)

<comment><USER11> The natural response to that is civil disobedience without actually getting caught. As a first approximation a single person operating alone could probably do around a billion dollars worth of damage without directly inflicting physical harm on anyone. Get a few thousand people to do the same and individually there going to be less effective, but things break down. Keep pushing and government becomes meaningless.

However, people are generally fat and happy. A surveillance state in wall-e's world just does not mean the same thing without the downtrodden masses which have fueled past revolutions.

</comment></comment></comment></comment><comment><USER12> I've said in a previous comment, sci-hub is pure civil disobedience of the noble kind. In fact i thought it would be impossible to ever sue her, because there is no moral argument that can outweigh what she offers. I use sci-hub all the time because my institution doesn't have access to all the sci journals, and i want access to ALL the knowledge for which millions of taxpayers have paid for (but they don't know where that knowledge ends up). Its the perfect complement to open access journals, for the (i believe) transitory period until we do away with impact factors and citation factories for good, like we did away with alchemy and phrenology.

I hope this increased publicity won't lead to shutting down the site or freeze its paypal account.

<comment><USER13> According to MLK, civil disobedience is only just when you accept the penalties prescribed by law.

<comment><USER14> Admittedly MLK probably didn't foresee that people could go to prison for downloading information from the internet.

</comment></comment></comment><comment><USER15> What an amazing woman. She tries to downplay her accomplishment by mentioning in the comments that she was only responsible for grabbing over 20 million papers. Nice try, but that's still a remarkable job!

Edit: not sure if the commenter is her, but she's done amazing work regardless.

</comment><comment><USER16> Although everyone seems to celebrate it, I feel a little uneasy about this.

The whole publishing scam is _not_ about giving (or restricting) access from research to peons, it is to justify sucking the money of public institutions.

When the situation sucks for people, they get motivated to try to change the system. It is already hard enough for people to try to change things, because research is hard enough, and then if you start to refuse publishing in unethical journals the next idiot is going to do it as well as you would.

Comparatively, many people are for the status quo regarding copyright in general because they know very well that piracy is an option. At a RMS talk, the (university!) crowd mostly unanimously cheered when he said "yes piracy" and booed when he said "... thus we have to change copyright". He was then asked "but what if hollywood disappeared with the change?".

<comment><USER17> > He was then asked "but what if hollywood disappeared with the change?".

What if they did?

It's the lie of all middle-men that they are necessary to the process. Composers didn't stop writing music when the concept of a noble benefactor died: the business model changed. Architects didn't stop building beautiful buildings because religions couldn't afford giant churches and kings stopped building palaces. Movies will continue to be made if Hollywood dies, and if history is any indicator, the art form will develop more rapidly once it's no longer hampered by outdated business models.

</comment></comment><comment><USER18> Does Sci-Hub practice what it preaches? Do they provide a means of downloading their entire repository of research papers either via torrent or SFTP? Because if the entire repository isn't available to everyone, then how can we be sure that once they get raided those papers aren't lost forever?

<comment><USER19> SCi-Hub used hacked or shared university account to abuse the university proxies to get content. For big research institutions, it's an annoyance. To small colleges and university library it is a major headache that can get the whole university locked out from resources the library is paying dearly for.

Source: My partner is a librarian at a smaller university, this happens every few months and they lose access to major online platforms for days if not longer every time.

<comment><USER20> I feel like you're blaming the wrong party here.

<comment><USER21> The "blaming" is in your head. People who explain how things happen do not "blame".

<comment><USER22> Don't talk bullshit.

There clearly is blame put on the proxies

> it [the proxy] is a major headache that can get the whole university locked > out

Whether either the practice of lock out should be blamed for immoral licensing or the proxy servers blamed for immoral illicit access, is not properly explained here.

<comment><USER21> Don't talk bullshit.

Just because you think in categories of 'blame' doesn't mean everyone else does.

For some people there is a difference between causality and guilt.

</comment></comment></comment></comment></comment></comment><comment><USER23> One aspect of Aaron Swartz's case that I found both interesting and way under- reported, was that he supposedly was going to perform meta-analysis against the collected documents. And that a previous meta-analysis of a similar sort had revealed some surprising and useful results.

With all the pieces locked up, it is difficult if not impossible to do such science and statistical analysis. In other words, it is antithetical to the nature of science. I didn't see/read enough to confirm this, in Schwartz's case. Nonetheless, I think the idea and the opinion are valid.

<comment><USER24> Tangentially related: is anyone here familiar with automated analysis of research papers? I feel this is a field that would perfectly lend itself to machine learning, or similar approaches.

It would require extraction of meaningful data, be therein lies the rub. There are several large projects looking into collecting/sharing scientific data, mainly focusing on persuading and empowering researchers to share their full data sets.

But what can we do where the raw data is not available? Is anyone working on ways to reliably extract data from the millions of PDFs in research databases?

There are several open-ended questions here. If anyone knows of work in these areas, or is interested in this, I'd be very keen to talk.

<comment><USER25> Only materials related, but at [http://www.citrine.io/](http://www.citrine.io/) they do something like that. I recently attended a talk given by one of the team members (Bryce) and they seemed to be quite open to discussions.

</comment></comment></comment><comment><USER26> I'm surprised there aren't more comments here. Maybe it's because most agree, and there really isn't much more to be said?

Anyways, thank you very much for sharing.

<comment><USER27> Authors/Scientist have little problem with getting their work into out there to as many people as possible. People want the knowledge. So the only ones really objecting to this are the publishers.

As someone who works at the edge of biology, I have noted that the publishing business reviews the papers get knowledgeable peer-reviewers(pressed for time) and then ask difficult questions of the authors. Some papers aren't good enough, they get rejected. This culling has some value, much as it irritates the scientists. Scientists are always trying to get their work published/sited and ranking journals is something that happens with some regularity [http://www.citefactor.org/journal-impact-factor- list-2014_0-...](http://www.citefactor.org/journal-impact-factor- list-2014_0-A.html)

Of course, want to download an article on the first genome assembly of drosophila from 15 years ago... $30. (I can access articles like this though my work in academia, if I go though the library and cross my fingers the moon is in the right phase, the the pdf link might show up).

Holding knowledge and parsing it out for profits is somewhat suspect.

Note that anything US government NIH funded since 2005 is required to have access. [http://www.ncbi.nlm.nih.gov/pmc/about/public- access/](http://www.ncbi.nlm.nih.gov/pmc/about/public-access/)

for example. This paper, published by a coworker in an official "journal" but still available for free via pubmed. [http://www.ncbi.nlm.nih.gov/pubmed/23443684](http://www.ncbi.nlm.nih.gov/pubmed/23443684)

I get the "joy" of uploading lots of data from screens to the us goverment pubchem website. I'm not sure anyone downloads it, but its there. They have an interesting embargo system which holds the data till publication (I've never used that, some people really care about that stuff)

Researchers in academia also pass around papers. The peer reviewers do to, if they find something interesting. We've gotten requests for data from unpublished papers.

<comment><USER26> Great to hear.

I've found another way to get access is to simply email the author of a paper. Of course, this mostly only works for those still enrolled at the academic institution where the research took place, but I've had good luck with it.

</comment></comment></comment><comment><USER28> I am poor, which I use as my primary excuse to pirate software, movies, books, etc.

To those who are not financially struggling: Why do you pirate? What is your moral justification?

<comment><USER29> If you aren't affiliated with a university, scientific papers cost something like $30/each to read.

If papers can be accessed for free, then the only expense required to follow cutting edge science is time, and everyone with an internet connection can join in. At $30/paper, suddenly only the rich (or those with rich patrons, such as tenured professors at rich universities) can afford to play.

Researching a topic, it's easy to skim 100 papers in a couple days, trying to figure out which ones are relevant and which ones aren't. Paying $3000 just to sort through and find the 5 out of 100 papers which bear closer scrutiny is very expensive, even for someone who isn't "financially struggling". (No, the abstracts alone aren't sufficient for this.)

And it's not clear that the money is going anywhere useful. The scientists who wrote the papers never make a profit: all the money goes to a couple of rich corporations, who will use the money to further impede scientific progress.

<comment><USER30> arXiv? I mean, you're looking at preprints, and you might have a higher dross- to-gold ratio, but can't you do your research there, without the moral dilemma?

<comment><USER31> arXiv is great, but represents a very small fraction of the total published work, is extremely field specific, and old papers are still very relevant. I find myself regularly relying upon citations from as early as the 1910's.

It's not even close to an alternative.

<comment><USER32> For my PhD thesis I used some papers from the 1800s along with many from the 1900s to 1950s because those were when the foundation papers were published in one area I was working in.

</comment></comment></comment></comment></comment><comment><USER33> This whole story and the narrative of "civil activist Alexandra Elbakyan" sickens me to no end.

In reality, Alexandra is an avid supporter of Putin's regime and a borderline fascist (in the original sense of cooperative state). Last year, when Dynasty non-profit fund (which sponsored thousands of Russian researchers, mostly in natural sciences) got expelled from Russia because of political reasons, she welcomed it because "they were against Putin". Here is my translation of her words [1][2]:

> [Dynasty] fund is one of the main supporters of "Liberal mission" fund. It > supports, develops and advocates "liberal" ideas. In Russian realities it > means support of Maidan [Ukrainian revolution], Bolotnaya [location of major > protests in Russia in 2012], propaganda against Putin and Russian > government.

She hates liberalism and the West in general. To add to the picture, here is what she really thinks about Sci-Hub (context: someone questioned her moral authority) [3][4]:

> We will fight you not only with bans [5], I warn you [6]. … I understand why > you are so against Sci-Hub - it's one of the projects that Russia can be > proud of and this doesn't fit into your narrative of "everything is bad in > Russia". And given that I support the government - it turns into a nightmare > for people like you.

So yeah, when I see comparisons like

>Actions of civil disobedience like those of Aaron Swartz and Alexandra Elbakyan

it's very hard not to throw up. No, it isn't "civil disobedience"; she is a complete, utter opposite to civil activists like Aaron Swartz. Sci-Hub is born out of indifference to intellectual property, total invulnerability to lawsuits (Alexandra lives in Russia and Kazakhstan, which aren't aren't very fond of cooperating with US institutions) and poverty of Russian scientists. Yet she manages to see it as a "thing Russia can be proud of".

It is very sad that media picks up the accustomed narrative of activist underdog and completely misses the context of her true views. Doing this, journalists both distort reality and miss a very interesting and provocative story of a statist (and probably somewhat mad) woman which found herself in a right position in a right time to get Western media attention.

[1]: [http://vk.com/wall-36928352_2614](http://vk.com/wall-36928352_2614)

[2]: «Так, например, фонд Зимина являлся одним из основных спонсоров фонда «Либеральная миссия». Этот фонд поддерживает, развивает и пропагандирует «либеральную» идею. В Российских реалиях это означает поддержку Майдана, Болотной, пропаганду против Путина и Российской власти.»

[3]: [https://vk.com/wall1396880_8792?reply=8796](https://vk.com/wall1396880_8792?reply=8796)

[4]: «Если потребуется — и не только с помощью банов с вами воевать будем, имейте это ввиду. … Я понимаю, почему вы так против Sci-Hub — ведь это один из проектов, которым может гордиться Россия, и это очень не вписывается в вашу пропаганду о том, как всё в России плохо. А на фоне того, что создатель проекта поддерживает государство — это вообще оборачивается кошмаром для таких, как вы.»

[5]: She bans everyone who even remotely don't support her views in VKontakte Sci-Hub group.

[6]: Yep, it seems like a physical threat in the context. Sorry if I wasn't able to convey it in the translation.

EDIT: formatting, typos

<comment><USER34> So maybe this shows that people don't fit neatly into categories like "good" and "bad", they can do both, and shades of grey in-between.

<comment><USER33> Or maybe this shows that bad people sometimes do good things out of serendipity; sometimes they even get credit and manage to hide their evilness. It doesn't make them in any way similar to Aaron Swartz, though.

<comment><USER34> "evil" people are pretty rare, most people believe they're doing the good thing. Sometimes they're correct.

</comment></comment></comment></comment><comment><USER35> Can anyone explain why it is not possible to publish your work as a PDF or to a free/open platform at the same time you submit it to the journal? It this somehow prevented legally in the terms of the journals?

<comment><USER36> I'd also be interested in this answer. What I have done is email researchers directly asking for a PDF and they have been happy to give me one. Often some of the authors have their own site where they add links to their own papers.

Now, is this because these authors are exercising their own right to distribute their work, and many other authors don't want to expend that effort? The copyrights don't change after all, so collating these and redistributing could also be the same as sci hub.

<comment><USER37> It all depends on the terms of the copyright transfer, which is based on the historic relationship between author and publisher.

It used to be that when you published a paper and transferred copyright, you received a number of copies of the publication for yourself. You could send these out to others who wanted a copy. This was especially important before xerography machines were common, as it was difficult to make your own copies.

Even in the 1990s, an academic researcher might get a postcard asking for a reprint to be sent in the mail.

So research publishing has long had a tradition where the author has a limited right to redistribute copies directly. (To be certain, the publisher made the reprints so this wasn't part of copyright but rather a codified expectation of the author/publisher relationship.)

This tradition was carried over into electronic publication Why? Consider the 1990s and early 2000s when the transition to electronic journals took place, and imagine the authors up in arms for having that distribution ability taken away.

This is why you can get a copy from the author. It's sometimes also formally specified that author may also distribute a copy on a personal or research web page. Sometimes this permission is for the author's final draft version, and not the proof copy used for the publication.

But there wasn't a tradition of sending the paper to a tertiary reprint service.

</comment></comment></comment><comment><USER38> Does anybody know how to find the corresponding torrents for the papers that sci-hub has fetched? Is there some sort of prefix I can punch into torrentz.eu?

<comment><USER12> i think scihub uses various proxies in multiple universities to fetch the articles.

<comment><USER39> I think the question was whether we can easily mass-download the papers from Scihub.

</comment></comment></comment><comment><USER40> Scientists do research paid for by institutions, universities, large charities and other scientific institutions. They write research papers and they have it published and those same institutions pay to get that information back. I can't think of any other area where such a crappy business model prevails.

The scientific community should reclaim their intellectual capital.

</comment><comment><USER41> as i understand it the argument for pay walling is the costs of peer review?

so how about a service where you publish the paper, but pay for the peer review that grants your paper an accreditation of significance or legitimacy

this current system of peer review to enter our journal to determine legitimacy is prone to too many errors: corporate buy outs, poor standards, peer review is only done once prepublication; and needs a revisit

i'd love an arxiv`like to have for hire peer reviewers with associated reputation, question the significance of a paper that is peer reviewed? Hire your favourite reviewer to take a second look

that said, i think all papers should be open for reading regardless of peer review, the indicator of research should be on review stead access

<comment><USER42> Peer review is done for free by the reviewers.

<comment><USER41> my understanding is: unpaid work is the model but the current appointment of that model makes liberal use of the term honorarium(o)

(o) [https://www.google.com/search?q=honorarium](https://www.google.com/search?q=honorarium)

</comment></comment></comment><comment><USER43> A simple fair circumvent: Any freely published article should not be allowed to be cited by a non-free one.

This puts the power into the hands of the author, who could decide if they want this as part of the open license. Rather than the current robinhood type system.

Does this already exist, anyone seen anything similar?

(edited, typo)

<comment><USER44> This is as absurd as prohibiting hyperlinks because the site being pointed to might have copyright violations

Not to mention completely non-enforceable. You're forbidding people to say "David et al. [3] did a work in this area"

<comment><USER43> Ok, that makes sense. But is it an absurd proposition to say that people who release non-open articles should not be allowed to directly build on the work of those who did it for free?

So how about if part of the agreement of accessing an open article was that any published work citing it would have to be open too. is that absurd in principle? and is it really non-enforceable?

An offshoot of this would be that anyone who goes on to cite something in a non-open article, could academically be accused of not reading the article their citing, and therefor mis-attributing information. This could lessen there creditability or even be grounds for a specific type of plagiarism.</comment></comment></comment>


A lesson for Nokia: what the JRuby team did when Sun went down. - <AUTHOR> http://arstechnica.com/open-source/news/2009/07/jruby-team-leaves-sun-joins-engine-yard.ars

<comment><AUTHOR> Since Maemo is OSS, there's nothing stopping them from resuming work at another company as an entire team.

Who wants to buy Maemo?</comment>


Dear Apple: Fix the App Store with a Measure of Credibility - <AUTHOR> http://www.andrewljohnson.com/article/Fixing%20the%20App%20Store%20with%20Measure%20of%20Credibility

<comment><USER2> I'm reminded of programming, and how code comments don't always describe what the code does. Trying something really is often the _only_ way to know if it does what you want.

So perhaps Apple should have a "try" feature: to download something you have to "pay" for it, but you have a short grace period (say, 10 minutes) in which to decide to completely "scrap" the application. If you scrap it, you aren't charged for it, and you can never use it again. This allows people to never be fooled into paying for something they discover they don't like.

<comment><USER3> That would solve a lot of problems. Now developers typically make a free version of their app, but Apple restricts how those can be used. They can't have grayed out buttons representing features available in the pay app, and can't be time limited. It can be hard to feature limit it and still have the free app be neat.

Our app has an offline data store of maps on the phone, so it's additionally painful that when people 'upgrade' to the pay app, they have to redownload all their data.</comment></comment>


Ask HN: What separates a Moonshot from conjecture? - <AUTHOR> I'm applying to startup school and was wondering what to characterize my application as. Based on research I did in grad school in AI and Public Health, I'm building an early warning system for various diseases (malaria, dengue fever) and public health problems. Some large orgs have shown interest, but the technology itself needs more development.
Would this be considered a moonshot by YC? Or an AI category product?<p>Also, when things like Rigetti computing presented themselves to YC, how do they assess whether it's even remotely possible? <comment><USER2> Late reply, but I'd say this sounds more like moonshot. Also, sounds awesome and hope this works.</comment>


ASK PG/HN: Do you have a video of the talk that Paul Buchheit gave at SS09? - <AUTHOR>

I have been searching for it online but the usual suspects only have short feeds for them. I missed the initial talk and hoped that there would be an archived feed. That was one of the main talks that I was really interested in. <comment><USER2> +1

</comment><comment><USER3> +1</comment>


Command Line, Console and UI Based Open Source Plain Text Accounting - <AUTHOR> http://hledger.org <comment><USER2> Related to this, I wrote a CLI tool called Ledger Reconciler[0] to automate the process of logging into all my financial accounts and inputting those transactions into Ledger. It uses headless Chrome underneath (via the puppeteer API[1]) so it could theoretically work with any system without needing to rely on public/private banking APIs.

I also tried to make the plugin system as friendly as I could so people for people to add their own financial instutions. I'm optimistic someone will contribute a plugin one day :)

[0]: [https://disjoint.ca/projects/ledger- reconciler](https://disjoint.ca/projects/ledger-reconciler)

[1]: [https://github.com/GoogleChrome/puppeteer](https://github.com/GoogleChrome/puppeteer)

<comment><USER3> How do you handle authenticating to the financial institutions?

<comment><USER4> I think we all know the answer to that question :)

I would note that professional bank feed apis (e.g. yoodle) also use username password pairs and not only break when you change password but can result in a locked bank account due to too many retries.

That said, there is no universal solution in the US.

<comment><USER5> > I think we all know the answer to that question :)

May I ask-- for those not in the know --that what is it? :) From your comment I'd infer that username/password, but that hasn't been the case for many years, at least at the banks I use (in Europe).

<comment><USER6> If he's using headless Chrome then it's likely form filling. I.e. just automatically typing in the answers to the authentication prompts on the pages as you'd normally do when you log in. Most banks, even those with TOTP or alternate MFA methods, allow you to login with standard authentication details to get access to statements and not make any changes.

</comment></comment></comment></comment></comment><comment><USER7> Does anyone have any advice on how to be successful with ledger? I've tried a few things. I entered a lot of things, but the issues I have are more structural nature and probably general lack of accounting experience:

What types do you use is there a useful list, that I can then translate to SMB or Personal tax filing? A somewhat standardized structure of assets, liabilities, accounts payable, accounts receivable etc.

How do you deal with Cash? (EDIT: most answers assume that you only spend little cash, but in some countries cash can account for a decent amount of your income)

How do you deal with interest?

How do you keep your bank statements? I've tried using [https://github.com/johannesgerer/buchhaltung](https://github.com/johannesgerer/buchhaltung) which supports paypal ofx and hbci

How do you convert your balance sheet into a tax filing compatible format?

How do you track things like rent? Do you keep it as a recurring entry i.e. `~ Monthly` or do you only book it when you pay with buchhaltung?

Do you keep completely separate ledger files if you're sole proprietor or can you keep them semi separate and work with includes to create separate reports for private and sole proprietor tax filing?

How do you track small expenses on the go? There's a web app called fava from beancount which looks nice, but i'm not sure you want to expose it to the outside world and i'm not sure the files are fully compatible with hledger [https://beancount.github.io/fava/](https://beancount.github.io/fava/)

Any help would be greatly appreciated.

<comment><USER8> > How do you deal with Cash?

Move money from Asset:Checking to Expenses:Cash

> How do you deal with interest?

Move money from Income:Interest to Asset:Depot

> How do you keep your bank statements?

I download CSVs once in a while an import them with hledgers import tool (see [https://bloerg.net/2016/02/22/ledger-import-of-deutsche- bank...](https://bloerg.net/2016/02/22/ledger-import-of-deutsche-bank- data.html))

> How do you convert your balance sheet into a tax filing compatible format?

I don't, sorry. But it's actually pretty easy to get the relevant data if you split your taxable incoming accordingly.

> How do you track things like rent? Do you keep it as a recurring entry i.e. > `~ Monthly` or do you only book it when you pay with buchhaltung?

It's part of the bank statement, so yes it moves money from Assets:Checking to Expenses:Rent whenever the transaction happens.

> How do you track small expenses on the go?

I don't. After a while I found it's not worth the effort. I rarely make financial decisions based on how much I spent on beers and sandwiches, so most of these little things are just subsumed in Expenses:Cash.

<comment><USER9> > > How do you deal with Cash?

> Move money from Asset:Checking to Expenses:Cash

I've been moving from Assets:Checking to Assets:Cash, and then from Assets:Cash to Expenses:Haircuts (or whatever)

<comment><USER10> You can also move from Assets:Checking to Expenses:Cash, and then from Expenses:Cash to Expenses:Haircuts. This allows you to track the larger cash expenses if you like, while letting the small purchases accrue in Expenses:Cash.

<comment><USER9> I just found that workflow described in a blog post elsewhere too:

"Pro-tip: tracking cash withdrawals and purchases can be a pain. I typically just maintain an Expenses:Cash account for these sorts of things. If I can remember how I spent a bit of cash, great: I can just debit Expenses:Cash and credit the appropriate account for what I spent the cash on after the fact. Otherwise, I just leave the amount attributed to Expenses:Cash. It's often not worth the trouble of trying to keep finer track of cash purchases than that." \-- [http://matthewturland.com/2014/03/29/ledger-basics-and- habit...](http://matthewturland.com/2014/03/29/ledger-basics-and-habits/)

Sounds smart, though at the scale of my personal finances, I get much more value out of tracking the petty transactions. I guess I'm pound-wise but penny-foolish.

</comment></comment></comment></comment></comment><comment><USER11> I use GnuCash [1] with non-compressed save files enabled so that I can store changes on Git. It works flawlessly for me: I can edit accounts across different computers, get the full force of a mature accounting package to do my finances, and can use/write scripts that plot some statistics using various bindings (e.g. Python) [2].

[1] [https://gnucash.org/](https://gnucash.org/)

[2] [https://github.com/SeanDS/gnucashxml](https://github.com/SeanDS/gnucashxml)

<comment><USER12> Have you found a sane way to handle massive initial imports? As far as I can tell, there's no way to bulk-categorize transactions. That means I'd have to go through each of my 6k+ transactions one-by-one for categorization.

<comment><USER11> Not sure - I moved to GnuCash with only a few hundred transactions to import, so I did it manually. It may be possible to use the search tool to find the transactions you want to give a particular category to, and then categorise all of those (I'd be surprised if there wasn't a way to bulk categorise a list of transactions).

</comment></comment></comment><comment><USER13> This is very impressive! I have a SO pursuing accounting, and there are so many subpar proprietary systems in the accounting world it's crazy. I have used gnucash, but there are many issues with gnucash that text only systems like ledger address, but the learning curve for them is daunting to your average user. So I could definitely see great opportunity for a text based ledger system and all its benefits along with a GUI overlay.

All that, with a gplv3 license, this is the kind of project I can get behind!

</comment><comment><USER14> I've been trying to replace the last dependency I have on Google products, a "finacial forecast" Spreadsheet integrated with Calendar. It lets me see approximately how much cash I'll have on hand at future dates, basically letting me live paycheck to paycheck without too much stress :) Can this do something like that? If not, does anyone know of something with similar features (local program, simple data files, etc.) that can?

<comment><USER15> I don't see why not, if you're willing to get deep into the ledger file format. Look at "Forecast" and "Budgeting."

[http://plaintextaccounting.org/](http://plaintextaccounting.org/)

Integrated right into your calendar, I don't know about that part.

<comment><USER16> > Integrated right into your calendar, I don't know about that part.

Depends on the calendar -- surely it could be integrated with cal(1).

<comment><USER14> integrating with cal would be perfect. I'm going to look into this as soon as I can.

</comment></comment></comment></comment><comment><USER17> At CurrySoftware GmbH we use ledger + recutils + aqbanking + custom bash scripts to automate many business processes.

Some of these processes are required by german law and many companys outsource these to accountants or tax advisors or buy shitty software from DATEV.

The great advantage in staying fully plaintext is, that you do not need to automate everything for the whole system to work. Incoming invoices are frequent -> write a bash script, Outgoing invoices are frequent -> write a bash script. Paying the state back some VAT happens once a month, use emacs to do it manually. I really love it.

<comment><USER18> It would be interesting if you could share your workflow/scripts.

Especially how did you implement SKR03 (guess you use that) into your hledger account hierarchy, and how do you generate e.g monthly VAT reports, or yearly VAT and trade tax reports.

<comment><USER17> The system is not yet ready to be published but drop me a mail and I will send you some bash scripts and will try to answer questions.

We use a custom "Kontenrahmen" based on the same ideas as SKR03, as using numbers is not necessary with ledger. That also means we can only communicate with tax advisors through paper. But as we communicate with the Finanzamt directly, thats ok.

</comment></comment></comment><comment><USER19> [https://github.com/narendraj9/hledger- mode](https://github.com/narendraj9/hledger-mode)

I have been using this for almost two years now. I am not sure if there are any other users but I find it very convenient. (Disclaimer: I wrote the extension).

<comment><AUTHOR> Thats awesome, to use it with emacs

</comment></comment><comment><USER20> I've been using beancount for the last 3 years. The fava project provides an awesome web UI for beancount and I'm really happy with that.

<comment><USER21> I've been using ledger for years. I really wanted to give beancount a try, but keeping documentation in google docs aggravates me. I understand that it might be easier for the writers to write and keep up to date. But it's an absolute pain to read/browse/skim/search, which is the whole point of documentation.

</comment></comment><comment><USER22> I successfully switched to ledger a few years ago after a few failed attempts years earlier. I attribute success to: [https://github.com/quentinsf/icsv2ledger](https://github.com/quentinsf/icsv2ledger)

It is the transaction import script that has really worked well for me without quirks. A lot of people recommend "reckon"\--which I've tried--but I found that while Bayesian scoring sounds good, a solid set of regex work better. icsv2ledger does a great job of allowing you to build up a pool of search terms to map inputs to categories. And since the regex set is an artifact that you can tweak and add to, you end up with a very stable and comprehensive import process.

I also like that it will (optionally) add the raw CSV that was imported as a comment in the transaction, and use that to avoid reimporting transactions. While this is a little visually noisy in the ledger file, I'm blind to it by now and it makes it rock solid to reimport CSV files without fear of duplicates. This also provides a neat way of dealing with each side of a transaction coming in separately. For example, both my checking and credit card list the transaction that corresponds to me paying my bill, but that is only one transaction in the ledger file. After I've done the import of everything, I simply delete one of the new entries but first copy its CSV comment into the remaining remaining transaction. So one ledger transaction lists both CSV comments, and subsequent imports from either account are duplicate free.

</comment><comment><USER23> This looks like a very well done project. If you are more a traditional software kind of person, I recommend heavily KMyMoney[0]. I have my full family accounting since 2007 in it (every single income/expense) and it works flawlessly allowing me with the great reports to know exactly where our family money is spent and saved.

[0]: [https://kmymoney.org/](https://kmymoney.org/)

</comment><comment><USER24> I've tried using ledger in the past. I ended up moving all the data to a database. Ultimately it is much easier/flexible to work with SQL, than arbitrary command line tools. Also it's easier to manipulate and process the data from anywhere (psql, cli app, web app, ...), without fear of corruption.

<comment><USER8> How do you model double-entry book-keeping in your relational database? And where comes the fear of corruption if .ledger files are merely text?

<comment><USER24> I can't share everything here, but basics are this. Three tables: accounts, transactions, entries. Entries have amount, unit, notes. Accounts have name, parent_account. Transactions have subject, notes, date. That's the data. Then I have a view that flattens accounts tree, so that each has name and fullname. Name being Fun, fullname a full path to that account, eg. Expenses:Fun.

With that base structure, you can start answering any questions by just summing entries groupped by accounts filtered by a regexp on fullname and fitler on date. What is my cashflow for any time period, what is my equity, what is my balance sheet, what I actually have, what changed recently, how any of this develops over time...

</comment></comment></comment><comment><USER25> Are there any plain text accounting systems that provide an android app as well? I'm using gnucash, mostly because it has a nice companion app.</comment>


Becoming a Real Startup – On Growing and Turning an Idea into Something Tangible - <AUTHOR> https://blog.trytomo.com/becoming-a-real-startup-on-growing-turning-an-idea-into-something-tangible-fast-55e18120e6ff <comment><AUTHOR> Let me know if you have any questions!</comment>


Things Humanity Could Do Right Now to Save Planet Earth from Climate Change - <AUTHOR> https://medium.com/@datarade/things-humanity-could-do-right-now-to-save-planet-earth-from-climate-change-2742ccebca14#.wui3igsb7 <comment><USER2> First point is kinda weak (Overfund Planned Parenthood Organizations) in terms of limiting population. This is probably the most important factor in saving our planet. How about limiting everyone to only one child per family, and not how China did it. I'll go one step further and recommend we have testing for potential parents to see if they are fit for having children. I mean we need a license to drive a car but anyone can have a child? Our society needs more in place to ensure kids have the best of everything.</comment>


HN:Help Pivot Our Startup -- Eat.ly - <AUTHOR> http://www.leveragingideas.com/2010/09/14/pivot-our-startup-eat-ly/

<comment><USER2> _The idea was born based on the observation that most people can't remember what they had for lunch two days ago. However, stopping to take a photo (and optionally sharing it publicly) adds both accountability and an element of fun._

A bunch of my friends will occasionally post pictures of their meals on facebook -- either because they're eating out somewhere particularly awesome or because they've cooked something themselves which they want everybody to see. I can't imagine anyone wanting to photograph _everything_ they eat just for the sake of keeping track of it, though.

Definitely need an iPhone/Android/whatever app. Email is too much effort.

Also, a facebook app. Once food photos start showing up in my newsfeed with "Posted from eat.ly" under them, I might start checking it out.

I'd de-emphasise the "health" and "calorie counting" side of things and play up the "food porn" side of things. I don't want everybody to know about the boring healthy salad I had for lunch today, but I _do_ for some reason want everybody to know about the delicious foie gras risotto I had at l'Atelier de Joel Robuchon the other night (as proven by the fact that I just found a way to brag about it in this very comment. nom nom nom)

Maybe provide a way to rate things by delicious-looking-ness? (You might want to think of a better word than delicious-looking-ness.) People could compete to have the most delicious-looking food on the entire site (like hotornot but without _quite_ as much potential for ego shattering).

<comment><AUTHOR> Agree with the iPhone app. We do have a basic Facebook integration that allows users to post to their feed. Also agree with deemphasizing the calorie/health aspects, though overtime, I still believe the stronger value prop for an app like this is to become the mint.com of eating rather than another food porn site. But, yes, in the near term more emphasis on social, food porn, etc is prob the way to go

<comment><USER2> There's already a _lot_ of calorie-counting apps out there if that's what somebody is interested in. dailyburn.com, for instance, is pretty good. It's also a _lot_ of work to replicate one of these since you'll need to enter a huge database of possible foods.

The problem with them is that counting calories is too damn much work, so adding an _extra_ step, where you take a photo of your food and _then_ count the calories in it, doesn't seem like a huge gain. On the other hand, I suppose if you had photos of everything you'd eaten you could more easily count the calories only once every couple of days.

Still, I think you'll need to decide between "food porn" and "healthy eating" rather than try both, since they're pretty much opposed.

</comment></comment></comment><comment><USER3> First off, have you measured the time it takes to load your site? It seems awfully slow to me (maybe it's the HN posting that's slowing it down) and could be a reason people are not spending more time on the site.

Before you change your vision, have you considered reducing the friction to use the site? Having to email photos and put in the name of the restaurant is cumbersome compared to opening a phone app and taking a picture that automatically gets posted to the site with your location from the phone's gps.

What sets this site apart from the other food photo based sites?

<comment><AUTHOR> We're definitely aware that site is uuuber slow. Because we're 100% bootstrapped (and not generating revenue) we recently scaled down from Engine Yard to a small instance on Slicehost. We figured we'd cut our costs while deciding what to do. EY was about $240/mth but 10x faster

</comment></comment><comment><USER4> Allow me to offer you an idea:

When I go to Yelp to search for a restaurant, the filters I have available are type of restaurant, proximity, rating and price. My next step is to either sort by those filters, or individually click a restaurant. Looking at an individual restaurant, my options for consideration are photos of the restaurant, consisting of either interior/exterior or images of the food, and user reviews. The user reviews consist of reviews about the service, atmosphere and food, but I have to chug through each review to get this type of specific information. And with a task of choosing a restaurant and having to repeat these sets of actions is immensely tedious.

What's missing?

A way to discover restaurants near me that I can discover by their actual dishes. The user flow would be this -> I'm in the mood for a burger. Show me restaurants that serve burgers. I get a range of places from diners to more high-end to delis and more, but instead if getting a list of joints, I get a sortable grid of user's burger photos from places near me. Then, all I need to do is say, that burger is the one I want and, barring other misc info such as price or needing reservations and what not, I will go there and get the burger that I saw with my own eyes.

The advantage you will have over Yelp is that While Yelp provides a way to discover restaurants by very general filters, you will provide a way to directly discover restaurants by the actual individual dishes.

As a side note, think about when you go to a restaurant and get a menu that has pictures. It gives so much more reference to choosing a dish, because the mystery is taken out of the equation. Your service would do exactly this. Remove the mystery of what you're going to a restaurant for.

Also, while you can see some food items on Yelp in the individual restaurant pages, #1 they come with no context and #2 I can only access these photos on these individual pages which is tedious.

There you have it.

<comment><USER4> Just want to add that of course, you'd be able to integrate food ratings, friending, favoriting, etc etc etc all that standard social stuff, but IMO, in order for this to be successful, you should consider boiling it down to the absolute simplest of functions and tasks to minimize user friction.

So with the above idea, I would recommend the user need only simply 3 things: 1) photo 2) restaurant 3) whether they liked it or not and an optional 4) description.

If it's an iPhone app then you automatically pull geolocation, but if it's through email, I'm not sure how this would be done and you'd have to explore what you can do.

</comment></comment><comment><USER5> You're focusing too much on what your users aren't doing. Find out what they are doing and retarget the app based on that.

<comment><USER6> Precisely. If you could so easily change your users' behavior to suit your needs you would simply get them to pay you money for sitting at home playing Halo:Reach. People pay money to companies that either solve their problems or entertain them.

</comment></comment><comment><USER7> Emailed Eric something akin to the "Hunch for food" idea a while back.

How I roughly imagined it:

Scrap the calorie count. Unless you're at a fast food restaurant where it's posted on the menu, people really are bad at guestimating these things.

Change the health ranking to a taste ranking, and have it be for specific dishes at specific restaurants. While Foodspotting is sort of playing in that space, they are just asking people to submit pics of stuff they like, sans rating. While it's great to know what people like, I also want to know what to avoid. And it's much harder to sort through comments than it is to glance at numerical ratings.

Use case: I go to Corner Bistro for the first time. I do a search for Corner Bistro on Eatly. I see that Eric has had the cheeseburger and that Mike has had the chili and fries. With ratings added, I see Eric gave the burger a 9, but Mike gave the chili a 5, so I'm going with the burger.

In the same vein, using tags and descriptions, aggregate photos and average scores for menu items at restaurants. So I look up Corner Bistro, and I'm presented with a photo (the top ranked photo?) and average taste rank for each item on the menu. It's the Waffle House of Menupages [and I just created a new vertical].

And, yes, eventually it could suggest restaurants and dishes based on my activity and that of friends.

Obviously need to make the entry process as frictionless as possible. Few different thoughts there:

1) Expand current email capabilities, a la Posterous. Use special characters to note the restaurant name, add tags, rate the item, etc. 2) Automate and outsource: map EXIF data to restaurants (imagine you guys could find a database like that somewhere...), superusers for aggregating and tagging, etc. 3) And has been mentioned, iPhone app that makes it all pretty and easy

</comment><comment><USER8> With regards to encouraging users to contribute photos: Have you thought about creating a few short blog posts about good ways to photograph food? Things like lighting, angles, how to try to take a decent picture using a smartphone, etc?

Food is pretty hard to photograph well, we tend to have overly high standards for what a good food photo looks like because of all the food porn around these days. Just like regular porn, the real thing is going to look a little different. If your users hate the pictures they're getting when they take pictures of their food, they're never going to upload them. They'll probably give up completely after a couple of attempts.

<comment><AUTHOR> Well version 1 was really not meant to be about the quality of photos (i.e. food porn), but if we do move in this direction it's a great suggestion

<comment><USER8> Ah, my point was just that if the users don't like the photos they take, they're not going to upload them.

<comment><USER9> To reinforce this point, we did not have core competency to build an iPhone app and wanted to keep the project bootstrapped. Given the feedback here it seems an app would in fact make a big difference.

</comment></comment></comment></comment><comment><USER10> Ideas, some already mentioned:

\- Android/iPhone app indeed seems like a must, maybe not even so much for ease (tho that would help), but for branding of Eat.ly as a legit product in the consumer's eyes.

\- I wouldn't be into the health aspect personally, but I would be into the food porn aspect. It seems too complicated guessing calories etc. BUT maybe you can have two "modes". Turn on "Food Porn" and that is what you get, switch to "Health Porn" and that's what you get. I probably would peek at both.

\- This is probably not doable, but if you could integrate menus (like menupages) and calorie counts from restaurants, people could take the picture, pick the food (like on foursquare you pick the venue) and post. Food numbers would come along.

\- Single player mode would work better in health mode than food porn mode. You could have numbers to strive for, but then why would you post pictures really? You could just post food.

\- Multi player mode could work for sexiest looking food, or for "I'd Eat That!" points. Maybe unlock a badge swarm style.

\- Not to mix day job with side project, but it could be pretty sweet to integrate the Eat.ly app (if there will be one) with a "post to foursquare" option so that if you check in your foursquare status can have an eat.ly link to a picture.

</comment><comment><USER11> As many others have said, iPhone app definitely. This will make the site more casual.

For the "pivot" i'd perhaps suggest trying to get people to document recipes with multiple pictures, this lets you target the more hard core user. Theres an excellent blog that i love reading that gives a good example of what im describing: <http://smittenkitchen.com/>

<comment><USER12> I like the recipe ideas, there's not a lot of good recipe content out there, and photos are key

</comment></comment><comment><USER13> Man, I saw you guys present at the Quantified Self Meetup in NY (either late last year or early this year) and I thought, man these guys have a great idea, and are waaaay ahead of my "track your daily activities" project. Sorry it didn't take off!

<comment><AUTHOR> Awesome! That was great fun night. We've tried to hold true to our roots with quantified self/personal tracking. However, most feedback (here and from friends) suggests we pivot away from this

<comment><USER13> There's something really strange about personal tracking - it seems like such a good idea, full of great potential for people, and yet the same problem happens over and over again - people start using it, fall off and never pick up.

</comment></comment></comment><comment><USER14> What about a leaderboard or rankings for popular dishes. Users "vote" by taking pics of delicious food. Restaurants would be more likely to promote the site - "Eat.ly votes our cheesesteak best on the East Coast".

You could also run contests for users - each week a different dish. Submit the most beautiful salad, win a prize.

Finally, promoting the site through restaurant weeks might work out well. There are 120+ restaurants participating in Philly's restaurant week right now. Best photo wins another meal.

<comment><USER9> This is a great idea. The problem with leaderboards and points is a network effect that is not currently there yet. Its a chicken or the egg problem that exists, and we tried to get something out that didn't require mass participation but rather had a strong single player game model. Now that we have some scale (tiny!) this may be a good thing to try

</comment></comment><comment><USER15> Why _wouldn't_ you have an iPhone app? It would make sending the pictures much easier. It's a small revenue stream, but it is one.

<comment><USER15> Also, why not pivot into the diet/accountability market? It seems a bit skeezy, but people might pay for it.

<comment><AUTHOR> It's possible. One thing we looked at was working with personal trainers, or programs like weight watchers. Another variation was to use mechanical turk to 'guesstimate' calorie counts and charge as a premium service

</comment></comment></comment><comment><USER16> What is the objective of the pivot? Cash? Users? Pictures? Something else?

<comment><USER17> Sounds to me like they're startup enthusiasts, and want eat.ly to get bigger and better without sacrificing the stability they have right now, with their full-time jobs.

In other comments, people are saying the iPhone app is important. I think they already know this, and what would be more interesting to them is any shortcuts they could take to build their iPhone app.

I think they ought to find an iPhone developer who has developed an app similar to what would be needed for eat.ly, and is in a similar position where they are enthusiastic about startups but not doing it full-time.

<comment><USER16> As far as shortcuts I'd recommend doing the "app" as a jqtouch view of their rails? app. That should work for the vast majority of users and avoid the whole outlay for the app store. They could also look at phonegap. It's far better to figure out down the line that you need a full blown app than to figure out that your app could have been done in HTML. iPhone is a pretty big learning curve.</comment></comment></comment>


How to tune a guitar with Ruby and FFT - <AUTHOR> http://makaroni4.com/ruby/hacking/2014/03/26/how-to-tune-guitar-with-ruby/

<comment><USER2> "As you can see we need to pull the string for 77.09Hz :)"

What does that mean?

Edit: 329.63 (desired) - 252.54 (actual) = 77.09

I was confused because the frequency/magnitude plot had the maximum-magnitude frequency in the title. Perhaps that point should be labeled instead.</comment>