Ask HN: need advice on how to write lots of data fast - <AUTHOR>

I'm working on an analytics app that will be generating several http requests for every page view. Probably an average of 2-3. I'm mainly a front-end guy and my knowledge of best practices for storing data is kind of lacking.
I know that writing to log files is hard drive intensive. I also know that the preferred method is to write to a database, which I know how to do and can easily setup. But I'd like to stretch my boundaries a bit here and learn something new.
I had an idea the other to use nginx as my webserver to handle the requests. Very fast & lightweight. It would dump any new data into a memcached instance, with a cron running every couple minutes to suck the memcache stuff either into a db or into a flat file. Even I can think of several reasons why this might not be optimal, but I thought it would a cool way to learn about how to use memcache and get experience with building light-weight systems.
So does anyone have any thoughts on anything I've said, and maybe can make some recommendations?
I ran into a related project called memcachedb, which purports to use the memcache api calls but writes to a berkelydb backend. I haven't looked into it much, but it does seem interesting. <comment><USER2> _I know that writing to log files is hard drive intensive. I also know that the preferred method is to write to a database_

No. Databases are _far_ more disk intensive than logging; they use vastly higher disk bandwidth, and depending on the database, often use lots of disk seeks too.

<comment><USER3> It's generally true for traditional DBs, a few new generation DBs have options to buffer writes in memory and achieve durability via in memory replication in other nodes and only writes to disk sequentially in large chunks, which is better than simple logging as you'd have to sync to avoid data loss upon hardware failure.

<comment><USER2> If you're going to consider a database which buffers writes in memory, you should also consider logging which buffers writes in memory.

<comment><USER3> Maybe my comment was not clear enough: buffering writes in memory on a single node alone has no durability to speak of. You need either a DFS or distributed DB that does replication of the buffers on different nodes to provide some reasonable durability.

</comment></comment></comment></comment><comment><USER4> I think you're looking for scribe:

<http://developers.facebook.com/scribe/>

</comment><comment><USER5> One of the questions you need to ask is: what you are going to do with all that data? If all you need is to store the data and never read it again except sequentially, then flat log files are probably the best way. But if you ever need to, say, calculate aggregates or do any kind of visualization, a database would probably be a better initial choice.

You'll want to keep in mind that whatever you choose should be pretty flexible, because as you discover more about the analytics you're providing, you'll need to slice and dice that data differently. You won't know what data and how it will answer the questions until you know the questions.

</comment><comment><USER6> Put the data in a message queue asynchronously. A second process can take the data from there and put it in a database or wherever it's easiest for you to analyse.

</comment><comment><USER7> discodb looks great for analysing analytics data

appending data to a file is a pretty cheap operation, just make sure you append properly and dont do something like read the whole file, append the string, then write it all, a lightweight server like mochiweb will probably have you handling more traffic than you can handle for a while before having to do a proper logging queue.</comment>


What's New in Docker 1.13? - <AUTHOR> https://blog.docker.com/2017/01/whats-new-in-docker-1-13/ <comment><USER2> Docker for Mac is a pretty terrible experience. [https://github.com/docker/for-mac/issues/77](https://github.com/docker/for- mac/issues/77) has eaten a huge chunk of time for my team and the issue has been open and known about since the beta. Yes there are hacks to make things better, but as of a month or two ago the Docker team seems to have gone radio silent after providing periodic updates.

<comment> duncan-donuts I have given up on using docker on macOS. I now use Ubuntu at work, and it's interesting to watch a lot of my co-workers give up on macOS too. I totally agree it is a horrible experience.

<comment><USER3> Use Docker on Ubuntu on my own machine, but forced to use a Docker on Windows at work. Using Docker on Linux is an absolute joy. I totally advocate it. Dockerhub is my first port of call when I'm installing something new on my machine because I know if there's an official repo it will work.

Docker outside of Linux is another story. I've run into config issues, VM memory problems, proxy issues, ect. Painful.

<comment> duncan-donuts Same here. I was struggling using docker at work on my Mac, but when I was working at home on my Linux box it was amazing. Docker is awesome on Linux and I have never been more productive.

</comment></comment></comment></comment><comment><USER4> Hopefully native squashing will allow some people to avoid writing Dockerfiles that look like

      RUN this && that && thistoo && thattoo && rm this && rm that

in order to avoid generating extraneous layers.

There have been some third party tools and registries that implement this feature, but it's nice to finally have it upstream.

<comment><USER5> Honestly, I see this as an antipattern, not because it generates extra layers, but because a Dockerfile isn't the right place to write a provisioning script.

When building apps, I keep a small number of shell scripts that live in a `bin` directory, that perform essential operational tasks.

One script installs all the dependencies required to either run or develop the app; another runs all tests and exits with either success or failure; and the final script just runs the app.

Docker then just runs those scripts, which are written deliberately to be easily read, and thus function as living documentation for the project.

These same scripts get used, both by developers on a daily basis, as well as by our CI/CD system when prepping containers.

This also makes onboarding a snap: you run `bin/setup`, and your Mac or Linux box is good to go. And, because that script gets used every time the CI system spins up a build, it will never go out of sync with reality.

<comment><USER6> The problem is that if you're building images a lot, Docker doesn't cache COPY calls because it doesn't know if the file has changed or not when calling the COPY command. So if you constantly build images and there are a lot of computationally intensive tasks within those scripts, but you've only changed one small item near the end of the script, your build time may be significantly longer by using the script than by putting it directly in the Dockerfile using the RUN command.

I build a lot of modules within one of my Docker images for work, and while it doesn't change often, I would really not like to wait the 20 minutes if I really needed to push to production when I only change some certificate population segment at the end of the Dockerfile or something after a particularly intensive module build RUN step.

<comment><USER7> Docker does cache COPY calls AFAIK.

If you're copying your whole directory though the cache breaks if any file changes.

The normal way of doing it is to copy stuff like requirements.txt, package.json, Gemfile etc first and install stuff, then copy everything else at the end

<comment><USER6> Ah, maybe that's true, I always figured it didn't cache it since it wasn't text. In this case it doesn't matter since the script would change, but if the scripts were broken apart and ran using RUN commands it would be a better middle-ground solution than wholly putting it all in a script with a single RUN command.

</comment></comment></comment></comment></comment><comment><USER8> I'm very excited about the clean-up commands!

Docker _desperately_ needs this, it's so frustrating constantly having full disk space due to untagged containers and unbounded volumes.

<comment><USER9> If you're on Mac, it's still a bit of a problem; hopefully the SSD-gobbling Docker.qcow2 file problem[1] is fixed soon!

[1] [https://github.com/docker/for- mac/issues/371](https://github.com/docker/for-mac/issues/371)

<comment><USER10> Looks like it is fixed in [https://twitter.com/kelseyhightower/status/82223709949555097...](https://twitter.com/kelseyhightower/status/822237099495550976). Im looking for official changelog for confirmation though.

<comment><USER2> Judging by the github issue list there are a lot more issues with Docker.qcow2 now than before. Mostly related to Docker.qcow2 preventing docker from starting.

Looks like a bad release. Hopefully 1.13.1 is soon.

<comment><USER11> (I work on Docker for Mac.)

Apologies for the inconvenience this has caused.

There was a race condition in a previous release which could allow multiple hypervisor instances to open the Docker.qcow2 simultaneously. Unfortunately this can corrupt the file by allocating the same physical block (cluster) twice, resulting in bad things happening. When this file-locking bug was fixed we also added an integrity check which checks the structure of the Docker.qcow2 on every application launch. For safety the app refuses to start if corruption is detected.

I believe that in these cases, the corruption happened in the past and is now being detected since the upgrade. Unfortunately if the app refuses to start it makes it difficult to reach the "Reset to Factory defaults" menu option. The workaround described here [https://github.com/docker/for- mac/issues/1159#issuecomment-2...](https://github.com/docker/for- mac/issues/1159#issuecomment-273899215) is to remove the qcow2 and restart the app. Unfortunately containers and images will need to be rebuilt.

For what it's worth after the integrity check and the locking fix went in, I've not seen any recurrence of this error. Please open an issue if you see any other problems!

</comment></comment></comment></comment></comment><comment><USER12> The "Use compose-files to deploy swarm mode services" is really intriguing to me, especially when they quote this one liner:

> docker stack deploy --compose-file=docker-compose.yml my_stack

But there's no links to further documentation or anything. Can this be used to deploy easily to a cluster of Droplets for instance?

I feel like the low end/longtail deployment of Docker is really underserved. I want to use Docker for its devops merits, but I have yet to find a clear, concise guide for deploying a simple web app to one or a cluster of VPS instances for a modest traffic project.

<comment><USER13> Have you looked at Kubernetes? It's pretty dead simple to get a cluster up these days and the general abstractions they chose are really great. Service discovery, config and secret storage, control loops, autoscaling, even stateful containers these days.

100% worth giving it a shot, even just to say you've tried it.

I've been using Docker for over three years now, and Kubernetes really is the realization of what I imagined containers would be like when I was just starting to use them in development.

<comment><USER14> > It's pretty dead simple to get a cluster up these days

The standard Kubernetes setup instructions are horrible. You need to run a bunch of shell scripts including cluster/kube-up.sh, which are documented to work on AWS, but which are completely untested before release, and which were recently broken for almost a month.

I'm currently running Kubernetes under Rancher. It's still a pretty steep learning curve, with some very well-hidden but essential configuration parameters, but at least it actually works if you follow the instructions.

<comment><USER15> The situation has been improved quite a bit with kubeadm. Getting a basic cluster set up is a lot easier with that.

[https://kubernetes.io/docs/getting-started- guides/kubeadm/](https://kubernetes.io/docs/getting-started-guides/kubeadm/)

</comment></comment></comment></comment><comment><USER16> In addition to crappy mounted volumes, Docker for Mac eats up CPU like crazy: [https://forums.docker.com/t/com-docker-hyperkit-up-cpu- to-34...](https://forums.docker.com/t/com-docker-hyperkit-up-cpu- to-340/13057/2)

And, also like the mounted voluems issue, despite this being on of the top things on the forum, it's still been a problem for like 8 months with no attention.

</comment><comment><USER17> This release broke our Azure Container Service Kubernetes deployment. Had to downgrade back to 1.12 for now, but not without suffering a day of downtime trying to trace the issue.

<comment><USER18> What was the issue?

<comment><USER17> [https://github.com/Azure/ACS/issues/3](https://github.com/Azure/ACS/issues/3)

</comment></comment></comment><comment><USER19> Use Terraform to provision some virtual machines, load balancers, bastion host, vpc, etc. [https://news.ycombinator.com/item?id=13436415](https://news.ycombinator.com/item?id=13436415)

Use Docker to setup development and production environments for a sample Flask application with CI/CD. [https://news.ycombinator.com/item?id=13436452](https://news.ycombinator.com/item?id=13436452)

</comment><comment><USER20> Huge news, I really hope this is gonna help the workflow and give another alternative for scheduling. I have been waiting for the update since last November (iirc it was scheduled for end of 2016).

BTW - Am I the only one finding the quality of the linked video very bad?

</comment><comment><USER21> > docker system df will show used space, similar to the unix tool df

I thought 'df' is for free space, 'du' is for used space. Can anyone explain why the went for 'df', not 'du'?

<comment><USER22> df shows both used and available space on a whole disk/device.

du is for space used by particular files/directories.

</comment></comment><comment><USER20> I can't find anything about secrets in compose files... Any ideas about this?

<comment><USER20> Ok, it still seem to be in development: [https://github.com/docker/docker/pull/30144](https://github.com/docker/docker/pull/30144)

</comment></comment><comment><USER23> why does the squash flag squash everything? Wouldn't it possible to only squash the layers introduces in the localhost build? The you could share the other layers again.

<comment><USER24> It only squashes the layers produced by the build.

</comment></comment><comment><USER25> I moved on to LXC

Never looked back

Congrats to the docker team in the new release anyhow.

<comment><USER26> What do you use instead of "docker build" and "docker pull"?

<comment><USER27> Could use the distro's package manager?

<comment><USER26> What do you mean? My Dockerfiles already use distro's package manager to install packages.

Do you you mean "create a long-lived LXC container and use distro's package manager to install packages as needed?" The problem with that approach is you have no idea how to duplicate the server, nor how to roll back the changes reliably. For example:

\- Your distro upgraded a library, and the upgrade introduced a bug. For extra fun, this was not security upgrade so unattended-upgrade process did not install it, so only half of your servers have the bug.

\- Your package list is incorrect. Somehow, your old server ended up with an extra package (previous software version? installed while trying to troubleshoot problems), and your new server does not have it.

\- You have leftover files -- either from the previous version of your software, or from the package you have had installed before.

These are definitely fixable in LXC with enough effort -- after all, Docker is not magic, and you can achieve a lot with LXC + shell scripts, and even more with LXC + shell scripts + chef. However, Docker is just so much easier and reliable than writing these scripts by hand.

<comment><USER25> LXC is sane to me. Upon first glance it is understandable and works everytime.

I went all in with Docker, it was all fun and games up until I tried it in production, and I started relying heavily on it.

Docker introduces container linking voodoo which have burnt me badly. Sure enough voodoo comes with its upsides, we now have Kubernetes which obviously I think is very cool. Just far outside of my use case.

Serenity now!</comment></comment></comment></comment></comment>


On Lowered Expectations: Transactions, Scaling, and Honesty - <AUTHOR> http://blog.foundationdb.com/on-lowered-expectations-transactions-scaling-and-honesty

<comment><USER2> Hey guys, I'm the author of the blog post. I'm happy to answer questions about performance results, argue about the necessity of acid transactions, or discuss anything else in the post.

</comment><comment><USER3> I've noticed a trend of naming articles "On $topic1, $topic2, and $topic3". Seems like a lazy way to title your article. Yeah, it tells us what your article is about, but not specifically what it's about. Just lists 3 topics that the author will somehow involve in their discussion.

<comment><USER2> That's fair. My runner up title was Scaling All the Things: An Honest Look at Limited and General Transactions. Better or worse?

<comment><USER3> A little better. The current title includes the words honesty, transactions, and scaling, but didn't show how they were related. Your original title at least strings them into a coherent sentence that more accurately describes the content: how databases really perform in terms of scalability, and how transactions affect that scalability.

<comment><USER2> Makes sense to me. Thanks for the feedback.</comment></comment></comment></comment>


How to disrupt the brogrammer vibe with 1 simple change - <AUTHOR> https://www.atlassian.com/blog/inside-atlassian/how-to-fix-brogrammer-culture-hire-for-values <comment><USER2> tldr: The change is don't hire Brogrammers.

<comment><AUTHOR> Well, that's the result.

The change is not to interview for "cultural fit" but for "values alignment".</comment></comment>


Faster pf - <AUTHOR> https://blog.pfsense.org/?p=1473

<comment><USER2> Is there any kind of tutorial for unexperienced network admins on how to use pfSense and how pfSense differs from: (1) a cheap home router; (2) Mikrotik RouterOS; (3) A full Linux router (involving iptables, DHCP server and other tools)?

I would like that.</comment>


Climate Change in a Nutshell: The Gathering Storm [pdf] - <AUTHOR> http://www.columbia.edu/~jeh1/mailings/2018/20181206_Nutshell.pdf <comment><USER2> Climate discussion now seems to come from Yes, Prime Minister. (1986). We appear to be at stage 3.

I don't understand US politics.. Most UK voters, of both political sides, agree there is an issue, and want constructive solutions[0]. Some more US-like differences at the margins, and differences in urgency, but there appears to be a broad consensus among the public. Less so amongst politicians themselves.

Does political consensus no longer exist on any issues in the US any more? Is all science politicised?

Sir Richard Wharton: "In stage one we say nothing is going to happen."

Sir Humphrey Appleby: "Stage two, we say something may be about to happen, but we should do nothing about it."

Sir Richard Wharton: "In stage three, we say that maybe we should do something about it, but there's nothing we _can_ do."

Sir Humphrey Appleby: "Stage four, we say maybe there was something we could have done, but it's too late now."

[0] [http://www.lse.ac.uk/GranthamInstitute/news/the-public- suppo...](http://www.lse.ac.uk/GranthamInstitute/news/the-public-supports-uk- climate-leadership/)

<comment><USER3> In the U.S. and apparently also in Australia, it's all politicized by the right wing climate science deniers, who have swallowed petrol industry propaganda, who view the embrace of basic science by anyone, regardless of their actual place on the political spectrum, as an Obama/Clinton/"libtard" hoax aimed at taxing their precious wallets and taking away their freedom. You can see it in the comments here denying climate science and making false claims about the existence of "many scientists" who disagree with the consensus and think there are other causes of climate change, ignorant nonsense about coming ice ages, and so on.

</comment></comment><comment><USER4> I'm in my thirties. I don't have kids, but want them. I don't want to admit to them that I did nothing about global warming when it counted. The thing is that I don't know what I should do; it seems so big, so intractable.

Articles like these stir me to action, but what action? Where's the mass movement on this? Where's the group, the party, the leadership?

<comment><USER5> First of all, since this is a community in which many fancy themselves engineers, ground yourself in the numbers. The main ones being:

\- number of gigatons emitted to date

\- rate of ongoing emissions

\- change in the rate of ongoing emissions

\- what total emissions numbers correspond to what PPM in the atmosphere

\- what ppm in the atmosphere correspond to what degrees of C warming (in various timeframes)

here is a nice infographic from the latest ipcc report that shows some of them: [https://www.ipcc.ch/site/assets/uploads/sites/2/2018/10/SPM1...](https://www.ipcc.ch/site/assets/uploads/sites/2/2018/10/SPM1_figure- final-e1541758557589-947x1024.png) pay particularly close attention to the one in the bottom left.

\- what degrees C warming correspond to drought, desertification and crop failures in what areas/river-valleys (note: sea level rise won't actually be a problem until after the famines, focus on the famines)

Once you get a feel for the numbers the "what to do" becomes a fairly obvious "everything/panic/get-super-duper-radical". But I can't elaborate on that more here because it ranges from "borderline communist levels of leftism" to "out and out eco-sabotage".

DO NOT fall for the false framing that a rump dumbass 20-30% of the population doesn't "believe" in global warming. That is smug nonsense that people use to justify a false-framing of some kind of stalemate. No society ever convinced everybody of the way forward. The _real_ problem is the litany of excuses and magical thinking that the so called "believers" will come up with to do nothing. Such as:

\- there's nothing we can do, give up, its too late

\- magically innovative technology will save us just hold your breath till it comes along

\- teh markat will provide! if we can just do this spreadsheet game with tax credits or something

\- i can't do that thing i should do because even though I'm one of the richest humans on earth and that ever lived I've cast myself as a victim of the current economic order and cannot afford to do anything.

Now, all that said as context, here are the top 5 things, IN ORDER (assuming you're a middle class american), you can do to curtail your and your families carbon footprint:

1.) have a negative-population-growth number of children. that means somewhere in the <2.1 range (stupid joke about cutting parts of babies goes here).

2.) do not use a car on a daily basis. ideally don't even own one. if you _must_ own one make it electric.

3.) live in a 5+ unit building. this cuts your heating/cooling/lighting footprint roughly in half, and ties into the car thing above (mid rise apts correlate strongly to walkable areas). no amount of solar panels and electric cars will make suburban-sprawl a sustainable land use or energy consumption model in your or your childs lifetime. this is the absolute hardest part for americans to swallow, they will pitch a temper tantrum fit and write off the messenger rather than grasp this one.

4.) fly (round trip) less than once a year

5.) don't eat red meat, or at least cows. even if you're not ready to give it up, you could deliberately replace red meat meals with chicken fish or vegitable alternatives more often.

However, I want to stress that viewing things through the lens of individualized personal consumption levels is a TERRIBLE focus. This is the the most universal problem humanity has ever faced. You want to do the 5 things above not because it will cut your footprint 10 - 20 tons per year (~50 - 80%, which it will), but because you will be contributing to building a society in which _everyones_ footprint is cut. You want to normalize apartment dwelling, walking, public transit, avoiding unecessary flights, and avoiding red meat. That is how you contribute to building a sustainable future society. You need to walk the walk not for a few tons, but to show your children and neighbors the way forward.

<comment><USER6> I'm starting to wonder if telling people what they can personally change in their lives is actually counterproductive. The gains to be had there are not that great.

Sure it's good to minimize energy use as you outline. But where does it leave you in terms of tonnes of CO2/year?

AFAIK the Paris accord goals (inadequate as they are) imply a target of around 1500 kg of CO2 per capita yearly. Homeless people achieve that in the west. You won't get near those levels, hower much you try to reduce your personal consumption.

We're not going to individual-responsibility our way out of this, and I wonder if getting people to try will just give them a false sense of accomplishment.

<comment><USER5> you're not wrong but careful not to fall into the internet's favorite form of broken depressive reasoning: splitting

the way you get mass action is by getting a mass of individuals to act. there is technically no other way. thats why I left the 'individual' stuff to the very end and made sure to explain how its not really about your footprint as much as its pulling your oar in the overall solution.

</comment></comment></comment></comment><comment><USER7> I still remember watching TV in early 90's and seeing clips from Hansen's 1988 senate testimony and hearing first time the "Now is the last time to act on climate change." from someone in TV.

Key numbers to get the point across.

* Starting mitigation in 2000 would have required mitigation rate 4%/year. (1.5 C goal)

* Starting mitigation in 2018 will require mitigation rate 18%/year. _18 percent!_ (1.5 C goal). There is no realistic scenario where this can happen.

* Nine years from now complete halt of all manmade CO2 emissions is too late to prevent crossing 1.5C.

* Global fossil and cement CO2 emissions keep going up, not down: [http://folk.uio.no/roberan/img/GCB2018/PNG/s09_2018_FossilFu...](http://folk.uio.no/roberan/img/GCB2018/PNG/s09_2018_FossilFuel_and_Cement_emissions_1990.png)

* CO2 mitigation curve becomes steeper and steeper. [http://folk.uio.no/roberan/img/GCB2018/PNG/s00_2018_Mitigati...](http://folk.uio.no/roberan/img/GCB2018/PNG/s00_2018_Mitigation_Curves_1.5C.png)

<comment><USER8> One thing I'd be interested to read is the difference between 1.5C and, say, 2.5C of warming. Do you know of any reports (or maybe just part of the IPCC reports) which lay out the incremental risks?

I just read for instance an editorial from nature which said that 10-30% of coral reefs survive at 1.5C but none survive at 2C. I'd like more of that kind of information to understand the risk profile.

<comment><USER7> carbonbrief.org has interactive comparison between 1.5C, 2C and 3C. [https://interactive.carbonbrief.org/impacts-climate- change-o...](https://interactive.carbonbrief.org/impacts-climate-change-one- point-five-degrees-two-degrees/)

<comment><USER8> Perfect, sincerely thank you for the link.

</comment></comment></comment></comment><comment><USER9> I've long been in the camp that we worry about climate change more than makes rational sense.

I'm not saying we should ignore it, but we often overstate. It's worth remembering that the effects on QoL will be completely eclipsed by technological progress within that period and that the timescales that GW happen over (50+ years) are long enough that most people will hardly ever notice it. Gradual relocation will be our primary means of fighting it. Other cities will be raised, like Chicago was in the 1850s.

FUD is not a genuine approach to invoke action - which has sadly been the main approach of many. We should remain reasoned.

<comment><USER8> If the world was composed of countries living at the quality of life of Western Europe or America and with that level of technical and economic expertise this would I think be reasonable, but in fact billions of people are living hand to mouth. Look at the effect that Syrian migration to Europe or indeed Mexican migration to the USA has had on Western politics. Now imagine instead of 5 million people from Syria 100 million or 500 million are on the move? Is the population of Bangladesh going to gradually relocate, and if so where to? Its nuclear armed neighbour on one side, with their own hundred million to be displaced from subsistence agriculture, or the country conducting ethnic cleansing on the other? You have assumed a greater degree of stablity than is reasonable.

<comment><USER9> People will primarily move more inland within their own countries. Some cities will be raised. Neighboring countries will absorb some. Small border shifts may happen, as have happened for centuries.

<comment><USER10> You have no reason to believe this.

From: [https://www.esquire.com/news-politics/a25422366/trump- ambass...](https://www.esquire.com/news-politics/a25422366/trump-ambassador- to-canada-climate-change-scientists/)

"The fourth and latest National Climate Assessment put together by 300 scientists from 13 agencies of the U.S. government and released last month found that climate change is real, man-made, and will cost the U.S. 10 percent of its economy by 2100. Midwestern farmers will lose 75 percent of their crop yields, and trillions of dollars in coastal real estate will be at risk. The wildfires out west, already unprecedented in their destruction, will get worse. Hurricanes and typhoons will grow more ferocious. Epidemic diseases will flourish."

And that's just for the US. Large portions of Africa and the Middle East are predicted to become uninhabitable. The threat to human life on a time-scale of 100 years or more is gargantuan.

<comment><USER11> Wow, by 2100 it will cost the US 10% of its economy? You mean a small size recession? And you all are such a fuss about? And for that you want to deprive 3rd world people from gaining the same benefits you get from energy?

This is alarmism on the grandest scale and a horrifying hypocritical at that. There is nothing we can do about CO2 right now. We can just continue to work on safer and cleaner ways to generate electricity (atomic mainly, fusion hopefully, solar for the local homes, etc.)

<comment><USER8> The predicted costs of tackling it are significantly lower, so why not just tackle it and avoid the recession.

Also, a permanent loss of 10% is not a small recession, most recessions lead to no permanent loss, just a demand shock which is made up afterwards. A permanent loss of 10% is more like the financial crisis, and I'd rather avoid that if I can. One brought Trump and Brexit, I'd rather not add another to the mix.

</comment></comment></comment></comment></comment></comment><comment><USER12> Some constructive criticism: I think this document is written backwards and does not contain what CEO's are interested in. The disastrous effects should come first and preceding that should be forecasts of the effect on the bottom line (profits). The business implications of climate change are what might capture their attention. Market disruptions for example, trends in adoption of EV cars, growth in solar and wind, tech industry use of alternative energy, costs for pollution being added to government and other plans, current problems with changes in forests and their effects on industry, plans to deal with coastal changes. Changes in behavior from industry and government are more likely to sway them. They attempt to control those, but a list of existing trends that are outside their control could be persuasive. They have hired scientists to confuse the issue, so they are not going to be swayed by a scientific argument. Leave that as an appendix for their subordinates to read. It's unfortunate, but I think you have to speak their language.

<comment><USER13> My experience is that CEO's don't need persuading on this issue, they are convinced but their fiduciary duties limit the scope of their actions. What they would like is sharp, clear regulation that provided a level playing field. The people who need persuading are voters.

<comment><USER12> Hansen does say in the document "My target is the level of a Chief Justice or a fossil fuel industry CEO", so I was thinking of fossil fuel industry CEO's (who probably have never experienced a level playing field). Voters need someone already persuaded to vote for; I think polls show the voters themselves weigh on the convinced side. Fiduciary duties are only part of what a CEO should be concerned about, but certainly major changes in what other industries are doing should be part of fiduciary duty. Like VW saying they are going to drop internal combustion engines in the near future, forest blight affecting logging, Apple and Google going solar, real estate trends in coastal areas, cities creating climate change plans, and more.

</comment></comment></comment><comment><USER14> Skimmed through this, but I can't make it all the way. It's so jarring and too depressing.

I lump climate change deniers into the same category as flat-earthers. Although I have more respect for the latter, as at least they aren't blindly swallowing self-serving nonsense spewed by ignorant politicians and certain news outlets.

<comment><USER15> I don't think there are scientists who _deny_ past or present climate change. Do you mean to disparage the many scientists who research the causes of past climate change, and who hypothesize that the current changes are, like all past changes, caused by other factors than the man-made, industrial-age rise in atmospheric carbon dioxide?

<comment><USER16> There is overwhelming consensus that the current rate of climate change, which by the way is totally unprecedented in history, can be from no other source than anthropogenic CO2. You can read the research and see how thoroughly and completely every possible confounding variable was accounted for.

The scientists who claim "current changes are, like all past changes, caused by other factors than the man-made" are not taken seriously by the mainstream, form an unimaginably slim minority and frankly are dangerously wrong.

[https://www.ipcc.ch/report/ar5/wg1/](https://www.ipcc.ch/report/ar5/wg1/)

<comment><USER17> Let them be wrong then. But science is not a democracy. You can have overwhelming expert consensus and still be completely wrong. You can get experts to hold a vote and agree some new idea is complete garbage but that vote does not change immutable truth. It's happened countless times in history.

For example, Ignaz Semmelweis (from wiki):

> Despite various publications of results where hand washing reduced mortality > to below 1%, Semmelweis's observations conflicted with the established > scientific and medical opinions of the time and his ideas were rejected by > the medical community. Semmelweis could offer no acceptable scientific > explanation for his findings, and some doctors were offended at the > suggestion that they should wash their hands and mocked him for it. In 1865, > Semmelweis suffered a nervous breakdown and was committed to an asylum, > where he died at age 47 of pyaemia

</comment></comment></comment></comment><comment><USER18> I invite everybody to watch this [https://www.youtube.com/watch?v=vaiZ5BHaUMY](https://www.youtube.com/watch?v=vaiZ5BHaUMY)

A calm, collected, scientific representation of both sides.

<comment><USER19> _There are literally no 2 sides_. There is a ~97% scientific consensus, then there are these people: [https://en.wikipedia.org/wiki/List_of_scientists_who_disagre...](https://en.wikipedia.org/wiki/List_of_scientists_who_disagree_with_the_scientific_consensus_on_global_warming).

John Christy (the other side in your video) represents the ~3%. His research was shown to be wrong over and over again. Most recently in 2017: [https://journals.ametsoc.org/doi/abs/10.1175/JTECH-D-16-0121...](https://journals.ametsoc.org/doi/abs/10.1175/JTECH-D-16-0121.1). [https://www.theguardian.com/environment/climate- consensus-97...](https://www.theguardian.com/environment/climate- consensus-97-per-cent/2017/may/11/more-errors-identified-in-contrarian- climate-scientists-temperature-estimates)

<comment><USER20> Check out this wikipedia page that casts scientific consensus in a slightly different light:

[https://en.wikipedia.org/wiki/Superseded_scientific_theory](https://en.wikipedia.org/wiki/Superseded_scientific_theory)

<comment><USER21> "John, when people thought the earth was flat, they were wrong. When people thought the earth was spherical, they were wrong. But if you think that thinking the earth is spherical is just as wrong as thinking the earth is flat, then your view is wronger than both of them put together"

[https://chem.tufts.edu/answersinscience/relativityofwrong.ht...](https://chem.tufts.edu/answersinscience/relativityofwrong.htm)

<comment><USER20> A bit over half a century ago, the scientific consensus in Germany was that Jews were subhuman. They also believed in a mystical ice planet orbiting the earth that possessed all sorts of wonders, the knowledge of which was suppressed by the anti-Arian cabal. Consensus bears a funny sort of relationship with the truth.

<comment><USER22> The scientific consensus in a single country is very different bar than a scientific consensus for the whole world.

I am pretty sure that if you looked at North Korea today, you could find some weird scientific national consensus, which obviously does not mean anything.

<comment><USER20> Is it true this is the consensus of the whole world?

It seems to be the consensus of a portion of the scientific establishment, which is certainly not representative of the whole world.

And, if, for sake of argument, we grant this scientific consensus represents the perspective of the world, it still does not follow this somehow precludes faulty reasoning and groupthink. "Science" is not a magic wand that automatically eliminates the normal sources of human bias, especially as recent news illustrates the rampant fraud and unreproducible results in the scientific establishment.

</comment></comment></comment></comment></comment></comment></comment><comment><USER15> After solving global warming, should we be more concerned by the approaching end of the current interglacial period? The next glacial period of the current Ice Age is going to be terrible for our stage of technology and civilization.

If we are serious about the current issues of climate change, and about potential asteroid impacts, why should we not also be planning for the mid- to long-term risk of the return to an Ice Age?

<comment><USER20> This is an interesting question. Haven't there been much warmer periods in our planet's history that were followed by an ice age?

It is hard to sort out exactly what climate change means.

<comment><USER19> Those warming periods are either local, are much, much slower and they are not supposed to be in sync with human activities either.

This one is much faster. Not everyone sees it of course. Change blindness is what they call it.

[https://www.npr.org/sections/krulwich/2014/02/05/257046530/b...](https://www.npr.org/sections/krulwich/2014/02/05/257046530/big- fish-stories-getting-littler?t=1531204154500&t=1537827635100&t=1544136887700)

<comment><USER15> This is not a correct description of past climate changes. The glacial and interglacial periods of the last 2.5 million years have been global and swinging temperatures by 6-9°C in either direction, with periods between 40-100k years.

During the last interglacial (Eemian, ca. 129–116k years ago) sea level was 6-9 meters higher than today; Scandinavia was an island. The water temperature of the North Sea was about 2°C higher than at present; hippos lived as far North as the Rhine and the Thames. The onset of the Eemian took just a few centuries, during which global temperatures shot up >5°C.

<comment><USER19> [https://www.nature.com/articles/s41467-018-05314-1#ref- CR8](https://www.nature.com/articles/s41467-018-05314-1#ref-CR8)

The Eemian was caused by reduced North Atlantic ocean circulation, a courtesy of the previous Saale glaciacion period, too much fresh water, and continued transfer of heat from the south.

It produced abrupt effects localized to high altitude areas. Most of what you described occured in the later stages of this 15000 year long period.

The Eemian cannot be really compared to the current global warming, a potential AMOC (Atlantic meridional overturning circulation) instability is just one out of the thousand problems we are facing.

</comment></comment></comment></comment></comment><comment><USER23> The problem with many discussions around global warming is not scientific, but cultural. Advocates of change often take a sanctimonious attitude toward anyone expressing the slightest doubt.

Have a look at the episodes of the Netflix series "Bill Nye Saves the World" on climate change. If you can watch without cringing, congratulations. I can't. Or take this interview Nye did with Tucker Carlsen:

[https://www.youtube.com/watch?v=qN5L2q6hfWo](https://www.youtube.com/watch?v=qN5L2q6hfWo)

Nye, and those who share his intolerant attitude toward justified skepticism do their cause more harm than good.

 _Every_ hypothesis should be questioned. The more sweeping the claims, the more pointed the questioning should be. Science is not immune to groupthink. The problem is magnified when a scientific hypothesis takes on a political dimension.

Take this quote from the article:

 _Climate has always been changing, but humans are now the principal drive for climate change, overwhelming natural climate variability._

I challenge any reader of the article to find a clear, logical, undeniable chain of evidence from hypothesis to the conclusion that humans are responsible for rising planet temperatures.

<comment><USER19> Challenge accepted. Proof that humans are responsible for rising planet temperatures: [https://climate.nasa.gov/evidence/](https://climate.nasa.gov/evidence/). Follow the references.

"every hypothesis should be questioned."

It is, constantly, literally on a daily basis. That's how we know.

What does pop culture (Bill Nye) has anything to do with this.

Also many of us are a bit beyond just theoretical discussions. It's 2018. We are increasingly affected by climate change to the point that many of us are packing, because the shit already hit the fan. What's a fun theoretical discussion for you (at the moment) is actually life and death for many.

<comment><USER23> Thanks for the link and response, but I was talking about evidence in the OP article. Like far too many pieces, it fails to highlight the evidence between human activity and rising temps, taking it instead as an article of faith.

Even the link you gave fails to enumerate the evidence for a human cause, and itself links to another article.

I realize that some have made up their minds, but the extraordinary claim of a mainly human cause requires extraordinary evidence. And that evidence is hardly discussed, nor are alternative hypotheses given their due.

This is the problem I'm trying to highlight. The discussion has veered from scientific to religious.</comment></comment></comment>


TV Solidarity - <AUTHOR> http://www.idlewords.com/2007/04/tv_solidarity.htm <comment><USER2> Great story, but it really ought to have "radio astronomy, pirate TV" as a subtitle.</comment>


Ask HN: Is it even worth it to have automated translations on your site? - <AUTHOR>

What I mean is, if the options are to either:
1) spend time putting a system on your site that will translate it into another language using Google Translate or 2) ignore translation altogether and just deliver an English site
...is it even worth it to do #1? Or are non-native English people already accustomed to running English sites through something like Google Translate, in which case me baking that into my site doesn't really add any value?
Note that providing a manual, human-made translation (i.e. The Right Way) is not an option here, hence this question. <comment><USER2> Just putting a feature on the site that links over to the Google Translate version really isn't worth the time, since it clutters up the page, and those translations aren't very good. However, despite you saying it isn't an option, if you actually serve good translations through the site that are crawled, that may draw more visitors.

If it is a saas app and you want it to be used internationally though, I say go for it and use google translate to attempt to translate labels, etc. that you use. Provide a feedback area so that people can tell you if the translation is bad, if they feed like it. But be aware that there is much more to i18n and i10n than translation (legal, other gov't issues, cultural issues, timezone, character set issues in page and DB, layout differences).</comment>


Web Browser based control system for the RaspberryPi - <AUTHOR> https://github.com/NeonHorizon/berryio

<comment><USER2> I'm particularly interested in the promise of no keyboard mouse or monitor based wifi registration of the Pi. It doesn't seem like this is complete though. I'd love to see a Raspbian version that, if it detects a wifi dongle, automatically starts an open ad-hoc network usable for configuring the rest of the connection. Something like the way Nest or Chromecast bootstrap.

<comment><USER3> Isn't WPS mode (WiFi Protected Setup) the right solution here?

</comment></comment><comment><USER3> If you are interested in the programmatic version of this, I'm working on a project called Nitrogen that enables you to manage a fleet of Raspberry Pis (or any embedded device) and write applications that work across them.

See [http://github.com/nitrogenjs/service](http://github.com/nitrogenjs/service) for more details...

</comment><comment><USER4> What's the target market, aside from hackers trying to get familiar with the board, or maybe some really basic debugging?

Is exposing pin-level hardware details over an HTTP API really helpful for building larger applications? And why not run that larger application directly on the ARM?

</comment><comment><AUTHOR> Author's announcement on Google+: [https://plus.google.com/109352235257103413028/posts/9X66y4jo...](https://plus.google.com/109352235257103413028/posts/9X66y4jogUk)

</comment><comment><USER5> While I'm not into the skueomorphism, I think this is a fantastic idea. Lean heavy on a restful API that makes it easy to skin it ourselves and integrate it into bigger systems. Awesome nonetheless.

</comment><comment><USER6> isn't this the same idea running a webserver on devices such as a wifi router that let you control the device via a browser remotely, or is there something new here?

</comment><comment><USER7> What a great idea!</comment>


Assange Hearing Day 8 - <AUTHOR> https://www.craigmurray.org.uk/archives/2020/09/your-man-in-the-public-gallery-assange-hearing-day-8/ <comment><USER2> "Lewis asked why Timm had omitted Kromberg's reference to the grand jury decision? Timm replied that it meant very little: 99.9% of grand juries agree to return a prosecution. An academic study of 152,000 grand juries had revealed only 11 which had refused the request of a federal prosecutor to prosecute."

That's some statistics ...

<comment><USER3> It's almost worth asking what those 11 were. The prosecution must have really failed

</comment></comment><comment><USER4> Independent of the more important points of the article, this quote provided some amusement :)

> Lewis then asked Timm if he had seen the actual evidence that supports the > indictment. Timm replied only some of it, in particular the Java script of > the messages allegedly between Assange and Manning.

<comment><USER5> Clever of them to communicate in Java Script.

<comment><USER6> I'm imagining this github repo with rules like "Locations are the 3rd letter of private fields. People are first letter of public properties"

Just throw a few bits of data into a large codeset

<comment><USER5> I'm sure if you start analyzing everything on github using those rules a lot of real names are going to turn up.

</comment></comment></comment></comment><comment><USER7> It strongly seems to me that the British establishment is incredibly pissed at Assange for the trouble he caused by holing up in the embassy, and is therefore making life as difficult for him as possible.

<comment><USER8> He made us waste so much money, lets waste even more money.

If that were the reason for this whole thing than its kindergartner level of spite.

<comment><USER9> ... and?

I've seen similar, in all walks of life. Politicians and civil servants are by no means immune to pettiness. Especially when setting an example might spare them work and money in the future.

<comment><USER10> Civil servants in the UK are renowned for being inherently lazy, incompetent and devious.

Its no wonder when you have to interact with politicians on a day to day basis that the cultures should mirror each other.

I would much prefer Microsoft Civil Servant sp4 or an IBM Civil Servant AI powered by Big Blue.

<comment><USER11> "Civil servants in the UK are renowned for being inherently lazy, incompetent and devious."

Depressing to see such low-quality nonsense on HackerNews.

<comment><USER10> Likewise with your comment mate

[https://www.clivebates.com/documents/lazycivilservant.pdf](https://www.clivebates.com/documents/lazycivilservant.pdf)

<comment><USER12> I'm not sure that an amusing leaflet, ostensibly about civil servants but equally humorous to anyone in a large private enterprise, refutes the parent's point.

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> Thing I don't get about this case: the UK will refuse to extradite Assange because he will be tortured if they do that (ironically for exposing proof that the US tortures prisoners).

And then the UK proceeded to torture him themselves (torture are the words "The Lancet" used [1], they cannot reasonably be dismissed as cranks) ... Illegally, by UK laws, and refuse to be held accountable for that in their own courts. Not that the judge even tries to demand reasonable treatment for Julian Assange from the prosecutor.

UK laws are very clear on what should happen in this case: because of the government's treatment of Assange the case should be thrown out, because obviously the government has acted illegally against him on MANY occasions, they don't even give him his rights INSIDE THE COURTROOM (to talk to his lawyer privately and however much he wants without interference). The government DOES NOT have the right to sue somebody for anything once they have tortured him. Once they have lied about the case, they lose the right to sue. Once they refuse medical aid to a prisoner, they lose the right to sue. They have spied on his communication with his lawyer, again grounds for dismissal. They are obviously not feeding him right, which again would normally disqualify the government's case. Or at the very least, warrants a delay in the case with him decently taken care of until he is once again strong and fit enough to stand trial. Note that this is completely independent of the merit of the case itself: the QC in any normal case like this would never be allowed to make their case without fixing these issues first. Some of these reasons for dismissal the judge has seen happen right in front of him with his own eyes.

Then there is the fact that the UK government does not have evidence against Assange on this crime (because he violated US law, not UK law), furthermore I do not see how it could -legally- obtain any such evidence.

And yet this farce continues. The judge systematically refuses to face issues of fairness of the case and is obviously intimidated by the government.

This shows one thing very clearly : the United Kingdom's government (the "executive") has ZERO intention of following through on their commitment to either follow their own laws, respect human rights and has no qualms whatsoever to threaten their own branches of government, when they want to violate laws.

[1] [https://www.thelancet.com/journals/lancet/article/PIIS0140-6...](https://www.thelancet.com/journals/lancet/article/PIIS0140-6736\(20\)30383-4/fulltext)

</comment><comment><USER14> The surreal aspect of this is the UK prosecution's assertions that the US prosecution is not politically motivated. All you have to do is look at the speech that CIA Director (now Secretary of State) Mike Pompeo made about Wikileaks and Assange in April 2017.[1] These passages give a taste of his remarks:

> It is time to call out WikiLeaks for what it really is – a non-state hostile > intelligence service often abetted by state actors like Russia.

> No, I am quite confident that had Assange been around in the 1930s and 40s > and 50s, he would have found himself on the wrong side of history.

> We know this because Assange and his ilk make common cause with dictators > today.

> That Assange is the darling of terrorists is nothing short of reprehensible.

> No, Julian Assange and his kind are not the slightest bit interested in > improving civil liberties or enhancing personal freedom. They have pretended > that America's First Amendment freedoms shield them from justice. They may > have believed that, but they are wrong.

> Assange is a narcissist who has created nothing of value. He relies on the > dirty work of others to make himself famous. He is a fraud—a coward hiding > behind a screen.

> First, it is high time we called out those who grant a platform to these > leakers and so-called transparency activists. We know the danger that > Assange and his not-so-merry band of brothers pose to democracies around the > world. Ignorance or misplaced idealism is no longer an acceptable excuse for > lionizing these demons.

> Third, we have to recognize that we can no longer allow Assange and his > colleagues the latitude to use free speech values against us. To give them > the space to crush us with misappropriated secrets is a perversion of what > our great Constitution stands for. It ends now.

But if you listen to the UK prosecutor, Assange is just being prosecuted because some low-level prosecutor in the US decided, on their own initiative, to build a case against him. The fact that high-level officials in the US government have been openly calling for an example to be made of Assange is supposed to be irrelevant.

The UK should have dismissed this extradition request out-of-hand when it was first made, given the obvious political motivations behind it. The US-UK extradition treaty makes it very clear that extradition is not allowed in such cases.

1\. [https://www.cia.gov/news-information/speeches- testimony/2017...](https://www.cia.gov/news-information/speeches- testimony/2017-speeches-testimony/pompeo-delivers-remarks-at-csis.html)

<comment><USER15> I think in this case politically motivated would mean something more like "Assange is a journalist sympathetic towards Biden so we're trying to get him for tax fraud".

In this case it's "Assange is working for Russian Intelligence against the interests of the US let's get him for a specific crime he committed towards that end".

You can of course disagree with the Russian asset part of the statement, but I don't think that qualifies as "politically motivated" in the kind of sense it would (or should) bother an extradition hearing.

<comment><USER14> He published material that _embarrassed_ the US government - information like the US government spying on the UN, American helicopter pilots killing Reuters journalists in Baghdad (and the US military then covering it up), and the US tolerating and working with Shiite death squads in Iraq.

When governments go after people for publishing embarrassing truths, that's political. Everyone knows why the US government wants Assange behind bars. The CIA Director / Secretary of State made it abundantly clear in his speech that the US intends to go after Assange in order to send a message that the 1st Amendment is no barrier to prosecuting journalists for what they publish. The UK prosecutor is playing dumb.

</comment></comment></comment><comment><USER16> Odd characterizations.

"At this point he was grinning very strangely indeed, looking up at the judge, leaning back with one arm wide across his chair back, in some sort of peculiar alpha male gesture."

Similar to a roald dahl book.

<comment><USER17> Murray is a 'colourful character'. He thinks the Skripal poisoning was a false flag executed by the UK government. He produced an string of soon-debunked conspiracy theories about it, but has stuck with it nonetheless.

<comment><USER18> Murray did _not_ state that Skripal was a false-flag by the UK government. He did ask several questions about the nature of the official UK government's explanation which you find at [1].

Can you point me towards _credible_ answers to some / all the questions raised in [1] please?

[1] [https://www.craigmurray.org.uk/archives/2019/03/pure-ten- poi...](https://www.craigmurray.org.uk/archives/2019/03/pure-ten-points-i- just-cant-believe-about-the-official-skripal-narrative/)

<comment><USER19> > I had never claimed the poisoning, if any, was not carried out by Russians, > only that there were many other possibilities

 _if any_ being the important part. My mind is open to the point that my brain is on the desk?

I don't doubt for one second that the intelligence services have cleaned up the story a little but it just looks like conjecture e.g. Craig Murray knows nothing about the roof of the house.

His doubts as to their identities look really convenient, given that "someone" tried to give him DNC emails in the past.

Edit: here is a bellingcat vs. murray catfight over the identities of the Russians

[https://mobile.twitter.com/eliothiggins/status/1047597999105...](https://mobile.twitter.com/eliothiggins/status/1047597999105462272?lang=en)

</comment></comment></comment></comment><comment><USER20> While I appreciate the reporting from inside the court room, Craig Murray is clearly biased and thus predisposed to seeing what he wants to see.

His characterizations paint the prosecution as ridiculous characters and their case as laughable at best, which may not reflect reality as much one would hope.

So what he is describing might be very unlike what the judges are experiencing, and if they end up giving the extradition of Assange's the go- ahead, a lot of people are going to be very surprised.

Edit: This rollercoaster of upvotes and downvotes on a post that just reminds people to not blindly trust someone who tells them what they want to hear is interesting in itself.

<comment><USER21> I'm pretty sure people who read Murray/support Assange believe his extradition is a foregone conclusion.

<comment><USER22> I've been reading Murray since about 2005. He's always interesting and it would be a mistake to dismiss him on one opinion. I lose count of the number of big stories he's broken, or found himself in the middle of. He wasn't as radical in 2005 as he's become. His book on Alexander Burnes is serious and fascinating history. If you google his site for "Guardian" or "Rusbridger", you can follow the evolution that has led me, and perhaps readers of his site, to have a great deal more confidence in what we read there than in, scary quotes, the main-stream media.</comment></comment></comment>


Show HN: Lychee — Self-hosted photo-management done right - <AUTHOR> http://lychee.electerious.com <comment><AUTHOR> I developed this project in my spare time. It's a self-hosted photo- management-system and a great place to share, store and manage your photos and inspiration. I hope you guys here on HN like it. If you have any questions or feedback, just write it here :) Thanks!

<comment><USER2> Private albums with a random access token in the url would be awesome (instead of passwords).

On a side note, I'd donate 100 bucks for a Lightroom Sync Plugin any minute.

<comment><USER3> I second that - I have been looking for something like this, but it would be extremenly conventient to be able to upload directly from LR.

Another request - I love the free flow, tiled display of images a la Flickr photostream - any chance that could be an option?

Otherwise, great job! - I was pleasently surprised at the quality of this applications.

</comment></comment></comment><comment><USER4> For years and years, I used Gallery/Gallery2 to self-host photos, but more and more, I found myself taking photos with my phone instead of a regular camera. From the smartphone, it was much easier to upload to Facebook (and now, G+) than it was to get photos into Gallery. Once every so often, I would bother connecting my phone to my computer, sync the photos over, and then upload them into Gallery. But by then, they were already shared among my friends and family.

My gallery is still up, but the last time I updated it was apparently in 2011. There was a android client that came out that I had tried, but it didn't really work that well. I still backup photos from my phone to my computer and NAS, but couldn't be motivated to go beyond that.

I've looked at times to improve this situation. I much prefer to have my photos self-hosted. I haven't really found a gallery type program that worked as well as Menalto Gallery, but Lychee looks like it's a good contender. I also like that it can share out to other services.

At first I was thinking that since this doesn't have an android client, it would fall into the same disuse as my Gallery installation. But I noticed that it could read from an uploads folder and you could essentially ftp upload to the server. This made me wonder if one could just have an auto-upload (scp/sftp or otherwise) to a server.

I found a couple apps that do this, but [Auto- Share]([https://play.google.com/store/apps/details?id=com.dngames.au...](https://play.google.com/store/apps/details?id=com.dngames.autoshare)) looks the most polished. So I can see this working out.

tl;dr Looks like a great photo manager, love the option to share out to other service. There may be some Android Apps to automatically upload, even if it's not natively supported in Lychee.

<comment><USER4> Ok, so I installed it and added some pictures into uploads/import. Lychee imported these and dropped them into a smart album called Unsorted. I then wanted to organize these into albums. The first thing I noticed was there was no way to select photos. You have to open them one by one and move them. This would be my feature request numero uno.

Second thing I noticed is there doesn't seem to be support for sub-albums. I guess this isn't necessarily a deal killer, but I had a system of organizing albums in YEAR/MM type albums. If there was a lot of photos or an event, that would be a sub-album within the month it took place. If a single event took place between months (New Year's Eve party) I just handled that on a case by case basis.

<comment><AUTHOR> Tip: You can right-click on single images/albums to rename, move or star them. By now, it may be easier to upload photos directly into the album of your choice.

Multi-select is planned.

</comment></comment></comment><comment><AUTHOR> Our server gets hammered by you guys. Here is the direct link to GitHub: [https://github.com/electerious/Lychee](https://github.com/electerious/Lychee)

<comment><USER5> Hi, great tool guys! Lovely. During configuration I keep getting an error during the creation of the database connection to mysql: "Connection failed. Unable to connect to host database because access was denied. Double check host username password to ensure that access is permitted"

Can't I write whatever username and password I want? Should I create the database before hand?

<comment><AUTHOR> This could help. Here is how the php/config.php should look like: [http://cl.ly/TVft](http://cl.ly/TVft)

</comment></comment></comment><comment><USER6> > "Change the permissions of uploads/ and php/ to 777, including all > subfolders"

> > [https://github.com/electerious/Lychee/blob/master/docs/md/In...](https://github.com/electerious/Lychee/blob/master/docs/md/In..). > I don't think this is ok

<comment><USER7> That bothers me. That should never be necessary.

It looks like they intend everything to be in the webroot, which is a problem in and of itself. Setting everything in the uploads folder to be executable without any .htaccess directives to prevent that seems like a potential issue. If they're not validating images (properly) or sandboxing uploads, or thinking about mitigating directory traversal attacks, then there could be issues with remote code execution.

</comment></comment><comment><USER8> It would be so great if it could simple serve files from a directory (without in-app upload). Then I could simply sync the photos using btsync (or anything else).

<comment><USER9> That's what I do for my own photos. [http://chir.ag/gallery/](http://chir.ag/gallery/) just shows everything from a folder on the server. The thumbnails shown in the gallery view are random and auto-generated. I use subfolders to caption/group the photo in large albums: [http://chir.ag/gallery/alaska- cruise-2013](http://chir.ag/gallery/alaska-cruise-2013)

The folder names are underscore delimited column data. E.g. "alaska- cruise-2013_2013-07-27_5th Anniversary Alaska Cruise_critters,cruise,hiking,juliet,travel" => "alaska-cruise-2013" is the permalink, 2013-07-27 is the start date, "5th Anniversary Alaska Cruise" is the gallery title, and the rest of the comma-separated words are tags for filtering, e.g. [http://chir.ag/gallery/tag/cruise](http://chir.ag/gallery/tag/cruise). I have a few local scripts to resize the raw photos and rsync everything with the server.

It's really not complicated code. Pretty simple stuff that anyone with half a year of coding experience can build. I have been taking and sharing photos on my 'blog since late 2001. I have tried many, many things over time and finally settled on the current method. While my gallery looks/works very simply, it has taken years to refine this exact strategy. I LOVE the Boston Big Picture view and always wanted that. But I also like the thumbnails to jump. I love clicking on the left/right of a pic to advance but I also like to scroll up/down if needed. Sorting gallery by name vs. date, filtering similar albums by tags, and of course captioning without using a text file or DB were things I always wanted but didn't want to deal with a complicated system. I re-did my gallery after our Alaska trip and have been very happy ever since.

</comment></comment><comment><USER10> This is _so_ smooth. Wow. It's been a while since I saw something labelled as "stunning" actually stun me.

The ability to tag photos and a commenting system would make it just perfect. Maybe an RSS feed for photos and comments, too?

<comment><AUTHOR> Tags and RSS feeds for albums are already planned.

</comment></comment><comment><USER11> I like it very much; both the code and interface seem clean and lightweight.

You should use mysqli prepared statements though, after a quick glance over the code I can't quite shake the feeling that there might be some sql injections lurking.

<comment><AUTHOR> All calls to the API are going through the api.php where every $_POST gets escaped. I'm not an expert when it comes to SQL injection, but I think this should prevent it. See line 61 in php/api.php. It would be great to know if this actually helps or not.

<comment><USER11> Having read your code and quickly brushing up the little bits of PHP knowledge from years ago, I'd say it helps only a little. mysqli_real_escape_string alone doesn't protect you completely from sql injection attacks, it just makes it harder - see the second answer at [http://stackoverflow.com/questions/5741187/sql-injection- tha...](http://stackoverflow.com/questions/5741187/sql-injection-that-gets- around-mysql-real-escape-string) for an example. Also, the homegrown $albumID validation allows bad payload to reach mysqli_real_escape_string().

<comment><AUTHOR> Thanks! Good to know.

</comment></comment></comment></comment><comment><USER12> Dude, you could've easily made money out it. I for one would've definitely bought it. Your product is superb, simple and gives us capability to add stuff on top of it like authentication

<comment><AUTHOR> Thanks! Donations are welcome :)

<comment><USER13> No - you didn't read what (s)he wrote. You should charge. Charging people real money gets you real users who give you real feedback. It allows you to allocate more money to the development of the project.

Providing this tool for free is a cost to you.

Sure, allow a certain number of photos free of charge and charge for more.

<comment><USER14> I also agree. I want to pay for this, not donate. This can be very important for my personal needs and I want to rely on it. If I donate, I don't get the feeling that the project is moving forward.

<comment><USER15> Simple solution, they provide a package that is the program and a friendly service contract which provide help setting it up or adding customization.

<comment><USER16> OK, seems everybody wants to pay per copy, am I rite? I think that is a pretty bad idea that will slow down adoption and development. Yes, this is an open- source project and more developers could help out. I think developers are less likely to jump on the wagon to help out a duo that gets all the money. Instead, don't try to monetize this now. Make adoption and interest grow. Then sell a hosted variant at lychee.io, Wordpress stylee. Or be more imaginative than that and come up with a better idea.

Great software anyways! Thanks!

<comment><USER13> Make something of value -> now sell it.

Why be more imaginative? It sounds a pretty simple offering to me, and is highly transparent.

A hosted variant isn't a bad idea (I was going to ask if there were recipes for EC2, Docker etc... some people might want to "self" host on a machine they control) but it's more "value added" than the original codebase.

</comment></comment></comment></comment></comment></comment></comment><comment><USER17> This is beautiful. AFAIK, the alternatives are either a self-hosted blog (not ideal, strips out EXIF data etc.) or something hideous like Piwigo.

Awesome!

<comment><USER18> Koken might be an other alternative. [http://koken.me](http://koken.me)

<comment><USER16> I'm running Koken right now and I don't think it's a Lychee alternative. Koken seems geared towards users that are building a photographic or design portfolio with a limited selection of images (and that's what I'm doing). I think Lychee is something else. A way to store and display photos en masse. Perhaps a way to share all your photos from your family vacations 2005-2013.

</comment></comment></comment><comment><USER19> Nice. A few questions (I might add more, just using the demo)

1\. How does it deal with mixed landscape and portrait images?

2\. Is preloading the next image possible?

3\. Scrollbars are appearing, I can't work out why, e.g. on [http://electerious.com/lychee_demo/#3/13901681813613](http://electerious.com/lychee_demo/#3/13901681813613)

<comment><AUTHOR> 1\. If the image is to tall/wide it will be scaled to fit into your browser. This happens via CSS. Photos won't be compressed or scaled, Lychee keeps their original quality and dimensions. You can click the 'Full Photo' button on the top right to see the file.

2\. Nope. Not at the moment.

3\. Does this happen only sometimes or always? Which browser and OS are you using?

<comment><USER14> I also have the same problem (3). It happens when using chrome 32 on windows 8.0 with full screen mode enabled. The scroll bars appear when I open an image and click the right arrow on my keyboard to switch to the next. For me it doesn't happen if I don't navigate to another image or if I'm not browsing in full-screen mode (f11).

<comment><AUTHOR> Thanks for the report! I will look into it.

</comment></comment></comment></comment><comment><USER20> I love it. I've been playing with ownCloud because I no longer want to store my pictures on Dropbox (I'm running out of space, and the first tier is 100GB which I don't need).

I'm going to toy around a bit with it and if I like the API I may hack together something to automatically upload new pictures from iOS devices :)

<comment><AUTHOR> This plugin lets you sync OwnCloud with Lychee: [https://github.com/GustavePate/lycheesync](https://github.com/GustavePate/lycheesync) It might not work with version 2.0, but the code may help you to start developing some great stuff.

</comment></comment><comment><USER21> I really like this! I've been wanting to work on a project where I can just insert a SD card into my home server, and it imports all the images into a locally hosted photo-management automatically. This looks like the perfect solution for that. Thanks for sharing!

</comment><comment><USER22> Animated transitions are really distracting, I hope there's a way to turn this off (and it should be turned off by default..)

</comment><comment><USER23> Thanks - I've been looking for something like this since Flickr went a bit mad. I'll install and play around.

Thanks for doing this!

</comment><comment><USER24> This may be exactly the photo sharing software I've always wanted. Will definitely be setting this up soon.

</comment><comment><USER25> Awesome. Certainly one of the most elegants solutions. Any plans to add video support?

<comment><AUTHOR> Not at the moment, sorry.

</comment></comment><comment><USER26> I can really see this as a frontend to my crashplan data. They do have a nice API :)

</comment><comment><USER27> good job! I like the simplistic and elegant interface. Like what other commenters said, it would be great if it can serve files directly instead of having to upload through the web app.

BTW, why did you name it Lychee?

<comment><AUTHOR> I think it's a great name for a product. Almost every name which has something to do with photos is already taken. And I like Lychees ;)

</comment></comment><comment><USER28> You don't need to blur the image when showing information about it.

</comment><comment><USER29> I'm looking at something like this but with storage in S3

</comment><comment><USER30> This looks fantastic, very well done.

</comment><comment><USER31> Wow, it is incredibly good looking!</comment>


At 94, Carmen Herrera Is Art's Hot New Thing, and Enjoying It - <AUTHOR> http://www.nytimes.com/2009/12/20/arts/design/20herrera.html?hp <comment><USER2> Nothing against the nice old lady, but from what I saw of her work in the slide show, to somehow pass this off as high art is a tribute to fakery.

<comment><USER3> Nothing against jackfoxy, but from what I saw of his comment, to somehow pass this off as art criticism is a tribute to trollery.</comment></comment>


Lavaboom launches crowdfunding campaign - <AUTHOR> https://www.indiegogo.com/projects/lavaboom

<comment><AUTHOR> Lavaboom CTO here, I'm happy to take questions during the next couple of hours.

</comment><comment><USER2> What is Lavaboom?

<comment><AUTHOR> We're a secure zero-knowledge encrypted email service, focused on privacy and great UI/UX.</comment></comment>


Send Mail using SMTP and PHP. - <AUTHOR> http://9lessons.blogspot.com/2009/10/send-mail-using-smtp-and-php.html

<comment><USER2> There are a multitude of very well established projects making the sending of email trivial in PHP. This feels like re-inventing the wheel for the sake of a blog post.

Also, for me personally, I think you're going too far over towards self- promotion. There is nothing wrong with self-promotion but save it for your best articles rather than submitting every article you put together.

</comment><comment><USER3> I like how the sending code totally ignores any response from the SMTP server. You have no idea if it even accepts EHLO, and you have no idea if your message is queued successfully.

I also hope you don't put data from the user into this class. If your email address contains ">\r\n" and some SMTP commands, your PHP script just became an open relay.</comment>