Scaling the Facebook data warehouse to 300 PB - <AUTHOR> https://code.facebook.com/posts/229861827208629/scaling-the-facebook-data-warehouse-to-300-pb/ <comment><USER2> Based on my current connections, and experience, I can't fully see myself ever getting the chance to do so, but I'd love to have the opportunity to work on similar problems - at Facebook or anywhere. I've always loved working on lower level problems such as this. I learned to program by writing computer game hacks, reverse engineering games and coding in C / ASM. Currently, I find myself writing C# every day for a small company at which nobody understands a single word I say about programming.

<comment><USER3> Yeah, it's interesting how Facebook does something which on its face seems trivial and unimportant, but due to scale, has some really amazing engineering and infrastructure challenges -- and they solve them by building tech to make things scale from commodity systems (like Google, etc have done), vs. buying third-party big iron solutions (which is what eBay did when they had similar challenges, and what most big companies do).

<comment><USER4> Do you think its a cost/control to not buy big iron (outsource to a degree) vs build/scale themselves? I'd like to hear some HN thoughts if some don't mind sharing.

<comment><USER5> "Big iron" isn't all it's cracked up to be. Everything is a trade-off. Very few people are doing pure computation and that is where those machines excel (in addition to lots of aggregate I/O). The government research labs and the like get a lot of use from these machines.

If you trying to scale an Internet-style app on one of these machines, you might need to expand past one machine after a while. By staying on one machine, you're avoiding all the complexity needed in your software to coordinate between multiple machines. If you lose the ability to fit on a single box, you'll need to add that complexity in anyway. So what does 10 beefy boxes buy you as opposed to 1000 smaller ones? There is of course an operational/DC/power cost involved with more boxes, but I think most shops consider that an easily solvable problem. For example, a maxxed out POWER7 box from IBM will give you 256 processors and all the memory and I/O trimmings you need. If you need more than 256 processors or the local amount of RAM, you'll pay the software complexity cost anyway.

<comment><USER6> Well, the 10 beefy boxes will be much, much faster if your problem is not very distributable. Say, Facebook as an application shards very easily, because most users don't interact much with each other. Other applications, might have much more interactions.

What you're really paying for when buying a 256 processor POWER7 box is the fact that the interconnect (and therefore the time to acquire a lock/update data from another node) is much faster and more reliable than commodity networks/kernels/stack.

<comment><USER7> Depends on what you are programming on. If its in a language far removed from the machine your mileage may vary.

I have had the opportunity to try out Google's implementation of mapreduce implemented in C++ way back in time (6 years ago). These would run on fairly impoverished processors, essentially laptop grade. Have done stuff on Yahoo's Hadoop setup as well, these used high end multicore machines provisioned with oodles of RAM (I dont think I should share more than that). If I were to be generous, Hadoop ran 4 times slower as measured by wall clock times. Not only that, Hadoop required about 4 times more memory for similar sized jobs. So you ended up requiring more RAM, running for longer and potentially burning more electricity. This is by no means a benchmark or anything like that, just an anecdote.

That Hadoop would require much more memory did not surprise me, that was expected. What was really surprising was that it was so much slower. JVM is one of the most well optimized virtual machines we have out there, but its view of the processor is very antiquated and it does not surface those hardware level advances to the programmer. You pay for a hot-rod machine but run it like an old faithful crown victoria.

Four times might not seem like much, for one thing I am being generous, and it makes a big difference when you can make multiple run through the data in a single day and make changes to the code/model. Debugging and ironing out issues is a lot more efficient.

I think Hadoop gave Google a significant competitive advantage over the rest, probably still does.

</comment></comment></comment></comment></comment></comment><comment><USER8> When they introduced ORCFile, I was kind of hoping that the next iteration would be the URUKFile.

<comment><USER9> Or HUMANFile.

</comment></comment><comment><USER10> I'm actually wondering, are their any ways to play with data like this (eg. downloading data from StackExchange [http://blog.stackexchange.com/category/cc-wiki- dump/](http://blog.stackexchange.com/category/cc-wiki-dump/) )

Any other ways? I don't believe there is a VM for this sort of "experimentation"

<comment><USER11> [http://aws.amazon.com/datasets](http://aws.amazon.com/datasets)

</comment></comment><comment><USER12> A lot of fancy tech for only 805MB per user (based on estimate of 500 million users). What if we had a pure p2p web app using webrtc & local storage to replace Facebook? Think everyone could spare 805MB?

Calculation: [https://www.google.com/search?q=(300PB)%2F(400000000)](https://www.google.com/search?q=\(300PB\)%2F\(400000000\))

<comment><USER13> You're a bit behind, it's 1.23 billion monthly active users or if you feel that's a bit disingenuous: 757 million daily active users.

Source: [http://newsroom.fb.com/company-info/](http://newsroom.fb.com/company- info/)

PS. you calculated using 400 million.

<comment><USER14> 300 pb / 10^9 ~ 300 10^15B/10^9 ~ 300 10^6 B ~ 300MB/user.

Considering my latest fb backup was ~18MB (unzipped), out of which 14MB was pictures, this doesn't sound too unreasonable to me. If anything, it sounds very conservative. If I was actively using fb for photos, I'd easily have at least 100 times as many, maybe 200 times as many. Not to mention that the single (short) video I've uploaded is 1.4 MB.

</comment></comment></comment><comment><USER15> I wonder, how does vertica link to this. ([http://www.vertica.com/2013/12/12/welcoming-facebook-to- the-...](http://www.vertica.com/2013/12/12/welcoming-facebook-to-the-growing- family-of-hp-vertica-customers))

</comment><comment><USER16> Maybe they should just delete something once in a while. How about actually deleting data for users that close their account?

</comment><comment><USER17> Where's the power coming from to scale like that? If Facebook is relying on power from the Columbia River for their data centers, they better talk to the tribes and the salmon. Those dams are failing and many parties in the Northwest want to see them removed.

A better bet might be for them to scale up in Utah and harvest methane from waste to generate power.

<comment><USER18> Because Utah has a lot of methane? I'm not quite following. I'd think methane production would be greater in a cow state like Iowa or Ohio than in a desert state like Utah? _joke_

Utah's power grid is primarily coal fired, but they use a fair bit of natural (methane) gas as well. [1] Utah has a couple of major hydroelectric dams (glen canyon, flaming gorge), and the use of solar and wind are on the rise.

NSA is building a major data warehouse in Utah; one of the considerations would have to have been cheap power. I'm guessing the Columbia river produces cheaper electricity than pretty much anywhere else, but Utah has very diverse (and affordable) power production overall.

[1] [http://www.deseretnews.com/article/700051087/Coal-mostly- pow...](http://www.deseretnews.com/article/700051087/Coal-mostly-powers- electricity-hungry-Utah.html?pg=all)

[edit] failed on humor the first time, trying again.

</comment></comment><comment><USER19> The things facebook does to keep people's shower thoughts readily available.

<comment><USER20> _...not to mention our uber-lords at the NSA..._ (yawn)

Actually all tech can be criticised for banality. Think of the telephone - _' they put in all those cables just so she can talk to mother...'_. Or television - _' they dug up teh street just so they could lay those fibre optic cables so your gran could watch the wresting...'_. Or even the trains - the train stopping at my home town does seem a waste of time, I can't believe they bother when you look at who gets on.

<comment><USER21> It's a fallacy that all technology can be criticized for banality. Technologies are different. Some are more criticizable than others.

Telephone was the first infrastructure to provide real-time voice communication. It enables families staying in contact, but it also enables economic growth, and a more effective society writ large.

Television is now a mindless wastefield of race-to-the-bottom drivel, but there are newer networks have haven't yet succumbed to drive, mostly on digital cable. I only have the respect for science and technology that I do because I grew up watching _The Magic School Bus_ and _Bill Nye the Science Guy_. My parents watched the moon landing on television.

Facebook does not do anything novel, nor has it ever been used for anything terrifically insightful. It provides some social value and exists for that reason, but it is clearly not equivalent to all other technologies.

<comment><USER22> I don't quite get it. Being able to transmit images is "novel" but being able to query more information that has ever been available before in the history of the human race is not?

You should really separate the _application_ of the technology from the technology itself. Your last sentence can be said the exact same way about the telephone "It provides some social value and exists for that reason" but that is clearly a ridiculous statement to make about the telephone.

</comment></comment></comment></comment><comment><USER23> And the moat keeps getting larger. Good for them. Yes, they are kind enough to open source this, but how advanced is the tech that they don't? And how much time have they enjoyed as the first to benefit from this impressive tech? Again, good for them.</comment>


Turning on Project Loon in Puerto Rico - <AUTHOR> https://blog.x.company/turning-on-project-loon-in-puerto-rico-f3aa41ad2d7f <comment><USER2> When I did my high altitude balloon I was struck by the changing wind directions at different altitudes. In this blog there's a picture of the actual drifting flight path: [http://blog.jgc.org/2011/04/gaga-1-flight.html](http://blog.jgc.org/2011/04/gaga-1-flight.html)

This flight path was pretty closely predicted 48 hours before using NOAA predicted wind data (see this blog: [http://blog.jgc.org/2011/04/gaga-1-looking-good-for- sunday.h...](http://blog.jgc.org/2011/04/gaga-1-looking-good-for- sunday.html)). The difference in size between the two 'loops' is caused by a slow ascent and rapid descent.

If I'd had altitude control (like Loon) and uploaded data on wind speed it would have been possible to make the balloon hang around a target area.

<comment><USER3> That's interesting, it seemed like the most challenging problem to solve given the balloons are always moving. They mention using machine learning to perfect the coverage:

> This is the first time we have used our new machine learning powered > algorithms to keep balloons clustered over Puerto Rico, so we're still > learning how best to do this. As we get more familiar with the constantly > shifting winds in this region, we hope to keep the balloons over areas where > connectivity is needed for as long as possible.

</comment></comment><comment><USER4> Check out the balloon flight paths by clicking on them [https://www.flightradar24.com/18.22,-66.59/8](https://www.flightradar24.com/18.22,-66.59/8)

<comment><USER5> That's amazing! Check out the difference in altitude compared to the commercial jets flying near-by:

Boeing 767-316 - Alt: 33,000 ft

Balloon - Alt: 61,900 ft

<comment><USER6> In meters:

Boeing 767-316 - Alt: 10058 m

Balloon - Alt: 18867 m

<comment><USER7> Aeronautical measures are generally done in feet, even outside the U.S.

<comment><USER8> No, it's not true.

<comment><USER9> Sorry, the only countries that specify altitude in metres for aviation purposes are Russia, China and North Korea-- and Russia is switching to feet this year: [http://aerosavvy.com/metric- imperial/](http://aerosavvy.com/metric-imperial/) [https://en.wikipedia.org/wiki/Flight_level](https://en.wikipedia.org/wiki/Flight_level)

</comment></comment></comment></comment></comment></comment><comment><USER5> Definitely the most interesting part of this project in my opinion.

> This is the first time we have used our new machine learning powered > algorithms to keep balloons clustered over Puerto Rico, so we're still > learning how best to do this. As we get more familiar with the constantly > shifting winds in this region, we hope to keep the balloons over areas where > connectivity is needed for as long as possible.

[https://blog.x.company/how-project-loons-smart-software- lear...](https://blog.x.company/how-project-loons-smart-software-learned-to- sail-the-winds-ec904e6d08c)

Apparently, back in 2016 they launched the balloons from Puerto Pico to do their first test of the autopilot system in Peru. Now they are flying the balloons from Nevada to implement it in Puerto Rico.

> we had figured out how to cluster balloons in teams, dancing in small loops > on the stratospheric winds, over a particular region.

<comment><USER10> This is insane. They're using ML to predict wind speeds at different altitudes and how to move the balloons with their various speed and power restrictions to optimize on-the-ground connectivity while maintaining close enough clusters and decent backhaul. Incredible.

<comment><USER11> To be clear, it's not so much they are using ML to predict wind speeds at different altitudes; they use a forecast model like everyone else for that. The clever bit is being able to control your altitude such that you find favorable wind-altitude tuples to be in to keep you generally in one spot over days/weeks. It's sort of like what ocean/circumnavigating sailors do but in a higher dimensional space, both literally and in your search algorithm. Fascinating stuff.

<comment><USER12> The loons don't need to stay in one spot - for example, two loons could rotate around a common center point and provide good service. Allowing the loons to wander while optimizing ground connectivity is harder computationally but makes it possible to maintain good connections even if the wind isn't entirely cooperating.

</comment></comment></comment></comment><comment><USER13> That is pretty awesome. I didn't realize the scale and complexity of what they were doing. When I first heard reference to this, I for some reason assumed that these were simple tethered balloons, not stratospheric swarms of AI balloons!

</comment><comment><USER14> "The 100G program is exploring high-order modulation and spatial multiplexing techniques to achieve the 100 Gb/s capacity at ranges of 200 km air-to-air and 100 km air-to-ground from a high-altitude (e.g. 60,000 ft.) aerial platform. The 100G program is developing the technologies and system concepts to project fiber optic class 100 Gb/s capacity via airborne data links anywhere within the area of responsibility (AOR).

Modern expeditionary military missions generate and exchange massive amounts of data that are used to produce situational awareness and guide decision- making. Much of the data must travel long distances along backbone communications networks composed of high-capacity links that interconnect command centers. While optical fiber services can provide a multi-gigabit data backbone in many parts of the world, modern expeditionary forces require a similar capability in places where fiber access does not exist. SATCOM services can provide some capacity to remote areas, but cannot provide the capacity needed to support the amount of data generated by emerging ISR systems." [https://www.darpa.mil/program/100-gb-s-rf- backbone](https://www.darpa.mil/program/100-gb-s-rf-backbone)

</comment><comment><USER15> Haven't seen Loon in the headlines for a while. This is really cool. In hindsight, disaster relief seems like such an obvious application for a technology like Loon.

Google could catch some serious tax breaks as well as goodwill if they worked out deals for providing impromptu connectivity services in disaster zones.

<comment><USER16> Yea, loon was unheard for a while, together with tesla both can get Puerto Rico back online ASAP.

<comment><USER17> Is SolarCity considered a quicker replacement to the damaged power grid and supply?

That a truck can carry far more energy in the form of petroleum than it can in the form of batteries is one of the arguments against _wasting_ fossil fuels on non-extreme conditions, like regular transportation and non-emergency electric power generation.

For Puerto Rico right now cheap, even if relatively inefficient, generators and lots of fuel seems like a good way to help in the very short term. Military surplus diesel generators, for example, are really cheap for how much power they can output with enough fuel.

<comment><USER18> They aren't directly comparable. You ship a truckload of PV and once installed it keeps providing daytime power without further delivery; you ship a battery to somwhere with PV and that power can now be made in the day and used at night, every day/night for a few years.

Even without batteries, shipping PV will at least keep the freezers cold (and food fresh).

</comment></comment></comment></comment><comment><USER19> AFAIK, many other big-name companies have been donating manpower and resources to help get PR back on its feet. What makes this announcement any more noteworthy? I'd get it if it included a bit more information, but the blog post reads as marketing fluff.

Does anyone know of any reliable sources showing in-depth sample numbers from previous deployments? I'd be really interested in getting an idea of what area size these balloons could cover, for how long it can be reliably maintained over an area, and their expected throughput. The Helping out in Peru [0] post has a bit of info, but it's very light on details.

If anyone is familiarized with these industries, I'd love to read up on the kinds of challenges faced with getting something like this deployed. Looking at it from an outsider's perspective, being without connectivity for over a month seems like a pretty long time. It's great that Project Loon is seeing improvements in their deployment time, but the article doesn't shed any light on the process, which is rather disappointing.

[0] [https://blog.x.company/helping-out-in- peru-9e5a84839fd2](https://blog.x.company/helping-out-in-peru-9e5a84839fd2)

<comment><USER20> It's newsworthy because an experimental Google X project is being used to solve a real world problem.

The US military might spend a billion dollars in aid, for example, but the world won't be any different in the future, beyond the lives directly impacted, because of this.

</comment></comment><comment><USER21> Im sure there are a few people who will find Loon's service useful.. but 80% are without power, so they're mostly not going to use internet anyway -- no power means cell phones cant be charged, etc. Some have generators, but are they able to get fuel? Emergency personnel cant even get fuel to deliver food and medicine.

Internet and cell service is being restored at a similar rate to power.. probably because these two depend on each other.

So it seems like this is only really useful to people traveling from one of the areas with power to an area without power (first responders, workers doing repairs, etc).

Still a good use case for Loon though.

<comment><USER22> You can easily charge a small device like a phone off solar panels, esp. in Puerto Rico a place known for sun.

<comment><USER21> good point.

I wonder how many people had portable solar panels before the hurricanes though. It's not like they could order it after the hurricane. And the solar panels on people's houses are probably damaged... two hurricanes would do that.

<comment><USER22> logistics of distributing small solar panels are almost certainly easier than generators + having to resupply them with fuel regularly.

<comment><USER21> sure.. but they have issues distributing food and medicine.. do you really think they're distributing a lot of solar panels?

<comment><USER22> If they decided they needed to get emergency power for radio communications to people and the choices were between generators w/ a supply of gas or portable solar charging kits I know which I would choose.

</comment></comment></comment></comment></comment></comment><comment><USER23> Hmm what kind of signal are they sending out? LTE/wifi?

<comment><USER24> It's a custom LTE base station.

</comment></comment><comment><USER25> Be nice if they could also airdrop some cheap already charged 2g phones.</comment>


Free porn videos to hit Xbox 360 following Internet Explorer update - <AUTHOR> http://www.trustedreviews.com/news/free-porn-videos-to-hit-xbox-360-following-internet-explorer-update

<comment><USER2> Summary: "It will be possible to browse the web on Xbox" "O noes, think of the children!"</comment>


Teardown and exploration of Apple's Magsafe connector - <AUTHOR> http://www.righto.com/2013/06/teardown-and-exploration-of-magsafe.html

<comment><USER2> So, the _charger_ has a microprocessor with twice the potential address space as the 6502 in the original Apple ][ .. Wow

<comment><AUTHOR> Yes, it's amazing the processing power that can be tossed into something like a charger these days. (The address space doesn't scale like that, though, and the 6502 had a 16-bit address bus.)

The charger microcontroller is in the TI MSP430 family; it's custom so I don't know it's exact properties. It probably has just 1 or 2K of flash and 256 bytes of RAM. But it's much faster, with a 16MHz processor, compared to 1MHz in the Apple II. This chip monitors the output voltage and current to handle power up and fault shutdown. It doesn't do the actual switching power supply control; there's a separate SMPS controller chip for that. And the 85W charger has another controller chip for power factor correction. It all seems a bit over-engineered to me.

</comment></comment><comment><USER3> Very cool! I wonder if everything holds about the same way for the new 'L-shaped' connectors?

<comment><AUTHOR> I read out the ID from an L connector and its basically the same. The circuit board must be slightly different since the wire comes in from the side but its the same 1-Wire device inside. I'm reluctant to sacrifice a charger to tear down the L connector just to see though.

<comment><USER3> Predictable enough, but who knows.</comment></comment></comment>


Show HN: Listory – discover, follow and read newsletters out of your inbox - <AUTHOR> https://www.listory.com/d <comment><AUTHOR> Still early stage, and any feedback will be appreciated! The goal is to extract from newsletters the most important and most recommended stories. Kinda like the Michelin Guide, but for content</comment>


MMO 2D Competitive Space Action Clone of Cosmic Rift - <AUTHOR>

I've created a clone of 'Cosmic Rift' called 'Astral Rift'. It is played in the browser and made with Javascript, NodeJS and JoyJS.
The codebase and the gameplay instructions are here https://github.com/esuen/AstralRift.
The game is playable here http://www.astralrift.com.
Please try it out and give me feedback. <comment><USER2> Some notes:

Add stars to the background for orientation.

Make the ships a lot easier to control.

I escaped the arena by moving to the bottom-left corner and holding down.

Make the bullets move faster.

Add instructions; just "shift = mine; z = shoot; etc."

I didn't understand the life system. I guess getting hit makes you lose energy, but so does shooting? I shot a guy a bunch of times but nothing seemed to happen. Maybe add more feedback when you hit somebody?

Right now it's not very fun.

<comment><AUTHOR> I'll be adding stars in the background soon.

Could you elaborate more on why the current controls are difficult? What could make it easier?

Yes, there is a couple bugs with escaping the arena. I may make the bullets faster.

Yeah, good point on adding instructions.

The life system is this: you're energy is both depleted when you are hit or use a weapon.

More special weapons and features added later will add to the enjoyment of this game.

<comment><USER2> I think more friction would make the controls easier. Right now it feels like you're sliding on ice.</comment></comment></comment>


Smog Chokes Delhi, Leaving Residents 'Cowering by Our Air Purifiers' - throwaway-hn123 http://www.nytimes.com/2016/11/08/world/asia/india-delhi-smog.html <comment><AUTHOR> previous discussion - [https://news.ycombinator.com/item?id=12886672](https://news.ycombinator.com/item?id=12886672)

and relevant info on the cause of the smog. Its not industrial pollution, but agricultural burning that's happening in neighboring states - [http://mobile.nytimes.com/2016/11/03/world/asia/farmers- unch...](http://mobile.nytimes.com/2016/11/03/world/asia/farmers-unchecked- crop-burning-fuels-indias-air-pollution.html)

[https://www.bloomberg.com/gadfly/amp/articles/2016-11-04/who...](https://www.bloomberg.com/gadfly/amp/articles/2016-11-04/who- s-choking-new-delhi-not-carmakers)

As last time, here's the problem to solve : what is a cheap, recyclable way to plant seeds without burning (last harvest's) hay ? The current solution of using a "seeder" costs 1900$ that is unaffordable by farmers. This was posted in November 2015 - [http://scroll.in/article/770176/why-we-should-be-alarmed- at-...](http://scroll.in/article/770176/why-we-should-be-alarmed-at-nasas- images-of-burning-fields-in-punjab-but-also-get-used-to-them)

<comment><USER2> Feed it to goats.

</comment></comment><comment><USER3> Aside from this specific event, do you have the feeling that we will be unable to stop catastrophic changes on earth due to pollution and resource exhaustion (overfishing etc)? Think about runaway greenhouse effects, floods, loss of biodiversity...

I think Elon Musk is right in his urgency regarding the Mars colonisation plans. Sure, you can always say "let's wait a couple of decades, with the new tech we'll have then it will be much easier and cheaper", but you could be mistaken. Technology and human civilization does not always move forward. We've been thrown back many times in the past and eventually there will be the day when we will not recover to previous levels: The beginning of the end. By then, we will no longer be able to tackle a huge challenge such as Mars colonization because we will be fighting for survival.

<comment><USER4> Realistically, whatever damages we could do to Earth's ecosystem, it will always be _much_ easier to fix (or cope with) things here than to create a livable ecosystem on Mars.

<comment><USER3> Earth is not just threatened by human caused problems. There could also be some other catastrophic event such as supervolcanoes, meteorites, killer virus. We may not have enough time to "fix" it. A second outpost drastically increases our chances of survival.

</comment></comment></comment><comment><USER5> People blame crop burning as the reason. Probably the farmers are polluting this place for 15 days, but the same farmers are absorbing our emissions with their farm plants for the remaining 350 days in the year. Nobody seems to mention it ever. The farmers also give us food to eat.

The farmers only need about 2000 dollars for a machine called "happy seeder" to stop burning. It is not a lot of money but the political class will only "subsidize" it, as long as the illiterate farmer fills out a big application form, takes a crowded train to the far-away capital city and submits to the right official along with some bribe.

Given those conditions, I really cannot blame the farmer, I only feel sorry for them.

By the way, I live in Delhi. I am suffering because of this pollution but I will not blame the farmers like media and other people of Delhi. Farmers are nice and simple people, they give us food to eat. Their farms absorb the emissions from my petrol car till I can afford to go electric.

</comment><comment><USER6> Interesting to remember that much the same thing happened in London not so long ago: [https://en.wikipedia.org/wiki/Great_Smog](https://en.wikipedia.org/wiki/Great_Smog)

So maybe there's hope Delhi will clean itself up in the near future.

<comment><USER7> London is still really bad, continually missing European Clean Air Quality targets and has been fined several times. The Great Smog was visible due to sulphur from burning coal etc. Whereas today, you cannot see the smog... but it is still very much there.

<comment><USER8> Yes, London is bad. Paris too. Especially with the reeking fumes near any major road from all the dirty diesel vehicles. Air quality in both cities is totally unacceptable in my opinion.

But it's nothing remotely like the scale of disaster that India has, where there is often constant choking smoke enveloping entire cities and regions. Even China is clean by comparison.

</comment></comment></comment><comment><USER9> Fueled by this mess in our capital me and a couple of friends spoke to a regional administrative officer for an anti pollution drive that will bring some sense in people. We were told to come back when the pollution is high enough. We're fucked in this country.

<comment><USER10> Who was it? Maybe we can shame them to action via twitter, etc. Or at least make them aware that this isn't cool

<comment><USER9> Twitter? I'm sorry this is rural India. No one knows or gives a shit about twitter. A more probable event is that police arresting me :D

But thanks for your concern we have contacted a couple of rich heads to use the situation to their advantage.

</comment></comment></comment><comment><USER11> Delhi badly needs a good solar transport story yesterday. There is a big market opportunity here, I hope someone grabs it.

<comment><USER12> Can't have solar power if you don't have sun due to smog. Besides, who is going to pay for it? It can't compete with cheap scooters.

<comment><USER11> This smog is temporary (because of the moronic cracker fun on Diwali). Delhi gets a lot of sunshine throughout.

Government is subsidising all things solar.

Also I meant electric vehicles, sorry mixed up both the things.

</comment></comment></comment><comment><USER13> Yet they still continue to use their motorbikes, without knowing the causality link between that and the pollution.

But yes, must be hard when "your actions are so small they don't count".

<comment><USER14> While your point is valid, a little compassion for those who suffer under this pollution, which is caused by a lot harder problems than a few million motorcycles.

<comment><USER15> A few million motorcycles is definitely a significant cause of the problem.</comment></comment></comment>


The Flynn effect - <AUTHOR> http://en.wikipedia.org/wiki/Flynn_effect

<comment><AUTHOR> My favorite part: "if American children of 1932 could take an IQ test normed during 1997 their average IQ would have been only about 80,[3] which would be classified as having borderline mental retardation or worse".</comment>


I automated all of my social media - <AUTHOR> https://patwalls.co/how-i-automated-all-of-my-social-media <comment><USER2> > Let's be honest - the main downside to automating social media is less > audience engagement (likes, comments, etc).

This is the first point under the "There are some downsides to automating" section. I think that's the most important thing to read and realize whether you should even continue to read or not.

If social isn't a marketing strategy your startup is using _at all_ , yeah, do this. It doesn't really matter too much then. But if you're actually trying to use social for _any_ value, doing this is a terrible idea, in my opinion. The author/OP phrases it as this is all a good idea if social isn't your _main_ marketing/acquisition channel, but I think this is still a bad idea if social is even your secondary or tertiary marketing/acquisition channel. Your engagement will tank on social, and people will be able to smell it a mile away if your social is worth engaging with or not. It's quite easy to tell if stuff like this is automated or not. I feel like the OP downplays that too much.

With that said, you do you, and decide what works best for your company. ¯\\_(ツ)_/¯ This is still a useful post if you put what I mentioned above aside, and decide to do this for your social strategy.

<comment><AUTHOR> OP here.

Yeah I agree, maybe I should have said if it's not in your "Top 3"

I think it also depends on your resources. If you're a solo founder then it can be hard to justify much time on social media.

And I also think you can have both. You can automate some posts, and also do ad hoc stuff. In my use case I publish articles on a consistent basis, why not automate the social media side of that?

<comment><USER2> That's a good point! I def agree it can be a good strategy to do both - automate some and still do manual work. That's a win-win to break it up!

</comment></comment></comment><comment><USER3> I automated my social media by deleting all of my accounts

<comment><USER4> Deleted code is debugged code.

<comment><USER5> This sentence gave me a miniature zen moment. It's gone now, though.

<comment><USER6> To become Zen you must first delete Zen.

</comment></comment></comment></comment><comment><USER7> Perfect, bots talking to bots. Now if we can only automate being influenced by ads the whole system will work perfectly with zero effort.

<comment><USER8> With this quick hack I drained my bank account! I set up an Instagram bot to browser for 30 minutes a day. It had a 2% chance to click any of the ads I saw. This aggregated URLs of products I wanted. To simulate bad late night purchase I set a Chrome headless instance to crawl these page at 11pm each night. This had all my form-saved information. To model drop off from bad UI, it would recursively auto fill the forms and try to proceed through the purchase, this gave me about a 70% drop off rate once in the checkout process.

(Just to be clear I didn't actually do any of this.)

<comment><USER7> What if we made a version of product search that sorted results by the number of times your reading bot saw the product advertised in your social feed (from bots). The entire social media enterprise could be supported with zero human effort.

</comment></comment></comment><comment><USER9> It's cool, but having broken icon image links gives away that its automated too much.

[https://pbs.twimg.com/media/Dvcbv4ZXcAAAEJR.png](https://pbs.twimg.com/media/Dvcbv4ZXcAAAEJR.png)

Also maybe add some more variation to the posts so that it looks human. When you pick a random article, and then stick a random block quote from the article on a picture... maybe just post your random quote as text and forget the picture every once in a while. Just to break up the visual rhythm.

Also throw in a retweet from a list of accounts you like every once in a while.

Edit: And last thing... really. Real humans rant. We get stuck on idea and make several posts on the same subject in a row.

You can easily schedule a years worth of ranting in a couple hours with a tool like buffer. Just post 3 things about the Steve Jobs biography in a row like you just now got around to reading it.

</comment><comment><USER10> Everyone should do this with their personal content too. Post to your own website and syndicate elsewhere. That way you maintain ownership of your content and audience, and people don't need to use Twitter/FB to follow you.

<comment><USER11> POSSE, as they call it.

[https://indieweb.org/POSSE](https://indieweb.org/POSSE)

</comment></comment><comment><USER12> This an absolutely fantastic post, thank you for writing this. Start Story is also awesome, keep up the good work!

</comment><comment><USER13> Isn't that like automating your cocaine addiction?

</comment><comment><USER14> Wait, no source code?

</comment><comment><USER15> TLDR; "Used bufferapp API to post all social networks</comment>


Things your grandchildren will probably say in the future - <AUTHOR> http://deepakbala.me/2012/11/02/5-things-your-grandchildren-will-probably-say-in-the-future/ 5 things your grandchildren will probably say in the future <comment><USER3> Agree with everything except e-mail. I see no reason to believe that's going away anytime in the next millennium.

</comment><comment><USER4> Modern hard drives don't look that different ...

RECENT : [http://www.newegg.com/Product/Product.aspx?Item=N82E16822136...](http://www.newegg.com/Product/Product.aspx?Item=N82E16822136555)

OLDNESS : [http://deepakbala.me/wp- content/uploads/2012/11/hard_disk.pn...](http://deepakbala.me/wp- content/uploads/2012/11/hard_disk.png)

</comment><comment><USER5> What's wrong with email?

<comment><USER6> Younger generations seem to use various other forms of "inboxing" (messaging apps, FB, etc) already. We can see some of the business documents now just being shared over the cloud, more communication over Skype and just meeting invites over email. As for personal mail, form my experience, the only email I receive today are newsletters, notifications, and the occasional BCCed broadcast mail (wedding invites, etc).

I think email is still relevant only for a document trail for business. If its about leaving a document trail, maybe there should be something new and more suited for it - for approvals, discussions, etc.

Its not like people are looking for a solution for a "problem", but then someone is going to create that something - which will make email less relevant even for the business use.

<comment><USER7> I don't see email disappearing. It is one of the classes of communication, from least disruptive to most disruptive:

Messaging - email, fb, linkedin, etc

Broadcasting - twitter, fb wall, rss, etc

Instant Messaging - skype, irc, etc

Calling - skype, phone, etc

In Person

That people are doing more messaging through fb, linkedin etc doesn't change that what they are doing is essentially email.

<comment><USER6> LinkedIn - <USER2>

FB - how much ever they make it look like email and function in a compatible manner, I don't think its used the same way email is(talking user behavior). It is just an asynchronous text messaging system. It is closer to chat/IM than to email in the way it is used. FB's bid to make it email-like seems to be an effort to move more of your messaging to FB or have the "unified inbox".

Is it really tough to imagine that 10 years from now we won't use "email" for more than 5% of our communication? And that subsequent generations will probably not use it at all?</comment></comment></comment></comment>


The Apollo Explorer - <AUTHOR> https://www.apollographql.com/blog/introducing-the-apollo-explorer <comment><USER2> Looks neat! Will this ever be something that we can run locally (similar to the GraphQL Playground)?

<comment><USER3> Maybe one day, but for now it's available entirely for free in Apollo Studio</comment></comment>


How To Design For Android Devices - <AUTHOR> http://blog.mengto.com/how-to-design-for-android-devices/ <comment><USER2> > All the resolutions on Android are pretty confusing, i.e. we have xhdpi > (2x), hdpi (1.5x) and mdpi (1x). They sound extremely technical. When I > first started almost a year ago, the engineers couldn't even explain to me > what the ratio were for each of those. I had to figure it out by asking a > lot of questions. The resolutions you should design in are 720×1280, 540×960 > and 360×640. Make sure to include hardware buttons and the status bar on > your design for high-fidelity mockups.

Uh, no. There is no standard resolution on Android. You shouldn't make any assumptions about it. Build layouts that scale themselves and adapt based on the screen width & height in density-independent pixels.

> Fonts are unfortunately device-specific so better expect crappiness on some > devices.

Fortunately, starting with 4.0 you can force the stock Holo theme in your app. Unfortunately, the manufacturer apps are stuck somewhere in the 2.x era and have completely alien user interfaces in comparison.

> In 2x, the minimum pixel dimension should be 28px for text, 96px for menus & > header and 116px for the footer.

It's a bad idea to talk pixels in Android at all. Instead, use density- independent pixels (and the sp unit for font sizes which also scales with the global font size) which are calculated based on the screen resolution and size/density. As two examples, the minimum size for a touch button should be 48x48dp to ensure it's big enough to touch, and a 7 inch tablet is at least 600dp wide.

<comment><USER3> Sorry for the downvote, but a lot of design advice coming from Google is like jedi hand waving. Of course there are standard resolutions and screen sizes, especially when you use dp instead if dip.

<comment><USER2> > Of course there are standard resolutions

No, there are not. A 720p screen might be a de facto standard on many devices right now, but if you are designing your app for a couple of resolutions and not resolution independence, you are in for a world of hurt.

For example, the Nexus 10 has a 2560x1600 resolution. Flagship phones this year are expected to go 1080p (the same way they went to 720p from 800x480 which was the most common resolution in 2010).

There is no standard resolution. Don't count on it. End of story.

> especially when you use dp instead if dip

dp and dip are the same thing.

<comment><USER3> Thnx. I use "dip" as special notation in my layouts and forgot they were actually dp.

I will concede one big difference: a lot of designers work in Photoshop so they're dependent on pixel resolution. I do everything in Illustrator, so I rarely think pixels.

If I think about resolution in old terms of strictly pixels then yeah, I'd be lost. But since I do everything in vector format, I still do my basic phone layouts at 320x480 and then scale from there.

</comment></comment></comment></comment><comment><USER3> There's a lot of shortcuts in this article. I appreciate him taking the time to put it together but a lot of important Android design practices were overlooked or misquoted.

There's 6 resolutions: ldpi, mdpi, tvdpi, hdpi, xhdpi, & xxhdpi. Of these, ldpi is often skipped, and tvdpi is optional (the Nexus 7 and Google TV use it). The new one, xxhdpi is important for new 1080p phones like the Droid DNA and upcoming Galaxy S4.

Resolutions are multiples. It's easiest to remember them as 48, 64, 72, 96, 144 (these are the sizes of your app icon). xhdpi is 2x mdpi. hdpi is 1.5x mdpi. tvdpi is 2/3 of xhdpi. xxhdpi is 3x mdpi & 1.5x xhdpi.

It's not pixels, it's usually dp (density-independent pixels) which are based on mdpi. As such, it's a good idea to design layouts at mdpi scale - it just makes it easier to match your dp values. I lay out for 320x480 and 1280x720.

I wrote a detailed blog post on 9-scale (<http://radleymarx.com>).

For most Nexus devices, the Action Bar is the only access to the Overflow Menu (aka the old Options Menu). So although it's only "recommend" to use it, there's no way to use the menu without it.

Fonts can be specified in the layout XML. You just need a custom TextView class. Note: widgets won't let you use custom class layouts. Fonts are added in the assets folder of the project. Some devices are picky about how you name fonts so it's best to name them as lowercase-name.TTF (uppercase file type).

Most often you'll want to match font sizes that other apps use. The minimum font size you can clearly get away with is 10sp on phones & 11sp on tablets. Users can change text scaling in their device so you should use sp instead of dp for text. If you don't want it to scale (i.e. a decorative layout) you can use dp instead.

The best way to design for devices is to buy one of each screen type. Some groups start a pool of devices to share for testing. For widgets I design for GS3 & GNexus simultaneously, along with a Nexus 7. When the layouts are complete, I start checking on other devices.

<comment><USER4> One other thing I would add is to check on different variations of Android (stock, HTC's Sense, Samsung's TouchWiz, etc..) as well as resolutions, since they have different default widget styles and colors that can affect your app in different ways. (Android 4 is much better about this with Holo, but anything before it can vary a lot.)

<comment><USER3> If you stick with native styles for device elements (such as the action bar & notifications) that should cover 99% of requirements.

Widgets are a different matter. Suffice to say the big difference there is between Samsung/HTC who displays full-sized widgets and Nexus/Motorola/Asus who force widgets to use way too much padding.

</comment></comment></comment><comment><USER5> Speaking as a programmer that has worked on adapting Android designs from mockups to reality this is probably a very useful article for would-be Android designers.

The "up button" description is a bit oversimplified but probably sufficient for an intro to design post.

Also fonts aren't exactly "device specific" as stated in the post. You _can_ embed your own TTF or OTF fonts into your app as a resource and use them. There are some annoyances to doing this (particularly when viewed from a designer's standpoint, so I'll let the article slide on this one) like the XML layouts won't recognize these fonts but you can programmatically assign the embedded fonts to Android's various text controls very easily.

Of course the hardest part as always when dealing with fonts is the licensing which is slowly getting better but remains years behind the licensing of other digital media formats.

Also on the issue of fonts, he mentions that Droid Sans is popular (since it was previously the default system font) but he should probably have mentioned Roboto, which is the newer and IMO much nicer default font, which is also freely downloadable, the entire set of Android fonts are available for use under the Apache license and can be used freely even in non-Android contexts.

<comment><USER6> Thanks for mentioning Roboto, I agree that it's a much nicer font. It's also good to know that it isn't too hard to embed custom fonts. I was definitely looking for that answer.

Also, can you elaborate on the "up button"?

<comment><USER5> My basic issue with the up button is the way you describe it conflates it with the back button, while the two should be considered different things (even though in many cases they will do the same thing). I blame Google more than anyone for this as they themselves conflate the issue mightily by, for example, styling the "up button" so it looks exactly like a back button.

<comment><USER6> How would you detail the differences between the 2 buttons?

<comment><USER7> Lets assume for a moment you are using the browser on your phone when a notification pops up alerting you to a new email. Clicking on the notification brings up your mail app and displays the email you just recieved.

At this point if you click the back button you will be returned to the browser session you were previously viewing. If, however, you were to click the up button instead you would remain inside your mail app and go "up" from your message to the inbox.

</comment></comment></comment></comment></comment><comment><USER8> He misses a lot of the intricacies of the Android design site ( <http://developer.android.com/design/index.html> ). You are actually recommended to make your content touchable rather than putting every action in the top right as an icon. If I made a weather app, for example, I may have little day or hour graphics that can be tapped for more about that item. Even for the action bar there is a split mode that puts actions on the bottom as well.

Similarly, he has the density classes wrong. There's XXHDPI, XHDPI, MDPI, and LDPI and they don't correspond to set resolutions. It's recommended by the Android Developer blog to actually change layouts based on a certain DP width (DP being a density independent pixel that normalizes measurements across density classes) rather than do x sizes of the same screen for each density. E.g. so screens with a lot of pixel real estate across get an extra list on the left, things like that.

<comment><USER3> You left out HDPI and TVDPI. While they don't correspond to specific pixel sizes, it's easiest to think of dp as px that scale for you based on a 320x480 screen.

</comment></comment><comment><USER9> I wish the designer in my company could be interested in these kind of articles. Is quite frustrating to receive time and again all the wireframes focused on an iPhone-like look, no matter what.

<comment><USER6> I was one of those designers. I'll try to get more designers to change their approach. :)

</comment></comment><comment><USER10> The resolutions seem off. The most common xhdpi resolution is correct (720x1280), however the hdpi one is 480x800 and mdpi is 320x480. Might be my mistake though, is there a clearer explanation why you chose those resolutions? Edit: This is in the case of phone sizes (tablets are another story). And even in the case of phones, you have varying resolutions inside those buckets. Yeah, explaining this to iPhone designers is always fun :).

Custom fonts are a problem, but there's a nice way to implement them (backwards compatible and all), and when you do it once, it will always be very easy. I don't have a code example nearby, shoot me an e-mail (in the profile) if you are interested, and I'll make sure to get a proper answer to you.

</comment><comment><USER11> Awesome article! Twitter apps, please take note of this:

> Finally, if you respect their guidelines, all actions should be on the far > top right. That includes functions like Refresh (because Pull to Refresh is > extremely uncommon on Android)

Pull to Refresh feels _really_ out of place in an Android app, don't use it! Most apps do not, but Twitter apps seem to be the exception, for obvious reasons.

<comment><USER12> I don't understand this complaint. Pull to refresh didn't exist on iOS either. Twitter introduced it, people liked it, it spread to other apps and it eventually became API. This is how evolution happens, why would you want to prevent it?

</comment></comment><comment><USER13> Thank you for disseminating this information. It would be a great world if more designers really understood Android. The one's I've worked with haven't given much of a care about it, and just stiffed me with wrong size images, and I ended up having to sample them and make my own 9-patches.

I wish Adobe would make a 9-patch tool that works with their stuff. draw9patch isn't that good.

</comment><comment><USER14> Very negative leaning.

<comment><USER15> Why do you say that?

Fonts are a true point of pain in Android dev.

<comment><USER16> Typography is pretty awful on Android in general. Both Roboto and Droid Sans are nightmare Frankenfonts.

Font rendering on Android is bulky, blocky, and fails to consider how humans read letterforms. So, even if a designer can convince the team to license a better font, it'll end up a smeared mess on users' screens.

The API calls to layout type on Android suck. (iOS sucks slightly less depending on how willing your UI engineer is willing to devote to this most basic foundation of design.)

"U MAD, BRO?"

Yeah. I get paid to be a picky snob about what the user sees and how they feel about the product. It pisses me off to look at Android because its not disappointing, the way Win8 is, but actively anti-design. Why pay to license good fonts? Why make a system for rendering type nicely? Because this stuff is basic user experience. It's like going into a hotel and discovering that the paint is yellowing. That hotel is more likely to have dirty towels, no hot water, smelly sheets, and bed bugs.

<comment><USER17> > Typography is pretty awful on Android in general. Both Roboto and Droid Sans > are nightmare Frankenfonts.

I like Roboto, but what do you find to be bad about those fonts?

> Font rendering on Android is bulky, blocky, and fails to consider how humans > read letterforms. So, even if a designer can convince the team to license a > better font, it'll end up a smeared mess on users' screens.

This isn't really true when setting custom typefaces on various views. Just subclass them and set up the typeface in the constructor and you are done. You just end up using that subclass your in layouts instead of a standard TextView.

> It pisses me off to look at Android because its not disappointing, the way > Win8 is, but actively anti-design.

I'm not really sure what you mean w.r.t to text. The text views and code cover most use cases and offer ways to handle flowing text in some cases, but you can always fall back to WebView rendering text content (which is easier for writers or editors to handle when making the text content).</comment></comment></comment></comment>


Ask HN: How do I market this thing? - <AUTHOR>

So I built a music video site called ChartBRAIN (http://chartbrain.com).  It's at the point where I think it's ready to try and get some users, but I have to admit I'm a better coder than marketer. So I'm wondering if the HN community can offer me some ideas to get some users and traction. I'd maybe even consider partnering up with someone with a proven record in driving traffic if they liked what I've built and felt it had some potential.  I think a lot of people think music videos are old news but Cull.tv just got bought by twitvid and tape.tv just raised 6.5 million. <comment><USER2> _I'd maybe even consider partnering up with someone with a proven record in driving traffic_

Anyone fitting this description would be more of the mind "I _might even consider_ partnering up with you." The world is sort of your oyster right now if you can do that. If you ask the kind of person you need what their next best alternative is, you will not hear "I got offered 5% equity and all the cola I can drink from an unfunded startup and am pretty darn tempted by that."

The traditional way to get traction for music sites and video sites is to turn a blind eye to folks stealing things that the established players can't turn a blind eye to. This lets you perform, essentially, visibility/legal-risk arbitrage. That's the on-the-ground reality, I personally wouldn't endorse acting on it.

You then use your early adopters to promote it to their friends, focusing feature development resources on implementing and optimizing things about the site which encourage social spreading rather than ones which merely encourage passive consumption.

<comment><AUTHOR> Good advice, Patrick I appreciate it! I should clarify some things, I guess. Fortunately, I'm adhering to all the YouTube api terms etc., so there are no legal issues (yet!). I suppose it's possible some entity like Billboard might take issue with using their name and/or logo to promote the site.

When I am talking about partnering, I know building a site is only part of the battle, so in terms of equity it would be much more substantial than 5%.

I guess maybe a better question then, is how to get those early adopters?

</comment></comment><comment><USER3> I'm someone with a track record of driving traffic and my company was acquired about a year ago. Totally different industry...but here are a few thoughts that might help you.

Interactivity - Right now the site is fairly passive. I think you would produce more engagement with the site if you allowed users to interact with the site. Maybe something like the ability to guess the next song in a quiz type fashion. Then, they get a score they can share across social networks. Or, some kind of voting mechanism that tracks your viewers opinion of the song/video.

Personalization - You will have a much greater chance of someone sharing your content across social networks if they can personalize the content and make it say something about themselves. You could think about giving the ability to the users to drag and drop the videos so that it changes the order in which they are played. Then, they might be able to share it in their social circle and say this is the order they think the songs should fall.

I realize these are feature suggestions that might take a significant amount of time, but building in features that drive the marketing is a great way to grow. Overall you did a great job on the site and good luck!

</comment><comment><USER4> Write a pitch and send it to techcrunch or mashable or any other site like that. Look for key editors on twitter and tell them about your site in less than 140 characters.

Give a reason on why people should follow you on twitter and facebook. For instance "Vote for your favorite video on our facebook page" or something like that.

Try facebook ads for a while if you have good organic numbers on your google analytics for instance a bounce rate less than 40%

Get it SEO friendly and try the basics with google webmaster tools

Work hard and be nice to people!

</comment><comment><USER5> Hey no real advice marketing wise, but one in the front of ux, please consider consider adding some sort of video lightbox effect on the videos (for example here: <http://chartbrain.com/chart/videos/hot>).

For me it was a big turn of in this kind of site at least to have to go to a new page to see the video, I wanted to sort of go quickly through many of them...

Otherwise, I kind of liked the site, not sure if I would personally use it but it seems ok...

</comment><comment><USER6> I have the impression you did it the wrong way. It is very difficult to create a product and push it somehow into the throat of people. It's much easier to other way round. Find what people want and then build the business.

Cull.tv had probably good arguments, but I don't think raising money is a proof of sound business in these days. BTW, I hope they are not your competitors.

Edit: I wanted to suggest to contact patio11, but I see he just answered.

<comment><AUTHOR> Well I agree to some extent, but I think the success of other video sites is proof that there is a market for a music video site with added value. I did build chartbrain.com to scratch an itch, but I do think it is a great way to enjoy music videos and stay on top of the charts.

<comment><USER6> What is your business model ?

I'm not sure I understood what you do. What I know is that people like to be part of the show or feel like so. Add a like button and give feedback on the number of time a music is played. Open it to submission.

Create a tribe of people listening to music videos and the cosy feeling to be part of this tribe. Add a live chat for instance or commenting on songs. Make your place different of all the others. Look for animators like a chat DJ.

You could leverage on musician desperately trying to find an audiance.</comment></comment></comment>