Gmail Ran out of space - <AUTHOR> http://daggle.com/pondering-email-conservation-hitting-gmails-storage-limit-1395

<comment><USER2> I _hate_ the fact that I can't search / sort by size in my gmail account. I'm sure I have at least 1gb of space that's being wasted by attachments that I no longer care about; but I don't want to go through all my messages to see what I want to delete and what I want to keep.

<comment><USER3> What you can do is add your gmail account as an IMAP account in Outlook or another email client and sort it there. Hard to believe that we must do that because of their anti sorting stance.

<comment><USER4> It isn't really an anti-sorting stance. If you have worked on appengine and with Google File System you will see that it is fast almost solely because there is no complete dataset. So sorting is really tough on 64K or 64MB chunks spread around many many machines. Google is so fast and scalable exactly because there is no complete dataset easily attainable. GFS works in 1000 item chunks and that is about the best you can get with that setup. Searching, counts/increments and metadata about ALL of your data in the GFS or email in this case, is a tough problem. The same underlying data system is used for gmail, reader, etc.

<comment><USER2> I wonder how difficult it would be for them to create a metadata file per account that would only contain things like sent from, sent to, size, subject, date sent (standard email metadata). Then if someone wants to go digging around, send the metadata down as JSON and let the sorting happen client side and let the client return the order the messages should render.

<comment><USER4> It is possible but it also means creating more data. I could easily see this being a pay element to gmail in the future. Basically it could be almost like a bot/spider/worker that gets a fairly recent snapshot of your data as metadata and allows you to clean up or organize based on it. It could be pretty tasking though and may need alot of engineering because right now your gmail data might be spread across thousands of machines. With more data/years this only going to get more tasking and costly.

At a certain point with data we have been living in a relative small sandbox in terms of data. As our lives spread to terrabytes of data and across many services, we to will be unable to run atomic operations on the whole of your data.

</comment></comment></comment></comment></comment><comment><USER5> Kind of misleading title - it implies that the whole of GMail ran out of storage space, whereas it was in fact just this guy.

</comment><comment><USER6> This article is far too long for something that can be summed up in these words: I am an abnormally large user of my email. Google's "buy more space" feature is broken because of a bug. It should be fixed.

<comment><USER7> Not exactly. The point is that the ratio of gmail's storage increase is so low that more and more people will inevitably bump against the limit in a couple of years.

It's not about how full the average inbox is today, but where it's headed.

<comment><USER6> Not really. Look at the stuff he's deleting -- hundreds of Facebook notifications = 10s of MB. He goes through and deletes thousands of emails and doesn't really recover more than 100 MB. My first feeling when I read this was that he's obviously deleting the wrong stuff. What he should really be looking for are those picture album emails at 10 MB each. That's what taking up his space and that's what most people aren't doing. Google is practicing the old idea of 80/20 and doing it very well.

</comment></comment></comment><comment><USER8> Simple solution with an additional filter rule:

[x] Delete message after __ days

GMail's web UI is really overrated. Missing many features that desktop clients and even other webmail platforms have offered for years. I still prefer traditional clients with the web UI as a nice fallback.

<comment><USER9> I do enjoy the 'conversation' view though. It's nice that it will pull in the messages that I sent into a thread, rather than just relying on people quoting the whole message in replies.

<comment><USER10> If you use OS X, Postbox will do this.

<http://www.postbox-inc.com/>

I'm sure clients exist for other platforms which do the same.

<comment><USER9> I usually use mutt outside of the Gmail interface, but IIRC there is a Ruby- based console mail reader that aims to do the 'conversations' thing.

<comment><USER11> <http://sup.rubyforge.org/>

</comment></comment></comment></comment></comment><comment><USER12> Regarding wasting storage on crud, I would be very surprised if the gmail storage system does not duplicate large attachments that have been forwarded around.

It would not be difficult to store one copy of every attachment or identical newsletter and link to it. This has to decrease their storage requirements dramatically.

<comment><USER13> Off the top of my head:

1) What if the disk your canonical copy is on dies?

2) You just introduced a dependency for cross-account reference counting (Alice and Bob share CryptoBible.doc attachment, Alice deletes the message it is attached to, Bob better still be able to access it). Your programmers will love you for that one. Don't forget all sorts of fun edge cases like "Does a suspended account peg an attachment on disk for forever, on the theory they could be unsuspended?"

3) Define "identical newsletter". No, really, pretend this is a job interview question. _waits_ Did you take into account the headers? _waits_ How about the salutation (Dear $FIRST $LAST,)? _waits_ How about the tracking pixels and unsubscribe links, which are by necessity personalized?

4) What is the performance impact of the algorithm to determine "identical", with respect to your kinda-sorta-not-really "identical" developed in question #3? Can it classify billions of messages in realtime to make nigh- instantaneous retain-or-delete-and-add-pointer decisions?

5) Does this algorithm introduce multiple single-points-of-failure into Gmail to save a fraction of a sliver of the costs of running the service?

<comment><USER14> 1) Why do you think Google has _the_ canonical copy on _a_ disk?

2) Why do you think Google ever intentionally deletes anything? Concurrent deletes are hard and expensive no matter what, and all of Google's tools make it much more so. They've gotten in all kinds of trouble in the EU for not being able to provide guarantees about how long they retain data.

3) Store headers, body, and MIME chunks separately. Your datastore will make hashes for its own reasons, and can make the de-duplication decisions independently of your app.

4) Your datastore is using hashes as the keys for retrieving the data already.

5) No, because there is no algorithm for this feature -- you got it for free because of other architectural decisions.

<comment><USER9> > _They've gotten in all kinds of trouble in the EU for not being able to > provide guarantees about how long they retain data._

The thing that I don't get is why they can't just 'wipe' the data. Maybe I'm misunderstanding GFS/BigTable here, but you don't necessarily need to 'remove' the information from that database. Just 'zero out' the data in there (or overwrite it with random data).

<comment><USER14> For starters, there are tons of indexes and snippets extrapolated from the data, and they're cached all over the place.

More fundamentally, why do you think any of the records are mutable at all? You reap enormous benefits at that kind of scale by making all writes log- structured.

</comment></comment></comment></comment></comment><comment><USER15> A month or so [1] after Gmail launched, The Screensavers [2] got a Gmail address and, on the air, invited viewers to send them emails to fill it up and see what happened. Unfortunately, I didn't see the end of the show.

[1]I don't know what took them so long. I know that it wasn't that soon after launch because Gmail came out in April '04, but this show was in May. (I didn't have a television at college, but my parents did). Gmail invites were scarce, but one would think they could've gotten one.

[2]This was on TechTV. It must've been just before TechTV was folded into G4 -- literally the last month of programming.

</comment><comment><USER16> What Gmail needs is an optional rolling deletion policy. When your mail gets to the ~7GB limit Gmail just deletes the oldest mail to stay within the limit. It could suggest you turn on this feature when you reach your limit.

MythTV does this when you fill your hard drive up with TV shows. It's much better than refusing the accept any new data,

</comment><comment><USER17> What we need is a "keep for 30 days" button, rather than "archive". Sometimes I mean keep forever, but more often my reaction to mail is that I don't expect to need it, and if I don't need it in the next month, I definitely don't need it, but maybe I'll change my mind next week and want it.

<comment><USER18> Actually that is what the Delete button does by default, keep it for 30 days.

<comment><USER9> Yea, but sometimes people will empty their Trash though. If it was archived with a 'remove after 30 days' tag on it, it wouldn't really be possible to accidentally purge all such emails.</comment></comment></comment>


The right to bear ARMs: Apple eyeing switch from Intel to ARM in 2013 - <AUTHOR> http://www.bgr.com/2011/05/06/the-right-to-bear-arms-apple-eyeing-switch-from-intel-to-arm-in-2013-report-claims/

<comment><USER2> Please please please please don't let this happen. Intel appears to be what made Apple viable for a lot of people, especially in the enterprise. Specifically, full-speed Windows in a VM or BootCamp.

It may be something we never actually use, but having that parachute is something I'm superstitious about losing. Never mind the return of the Whose is Bigger CPU wars.</comment>


Retraction from Tumblr - <AUTHOR> http://istumblrdown.com/retraction <comment><USER2> Why no appreciation of the apology? If all people are going to do is find fault with people apologizing, why bother? How should Tumblr reacted? What are they supposed to do?

<comment><USER3> Why should we appreciate the apology? We might blame tumblr a little less now than if they'd stayed their course, but I don't see why it should swing over into appreciation of what they're doing.

Perceivedly, it happens a lot that big scary companies bully smaller entities with meritless but expensive-to-fight legal threats. That sometimes there's enough of a public notice of that to make the company reconsider and to limit the damage done is just part of the game and doesn't really leave them without blame, it just means they got "caught" this time.

<comment><USER4> I think you are missing the point here. It is stupid to blame Tumblr for all the other big scary companies. They made a mistake. They apologized and fixed the problem, and according to the apology they found a bug in their process and fixed it.

Instead of bashing Tumblr for apologizing, we should be happy they retracted. If HN was the reason why they retracted, then great. But let's stop looking down on Tumblr purring ourselves at the top.

We are all at the same boat and same level. We are all trying to make our companies work.

<comment><USER5> He is missing the point, it's not even his apology to accept, it's Zach's. He's appeared to do since he considers the apology 'genuine'.

The thing to note in all this is:

"Our legal department very recently started using a third party vendor to assist us in pursuing trademark infringers and, due to an error in our new process, your domain was mistakenly caught in the cross-fire."

If tumblr has switched to yahoo! legal, that is not a nice side effect of the acquisition.

<comment><USER3> > He is missing the point, it's not even his apology to accept, it's Zach's.

Of course it's not for me to accept or not accept that apology and no one should care about my opinion on the matter. But I think not appreciating the apology is a defensible position to take and I'm trying to articulate why I feel so. Feel free to dismiss me. :)

</comment></comment></comment></comment></comment><comment><USER6> That means, unless you are on HN's front page you don't have rights to use word tumblr in your domain

<comment><USER7> It's become an interesting phenomena now. The rise in availability of the internet and pervasiveness of social media is making companies accountable for their wrongdoings. Of course, the more prominent/visible the complaint, the more likely it is that a company will try to act on it.

However, I don't believe that most companies would just handle problems like this in a case by case basis. IsTumblrDown made some noise, and that noise led Tumblr to recognizing a flaw in their process. At the very least, it'll make them more cautious going forward. Incidents like this can lead to long-term improvements.

<comment><USER8> Good

It means that legal departments have to stop the "shoot first ask questions later" strategy

It's very easy to just go around threatening legal action against people unchecked.

</comment></comment></comment><comment><USER9> It doesn't matter to Tumblr if HNers boycott Tumblr the web site.

It might matter to Tumblr if HNers boycott Tumblr the employer or Tumblr the business partner.

<comment><USER10> It's also worth considering Yahoo! the employer or Yahoo! the business partner.

</comment></comment><comment><USER11> So essentially, a legal team was spamming anything with the word tumblr in it?

<comment><USER12> > a legal team was spamming anything with the word tumblr in it

There ought to be some comeback on people who do this, just like there ought to be for people who send out false DMCA takedown notices.

<comment><USER13> Lawyers basically say until a cease and desist is received (confirmed received), then they've spent money and are serious.

</comment></comment></comment><comment><USER14> IsTumblrsLawsuitDown.com

</comment><comment><USER15> The cynical reaction would be to say this is all about damage limitation to their brand since it appeared on HN, as they meant to do it rather than it being a 'clerical error'.

Good thing I'm not cynical.

<comment><USER16> HN, and its readership, simply isn't that important to tumblr. If it were reddit, maybe the cynical reaction would be accurate.

<comment><USER17> Yes it is. Tumblr's parent is Yahoo and Yahoo will be keen on being friendly to the developer community to attract top talent. Much of which reads HN.

<comment><USER18> I know it seems that way, but that's actually reddit and r/programming. Based on sheer numbers, I think most engineers don't frequent Hacker News.

I was just at dinner with an ex-Zynga game dev last night who had never even heard of it.

<comment><USER19> >ex-Zynga game dev

Well, he did say "top talent"!

<comment><USER18> I'm not sure if you're being sarcastic, but it's unfair to judge someone's ability based on where they used to work.

This friend of mine was laid off in New York and received his severance; based on what I know about him, he could work in a multitude of places right now if he wanted to.

Zynga's vilified image aside, there is no reason to believe their engineers are not "top talent."

</comment></comment></comment></comment></comment></comment><comment><USER19> "So I'll be bringing it online in a few weeks again."

Wait, what? He gets to keep his product but post-pones its availability by a "few weeks"? Opportunity missed. Or am I missing something?

<comment><AUTHOR> It's not really a product. It's a page that says "Tumblr is up" or "Tumblr is down." Very very niche and incredibly limited functionality.

</comment></comment><comment><USER20> > _...pursuing trademark infringers..._

That's the part they should be apologizing for.

Unless their business is actually getting hurt in any way (is someone selling fake Chinese made Tumblr t-shirts on the black market?), this is just a systematic exercise in corporate bullying.

They shouldn't be engaged in this in the first place. Tumblr is not friggin' Nike.

<comment><USER21> There's no choice in the matter. Do you realise trademark holders have to vigorously defend their marks in order for them to remain protected? Failure to do so jeopardises protection.

Disclaimer: IANAL

<comment><USER22> ... and istumblrdown.com is somehow stealing tumblr's business?

<comment><USER23> Only when tumblr is down.

</comment></comment></comment></comment><comment><USER24> This seems like a risky move for tumblr, no? Doesn't this mean they're not actively defending their trademark? In which case this could cause them legal problems in the future.

<comment><USER25> IANAL, but to me it seems more like "we are actively defending our trademark (ala the hiring of the legal company), but we do not believe that you are infringing on that trademark and what you are doing is fair use, so carry on"

<comment><USER26> DannyBee (who is a lawyer) pointed out in the thread last night that this doesn't need to be "defended" against. He explained that, since they use the term "Tumblr" to refer specifically to the real company Tumblr, and since it's clear that they're not trying to masquerade as Tumblr or cause any confusion, that this is a legitimate use of the term and does not need to be "defended against" in order to protect the trademark.

<comment><USER27> The legal term here, incidentally, appears to be "nominative use". (I learned a lot from that thread!)

</comment></comment></comment></comment><comment><USER8> And by the way, this is not (mainly) because it was front page on HN.

HN users do not overlap (a lot) with the target audience for Tumblr. (not a lot at least)

HN is of course an important news outlet in the industry.

<comment><USER28> true, but it is same with google reader. it is not profitable, but who is target audience? the guys who install and recommend most of the software for casual users. So in some cases, you need to please a smaller group, to win over bigger group.

I think any company can win market, if your local it friend/guy starts recommending you specific brand. For example, I changed 10 peoples mind how to think about AV. They went over from AVG to MSE, because I recommended. Why? Because, there is less chance for them to fuck it up and install trial.

</comment></comment><comment><USER29> I wonder what the threshold is for pursuing trademark infringement...obviously imposters but what else?

</comment><comment><USER30> With our powers combined....

</comment><comment><USER31> This is a classic case of TDC. (i.e. Tumblr Damage Control).</comment>


Erlang R19 Release Candidate Available - <AUTHOR> https://github.com/erlang/otp/releases/tag/OTP-19.0-rc1 <comment><USER2> Wow, lots of goodies. Erlang team has been unstoppable in the last few years.

Some changes:

* New state machine module: gen_statem (ssh module uses it as a first example).

* 3-5x times faster open_port(spawn, ...)

* Must faster and better tracing (also lttng support)

* Keeping messages off heap for processes

* mnesia-ext : pluggable storage for Mnesia. This removes the 2GB per shard limit.

* Allow limiting process's max heap usage

* Observer tool allows configuration update frequency and length of graph windows

* crypto uses EVP calls so should be accelerated by hardware were supported

* Parallel/faster module code loading

<comment><USER3> > Allow limiting process's max heap usage

What happens when it hits that limit? Crash, or try to GC, or... ?

<comment><USER4> [http://erlang.org/documentation/doc-8.0-rc1/erts-8.0/doc/htm...](http://erlang.org/documentation/doc-8.0-rc1/erts-8.0/doc/html/erlang.html#process_flag-2)

Configurable, but can kill the process in question and/or log when it occurs. Their suggestion is to log it first and tune it from there since heap size isn't easily predictable.

EDIT: Scroll down a little bit to get this particular flag documentation.

</comment></comment></comment><comment><USER5> Awesome stuff. [http://erlang.org/download/OTP-19.0-rc1.README](http://erlang.org/download/OTP-19.0-rc1.README) has the complete list.

It's so beautiful to read such CHANGELOGs. The developers clearly love their product and have put in the effort to clearly explain each change, background and it's purpose.

<comment><USER6> I think it's partially from an ethos driven with the hindsight of the original core Erlang team. They wished in retrospect that they'd spent more time documenting "why" they did something rather than "what" they did.

Not wanting to recommit sins of the past and all that :-)

</comment></comment><comment><USER3> I miss working with Erlang. I still think Ruby is better for getting something out quickly for many businesses that charge for a product(†), but Erlang was very interesting, challenging and generally a good community.

† If you charge for a product, you don't generally have to worry quite so much about scaling because you make a lot more money per user than some site that's all about ads and trying to keep the cost per user really low because the ads don't actually pay all that much.

<comment><USER7> Have you evaluated Elixir yet? Targets Erlang BEAM + OTP backend + Ruby like syntax. Best of both worlds, maybe? The Phoenix framework sweetens the deal too. I would also add Ecto ORM for popular RDBMS backends.

<comment><USER3> I haven't had a serious look at it yet, but the Ruby/Rails ecosystem is really big. I'd bet Elixir isn't quite there yet. I like what I've seen so far, though.

<comment><USER8> I'd argue that in the vast majority of cases, anything 'missing' from the ecosystem is probably one of those gems that probably shouldn't be a project dependency anyway. That doesn't mean it won't filter its way over to the Elixir ecosystem at some point, but I don't think you'll find any of the major pieces missing.

</comment></comment></comment></comment><comment><USER9> You will be able to use leveldb with Mnesia in R19.

</comment><comment><USER10> what does "Keeping messages off heap for processes" mean?

</comment><comment><USER11> What are the possible implications of this for Elixir?

<comment><USER12> Elixir immediately gets access to all of these features in the VM. Over the next coming months, there may be the creation of more Elixir-friendly access wrappers to the features.

However, there is the need to make sure Elixir can work with R19 in the first place, but Jose Valim and co. has usually been quite good at getting that working.</comment></comment>


Rate-design wars are the sound of utilities taking residential PV seriously - <AUTHOR> http://blogs.berkeley.edu/2013/11/12/rate-design-wars-are-the-sound-of-utilities-taking-residential-pv-seriously/ <comment><USER2> Higher fixed monthly charges will only delay the inevitable. In places like Hawaii, it could actually speed it up.

Suppose you were a homeowner with solar panels installed. Right now, you're not paying very little because of net metering. Two years from now, the utility decides you'll need to pay monthly for grid connection.

It might then make sense to just cut the grid connection. Adding panels and batteries is the obvious choice, along with upgrading appliances with more energy efficient ones. Batteries are expensive, though they could be in your electric car.

One step further, if neighbourhoods or condo associations organize in micro- grids, big utilities are again stranded with useless assets.

<comment><USER3> Batteries are _very_ expensive.

It costs $10,000 to buy a battery system large enough to carry your whole house. And you're cycling the batteries every day, so you'd be lucky to get 10 years use out of them.

Even with the higher fixed charges mentioned in the article, it is still cheaper to connect to the grid than to buy the batteries.

</comment></comment><comment><USER4> I can understand why people are hostile towards utilities for trying to change their pricing. However, keep in mind that utilities are some of the most regulated private companies in our economy. Their rates have to be approved by the government. They are allowed to make a profit, but not a big one. If they start making losses, then the lights are going to go out ...

For those who have suggested going off the grid, keep in mind that even in sunny California, we do have a rainy season. I've reviewed my friends' electric bills with them, and even those with quite large PV installations are still net consumers through most of the winter. To make it through, you'd likely need a generator (dirty, loud, expensive).

I think the bigger issue here is that people have made long-term commitments (20 year lease agreements) to finance solar installations, counting on net metering and a continual increase in power rates. If the utilities change the game, you're going to screw over a lot of fairly well-off consumers (vs. the mean), which doesn't usually go over well.

</comment><comment><USER5> Hawaii gives us a glimpse of the future of what solar will look like in the US. They are the hottest solar market in the US due to having the highest electricity rates combined with excellent conditions for solar. The primary utility in Hawaii (HECO), has started denying many permits to homeowners due to grid saturation on many circuits.

Infrastructure investments must now be done by HECO to allow for increased solar adoption - something that may not be in the best interest for the utility.

<comment><USER6> I'm uninformed, "grid saturation" meaning what?

<comment><USER7> I'm not positive, but I might imagine that it's that the solar power generated by that particular grid either equals or exceeds the amount used by it. If there's multiple grids it could be that the residential area is nearly self sustaining during the day, and they don't have the capability to move that generation successfully to the other places that need it because the grid interchanges weren't built with that in mind.

<comment><USER4> That's exactly what it is. See presentation here: [http://www.heco.com/vcmcontent/StaticFiles/pdf/LVM/LVMMECOPu...](http://www.heco.com/vcmcontent/StaticFiles/pdf/LVM/LVMMECOPublicNEW.pdf)

It appears that there are neighborhoods in Hawaii where the PV generation capacity is approaching the max load of the circuit.

</comment></comment></comment></comment><comment><USER8> > What makes the policy debate so difficult to resolve is that tariff design > is a very indirect way to support residential PV. In Germany, they've gone > with feed-in tariffs for solar PV instead – a direct subsidy for every kWh > of energy coming from your PV system. Much simpler, and allows a reasoned > debate on tariff design apart from solar PV policy.

I'm fairly confident this section about Germany is wrong - Germany has feed in tariffs which operate in addition to net metering, not as an alternative. That is certainly the case in the UK, whose system was modelled on Germany's.

Still, the basic point of this post is correct, net metering is an indirect subsidy, but there is an obvious technical solution: using the article's metaphor, to encourage people to eat their own zucchinis, or to use the solar energy on-site rather than automatically exporting it to the grid.

At present, PV systems export generated electricity directly to the grid, separately from the grid-connection used to bring in electricity which is consumed on-site, all of which is imported. Solar could fulfil normal electricity demand, and also be used to charge up storage heaters, or storage a.c. To the utilities, the house would look like any other, except with lower electricity demand.

<comment><USER9> Sure, PV exports to "the grid", but practically speaking you are reducing the power requirements for your local area. Since peak demand to the area is lessened, utilities get to defer infrastructure upgrades, thus saving money.

<comment><USER10> Until there is enough renewable generation that you're actually reducing or eliminating their profits on the fixed-cost of generating equipment they have in play.

I think that's the concern. It was all, "Ha ha look at solar, how cute and expensive." Now its more, "Its going to eat our lunch." and its only getting cheaper year by year.

</comment></comment></comment><comment><USER11> Hilarious.

"they want to further reduce the incremental energy price by implementing a fixed monthly charge on each customer, aimed at covering some of the costs of retail distribution, metering and billing."

That's straight out of the 1975 hippie classic "Ecotopia: The Notebooks and Reports of William Weston" \- and is one of the Casus belli California cited when seceding from the union in that book.

<comment><USER12> I keep getting notes from SoCal Edison with each bill wanting to tack on fees for every little thing. New distribution lines, cleaning up their nuclear mess at San Onofre. In other businesses those are costs of doing business and are factored into their balance sheet rather than getting directly passed on to the customer.

<comment><USER11> Just so long as you don't start setting up artillery on the sierra nevadas.

</comment></comment></comment><comment><USER13> The IBP (tiered rating) isn't a birthright of utilities, geez. It's a conservation measure!

Whether I shave off my higher-tiered consumption by changing my thermostat, changing my lighting or by putting in solar, that lost high-profit consumption isn't the company's property, it's mine to do something else with.

<comment><USER4> I don't think most utilities have a beef with shaving off high-tier consumption (well, I'm sure they miss the revenue, but they are not stupid enough to go back on years of telling their customers to conserve).

Their bigger concern is net metering, by which I can sell them solar power during peak hours at their retail rate of, say, $0.45/kWh and then buy it back from them at, say, $0.06/kWh at night, or maybe even 6 months later in the dead of winter.

Not paying for power you didn't use is one thing, but using the power company's own transmission network to arbitrage its rates with no fee does seem long-term unsustainable. If the current rates reached their ultimate conclusion, the power companies would have to make massive, massive investments in storage systems capable of buffering months worth of demand, as their net revenues trend towards $0.

</comment></comment><comment><USER13> I am a net-metering solar PV owner who's not averse to a reasonable, independently audited "connection charge".

But I would be adverse to having to pay via a "connection charge" for what looks to me like a failed uprating of a nuclear plant, if I were I a Southern California Edison customer.

</comment><comment><USER14> Dig a 15 meter pit 5 meters across and suspend a 5 meter cube of ferro- concrete on a cable feeding onto a drum above it connected to a gearbox and motor/generator. Attach this to your PV and then relax.

<comment><USER15> Google says density of concrete is 2400 kg / m^3. Multiply that by 5x5x5 m^3 and you have 300,000 kg of concrete (300 metric tons).

       m = 300000 kg
       g = 9.8 m/s^2
       h = 10 m

The total amount of potential energy you can store is:

       m * g * h = 29400000 J = 29.4 MJ

The more customary unit of energy in electric circles is the kW hr. Converting a kW hr to joules gives:

       1 kW hr = 1 kJ hr / s
               = 1000 J hr / s * (60 s / min) * (60 min / hr)
               = 3600000 J
               = 3.6 MJ

So the potential energy stored in the parent's system is:

       29.4 MJ * (kW hr / 3.6 MJ) ~ 8.17 kW hr.

Probably needs to be bigger to supply a decent sized household. Not to mention issues with inefficient conversion of potential energy to electricity, maintenance, and a backup plan for cloudy days.

<comment><USER14> Thanks for that, I thought it would give around a couple of days of backup, however I had completely fudged the numbers somewhere along the line.

</comment></comment></comment><comment><USER16> Interesting, it is getting more "real" to the utilities. It should also be shifting the transmission load around, I don't know if that helps or hurts.</comment>


Ask HN: Email token service - <AUTHOR>

Hi HN,
there is any "email token" service out there ?
I mean that I would like to generate every time a new email address (something crazy like: firstname.lastname@example.com) and use that email address to register to only one single service.
When I receive an email to that address, the email it forwarded to my own, "real" address.
As soon as I start to get spam from a particular token I can simply stop the forwarding of email.
Is anything like that available ? <comment><USER2> Shameless self-plug.

If you are running postfix as your mail server, you could use something like [0] (or write your own solution, utilizing a database or whatever, of course).

[0] [https://github.com/afics/vmailmgrpy](https://github.com/afics/vmailmgrpy)

</comment><comment><USER3> yea the phrase used for indexing on google is "one time email address" 10minutemail i guess?

<comment><AUTHOR> Dear friend,

your message show that you haven't even read my message.

I appreciate your reply, but next time, please, take time to read the message you are replying to.

Best.

</comment></comment><comment><USER4> You could easily use something like Mailinator (or their Pro service) for something like this.</comment>


Trello X, A Unofficial Mac OS X App for Trello - <AUTHOR> http://lingsdesigns.com/trellox/

<comment><USER2> Might be nice, but the site looks extremely sketchy with zero details about the app you'd like me to download and run on my computer.

Sorry, not gonna happen.

</comment><comment><USER3> Screenshots? Details?

Trello is already quite a good UI on its own. What benefits do I get for trusting your app with access to my account?</comment>


Announcing Beta For Local SSD - <AUTHOR> http://googlecloudplatform.blogspot.com/2014/10/announcing-beta-for-local-ssd.html

<comment><USER2> "No planned downtime: Local SSD data will not be lost when Google does datacenter maintenance."

Only unplanned downtime then :)

Live migration maybe adds another '9' to reliability, but customers still need to plan for failure.

<comment><USER3> (note: I work on GCE networking)

Yes, certainly, it's always possible that hardware will fail with no notice, and any architecture needs to account for that (this is equally true if you're running your own infrastructure, of course). For some applications downtime in these situations may be the right value tradeoff, for others they'll need to architect things assuming always-on redundancy. In terms of data durability, though, local SSD gets you a performance win by being local. If there's a catastrophic failure of that machine, data is gone.

What live migration buys you, the prospective cloud customer, is transparent avoidance of (a) Google's planned hardware maintenance windows (for example, network or power infrastructure maintenance) and (b) outages due to hardware failures which can be detected and migrated away from before they cause data loss.

The second category includes any number of situations which, if you can't migrate your workload, require taking a machine out of service, replacing some gear, bringing it back up, and hoping the replacement fixes the issue[0]. If we can instead migrate your VM away from the issue (say, for example, it's a bad root hard disk -- we need to replace the drive as lots of things in the VM hosting environment depend on it, but your workload is otherwise completely unaffected), we are able to service the affected machine with zero downtime for your service.

[0]: Think about the number of things that could manifest as a flaky network connection. Could be any (or all) of: bad cable, bad NIC, bad motherboard, bad CPU, or even bad RAM (since NICs use bus-mastering DMA from host memory in most cases). After migrating the workload away, we can take as long as necessary to reliably diagnose and fix the machine. Huge win for us and for our customers.

</comment></comment><comment><USER4> With the provisioning flexibility I wonder if these are internal or external? If internal, seems like they eat a lot of cost if they are never provisioned. If external, how do they keep them as fast as local?

<comment><USER5> They're using NVMe, so the SSDs are either really local or at least in the same rack using a PCIe switch. One could imagine that different servers have different numbers of SSDs and when you create a VM with N SSDs they put it on a server with that many.

<comment><USER6> That is quit interesting. I am currently looking into seting up a virtual platform that will use SSD disks.

Do you have a source I can look at for more info on what SSD setup Google Compute Engine uses?

<comment><USER5> TFM: [https://cloud.google.com/compute/docs/local- ssd](https://cloud.google.com/compute/docs/local-ssd)

</comment></comment></comment></comment><comment><USER7> _Optimization considered harmful: In particular, optimization introduces complexity, and as well as introducing tighter coupling between components and layers._ \- RFC3439, via [https://github.com/globalcitizen/taoup](https://github.com/globalcitizen/taoup)

<comment><USER7> Pfft. Classic Hackernews... downvote to pluto without discussion despite validity of alternate perspective. One day you kids will learn :)

To spell it out, "Optimization considered harmful" means time may be misspent caring about these things, and/or actually result in changes to your application or service that will take it backwards in measurable ways.

"In particular, optimization introduces complexity, and as well as introducing tighter coupling between components and layers." means, for example, that relying on Google Cloud's SSD support to run your service locks you in and makes you dependent on them, their ability to provide availability, their internal decision on how long they choose to maintain that service offering, and their internal decisions on pricing. It further suggests that this time may be better spent getting to the heart of the problem, eg. removing the disk I/O performance constraint from your application that made you want an SSD in the first place by sharding your datastore.

This is a pretty valid line of thinking as an antidote to 'rah rah'.</comment></comment>


Apple, Google lose bid to avoid trial on tech worker lawsuit - <AUTHOR> http://www.reuters.com/article/2014/03/29/us-apple-google-ruling-idUSBREA2R20W20140329

<comment><USER2> I wonder how big this could get, settlement-wise.

My son lives in the Bay Area and even though he is a tech enthusiast (lots of Apple gear, active GMail user from two years before public release) he is very down on tech companies right now. I get the impression that this is a common view in the Bay Area.

So if this goes to trial a local jury might be inclined to go hard on the defendants.</comment>


What's a software service you would pay for? - <AUTHOR> Let's keep the service charges to $5, $25, and $100 a month. <comment><USER2> Maybe an alert system that monitors selected websites for certain content (key words) or even lack of content, such as outages. RSS can potentially do this, but you have to accept lots of content to filter it locally (client-side), and its not portable. Have a smart-phone app, emails, and web-only interface as options for receiving notices.

<comment><USER3> I am working on something kinda-sorta like this, but I'm not so sure how to productize it. I think the industry as a whole has been held back by the "phone app" phenomenon because it discourages people from taking full advantage of the power of PCs.

The trouble is that you have to have an "irrelevance model" and that may be even more user specific than a relevance model. (For instance, anything about Apple is in my irrelevance model for HN) The irrelevance model may be more important than the relevance model because 99.9% of everything is junk, so the problem of finding actionable "news" is like making glasses that let you look at the sun.

<comment><USER2> Good filters have both "include" words and "exclude" words. It could also have something like Outlook's filtering "wizard" that allows fairly complex logic trees. But one has to be careful to not get carried away early in a project.</comment></comment></comment>


Cross Site Scripting is a Big Problem [bug fixed] - <AUTHOR> http://nycs.bigheadlabs.com/search1/?q=%3C%2Fh3%3E%3Ch1+style%3D%22font-size%3A+100px%3B%22%3ECross+site+scripting+is+a+big+problem.%3C%2Fh1%3E%3Cscript%3Ealert%28%22One+must+filter+users+input+or+render+it+harmless+or+bad+things+can+happen.%22%29%3C%2Fscript%3E <comment><USER2> I guess it's a good thing there's no authentication on that site.

<comment><AUTHOR> Any domain cookies for .bigheadlabs.com are vulnerable, which _could_ be a real problem (Wordpress admin maybe?).

Domains are so cheap now that I almost always buy one for every project (even hacks) these days, partially just to isolate potential XSS issues.

I didn't mean to imply anything disparaging towards you, this kind of annoying stuff pops up even at Google. It's so easy to miss a spot, especially on quick hacks.

Thanks for creating that site, it's an awesome contribution.</comment></comment>


People who lived through the "Golden Age" of BBSes write about their experience - <AUTHOR> http://textfiles.com/history/ <comment><USER2> BBSes made such a difference in my childhood. I think my parents wished I was chasing a hoop down the street with a stick, playing stickball, or some other confused memory they had of their own childhood that was probably just an episode of Leave it to Beaver. But BBSes were where I learned about technology, programming, and collaborating with others to build something beyond what I could do alone. They're where I made friends with the cute girls in my city that 13 year old me wouldn't have thought to approach otherwise. They let me message to people on the other side of the planet back when $2/minute long distance calls and weeks-long mail delivery letters were the norm among most adults.

I ran a moderately popular board with my brother called "virtual unreality systems", which seemed like the coolest name to me as a kid. I may add to that page once I'm back in front of my laptop

</comment><comment><USER3> The audio files are worth checking out too.</comment>


Multiple Monitors and Productivity - <AUTHOR> https://blog.codinghorror.com/multiple-monitors-and-productivity/ <comment><USER2> From 2004, when people used CRTs and not LCDs.

Maybe still relevant, but you should at least keep this in mind.</comment>


"I am writing to share some alarming signs of Bitcoin price manipulation." - <AUTHOR> http://www.ofnumbers.com/2017/11/09/a-note-from-bob-on-the-transparency-of-tether/ <comment><USER2> Is not the SEC aware of these mysterious "USDTs?" I still don't really get what they are or how they come to be, and furthermore how they are used to manipulate price.

It's like a guarantee that there is a cash representation right?

Also, most federally-backed banks have but a fraction of all balances on hand.</comment>


Ask HN: Tech scene in Raleigh, NC? - <AUTHOR>

I'm contemplating leaving the bay area in about a year for reasons deserving its own post. I keep reading about a tech scene in Raleigh, but I'm curious to hear from HNers who live there about what Raleigh is really like. Do you enjoy living there? How are the people? Is it hard to find a tech job that pays a decent salary? Is there much of a startup scene? How's the weather? How's the food? Is the cost of living reasonable?
If it helps, I'm a full stack web dev with some iOS experience and technical project management skills (see my profile for more info).
I appreciate your insights! <comment><USER2> Upvoting as I'm from Charlotte, NC and want to hear how our neighbor Raleigh is doing! From my time visiting there and from what my friends who live there say:

It was a great, relaxed place to live. There are lots of smart people (it's in the Research Triangle). Weather is truely seasonal. Hot in the summer (80-100+), cool autumn and spring, cold winter. Not cold enough to have the snow stick regularly, but it will melt in the day and freeze at night. So beware black ice.

I'm also in the Bay Area. Would love to hear why you're leaving over coffee if you're open to it. Maybe I can intro you to some good people if you end up moving over there too. Email is in my profile.

</comment><comment><USER3> Raleigh is great. Lots of educated people. Two hours to the beach. Good jobs. Cheap labor.

</comment><comment><USER4> Hi. The Raleigh/Durham/Chapel Hill area is a tech hub and has been for years, with a number of big IT companies in Research Triangle Park (RTP). RTP is kind of the center between those three cities.

RTP historically has a bunch of big IT in it - IBM, SAS, and Nortel were the big three, but Nortel melted away, not sure what IBM is doing on the hiring front there, SAS is still chugging along. Currently, I think Fidelity, Deutsche Bank, and MetLife are out in RTP. Fidelity seems to be recruiting this year consistently and the other two as well. These big places all pay pretty normal salary ranges. I'm out of the corporate dev world, but anecdotal info would seem to peg salaries in the $90K to $110K at those places for experienced engineers. Friends and I have heard of $125K salaries plus bonus, but none of us can actually point to a verified example of that base salary as a direct hire. I think even these big enterprise corps want to hire a little lower level and promote up. For contracting at the financial corps, all bets are off. In 2008, I knew contractors in test groups doing $100-$150 an hour, which was insane given the skill sets I observed on some of those contractors (minimal). I wouldn't count on that today. I tend to put the Triangle (shorthand for Raleigh, Durham, CH, and RTP) into a "third tier" of tech cities, with SFB/NYC tier 1 and Chicago/Austin as tier 2.

I spend more time kind of looking at Durham for startups and small consulting firms. Adzerk, iContact, and back out in RTP, MaxPoint are startups at various points of existence. I have no idea what pay is like at startups. You might want to check out [http://bigtop.it/jobs](http://bigtop.it/jobs) \- they have both a job listing and host semi-regular mixer/job fair events. I went to one and didn't really pursue anything. It may have been a little entry-level leaning.

Oh, don't forget the local university tech scene. NC State is there in Raleigh (and RedHat is on Centennial Campus), Duke is Durham, and UNC-CH in Chapel Hill. If you are interested in being a staff coder in academia, there are opportunities in those places. Duke and UNC-CH also have big hospital orgs that hire buckets of IT staff - if you can shoehorn the word EPIC onto your resume, you can land something in one of those two quickly.

Weather is lovely, but hotter than the Bay area. Humidity and 99F is . . . nasty. And I'm a native NC'er. Good mix of sun and seasons during the year. Excellent beaches (and the Outer Banks) two to four hours driving. Same drive time West puts you at 4000 to 5000ft in the Appalachian mountains of NC.

Cost of living is ok. Housing cost varies too much to generalize. Cheaper, sometimes much, outside the cities. Good food given the college town status in all three. Reasonably active tech meetup scene.

If I was moving to the area and didn't have a partner or family, I would probably lean towards downtown Durham. I feel like it might be a little more funky and interesting than the other places. If I had a family or partner who wanted to start one soon, I might shop around for a school district I would be happy with and try to land there.

Oh, last thing - the Triangle has good political diversity, but I think once you leave it, the state is generally conservative with pockets with more progressive presence.

<comment><USER5> FYI - Red Hat is now located in downtown Raleigh.</comment></comment>


On the Berenstein Bears Switcheroo (2014) - <AUTHOR> http://woodbetweenworlds.blogspot.com/2014/06/commenting-on-berenstein-bears.html <comment><USER2> Posted in today's NY Times: [http://www.nytimes.com/2015/05/15/nyregion/witness- accounts-...](http://www.nytimes.com/2015/05/15/nyregion/witness-accounts-in- midtown-hammer-attack-show-the-power-of-false-memory.html?ref=nyregion)

NYPD shoot a man wielding a hammer in midtown. One witness says he was shot while fleeing. One says he was shot while lying down and handcuffed. Only they have a surveillance video that clearly shows him swinging his hammer at the other cop when he is shot. The problem is your brains.

<comment><USER3> Sorry if I'm missing something, but what does that have to do with the Berenstein Bears article which this is a comment to?

<comment><USER4> When something happens (911; a crime you witness; a cop shooting someone that you witness) you get a bunch of people to write down, as soon as possible after the event, what happened and what they were doing and where they were.

You then check again 5 years later. You'll find a lot of differences between the two accounts.

What's weird though is when you show people their original account, in their own handwriting, and those people will deny the original account. "Oh no, I must have been mistaken when I wrote that" they'll say.

In the case of the books people are _convinced_ that they have some special version of the book with the Berenstein spelling, and they can't believe that their book is just like every other book with the Berenstain name. They'll argue about it, tell you you're wrong, and then dig out a box from the attic and discover that actually they're wrong.

This is a useful natural experiment that lets us know that eye witnesses are not reliable, especially when time has passed.

</comment></comment></comment><comment><USER5> I thought it was -stein too, but I posit that we all think this way because surnames ending in -stein are very common, while surnames ending in -stain are rare. We mentally corrected the obscure variant spelling we only see in one place for one we see everywhere else. Hell, my own cousins had a -stein name, so it's definitely something I would've been exposed to a lot.

And I always did pronounce it Bernstein as a kid, so my memories are just crap in general.

</comment><comment><USER6> I came up with the same theory independently. There used to be a restaurant here in Brisbane Australia called Pastoodles. I know the road where it was, the exact shop, I drove past it many times intending to go there. One day it wasn't there any more. In its place was a different shop that had obviously been there for years. Google doesn't remember Pastoodles, nor do old paper copies of the white/yellow pages, nor do registered business listings from the years I passed it. I have looked into this. My best friend and I discussed going there multiple times but she has no memory of the conversations. The only conclusion I can come to is that Pastoodles exists in an alternate universe and I've traveled between the two.

<comment><USER7> I don't know about alternate universes (though it's a cool theory), but apart from the Berenstein/stain thing, I've also had so-called "false memories" from time to time. I'll bring up conversations I would swear on my life I've had with someone, and they won't have a clue what I'm talking about. It has happened to other family members too; sometimes my mother will bring up a past event or conversation and the rest of the family will assure her it never happened. My sister can remember things from our childhood that none of the rest of us do, and we don't believe her until she breaks out an old diary and it's there in black and white.

Another example of the brain playing tricks: There are so, so many times I get stopped in public or at my job by a customer, someone who swears they know me from somewhere, or that I must be related to so-and-so, when there is no actual connection. Apparently I have "one of those faces" and I seem familiar to many people. One of the most interesting times that happened was when I was 18 and browsing around in the local Head shop, this girl came up and hugged me from behind. I didn't recognize her, but she started asking me when I moved to the little Georgia town we were in, or was I just there to visit her? Despite my protests, she swore I was her high school sweetheart from Oregon, right down to my accent, mannerisms, smile, eyes, and build. I had to show her my license before she could accept I wasn't him. I would have thought she had a mental disorder if not for her best friend right beside her going on and on about how I could be his twin.

I'm sure a geneticist or anthropologist could offer a theory on my uncanny resemblance to her friend in that particular instance (that, or my mother had twins and never told anyone), but it doesn't explain why so many other people see a familiar face when they look at me. Maybe I'm just that plain-faced that it's easy for others to project a known face onto mine.

<comment><USER8> Once, as a child, I recall waking-up and running to the cupboard expecting to find it recently replenished with food, only to discover it was bare. My mom was still out shopping. I realized that I had only dreamt about her returning home with groceries.

It's probably not applicable in the Berenstein/stain case, but I've long suspected that a lot of false memories originate in dreams that I've forgotten about having. The contents of the dream lives on, along side legitimate memories, while the fact that it was a dream fades away.

The empty cupboard experience was unique because the dream was recent enough for me to recognize the false memory as a dream I had just had.

<comment><USER7> I think a lot of it is indeed half-remembered dreams. I've woken from dreams believing that what happened was a memory, convinced of it, even though I was aware it was also a dream. Usually this only happens when I'm physically ill (I've been fighting a bad sinus infection and I've had some hyper-realistic dreams all week).

</comment></comment></comment></comment><comment><USER9> And then, as you stare blankly at the evidence, unyielding and undeniable in its contradiction like the ocean before the shore, you realize that reality as you knew it has never been more than an elaborate fiction, and not even one that cares to tell you the truth.

 _Berenstain._

</comment><comment><USER10> I think a large part of it is due to some sort of quirk in how we read words.

In the word "Berenstain", the first two vowels are e's, and I think the brain naturally expects the next vowel to be an e as well. Also, it's a show/book series aimed at children, and children are much more likely to misread/misspell/misremember things. It'd be interesting to see the demographics of people who remember the "E" spelling vs. those who remember the "A" spelling.

It doesn't help that some people, like myself, only ever read the books, and never heard it pronounced on the TV show. And even then, the "ain" pronunciation is still plausible for the "E" spelling (like "vein").

This is all purely theoretical, and I'm no expert, but it seems plausible. I experienced a similar effect when I realised I'd been spelling and pronouncing "Seinfeld" as "Seinfield" for a long time. I was much more willing to accept that I'd been wrong about that though, since, once I started listening a bit closer, I realised there was no way you could pronounce "SeinFIELD" as "SeinFELD".

<comment><USER11> If you listen closely, when it comes to the show you can hear them pronounce it two different ways; as -stein in the opening segment, and then as -stain on the network bump before commercial. As a kid I'd probably have paid more attention to the one in the show.

[https://www.youtube.com/watch?v=AS9lB57IQco](https://www.youtube.com/watch?v=AS9lB57IQco)

At the least we have indication that some adults wanted to say it as "-stein" around the time of the TV show's creation.

(Stan and Jan pronounced their name '-stain', so that is correct, but it just doesn't roll off the tongue, does it?)

</comment></comment><comment><USER12> Eerie timing - or inspiration?

[https://news.ycombinator.com/item?id=9552210](https://news.ycombinator.com/item?id=9552210)

It bothers me I remember it distinctly as "Berenstein", but I try not to dwell on that too much.

<comment><AUTHOR> Yes, your comment was the inspiration, I went to google the spelling because I vaguely remembered Berenstein, and got sucked into reading about the issue.

I think the most plausible explanation for me is that they pronounced it "Bear-en-steen" in the videos so as a kid I just assumed that was reflected in the actual spelling.

<comment><USER13> I remember my (first grade?) teacher made a point to tell us that even though the name was written with an "a", it was read as an "e" to demonstrate how English violates so many of its supposed rules.

So thankfully I never struggled with the actual spelling and questioning my sanity over it.

</comment></comment></comment><comment><USER14> I would have sworn up and down, and bet all my money it was Berenstien, but when I see the Photoshopped logo, it looks... wrong. The e doesn't mesh with how I remember it looking.

It's like when you scratch a bug bite, but then it starts bleeding.

</comment><comment><USER15> I thought it was "Ford Perfect" for a decade before realizing it's spelled "Ford Prefect" in all the HHGTTG books. The mind sees what it wants to see.

<comment><USER16> Now I'm afraid to look -- is it LiberOffice, or LibreOffice? Spellcheck says the second one, but...

<comment><USER17> LibreOffice. I remember at the time everyone complaining that it's very inconvenient to pronounce.

</comment></comment></comment><comment><USER18> This is fascinating. I remember Berenstein myself, probably owing to the fact that that's how we pronounced it ("steen") and when I first encountered them, I couldn't read.

I'm going to ask some people tonight out at the bars to write the word and see if they'll put money on their spelling, which will likely be wrong if we're any indication (don't worry, I won't take it).

</comment><comment><USER19> I'd extend this further and suggest that our existence in Universe A instead of Universe E hinged entirely on the attentiveness of an unknown immigration officer when attempting to understand a Jewish surname. Somehow a bunch of us shifted from one universe where (s)he wrote down "Berenst _e_ in" to one where (s)he wrote down "Berenst _a_ in", and now - thanks to that random stranger from the 19th century - thousands (perhaps even millions; I've yet to encounter anyone in the wild prior to today who didn't spell it or pronounce it "Berenstein") of folks are rather lost.

It doesn't help that - in the videos - "Berenst*in" is spoken with an Appalachian (?) accent, making it incredibly ambiguous what the normalized pronunciation would be.

Whether or not this is a case of quantum physics or social psychology at work seems to be elusive, to say the least :)

</comment><comment><USER20> I _insisted_ to my high school drama teacher that Neil Simon was not just a playwright but also a famous songwriter. I guess Neil Diamond and Paul Simon got crossed in my brain somewhere.

</comment><comment><USER21> How long til the copyright expires? Then we can print new editions that make the requisite correction.

</comment><comment><USER22> I've noticed this phenomenon with titles that might contain plurals or articles, but also might not: e.g. "Sid Meier's Civilizations" or "Mad Max: Beyond the Thunderdome". It's typical to eliminate these elements for brevity, but more jarring when people remember them when they aren't there.

<comment><USER23> A common example: the last book of the Bible is often mistakenly referred to as "Revelations".

</comment></comment><comment><USER24> Probably lossy compression. The memory was reconstructed from incomplete memory and adult knowledge of nomenclature.

</comment><comment><USER25> This reminds me of the people who swear that the old cartoon Captain Pugwash was a cesspool of filthy double entendres.

[http://www.snopes.com/radiotv/tv/pugwash.asp](http://www.snopes.com/radiotv/tv/pugwash.asp)

</comment><comment><USER26> I get this sort of feeling with audiobooks a lot, due to word pronunciations. I grew up saying and hearing a word a specific way for years of my life, then this audiobook professional reads it in a different way, but I have to accept him as the authority, since he's the official "reader", and I can only assume got the correct pronunciations from the author(s).

One instance was the word "sietch" from the Dune series. I had always pronounced it as one syllable, 'seetch', but the audiobook said 'see-itch'. And this was the multiple-cast series, and different voice actors said it 'see-itch'. I was actually yelling at them: "NO! It's SEETCH!"

</comment><comment><USER27> This whole "universe splitting" theory seems like a load of utter hogwash trying to explain the simple fact that humans tend to have really bad memories.

<comment><USER28> Somebody didn't get to point 9: "I don't really believe we switched universes. Obviously, with my rational mind, I understand that the most reasonable explanation is that I misremembered. Occam's Razor and all that."

<comment><USER29> #9 was absent in his universe, no such key exists on his keyboard.

</comment></comment></comment><comment><USER30> Similarly, I was absolutely baffled when looking back at 80's history that the TV show and pop icon named Max Headroom, was not, as I remembered and would insist as being named: Max Head-rom

</comment><comment><USER21> They should make a Star Trek time travel episode to explain this.

Most of the Star Trek time travel stuff is already pretty stupid, so that's not a reason to not do it.

<comment><USER12> There's actually a Mandela Effect regarding a Star Trek episode!

There is a following of people who believe a certain character was killed off on a very specific episode. The thing is that the write of Star Trek claims that character was never killed off.

[http://mandelaeffect.com/star-treks-chakotay-story- line](http://mandelaeffect.com/star-treks-chakotay-story-line)

<comment><USER21> I am actually working my way through that series right now.

I think there may have been a confusing episode where it seems like he dies, but he didn't if you pay close enough attention.

Because I do remember being confused at one point and thinking he was dead, but I think my confusion got cleared up somehow.

Star Trek tends to do things like that. e.g. the episode opens and bad stuff happens, but then you find out it was in the holodeck. Or a dream. Or alien brain manipulation.

In fact there was an episode where Chakotay led a rebellion against the captain and (probably) died (I don't remember now), but it turned out to be a "holonovel."

</comment></comment></comment><comment><USER31> reminiscent of the Marshall McLuhan "medium is the massage" thing. Everybody "knows" it's the medium is the "message" because, of course, it's about media and communications... But that pesky objective external physical reality keeps sneaking back up and proving long held memories to be bogus.

</comment><comment><USER32> My last name is Bredensteiner (and was when I was a child reading the books) --- It is and was always stein in my head.</comment>


Bjarne Stroustrup on Educating Software Developers - <AUTHOR> http://itmanagement.earthweb.com/features/print.php/3789981 <comment><AUTHOR> "The companies are complaining because they are hurting. They can't produce quality products as cheaply, as reliably, and as quickly as they would like. They correctly see a shortage of good developers as a part of the problem. What they generally don't see is that inserting a good developer into a culture designed to constrain semi-skilled programmers from doing harm is pointless because the rules/culture will constrain the new developer from doing anything significantly new and better."

Wow. I was a little anti-Bjarne because I'm no big fan of C++, but this is gold.

<comment><USER2> > as cheaply

In other words, talent costs money. Companies expect the good developers to come out of the woodwork for a 35-40k junior dev position and wonder why they come up with dregs.

But I agree. In the one year I spent at a university I ran into many CS majors. Most were in it for the money.

Now these students were only taught in Java as far as I could tell, yet they seemed to have no interest in the language. I'd ask questions about Java web technologies that I'd only heard of (being more of PHP guy) like Struts and what not, drawing blank stares. I'd ask all sorts of things that I'd heard about the language, hoping they'd relate some experiences in regards to what I'd heard, that the code was rigid, verbose, etc. I was especially interested in what they knew about OOP which at the time I was teaching myself. More blank stares.

Didn't take much more digging to discover that they were pursuing the degree for the money and nothing more. I thought that it was very sad that I, a History major at the time, had more passion for computer science than most CompSci majors I met.

<comment><AUTHOR> _Companies expect the good developers to come out of the woodwork for a 35-40k junior dev position and wonder why they come up with dregs_

It's not that they expect good devs at that price, it's that they _can't tell who the good devs are_. For grads, all they have to go off of are grades and maybe an intership. Otherwise, it's just a resume. Without an effective way to recognize good devs and with no portfolios, it's just a crapshoot. It's the same information asymmetry that makes it hard to buy a good used car. As pg (?) said, the only way to recognize a good dev is to work with them for a while, and most companies and job seekers are unwilling to do that.

</comment></comment></comment><comment><USER3> A major part of the problem is that many schools are still trying to cram everything into a BS Computer Science degree. Both the students and industry would be better off if schools offered alternatives such as a BS Software Engineering and/or BA Software Development, which would focus more on construction and somewhat less on theory. Some schools are doing that, but not enough.

<comment><USER3> By the way, Richard Gabriel (one of the early Lisp experts) had some good ideas for an alternative academic program. <http://www.dreamsongs.com/MFASoftware.html>

</comment></comment><comment><USER4> So, is it just me, or he complains that students are pointed in the wrong direction and therefore don't write "real/maintainable" programs, instead focusing on more CS (algorithms, data structure) - but when talking about java, he says the education needs to focus more on algorithms, data structure.. ?

When talking about magic constants on students' code: "I have seen the result of that attitude in new graduate students: It is rare that anyone thinks about the structure of their code or the implications for scaling and maintenance – those are not academic subjects. Students are taught good practical and essential topics, such as algorithms, data structures, machine architecture, programming languages, and "systems," but only rarely do they "connect the dots" to see how it all fits together in a maintainable program. "

When talking about the use of java in universities: "Education should prepare people to face new challenges; that's what makes education different from training. In computing, that means knowing your basic algorithms, data structures, system issues, etc., and the languages needed to apply that knowledge."

To me this is like a "everything is a priority" problem. Students should learn all of the CS stuff, and also all of the "realworld" stuff.

<comment><AUTHOR> His emphasis is on problem solving. That's his complaint about having students use Java libraries to build stuff - they don't learn how to solve problems. You need algorithms, data structures and theory since it's the language of solving problems, but you also need practice, feedback, and mentoring to get good at programming. He objects to those academics who think programming isn't an essential activity.

</comment></comment><comment><USER5> Count me among those who who thinks the CS vs. Software engineering debate is silly.

I never understood how you could teach CS theory w/o a lot of programming and I have no idea how you'd teach programming w/o the theory. If you can't figure out the big-O complexity of an algorithm you're useless and if you can't implement it you're useless.

I also think that engineering degrees should not be crammed into 4 years. Just to get the basics requires way more credits than a normal BA. Perhaps a BS should be viewed as something between a BA and an MS and 5 years should be standard.

<comment><USER6> Because most people working in software development aren't doing anything remotely algorithmic. The vast majority of corporate/commercial software does two things: it gets input from somewhere, either a user or another program, and writes it to a database. It gets records from the database and formats a report for a user or sends it to another program. And that's it. That's all most websites do too. If it's XML then the programmer doesn't even need to know parsing. You can - and many people do - work in software and be good at your job, without _any_ theoretical underpinnings at all.

That's not to say that that's a good thing, mind, it's neither good nor bad, but it's a fact that maybe 1 in 10 people employed in the software industry will need to know big-O, perhaps even less than that.

<comment><USER7> _That's not to say that that's a good thing, mind, it's neither good nor bad, but it's a fact that maybe 1 in 10 people employed in the software industry will need to know big-O, perhaps even less than that._

I don't think the ratio is that high, simply because most people who do software development aren't in the software development industry. They are members of an IT team in an organization who profits come from the sales of things other than software. Although the organization they are at might benefit from software that solves real technical problems relating to their product line, most of the time, they _are_ just writing ETL programs that require little more than getting table names right. Of the people who work at companies that see software development as one of their core disciplines, I imagine the problems are sufficiently hard to make some knowledge of theory useful.

</comment></comment></comment><comment><USER8> _Had I been given such books as a 1st year student, I would have changed fields. Those books were trying to spoon feed the students. Good students need to be challenged! The contemporary Math, Physics, and Biology books I have seen are far, far more conceptually challenging than what we present to CS and engineering students in the area of programming._

Let me testify from personal experience that Bjarne is on to something here.

</comment><comment><USER9> Bjarne has obviously never heard of the "School of Hard Knocks".</comment>


Python performance the easy(ish) way - <AUTHOR> http://jiaaro.com/python-performance-the-easyish-way <comment><USER2> I've tried this with GCC 4.7.1 on Debian x86_64 and did not get it to work in constant time with -O2.

I'm guessing (from the OP's usage of -install_name) that he's been compiling this on OSX. I wonder what did my compiler miss that the OP's didn't?

EDIT: just tried with clang and got constant time behaviour, interesting

EDIT 2: reading the comments in the post, I now suspect it has to do with integer overflow. However, compiling with -fwrapv did not change anything. Need to dig into it more.

EDIT 3: it seem that clang simply notices that the computation can be done in constant time, whereas gcc does not. I'm not sure if it's actually useful in real world code, but it's certainly somewhat magical to see a compiler understand that you can substitute the entire loop with a simple calculation

<comment><USER3> I assume the OP would be using llvm, as this is the default OSX compiler, and the gcc command is usually just an alias or link to clang.

In real world code this kind of optimisation is designed to catch things programmers aren't aware of, for a large decrease in required computation. In this case, sum(1 .. n) == n*(n+1)/2.

</comment></comment><comment><USER4> Did anyone even read his code? The C code gets the wrong answers once its integers overflow. The Python code will produce the correct answer using longs once the integer calculations overflow. Ctypes is the right answer for a different problem.

<comment><USER5> I agree that it'd be more correct to make the C and python code output the same answer.

The important thing here is how to hook up C code to python.

the benchmarks are still (mostly) valid because it's still the same number of addition operations (regardless of the overflow).

Honestly, I'm not a very good C programmer (as you can see). I was really just documenting how to do this for myself. I never expected to get such a big surge of traffic from HN.

That being said, I'll take a second look at the code and try to make it more correct, suggestions welcome!

</comment></comment><comment><USER6> I'd also recommend the cython library (<http://docs.cython.org/src/tutorial/>). It is as easy as writing usual python code with some type annotations to achieve similar speedups.

</comment><comment><USER7> So I've done some limited work with writing C/numpy extensions (for numerical work) and it's gone swimmingly. However, a lot of performance-critical things I need to do don't necessarily fit into array/matrix numerical computing - lots of times there are difficult,complex datastructures involved that need to be accessed in the "inner loop" (not really a loop, but you get the point). I typically end up using Java for these sorts of problems, although sometimes Pypy does okay.

I rather dislike the constant advice I see doled out to those who mention Python's performance issues - namely, that they can just rewrite the performance critical parts in C. Lots of the time, the performance critical parts are just as sophisticated and require just as much abstract, high-level coding as the rest of the program. Some awareness of that would be nice.

<comment><USER8> I hate responding with such a flip answer, but have you tried Cython? If you have to do some serious munging inside inner loops and need something that is a bit more aware of data structures and the shape of your data I have found Cython to be a nice option for building the equivalent of a Python C-extension module with mostly Python-like code and structure.

</comment></comment><comment><USER9> CTypes is neat for simple tasks but Cython is the industrial strength solution for large extensions.

</comment><comment><USER10> First of all, the C code will return wrong answers because of integer overflow for sufficiently large inputs. But since this is only for the sake of an explanation on Python performance, I'm willing to take that.

What disturbs me much more is that the article basically concludes, that if Python performance isn't good enough for you, write it in C? How is that Python performance? If speeding up Python is what you want I'd suggest having a look at PyPy, which is a JIT python runtime. It doesn't work with some libraries (for reasons I didn't look into yet) but works wonders on performance!

Oh yeah and if anyone ever sums up large amounts of continous integers using a linear time algorithm: Please use n*(n+1)/2. This is obvious, but since the article ommited it I though I'd say it here.

<comment><USER11> As awesome as PyPy is it isn't going to give you anywhere near the sort of speedup you'll get from dropping down to C for you critical functions. The best I've seen on actual code I've written (as opposed to benchmarks) is 4-5 times speed up, which is certainly nothing to sneeze at, but still a long way to go before you can start comparing with C. For what it's worth In the contrived example used in the article I only got a 1.2-1.3 time speedup using pypy (pypy 1.9 vs python 2.7).

(of course you can use ctypes from pypy as well, so you can get the best of both worlds)

<comment><USER10> This is true, yes. The kind of performance C delivers is often only achieved by C, Fortran and some other compiler languages. But that's not the point of what I was saying. I was talking about speeding up the Python _language_. That's what PyPy is incredibly good at.

I once wrote a small traffic simulation in Python. Not real-time. It would take about 15 minutes to run a medium sized simulation using the CPython interpreter. PyPy ran the same simulation in slightly less than a minute. This is a substantial speedup. In this case it was because function calls can be incredibly slow in Python and PyPy takes care of that.

I'm not claiming that implementing parts of the program in C is bad (I'm mainly a C programmer anyway). All I say is that it's not really "speeding up Python" when you're not using Python. It's definately an option though and a very good one too!

ctypes might not be the best way to do it for some use cases though. In my opinion, the most beautiful way is extending the Python API with native C. That option produces the most LOCs though, and therefore probably more bugs. However, if you've got large portions of a Python program in need of speedup and PyPy won't cut it, it's probably the best option to write a decent part of the low level stuff in C and provide native Python bindings. This way you get the best of both worlds: The performance of C for your underpinnings and the flexibility of Python for developing all the other parts, saving you time and therefore money and sweat.

</comment></comment></comment><comment><USER12> If you like this kind of dynamic code generation and usage, LLVM with ctypes is also a promising avenue, as you don't (necessarily) have to invoke an external compiler. You can build the module on-the-fly.

For example see bitey, an abstraction to directly import LLVM bitcode files: <https://github.com/dabeaz/bitey>

That said, there's so many ways to speed up Python these days...

In most practical cases where I need performance I'd try to run it in PyPy (easiest), use Cython (great python-C hybrid), use Theano (to generate GPU code), or even write a plain Python C API extension (no dependencies to deploy, and in contrary to ctypes you can directly offer Python classes without Python wrapper).

</comment><comment><USER13> Thanks for showing it was that easy. I knew this was possible, but I thought it to be way more complicated. Now I'm more inclined to use it (if I really need that level of optimisation).

</comment><comment><USER14> I wrote a program in assembly a few weeks ago to do this exact thing.

It can spawn worker threads to speed up the summation. I have a quad core CPU, so I've set it to 4. If you have a different number of cores, you can modify line 8 to change the number of workers it will spawn.

<https://gist.github.com/3369946>

</comment><comment><USER15> I'm not a big user of Python, but it seems to me that any language that requires you to embed a different language for performance is inherently broken.

To regular pythonistas: Is this a common issue with Python, or only in the author's contrived scenario of summing a ridiculous amount of numbers?

<comment><USER15> Wow, I don't think I've ever had such a visceral reaction to anything I've posted on HN before. By people who clearly didn't take the time out of their day to even read what I posted!

Basically, I don't use python very much and wanted to know if python had performance issues that required you to drop down to C code often, or if that was just for a scenario such as this.

I think that any language that REQUIRES you to drop down to a lower level for performance (not one that SUPPORTS it, but one where it's REQUIRED) is inherently broken. I would think that would be a pretty noncontroversial statement - if every python script you wrote had to have embedded C code because otherwise it was too slow, that would be pretty broken. That's why I wanted to know how common this was.

<comment><USER16> I think the reaction was because you put Python in a broadly negative camp by infering ctype extensions are required for any sort of performant code. Its actually relatively uncommon to use ctypes and most developers who use Python know this.

Python is a great general purpose language and is used as such. No one would be silly enough to solely rely on it in highly performance sensitive environments (IE: 3d renderers or high frequency trading). Just like no one would be silly enough to write an entire CRUD web app in C.

The point is, Python isn't broken and claiming that it may be because you read a short article on a Python feature got you the negative reaction.

<comment><USER15> I never inferred that ctypes were required - in fact, I explicitly did the opposite. I asked if it was required for performant code. See the part of my post where I said "To regular pythonistas: Is this a common issue with Python, or only in the author's contrived scenario of summing a ridiculous amount of numbers?"

I also never claimed that python was broken.

<comment><USER16> > I'm not a big user of Python, but it seems to me that any language that > requires you to embed a different language for performance is inherently > broken.

It seems like you imply both of those things, though perhaps not as explicit as "Python is broken because ctypes are required", which you did not say.</comment></comment></comment></comment></comment>


Software Licenses Explained In Plain English - <AUTHOR> http://bluefletch.com/blog/software-licensing-the-intersection-of-technology-law/

<comment><USER2> Author is clueless. Linux is provided as an example of open source software, even though it is publicised under a free software licence.

<comment><USER3> Actually, no, the author is correct. Most people don't get the licenses correct nor truly understand them. The notions of "free" vs. "open" are often conflated or not well defined.

This author has correctly identified the distinction and tries to lay it out in human terms.

Before the Internet went mainstream, I used to see a lot more GPL software that was pay-for. If you paid and got the binary, then the GPL entitled you to also get the source code. But if you didn't pay, there was no way to get anything since it wasn't distributed to you.</comment></comment>


UTM Mangler - <AUTHOR> https://github.com/huntwelch/UTM-Mangler <comment><USER2> For those unfamiliar with UTM:

Urchin Tracking Module was an analytics package that Google acquired way back when.

UTM defines a set of query parameters that Google Analytics, and other web analytics platforms, use to capture and provide and information about a referring source (or 'channel') to a webpage. At the highest-level, the buckets are direct, organic, referrer, social, email, display, and paid search.

The three UTM parameters that are required to track 'campaigns' in a web analytics tool, such as Google Analytics:, are Utm_Source, Utm_Medium, and Utm_Campaign.

Example: [http://stilldrinking.org/utms/utm- mangler.html?utm_medium=so...](http://stilldrinking.org/utms/utm- mangler.html?utm_medium=social&utm_source=facebook&utm_campaign=jan-15-hn- woop-woop)

If you clicked on the link above, it would inform stilldrinking.org's Google Analytics property that you, the visitor, came from Facebook and are part of the 'jan-15-hn-woop-woop' campaign.

</comment><comment><USER3> Unfortunately, this won't work for any analytics platform that uses first interaction attribution, which most do. That means, they'll immediately store the first value seen and treat that as the "source" for the visit. Changing it will tweak per visit parameters, but won't really affect most people.

Still cool, but a better solution would be to make a plugin that mangles them before the request is sent :)

<comment><USER4> Can't speak for other analytics systems, but this is definitely not the case for Google Analytics (which is what most utm_* parameters are targeting).

Google Analytics' default attribution (outside of Multichannel Funnels) is _last_ touch attribution.

<comment><USER3> GA is definitely ubiquitous, but people serious about attribution aren't limiting themselves to GA :P

</comment></comment></comment><comment><USER5> Worst of all is when these are used in cache keys especially when some of these get unique values. I did write a simple Nginx set of rules to strip this off when used in the cache key or sending to the backend. Here's a fragment: [https://gist.github.com/nikolay/e58fb0de98acd8aed932](https://gist.github.com/nikolay/e58fb0de98acd8aed932)

</comment><comment><USER6> Or just strip it entirely:

[https://github.com/wistia/fresh-url](https://github.com/wistia/fresh-url)

<comment><USER3> This one just removes it after people land on the page, it won't strip attribution which is what it sounds like the OP is going for.

fresh-url is meant to aid site owners so that the users don't see the ugliness of UTM codes :)

</comment></comment><comment><USER7> What is UTM?

<comment><USER8> I was expecting something Mapping/GIS related, given [1], but maybe that's just the mental scar-tissue from a previous project involving various data sources that were in formats not quite entirely unlike the ones they purported to actually be. Which meant, along with one incredibly badly designed viewer application, I could legitimately say "I spent almost an hour dragging myself out of the ocean" in a worklog entry :P

[1] [https://en.wikipedia.org/wiki/Universal_Transverse_Mercator_...](https://en.wikipedia.org/wiki/Universal_Transverse_Mercator_coordinate_system)

<comment><USER9> Back in a previous life I used to work for a software company in the defense/GIS space. It's pretty incredible how much labour goes in to making the earth somehow representable as a series of X, Y and Z tuples. The difference between floating and fixed point could mean being off by dozens of metres.</comment></comment></comment>


What music sounds like through an auditory implant - <AUTHOR> https://theconversation.com/heres-what-music-sounds-like-through-an-auditory-implant-112457 <comment><USER2> Why do the implants only have 8-22 electrodes?

The cochlear nerve is about 0.2mm thick, which means if mems technology is used to make the electrodes, one could cram over a million in a 1000x1000 grid and still be orders of magnitude from the limits of silicon technology. Using that, there is a very real possibility that a user could end up with super- human hearing.

Either there isn't enough money in implant R&D to justify the use of the latest tech, or delays in medical certification processes are forcing the use of 30 year old technology.

</comment><comment><USER3> The original IBM PC had a speaker in it, that was driven simply by connecting one lead to an I/O pin and the other to ground. The speaker could be driven only by a square wave. The sound waves were simply converted to a square wave turning the I/O pin on and off.

This hopelessly crude device worked rather amazingly well. Certainly far better than the sound samples in the link. I wonder if that technique could be applied to the implants.

<comment><USER2> These implants stimulate nerves directly. Nerves aren't good at receiving high frequency information (an individual nerve can't fire much more than 10Hz).

Instead many nerves exist, each representing a specific combination of volume and pitch.

These implants can't precisely target the nerves though, and end up hitting a random selection of nerves. That's the core of the issue.

</comment></comment><comment><USER4> I've had a cochlear implant since 2002 and always cringe at these kinds of articles because the samples always sound RIDICULOUSLY terrible compared to what I usually hear. Like, NES levels or early PC speaker levels of terrible. I wonder why.

<comment><USER2> How many electrodes does your implant use?

The demos sound like they only use ~4.

<comment><USER4> It's the HiRes 90k, so 16, IIRC. They bury the information pretty deep under lots of marketing talk about "virtual channels" and such.

</comment></comment></comment><comment><USER5> This series of six talks by Graeme Clark the inventor of the cochlear implant gives a good insight into the struggles and hopes of this type of research [https://www.abc.net.au/radionational/programs/boyerlectures/...](https://www.abc.net.au/radionational/programs/boyerlectures/restoring- the-senses/3210174)

</comment><comment><USER6> I set off an alarm recently that triggered much the same auditory effect. Also similar is if I listen to binaural beats for 10 minutes and remove the headphones. I have no idea what's going on there.

<comment><USER7> I wonder if it's the audible equivalent of the afterimages you see when you stare at a certain color for a while.

<comment><USER8> This would explain so much of electronic music. It also makes me wonder if there are any 'movies' of moving color that actually take place solely in the mind, with the aid of the after-images of what has come before?</comment></comment></comment>


Ask HN: How do you document your architecture? - <AUTHOR> I'm wondering if there are any best practices to document your software architecture design. Related to this, are there are any public examples of a good software architecture documentation? <comment><USER2> I've had to explain our architecture to new hires a while ago, and it was a real mess. I was screen sharing while talking and drwing an ugly graph in inkscape at the same time.

It gave me an idea though, I'm working on a project that'd turn a plain text document (dot language style) into an narrated video made up of graphs showing how things work, who does what when..

Idk if there's a market for that but it's really fun to program anyway :)</comment>


Ask HN: How do I create a chatbot for a specific domain? - <AUTHOR> Something like this - https://looker.com/blog/slack-your-data but much more in depth. For example, parsing questions like<p>"how many men are in NYC between the age of 25 and 35 who are single, like to play tennis and love Chinese Food?"<p>or something like that, assuming I have all the data that is required to run such a query. Are there tools, libraries etc that help process questions like above and help convert them into SQL? <comment><USER2> You're looking at a large amount of NLP and machine learning. You're going to need to do tons of work to implement it for any way of phrasing the questions.</comment>