Storing 25 petabytes of Megaupload data costs us $9,000 a day - <AUTHOR> http://arstechnica.com/tech-policy/news/2012/03/isp-storing-25-petabytes-of-megaupload-data-costs-us-9000-a-day.ars?clicked=related_right <comment><USER3> I've dealt with e-discovery sets. No one really has answers to what to do when you have a litigation hold on data. Legislation commonly requires "retention of anything related to X case", but how do you know what's relevant and what isn't? When you are a third party the ambiguity increases. So you end up with an _everything and kitchen sink_ data dump. Even with _everything_ the data is commonly useless without context. You have files without access logs and logs referrencing local namespaces etc...

With a 25 petabyte discovery, I'm not surprised that everyone's scratching their heads on what to do next. This isn't just an MPAA/Megaupload problem. Even a smaller dataset like a 10-20TB discovery has numerous problems. Hosting/indexing/classifying/reviewing millions of documents is an open issue for the legal field. What do you do when there are multiple parties who all need to see "everything"? If everyone does their own thing how do you reference materials in a consistent manner across the interested parties? If you all agree to host the data in a neutral place who pays for it? What if the technology of that host benefits one party at the expense of another?

For years the legal field has had a "print it all out and have a team of paralegals go over it" viewpoint. Clients don't pay for computers, but they do pay for paralegal hours. Only recently has that become untenable. Discovery sizes are growing exponentially per year. It's common to have a new discovery set come in larger that every previous set combined, and the legal industry doesn't really know what to do about it.

<comment><USER4> If it's anything like the usual proceedings, the lawyers involved would probably prefer if they had several copies of the data, too.

I'm sure most of these drives were arrayed in such a fashion where they're unreadable unless in the proper equipment. It's not like you can just buy a pile of off-the-shelf external drives and start copying, either, as the contents might be unreadable unless the proper software is installed and configured correctly.

<comment><USER5> Not to mention that it would cost (1Tb * $109 @ newegg) * ( 25,000 Tb) $2,725,000 just to replicate it.

<comment><USER6> I'm sure you could get a bulk discount though.

</comment></comment></comment></comment><comment><USER7> The federal government does have a process for this sort of thing, if they seize an alleged drug dealer's house and that house has a mortgage the United States Marshals Service will pay the mortgage. If they seize cars, furniture, other assets the government is responsible for the storage of those items until the case has been resolved. [1]

I would guess that MegaUpload's lawyers will make the claim that the data on those servers is critical to their defense and must be maintained. That is probably an accurate claim, DotCom will want to present evidence of compliance with DMCA notices, counter the claim that a "majority" of the content was under copyright, etc. Best case for DotCom would probably be that his lawyers argue for retaining the data and the judge lets Carpathia destroy it anyway. That would give DotCom reasonable grounds for appeal.

[1] <http://en.wikipedia.org/wiki/Asset_forfeiture>

<comment><USER8> You hit it on the head. I used to have to deal with alot of litigation-related preservation of data... we started calling discovery "Mutually Assured Destruction".

The lawyers basically try to make things more and more onerous in order to encourage a settlement. It's amusing, as long as you aren't accountable for the data!

</comment></comment><comment><USER9> It sucks, but that's the price of doing business. They chose their customer, and now are (unfortunately) tied to consequences. Same thing happens to building owners who have a crime committed by a tenant, the leased space becomes a crime scene until the police/govt are done with their investigation.

<comment><USER10> But instead of just losing use of those servers, they have to actively maintain them. It looks like they just want someone to pay for the $9000 per day that it costs to keep the servers running, they aren't looking for money that they lost from not being able to use the servers.

To go with your analogy, sure building owners aren't allowed to rent the space back out, but they most certainly not asked to pay for usual water, gas, or electricity bills (because they aren't/shouldn't be being used).

<comment><USER11> Why are the servers still running? Surely it's easier and cheaper to just turn off the servers, power down the racks and lock the cages?

<comment><USER12> I'm wondering this, too. The data would still be there, and they wouldn't have to worry about maintenance costs at least, right?

<comment><USER13> Shutting them down could easily destroy evidence depending on how it's done. EX: Many redundant systems try replicating data from machines that are taken off line. What happens when the full network is taken off line quickly has probably not been tested.

<comment><USER12> I see, so even just shutting down the servers hosting MU's data could be damaging to the whole network?

</comment></comment></comment></comment></comment></comment><comment><USER14> So, 25 petabytes ... 25 million gigabytes. Anyone care to guess how much of this data is illegitimate? And how much of _that_ is under MPAA's copyright?

Back-of-the-envelope calculation: Just did a search for "1080" on some unnamed site and it appears a bluray rip of a movie encodes to roughly 10GB. So that would be _2.5 million_ movies in 1080p quality. I don't think we've made that many, have we? Especially if you consider that movies that came out before the "high-definition era" are encoded to about a 10th of that size (700MB-2GB roughly, afaik).

Maybe I'm missing something obvious.

Not counting TV series for instance (are they also intellectual property represented by the MPAA? I'm not in the USA so I never really dug into that).

Movies duplicated in different quality formats are usually a 10th or less of the size of a 1080p Bluray rip as well, as an upper limit I could add a factor of x1.5 for that.

But then, the "long tail" of movie rips are 700-800MB and do not have duplicates.

Unless ... is the MPAA also representing porn? Because then all bets are off and I can easily accept that this 25 petabyte consists mostly of MPAA protected intellectual properties.

But otherwise, what percentage of these 25 petabytes would you estimate actually represents illegitimate data owned/represented by the MPAA? 2% ? 10% ?

Is that fair to the owners of the other 90% of the data? Even if it's probably mostly porn? (I'm fairly sure most of the data has to be porn)

I'm just wondering. Also because it's interesting to speculate what could be in these 25 petabytes. If you have a better guess I'd love to hear it :)

<comment><USER15> Think of the problem from the opposite direction. What the hell _else_ could the bulk of that possibly be?

I've got 4TB of storage on my media server, last time I checked at ~60% used.At _best_, possibly 0.5% of that is stuff that I've personally created and have copyright over. Hell, all the email I've sent _or recieved_ that wasnt spam filtered since mid 1995 only comes to a few hundred meg - <USER2> attachments! Smething less than half a TB of it is music which I have some kind of right to have as digital files (some of it purchased as files, some of it ripped from cd and vinyl - which is somewhat less clear legally with respect to my rights to have a "copy" as a file on my hard drives). Realistically, outside of academia and industry (who presumably aren't significant users of MegaUpload) chances are so close to 100% that any 1GB+ file is copyright encumbered in a way that gives the MPAA an interest that it doesn't matter. The nearest I cold come to justifying the rest is that some of it it "time shifted" TV (from the PS3 TV tuner/PVR), some of it is DVD backups for discs I own, some migh euphemistically be referred to as "timeshifted DVD rentals or loans", but a _lot_ is copyrited content found on "channel BitTorrent" or downloaded from YouTube. If the copyright police confiscated _my_ hard disks and catalogued them in front of a judge, I'd have a very hard time looking him in the eye and saying "I didn't think I was doing anything wrong!"

I fear that line of argument bodes badly for dotcom…

<comment><USER14> Just because you don't happen to have activities that generate enormous amounts of data doesn't mean nobody else does.

There's also people that make a lot more video data than you apparently do, same for sound recordings. Do you believe that the majority of recorded video data in the world is MPAA's? That would go against everything we know about the "long tail". You do realize that of all text (or books) written in the world, less than 1% actually gets published? Why would video or audio be any different? Just because _you_ don't produce it, doesn't mean the MPAA industries are the only ones that do.

Also as you say, there's academia, PHD students I know use equipment that generates gigabytes per second. Or without equipment there's computer programs that do it from calculations. Now I agree it doesn't seem likely they'd use megaupload for that.

And that's just a few things I can come up with right now. Who knows what sort of computer stuff other people do that generates retarded amounts of data they need to share?

I'm also not saying that all that copyrighted stuff isn't there, it's just that if it's _25 petabytes_ worth of data, it just doesn't add up, the MPAA- represented copyrighted part of those 25 petabytes can only be a tiny fraction of that amount.

<comment><USER15> " … it just doesn't add up, the MPAA-represented copyrighted part of those 25 petabytes can only be a tiny fraction of that amount."

Interesting. My completely uninformed assumption is diametrically opposite to that. I'm guessing there's not much sophisticated de-dup going on, and _way_ more of the diskspace on MegaUpload is probably various bit-wise different rips of the same smallish set of Hollywood blockbusters. And while I agree the long tail suggests there's almost certainly lots of people out there with lots on non-mpaa-copyright encumbered files - I'd be quite surprised to find the area under the "long tail" was withing 2 or 3 orders of magnitude of the "fat head" occupied by all the copies of all the dvd rips and broadcast tv recordings.

I wonder if there's any believable data anywhere to see whether I'm wrong?

(Note: I've got a non-US-centric view of this too, here in Australia internet connection plans lag behind the US in terms of speed and bandwidth caps, so even though I've got friends who generate lots of GoPro footage for example, but they'll in general be storing them on locally attached harddrives, not trying to push gigabytes of raw data out into "the cloud". That might explain why I make tghe possibly-incorrect assumptions that I do…)

</comment></comment></comment></comment><comment><USER16> In civil or criminal asset forfeiture, the state can conceivably confiscate property if used for or if it enables a crime. In some jurisdictions it doesn't even matter if the owner of the property and the criminal have really nothing to do with each other. (i.e. Your stolen SUV was used to rob a liquor store.)

Also, the government could have probably seized everything anyway as evidence. The problem with that is setting up that much rack space and network infrastructure isn't cheap.

That's Carpathia's basis for compensation. They are providing a service to the government. Seems like a no-brainer.

</comment><comment><USER17> Help me out with the math here:

1 terabyte costs them $128.41 per year, right?

Amazon S3 would cost them roughly $444 per year, if they were using the Reduced Redundancy Storage.

The cheapest HD that I see on pcpartpicker (in terms of Price/GB) is the Western Digital Caviar Green 2.5 TB (5400 RPM) for $135.43, which is $0.054/GB. That's $54.17 per TB.

If you want a single backup, that's $108.34 per TB. Two backups (3 copies of each file), is $162.51 per TB.

So, if I'm doing this right, as long as their HDs last at least 15 months, on average, they have triple-redundancy, and the cheapest price ratio for consumer hardware. And I'm not even counting their power, network, cooling, or puny humans to maintain it all. That means their HDs, if they were made out of the cheapest parts I could find, would have to last significantly longer than 15 months, on average.

They're actually doing really good on price, if you ask me.

Or am I missing something obvious, or doing the math horribly wrong?

<comment><USER18> The hardware cost 1.25 million. It cost $9000/day in electricity/connectivity/rackspace.

<comment><USER19> True. I'm honestly surprised they didn't declare daily depreciation, for $1.25 million in assets that are obsoleted by new technology at Moore's pace, I'd expect that could arguably be quite high.

</comment></comment></comment><comment><USER20> Its the storage disks the government need, not the rest of the server hardware, if they cant come up with an agreement then shutdown all the servers, take out the disks, catalog, and put in a warehouse somewhere. They are now free to re-use the rest of the server for something else.

That would satisfy the needs of the government if they need access to the data, preserve it if in the future people are allowed to download it, and prevent the MPAA from complaining that it was given back to Megaupload.

I'm sure the cost of storage would not be minimal, but they could still use the rest of the hardware and not have to keep the servers powered up.

Possible problems:

\- Maybe the servers cant be shutdown and brought back up without certain passwords or encryption keys

\- Labor cost of shutting down and catalogging all those disks ( if done progressively would probably work )

\- Others?

<comment><USER21> You want to pull 1,103 server's disks?

The compatibility problems you have trying to get data off a disks in a hardware raid make it impractical (do you have the EXACT SAME version hardware revision & firmware; without this you can't guarantee you can read it back)? Its either that or you have to pay data recovery guys to rebuild it.

Not to mention hard drive costs are still high, post thai floods. For enterprise gear we are getting most quotes ~ $300 AUD a disk, for consumer gear its ~ $130. Most servers are running at least 2 disks... that $286K worth of disks alone if your talking cheap - low capacity disks, not including labour to change the disks and test the hardware before you deploy a workload to it.

<comment><USER22> >The compatibility problems you have trying to get data off a disks in a hardware raid make it impractical (do you have the EXACT SAME version hardware revision & firmware; without this you can't guarantee you can read it back)? Its either that or you have to pay data recovery guys to rebuild it.

Forgive me for for sounding like a member of Anonymous but..

So? That's the government's problem. I don't see why a private company should be in any position where they're required (at wallet or gun point) to help in an investigation at their own expense. Pull the drives, warehouse them, and let the FBI do what they have to do. They have IT to rebuild the RAIDs.

<comment><USER23> Carpathia wants to be paid $9000 a day like they were before the Megaupload case started and pulling the drives and telling the FBI that it's their problem doesn't do that.

<comment><USER22> No, but it stops the costs from accruing. Legal action could then be brought to recover their other costs.

</comment></comment></comment></comment></comment><comment><USER24> This is why the US Government shouldn't have seized the site first, and asked questions later. They should've filed a trial against them, and let them keep hosting the data, and if found guilty, _then_ take it down.

<comment><USER25> I agree, standard operating procedure when dealing with digital information should always include a generous window for destruction of evidence.

<comment><USER26> I disagree. Servers should be inaccessible, but not at all accessed by the law enforcement agency coming up with claims in court - why would they need to look at that amount of data anyway and why should tax payers suffer.

Just - put them behind bars. I said so.

</comment></comment></comment><comment><USER19> The urgency is because Carpathia's lease has run out, they can't stay at the $9k/day facility.

Carpathia has to pay $65k to move the servers, then $37k per month to keep them in a climate controlled facility while powered down. Lost profits are still a relevant consideration. This is a doozy of a damages calculation. What's depreciation on assets that are rendered obselete by (something like) Moore's law?

I'd say Carpathia deletes the data and then supports the petitioners (those with lost data) in the takings clause case against the government. Carpathia claims indemnity against claims by pointing at MegaUpload and the Feds, but probably gets joined in a bunch of messy lawsuits. Real roll of the dice.

</comment><comment><USER27> Why are options that would destroy any chance at Megaupload conducting business in the future even on the table before a trial is finished? I suppose a large amount of damage is already done, but it would be a gross injustice to kill their business before anything started. The government should pay to keep this up until they've conducted their trial. If they don't, and they lose somehow, I hope they get hit with a massive countersuit.

</comment><comment><USER28> For reference, this amount of data would require 190 backblaze storage pods ($7,384 for 135TB) totaling $1.4 million.

<comment><USER29> Is that pre- or post- Thailand flooding prices? (Have drive prices settled down again?)

And is there any redundancy included in your figures? (25 petabytes / 135 terabytes == about 190 pods)

<comment><USER28> Backblaze quoted this price before the flood at $120 for 3TB. I'm not sure if post-flood prices are back to these prices or what range of prices you can expect at this volume.

This figure doesn't include redundancy. Backblaze uses raid6, so the usable capacity is actually 117TB per pod[1]. With this configuration the final cost should be closer to $1.6 million.

      [1] http://hardware.slashdot.org/comments.pl?sid=2341206&cid=36834390

<comment><USER30> > _$120 for 3TB_

So even if you throw out the rest of the hardware and have zero redundancy, $40/TB * 25000 terabytes = $1,000,000.

(Of course, somebody has to ship and store over 8,000 HDDs, too.)

</comment></comment></comment></comment><comment><USER31> Can someone more familiar with this stuff explain why Carpathia's still paying for "power and connectivity"?

I would have assumed that the FBI would have actually seized the servers, or at the very least pulled the network cables out.

<comment><USER32> Not if the evidence is on the servers.

</comment></comment><comment><USER29> I'm really confused by this. Is Megaupload (or any megaupload employee) facing a criminal trial? How can any "evidence trial" (or whatever they call it) be maintained if a law-enforcement agency doesn't have the drives?

Have any hashes been taken of the drives?

</comment><comment><USER21> The costs of moving that amount of data is crazy. I am surprised the government hasn't seized the hardware, and chucked it in a warehouse.

I did some back of an envelope calculations... and its absolutely crazy. Tape would require over 17,000 Ultrium tapes. Now you could De-dupe... but the hardware to process and dedupe that much data.... not really an option. Not to mention the time to write that many tapes...

Something like thumpers (48 disk sun x86 boxes) would be expensive, last time I looked they were around say $30k for a large order... 160tb usable assuming 4tb disks are the thumper is split into 4 Raid 6 arrays... thats 160 thumpers... 4.8 Million

Even backblaze pods would likely be well over a Million...

This doesn't even cover hosting costs, transfer and such. Not to mention to be usable in court there are going to have to be processes in place to document compliance and validity of the copy....

All in all not a great place for Carpathia to be in.

<comment> Tuna-Fish > Now you could De-dupe... but the hardware to process and dedupe that much > data.... not really an option.

FYI, the data is already comprehensively de-duped.

<comment><USER21> I know they dedupe on the file level, but I wonder if they are doing block level deduping... as without a big shared storage infrastructure block level deduping becomes pretty hard to serve at high speed from as the reads potentially become distributed across hundreds of nodes...

To build out web scale systems you generally use commodity gear and accept the overhead of duplication, heavy deduping requires massive IO, and there is no way i can see you can be dealing with that much data have that level of IOPS and be profitable charging what they charge.

</comment></comment></comment><comment><USER33> This post really hurts my soul.

Can Carpathia sue the Federal Government for NOT seizing assets. It's the data, not hardware. Data is transferable. They want it, take it.

Can Carpathia sue? This kind of injustice just makes me boil.

</comment><comment><USER34> Pardon my ignorance, and this is a serious question, but why can't they just turn them off? I realize it doesn't address all the costs, but surely it could reduce them significantly.

<comment><USER35> While I'm obviously uninvolved, there are several reasons why turning them off might be a hassle. Primarily is that if anything is encrypted, the encryption key is currently in memory and the disks are already open - turning the servers off might require reentering details.

Alternatively, in a setup of this size, I imagine there'd be no end of redundancy configurations - RAID for individual disk sets, DRBD (/a SAN equivalent) across servers - turning them off would turn all that HA tech off. Meaning that, when you try to bring the system back up, the redundancy implementation might say "oh no, I've lost x peers from my set of n" and fail itself completely.

Shrug. I've no doubt explained it badly, but there are good reasons to keep them running. It's not just a case of "pop the hard drive out and use it elsewhere"; the logic associated with keeping 25 petabytes of data would also have to be restored to its current state.

</comment></comment><comment><USER36> This gives an idea of the economic activity generated by services like megaupload and what is being removed from the economy by killing the company. Roughly $3.2 million in hosting fees, and could be more if that's just the cost price. Also salaries, over $1 million in hardware, and the various other suppliers. One wonders about the GDP of the recording and movie industries relative to the businesses they're going after.

</comment><comment><USER37> What's the big deal? Just delete the data. Customer pays for storage. Company stores. Customer stops paying for storage. Company deletes.

Sure, a bunch of pissed off people will certainly be upset - but it's not the company's fault - they shouldn't have to bear this burden. I can't see how they could be sued by users for this, they didn't enter into any kind of agreement with the users, only with the customer.

<comment><USER38> The article also mentions that they need to hold onto the data because it may be used as evidence in the court case.

<comment> brown9-2 It boggles my mind that the court hasn't taken position of the hardware/data then.

If the hardware is left running, and not in official custody, how do any authorities know that the data isn't being tampered with?

</comment></comment></comment><comment><USER39> I can definitely understand the lost potential revenue of having unused servers. But I wonder why they are saying that cost includes power and connectivity for the servers? Seems like they would be powered down. I would actually have assumed the servers to be confiscated and taken off-premise by the FBI.

<comment><USER40> It's possible (though I'd doubt it is the case) that those servers have data from other clients as well.

<comment><USER41> Any large datacenter has storage allocated in complex ways. IT may be challenging to isolate one customer's data physically from another. For instance, physical disks can be virtually concatenated then repartitioned into virtual storage containers, which may reside on part of 1, or parts of many physical disks.

They could of course migrate all of the contested data onto new storage. But its large; who would pay for that?

<comment><USER39> That makes the most sense to me.

</comment></comment></comment></comment><comment><USER42> There are legal obligations for the government to reimburse telco companies if they are asked to spy on their customers on the governments behalf. Also, obviously, if you want to use data as evidence in a trial, it needs to be stored safely by the police and sealed off, to ensure that its integrity is preserved.

So either the government needs to pay up, store the drives themselves or dismiss these thousands of harddrives from the witness bench.

Also, I cant see how the EFFs claim has any legal merit. Theres no obligation for a site to enable you to access data you sent them.

</comment><comment><USER43> Megaupload had a lot of assets that were frozen. I don't know about the legailities, but it would be reasonable to use frozen funds to pay for this.

<comment><USER44> I really doubt that would be legal. It would be akin to the government can forcing you to pay for them to prosecute you.

Of course, I think the whole category of forfeiture law it blatantly contrary to the Fourth Ammendment, so what do I know.

<comment><USER45> > It would be akin to the government can forcing you to pay for them to > prosecute you.

Taxes?

</comment></comment></comment><comment><USER46> I know any legitimate hosting company would never do this, but it would be amazing if they just "happened" to have very loose security on the servers that hold Megaupload's data, and if some hacker were to..."gain unauthorized access" and wipe all the data.

They wouldn't be held responsible for a breakin, would they?

<comment><USER11> If a court orders them to take "reasonable, and industry best practice computer security approaches to prevent the data being lost", then, yes, they would be responsible for "accidentally-on-purpose" leaving the servers accessible. If it were to happen, someone would do a post-mortem, find out that they intended for it to be broken into, and then they would be guilty of contempt of court/destroying evidence/etc.

<comment><USER46> Yeah, realistically it just wouldn't be worth it. But it's fun to imagine.

</comment></comment></comment><comment><USER47> Are there seriously people that used Megaupload as their sole and only place to store their data? What if there was a fire in the server room? Or some MU intern typed rm -rf /?

<comment><USER48> Believe it or not, there are even people who don't have backups at all. I know, right?

<comment><USER49> Not backing up your important data is passive stupidity.

Backing up your data on Megaupload and then not keeping a local copy is active stupidity of such vast scale that I refuse to believe it has ever happened in the real world.

<comment><USER50> > Backing up your data on Megaupload and then not keeping a local copy is > active stupidity of such vast scale that I refuse to believe it has ever > happened in the real world.

The purpose of having a backup is so that you can restore your data after the original is lost, because there is a good chance the original will be lost. Explicitly retaining it on your local system is not enough to keep it safe. If it was, there would be no reason to backup in the first place.

If a million people used the service for backups, it wouldn't be unreasonable to expect several of them to have a drive failure each and every day.

<comment><USER49> Do you really think a million people used Megaupload for backups, when the service was nearly completely unusable for that task?

Do you think they were zipping up the contents of their own systems and then uploading them daily?

</comment></comment></comment></comment></comment><comment><USER51> Making Megaupload pay this $9000/day seems unfair, too. The US government has cut off all of Megaupload's revenue streams, and so they would be forcing Megaupload to keep paying for a service that they can no longer make money from.

Regardless, why is the cost so high if the server is down? Does this $9000/day reflect the loss that Carpathia suffers from not re-allocating this storage to other customers? It would seem to me that given Megaupload's current state, it would be sufficient to leave the servers powered down and unplugged until the legal dispute is resolved... surely the cost of leaving a server idle is not $9000. I don't really know though...

</comment><comment><USER52> Wait, why do they need to keep the power and connectivity on if they are not being actively accessed? That would same a bunch of money it they were kept off.

</comment><comment><USER53> _"and argues that if that data needs to be preserved, someone else—the government, Megaupload, or an interested party such as the MPAA or EFF—should bear the costs of preserving the data"_

Fucking exactly!! Have fucking MPAA pick up the tab.

EDIT: its going to be amazing (and will take years for sure) to see if this won't bite MPAA in the ass if the judge will rule that yes they do have to pay. Would looove to see that. This should be actually a rule of thumb -- if MPAA believes someone is infringing, court suit is entirely fine, but you guys (MPAA) will pay to keep the light on in the meanwhile.

</comment><comment><USER54> I wonder if this is an opportunity for a startup, and/or an insurance product sold to SaaS end users, hosting facilities, or developers.

</comment><comment><USER55> I don't feel one bit of sympathy towards Carpathia, they most likely had all the warning signs on their door - dmca notices, legal notices and more but they willingly provided service to a company with garbage morals.

</comment><comment><USER56> In my mind, the MPAA is certainly the best choice to pay these costs. They're the ones with the problem, they should be the ones to bear the burden. Especially considering Megaupload offered to take the data and they explicitly forbade it. If the MPAA didn't like the solutions offered, but can't come up with something better, then I think Carpathia should get to do what it wants.

Edit: The opinion above has NO legal basis whatsoever. As many have pointed out, it's not even legally possible. I made this comment solely from a "In a perfect world..." standpoint.

<comment><USER57> The data is being held (from what I understand) as part of an FBI investigation. The MPAA may be responsible for the case existing and bringing the supposed crime to light but the FBI are the people trying the case and this is beyond the MPAA now. The responsibility lies with the government agencies.

<comment><USER58> The FBI, as MPAA's minions, should be the ones paying it. Either their "confiscate" the data or they do not. If they do, they have the responsibility to keep it up, if they do not, they should have to give it back to its "owner".

<comment><USER59> Unwitting puppets perhaps, but I doubt the people of the FBI want very much to be dealing with copyright drama. That's like fixing bugs in form code to them. It must feel like a complete waste of their time and talent.

<comment><USER60> Then they should choose not to do it.

</comment></comment></comment></comment></comment><comment><USER61> At that price, it would only take them 150 days to stop losing money if they started building some Backblaze servers ([http://blog.backblaze.com/2011/07/20/petabytes-on-a- budget-v...](http://blog.backblaze.com/2011/07/20/petabytes-on-a- budget-v2-0revealing-more-secrets/)). 25,000TB / 135TB * $7,384 = $1,367,407 minimum cost of commercial hardware to store that much.

"historically and mind-bogglingly large amount of data" - you could say that again.

<comment><USER62> Did you read the article at all? They can't lease the equipment to anyone else, which is what's costing them money. The servers were estimated to be values at $1.25M, but they are also unable to lease all the space that they are using, which is significant.

<comment><USER61> Yes, I get that. That's why I said "stop losing money." It seems like their only solution would be to transfer the images of the data to cold storage and then find some source to recoup the cost of the cold storage devices (which is why I thought the backblaze numbers would be interesting.)

<comment><USER62> I doubt they are legally allowed to do that. Otherwise it would be a simple matter of a lot of tape drives.

</comment></comment></comment></comment><comment><USER63> Surely it wouldn't take too long to contact both the legitimate users of Megaupload and ask them to download their totally legitimate files?</comment>


Show HN: Hndex.org – a full-text search engine of articles submitted to HN - <AUTHOR> https://hndex.org/ <comment><USER2> I was just telling a friend about a post I had read about buying cheap tools to start out. I couldn't find it at all googling, and the HN search didn't work. I saw this post, searched "cheap tools quality", and immediately found it. Tnank you!

The post in question: [https://www.johndcook.com/blog/2020/07/25/worst-tool- for-the...](https://www.johndcook.com/blog/2020/07/25/worst-tool-for-the-job/)

<comment><USER3> where actually the best happens in the comments under hn submission to this article (in short: the claim seems not valid for certain domains)

</comment></comment><comment><USER4> I love this. The search box had an option to autocomplete terms I had recently searched, I clicked on one (a term I was interested in, obviously) and immediately found an interesting HackerNews post. Nice!

I think HackerNews and other link aggregators (e.g. reddit) have a kind of recency problem, where there is a lot of great content, but people only see the recent stuff. This seems like a great way to uncover some of the latent value of old HackerNews content.

Is there a way to suggest content too? e.g. If you liked that, you'd probably also like X, Y, and Z.

<comment><USER5> >I think HackerNews and other link aggregators (e.g. reddit) have a kind of recency problem, where there is a lot of great content, but people only see the recent stuff.

With Reddit and HackerNews, I want a relative ranking index. I can search by the top content, but something 5th today could have more votes than the top submission of 2015 because of forum/subreddit growth.

I want them ranked by something like the ratio of views to upvotes or upvotes compared to total upvotes for that day.

<comment><USER6> > I want them ranked by something like the ratio of views to upvotes or > upvotes compared to total upvotes for that day.

Yeah, this is definitely possible with public data. I did something similar on reveddit [1] for removed reddit content. Hovering over the graph shows the item with the highest vote ratio [3], and clicking skips to that point in time. Code is here [2] for anyone interested and I apologize in advance..

[1] [https://i.imgur.com/p3Bi5IS.png](https://i.imgur.com/p3Bi5IS.png)

[2] [https://github.com/reveddit/ragger/blob/master/revddit_aggre...](https://github.com/reveddit/ragger/blob/master/revddit_aggregator.py)

[3] [https://www.reveddit.com/r/worldnews/?showFilters=true](https://www.reveddit.com/r/worldnews/?showFilters=true)

</comment></comment></comment><comment><AUTHOR> I spend a lot of time trying to find articles I once read on here via Google with...

      site:news.ycombinator.com some term

...but a many times "some term" is not in the title of the HN post, but in the body of the article. I didn't see any other tools that did this for HN.

<comment><USER7> There's also Algolia, [https://hn.algolia.com](https://hn.algolia.com)

<comment><USER8> It doesn't index the article contents though; only the story titles and comments, same as Google.

<comment><USER9> How did you come to this conclusion with respect to hndex?

Most articles have no comments. Yet one can search on hndex for terms only found in the article, not in the title.

<comment><USER8> I wasn't really saying anything about hndex, apart from that it indexes the contents of external articles submitted to HN, as you said.

The correction I was making was about HN Algolia search (which is linked from the search box at the bottom of each HN page), which only indexes content on news.ycombinator.com itself – i.e., article titles, comments and text-only posts like Show/Ask HN – but not the external content in submitted articles.

</comment></comment></comment></comment></comment><comment><USER10> So the first term I searched was "cognitive". Search for it yourself, and click the first result.

Go back and click "cached".

That was a cool experience.

<comment><USER11> I wonder how cached results are created and is there some kind of copyright violation because they store the content.

<comment><USER12> I suspect it might be something mentioned in [https://github.com/masukomi/arc90-readability](https://github.com/masukomi/arc90-readability). I believe it's what powers "readability" view in modern browsers (firefox [https://github.com/mozilla/readability](https://github.com/mozilla/readability), safari).

</comment></comment></comment><comment><USER13> That's awesome! I searched for MuleSoft, not expecting anything in particular, and found a random fact (at the end) that certainly wasn't in the title of the article.

Bug report: in Safari on dark mode, the text in the search box is almost the same color as the background (white on white).

From "GitHub was also talking to Google about a deal, but went with Microsoft instead" [https://www.cnbc.com/2018/06/05/github-interest-from- google-...](https://www.cnbc.com/2018/06/05/github-interest-from-google-and- others-revenue-about-300-million.html?__source=twitter%7Cmain) \- Salesforce got MuleSoft for $7.5 billion, Microsoft got GitHub for $7.5 billion.

</comment><comment><USER14> I searched for "red light therapy" and none of the articles matched, not even the most recent article on red light therapy that was on the homepage. Same for "red light".

</comment><comment><USER15> It's fast and light weight! What stack are you using?

My only wish is that it should be possible to sort chronologically.

</comment><comment><USER16> Neat project. It's a bit depressing how many of the article links don't work anymore. For example, only the official signal vs noise article works on the first page of results for Basecamp: [https://hndex.org/?q=Basecamp](https://hndex.org/?q=Basecamp)

How does ranking work? Apologies if I've missed the explanation.

<comment><USER17> Articles which have been posted to HN often end up in the archives of Archive.org and Archive.is in my experience.

The OP site also has a "cached" link for each article, don't know if you saw that.

Also, to the maker of HNdex, consider adding links to Archive.org and Archive.is next to the cached link you have, so that readers can click through and check if they have a version of it in case there's images etc

<comment><USER18> Nice.

HN feature request: Add submissions to archive.org (or equiv), include that cache link with story.

</comment></comment></comment><comment><USER19> What does this offer over [https://hn.algolia.com](https://hn.algolia.com)?

(Algolia was backed by YC btw)

<comment><USER20> Looks like this search engine searches the _articles_ and links that have been shared, whereas hn.algolia.com only searches title, author, and the text of a post if it is a text post.

</comment></comment><comment><USER21> Ohhhh I first thought this was a search engine for hacker news and I was wondering why the hell anyone would reimplement this, but this is fricking cool. Tbh I wanted to implement something similar for more than hackernews, but this is a nice thing to have. I especially love that you implemented a cache :)

Seems nice. Sort and filter functionality would probably add to this, but I will bookmark this for sure and try it out as a search engine for tech topics in general.

Thanks for creating this!

</comment><comment><USER22> May I suggest you add time as well to the search results? It's super relevant in the context of news and technical articles.

</comment><comment> arc-in-space Holy crap this is fast! At first I was mildly disappointed it wasn't a live search like hn.algolia.com, but I was seriously not expecting the results page to load so quickly. I guess goog et al. trained me to expect garbage latency in internet search forms.

This seems to be a good complement to algolia, given that it searches through the linked pages instead of comments.

Minor nitpick: would it be possible to make it give a 'past' link, to search for all discussions on a result? Some of the 'comments' take you to duplicate posts with no comments instead of the more popular cases.

</comment><comment><USER23> I wrote something similar to this a year ago (though shabby looking by comparison!): -

[http://kakapo.susa.net:8080/cfs/](http://kakapo.susa.net:8080/cfs/)

I abandoned it when Google deprecated the WebRequest API, but the code's still available on GitLab. [https://gitlab.com/ksangeelee/cfs_build](https://gitlab.com/ksangeelee/cfs_build)

It allows article score and uBlock Origin 'hits' as search criteria.

<comment><USER11> Very cool implementation. I am wondering have you considered using something like [https://github.com/cliqz-oss/adblocker](https://github.com/cliqz- oss/adblocker) which can sit on top of headless browser and do not require bridging to an extension.

</comment></comment><comment><USER24> This is neat!

\- What is your stack?

\- How often is your database updated?

\- And will it be open source in the future?

</comment><comment><USER25> This is fantastic. Way better than the Algolia search which seems to be the most "official" one. I love that you cached the article content too.

</comment><comment><USER26> What is the criteria for an article getting into the index? To test it out I tried several of my submissions and could only find one in the index.

</comment><comment> wh-uws this is great whats the stack / architecture you built this with? Would love some insights.

I often find myself searching HN for opinions on different technologies over time and this will be invaluable for that use case.

</comment><comment><USER27> So many articles found that are gone from the original sites, but still readable from the cached version, that's pretty useful!

</comment><comment><USER28> Nice tool, though I tried a few searches, and a few searches of <same text> \+ ' hackernews' on google, and I gotta say I like the google results better. Search options for comments/titles/users, date range, ask/show/jobs, sorting by date/votes (weighted?), etc would be nice additions.

</comment><comment><USER29> Love it, love it! Thank you.

Is it possible to give a little inside peek of the techniques, algorithms and tech stack etc. you have used?

</comment><comment><USER30> Tried to find an article from earlier this week, no dice. [https://hndex.org/?q=open%20source%20security](https://hndex.org/?q=open%20source%20security) [https://hndex.org/?q=openssf](https://hndex.org/?q=openssf)

Great idea, I appreciate how fast it returns results! Just needs more more control over the search parameters and figuring out why articles like the example I posted above aren't working and you got yourself a nice HN search.

</comment><comment><USER31> Here we are in 2020, and web search is not what I'd hoped it would be. I would love to see more niche, small search tools like this rather than better general-purpose search. Building upon that, maybe an aggregator of search results from many small niche search engines. Actually, just writing that sentence reminded me of Searx [0], which I have to admit I haven't tried in earnest but really should.

0: [https://searx.me/](https://searx.me/)

</comment><comment><USER32> There's also Falcon[1] Chrome extension which does full text indexing on your browser history so if you read something and can't either Google it or find it in the browser's history, Falcon will broaden the search scope.

[1] [https://chrome.google.com/webstore/detail/falcon/mmifbbohghe...](https://chrome.google.com/webstore/detail/falcon/mmifbbohghecjloeklpbinkjpbplfalb?hl=en)

</comment><comment><USER33> Well done. One nit...if you enter something it doesn't like, the error that pops up isn't helpful: _" Please match the requested format"_

</comment><comment><USER34> Found a bug! Searched for my name and found a bunch of unrelated BoingBoing links.

[https://hndex.org/?q=Terence+Eden](https://hndex.org/?q=Terence+Eden)

Why? Because those old posts have a "what's new" set of links. One of which contains my name. I'd suggest only searching the `<main>` element, perhaps?

</comment><comment><USER35> Thanks for making it. great resource. Two things:

1\. Show dates in results.

2\. Are results better than searching on Google with "site:news.ycombinator.com"

<comment><USER36> I think it indexes the content of articles referenced on hachernews which makes it different from Google with "site:news.ycombinator.com".

</comment></comment><comment><USER37> This produces such a unique set of results for things.

Fantastic project and well worth creating!

As another commentor has noted - this totally disobeys recency bias and throws up interesting articles for a topic.

Edit - it is disappointing how many of these links 404. But even if that's the case the headline and intro is a set of time capsules of sorts nonetheless.

</comment><comment><USER38> Could you add post points and how much time since/when was the article posted? It would help as some topics do not age as well as others. Having sorting (most popular, most recent) and range search (last year, custom range) would also help those who want to narrow their search.

</comment><comment><USER39> I love the option to view cached versions of websites. This is a nice project. Thank you, Mcovalt.

</comment><comment><USER40> How does the "full-text search" work? I tried to search my previous post[1] but no luck, even tried the exact title.

[1] [https://news.ycombinator.com/item?id=22830472](https://news.ycombinator.com/item?id=22830472)

</comment><comment><USER41> can i sort by date? i searched for kubernetes and the first page results were from 2016.

<comment><USER42> Yes, with so much context missing from the results it's pretty confusing.

Ideally for this to be useful I'd want date on HN, date of article (although I see that'd perhaps be hard to extract from unstructured pages) and also the HN points, as those are a major proxy to quality usually.

As others have said it has a nice crisp UI and i like that it's so quick.

</comment></comment><comment><USER43> What stack did you use to make this?

Ps: show the date / age of the posts on the results page

</comment><comment><USER44> I would love to see the source and/or a write up about the stack used here. I've thought about making something similar, but got distracted while in the planning phase.

</comment><comment><USER45> Very cool! How frequently does it index the HN content? I've just tried searching for DevComrade which I submitted 14hrs ago and nothing shows up.

</comment><comment><USER46> Beautiful! Please consider adding a dark mode option.

<comment><USER47> I don't know why but to me the background is #000 black with #fff text by default...

It's such an intense "dark mode" that I couldn't read the first full article of interest I found because the contrast was killing me, and then coming back to regular HN left me nearly blinded for a few secs

</comment></comment><comment><USER48> I would love to search books, video recommendations that people post on HN. It's one of the cool things I like here

<comment><USER49> There is this site / newsletter for books mentioned in hn comments

[https://hackernewsbooks.com/](https://hackernewsbooks.com/)

There are much fewer books in the last six or so weeks editions. Don't know why this is. Whether it is some scraper not working, or actually no books being mentioned.

</comment></comment><comment><USER50> This is absolutely fantastic, is there any way to display the dates on the cached version and/or result?

</comment><comment><USER51> How does a site which can index text inside a document usually work? Is it similar to Linux grep?

</comment><comment><USER52> Hi Matt, can you provide any details on the code/backend?

</comment><comment><USER53> Wow - would love an RSS feed feature to subscribe to a search

</comment><comment><USER54> Would you be willing to provide an API for this service?

</comment><comment><USER55> awesome! would be great to have a submissiondate/votes/num comments/link to the hn submission in the results!

</comment><comment><USER56> this is really cool, now I can filter by topics I really care about, well done!

</comment><comment><USER57> I'm worried making it public will encourage companies to "SEO"ify HN.

</comment><comment><USER58> Does this work with articles behind a paywall?</comment>


FEMA Contract Called for 30M Meals for Puerto Rico. 50,000 Were Delivered - <AUTHOR> https://www.nytimes.com/2018/02/06/us/fema-contract-puerto-rico.html <comment><USER2> >For this huge task, FEMA tapped Tiffany Brown, an Atlanta entrepreneur with no experience in large-scale disaster relief and at least five canceled government contracts in her past. FEMA awarded her $156 million for the job, and Ms. Brown, who is the sole owner and employee of her company, Tribute Contracting LLC, set out to find some help.

>Ms. Brown, who is adept at navigating the federal contracting system...

I'm guessing that is why she got the contract despite having a poor track record and insufficient experience. She's good at playing the contracting system.</comment>


GE 'badly' misjudged the clean energy transition, costing investors almost $193B - <AUTHOR> https://www.utilitydive.com/news/ge-badly-misjudged-the-clean-energy-transition-costing-investors-almost/556420/ <comment><USER2> That's quite a stretch to blame everything on GE's Power division. I've been following GE for a while and the problems at GE go back many years. At the heart of the problem isn't its Power division (which is bleeding money) but GE's Capital division, which has a lot of liabilities still from the 2008 recession, its opaque finances, and its massive pension liabilities. The Alstrom acquisition (cost GE $10B) and being late to the wind turbine game can't explain $193B loss in market cap. It was basically really badly managed by Immelt for a decade. GE used tens of billions to do stock buybacks rather than pay down its liabilities.

The clean energy transition is just one of many problems for GE and isn't even its most urgent. Companies like GE make bad decisions like that all the time but it takes a series of bad steps and mismanagement to push a conglomerate like GE to the brink.

That said, I still think GE has a lot of very valuable assets that generate a lot of cash flow and things are starting to turn around. The massive amount of cash it got from offloading its biopharma division will give it a bit more runway so it doesn't have to sell its best assets (aviation, healthcare, etc.) to finances its bad debts.

Edit: Full disclosure -- the reason I know about GE's issues is because I've a long position on GE. So please read my comment with the view that I'm biased. Otherwise, I wouldn't be going long on GE.

<comment><USER3> > GE used tens of billions to do stock buybacks rather than pay down its > liabilities.

Is this such a bad plan if equity funding is expensive but debt funding is cheap?

If you can secure a loan against your assets at 3% and buy back preferred or common shares paying 5%, then it can make sense. It's one way to avoid a dividend cut while decreasing dividend expenses. You can always issue the shares again later.

<comment><USER2> At a high level, I don't disagree with the idea as a whole but the problem with a plan like that is when you're facing a liquidity shortage, your shares may not be worth a lot. The problem with leveraging debt is that it can also compound and magnify your problems. Especially in the case of GE, the huge debt caused its debt rating to go down and that made the cost of borrowing higher, thus compounding the problem. If GE had paid down its debts, its book value would have gone up and eventually that would drive up its share prices (eventually... in a rational market).

</comment></comment></comment><comment><USER4> > _E is "a case study in how rapidly and unexpectedly the global energy > transition away from fossil fuels travels up the economic chain and destroys > value in the power generation sector," the report says. It points the finger > at large shareholders, like Vanguard, BlackRock, State Street and Fidelity, > who IEEFA analysts say should be doing more to push companies away from > fossil fuels._

This is interesting way to phrase things? It's the fault of the index/hedge funds, who likely did not have any say, for the reason that the executives running GE weren't forward looking enough?

<comment><USER5> It's absolutely not the role of index funds to dictate strategy to the companies they own. The whole premise behind an index fund is investors don't know which strategy is best, so just buy everything.

Where funds could have an impact is tying more of management compensation to long-term results. Longer vesting schedules on stock compensation would be an easy place to start.

<comment><USER6> Theres an observer effect type thing going on either way. If they invest in certain behaviors, companies will start behaving that way to get investments.

At the end of the day, the investors have agendas (the large stakeholders agenda is generally global stability aka lack of disruptive change) and they invest in ways to achieve said goal. GE fit that model AND their collapse is a great illustration of why attempts at safe passive investing eventually lead to stagnant and obsolete investments.

</comment></comment></comment><comment><USER7> A lot of people seem to have misjudged the clean energy transition. Here are the International Energy Agency 'World Energy Outlook' projections and reality [https://twitter.com/AukeHoekstra/status/866313289306963969/p...](https://twitter.com/AukeHoekstra/status/866313289306963969/photo/1)

It kind of fits with what Kurzweil bangs on about, about people thinking linearly and misjudging exponential growth. I hope this exponential continues as it would be a major plus for combating climate change. Here he is in 2011 saying solar will power the world around 2027. Which sounds a little optimistic. We shall see. [https://bigthink.com/think-tank/ray-kurzweil-solar- will-powe...](https://bigthink.com/think-tank/ray-kurzweil-solar-will-power- the-world-in-16-years)

</comment><comment><USER8> [http://ieefa.org/wp-content/uploads/2019/06/General- Electric...](http://ieefa.org/wp-content/uploads/2019/06/General-Electric- Misread-the-Energy-Transition_June-2019.pdf)

Here is a link to the actual report.

This reads like they started with the conclusion, and then kept googlig for facts to support it. Importantly, no where does it connect the energy transition to the number 193 billion. They say a thing happened in 2016 (an acquisition), in 2018 another thing was much worse (GE's mkt cap), so _obviously_ that 2016 thing caused the 2018 thing.

Dissapointing givien the high caliber of donorsto the IEEFA.

<comment><USER9> Yeah kinda reads of green energy propaganda imo. Not opposed to renewables at all (own PEGI).

</comment></comment><comment><USER10> >GE is "a case study in how rapidly and unexpectedly the global energy transition away from fossil fuels travels up the economic chain and destroys value in the power generation sector," the report says. It points the finger at large shareholders, like Vanguard, BlackRock, State Street and Fidelity, who IEEFA analysts say should be doing more to push companies away from fossil fuels.

Big finance is finally waking up to the fossil fuels transition [https://www.nytimes.com/2019/06/11/climate/climate- financial...](https://www.nytimes.com/2019/06/11/climate/climate-financial- market- risk.html?action=click&module=Well&pgtype=Homepage&section=Climate%20and%20Environment)</comment>


Omea: The Integrated Information Environment - <AUTHOR> https://www.jetbrains.com/omea/ <comment><AUTHOR> Created by Sergey Dmitriev, the co-founder of JetBrains.

"I've started Omea project (the personal integrated information environment) at March-June 2003 and manage it now as product manager)"[1]

[1] [http://wiki.c2.com/?SergeyDmitriev](http://wiki.c2.com/?SergeyDmitriev)</comment>


Ask YC: Where do you work? - <AUTHOR> For those of you that aren't working on your own startups yet, where do you work right now? <comment><USER2> I'm working at 1-800-Free-411, a company that's moving through it's post- startup, not yet a midsized company phase.

The company is actually a lot of fun to work at. The product is useful to a lot of people (Advertisement supported 411 Directory Assistance), and it's a great place to work, learn, and play.

What I like best about the company is that it has a very high engineer to manager ratio ;) That's always one of my metrics when looking at new companies.

Before that, I worked at Irrational Games (now 2k Games), as the IT manager of the US and AUS offices, working on Bioshock and other games.

<comment><USER3> How is your company doing now that 1-800-GOOG-411 is out? A lot of Google's side businesses are just for the data but some are out to compete for real. It'd be interesting to hear from the inside.

<comment><USER2> It's great to see Google coming in and validating the market that we're in. The company's doing well financially, and continue to grow callers. It's an exciting place to work, and I'm glad to be here.

When Goog-411 first came out, I was impressed: They do Call-Completion (auto- dialing the number you called), which we don't, and I'll admit it's a nice feature.. It's just expensive to do. (They eat those minutes)

On the other hand, we have live operators, and were Google prefers automation, so there are always tradeoffs. .Our Automation is pretty great, but we know that some people won't get recognized, so we give a Liveop backup, and Google doesn't.

I'm afraid I can't really go into details more than that without breaking agreements.

<comment><USER3> That's enough for me, I'm glad to hear things are going well. While it's hard to compete with Google's pocketbook funding the minutes, I think for a lot of Google services, they have a hard time competing with any competitor that has a human available in loop. People really, really value that and I'm sure you guys are reaping rewards from doing it.

</comment></comment></comment></comment><comment><USER4> I'm a full-time student, a free-lance web developer and designer and marketing consultant, working on my own startup (ratemystudentrental.com), lead guitar for Flint-based rock band, Moment of Inertia, and work at a tier one automotive supplier as an engineering co-op.

</comment><comment><USER5> I work as the IT manager in a distribution company in Trinidad in the West Indies. I've built the company LAN from scratch, with Windows/Red Hat/Slackware/VMware virtual servers/thin clients and the company intranet with Apache/PHP/FreeTDS. We program reports directly in PostScript just for the hell of it. I fall asleep every day in my office and deal of escaping to an all-UNIX environment, just me and a room full of servers. Lord, I hate Windows!

</comment><comment><USER6> I work in a cube every day and strategize/dream of getting out of my cube, sometimes hourly

</comment><comment><USER7> I alternate between being a freelance programmer and a lighting designer for a band. A project I'm working on fuses the two; I'm writing better lighting control software.

</comment><comment><USER8> I work for me, but I do have a little company: <http://www.extravalent.com/> which is really just me.

</comment><comment><USER9> I work from home as an ATG (www.atg.com) Architect. At the moment there is a ton of demand, which lets me do things like work from home. I work on side projects and startups as much as time permits.

I love working from home. My stress level has dropped immeasurably since I left the office buildings of AT&T. No commute, no politics, no spending 5 hours/day trapped in a small conference room. My happiness, health, and waistline have all benefited from the change. Granted some folks can't work effectively from home, but I've found that since I generally really love what I do, it's easy for me to be productive without oversight.

</comment><comment><USER10> Trulia. Someone else's startup. It's amazing, and I highly recommend it.

<comment><USER11> Can we get in touch?

<comment><USER10> Yeah. I'll send you an email.

</comment></comment></comment><comment><USER12> I'm a full time student and working on my startup.

</comment><comment><USER13> Unemployed undergrad looking for an internship

<comment><USER13> I forgot to add that my e-mail can be found in my profile =). Looking for a software engineering summer internship located in NYC or Silicon Valley.

(Hopefully this isn't a faux pas =)

</comment></comment><comment><USER14> Web Developer for CollegeHumor / BustedTees also run priceadvance.com on the side.

</comment><comment><USER15> I am a grad student at the san jose state university and a freelance programmer.

</comment><comment><USER16> I'm a full-time student at the University of Maine, working on my world shattering thesis. I also work for the Department of Sustainable Agriculture and a software development house run by my academic adviser.

<comment><USER17> Cool, I live in Harpswell, work in Portland.

</comment></comment><comment><USER18> Developer for FedEx Services

</comment><comment><USER19> I'm a college student working at my university's tech support desk while doing freelance web design jobs on the side while simultaneously working on my startups. www.playforcleanwater.com

</comment><comment><USER20> I'm working for a startup wireless carrier in Oklahoma. We're working to bring wireless phone service and wireless broadband to rural areas in connection with another major carrier.

</comment><comment><USER21> Nokia Siemens Networks, in Portugal

<comment><USER22> ClusterMedia Labs, Aveiro :D Are you fom ist or feup ?

</comment></comment><comment><USER23> Junior dev at an ecommerce co. I too am plotting my escape.

</comment><comment><USER24> I'm a freelance web developer.

</comment><comment><USER25> At my desk:-)

Home office, Innsbruck, Austria. My company is DedaSys LLC (always looking for partners), and I do a mix of consulting and working on my own stuff.

</comment><comment><USER26> R&D unit of a actuarial consulting firm. Problem-solving and hacking in areas where pensions, investment, finance and technology meet.

<comment><USER27> what kind of background do you have? I'm an EE doing chip design but am looking to get into something more business-oriented.

<comment><USER26> Grew up with computers (started with a ZX-80) but for some reason lost interest as a teenager. Maths degree at university; mainly pure maths, group theory, number theory, topology etc.

Didn't want to do research so sought a mathsy job and joined an actuarial consultancy, advising pension schemes through calculations and mathematical models. Found out that it was less technical than I'd expected and consulting was quite repetitive. Unofficially carved out an hour or so each day to get better at broader finance theory (derivative pricing, corporate finance etc) and programming, and re-discovered why I liked computers so much as a youngster. Not a well-defined career path, but more a case of doing things I found interesting, earning trust, and pushing myself and the firm in different directions.

Then quit and did an MSc in artificial life for a year - unrelated but challenging and great fun.

Then back to the same company, doing similar things. I'm reminded of an article posted here a few months back saying that it's tough to be the best in a narrow field, but being in the top decile in lots of areas is easier and makes you valuable. I'm not the world's best programmer, mathematician, finance theorist, researcher or communicator; but I'm not too shabby at any of them and the combination seems to work well, particularly when coupled with a good knowledge of a particular domain (pensions in my case).

Some parallels to the startup world, even though I work for a big firm. In particular pg's The Power of the Marginal (<http://www.paulgraham.com/marginal.html>) resonates and translates to my own experience.

</comment></comment></comment><comment><USER11> i work at this place called expedited travel, we expedite u.s. passports in 24 hrs and get visas in less than 1 week. (web developer)

</comment><comment><USER28> I work for a small medicial anaylsis and storage company, primary making sure that the client's data is government compliant.

</comment><comment><USER29> Software architect at a small consulting firm.

</comment><comment><USER30> University of Pennsylvania School of Medicine.

</comment><comment><USER31> I'm working for Truviso, a startup in the Valley. We're building a data stream management system on top of PostgreSQL -- essentially taking PostgreSQL, and giving it the ability to evaluate SQL over live streams of data in addition to static tables.

(BTW, we're always looking to hire good hackers -- send me mail if interested.)

</comment><comment><USER32> Freelance web developer.

Previous jobs: research physicist, biophysics postdoc, semiconductor product engineer, and... web developer.

</comment><comment><USER33> I work for Brady Corp in the world of RFID & coach basketball at Cardinal Strich University, Milwaukee, WI

</comment><comment><USER34> ARM Ltd, Leuven office

</comment><comment><USER35> Exceptional Innovation (www.life-ware.com). We design hardware and software for smart homes.

</comment><comment><USER36> I work for Logitech

</comment><comment><USER37> Bug Labs

<http://www.buglabs.net/team?page=3>

</comment><comment><USER38> I am doing business on my own, but am also employed by avionics company Rockwell Collins.

</comment><comment><USER39> I'm a developer for NCEAS, an ecological research center, attached to UC Santa Barbara.

<comment><USER40> Very cool! Are you also a graduate student there?

<comment><USER39> I have a degree in physical geography, but now work full-time on some sweet research projects.

</comment></comment></comment><comment><USER41> I'm on my own startup now, but previously was at a small financial software startup.

</comment><comment><USER42> I have a home office.

</comment><comment><USER43> I work at <http://auctionpal.com> \- a startup in Waltham, MA.

(I'll admit, i only do it for the rails experience)

</comment><comment><USER44> I work at DentcraftTools.com as all around web guy.

</comment><comment><USER45> Unemployed/Graduating this spring/Looking for work

</comment><comment><USER46> Government organization (social insurance) in Saudi Arabia and developing a startup part time. Hope to make the jump some time soon.

</comment><comment><USER47> Full time undergrad student. Over breaks I do network support for a non-profit medical group in the NYC / Mid-Hudson area.

</comment><comment><USER48> At a financial start-up... and I'm looking into a tech startup. Neither is mine, though I am considering that route too.

</comment><comment><USER49> Full-time developer and researcher working on a data mining project for a refrigeration compressor industry, at Brazil.

<comment><USER31> Interesting. Can you talk a bit more about the nature of the data mining project?

</comment></comment><comment><USER50> I am a sys admin at an online bank.

</comment><comment><USER51> I work at Yahoo! in Orlando, FL

<comment><USER40> I too work for yahoo (just an intern)

</comment></comment><comment><USER52> Natural Gas software company

</comment><comment><USER53> Harvard Business School

</comment><comment><USER54> programmer in a gaming company

</comment><comment><USER55> tech support at (aka Hell)

</comment><comment><USER56> nobody works at starbucks?

<comment><USER10> Not lately.

</comment></comment><comment><USER57> startup kid

</comment><comment><USER58> Freelance web developer, designer, programmer, all around technical handyperson.

Hoping to become a startup :]</comment>