Ask HN: Pick a city, any city - <AUTHOR>

So, hypothetically... let's say you find yourself a divorced father, nearly 40 years old, who has sole-custody of a seven year old son(his mother just moved 1000+ miles away), you are a technical architect, and have a job where you work from home 100%. You are free to move anywhere in the nation (USA), and your hobbies are sailing, hot rods and fixing up houses.
Where would you move that has great schools, large lakes/ocean, scenery, reasonable cost of living  (buy a ~2000-ish sq ft home for around  $250k)?
You have family on both east and west coast, along with family in Minnesota and Nebraska.
If it matters, you currently live in a north suburb of Dallas, TX.
Where do you move, hypothetically of course? <comment><USER2> Portland is not bad. The winters are rainy and by the city can be expensive, but it's fairly relaxed and is close to absolutely gorgeous scenery.

Southeastern Virginia (York County and some of Virginia Beach) have some good schools and decently cheap housing.

Carolinas, Delaware, Florida have some nice spots.

Some parts of Maryland may fit the bill.

The Great Lakes areas are options too, if you're OK with colder winters.

</comment><comment><USER3> North or South Carolina, Delaware, Florida (advantage Florida to no state income tax. Although summers in south Florida can be miserable with humidity.)

North Carolina is probably one of the nicest states I have lived in. Loved it there and would go back in a heart beat given the right opportunity. Lots to do and easy drive to lots of other activities.

</comment><comment><USER4> I can see the Atlanta area working for you, although we do not have ocean. (But we have lakes.)

</comment><comment><USER5> San Diego or one of its adjoining suburbs.</comment>


Chromium Blog: Moar better graphics (Chrome 18 released) - <AUTHOR> http://blog.chromium.org/2012/03/moar-better-graphics.html

<comment><USER2> Great news! It also seems to me that his has improved the rendering of SVG, although I don't know if it's actually hardware accelerated now or just faster.

[http://ie.microsoft.com/testdrive/Performance/SantasWorkshop...](http://ie.microsoft.com/testdrive/Performance/SantasWorkshop/Default.xhtml)

Runs significantly faster and smoother for me now.

<comment><USER3> I have a quad core i7 SNB laptop with Nvidia 550M. That test gives me 140 presents per minute, while in IE9 it gives me 234 per minute.

These 2 tests also show Chrome 18 to be _twice_ as slow as IE9:

[http://ie.microsoft.com/testdrive/Performance/PsychedelicBro...](http://ie.microsoft.com/testdrive/Performance/PsychedelicBrowsing/Default.html)

[http://ie.microsoft.com/testdrive/Performance/WrigglyWords/D...](http://ie.microsoft.com/testdrive/Performance/WrigglyWords/Default.html)

So what's going on here? Is Chrome really that far behind in graphics performance, or is it because Google chooses to focus on other type of 2D graphics than Microsoft? I'm willing to give Google the benefit of the doubt, because these sort of benchmarks tend to be biased when a competitor makes them (and that goes for Google's own V8 tests, too, where IE9 performs poorly), but IE9 came out a year ago, and even though graphics performance might not be the most useful thing to have in a browser right now, I still can't help but feel a bit disappointed.

<comment><USER4> Results for me:

Psychedelic Browsing Chrome 19.0.1081.2 dev-m gives 5983 Revs per minute IE 9.0.8112.16421 gives 1815 revs per minute

Wriggly Words 10K words Chrome 11425 ms IE 19746 ms

Not sure what you are saying is correct.

</comment></comment></comment><comment><USER5> Nice. But the the title "moar better graphics" is not more friendly or personal, it's just dumb.

<comment><USER6> And it's signed: Posted by John "More CPU in Your GPU" Bauman and Brian "FPS" Salomon

</comment></comment><comment><USER6> If Transgaming sounds familiar: their flagship product is Cider, a proprietary fork of Wine aimed at running Windows apps on Mac OS (just before Wine switched from a MIT license to the LGPL).

</comment><comment><USER7> In light of the Adobe announcement that they're charging 9% for apps that use Stage3D and domain memory, SwiftShader WebGL in Chrome is pretty huge. One of the major advantages of Stage3D vs. WebGL was the fast software fallback for machines with sketchy GPUs or drivers.

</comment><comment><USER8> <http://www.howfastismypc.com>

17: 2 out of 10

18: 10 out of 10

IE9: 4 out of 10

:) M$ guys have been pushing this demo everywhere they go for IE9 vs every other browser. Only because they were the first to market with GPU acceleration. M$ your doing it wrong.

</comment><comment><USER9> It seems like they fixed a very annoying bug in their font rendering on Windows. First thing I checked:

<http://imgur.com/a/UXEPg>

edit: on a closer look Firefox still renders text better.

</comment><comment><USER10> Hmmm... I'm getting some weird artifacts in the new version on my 2010 iMac with this little canvas toy: <http://www.apejet.org/aaron/code/sparks/>

</comment><comment><USER11> I'm stuck on XP laptop for work and text now looks like crap. Can't get any response from the Chrome team.</comment>


Computer classes should teach regular expressions to kids - <AUTHOR> https://www.theguardian.com/technology/2012/dec/04/ict-teach-kids-regular-expressions <comment><USER2> That's a lot of text for endorsing teaching something you can learn in 10 minutes.</comment>


DeviantART data compromised - <AUTHOR> http://www.neowin.net/news/deviantart-database-compromised-email-addresses-leaked?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+neowin-all+%28Neowin+All+News%29

<comment><USER2> Title is incorrect, a database belonging to a third party that contained some user information was compromised, part of the data belonged to Deviantart but the database did not.

<comment><USER3> Also, in case no one connected the dots, Silverpop Inc was the compromised party for both this and the McDonalds incident yesterday. They recently pushed password resets and new login procedures for their clients so presumably a lot more people got affected.

<comment><USER4> I think it's telling that Gawker's database was dumped for everyone to see, why these haven't been released. Just shows you how much some people hate Gawker.

<comment><USER3> The motivations look fairly different imho. Gawker's case was clearly motivated by emotions, and the Silverpop case appears to be the age old black market business showing its ugly head on mainstream news.

</comment></comment></comment></comment><comment><USER5> From TFA: "email addresses, birth dates and usernames may have been leaked but passwords were not compromised"

<comment><USER6> So, identity theft only then.

</comment></comment><comment><USER6> 'Tis the season to be compromised. Fa la la la la, la la la la.

Change your passwords or be sodomized. Fa la la la la, la la la la.

<comment><USER6> Really... no one else thought this was funny? In fact, it got down-voted?

I thought you people had a sense of humour.</comment></comment>


Microscopy without lenses - <AUTHOR> http://physicstoday.scitation.org/doi/10.1063/PT.3.3693 <comment><USER2> Cool.

The Stanford paper microscopes are the cheap, analog alternative that use just a glass bead. For electronics, it makes sense to eliminate lenses and use physics to get a simpler microscope.

</comment><comment><USER3> IIRC when Intel was in the business of building educational add ons to desktops, they produced some microscopes something like these.

</comment><comment><USER4> where can I buy one ;)

Seriously though, this would be great for getting microscopes in tightly- integrated platforms.

<comment><USER5> The first three methods shouldn't be very expensive. There's no special purpose components, no moving parts. As soon as patents expire you will probably be able to buy one at the corner store.

<comment><USER6> The size of a red blood cell is ~100 um. The size of a nucleus of a cell is ~6um in a mammal (so says wikipedia at least, I know less than a 9th grader re: biology). You can get that resolution with < 10k worth of name-brand optics gear (Zeiss, Nikkor JP, whatever) off eBay.

Pair that to a x,y stepper and some Python OpenCV script and you have a ready to go "cell counter", which this paper espouses as one of its uses. Not only that but they only get 20nm with shadow imaging techniques which (I'd imagine[2])are inherently awful for actual examination of features.

Here's[3] a paper that uses wet-methods (i.e. good for biology, where you can add florescence/dyes/stains to your culture; not nearly good enough for modern chip fabrication) that gets you down to the 20nm range. And at that point your lab already has a microscope to hit those lengths.

Basically this lowers the price of this use-case: "oh at t(0) we have 30 blurry units we can sorta-kinda safely assume as eukaryotic; t(1hr) = 600 blurry features". I can see this being a market opportunity for having in- house cancer biopsy diagnostics.

From a research point of view, you're doing at the lower level (protein folding, crystallography, whatever), you're already equipped with the proper mass-specs/NMR/SEMs/TEMs you picked up from Waters or Agilent at half a million. I mean that's got the potential to save a lot of lives, which is awesome, but I probably sound disappointed because I was really hoping for an actual advancement in lens-less microscopy.

\--

[1] en.wikipedia.org/wiki/Diffraction-limited_system ~200 nm under ideal conditions, no pedantry please on apertures or practical things like chromatic/spherical abberation issues, etc. I realize these limits exist, we're just talking orders of rough orders of magnitude. [2] Not in the field, just remarking based on conversations with people who have had to setup labs at universities with limited NIH/NSF budgets. [3] [https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2645564/](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2645564/) There are a boatload of other papers out there, but this is the one I had off hand.

<comment><USER7> Building a science grade microscope that can be used to produce research-grade results is not a trivial problem. Sure you can strap a few lenses and steppers to a frame and make a microscope but that's a very long distance from being able to produce quantitative results of other scientists care about.

<comment><USER6> I think you misunderstood me when I said "name-brand". I don't mean some 30 dollar microscope with $20 glass lapped with 2 passes of alumina 35 micron by a 12 year old in a Chinese sweatshop. Those brands I mentioned were (and apparently still are) more than good enough to get you into industry primer publications. This[1] is literally a random paper I picked off of _Nature Medicine_. I intentionally chose the first DOI from _Nature Medicine_ I saw to get it as 'random' as possible. (Nikkor is the trademark Nikon used to use to indicate their high-end domestically manufactured equipment.)

Hmm, Nikon, check. Agilent, check. Zeiss, check. Hell, everything in that paper, from the Sigma-Aldrich sourced chemicals to the BioGen PCR sequencer is 100% bog standard. Go pick up this months copy of any journal of medicine (hard medicine, not like, psychiatry -- JAMA, NE Journal of Medicine, Cell, whatever) and see what hit-rate you have with those 5 brands. "Research quality science equipment" is bog standard.

In fact, I was curious to see what equipment the CRISPR-Cas9 guys were using, because I mean cleaving nt's is about as low-level as anyone with an MD/PhD is going to get. (It also meets your "research-grade" criteria, cause, you know, its _CRISPR-CAS9_.)[2] Same brands, more sensitive equipment, still pretty bog standard. Again, semi-conductor equipment from the mid 2000s still has it beat for requirements re: sterile conditions for SiO seed -> ingot growth, re: resolution by an order of magnitude (at least) for making chip masks, re: chemical purity when CVD'ing your poly/metals, and for resolution re: SEM/FIB'ing for inspecting your wafers.

[1] [https://i.imgur.com/IC56Nxj.png](https://i.imgur.com/IC56Nxj.png) [2] [https://i.imgur.com/EAQo77W.png](https://i.imgur.com/EAQo77W.png)

<comment><USER7> you're talking $50-100K of investment to buy that Nikon scope.</comment></comment></comment></comment></comment></comment>


My first experience with formal methods - <AUTHOR> http://zipcpu.com/blog/2017/10/19/formal-intro.html <comment><USER2> I think this is a very typical first response to formal methods. But eventually it becomes clear that ultimately it's not really solving any problem, but simply kicking the can to another problem. And in my opinion it's a step backwards as the can you're kicking it to is arguably more complex than where you came from.

Depending on the system you're using formal methods will rely on some set of preconditions (state before a function), postconditions (state after a function), and general state. There's nothing magical about formal methods that prevents errors, either logical/technical or by omission, here. And suddenly very small changes in program flow/logic can result in enormous amounts of work.

They're really fun and interesting, and also help you look at your code in a different way even when not using them. I think everybody has this sort of 'epiphany' moment the first time they actually play with them. But they don't work out in practice anywhere near as well as they do in theory or very limited scale. Something you quickly realize as the effort involved increases exponentially against linear increases in program size/complexity.

<comment><USER3> Without going to full functional proof in Spark (for example) you can go easily as a first step for proof of data flow peoperties ('ooops this buffer size variable is not initialized in this strange corner case'), and then without too much effort (compared to exhaustively testing) proof of absence of runtime error (aka this code won't crash). Of course as GP remarks, you'll have explicited all your preconditions and invariants (types or contracts). But that's a choice : either treat the damned corner cases in your code or expose them to your user ("this can't work on input buffers that aren't power- of-2"). If you don't treat the corner case, instead of putting it in the docs, or in the header comments, or asserting right when you start, add it as a precondition or type invariant/predicate. You then get all the power of automated proof tools (please use all my cycles and RAM but prove this) to prove that without the corner case you won't crash and your users get to check (by proof, static analysis, testing, fuzzing, 'careful code reading'...) that they never call your API the wrong way - if they want.

I think Frama-C & Spark should sell a bit more the AoRTE stuff. If at least your software won't crash or use uninitialized variables, you'll sleep better. The world would probably be a better place if things like x509, asn1 decoding code and all kinds of file format parsers were proved for 'just' the absence of runtime error...

The thing that is nice with most modern formal verification tools is that they're not all-or-nothing propositions. Suppose you already have some Spark code that you've checked (proof) for AoRTE, but didn't go for full proof of all the functional properties of your software (you went for pure-testing with a pinch of static analysis and some code review on critique stuff). You're already high on the correctness scale... You get a report from a customer or security researcher or code auditor that there is a strange behaviour in your code. They can't nail the reason but sometimes you send inconsistent data. No crash or memory corruption, just an output that seems impossible to you (even after full code review and exhaustive test) and out of spec anyways. No reproducer, no idea what was sent to your system, just 'please stop sending us inconsistent data'. One way to find the bug is to try to prove it's not there. Add a post-condition 'fields a and b and c should never be 0 at the same time' and use the tools. Maybe start with structured testing following the code yourself... Then go for unsound static analysis or fuzz testing. Then once you've ran 2 weeks of afl and hongfuzz and didn't trigger this post- and once you've exhausted all the codepeer (or PVS studio or Coverity) false positives and still they didn't find the code path to your bug, and you still need to find this bug, you can use the proof tools and be sure you'll get an answer (after some work) on this specific question.

2 big caveats (for Spark at least) : you'll have to restrict yourself to the Spark subset of Ada (mostly, no pointers, no aliasing - but this subset is deemed to grow once the proof tools improve...) and second : automated proof with floating point numbers is not there yet and it is a _hard_ research topic.

<comment><USER4> SPARK has been used for a microkernel (Muen), crypto primitives (Skein) that caught error in C version, and DNS server (Ironsides). Might be worth mentioning in these so people can see practical work. Far as floating point, I might take your word for it on how hard it is but what's your take on work like Gappa?

[https://www.lri.fr/~melquion/doc/14-smt12-expose.pdf](https://www.lri.fr/~melquion/doc/14-smt12-expose.pdf)

Think such methods can be expanded to handle more of it like SPARK does runtime errors? Or it will stay really hard?

<comment><USER3> I'd read about gappa in here : [https://hal.inria.fr/hal-01534533/document](https://hal.inria.fr/hal-01534533/document) and then [https://hal.archives- ouvertes.fr/hal-01522770/document](https://hal.archives- ouvertes.fr/hal-01522770/document) . Seems they're building on the work done with gappa.

My guess (bystander and user of the tech) is that it'll stay hard for 5-10 years (slow progress of research projects, lots of lead bullets, and far more cores on our dev machines...). If you're motivated and are ready for semi- automated proof you can probably use lemmas (pre-proven properties) in Spark but if you can't find your lemmas in the Spark toolset, then good luck with Coq or Isabelle.

Some encouraging stuff is going on in the FP static analysis world though, with projects like herbie (based on sampling only IIRC), the Tocatta lab, and I've heard some good things about numalis (start-up).

Edit: seems Gappa can be integrated through why3, and seems they did some work on collabaration with alt-ergo so it seems it should be usable through Spark. Something to check in the next release :-D.

</comment></comment></comment></comment><comment><USER5> Formal methods are particularly effective for verifying hardware, because a) the state space is much smaller than for software (although it can still be prohibitively large for complex designs and brute force formal methods) b) people are actually ready to shell out some decent money for getting the verification done.

<comment><USER6> Not only is formal methods more effective for HW, the incentives are also more tilted towards HW. If you fuck up the HW design, well..you're fucked. With SW, "you can always push out an update" mentality exists. I'm not saying this to criticize SW, but this is more an example of how you conform to the ecosystem in which you exist.

<comment><USER7> Which is why people are keen on developing formal methods for smart contracts, which generally can't be updated. Plus they're a lot smaller than most software.</comment></comment></comment>


How PostgreSQL's SQL dialect stays ahead of its competitors [PDF slides] - <AUTHOR> https://modern-sql.com/slides/PG-stays-ahead-20180718.pdf <comment><USER2> Is it correct that SQL Server does not support IDENTITY(slide 29)?

IDENTITY property is supported from as far as I can remember: [https://docs.microsoft.com/en- us/sql/t-sql/statements/create...](https://docs.microsoft.com/en- us/sql/t-sql/statements/create-table-transact-sql-identity-property?view=sql- server-2017)

Second approach is DEFAULT with sequence.

CREATE SEQUENCE seq; CREATE TABLE tab(i INT DEFAULT NEXT VALUE FOR seq, b VARCHAR(100))

<comment><AUTHOR> Thanks for pointing out!

The syntax used by SQL Server doesn't follow the standard. Or, in other words, the standard syntax gives an error in SQL server. However, I'll note that there is an alternative syntax in future versions of these slides.

Thanks again.</comment></comment>


A material web browser built with JavaScript - <AUTHOR> https://github.com/iiegor/chrome-material <comment><USER2> Looks fantastic. Will try once I get to my computer. Does it run on Linux?

<comment><USER3> yeah, check out the package json. I'm running osx, cloned it, npm installed and npm build --linux , and then npm start fired it up. Looks like it's still missing some nice hotkeys, and I couldn't get the settings to open, but it's some good work.</comment></comment>


MyGengo Raises $1 Million To "Own" Translation - <AUTHOR> http://techcrunch.com/2011/06/20/mygengo-raises-1-million-to-own-translation/

<comment><USER2> Robert,

Just want to say congratulations! I've heard only great things about what you're doing, and I hope you can improve the quality of web translations in Japan.

I've been involved in a number of multi-lingual Japanese sites, and I wonder how you feel about competition from Japanese companies.

As I'm sure you know, translation is one of those things that every large company knows they need, but doesn't want to spend a lot of time and or money on. Accessing an API and getting translations seems to be well beyond what most web companies here are capable of, and so you see companies like J-Service[1] which gives their 1-click solution to site translations. I've seen a number of sites (usually 市役所s or 観光協会s) that have used J-Service as a quick-fix for their translation needs.

While services like J-Service have a number of failings (SEO being the big one), their ease of deployment makes them a low-hanging fruit for a lot of sites.

Do you find yourself tempted to go in a similar route with MyGengo, or are you more focused on a global strategy with your API at the forefront?

Keep up the great work!

[1]<http://www10.j-server.com/pro/>

</comment><comment><USER3> Re: comments about much bigger players, as a developer at a compay (moo.com) whos revenue is a _rounding error_ on our competitors balance sheets, I can say from experience that you don't need to be bigger to provide an awesome product, amazing customer service and make a ton of money doing it.

"Owning" translation doesn't have to mean being bigger than the competition, it just means being better.

<comment><USER4> Ahh, moo.com, my favorite site to check whether I have internet (so much faster to type than google.com). Thank you for having a pretty site.

</comment></comment><comment><AUTHOR> Disclosure: My company. Factual note: Not actually a Series A, was a conv. note.

<comment><USER5> Just FYI: claiming you're going to "own" translation makes you sound idiotic, and I'll never use your services because of this. :)

Good luck, though.

<comment><AUTHOR> Guess you could use our services for a different reason instead. But we'll live.

<comment><USER5> That's the point: you won't with that attitude.

I can name 3 online translation services that are much bigger than you. How are you going to "own" a space that already has much bigger players?

I was a fan of your service (mainly because of our large Japanese user base) up until this point. But you know what, I'll live without your service. :)

<comment><USER6> Did you wake up on the wrong side of the bed this morning? You seem to be awfully hostile for little reason, hence downvotes.

Repeating "I won't be using your service" is hardly constructive for anyone. Also I don't see why you wouldn't use them if they proved to be better than the alternatives at _what they are actually doing_.

</comment></comment></comment></comment></comment><comment><USER7> Interesting. From a technical point of view, how does this work? Do you run texts through an automated system and then have real people proof read it afterwards?

<comment><USER8> They're farmed out piecewise to a pool of translators in the language pair. Say I had actually passed the Japanese to English translation test (which I didn't -- what can I say, must have been having a bad night, and I think I botched the heck out of a statement on the red tide's effect on productivity of fish stocks). I would log into myGengo, and there would be a list of J -> E translations awaiting completion, with titles. I could get the full text for one, get basically an editing lock on it (i.e. no one else is translating it while I am), and then after completion the customer reviews, accepts, and I get paid money.

This is optionally doable via API on the customer's side, which is a practical necessity for someone whose business was doing that transaction several hundreds or thousands of times per day. For example, you could imagine a major online retailer deciding "We'd like to offer all of our clothes to customers in the big six fashion markets, but drats, how to get 100k skus of descriptions of men's formalwear translated into six languages. AHA! We'll just build it into our CMS using the myGengo API."

Disclaimer: worked for them. They're cool folks.

<comment><USER9> As an aside, it seems their tests are very uneven. My Japanese is below 1-kyu level on the JLPT but I still managed to pass the Japanese to English test. However, I failed the English to Portuguese test twice, even though my native language is Portuguese and EN->PT is way easier for me than JP->EN.

</comment></comment></comment><comment><USER10> wait, how do they own translation? did they buy the .translation tld or something? or is that word now trademarked and no one else can use it?

</comment><comment><USER11> Not sure what this is about, since I refuse to visit TC anymore, but I do know that eu$1MM is not a lot of money in the translation game...how do they expect to own it? Oh wait, do the scare quotes mean they bought a domain name or something silly like that?</comment>


Show HN: I made a Twitch floating player with floating chat (for Chrome) - <AUTHOR> https://chrome.google.com/webstore/detail/twitch-mini-player/fjeahcfaibacboijpccppebdpihhbflk

<comment><USER2> It's awesome, so much better than what I was using: [http://bit.ly/1xsAH8b](http://bit.ly/1xsAH8b)

Although I think Chat is broken, because last night Twitch changed the IPs of their chat servers.

<comment><AUTHOR> Chat works for me, could you provide a screenshot?

<comment><USER2> Seems to work now. I know twitch was having problems earlier today maybe that was it.

</comment></comment></comment><comment><AUTHOR> This is an early beta, feedback and suggestions welcome

</comment><comment><USER3> As someone that uses Twitch a lot I think this is awesome... Its a lot easier than resizing my windows all of the time.

</comment><comment><USER4> This is amazing. Any chance on Firefox/Open source?</comment>


YC's Startup School Relaunching as Continuous Program - <AUTHOR> https://blog.ycombinator.com/startup-school-relaunch/ <comment><USER2> Awesome news. If you haven't done startup school, I'd _highly_ recommend it. We got paired with lots of people from our local city (which surprised me since I didn't know there was this much activity around here).

Seriously startup school is great. Cannot recommend participating enough.

And also: do it no matter what stage you are at. Full disclosure: the project that our team was working has been put on hold by us[1] (and going through SuS helped us realize we might be chasing the wrong thing), BUT the experience of going through it pushed me towards working much harder on a project which had previously just been on autopilot.

[1]We'll probably pick it back up eventually.

<comment><USER3> Oh getting paired with locals is very cool! I did it a couple years ago, and it was worth it -- but I wish some in my group had been from around my city, Austin. My project ended up shipping ([https://www.currentkey.com](https://www.currentkey.com)), and it was nice to be accountable to a few other people through an important stretch and have regular reality checks. I'd strongly consider doing this again for my next project (a mobile game), especially if I could be paired with locals.

If I had one regret, looking back on the experience, it would be a mistake I made: forming an LLC far before I needed to, largely to be seen as a 'company' by YC. You can (and I'd say in some cases _should_ ) validate your project, esp. if you're a solo founder - by launching an app as an individual developer vs a company account. I know YC itself is only interested in companies, but they should appreciate that if this school is open to all -- all people don't need a company to have a profitable project.

<comment><USER4> I'm working on a startup ([https://www.cloudternal.com](https://www.cloudternal.com)) in Austin. If you want to connect, my email is in my profile.

<comment><USER3> Thanks for saying hi, and done!

</comment></comment></comment></comment><comment><USER5> Astounding strategic play from YC here. Start up School has become popular enough to be viable as a social network. Imagine a VC having a live and data rich directory of most investment hungry early stage tech startups where that startup has shared its progress from the earliest stages of its development. What an incredible advantage that is as a VC firm. Even though I love YC, I find that idea quite worrying.

I've been through SUS and I really valued it. It's great content and some of the discussions were really valuable. I think it's fair to say, though, that the content is very opinionated. I understand why - it's trying to prepare people to apply to YC and YC have a particular way of doing things. I think that's fine. My worry is that the reach of YC on the broader tech start-up culture is so broad now (and getting broader) that these opinions and ideas become broadly accepted norms. When that happens, people stop recognising the fact that there could be a choice to make and we loose the diversity and richness of other opinions and ideas.

Here's an example: everyone knows that the only place to incorporate a company is in the state of Delaware. Creating something in your local region is out of the question, not because you've reasoned well about the pros and cons but because the culture has a generally accepted answer.

I think if we don't have diverse views and perspectives on what's possible and what kind of companies we should be creating, we'll end up with more of the same. That's a winning formula for YC but a bad one for culture as a whole.

We don't need one VC that dominates the early stage market. We need competition to keep the ecosystem healthy. Maybe what we really need is the other VCs, bootstrappers, open-source foundations and NGOs to get into the startup education/social network business too.

<comment><USER6> do you have any info on the Delaware vs your home state debate?

<comment><USER5> Should companies should be founded wherever the financial and legal friction is lowest or are there other important factors? What about the social responsibility a company has towards the community, city and even country that that company was created in?

I think there are a number of factors to consider here which will be specific to each individual context but just pointing out that the answer is only obvious if you've already decided what the important considerations are. VCs will always have a bias towards ruthless efficency but founders need to be more thoughtful. Not every decision is about the bottom line.

</comment></comment></comment><comment><USER7> I did Startup School in 2018 and was one of the inaugural grant winners.

The program was extremely helpful for getting me through to the launch. My product is a _moonshot sold to enterprises_ , so it took an enormous amount of work upfront to get something into users' hands.

The accountability of a cohort was critical for the lonely days of coding something that is both ambiguous and ambitious. Also, constantly being forced to explain my product really helped me hone my message and sales pitch.

I am still friends with some of the people I met back then. I seriously recommend this program to anyone who is starting out.

<comment><USER4> I've been working on an enterprise SaaS startup and have the same challenges. Explaining a general purpose tool and new capabilities that no one has seen before is really daunting. I'd love to hear more about your experience and how it turned out, especially the sales side.

<comment><USER7> I was pretty bad at explaining Empirical in the beginning. During alpha testing, I emphasized the technical merits (statically typed Dataframes!) rather than an application domain. Once I figured-out I needed to push _time- series analysis_ , then the user feedback became more focused.

After starting the public beta, I heard the following sentence several dozen times: "This looks amazing! Does it handle streaming?" And thus began the quest I am currently on.

About the only advice I can give is to keep practicing and keep an open mind. The feedback will get less negative the more you hone your pitch.

<comment><USER8> Big fan of Empirical! I'm excited to continue following the development.

Do you have customers, or potential customers in the pipeline?

<comment><USER7> The business model I've eyed is to license to service providers, like securities brokers and data vendors in the finance industry. That way, their customers can execute queries in the cloud.

For example, a crypto broker wants their traders to write their own models. A stock-exchange advisor wants their asset managers to compute their own transaction costs. Etc.

Basically, Empirical will be a value-add component for existing customers of these larger providers. Instead of shuffling market data over the Internet, the user will "send the query to the data."

Every firm I've shown Empirical to has had the same word-for-word response that most end users have had: "Does it handle streaming?" I put the sales calls on hold while I try to get this done.

</comment></comment></comment></comment></comment><comment><AUTHOR> Hey folks, really excited about this change. I've been wanting to make Startup School a continuous program for years, but it took a while to get all the right pieces in place. Here to answer any questions you may have!

<comment><USER9> What were your initial expectations for Startup School? Where do you see Startup School in 2030?

Thank you for this work, I've gained interesting and profitable lessons from the program.

<comment><AUTHOR> Initial expectations: we expected Startup School to be very popular with founders, and in the long term also hopefully encourage more founders to apply to YC's core program. It modestly exceeded our expectations on the former (we've seen over 100k users sign up in the last two years), and has vastly exceeded expectations on the latter. Concretely, in the latest YC core batch 44% of founders are Startup School alums. That's a far higher percentage than we were expecting at this point!

In 2030: no idea! Directionally though, I'd like to migrate all the "scalable" parts of YC (1-to-many lectures/writing, investor/founder matching, community, accountability) to Startup School where they'll be freely available to everyone, while leaving the unscalable parts (eg. 1-to-1 advice with partners, big checks) in the core program.

<comment><USER10> This is awesome!

I love that there a groups later in Startup School that are for later stage companies.

I can see a progression where based on the stage and growth of your company you get different peers, investors, advisors, and off the record content.

<comment><AUTHOR> Totally. We're making some early moves in this direction with a private forum channel for companies that self-identify as growth stage, but we'd like to take this a lot further.

</comment></comment></comment></comment></comment><comment><USER11> I was in the first version of the MOOC and it was better for me meeting the same people over the duration of the time (currently I believe you interact with different groups each time?).

I found it both interesting seeing how people had evolved during the programme as well as not wanting to let people down by being late or not attending.

I would suggest using some kind of Karma to improve the experience while still keeping the people you meet freeform.

1) Meet the same people over 4 weeks (say) for accountability

2) Attendance, Reports, Progress, Peer feedback etc. can be factored in

3) After the 4 weeks you are matched against people who are at your level of attendance/motivation/productivity/peer review etc. and a new set of group office hours is started.

Anyway, this would help me better to be more accountable, while job, family, hobbies friends etc. get in the way.

<comment><USER12> I've done Startup School for the past two iterations. I really enjoyed meeting so many random founders in each meeting, but we spent more than half of the meetings just getting to know each other. Some of my meetings had 7 teams! And then the next week, you wind up with new founders and you do it all over again. As time went on, I got less and less from the group meeting, where they were my favorite part at the start. Having the same group over and over would be fabulous, since we could really talk about what we did last week, whether we did it, what's up for next week, continuing talk about issues we're having, etc.

<comment><USER13> I'm torn on this a bit since I enjoyed having to practice pitching concisely to new companies outside of biotech to improve that aspect, and to get new perspectives on problems from different people, but I also did enjoy the times that we accidentally (or deliberately?) got regrouped with companies from the prior week and could catch up on progress made. Both have their place and their use, but some level of deeper relationships (as built in the first version that I went through when there were set cohorts) are definitely lacking in the current iterations. Part of it might be taking initiative to stay in touch on your own I guess.

</comment></comment></comment><comment><USER14> Low conversation quality and feedback was a pain point for me during SuS. Do you intend to implement any strategies to improve the quality (better matched, not saying anyone is better or worse) of the peers we are communicating with?

For example, can I be placed in a cohort with only people who are doing more than $100k ARR?

<comment><AUTHOR> Yes, we've reworked the matching algorithm! You can see the criteria we use for forming matches at the bottom of the weekly update when you sign up.

That said, it still isn't perfect and we have more work planned in that area. We also dropped the requirement for group sessions from 6 to 1 so that folks who don't find them a valuable use of time don't need to keep doing them!

</comment></comment><comment><USER15> This seems mostly aimed at SaaS/Uber4X type of startups where you can bootstrap an MVP in a weekend and have a customer by the next one. Where's the content for startups doing innovative stuff in the long run?

<comment><USER13> Having gone through SUS/winning a grant with a biotech, I'll say the style of thinking and the community/accountability was universally applicable. Getting to the killer experiment as quickly as possible to have a solid foundation to build on/raise money off of is a very similar and useful style of thinking to get an MVP launched asap. The more tactical content was sorely lacking for biotechs/hartechs/moonshots though. For bio and hardware, it would have been nice to have some co tent specifically around challenges with capital raising/capital efficiency, how to think about designing experiments. Maybe for hardware, a how to do kickstarter lecture (since there were some very granular tactical software lectures), and maybe a lecture on how to manage a supply chain/inventory. For moonshots, I dont know if theres a playbook that applies since by definition they're very different, but perhaps similar themes to biotech of how do you derisk as quickly as possible. Hopefully theres an opportunity to add content along those lines as they build out the next gen of SUS

<comment><USER15> Thanks for the experience, your post is encouraging, might apply! The lack of "more tactical content" is what I noticed and that is what caused me to have that kind of a perspective at this being aimed at quicker-moving startups.

<comment><USER13> It's not an application anymore I dont think, it's more of a just sign up to participate, but you have to actually participate to get the benefits sort of thing

</comment></comment></comment></comment><comment><USER16> Glad to see you guys doing this, let's see how this changes the future of startups! My feeling is that YC was always AA baseball and now it's a lot more like the big leagues.

</comment><comment><USER17> Really excited to see this being offered as a continuous program. I've participated in and followed SUS since its launch and I can't recommend this program enough!

I'm particularly interested in the addition of streaks as a way to increase accountability and unlock additional opportunities, have you considered further gamifying the program with similar metrics?

</comment><comment><USER18> I've done the program twice and was one of the grant winners. I found the program super valuable, particularly the groups sessions. I really like the new continuous format and excited to keep participating. Nice work Kyle and team!

</comment><comment> 100-xyz We participated in the last SUS and it was awesome for us. Lots of feedback, improvement, new users and a successful PH launch.

Its an on line animation platform [https://toonclip.com](https://toonclip.com)

</comment><comment><USER19> nice - great program and I really got a lot out of SUS! now can we talk about the "guaranteed YC partner interview" that was promised if SUS participants completed a tech quiz from Triplebyte (a YC company). All i got was spam from Triplebyte...

</comment><comment><USER20> Great news! Also, thank you and everybody for all your hard work :)

Will there still be the $15k grant consideration if you become certified when you submit your app for the YC Core program?

<comment><AUTHOR> We aren't currently evaluating companies for grants, but are likely to restart a version of that program later in the year. If and when we do, founders that participate in the program starting now will be eligible!

</comment></comment><comment><USER21> for anyone who has been in Startup School and successfully gone through it, would you say it's best to have a Startup that is a "product" instead of say a service-type business?

For example, I am working on a side-hustle that is more of a service that sells a product, versus a traditional app/SaaS app. So I'm wondering if Startup School is more geared towards product/app businesses than a typical service-type business.

</comment><comment><USER22> So if we graduated SUS in already do we get the Startup School Certified status or we have to go through the program again?

<comment><AUTHOR> You'll need to start again, because the way we're tracking things changed. That said, if you've been submitting updates consistently those will count towards certification, and if you go through and mark all the content you already viewed as complete you'll be most of the way there already.

<comment><USER23> Are there benefits of being Startup School Certified? It's not clear in the FAQ

</comment></comment></comment><comment><USER24> Does Startup School give $15,000 to anyone who gets in after completing the basics?

</comment><comment><USER25> Can we also get (I mean this respectfully) a specific answer from YCombinator on who exactly hosts the news.ycombinator.com website?

It appears to be the little heard of and quite hard to track down "M5 Security Solutions" also known as "M5 Hosting". What is strange is they do say they do private hosting for the Department of Defense. Can we get an expansion on that relationship? Thank you.

<comment><USER26> It's no secret. We were happy to answer this last week ([https://news.ycombinator.com/item?id=23473932](https://news.ycombinator.com/item?id=23473932)), happy a couple years ago ([https://news.ycombinator.com/item?id=16076041](https://news.ycombinator.com/item?id=16076041)), and happy now: we have two boxes (one active, one failover) at M5, a small hosting provider that has good support and better tooling than what we had before. They've served HN well. They seem to be run by engineers, and are engineer-friendly. The founder has been an HN user for 9 years ([https://news.ycombinator.com/user?id=mikiem](https://news.ycombinator.com/user?id=mikiem)).

I don't remember how Scott found them, but it was through some HN-sympatico source like an obscure technical mailing list. We wanted to switch providers, and it seemed like a good idea to pick one that appealed to people with similar tastes to us and sympathetic with the spirit of HN. It's worked out great so far.

Weirdly, this is at least the third time that you've posted strangely insinuating posts about how/where HN is hosted:

[https://news.ycombinator.com/item?id=21423248](https://news.ycombinator.com/item?id=21423248)

[https://news.ycombinator.com/item?id=23374955](https://news.ycombinator.com/item?id=23374955)

Three starts to look like an idée fixe. I still can't figure out what the insinuation is supposed to be...no, we're not an intelligence front. I really want to put a joke here, but I suppose it would probably only feed whatever this is, so I guess I'd better not.

<comment><USER25> I just saw this sorry. Reading now.</comment></comment></comment>


Ask YC:  How can I compete with someone who has a life story like this and who donates 50% of profits to charities? - <AUTHOR> http://www.reuters.com/article/pressRelease/idUS138780+26-Feb-2008+PRN20080226

<comment><USER2> He's got an ugly ugly site. Go to the closest bookstore and count how many Sudoku books they have. Don't worry about competition like that. Just make your puzzles fun to do.

</comment><comment><AUTHOR> "Heart transplant recipient starts new puzzle craze, donates profits to worthy causes"</comment>


Sleepless nights with MongoDB WiredTiger - <AUTHOR> https://blog.clevertap.com/sleepless-nights-with-mongodb-wiredtiger-and-our-return-to-mmapv1/ <comment><USER2> Document level locking as opposed to collection level locking was promising but seems unstable when the data doesn't fit in the cache.</comment>


I get by with a little help from 94552 friends - <AUTHOR> http://a.wholelottanothing.org/2009/07/i-get-by-with-a-little-help-from-94552-friends.html

<comment><USER2> Good, personal story by Matt about the struggles he's had keeping a very successful site up and running.

Well worth the read if you love the independent web and strive, like I know I do, to replicate his success some day.</comment>


Why remote culture will be better than centralized companies - <AUTHOR> https://hajak.se/distributed-culture-will-win-b2f06f24a2ae <comment><USER2> Here at Aula (35-person remote-first startup building a 'Slack for education'), we've done the following:

1\. Dedicate sync time to the stuff that's otherwise hard to do such as relationship building - 'remote coffees', an onboarding focused on getting under the skin of each other. Wholesome stuff. 2\. Your post mentions 'protocols for communication'. We've engineered templates for all team meetings to direct the conversation towards the topics we think matters the most. 3\. Documentation is needed in an async workplace; if you encourage commenting on/challenging everything, you foster an 'always in beta' culture where everyone can make edits.

We're still working on building a culture where remote team members proactively reach out to each other to 'hang out' on video calls if they feel socially under stimulated.

Has anyone solved that?

</comment><comment><AUTHOR> How has your company built a good remote/distributed culture? And where has it failed?</comment>


Ask YC: What do bootstrappers eat for dinner? - <AUTHOR>

I quit my job a week ago and moved into an apartment with a friend to work on an idea full-time. I'm 23-years-old. I want to know, what do you guys eat for dinner?! How do you buy food when you have no income? When I was an earning man, I managed to get by with meals at the office, lean cuisines, and hummus. Now that I'm on my own I have to figure out everything for myself. We plan on making Costco runs about every two weeks and going to Safeway to get anything else we need. How do you guys survive? <comment><USER2> My favorite dirt cheap (but nutritious) dinner is mujadarrah.

1 1/2 c. lentils (rinsed) 3/4 c. rice (rinsed) 1 onion 6 Tbsp. olive oil

Slice the onion into 1/4 inch thick rings. Fry them in the oil over medium heat for 15 minutes. They'll be kind of ropy when they're done, at which point you should take them off the heat. While the onions are cooking, put the lentils and 1 quart of water in a medium to large saucepan over medium high heat. Get the water boiling and cook the lentils uncovered for 15 to 20 minutes. Once the lentils have started to soften up, add the rice, a little salt, and a lot of pepper. Reduce the heat to low and cover. Once the water is all absorbed, stir in the onions, let it cool for a minute or two, and dig in.

Definitely stay as close to raw ingredients as possible. I've lived out of two cookbooks for the last several years: \- Mark Bittman's "How to Cook Everything" \- Deborah Madison's "Vegetarian Cooking for Everyone"

Learn to not be picky. Ramen may be cheap and taste sort of ok, but you need to take care of your body if you want a well functioning mind. Learn to love things like kale... cheap, available year round, VERY good for you. Boil it until tender and hit it with a little tabasco.

</comment><comment><USER3> _"... want to know, what do you guys eat for dinner?! How do you buy food when you have no income? When I was an earning man, I managed to get by with meals at the office, lean cuisines, and hummus. Now that I'm on my own I have to figure out everything for myself ..."_

Food is somewhat like hacking. You have to have access to the raw ingredients, the right recipes & kit. Most important is practice. One of the things I've noticed that differs from hacking is cooking is very time sensitive. Timing is everything.

So here's a couple of hints:

\- REFERENCE: find a good food reference ~ <http://www.jamieoliver.com> Jamie Oliver may not be to everyone's taste but he covers a good variety of tastes and communicates well.

\- BUDGET: set a budget, stick to it.

\- LISTS: create a shop list, don't deviate: fruit, vegetables, meat, rice, pasta, bread, nuts, oils then cheese should be at the top list.

\- PURCHASING: shop at markets and look for fruit/veg in season, shop at supermarkets also but be aware of specials. Limit purchases of non-essentials and learn to spot in-season consumables. For meats & fish wait till the later part of the day when the prices go down.

\- MEAT CUTS: learn how to make do with cheap meat cuts (but be inventive). In the restaurant trade the difference b/w mediocre and great chefs is the ability to make something good out of cheap meat cuts. Do the same.

\- BREADS: make your own bread (flour + water + yeast + oil) Unless of course you can find a cheap baker. My baker bakes bread 30% under supermarket price ~ <http://flickr.com/photos/bootload/543460411/> then <http://flickr.com/photos/bootload/1005146072/>

\- NO WASTE: don't overcook, portion out servings to avoid waste

The most important thing is to get as close to raw ingredients as possible. Then cook yourself. Cook when you need, don't freeze. If you go for processed stuff you are going to pay. The pay-off is better food, cheaper food ~ <http://flickr.com/photos/bootload/sets/72157600195801193/>

Peasants always eat better. They just have to work harder at it :)

</comment><comment><USER4> Definitely learn to cook. You will eat well and spend less. I will often spend $5 to cook a well-rounded meal that feeds me for dinner, lunch, and dinner again...

Just try not to get into an unspoken cooking competition with your roommate. Two months into the semester that my roommate and I decided to learn how to cook, we were spending $30 a meal at the grocery store, trying to outdo each other with dishes like Fillet Mignon with Caramelized Onions, and Grilled Swordfish. Don't do that. Stick to the basics.

</comment><comment><USER5> It's time to get to know your relatives better...eat dinner at their houses, they usually don't charge. Suprisingly, you can find meals at local eateries that may not be at all more than the ingredients, let along the additional costs of time, electricity, supplies, and wasted food, when you have to throw it away, at least at the beginning. So, get to know your relatives and your local food places (you'll easily find ~$3 filling meals, depending on COLA).

</comment><comment><USER6> Regardless if you're bootstrapping, or get outside funding, an early startup should always bootstrap. Bootstrap to me is to spend every dollar bill like it was your last. Now, when you start a startup, you won't have time to cook (good way to save money, but not good way to save time), instead you can spend that time coding. Going to a restaurant is out of the window (way too expensive). The best way to save money, and save time is to buy pre-cooked food. I think noodle is the best in that case. I love noodles; just throw it in the microwave for 2 minutes, and it's ready. You can even eat it while you're coding. Trust me I've experienced it before. Noddle will save you time and money.

good luck!

</comment><comment><USER7> Most days, I eat whatever my parents cook.

Living in my parents' basement while I get tarsnap up and running is the best business decision I've ever made.</comment>


Ask HN: Moving to Brussels, how's the tech scene? - <AUTHOR> Hi HN, I work for a rather large SaaS business and I'll be moving to Brussels in February.
Anyone from the region able to give me some pro tips?
Ie: How's the tech scene? What suburb should I live in? Things to do in and around Brussels, etc <comment><USER2> So, when I lived in Brussels, I found that most of the English-speaking expat community lived in Waterloo, or other towns nearby. But we lived in Ixelles, which also has some English-speaking expats.

The tech scene was actually pretty good, and I would highly recommend the FOSDEM conference. You should also check to see if the LinuxBierWanderung group is still around — they like Linux, drinking beer, and hiking, hence the name.

When I was there, I worked for Belgacom Skynet, but before they go re-absorbed into the parent telco. You may not have much options for getting internet access there, but I would encourage you to at least try anything but Belgacom.

Things to do around Brussels depends on what you're interested in. If you like architecture, there is Hotel Horta (which is the Horta museum), and other houses designed by Victor Horta. He is a famous Belgian Art Nouveau architect. It is located in Ixelles, not far from where I used to live on Avenue Louis Lepoutre.

Of course, there's always the Grande'Place, and Manniken Pis. The worlds second-largest cupola is found in the Basilica de Sacre Coeur, and there are a number of truly beautiful churches there. They also have movie houses, the opera, etc..., but most of the really old and beautiful stuff is outside of Brussels in Ghent, Liège, Bruges, etc....

Don't forget Castle Boullion, which dates back to Godfroid of Boullion, who was the leader of the first Crusade around 1000AD. It is close to Abbaye d'Orval, where they make the famous Trappiste Orval beer.

That's my first pass, off the top of my head.

<comment><USER2> Belgium also has lots of really nice museums, but we never went to many of the touristy places unless we had friends in town and wanted to show them around.

Speaking of touristy, there is also the Atomium, and Mini Europe (especially cool for younger kids).

</comment></comment><comment><USER3> Brussels is Brussels, there's so much more around, Belgium has many lovely places to visit and from Brussels they're nearly all reachable by public transport.

Living areas in and around Brussels are hit and miss sometimes, it can be quite hard to find decent priced places. A lot of expats I know choose to move out of Brussels to nearby towns, Leuven or Mechelen are examples, about a 20 to 30 minute train ride to/from Brussels (North station). The public transport works but always calculate time for delays, they are more common than bread unfortunately. Finding a decent area to live in Brussels requires a lot of research, some areas, like Elsene, are great to live in but some parts of that area are also to be avoided.

As for tech scenes, there's plenty, the Meetup app might be helpful for that.

There's so much I could tell you, as I myself live in Belgium and have been working in the IT scene for over 17 years, most of that in Brussels, so if you have questions or want to have a talk once you're here you can reach out to me via my profile.

Welcome to Belgium!</comment>


Why Tarsnap doesn't use Glacier - <AUTHOR> http://www.daemonology.net/blog/2012-09-04-why-tarsnap-doesnt-use-glacier.html

<comment><USER4> While I think this is a completely reasonable thing (not storing tarsnap data in Glacier), I'm not sure if the reasoning holds beyond, "For the way we designed tarsnap it doesn't make sense."

When you look at how GFS is implemented or Bigtable, or Blekko's NoSQL data store, there are layers, with the meta data as a pretty separable layer from the data. In all of the cases for these large stores that separation was put in to facilitate putting the meta data into lower latency storage than the 'bulk' data which facilitates fast access.

As cperciva relates, tarsnap spends a lot of time de-duplicating data (or writing only one copy of a block with identical contents) which is great for reducing your overall storage footprint which lowers your costs. But if the cost of storage is much much smaller than the retrieval cost, then your design methodology would be different.

So I would not be surprised if there was a way to build a function equivalent product to tarsnap that had lower storage costs if the bulk data was in glacier and the index data was in S3, or if the index data was designed such that a document recovery was exactly two retrievals (a catalog, and then the data). And of course such a system would not de-duplicate as that would result in potentially more retrievals.

It seems that if the price delta ratio between Glacier and S3 was higher than the de-dupe ratio, then Glacier would 'win' with no de-dupe. Else de-dupe would still win. Thoughts?

<comment><USER5> I think the most important part is separating the de-dup metadata from the data. I assume the de-dup is done via either digest or HMAC -- if you keep the digests (and the blocks of digests, et al.) and pointers to the data location in Glacier in S3, couldn't you do the de-dup using S3 but actual data storage using Glacier?

<comment><USER6> This is exactly what I've been wondering - obviously cperciva is 10x (100x?) smarter in this area than I ever will be, but I kept thinking, "Just keep the digest in online storage, and the actual data in (offline?) Glacier. You don't need to access Glacier during backups to calculate digests, because you already have the digests online."

I read through his posting carefully, to see if he would capture that possibility - but didn't really see it there.

<comment><USER4> Think that through a bit :-). So what counts as a 'retrieval' ? Well in the tarsnap case it appears to be a block which as far as glacier is concerned it probably thinks it is a file.

[ NB: huge huge guesses here about how tarsnap works ]

Now consider de-dupe in blocks (vs de-dupe in files) with an object store, combined with a backup. Now lets say you have a file that is 5 'blocks' long. You can get the original 5 blocks (5x fetches) then each block where deltas reside, to recreate the file you want at the time you want it.

Compare that to the 'stupid' way of doing it which is to store a full image of the file system for each time period. Where you have one fetch to get back the file from the copy of the file system that you were interested in.

But this is where the assumptions that make that stupid need to be evaluated. It is a poor choice because the expensive thing is the storage, and you trade compute cycles for storage. That is normally a winning idea because you only 'spend' the compute cycles when you reconstruct the file you want, but you continually pay for storage month after month.

Except that the pricing model of Glacier makes bulk storage cheaper and algorithmic reconstruction expensive.

Presumably the folks at Amazon are de-duping, after all they get to charge per the GB and if they can sell that exact same GB to two people, well that is a win!

So back to the question at hand.

Lets say your middleware layer is just like it is today, figures out just the deltas in your file system from the last backup and then pushes those. You've got a file system image plus a delta image and by applying the delta to the full image you can get back to the current image. However, since storage is now less expensive than compute, _at the Amazon instance_ you take the delta apply it to the latest full backup, and create a new full backup which you then store in Glacier.

So can this possibly make sense? That is the question, if you keep full copies of the latest copy of every file, and pointers to the reconstructed previous versions in your S3 meta data. Can you get a lower net cost for the service implementation?

</comment></comment></comment></comment><comment><USER7> I like articles like this.

1\. Explains what the new whiz-bang technology is.

2\. Explains what Tarsnap is.

3\. Provides technical explanation that lists both the upsides of the new technology along with the downsides.

<comment><AUTHOR> Thanks! I like blog posts like this too -- they appeal to my didactic instincts.

</comment></comment><comment><AUTHOR> Another reason, which I decided deserved to be written up as a separate blog post, is the surprising nature of the "data retrieval" component of Glacier's pricing model: <http://news.ycombinator.com/item?id=4475319>

</comment><comment><USER8> I'd bet that this use case is pretty common: I have only a small amount of data (a few hundred MB in my case) that I deem day-to-day critical, but hundreds of GB that I don't want to lose. My backup solution is effective but primitive: parchives on two offline hard drives, one of which is at my parents' house seven hours away. Backing up takes little effort but it's easy to forget, especially at the "remote site".

It's really annoying that there's lots of easy backup solutions for online data, but nothing for cheap backup of large amounts of data that can afford to be offline for a while. Glacier is the perfect solution but I dread having to figure out whatever I have to do--divide things into a 180-part archive and download it over a month?--to get data back. The first glaciated backup solution on the market that I think I can trust will get my dollars almost immediately.

<comment><USER9> Some people might not be aware that CrashPlan supports backing up between your own PCs (eg. to your parents) for free:

<http://www.crashplan.com/consumer/crashplan.html>

</comment></comment><comment><USER10> I was thinking about one time, single shot backups, but encrypted wtih tarsnap. No dedup. After all, storage is cheap and this is supposed to be zuper worst case scenario stuff, so maybe I don't want to go to the trouble of reconstructing the backup. I want to know it's there. But my interest is only theoretical.

<comment><USER11> If you have a command line tool to access Glacier, I think what you can do is make an archive (with tar), encrypt it with gpg, and just use the command line tool to upload the encrypted archive. So you don't need tarsnap for that, but if you want fancy deduplicating incremental encrypted backup, that's where tarsnap shines.

<comment><USER10> Yes, but I'm already using tarsnap. Running it with "--backup-to-glacier" is easier than rolling my own. I don't want two passwords, two super duper secret keys, two online services I have to pay, ....

</comment></comment></comment><comment><USER12> Does anyone know of products that already incoorperate Glacier. Basically, I am looking for a solution to back up my photo library. It's one big file with a "Write Once, Read hopefully never" Access Policy. I guess I could go and push it to Glacier by myself, but I'd rather have a library/product do it for me

<comment><USER6> Glacier was rolled out on August 21st - I don't think anyone has rolled out code to support it in a consumer friendly fashion - but drop a line to <http://www.haystacksoftware.com/arq/> and encourage them to give you the ability to target Glacier for your Archives, and I bet you'll see something fairly soon.

</comment></comment><comment><USER13> I'm confused about the reason a block can't exist in both S3 and Glacier at the same time, if the deduplication code decides that block is needed in a new archive.

Why couldn't you simply have a rule that each file is either S3 or Glacier, and S3 lists-of-blocks can only reference other S3 blocks, while Glacier lists of blocks can only reference other Glacier blocks?

In the worst case, where every block was in both archives, this would only increase costs by 10% if Glacier costs a tenth what S3 costs.

<comment><AUTHOR> What happens when you decide that you don't want that new archive to be in S3 any more and tell Tarsnap to migrate it over to Glacier?

<comment><USER13> To S3 it's like you just deleted the file; to Glacier it's like you just created the file.

<comment><AUTHOR> Except that then we're trying to store two different blocks in Glacier with the same hash ID.

<comment><USER14> From that I assume that if a block's hash matches something that's already in the archives then you retrieve the archive block(s) with the same hash ID in order to verify that it is exactly the same (byte-for-byte)? And this wouldn't be possible with Glacier as you can't just retrieve the block from storage to check there and then.

Do you have any stats on the number of collisions you've seen?

</comment></comment></comment></comment></comment><comment><USER15> I've got Terabytes of data that need to be archived and possibly recalled later (trading logs and market data preserved in case of an audit). I'm not at the scale of Glacier. If TarSnap had a readonly vault (files in said vault would never change) then it would be able to distinguish the files and split between the Glacier and S3 offerings.

<comment><USER6> Drop a note to <http://www.haystacksoftware.com/arq/> \- I'll wager they will have an option to create "Glaciered Archives" of folders within a few months. The developer there has already expressed some interest, and suggested that Arq would lend itself to pointing at a folder and storing it off to a Glacier Archive (with all the caveats around 4 Hour delay, 24 Hour of Availability, Costs associated with restores, etc...)

<comment><USER16> [I'm the developer] I'm hoping storing the metadata in S3 and the actual file data in Glacier would work well. I'm still looking into it.

[edited for clarity]

<comment><USER6> That's awesome news. Just being able to take a folder on my Drive, and say, "Archive this 5 Gigabytes of photos for the next 50 years" - and know it will cost me around $30 (and, as time goes on, likely less as the storage drops) and I won't have to worry (much) about it, will be a big win - even if you don't come up with an easy way of integrating with the normal S3 backups - <USER2> bet a lot of people will love that feature.

</comment></comment></comment></comment><comment><USER17> I think the best use-case for Glacier is large files that you would absolutely hate to lose. The thing that people cry over when their laptops get stolen.

For home use, all the family photos and videos. An archive of your emails (outlook.pst) because a lot of important data is stored in there. All your taxes and accounting data from years past. Bulky stuff that takes up space on your home system, but isn't used daily.

In business, many companies use "Iron Mountain" to archive their paperwork. Old invoices, reports, things that were important in the past and may someday be needed. That's what Glacier is for.

Glacier is archiving, not backup. You might want to take advantage of cheap storage by keeping your backups there, but that's a different issue.

<comment><USER18> I don't get the impression Glacier is right for any home user. The pricing scheme is just too weird, and there are weird delays and time limits involved. I believe you have 24 hours to retrieve your data before incurring more fees when you do a retrieval from Glacier--but if I store a backup of 150 GB of photos and music and fetch the whole thing, I'm not going to be able to get that from Amazon to my home computer in one day. Avoiding that problem will necessitate other expenditures.

In order to avoid the retrieval costs you'll have to limit retrievals to a small fraction of what you have stored. In my example, if I lose all my photos and music I'm going to want to restore the whole thing. Ignoring the transfer problem above, you're looking at paying for transferring 95% of the archive, 142.5 GB. I find Glacier's pricing model so difficult to comprehend I couldn't even guess at what that would cost, but Colin's math shows that what looks like it should cost $0.02 (retrieving 2 GB once at $0.01 per GB) winds up costing $3.60 (peak rate, percentage of archive fetched, etc.), so I wouldn't hold out a lot of hope for our use case of fetching the entire 150 GB archive.

As you say, this is certainly something businesses that need to archive lots of stuff may be able to use (if they can navigate the pricing structure) I just don't see a home user getting anything out of it but frustration and a confusing bill.

<comment><USER13> Assume I store 150 gigabytes of family photos. I pay $1.50 a month ($18 a year) for storage. Assume I've uploaded the files in 150 * 1 gigabyte archives.

I decide to retrieve it all. The 5% (6 gigabyte) retrieval allowance is negligible. The data transfer out fee at $0.120 per gigabyte will cost $18.

If I retrieve 150 * 1 gigabyte chunks every hour retrieval will take 1 hour; the peak hourly retrieval will be 150 gigabytes; the data rate will be 341 Mbps; and the retrieval fee will be 150 _720_ $0.01 = $1,080

If I retrieve 7 * 1 gigabyte chunks every hour retrieval will take 150/7~=22 hours; the data rate will be 15 Mbps; the peak hourly retrieval will be 7 gigabytes; and the retrieval fee will be 7 * 720 * $0.01 = $50.40

If I retrieve 1 * 1 gigabyte chunks every hour retrieval will take ~7 days; the data rate will be 2.2 Mbps; the peak hourly retrieval will be 1 gigabytes; and the retrieval fee will be 1 * 720 * $0.01 = $7.20

If I share an account with 20 other people with the same amount of data stored, the 5% allowance would be enough for my entire download; I could retrieve as quickly as I liked without incurring a retrieval fee. I would still pay the $18 data transfer out fee.

TLDR: Retrieval isn't as cheap as storage, but if you lost all your family photos, you'd probably be willing to pay it.

<comment><USER18> Thanks for doing the math on this!

</comment></comment></comment></comment><comment><USER19> Great write up, I've been curious about it, but I'm not particularly bothered. I think what I'm going to end up doing is to basically tar ~/media, pipe it through GPG, and pipe it through some upload-to-glacier script. Maybe automate that once a month or something and then have tarsnap for much smaller/rapidly changing stuff. As it stands now I have 100gb in ~/media that's costing me about $1/day, so it'd be nice to reduce that.

</comment><comment><USER20> Cperciva,

Could a similar system to Tarsnap exist for Windows? I'm not asking you to implement it, I'm just curious.

<comment><USER21> Tarsnap runs without any problems under cygwin/windows right now. It is very easy to set up: the Tarsnap downloads page lists all the dependencies - <USER3> just need to make sure to include them in the Cygwin setup and then run his makefile. Frankly, his setup instructions are far more easier to follow than certain other gui-based installs that look shiny but induce far more stress trying to guess how to opt-out of the complementary crapware (e.g., Skype, Adobe Reader, utorrent...)

Also, as he notes in his install page, it _definitely_ is worth checking out the source code: one of the cleanest C source I've ever seen, and very educational.

<comment><USER20> Thank you for the info!!</comment></comment></comment>


"Anti-internet piracy law" approved in Spain; sites can be shut down in 10 days - <AUTHOR> http://www.bbc.co.uk/news/technology-16391727

<comment><USER2> Why copyright / piracy?

I understand that Hollywood and other IP owners spend more, have better lawyers and lobbyists, but really, I no longer believe the idea that across the Western world, legislators are all coming round to the idea that they must somehow protect the status quo of music / film production.

This is feeling more and more like an excuse.

And if it _is_ an excuse, then finding the underlying concern / reasons that drive these laws is important

What do you think is driving legislators fears?

Is it a need to somehow get this internet thing under "our" control.

Is it a form of parochialism - we dont want this foreign sites coming into our country (be that sex.ru or facebook)</comment>


10 apps in 50 days: Launch #1 - <AUTHOR> http://radio.poetic.ly/

<comment><USER2> This is definitely a terrific way for learning new stuff (you have now the HN community watching you). However, don't also forget that for most applications, their success/failure will be also determined by the spit and polish applied on them. Don't get carried away over the "quantity", always be mindful of "quality" too.

Update: I read your intention behind this here: <http://terrorified.com/?p=786>

Now I better understand your motivation. All the best.

</comment><comment><USER3> I clicked the link, and the random band-picker picked Eagles for me. So the line on the top said "I'm in the mood for Eagles videos", and the video playing under it was Iron Maiden's Where Eagles Dare.

Just the old demo effect, I'm sure.

</comment><comment><USER4> I typed in "cats" and got exactly what I expect. Nice job :)

</comment><comment><USER5> How is this different than scraping the results of a youtube search and embedding a result?

<comment><AUTHOR> This keeps a playlist of what you liked and uses that to recommend videos, allowing you to personalize and filter down the videos to only what you are interested in.

Example: Searching for "haskell" will give you videos about people named haskell as well as the programming language. After adding a few example videos to your list, you will get only results related to the programming language. The idea is to start to combat the incredible amount of noise in search results: believe it or not, not everyone enjoys rick rolls and gophers.

<comment><USER5> Ok, that is awesome. You should put some text explaining that somewhere on the page.</comment></comment></comment>