Philip Greenspun Visits Berkeley - <AUTHOR> http://blogs.law.harvard.edu/philg/2010/01/10/visit-to-berkeley-california/ <comment><USER2> About Berkeley's poverty rate, it doesn't seem like he considered that just like rich people can move to other states, poor people can move too. Maybe the poverty rate of Berkeley is high precisely because it's better to be poor in Berkeley than in Oakland or Emeryville? When a city cuts down on social services, they are just externalizing the costs onto neighboring communities.

I know that Santa Cruz was trying to get rid of its reputation as "a good place to be homeless in" exactly because homeless people from all over the Bay Area came there.

<comment><USER3> This is exactly why the population of homeless people is so high. It's always high near a university, and it's especially high in a city where the homeless are borderline actively encouraged - the city even fought the university on tearing down a park to build more student housing (note: university owns the land) because too many people were living in it.

</comment></comment><comment><USER4> As one small point, Greenspun waxes eloquent about the "big government" stupidity of the Berkeley Tool Lending Library.

Well, what is the Berkeley Tool Lending Library? It's a couple of sheds and some low-hours guys there taking care of the equipment. We got wheelbarrows, we got various small power tools, we got rotor tillers, we got tree pruning equipment, some ladders, etc. Lots of people benefit but, by eye, I'd say the main beneficiaries are homeowners who otherwise would have to buy under-used equipment from Home Depot or hire a contractor. The budget is freaking small. The social benefits (including bottom-line contribution to the local economy) is freaking huge. He's talking through his hat on that point, at the very least.

<comment><AUTHOR> What happens when someone gets injured or killed because he or she was improperly using a tool from the tool lending library? What if the low-hours guys you mentioned were cutting corners on the tools' upkeep? Or is this really an issue?

I don't think that Philip is opposed to the idea of a tool lending library, he is merely questioning why the City of Berkeley needs to be running it. Why can't a group of citizens form a cooperative to do this, instead of involving the city government?

<comment><USER5> The city government IS a cooperative formed by the citizens of the City of Berkeley.

<comment><USER6> A cooperative is a group of people united voluntarily. A city is not voluntary (yeah, you can move. but it's not always possible or convenient until we have seasteading).

<comment><USER7> City residence IS voluntary. Very much unlike country citizenship, which, for practical purposes is involuntary for the vast majority of (non-European) people.

<comment><USER8> What do you mean? Country citizenship is not voluntary for EU citizens either. Did you mean country residence?

<comment><USER7> In practical terms, citizenship does not restrict residency in Europe for Europeans.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> _"I pointed out that California collects a larger percentage of its citizens' income than all but five other states (10.5 percent; source). Shouldn't it be possible to run the state on 10.5 percent of income?

Despite the fact that all of my interlocutors had university educations, sometimes including PhDs, all were so deeply invested in the idea that their insolvent state government is starved for revenue that they were unable to parse the information."_

Perhaps they were just stunned that someone with a PhD doesn't know that income tax isn't the whole story?

California state revenue, 2009: $88 Billion (source: [http://www.ebudget.ca.gov/pdf/BudgetSummary/SummaryCharts.pd...](http://www.ebudget.ca.gov/pdf/BudgetSummary/SummaryCharts.pdf), chart SUM-01)

California population: 36.8 million (July, 2008; source: US Census)

CA Revenue per capita: $2391.30

Texas state revenue, 2009: $78 Billion (source: <http://www.cpa.state.tx.us/taxbud/bre2008/BRE_2008-09.pdf>, page 3)

Texas population: 24.3 million (July, 2008; source: US Census)

TX Revenue per capita: $3209.88

"Conservative" Texas takes in almost a thousand bucks more per person in taxes and fees than does "liberal" California.

UPDATE: I made a mistake. The numbers for Texas are reported on a biennial basis, whereas for California, they're reported on an annual basis. See my reply to cwan below for details. When you get the annual numbers from the Texas budget summary, they appear to take in about 32% less per capita than California.

<comment><USER10> I'm afraid you are incorrect. The link you put for California is just the general fund, which does not include special funds, bond funds, or federal funds like the Texas one does. After you add those in, California's count is $208 billion in 2008-2009. See this for more: [http://www.dof.ca.gov/budgeting/budget_faqs/information/docu...](http://www.dof.ca.gov/budgeting/budget_faqs/information/documents/CHART-B.pdf)

Edit: I should say though, right on for pointing out the material issue. You got the meat of it right in that it's overall spending that matters, not just income tax. You "showed your work" correctly, you just plugged in the wrong numbers. That means you get an 8/10 from me.

Edit2: I read the parent's post too quickly in my haste to make a point. However, neither of us considered the contribution of local taxes, as discussed below. I believe that that definitively answers the question of which state spends more per capita.

<comment><USER11> And after you add them in for Texas, it's $158 billion. You're comparing apples and oranges.

Additionally, timr was correct to restrict his numbers to state tax revenue, which is the total under discussion.

If you do the math, it comes out to $6,500 per person in Teaxs, and $5,600 per person in California. So even considering the other sources of revenue (though it makes no sense to do so in this context), timr's point is still correct.

<comment><USER12> Apparently still not true. Have a look here (California State Per Capita Revenue numbers) 2010E - State: $4015; State+Local: $9050 [http://www.usgovernmentrevenue.com/CA_per_capita_revenue.htm...](http://www.usgovernmentrevenue.com/CA_per_capita_revenue.html)

and here (Texas State Per Capita Revenue numbers) 2010E - State: $2980; State+Local: $7041 [http://www.usgovernmentrevenue.com/TX_per_capita_revenue.htm...](http://www.usgovernmentrevenue.com/TX_per_capita_revenue.html)

These tie better back to the US Census numbers (which are only as at 2004 but are on an apples-apples basis): <http://www.census.gov/govs/state/04rank.html>

The problem is that states do not report revenues in the same ways and some are clearly more creative than others.

<comment><USER9> Yes, there's a problem with my original post; no, it's not related to federal transfers -- it's correct to exclude them from the calculation. The problem is that Texas (bizarrely) allocates its budget on a biennial basis. I didn't read the documents closely enough, and had assumed that 2009-2010 meant fiscal 2009-2010. Oops.

From the same Texas source, table A-12, the estimated total state revenue is $39.3 Bn for fiscal 2009. Thus, the per-capita revenue numbers for fiscal 2009 would be $1,617.30, versus $2,391 for California. So, I'm wrong. Texas takes in about 32% less per capita than California.

<comment><USER13> It's not that bizarre they budget biennially -- when you consider that the Texas legislature is part-time, and meets only every other year.

Given the eternal train-wreck that is California budgeting -- even in good years, they just give away the store in future obligations (like public pensions) that make the bad years twice as bad -- fewer, longer budgets might be a good idea here as well.

<comment><USER9> AFAIK, the biggest problem with California budgeting is the initiative-driven pre-allocation of huge amounts of money that would otherwise be discretionary income. Pensions may play a part, but not nearly to the extent claimed by certain partisan interest groups.

</comment></comment></comment></comment></comment></comment></comment><comment><USER14> I think it's clear that the problem with California is not the taxation rate (which after all at 10.5% is not much higher than the national average of 9.7% and ranks only 6th) but rather that the entire state government is incredibly broken.

A big portion of the state's revenues are constitutionally mandated to go to certain programs via the ridiculously easy referendum process.

Meanwhile, Proposition 13 requires a 2/3rds majority vote to pass any sort of budget, and a hardened radical element in the Assembly would rather see the state go down in flames than raise taxes (even when coupled with spending cuts).

I think the very short term limits (3 terms) for assembly members feeds this tendency because they don't have enough time to get good at legislating and compromising before being kicked out of office.

At the same time the (vast majority) Democrats in the assembly don't want massive spending cuts -- but you must admit they have the majority! A 2/3rd requirement to pass a budget is incredibly undemocratic.

The result is unrestricted borrowing and inevitable collapse.

</comment><comment><USER15> Suggested subhed: 'Foolish Californians fail to recognize obvious superiority of writer's free-market views, invite him to dine with lesbians.'

</comment><comment><USER2> I've always thought Greenspun's articles were interesting and well written, so I'm disappointed to see such a transparently ideological and sarcastic article.

<comment><USER16> Really? It seems like at least 4 out of every 10 Greenspun posts are transparently ideological and sarcastic.

<comment><USER2> Maybe it's because I've mostly read the ones about flying?

<comment><USER16> True, I subscribe to his blog and it's about 40% sarcastic ideology, 30% aviation, and 30% commentary and book reviews. Usually only articles from the 3rd category get posted here.

</comment></comment></comment></comment><comment><USER17> Move over Rush Limbaugh and Sarah Palin. The right has a new hero and it's ... Phil Greenspun? Who knew!?

Now Phil is surely much smarter than I. No argument there. I would never have the confidence to go to Harvard and try to straighten out the PhDs there about how Massachusetts should be run. They would call me names like Carpetbagger and such.

Where does Phil get his 10.5% tax rate number? From the Tax Foundation <http://en.wikipedia.org/wiki/Tax_Foundation>. Where do they get their numbers? It's vague. Does it include sales tax and workman's comp etc.? The same foundation complains about California Corporate tax burden being the highest in the nation at an outrageous 8.3%. Hmmm. 8.3 < 10.5. And don't give me any of that trickle down baloney about not taxing corporations.

Pensions are a great boogie man of the right wing. They cost so much. It's much cheaper to stop paying those folks. If they haven't amassed fortune enough to retire using that huge 40k salary by now, tough luck.

I had better stop now before I get all worked up. After all I'm just a dumb old computer programmer who doesn't know much about economics. Phil's a s m a r t old computer programmer ...

Sigh. I'm actually a bit of a fan of Phil's. It's just that the harsh conservative position is hard for an unemployed 50+ programmer to swallow these days. I don't have a "fat" pension. When I go for interviews I rarely see anyone over 40. Good luck to you all, and don't look down on us "over the hill" workers. Just look away.

</comment><comment><USER18> "None could accept the idea that their state had a spending problem rather than a revenue problem"

"For roughly 60 years, Berkeley has offered more services to its residents than virtually any other city in the U.S."

This should point out to him his mistake, I would think. He is assuming that the correct answer is to lower taxes (or keep them the same) and spend less. But this isn't a correct (or wrong) answer, it is one option. Maybe people in California would rather pay higher taxes and get more services. It works pretty well for a lot of European countries.

</comment><comment><USER19> I've never seen someone so bitter about California weather.

</comment><comment><USER20> One sentiment people in Berkeley would seem to share, however, is that they'd rather be naive about helping people than naive about being critical.

<comment><USER21> The argument is that the mismanaged spending and high taxation rate are actually pernicious to the community's welfare, this isn't just a matter of cynical reflex saying "look at those do-gooders thinking they're making a difference".

<comment><USER20> But the argument hints that they are missing the big picture by, as you claim, 'mismanaged spending and [a] high taxation rate'. My point is that it's better to miss the big picture while helping people, than to miss the big picture while criticizing people.

Greenspun visits Berkeley for maybe 3 or 4 days and he's now quite sure that people there have it wrong? He mentions Prop 13 in passing, as something that everyone in the East Bay keeps on talking about. Some might argue it's a much bigger issue than he realizes. Far from over-taxation being a problem, it would be that California has not taxed based on real property values for decades. I.e., Greenspun has side-stepped a large problem in the bigger picture by being hyper-critical.

Perhaps I'm overreacting, but I was just getting a little tired of Greenspun's hints that there is a problem with being naive about 'doing good' -- as you mention (quite well said, that is sort of how I read Greenspun's attitude). My point was to hint that there is also a problem in taking one's 'cynical reflex' too seriously.

After all, these actions, at a certain level, are really kind of similar movements. Hyper-taxation and hyper-criticism both attempt to optimize at the wrong time, too impatiently. However, in the former case, at least someone gets helped (though arguably by being critical Greenspun is helping eventually). At any rate, I don't think what he says is completely incorrect. There's just a flip side to it. E.g., in any sufficiently large Lisp program, a lot of time is spent in FFI.

</comment></comment></comment><comment><USER22> Do California residents get a greater than average amount of services for all their taxes?

<comment><USER23> That is an interesting question, but I don't think it is the right one. California can innovate and do better. Where there is waste it may be worth eliminating, even if it is a below average amount of waste.

If you are talking about federal taxes, then the answer is certainly yes. Californian's pay a lot more in federal taxes than they receive in services from the federal government relative to other states. This may be related to the fact that they have less federal senate votes per resident than most states.

Does incarcerating someone count as a service? California has a large prison population. I would argue that significantly as a result of federal drug policies.

The article was pretty light on the detail of where the money goes.

</comment></comment><comment><USER24> His anecdote about Macintosh repair costs is excellent. Case closed, Californians, you should have gotten Dells.

</comment><comment><USER25> Philip is so right. The Bay area stinks. Every body should move to Texas or to Boston where life is so much better. Real estate prices are way too high. The politics here are way to liberal. The government is way out of touch with what's realistic. The whole state is going to collapse any day underneath its own weight.

</looks through San Jose real estate listings></comment>


Meet the Saudi Arabian Metal Band Breaking the Law by Being a Metal Band - <AUTHOR> http://www.vice.com/read/anti-religious-black-metal-band-in-saudi-arabia-666

<comment><USER2> Brave, foolhardy, dangerous. Sounds pretty good too! I wish them luck.

</comment><comment><USER3> That's so Metal.

<comment><USER4> And so Medieval.</comment></comment>


DNS Security: Threat Modeling DNSSEC, DoT, and DoH - <AUTHOR> https://www.netmeister.org/blog/doh-dot-dnssec.html <comment><USER4> This is a great summary and I agree with the findings mostly. I think we should separate the "DoH the protocol" from the "DoH decision of using a public resolver". For "DoH the protocol", widespread adoption by most users' default resolvers is about as likely as widespread adoption of DNSSEC and DoT. For "DoH decision of using a public resolver", that decision can be done today with resolvers that support DNSSEC/DoT.

So if we want users to use a hardcoded default resolver different than their ISP's, come out and say that and stop conflating it w/ the protocol. If we want users to use the DoH protocol, we're in the same boat as asking them to use DNSSEC/DoT. Therefore, for the idea of a hardcoded default resolver in one browser, if it's really something a browser wants to do, why are they waiting on a protocol? And for the idea in another browser of checking known protocol support of the existing resolver, do/can they do that with DoT too? And can they do something besides a hardcoded list to check resolver capabilities (i.e. can I return an additional record/bit from my DNS servers that is essentially DNS equivalent of HTTP Alt-Svc to say I support DoH)?

<comment><USER5> The entire premise of DoH and DoT are that you don't want to use your ISP's DNS. Nobody needs to "come out and say that"; it's whole point. They're both mechanisms to tunnel DNS out of your network to somewhere safer. The difference between the two is that DoT has a network-operator kill switch, and DoH doesn't.

<comment><USER4> > The entire premise of DoH and DoT are that you don't want to use your ISP's > DNS

This is the discussion separation I am requesting. There is benefit of in- transit encryption regardless of who the endpoint is. There _may_ also be benefits to defaulting to non-system DNS, but those discussions should be separate IMO.

<comment><USER6> What's the benefit of encrypting your DNS traffic to the ISP? There isn't anyone between otherwise they'd be your ISP. The point of DoH is to provide a way to configure DNS to something that isn't the ISP without the ISP being able to block it, snoop it, or intercept it. There is no separating the concept because if you're doing it to the ISP then the reasons for doing it are now invalid.

Also "not the ISP" != "non-system dns". Don't conflate that just because browsers were first/most famous to implement.

<comment><USER4> I specifically said endpoints and not ISP because the proliferation of encrypted DNS traffic is a good thing regardless. Also, there is still MITM potential between you and your ISP's DNS server as many ISPs share pipes that have been known to be tapped at different PoPs. The value of encryption to DNS servers is obvious, even if we pretended that user-to-ISP traffic was all we were talking about.

I liken these arguments to the same ones made in the past by our more-naive selves concerning encryption of corporate intranet traffic. We should be smarter now.

<comment><USER6> Replace ISP with endpoint, the conversation is the same. Simply encrypting to whatever the other end of the cable tells you to is false security. If the other end of the cable is malicious it will just lie about what your DoH server is. You can establish identity with a CA infrastructure and a precoded default destination, this is what DoH is about

Encrypting traffic, internal or external, only makes sense if you don't trust whatever you happen to plug in to tell you what to trust. OWE in Wi-Fi has similar faults.

</comment></comment></comment></comment></comment></comment><comment><USER7> Great summary. The user's threat model could be more succinct and address the question of what the end user is really protecting, and from whom. Then the features of the different protocols would be mitigations to that risk.

I've got a horse in the race, but I might use it as an example input for the method in this post: [https://www.qtra.io/simpletm](https://www.qtra.io/simpletm)

Short version would be, <end user> wants to protect the <confidentiality and integrity> of the <contents of their browsing history> from:

<isps> who <using it to target ads and sell to 3rd parties>

<intelligence agencies> who <are engaged in bulk collection>

<employers> who <use browsing history as leverage to manage out staff.>

etc.

We need the who, why, and what, before the how. Excellent summary all around though.

</comment><comment><USER8> Great discussion. I also think that the more we talk about it, the more and better solutions will start appearing to solve all the problems that come with choosing DNS providers. There is also one issue that we don't discuss too much. I understand that by selecting a private DNS resolver over the ISPs DNS, you still provide your info to that resolver's operator, and they can monetize it. And here I see one crucial thing - choosing the provider that HAS to collect your data over choosing the provider that doesn't do it. Correct me if I'm wrong, but basically, any DNS provider located in the USA is required to collect and store your data by law. So it's not only easy to use that data for monetization, but also they would be required to give all that info to the gov if needed. I was going through a list of most popular DNS providers, and I didn't quite like what I saw. But I found this Trust DNS app ([https://surfshark.com/trust-dns](https://surfshark.com/trust-dns)) that as a product from Surfshark is also located in the British Virgin Islands. So technically, they are not required to collect any users' data, and they state that they don't. There is also an option to switch between DoH and DoT. The only downside is that it's only for Android right now. Any thoughts about it? Are there more options that are not based in the USA or other 14 eyes countries (or well, China)?

</comment><comment><USER9> More attention needs to be paid to QName minimization, IMO.

DNSSEC is the only truly hierarchical PKI with pervasive name constraints that we have. That makes it very valuable, even if it is difficult to leverage to solve other problems.

Like all PKIs, DNSSEC is vulnerable to authorities (that is, authoritative servers) acting as MITMs. WebPKI makes it very easy for nation states to acquire MITM credentials: just subvert a national CA and you're done because the WebPKI lacks name constraints. A real PKI makes this harder, because a nation state can only legally subvert authorities within its jurisdiction.

The root zone is special though, as long as there is only one, and that's why the root zone operators make such a special show of key signing events: to increase confidence that it has not been subverted. But the root zone is still a concern -- if not today, then tomorrow.

Enter QName minimization. By sending minimized queries -at the cost of having to do more queries, thus having higher latency- one can keep intermediate authorities from observing the full domainname that one is attempting to resolve. That makes is more difficult for authorities to MITM specific targets.

<comment><USER5> The opposite claim is closer to the truth regarding "true hierarchical PKIs" versus the WebPKI. In reality, if a state suborns a CA to misissue certificates, Mozilla and Chromium will nuke that CA from orbit. You don't have to wonder about this; Mozilla and Chromium casually destroyed some of the largest commercial CAs. Moreover, misissuance is very likely to be detected, because Chromium and Mozilla have moved towards requiring Certificate Transparency participation.

On the other hand, if a state suborns its DNS providers to inject bogus keys, there is no recourse. You can't revoke .COM. Is Google going to become Google.IO? Wait, no, .IO is controlled by Five Eyes as well.

<comment><USER9> I don't buy that.

First, we see that China, for example, has enormous market power that it uses to get its way in just about every controversy.

Second, CAs will get kicked off the TA list only if they get caught.

<comment><USER5> I addressed that in my comment: it's never been easier to get caught, since certificate issuance is now cryptographically logged.

</comment></comment></comment></comment><comment><USER10> > From my perspective, I believe that we should work on pushing for wider > adoption of DNSSEC, because I continue to come back to the core problem of > DNS results not being authenticated or verified.

Can we replace DNSSEC with something that doesn't give government control over root zones?

<comment><USER11> This comes to mind: [https://sockpuppet.org/blog/2015/01/15/against- dnssec/](https://sockpuppet.org/blog/2015/01/15/against-dnssec/)

<comment><USER10> Very yes. `tptacek has great takes on DNSSEC

<comment><USER12> tptacek has a tendency to deny the existence of problems that he doesn't consider to be important, and thus his criticism of DNSSEC tends to go a little bit too far. He does a great job of explaining what's wrong with DNSSEC and I agree with the general conclusion that it's not worth the trouble, but his blindspots are a bit annoying.

<comment><USER5> Feel free to provide an example!

<comment><USER13> Not OP, but in particular one of the major points of your post about the issues surrounding DNSSEC is that TLDs are controlled ultimately by the government with jurisdiction over them. Specifically I'm talking about the section titled "DNSSEC is a Government-Controlled PKI". You present the argument that the existing CA system is more protected from government interference than registrars. I think that argument is pretty weak considering that for the most part we're just talking about DV certificates and all a DV certificate from a CA signifies is that that particular CA verified that the entity they issued the certificate to controlled that domain. When ICE seizes a domain name, they can get a DV certificate no problem, they own the domain at that point. If Muammar Gaddafi ordered bit.ly to be seized for some hypothetical "Bureau of Information Technology", he could have absolutely gone to any CA afterwards and gotten a DV certificate.

Why should a litany of CAs with authority to sign certificates for largely any TLD be preferable to only allow the entity that's authoritative on domain ownership being able to sign valid certificates? DV certs are just indirectly checking with DNS anyways and there are pitfalls in that process and no real benefits in the end.

<comment><USER5> Google and Mozilla can (and likely would) destroy the CA that allowed an unauthorized Gmail.com (not DNSSEC-signed, by the way) certificate to be issued. They have destroyed some of the largest commercial CAs in the industry for less.

Neither Google nor Mozilla can revoke .COM, and the USG has repeatedly used its authority over .COM for public policy ends.

<comment><USER13> Yet they haven't, and never will remove CAs that sign certificates for domains that ICE has seized. If the US government actually took the extraordinary step of just flat out taking ownership of gmail.com under the guise of some anti- trust action then they would meet the requirements to be issued a DV cert anyways.

If the USG actually did seize control over gmail.com, and they had some CA sign a DV cert for them, how would that constitute a misissuance? The whole point of a DV certificate is that it only attests that the owner of the cert is the owner of the domain. The scenario we're talking about is a government becoming the new owners of a domain name.

But regardless of that, mitigating government overreach by means of having hundreds of sacrificial CAs is a pretty poor solution. What happens if Let's Encrypt gets blacklisted? That's a huge chunk of the internet gone right there until everyone gets around to replacing them. IANA doesn't have to have the root and TLDs delegated to a single entity. DNSSEC could be replaced by something requiring signatures from multiple different entities in multiple different jurisdictions to actually solve this problem. Have the root and global TLDs signed by at least 3 out of 5 entities with no more than 2 of those being from a Five Eyes nation. IANA already has a mediation process to resolve disputes with domain names, there's no reason why legitimate domain seizures couldn't be subjected to that process rather than just a unilateral court order to the registrar.

As for killing large existing CAs though, just look at how egregious Symantec was before Mozilla and Google ultimately decided to pull the plug. For years there'd be new ever more creative ways in which Symantec misissued certificates and their responses always seemed to be some variant of "Sorry, we didn't know we weren't supposed to issue certificates to entities other than the owner". There is still considerable friction to removing misbehaving CAs. As a matter of fact, Symantec in particular did misissue certificates for google.com back in 2015 during that whole test certificate debacle.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER14> I built a simple DNS (and HTTP headers, and SSL) tool that some may like - [https://DNSApe.com](https://DNSApe.com) \- and DNSSEC lookups are coming soon!

</comment><comment><USER5> This is the kind of analysis that makes sense if you read RFCs at face value, or columns from network engineers at CircleID. It does not reflect the real world.

Here's a breakdown of the real world implications of these three protocols. They are in fact simpler than this analysis suggests.

DoT and DoH are related efforts. DNSSEC is unrelated to DoT or DOH.

DNSSEC is a ~25-year-old effort to sign DNS records, so that attackers who control caches can't spoof records. In 25 years, through 3 revisions of the protocol, 2 of them major, DNSSEC has seen practically no deployment outside of Europe, where registrars enable it automatically (and thus control the associated keys). In particular: with the exception of Cloud Flare, which provides DNSSEC services, and some Paypal domains, no major tech companies sign their zones with DNSSEC.

For a bunch of reasons, DNSSEC is probably a dead letter (I'm not calling time of death yet, but I expect to within in the next 6 months). In particular: protocol-level defenses against record spoofing just aren't very relevant. Much of the Internet runs on TLS (if you don't, DNSSEC is the least of your problems). The WebPKI (TLS CA system) has in-progress mitigations against spoofing, and more in the pipeline, and all of them put together will cost a tiny fraction of what DNSSEC will cost. For a year or two, SMTP TLS was the great hope of DNSSEC deployment, the last major mainstream application that had a security gap addressable directly with DNSSEC. But then the major mail providers all ( _all_ ) got together and came up with SMTP-STS to address that gap directly.

It is hard to think of a real threat that end-user adoption of DNSSEC would address. I like to joke that the DNSSEC root keys could end up on Pastebin today and network engineers around the world could sleep the resulting Jira tickets for a week or two before deciding whether to just backlog them. Except I'm not really joking about that at all.

Which brings us to DoT and DoH.

The problem you do actually have with DNS today, especially if you're in the US, is that your large ISP is abusing DNS to monetize your traffic. I'm on AT&T, and, with their standard DNS, if I type a nonexistent domain into my browser, I'm on an AT&T landing page. ISPs are actively attacking DNS today for all their customers.

DoT and DoH tunnel DNS out of untrusted networks to a resolver on a hopefully more-trusted network (that is: there isn't that much point to DoT'ing or DoH'ing to your ISP's nameservers; you care about DoH/DoT if you want to use a third-party resolver, which you should).

Both DoT and DoH use TLS to secure DNS lookups. Protocol nerds will have fascinating discussions about the tradeoffs between the two approaches, but from a practical perspective, for home users, here's the real difference: DoT has a kill-switch for network operators, and DoH doesn't. A network operator that doesn't want you DoT'ing can stop you, because DoT deliberately runs on an oddball port. DoH, on the other hand, very deliberately runs on 443/tcp, so that it costs more to filter.

Do you want a kill switch in your DNS privacy tunnel? That depends on who you are. If you're an enterprise network operator, you want the kill switch, so you can (reasonably!) force your endpoints to funnel their DNS lookups through recursors you control and monitor. If you're a home user, you very much do not want the kill switch; if you decide you want to use Google DNS rather than your In-Flight Wi-Fi provider's DNS, it should not be up to your airline or their wifi provider whether you do that.

<comment><USER15> > with the exception of Cloud Flare, which provides DNSSEC services, and some > Paypal domains, no major tech companies sign their zones with DNSSEC.

I was curious and tested the domains of the Fortune 500 as compiled at [https://blog.dofo.com/fortune-500-domain- names/](https://blog.dofo.com/fortune-500-domain-names/). Here's the list I came up with:

      aetna.com
      boozallen.com
      centene.com
      cmsenergy.com
      comcastcorporation.com
      key.com
      magellanhealth.com
      paypal.com
      raymondjames.com
      raytheon.com
      simon.com
      theice.com
      thorindustries.com
      usaa.com
      utc.com
      xerox.com

I'm not agreeing or disagreeing with your statements or sentiments; I just didn't want my efforts to go to waste.

<comment><USER5> No, this is super helpful.

Booz Allen, United Technologies, and Raytheon are government contractors. For several years, the USG had a policy of requiring DNSSEC for its own internal zones and had sent signals that its partners would require DNSSEC as well. But a year and a half ago, the USG rescinded the requirement, and no longer recommends DNSSEC even for internal .gov zones.

Comcast is an interesting example because their adoption of DNSSEC broke a major project rollout: the week HBO NOW became available, it was unreachable from their networks due to a DNSSEC misconfiguration.

Another data point: of the Alexa Top 50 in the US, only FORCE.COM is DNSSEC- signed. But SALESFORCE.COM isn't; similarly, Paypal is signed, but its popular subsidiaries like Braintree and Venmo aren't. Or take the Moz 500; you'll find Mozilla is signed, as are a bunch of .GOV sites (see above), some European sites, and, if we're keeping score in the US, also Fox News, Time, Mediafire, and, weirdly, Themeforest. For reasons I do not understand, Stanford, Berkeley, and CMU have taken the time to sign as well. And that's it.

(For others reading: you can take a list of domains and do a quick spot check for DNS by just running "host -t ds (domain)".)

<comment><USER16> Where is there a rescindment of usg public dnssec requirements? It's still a requirement even for fedramp-low.

[https://www.fedramp.gov/search- results/?search=dnssec](https://www.fedramp.gov/search-results/?search=dnssec)

<comment><USER5> [https://cloud.gov/docs/compliance/domain- standards/#dnssec](https://cloud.gov/docs/compliance/domain-standards/#dnssec)

Note that cloud.gov is FedRAMP-authorized and does not itself support DNSSEC, which I point out in addition to the rationale cloud.gov provides on that link (which answers your direct question).

</comment></comment></comment></comment></comment><comment><USER17> The root issue (pun not intended) is that DNS is not fully encrypted, not even point-to-point, DNSSEC only provides an integrity check, no encryption; DoT/DoH is only used between a resolver and a client as a last-mile solution. No robust security can be derived from the absence of full encryption - <USER2> is none, ultimately the traffic is sent in clear over the Internet backbone. You must make your own choice of trust for sending traffic in cleartext - <USER3> local network at home/school/hotel, your rented VPS provider, your ISP, or Google/CloudFlare.

Given that CloudFlare already has 13%+ market share, as an end-user, if my primary threat is eavesdropping in the middle, not eavesdropping by CloudFlare (or by NSA's subpoena or NSL), it's actually reasonable to trust the tech giants. Using CloudFlare's DoH server helps me to end-to-end encrypt my query of 13% of the domain names! Also, CloudFlare has business plan to adopt private encrypted DNS links to other authoritative servers, and they already have a private channel with Facebook, it's even better! Don't start sending hate replies, I know the bigger picture is that it allows CloudFlare to effectively monopolize DNS is extremely harmful in the long run, and I'm worried. But I simply don't see a perfect alternative solution that allows one to self-host a resolver at home with end-to-end encryption to communicate with authoritative servers.

You see, the end results that everyone should work towards is introducing encryption to the missing resolver-authoritative server link.

DNSCurve was developed in 2005 as a practical and high-performance protocol for deployment on authoritative servers, and fill the missing link, the protocol is great, DNSCrypt was its direct descendant for last-mile client- resolver encryption.

It was ultimately not adopted, partially because of many hate DJB's personal style of taking an aggressive position over technical issue, especially his attack on DNSSEC. But also, the final reason was that the ICANN has stated that, in the case of the DNS Root zone servers, DNSCurve will never ever be implemented. First because its threat model doesn't include the authoritative servers themselves, and it gives them the private key. This is fine for private use, but the DNS root servers are controlled by many political entities, and ICANN doesn't believe trusting them is acceptable. DNSSEC was designed in a way to avoid giving the ultimate key to root servers, but not DNSCurve. Also, DJB hated DNSSEC so much that DNSSEC cannot be used to sign the pubkey for DNSCurve resolver - oops!

In the future, we should work toward a solution for resolver-authoritative server encryption. In the article, the author mentioned DNS-over-TLS is a potential solution, if so, we should push to deploy DNS-over-TLS on major authoritative servers, or perhaps one day, on the root servers if a solution has been worked out.

Only then, it could allow us to say farewell to CloudFlare's DNS and Google's DoH resolvers.

* [https://en.wikipedia.org/wiki/DNSCurve](https://en.wikipedia.org/wiki/DNSCurve)

</comment><comment><USER9> [http://stats.dnssec-tools.org/#](http://stats.dnssec-tools.org/#)</comment>


Indie developer wants to fix game discovery with $1.2M in funding - <AUTHOR> http://venturebeat.com/2015/02/25/shark-punch-raises-1-2m-to-fix-game-discovery-with-its-playfield-community-platform/

<comment><AUTHOR> Jiri from Shark Punch here. We had some issues with signups in the first hour, but it's all been fixed now.

Would appreciate you giving it a shot!</comment>


U.S. Economy Falls Short of Obama's Forecasts Every Year - <AUTHOR> https://www.scedast.com/visualization/us-economy-falls-short-of-obamas-forecasts/?utm_campaign=post&utm_source=hackernews&utm_medium=social <comment><USER2> Obviously it means we aren't working hard enough. Activate shock collars.</comment>


Robots will soon replace human fruit pickers - <AUTHOR> http://spectrum.ieee.org/automaton/robotics/industrial-robots/sri-spin-off-abundant-robotics-developing-autonomous-apple-vacuum <comment><USER2> Robots won't take all our jobs yet; apparently they're aiming for the low- hanging fruit.

</comment><comment><USER3> Once mainstreamed, I wonder how this will affect the immigration debates, probably in favor of more closed borders.</comment>


J. Robert Schrieffer (1931–2019) - <AUTHOR> https://www.nature.com/articles/d41586-019-03015-3 <comment><USER2> From the obit in The New York Times:

Late in life, Dr. Schrieffer's love of fast cars ended in tragedy. In September 2004, he was driving from San Francisco to Santa Barbara, Calif., when his car, traveling at more than 100 miles per hour, slammed into a van, killing a man and injuring seven other people.

Dr. Schrieffer, whose Florida driver's license was suspended, pleaded no contest to felony vehicular manslaughter and apologized to the victims and their families. He was sentenced to two years in prison and released after serving one year.

Florida State placed Dr. Schrieffer on leave after the incident, and he retired in 2006.

Richard Klemm, a professor of physics at the University of Central Florida, who collaborated with Dr. Schrieffer on a research problem in the 1980s, said he had tried to contact him but did not hear back.

"He wouldn't communicate with anyone," he said. "He apparently didn't want to have anything to do with his community any more after what happened."

</comment><comment><USER3> I remember going to an interesting lecture by Dr. Schrieffer back in (1997?) where he talked about effectively fractional quantum numbers in the electron wavefunctions in long chained molecules like DNA. I don't think much came of it, but it was an interesting idea.

Here is a review by Frank Wilczek: [http://www.frankwilczek.com/Wilczek_Easy_Pieces/330_Fraction...](http://www.frankwilczek.com/Wilczek_Easy_Pieces/330_Fractional_Quantum_Numbers.pdf)

</comment><comment><USER4> pretty unbelievable they don't mention this [https://www.foxnews.com/story/nobel-prize-winning- physicist-...](https://www.foxnews.com/story/nobel-prize-winning-physicist- gets-two-years-in-prison-for-deadly-crash)

<comment><USER2> In fairness, the obit in Nature is strictly about his career and discoveries. Let's see what the NYT obit looks like. However, thanks for pointing this out. I save obits of notable people, and in this case I concatenated both articles.</comment></comment>


Ask HN: Using spare VPS compute for? - <AUTHOR>

I have a VPS on DigitalOcean and I use it just to host my personal website which takes almost no resources. What are some cool (legal) things I can do with the extra compute I have?<p>It is the smallest droplet (512 MB, 1 vCPU), but I figured that's enough for something. <comment><USER2> Maybe you could run a Tor exit node, or setup a public mirror of your favorite code bundles. Or you could donate the processing power to a large distributed- computing project like Stanford's Folding-At-Home ([http://folding.stanford.edu/](http://folding.stanford.edu/)).

Or you could always just turn your server into a little bitcoin miner. Probably won't make any money from it, but it might be fun to do regardless.

<comment><USER3> Digital Ocean will likely ban you for bitcoin mining (and other software that constantly stresses any resources.) A quick search reveals tat a Tor exit node is likely to get you banned on there too. Cut-rate cloud providers like Digital Ocean are basically in the business of getting as many low use customers and getting rid of the everyone else [as long as it doesn't totally wreck their reputation/goodwill].

</comment></comment><comment><USER4> Cats! You can always post cats!

Now, on a more serious note: you could host your friend's sites (and probably can do a split of the payments), use it as a host for various tests (if you are a developer).

If you are living in a non US country you could make an instance in US then use it as a proxy to access various services that requires you to be in US (hulu, netflix, etc)

</comment><comment><USER5> You can search for alien life or help cancer research.

[https://en.wikipedia.org/wiki/List_of_distributed_computing_...](https://en.wikipedia.org/wiki/List_of_distributed_computing_projects)

</comment><comment><USER6> You could host a mumble/vent for a gaming clan/scene you're involved with.

Run a dedicated server for whatever games you could manage with 512 mb ram.

</comment><comment><USER7> Use can host and use ownCloud [https://owncloud.org/](https://owncloud.org/)

</comment><comment><USER8> You can run your own secure vpn, to access web when browsing in coffee shop and in similar places.

<comment><USER9> Which software(linux) would you recommend for this on the server side ?

<comment><USER8> Does not really matter, since majority of solutions will work on all flavors. I can speculate that most of tutorials will by on Ubuntu.

But you can look here for basically on-liner: [https://news.ycombinator.com/item?id=7586775](https://news.ycombinator.com/item?id=7586775) Also there is "full" solutions [https://www.digitalocean.com/community/tutorials/how-to- inst...](https://www.digitalocean.com/community/tutorials/how-to-install- openvpn-access-server-on-ubuntu-12-04)</comment></comment></comment>


Being Laid Off - <AUTHOR> What does it mean to be an engineer, laid off. It means I am studying algorithm challenges to get a job I have been doing for 20+ years. I will do it because that is the tacit agreement we all have made. To torture our fellows in the name of "not letting unqualified people in", so that we can feel deserved of those big checks.
--You can't virtual whiteboard a dp solution to my lss problem with contrived memory constraints?  Sorry I just can't take the risk of letting you write code on my team. Excuse me while I go back to copying and pasting stack overflow into my oauth library I'm rolling for scratch for some reason.--
We have all seen the coder who talked a good game and nested 10 loops deep and bragged about how much work their code was doing.  But apparently detecting them is impossible without puzzles in high stress environments.
Being laid off means I have to stare you in the face knowing full well I have more experience and make better decisions than you in most practical situations, but you will turn me away because I didn't catch your gotcha question where you cleverly described a scenario that could be solved using a single server. But instead I designed a system to scale to several thousand tps quickly, and forgot to ask scale requirements because this is the third architecture interview today and all have been aimed at the scale of your org.
Being laid off means that I am suddenly responsible for adding interview puzzles to my skillset after spending 11 months saving my company millions of dollars writing real software.
I have learned all my skills on the job. Truth is I will do better if you put me in a room with better people. If you measured the ability to develop expertise rapidly you would see that I'm unparalleled, if I may.  But you want me to tower of hannoi your fizzbuzz.
I'm gonna go play animal crossing and see if I can forget writing this post.  I need to start doing leetcode questions in the morning. <comment><USER3> The tests are used to enforce age discrimination bias indirectly to shield from lawsuits. But at the same time I can't stand those employers who are quick to hire and quick to fire usually they're too lazy to do decent job of vetting. Its a tough question how to analyze someone's capabilities. So there is no easy answer. If you were working tons of unpaid overtime your github may be a ghost town who has time for side projects when the main job is asking for 80+ hours a week. There's no easy answer here except we should stick to practical questions and avoid testing for computer science concepts which are only valuable for university teaching or research positions in computer science.

</comment><comment><USER4> Small software companies. (Not startups.) They really need people who can do the actual job, so they tend to interview for that. I've never done an "algo" challenge in an interview. I've been asked to write code though to solve a contrived problem, but that's unavoidable.

</comment><comment><USER5> I got laid off on my fortieth birthday. It hasn't bothered me at all. Been working on building my own SaaS. I refuse to take algo interviews. Recruiter is hitting me up to work at same place as before for more money. Get your hustle on and get your last laugh.

</comment><comment><USER6> I can relate to the interview part.

I have applied to horizontal positions in my company and been turned down so many times. They ask lots of questions that you would never hear on the job.

They had me working on obscure systems like FileNet and Neoxam. I even filled roles above my level, like tech lead (I'm a mid level dev). I got my AWS certs and it still took me a long time to move into a new position. It wasn't even a promotion, just a lateral.

The biggest thing I have learnt on the job is that you should not care about what the company or most others think. They are only there to F you over because they have this goal (and you should too): get in, make as much money as possible, get out as soon as possible. Skill and value mean nothing. Use the politics and people to get yourself ahead.

</comment><comment><USER7> I am sorry for your pain. I have never had the 'algorithm quiz' interview, and am proud to say I have never given that quiz.

IMHO If you are good with algorithms, your resume should show it, and you should be able to speak to it. If your resume does not show the skill then they should not grill you on it. Of course as a interviewer you must be skilled enough to recognize a valid understanding of the algorithm the CANDIDATE explains not just the ones you as an interviewer knows.

That said, I have seen plenty of resumes where people will list off doing in 3 months, that which would take a genius a year. Validating that you actually did XYZZY verses just attended a meeting on topic is a key objective of the interview.

</comment><comment><USER8> I am in the same boat now and, frankly, I am starting to care less. My medium to long term plan is to commercialize the tech that I've been developing privately for several years now. Were it not for that I'd happily throw myself into studying a big book of algorithms. However I resent spending time that would have otherwise been spent developing my tech -- which is about 1½ years from being ready, (still some R&D and several layers to build out) -- studying algorithms that most (standard) library provide.

</comment><comment><USER9> I feel your pain. Stay strong. Good luck.

</comment><comment><USER10> If you were in charge of the interview process, how would you design it?

<comment><USER11> The only way that is good is to call the candidate onsite[0] for a day and give them two practical assignments[1]. First half-day assignment without internet connection but access to books, and the other with internet connection. The candidate is free to ask questions to the team members at anytime[2]. Then at the end of the day have a 1-hour group discussion(not interview) conducted by the team on how the candidate attempted to solve these two assignments.

[0] This can be done remotely, but to avoid cheats the team members will have to remotely monitor the candidate's screen and so will be wasting their own time.

[1] Assignments should be from the challenges the team is facing during their day to day job or already resolved.

[2] Will provide indication of communication skills of the candidate.

This method will be both quick and better at finding right candidate for the team.

Edit: added the communication part during the assignment phase.

<comment><USER12> And then you will see threads complaining about wasting a whole day just to be rejected, spying on your computer remotely or focusing too much on very narrow domain knowledge.

Taking your idea to extreme would be having a long period of evaluation period, maybe a week or month, but for obvious reasons this wouldn't work either.

</comment></comment></comment><comment><USER13> This is why I am going back to school for an MBA and looking for roles outside of software engineering. Being a SWE is a great way to make decent money immediately out of school, but the reality is that, unless you're a true standout, it's hard to justify your cost as a senior engineer versus someone a few years out of school. You might think you can "make better decisions in most practical situations," but the reality is that you probably can't - <USER2> least not in a meaningful enough way to justify your significantly inflated cost versus a younger engineer to an employer. The experience curve in pure software engineering is asymptotic or perhaps logarithmic at best, and the situations in which additional experience matters are rare. In contrast, the impact of a good manager, recruiter, financier and so on is limitless. My peers that went into investment banking, consulting, and the corporate world are now getting director and vp titles, making partner at prestigious firms and so on. In 5 years they will be blowing me away financially unless I change something.

Like you, I've come to realize our interview process is stupid, despite taking part in it probably 100s of times by now. In fact, I think the entire concept of rank and hierarchy in software engineering is stupid. I'm tired of the pendantic arguments we get into, the endless architecture debates, design docs, managers using phrases like "impact at scale", listening to L7 wind bags spout off abstract technobabble and so on. In the end, I had to conclude that most of it is bullshit. I've seen the pattern enough times at some of the most iconic companies of our generation to determine that this is actually the way it works in most cases. It's as though we took the corporate hierarchy from 1950's GE and tried to apply it to the software engineering profession, when, in reality, software is much more like a trade. Ultimately, I think it stems from some sort of superiority complex or need to feel as though we are progressing towards something, so we set up a bunch of rules and hoops to make it difficult for others to reach our status as a vaunted FAANG engineer.

Ultimately, I had to look at myself in the mirror and think about my business value. For now, it's possible to ride the tidal wave of demand for software engineers at a select group of companies that happened to stumble upon a money printing machine. However, I'm not particularly confident that abundance will continue. My advice would be to take this as an opportunity to pivot into something that will allow you to more dramatically differentiate yourself professionally. Everyone and their mom is studying computer science these days. Companies are discovering they can hire remote developers. And, as you're discovering, this is no career path for old souls.

<comment><USER14> Yeah but soft engr you can get into a project where you get over the initial hump and can coast, do like 20 hours a week of work. Get paid pretty solid. Work remote and still do a fine job. Plenty of time to concentrate on your own stuff.

All of those other roles you're going to have to have facetime always. Playing politics. Is that worth it? Maybe.</comment></comment>


Shepard tone - <AUTHOR> http://en.wikipedia.org/wiki/Shepard_tone

<comment><USER2> It's also possible to make such a thing as a shepard rhythm, using much the same principles. It's a lot harder though. Here's an excellent example: <http://stretta.bandcamp.com/track/as-clean-as-fire>

I highly reccommend listening to and purchasing the whole album if you like bach style soothing-yet-maths-based music.

<comment><USER3> That's pretty cool! Is there an analysis of this song (or album) somewhere?

<comment><USER2> There's this blurb he wrote on the soundcloud page:

 _"When I see a photo of a modular synthesizer, I wonder, as I think many others do, what the thing sounds like. What possibilities lurk within this strange hardware? I want to hear something orchestrated and controlled. I want to hear someone commanding the instrument with authority, not merely floating on waves of serendipity. I want to hear something composed for the instrument, leveraging its strengths, not a orchestration of an existing composition.

'A Funneled Stone' is a pure modular synth release, tracked the old-shool, 1970's way: one monophonic line at a time. Every sound you hear was created, patched and recorded for that moment in time. When a new sound is needed, the patch is torn down and a new one is built. Polyphony is achieved by tracking each voice individually.

A modular album is, by definition, unapologetically synthetic. I also tried to take a more minimalist approach to orchestration, so the individual sounds can be more fully isolated and appreciated. I spent much of the final month of production taking elements out, and editing for length. Sometimes this results in the remaining elements merely hinting at the underlying harmonic movement.

As you can imagine, this process is very time-consuming, but fun. I hope you enjoy the results as much as I enjoyed creating it."_

For more super-hi-tech-math-music have a listen to this: [http://detroitunderground.net/blog/2012/05/03/vaetxh- libet-t...](http://detroitunderground.net/blog/2012/05/03/vaetxh-libet-tones- du17/)

That page has technical descriptions of what's happening in each track. Risset illusions are just the start of the psychoacoustic trickery which goes on in that record.

A word of warning however: unlike the stretta stuff, the Vaetxh tracks are _REALLY NOT SOOTHING._ If you are soothed by them, seek help.

</comment></comment></comment><comment><USER4> Hm, to me it sounds like a sequence of overlapping falling tones. There's a noticeable point where the falling tone is supplemented by one above it. This first happens about 20s in.

 _edit_ The noticeable change seems to be very close to 22s.

<comment><USER5> It does sound like an auditory illusion to many, as not everyone can isolate tones in a mixed sound.

I'm curious if you happen to be a musician or a sound engineer?

<comment><USER4> Nope, I'm not a musician or sound guy. I'm a coder. But I do enjoy music.

That falling tone on the Wikipedia page sounds like something from one of _coil_ 's tracks. I can't pinpoint which one though? they use similar sounds here <http://www.youtube.com/watch?v=c6IR90tgtDI> and here <http://www.youtube.com/watch?v=9_R-mXRONh0>

_edit_ Expectation may pay a part - when told "come and listen to this 'endlessly falling' auditory illusion" of course I'm going to try and pick out the subtle seams. I might not notice if it was in the background, unannounced.

</comment></comment></comment><comment> robert-boehnke This effect was also used for the infinite stairs in Super Mario 64

[http://www.youtube.com/watch?feature=player_detailpage&v...](http://www.youtube.com/watch?feature=player_detailpage&v=70J8SOX5w64#t=30s)

<comment><USER6> Yes! until the batpod replaced it from collective consciousness. Thanks for the memories!

</comment></comment><comment><USER7> Sounds like gödel, escher, bach <http://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach>

<comment><USER8> These are mentioned at least once in GEB.

</comment></comment><comment><USER9> Águas de Março is a famous Brazilian song by Antonio Carlos Jobim that employs Shepard tones. It is very easy to hear the use of the technique in the song: <http://www.youtube.com/watch?v=srfP2JlH6ls>

From Wikipedia: <http://en.wikipedia.org/wiki/Waters_of_March>

The inspiration for "Águas de Março" comes from Rio de Janeiro's rainiest month. March is typically marked by sudden storms with heavy rains and strong winds that cause flooding in many places around the city. The lyrics and the music have a constant downward progression much like the water torrent from those rains flowing in the gutters, which typically would carry sticks, stones, bits of glass, and almost everything and anything. The orchestration creates the illusion of the constant descending of notes much like Shepard tones.

</comment><comment><USER10> I hear a loop ~22 seconds long of a falling tone. I can clearly hear the higher pitch tone fade in at the start of each cycle. I'm not such I get the "gets no higher or lower".

I'm listening to this version: <http://www.youtube.com/watch?v=DfJa3IC1txI>

</comment><comment><USER11> Why is this the top HN story? There are tons of perceptual audio examples that have been around for decades. Check out Curtis Road's Computer Music book for an in-depth analysis of this technique and more.

<comment><USER12> Because it's cool and hackers like cool things. I would never have known about this if it didn't make it to the top of HN.

<comment><USER11> So can we come up with an meaningful definition of 'cool'? There is an infinite number of arbitrary 'cool' things in the world - audio perception is definitely interesting, but so is granular synthesis, binaural audio and physical modelling.

I'm just not sure if the role of HN is to educate users on 'cool' things. Maybe its because most of the US is currently sleeping...

<comment><USER12> _On-Topic: Anything that good hackers would find interesting. That includes more than hacking and startups. If you had to reduce it to a sentence, the answer might be: anything that gratifies one's intellectual curiosity._

<http://ycombinator.com/newsguidelines.html>

</comment></comment></comment></comment><comment><USER13> Cool find. It says that it can be interpreted either as a rising or a falling tone, but I only can hear a falling one, even if I try to interpret it as rising.

<comment><USER14> Each tone is either rising or falling. This example is falling.

</comment></comment><comment><USER15> You can also use this illusion to make 'octveless notes': <http://www.jefftk.com/octaveless/></comment>


What Defines a Meme - <AUTHOR> http://www.smithsonianmag.com/arts-culture/What-Defines-a-Meme.html

<comment><USER2> > [Richard Dawkins] Already one of the world's foremost evolutionary > biologists

Minor nit: I don't think Dawkins is actually a foremost evolutionary biologist; though he's the foremost popularizer of evolutionary biology. I do enjoy his writing on it.

single page: [http://www.smithsonianmag.com/arts-culture/What-Defines-a- Me...](http://www.smithsonianmag.com/arts-culture/What-Defines-a- Meme.html?c=y&story=fullstory)

> They interact with each other and with other mental forces in the same > brain, in neighboring brains

I prefer this description than Dawkins' because it includes competition and combination within the one brain, something that genes cannot do within the one body.</comment>


Ask HN: How do you organize and manage database migrations? - <AUTHOR> When building services that rely on relational databases, in my case postgres, what are some best practices and tools to help manage schema changes?
We've been using migrations and doing it all manually but it's become a bottleneck and a little bit of a nightmare with multiple consumers of the database needing to make schema changes.
Another concern is multiple environments, from local development to staging and production. We're using docker-compose for local development which runs the entire "full" schema and then I'm manually applying the migration files to staging and production before we deploy.
I've looked at some projects like flywaydb[1] and liquibase[2] but both are not completely free and seem proprietary. Does anyone know of another open source system that could help manage database schema versioning and migrations?
Thanks so much HN, this is something that I have been struggling with.
1: https://flywaydb.org/
2: https://www.liquibase.org/ <comment><USER2> Honestly? I don't know of any tool that handles database migrations as well or as easily as Ruby on Rails and ActiveRecord.

I have literally spun up a barebones Rails app _just_ to manage my schema and migrations for a completely separate Python/PostgreSQL project before.

[https://guides.rubyonrails.org/v5.2/active_record_migrations...](https://guides.rubyonrails.org/v5.2/active_record_migrations.html)

<comment><USER3> I second this - migrations in Rails are fantastic. Even if you aren't using Rails you can get the styles and source code here [https://edgeguides.rubyonrails.org/active_record_migrations....](https://edgeguides.rubyonrails.org/active_record_migrations.html)

Large open source projects such as GitLab source code are also treasures to discover migration rules.

If you have a millions of users with high traffic tables you may want to check some migration guides for downtime, data migrations and batch processing here [https://docs.gitlab.com/ee/development/migration_style_guide...](https://docs.gitlab.com/ee/development/migration_style_guide.html)

If you want to have some concurrent ways of DDL operations you may benefit from [https://gitlab.com/gitlab- org/gitlab/blob/master/lib%2Fgitla...](https://gitlab.com/gitlab- org/gitlab/blob/master/lib%2Fgitlab%2Fdatabase%2Fmigration_helpers.rb)

<comment><USER4> That gitlab link was a good read, thanks!

One of the points was that migrations should always be reversible. I've struggled with this in the past when it comes to migrations that change or drop data. How do you write a downgrade script that recalls the previous data?

I've given up with downgrades now. I make sure to take a dB backup or snapshot before running an upgrade. I've never had to test this solution in a tense situation though...

<comment><USER3> One way is to keep a polymorphic (postgres unloggged) `backups` table with bson/json field to store any table's row. use this before and after migrations and backup and truncate this frequently.

</comment></comment></comment></comment><comment><USER5> I'm the author of an open-source schema management tool, Skeema [1], which uses a declarative infrastructure-as-code approach: you track your table definitions (and procs, funcs, etc) in a Git repo of *.sql files, typically one CREATE statement per file.

Instead of writing migrations, developers simply add / remove / modify these CREATE statements, going through the same pull request and code review process as for code. The tool knows how to diff the desired state (expressed in the repo) vs the actual state of any database environment (prod / stage / dev, etc) to generate the appropriate DDL.

It's a bit of a paradigm shift relative to traditional mysql and postgres migration tools, but it's an approach that has been used successfully by Facebook internally for nearly a decade. It's also a common approach in the SQL Server world. I've written a blog post [2] describing some of the advantages of the declarative approach.

Skeema currently only supports MySQL and MariaDB, but some declarative tools for Postgres include sqldef [3] and migra [4].

[1] [https://www.skeema.io](https://www.skeema.io)

[2] [https://www.skeema.io/blog/2019/01/18/declarative/](https://www.skeema.io/blog/2019/01/18/declarative/)

[3] [https://github.com/k0kubun/sqldef/](https://github.com/k0kubun/sqldef/)

[4] [https://github.com/djrobstep/migra](https://github.com/djrobstep/migra)

<comment><USER6> I was going to ask after rename support, but lack of it is documented over here along with a couple other things. [https://github.com/skeema/skeema/blob/master/doc/requirement...](https://github.com/skeema/skeema/blob/master/doc/requirements.md#responsibilities- for-the-user)

Interesting ideas though. Microsoft does something similar with a DACPAC where they record the schema and diff, but they also record a list of rename actions etc. and keep a table to track those "migrations" in addition to the diff process. [https://docs.microsoft.com/en-us/sql/relational- databases/da...](https://docs.microsoft.com/en-us/sql/relational- databases/data-tier-applications/data-tier-applications?view=sql-server-ver15) and/or [https://docs.microsoft.com/en-us/sql/ssdt/extract-publish- an...](https://docs.microsoft.com/en-us/sql/ssdt/extract-publish-and-register- dacpac-files?view=sql-server-ver15) — for rename and other SSDT details, see [https://docs.google.com/presentation/d/1DvC2gzCucjHFbGiBLa0R...](https://docs.google.com/presentation/d/1DvC2gzCucjHFbGiBLa0R5H8WZm48z2pDciY0aRcykBo/view?usp=sharing) (it's a "RefactorLog" if searching)

Having seen all that, the Rails ActiveRecord approach strikes a decent balance between specify migrations as imperative actions and keeping a declarative schema checked in. A comparison between the two approaches is at [https://blog.raph.ws/2019/01/migrations-vs- dacpacs/](https://blog.raph.ws/2019/01/migrations-vs-dacpacs/) but I find DACPAC to be useful but over-complicated for developers and not well-enough supported on non-windows hosts yet. (It basically requires VS for Windows or SSDT right now...)

Also, it's likely a migration system needs some kind of code review checks and automation around which actions you'd allow in your migrations if you don't have that already...

The next question after "how do you migrate?" is probably "how do you backup and how long would it take to restore after a bad migration and/or bad rollback?" These days your answer is probably either "no time at all" for smaller apps with SSDs or "it's all outsourced to the cloud," for the new cloud distributed data stores or for the smart DIY ones, "we only deploy one server at a time and can either afford the downtime or have copies in multiple AZs, perhaps eventual consistency..."

</comment></comment><comment><USER7> We built it ourselves.

* We have migrations written in SQL files named like 45_add_column_x_to_view_y.sql

* We track the migrations in git and we do merge requests and reviews

* We include a comment in each migration containing the original DDL of the object (1. it makes it easier for the reviewer to see what's changing, 2. it gives us a way to rollback, though it never happened so far)

* We track the applied migrations in a table _migrations_ looking like "2019-10-30T10:49:00" ! "45_add_column_x_to_view_y.sql"

* We have a CLI in the same repo that allow us to apply the migrations. It's 50 LOC of python that basically 1. gets the DB host/user/pwd from an .env, 2. checks the last applied migration in the table _migrations_ , 3. finds the new migrations to apply, 4. applies them following the order of the ids e.g. 45 before 46, 5. updates the table _migrations_

* We have 3 DBs (dev / staging / prod) in AWS RDS and I have an .env for each one of them

* We have a CI pipeline on the repo to automatically apply the merged migrations in staging and check if nothing goes wrong

* When the staging or dev is trashed, I delete it and recreate it from a snapshot of the prod (it takes 5 min) [https://stackoverflow.com/a/49878477/652669](https://stackoverflow.com/a/49878477/652669)

<comment><USER8> Which existing technologies did you look at and why didn't they fit your needs?

<comment><USER7> We looked at Alembic and our main issues were that

1\. we don't have the entire database modeled into SQLAlchemy and we don't plan to

2\. we have A LOT of views, triggers and functions and they are not very well supported by SQLAlchemy/Alembic (? I may be wrong)

<comment><USER9> Point 1 is not an issue IMO, you definitely don't need to model anything schema-wise into SQLAlchemy ORM to make Alembic useful, since Alembic operations translate pretty much directly to SQL. I've just built a project last month that uses Alembic but otherwise connects to the database and issues SQL directly.

Point 2 is indeed a big factor though. Alembic doesn't have much first-class operation support beyond data storage constructs, and if most you need is op.execute() (which runs plain SQL) one might just as well build their own tool like you did.

</comment></comment></comment></comment><comment><USER10> Flyway is free open source software (licensed under the Apache License 2.0).

There are pro and enterprise licenses in case you have additional requirements. These licenses as I understand it are proprietary and hence non- free but the the community edition absolutely is free software.

I've been using the community edition in multiple projects for several years now and it works great, particularly in the context of Spring Boot applications.

It might not be just as polished and well-integrated as ActiveRecord is for Ruby on Rails but keep in mind Flyway is supposed to be framework-independent so it can't be too tightly integrated.

That said, its integration with Spring Boot is really good. Just add it as a dependency and configuration and migrations during application startup will be taken care of automatically.

<comment><USER11> Been using Flyway for years with no problems and definitely open source, having contributed documentation changes in the past.

Repeatable migrations are great especially when combined with upserts in Postgres, if you keep config in your database as well.

</comment></comment><comment><USER12> I've been using Dbmate for a few years now across our entire stack. It does the job perfectly.

[https://github.com/amacneil/dbmate](https://github.com/amacneil/dbmate)

<comment><USER13> I can second Dbmate.

The process of a sql migration management is pretty simple and like other comments it's not hard to write something yourself which works well I just prefer to use something a little more battle tested in the deployment process.

Also it's a go app so you don't need to pull down a stupid amount of dependencies for it to work just pull from it's github release and mark it as executable.

</comment></comment><comment><USER14> Our stack heavily uses Django. We are therefor using Django's migration system. I worked with alembic before and some other Python based migration systems, but nothing comes close to the Django's simplicity.

We're heavy Postgres users and take full advantage of some of its features which are not very well supported by Django. Quite a number of our migrations involve `migrations.RunSQL` operations. That's fine. We've also developed a package [1] that extends Django with a lot of Postgres specific features. For example, recently we added support for partitioned tables. We can now set those up as normal Django models.

In order to reduce the number of f'ups, we also built a Github bot that inspects PR's for potentially disastrous migrations. For example, the Github bot will yell at you if you try to rename a field or blindly drop a table. The Github bot also makes sure your migration doesn't conflict with another migration that was just merged.

[1] [https://github.com/SectorLabs/django-postgres- extra](https://github.com/SectorLabs/django-postgres-extra)

</comment><comment><USER15> I use good old fashioned numbered SQL scripts. 01_foo.sql 02_bar.sql 03_baz.sql

In my local dev environment, I have a DB backup representing the current state of production. This is the starting point SQL scripts must be applied to. A deployment script restores this DB, then applies the SQL scripts against it. The entire deployment process takes about 2 seconds even with many gigs of data in the DB backup.

Sometimes I'll include an extra "dummy data" script applied at the end.

My local environment, Test, and Production are deployed in _almost_ exactly the same way. A few extra actions are done for Production of course. And Production is not restored as that would loose data. Local dev is optimized for a speedy restore/deploy.

Developers always want to treat the DB as if it's code. But it's not. It's a blob of valuable state that must be protected from corruption. You apply patches to that blob of state. The order patches occur __matters __. There 's no "re-build" like in pure code.

</comment><comment><USER16> Honestly, build it yourself. A good library of small shell scripts along with some well-commented .sql files will be easier to maintain and understand than any off the shelf migration utility.

</comment><comment><USER17> I've previously used some ORM-based migration tools, like Django ORM migrations (and when it was a separate tool, South), looked at others like Alembic. What I found is that they automate simplest cases, in turn making more complex cases even more complex.

So when I need to do some complex migration on a big database, I usually open postgres shell, open a transaction and I develop a migration like a code, in a REPL. And in case of these automatic tools, after I did that I have to go read their documentation, and port SQL to their syntax.

If I have junior members on the team who don't know SQL yet on a good level, easier tasks are automated by the tool, and for more complex task they completely lack skills developing a migration and have a much steeper wall to climb.

Another thing is that many tools have downgrade migrations. Downgrade migrations are a lie! How could I revert a migration that drops a NOT NULL column with some data? In case I really need to revert a migration I will write another forward migration. Which I did exactly zero times in more than ten years. So, writing downgrade migrations is a waste of time.

Another minor point is that sometimes migrations are running for a loooooong time. Not a second or two - it can be many hours. So I don't want this migration to start automatically during my deploy process. But I don't want to have it as a separate script, so on local dev installations these migrations just take the same `make migrate` route. It's much more convenient to take a part of a commented SQL and run it separately, than to take a part of a migration script written in Python and run it separately.

I'm using a Nomad tool [https://pypi.org/project/nomad/](https://pypi.org/project/nomad/) which is written in Python. I think any tool that supports plain SQL migrations, has dependencies between migrations and doesn't require to write downgrades would be acceptable.

</comment><comment><USER18> We have built projects where we used python Alembic or Ruby's Activerecord migrations to manage the db.

The actual application was in Go or nodejs.

I daresay that Activerecord and Alembic outclass Liquibase in a lot of things.

<comment><USER19> Another vote for Alembic, especially if you use SQLAlchemy in your app. I can't say enough positive things about the 1-2 punch of SQLAlchemy and Alembic if you are dealing with relational databases.

Some technical benefits to Alembic:

\- It will give you the structure for upgrades and downgrades.

\- Has a clean interface for DDL operations

\- Supports every (?) database that SA does

\- You can use it in "Offline" mode if you don't want to have Python and all the dependencies on the server or have to hand the migration off to someone else that has access.

\- The branch feature is really nifty if you are in advanced situations.

Some non-technical benefits with Alembic

\- It is open source

\- zzzeek, the author, is pretty active on here and has built both SQLAlchemy and Alembic so there is a lot of cohesion in styles.

\- The issue tracker is active and responsive

\- The code is stable (something you want in a migration tool) and is unlikely to go anywhere.

Highly recommend.

Edit: Formatting

</comment></comment><comment><USER20> I have used sqitch [http://sqitch.org/](http://sqitch.org/) in the past and found it good for postgres. It supports revert and verify too.

<comment><USER21> +1 for Sqitch. If you include a full deploy -> verify -> revert -> deploy flow in your automated testing suite you can pretty safely introduce changes that can be rolled back without issue. This has served my team well on several complicated databases over the past two years.

</comment></comment><comment><USER22> We have three applications that access the same database (ugly, we're working on changing that), so we have a fourth component that's solely responsible for the schema.

We use CI/CD for all components, including for the schema. The deployment playbook for the schema basically does a `flyway migrate` after installing the schema / migration files.

We have (again, historical reasons) a home-built schema diff tool for our subset of mysql that we use. For Postgres I'd look into apgdiff. We use the diff tool to generate initial versions of the migrations, potentially modify them by hand, and then `git add` them in the schema project.

If you don't like flyway, you could check out [https://sqitch.org/](https://sqitch.org/) which is fully Open Source.

</comment><comment><USER23> Listen to the SE Radio podcast episode on database evolution[0].

[0] [https://www.se-radio.net/2012/06/episode-186-martin- fowler-a...](https://www.se-radio.net/2012/06/episode-186-martin-fowler-and- pramod-sadalage-on-agile-database-development/)

Also checkout the links from the show notes:

[http://www.databaserefactoring.com/](http://www.databaserefactoring.com/)

[http://www.martinfowler.com/articles/evodb.html](http://www.martinfowler.com/articles/evodb.html)

[http://www.amazon.com/exec/obidos/ASIN/0321293533/agiledba-2...](http://www.amazon.com/exec/obidos/ASIN/0321293533/agiledba-20/103-9235322-6060601)? creative=327641&camp=14573&link_code=as1

[http://www.awprofessional.com/bookstore/product.asp?isbn=032...](http://www.awprofessional.com/bookstore/product.asp?isbn=032150206X&rl=1)

</comment><comment><USER24> Entity Framework in .Net is very easy.

\---

The logic for migrations is suprisingly simple:

\- A new migration will normally create a file with an Up or Down method and a "timestamp" \+ "description of the purpose of the migration" ( eg. 20191031_080603532_add_tags_to_contacts.cs )

\- Every migration, when run on the database will create a new entry in eg. the _migrations table . Some will store the PreviousContext which is a hash of the CurrentScheme before the migrations.

If the application has a live check on startup. It will check the hash of the current db schema, compare it to an entry in the database.

Check if that compares to the latest.

The latest is the latest "migration file with up or down". It will execute all newer once according to the timestamp. Untill the latest one is executed.

A tool also contains a "TargetMigration" for development purpose or for reverting a database back to a previous state.

Every Up or Down migration is run inside a transaction. If it fails, nothing is stored.

There is also a way normally to do a Seed of the database, after the migration is run. But this could be added in the Up or Down method.

PS. All other methods untill now were bad.

</comment><comment><USER25> Django is my favorite backend because its database migration tool is easy to use and can be customized to fit my needs. That being said, if you want database-first schema migration, the only way to do that (that I aware of) is by attempting to generate models for your existing schema using Django's inspectdb command, then run a fake migration.

I like how Wordpress does simple database-first migration: just call `dbDelta()` function with your new table schema in sql format (CREATE TABLE ...) and it'll figure out a way to update your table to fit your new schema. I wonder if there is any similar db migration tool in python that work like this, seem perfect for small projects that don't want to use an ORM.

[https://codex.wordpress.org/Creating_Tables_with_Plugins#Cre...](https://codex.wordpress.org/Creating_Tables_with_Plugins#Creating_or_Updating_the_Table)

</comment><comment><USER26> We're trying out a few things right now.

1) Use a migration tool (flyway, dbmigate, etc.)

2) We try and keep scripts idempotent to minimize on migration script explosion. Also helps to keep changes to similar assets located within the same file: easier to see the progress of a given asset as it changes throughout the lifetime of your app.

3) We have a dedicated repository for each database/schema. A CI/CD process triggers on a push to a given branch (dev, qa, stage, prod, etc.). The CI/CD process runs the migration script (in this case AWS Code Pipeline). Having the schema in its own repository decouples our databases from the service(s) that use them.

4) We try our best to separate schema changes from feature changes: a) push database changes first maintaining backwards compatibility, b) then push feature changes, c) then remove backwards compatibility. So, try to minimize on rollback script.

Local development is same as yours: docker-compose.

</comment><comment><USER27> Database migrations for RDBMS like mysql or postgres, we use knex or laravel's database migrations tools.

For bigquery, we wrote this nifty tool [https://github.com/SplitmediaLabsLimited/supermigration](https://github.com/SplitmediaLabsLimited/supermigration)

</comment><comment><USER28> I've developed a simple migration tool called mgrt [1] to handle database migrations. It takes plain SQL files, and runs them in order, keeping a hash of the contents of the file in the database to ensure that it cannot be modified once run for the first time. I've posted about it here once before [2]. Check it out if you like.

Right now it only supports SQLite, MySQL, and PostgreSQL.

[1] - [https://github.com/andrewpillar/mgrt](https://github.com/andrewpillar/mgrt)

[2] - [https://news.ycombinator.com/item?id=19517001](https://news.ycombinator.com/item?id=19517001)

</comment><comment><USER29> > but both are not completely free and seem proprietary

I don't mean to be "that guy", but unless your whole business's "thing" is no- proprietary-software, please try to pay for commercial licenses of good software. That money pays for bug fixes, security fixes, general maintenance, feature development, testing, support, and a host of other crap you probably don't want to have to do by yourself. If they provide a free tier, you're also supporting the free users.

Flyway is kind of the gold standard, I recommend it.

</comment><comment><USER30> We use flyway for the biggest monolith database in our company. We also use alembic for some microservices.

Alembic is extra convenient if you're already invested in sqlalchemy.

I've tried yoyo-migrations[1] once, a while ago.

With yoyo, like flyway, migrations are written as SQL queries. However, they're expressed as SQL steps in python files. Might worth a look if you're using python.

1\. [https://pypi.org/project/yoyo-migrations/](https://pypi.org/project/yoyo- migrations/)

</comment><comment><USER31> I've found MyBatis Migrations [0], to be very useful. I like the fact that you can do a DB dump of your current database state and set that as the 0th level of migration and then continue from there even if you've never done migrations before.

It's also extremely straightforward and simple as it doesn't try to be part of any ORM.

Highly recommended.

[0]: [http://mybatis.org/migrations/](http://mybatis.org/migrations/)

</comment><comment><USER32> Like half the people commenting here, I wrote my own ([https://bitbucket.org/koalephant/mallard/](https://bitbucket.org/koalephant/mallard/)) in shell, after using another similar tool that's written in python (and contributing some fixes to it).

Same general concept as most others use, apply sql files by date, track what's applied in the db itself, deploy the sql files with your code.

</comment><comment><USER33> I don't have a say in what we do at work, but in all my side projects (which I write in Go), I've used darwin[1], run in main before the server starts up.

1: [https://github.com/GuiaBolso/darwin](https://github.com/GuiaBolso/darwin)

</comment><comment><USER34> I'm currently very much into [http://knexjs.org/](http://knexjs.org/) since it is just a query builder rather than an ORM. It has just enough abstraction to allow it to be DB agnostic.

</comment><comment><USER35> I've used Alembic extensively for Python projects and sequelize somewhat less so for Node projects. It's not a huge sample space, but Alembic's design is much better IMO because it can detect and manage conflicts across merges.

</comment><comment><USER36> FWIW, a product I know uses

[https://github.com/vmware/upgrade- framework](https://github.com/vmware/upgrade-framework)

a little better than just sequential numbering (which breaks down on teams larger than 1).

</comment><comment><USER37> We use native rails DB migration.

</comment><comment><USER38> Having dealt with this before, if I could go back I would integrate an ORM to do the schema changes. Something like SQLAlchemy/Alembic where you can store your schema changes like git commits and roll forward/back as needed.

</comment><comment><USER39> I know it's payware, but shout out to Redgate SQL Compare (SQL Server for sure). I managed schema comparison from dev, staging, deployment for multiple clients - I would have gone nuts without it.

</comment><comment><USER40> [https://djrobstep.com/docs/migra](https://djrobstep.com/docs/migra)

</comment><comment><USER41> In javascript land I use Knex, which has built in migrations and CLI. However it only has basic features compared to ActiveRecord.

</comment><comment><USER42> I have been using flyway for over half a dozen projects and more than a few years. Works very well.

</comment><comment><USER43> If you are using Ruby but don't like ActiveRecord, you can use Sequel, which is really great

</comment><comment><USER44> Also check out MyBatis migrations tool. Similar to raw-sql approach, very simple.

</comment><comment><USER45> Liquibase is open source.

You only get support and templates by paying for it.

</comment><comment><USER46> We use sequelizer on postgres,nodejs

</comment><comment><USER47> I wrote a tool for that in our company (in Java, but could be written in any language) like 10 years ago or so, which is still in heavy use as of today in multiple projects, migrating tens of thousands of (SQLServer and PostgreSQL) databases each year. Its key feature is probably the usage of a schema hash, which seems to be pretty unique among DB migration tools (at least the common open source equivalents to what I wrote don't seem to do this). It works like this:

\- The tool can hash the current schema found in a live database that it's connected to

\- It also knows hashes for all schemas ever created for the application (these are versioned and usually originate largely from an ORM generator, but can also mix generated parts with manually-added ones)

\- In addition to knowing these "full schemas" with their respective versions, it knows forward (and optionally backward) migrations between the versions (these are entirely created by developers). Migrations usually perform DDL stuff, but may optionally also convert data either with pure SQL or procedurally (as migrations are effectively stored as Java code, they can do practically anything)

\- When run against a database, it can either initialize it to any version of the software using one of the full schemas, or migrate it from any known schema version to any other version to which it can find a migration path (pathfinding is a cheap Dijkstra implementation on the migration graph). It automatically detects at which version a schema is by hashing it and comparing the hash with all known schema versions and their hashes (with the ability to fall back to metadata of past migrations also stored in the database in case two versions result in a hash collision). Before and after every migration step, the hash is built from the current state of the database and checked against the stored values for before and after the migration step in question, to ensure the step finds everything as expected and resulted in the correct final state.

\- If a backward migration path can be built, migrating "down" is also possible (but I've never seen this actually used in practice, hence most teams today just skip writing backwards migration steps).

\- The hashes are also used to prevent accidents that might happen if someone runs a migration against a database on which some person manually modified anything schema-relevant. In such case, the hash won't match any version, and the migration will not even start. Whatever manual modification has been done must be undone first in order to guarantee that the migration steps can find the exact state that they were written for and thus yield the expected results (this actually saved our asses multiple times).

\- Finally, a variant of the tool was integrated in the CI process, which means that all migration steps currently checked in are tested against an actual database with each build and can fail the build if not working. Since the current schema can be generated from an object model, the tests also generate this automatically and check its hash against whatever the currently highest versioned migration would produce, failing the test if these are not equal. Effectively this results in developers being immediately notified if someone did change the database schema (usually indirectly via an object model change) but did not check in the necessary migrations to get old versions migrated upwards.</comment>


Twitter's Bootstrap 2.0.1 Released - <AUTHOR> https://github.com/twitter/bootstrap/commit/7ec09d0db9b1117d708682a08efaa55045e6926e

<comment><USER2> I'm glad to see that they're staying on top of this wave of momentum. The worst thing they could possibly do now is stagnate.

If you haven't given Bootstrap a try then I suggest you check out the examples: <http://twitter.github.com/bootstrap/examples.html>

Way to go Mark and Jacob; you're doing a fantastic job!</comment>