Type-safely embed DSLs directly into Java - <AUTHOR> https://github.com/manifold-systems/manifold/tree/master/manifold-core-parent/manifold#embedding-with-fragments-experimental <comment><USER2> Except it is not really embedded in Java, only in comments.

I sincerely hope this dies a quick death.

Java with its jars and infrastructure already makes it plenty convenient to work with resources without them having to be in your face when you are editing the file. Good IDE lets you move between the resource file and the code with a single click.

<comment><USER3> Yeah, I'm not sure I get it either. More power to the author for solving a problem they may have, but I have yet, in my 20+ years experience with Java, had a need to do this kind of code embedding in the source code (whether in comments or not). Sure, it's nice to be able to run other languages at run- time (e.g. GraalVM, etc), but is there really a need to have this kind of language interoperability at compile time?

<comment><AUTHOR> > but is there really a need to have this kind of language interoperability at > compile time?

Well, if you want to leverage Java's static type system (and why not?), the answer is, yes. I imagine you'd want type and member references to the other language to resolve statically using the compiler, right? Similarly, why not have the same functionality in your IDE? Plus code completion, usage searching, refactoring?

Now, as I mentioned in an earlier comment, the embedding part of this addresses just a small segment of use-cases e.g., scoped query editing. The vast majority of other cases work directly against resource files, type- safely. Read more about that here:

[https://github.com/manifold-systems/manifold](https://github.com/manifold- systems/manifold)

<comment><USER3> I'm not at all contesting your reasons, which I expect were plentiful enough to build this system. I'm simply saying that if I want to use Java's static type system, I'll just write in Java. Perhaps I'm an old codger these days in my ripe early 40s.

</comment></comment></comment></comment><comment><USER4> I'm curious how this works under the hood, looks like it's added as an annotation processor, and the DSL is embedded in specially formatted comments, but not in annotations. This implies to me the annotation processor is processing the comments in a source file? I did not know annotation processors could do this! Also implies comments carry through to the compiled .class files which I did not think they did either (runtime bytedcode weaving)?

<comment><AUTHOR> Hi, you can blame me for this. It's a Java compiler plugin[1], which is similar to an annotation processor, but can hook into the compiler at a much earlier stage, which allows it to contribute to all phases of the compiler, including the Parser phase. The Manifold plugin takes full advantage of this, hence its ability to analyze comments, contribute to bytecode generation, etc.

[1] [https://docs.oracle.com/javase/8/docs/jdk/api/javac/tree/com...](https://docs.oracle.com/javase/8/docs/jdk/api/javac/tree/com/sun/source/util/Plugin.html)

<comment><USER5> So you have a javascript frontend hooked into the java compiler? Or how does your javascript integration work?

<comment><AUTHOR> At compile-time manifold-js[1] parses JS and generates Java types (stubs), which forward execution to rhino at runtime. Basically, JS seamlessly mapped onto Java's type system.

[1] [https://github.com/manifold- systems/manifold/tree/master/man...](https://github.com/manifold- systems/manifold/tree/master/manifold-deps-parent/manifold-js)

</comment></comment></comment></comment><comment><USER6> Thank you for making this.

Have you considered an approach based on annotations and multi-line strings? I'm not sure string constants are valid targets for annotations but maybe that could be added to the language?

        @Language(name="Javascript")
        """
            function callBark(aBarker) {
                aBarker.bark();
            }
        """

<comment><AUTHOR> Thank you!

Yes, Manifold already supports Java 15 multi-line strings (aka text blocks) like this:

        var myValue = """
        [>.js<]
          function foo() {
            return "hi";
          }

          foo();
        """;

You can embed a resource fragment as either a declaration or a value. You use a string to embed a value fragment as with the JS example above. Note the [>.js<] header indicates the resource type for the string, similar to your annotation. As you surmised, an expression such as a string literal cannot be annotated.

</comment></comment><comment><USER7> The graphql example is not the greatest, but I would personally use GQL Code Generator to create a little strongy typed client for all your .graphql files:

[https://graphql-code-generator.com/docs/plugins/java](https://graphql-code- generator.com/docs/plugins/java)

Not sure if there's a strong use-case in general here. Even Groovy code, which is a popular choice for JVM DSL languages, has pretty excellent stub generation.

<comment><AUTHOR> But the main idea with this is to avoid the pitfalls of conventional code generation such as GQL Code Generator. See "The Big Picture"[1] in the core docs.

[1] [https://github.com/manifold- systems/manifold/tree/master/man...](https://github.com/manifold- systems/manifold/tree/master/manifold-core-parent/manifold#the-big-picture)

<comment><USER8> When I see "no code on disk" I see that as a downside. What happens when this generated code has a bug? How do I set a break point in this code? How do I see the code to know what it is doing?

Extending the compiler is great... until it isn't. Imagine a feature in java the language not working correctly due to a compiler bug. Debugging or working around this issue could be very difficult. With compiler extensions, you are now widening the opportunity for this type of bug to occur.

More principled code generation systems like Racket's macro system do in fact let you expand the code to see what's generated while avoiding having a separate build step, and even have a macro debugger for interactive debugging. Mainstream languages have a ways to go before we get this kind of tooling for language extension frameworks. Until then, I think I prefer having a separate build step with on-disk, visible, and debug-able code.

<comment><USER9> I definitely agree; the author clearly thought no on-disk produced code was a feature, but it makes me nervous. Similarly, the author then goes on to say how the framework supports circular dependencies between manifolds, or manifolds that each modify one another's types ... and I immediately think that will create debugging nightmares.

<comment><AUTHOR> In most cases, the devs that should be concerned about debugging generated code are the authors of the generator. If there are bugs in released code, consumers of the API should file against them. In any respect, as I mentioned in a previous comment when a resource is selected you can view its generated source via Edit | View Java Source.

</comment></comment></comment></comment></comment><comment><USER10> You can write other languages like this in Groovy using Intellij. Intellij allows you to annotate a multiline string as another language, then it will parse and syntax color, autoformat for you. The use case would be when you are templating frontend code snippets on the server side. I'm not sure if its just Groovy though, I'm betting Intellij can probably do this with other languages that have multiline strings.

<comment><AUTHOR> Right, but that is just syntax highlighting. This is resolving the content of the string type-safely, at compile-time.

</comment></comment><comment><USER11> This is cooool!</comment>


Why Google Offers Will Fail - <AUTHOR> http://crackedplastic.com/2011/08/04/why-google-offers-will-fail/

<comment><USER2> It will fail like Facebook offers because people are being absolutely inundated with these ridiculous deal sites. Someone come up with something original.</comment>


REIA language designer on Twitter/Ruby/Scala discussion - <AUTHOR> http://unlimitednovelty.com/2009/04/twitter-blaming-ruby-for-their-mistakes.html

<comment><USER2> I like Tony Arcieri, but this is a weak and somewhat unseemly argument, easily knocked down:

(1) Most urgently, Alex isn't bashing Ruby. He's at pains to point out that Twitter continues to use and like it. Pointing out the relative strengths of other languages isn't "bashing".

(2) The technical problems Alex has with Ruby are bona fide well known problems with Ruby. Put aside the green threads debacle and the MRI VM is still a dealbreaker. I have lots of long-lived EventMachine code, and MRI simply isn't up to the task. Arcieri even stipulates to this in his post.

(3) In deriding Kestrel, Arcieri is ignoring several of key facts that Twitter had to face when they built it: first, that they were already down the path of a house-built message queue (they didn't go to Scala to build Starling, which moots his critique of it); second, that apart from Apache, none of the competing projects were mature enough for Twitter to commit to at the moment in time we're bickering about; and third, that his preferred queue (RabbitMQ) would have required them to commit to yet a third exotic platform, Erlang, the Ron Paul of programming environments, and would have won them no more meaningful performance than they got with Scala, but would have robbed them of JVM compatibility.

It's the third point that rankles me the most. In a 24 graf jeremiad, we find only 10 grafs in that Arcieri can't make a performance-based critique of anything but a straw man (Starling), and can only do it by himself rejecting Ruby in favor of Erlang. This is the best the Ruby community can do to answer Alex's argument?

<comment><USER3> (1) I think the rub isn't so much that they're bashing Ruby as it seems like rather than evaluating all their options, they just jumped ship to some random unproven technology for whatever reason. They then turn around and say "well we switched because Ruby didn't do what needed" (but check out my book on the language that does!!). It makes one think they don't know if it did or didn't, since JRuby would've solved a goodly number of their problems (probably; we'll never really know). Just like when they wrote Starling, they seemed to have just decided to hack something out rather than make a reasoned technical decision.

(2) Yes, but JRuby and Ruby 1.9 both handle those problems much better. Moot argument.

(3) So rather than picking an "unproven" technology (by some measure of unproven), they write something completely new in a comparatively immature programming language. Yup. Much better choice.

And since when did JVM compatibility matter? The reason they chose the JVM was good threading and so on, but Erlang has that same support. It wouldn't be introducing a third "exotic" platform, but a second, different platform.

Of course, I'm saying this stuff from an outside perspective. I'm hoping he elaborates on his blog, because I would really be interested to hear a more technical explanation of their decisions.

<comment><USER2> I really think you're going to lose this argument. You're defending a blog post that says that Kestrel was a far worse choice than a single-developer C project with no major success stories. I don't think, and I don't believe that you think, that Scala is as likely to be a failure mode for Twitter as MQ is.

However immature Scala is --- and I'm not using it --- the Scala runtime is absolutely rock solid. I'm sure that's true of JRuby as well, but the comparison isn't between JRuby and Scala, it's between MRI and Scala, and for a company that tolerated high-volume messaging servers in MRI, we both know Scala is going to be like shangri-la by comparison.

<comment><USER3> I'm not defending the post, but countering your points. If my points happen to line up with his opinions, then that's merely coincidence. There are a _lot_ of MQ options out there, many of which I've used with great success (with Ruby no less). To argue that one of those is less acceptable than a home grown solution in Scala is, at best, dubious.

But you just danced around the real question: why _not_ JRuby? Scala's runtime == JRuby's runtime. They're both JVM languages, and if they were to use JRuby, there wouldn't be some big crazy rewrite. The only difference would've been "jruby mq.rb" rather than "ruby mq.rb." That's the decision that hasn't really been explained.

Even further, I'm not sure why you're insinuating that I think they should write a message queue in MRI. Either use JRuby or use something else. I totally agree MRI is not acceptable for something like this (but 1.9 may be; I haven't tried it but its performance is only a hair slower than JRuby), but there are other ways to solve the same problem that don't involve rewriting tons of code (either by using JRuby or by using a proven, solid drop in replacement, possibly with an API shim if they really needed it).

<comment><USER4> JRuby is not a 1:1 mapping of a Ruby to Java bytecode -- there's significant additional book-keeping that must be done by JRuby's runtime (for example, maintaining the Ruby call frames).

In contrast, Scala maps to the JVM as closely as possible. Scala classes are Java classes -- Scala and Java are bidirectionally interoperable, and Scala's performance subsequently benefits.

</comment></comment></comment></comment></comment><comment><USER5> _As I perhaps somewhat self-aggrandizingly consider myself one of the most knowledgable people regarding I/O in the Ruby world, I decided to peek around the Starling source and see what I discovered. What I found was a half-assed and pathetically underperforming reinvention of EventMachine, an event-based networking framework for Ruby which is the Ruby answer to the Twisted framework from Python._

The non-evented Starling was multi-threaded and used a thread pool to manage connections. I'm unclear on how that is a 'half-assed implementation of EventMachine' and not simply a multi-threaded network daemon.

<comment><USER2> EventMachine supports an actor-style threading model, which may be what Tony was comparing it to; pretty clearly, Tony Arcieri knows EventMachine and async programming --- he's one of the better known people in the EventMachine "community".

<comment><USER5> Does that make every multi-threaded network daemon that manages a thread pool of connections a 'half-assed implementation of EventMachine' then?

(I know who Tony is. I co-wrote Evented Starling.)

<comment><USER2> Ok. It wasn't clear from your comment whether you were saying Arcieri didn't know what he was talking about. Sorry.

</comment></comment></comment></comment><comment><USER6> Message queues are something the financial industry has been getting right for years (even down to custom hardware-based implementations), so the author brings a good point that it's stupid to reinvent yourself if you don't have an experience.

</comment><comment><USER7> His analysis of starling reinforces an impression I got of Twitter devs from articles/discussions back when they were having lots of uptime/scaling issues. The impression was they weren't that experienced or all that good.

It makes me curious how many early startups aren't composed of rockstar devs. How much (if at all) timing, luck, marketing matter more than dev ability at the beginning.

I tending to think it's not nearly as important to have experienced rockstars from day 1. It's not until you get enough success to become famous and start attracting experienced rockstars that it becomes critical to recognize and hire them.

</comment><comment><USER8> While complain Ruby, why not make some language level contribution? Or is it very cheap for Twitter to rewrite the whole stack? I am wondering...

<comment><USER9> [Note: I can't read the original link -- the site is down -- so I have no idea what the original submitter said. But let me take a guess about what you're saying.]

Try to put yourself in Twitter's shoes. Your viral app is a fantastic, unprecedented success. Your traffic is doubling every week. The Fail Whale is onscreen so much that it has its own name, its own fan club, and its own T-shirts. Techcrunch is rumbling about all the other entrepreneurs who are setting up to clone your service.

The idea that a language-level change to Ruby is a wise thing to pursue at this point is insane. Ruby has a big and complicated code base. You are not a language designer. You probably won't even figure out what you could do that would help. If you do, the change will probably result in an internal-only fork of Ruby that can't be reliably patched and that is incompatible with a random cross-section of your third-party libraries. Deploy that thing and you will be finding and fixing Heisenbugs all over the codebase for the next six months.

To actually get an official change into _Rails_ takes months, minimum. In the case of Ruby that might stretch into a year or two. Because you must first win a series of online arguments, and then you must wait for lots and lots of third parties to test your change against their apps and libraries and report or fix the bugs.

Yep, much cheaper to just rewrite your whole stack using different infrastructure. Several times, if necessary, as experiments. Twitter is expert at rebuilding their own stack -- what has been done before is easier to do again.

<comment><USER8> > I can't read the original link -- the site is down

Well, it is up. You might need some http proxy to read the article

> To actually get an official change into Rails takes months, minimum. In the > case of Ruby that might stretch into a year or two.

Twitter can obviously fork Ruby code base and maintain their own branch if they like.

</comment></comment></comment><comment><USER10> Great point about Twitter never explaining why it didn't just use one of the many awesome open source message queues already in existence.

<comment><USER11> He suggests the following message queues:

<http://www.rabbitmq.com/> \- first version 8th February 2007, but the first version not to have "alpha" or "beta" status was 1.5.1 released 21st January 2009

<http://memcachedb.org/memcacheq/> \- version 0.1.1 released 26th November 2008

<http://www.ejabberd.im/> \- not really a message queue

<http://xph.us/software/beanstalkd/> \- first public release 11th December 2007, hit 1.0 28th May 2008

<http://activemq.apache.org/> \- not sure when it was first released but the mailing list goes back to December 2005

The first public release of Starling (Twitter's first custom message queue) was 10th January 2008. Presumably they had it running internally for a while before they released it.

From this, we can see that when they built their own pretty much the only realistic open source option was ActiveMQ, which can hardly be described as a light-weight solution (not to mention it still doesn't have a stellar reputation under high loads). When the alternatives aren't rock solid yet, rolling your own (where at least you understand all of the code and how it works) seems like a perfectly practical alternative.

<comment><USER12> While it is possible that Starling had been running internally before its release, this does not excuse overlooking rabbitmq. A software package that had proven itself in real-world scaling and been designed by people with real experience in the problem domain (c.f. the financial services world) is going to be much better at "alpha" or "beta" quality than Starling is going to be even after the twitter devs hammer at it for a couple of years. The twitter devs were starting from scratch, writing something that other people out there actually had some experience with, and decided to not take an existing solution and fix/adapt it to their needs.

<comment><USER2> I take issue with the idea that the financial services world has real experience in Twitter's problem domain. My experience with the financial services world is significant technically, but casual in a career sense. That said:

I think hi-fi devs make lots of stupid decisions in the name of performance. In the few cases where their actual outcomes match up to their posturing, it's because their code is obsessively cobbled around one specific use case they've been working on since 1989.

Have you ever _read_ an order management system, or looked at Tibco Rendezvous on the wire?

Most of the hi-fi companies adopting MQ are built around straight AMQ, and bare-metal performance was out the window long before they bolted their crappy WebSphere app onto it. What these companies are looking for is predictability, not performance, and their problem sets are much simpler and most stable than Twitter's.

</comment></comment></comment></comment><comment><USER13> I'm getting a 404 message that is trying to sell me novelty gifts.</comment>


The Real Value of $100 in Each US State - <AUTHOR> http://taxfoundation.org/blog/real-value-100-each-state-2016 <comment><USER2> One other consequence of this is that a cleaning lady in San Francisco may make $70K per year and be near the poverty line, yet be in the same tax bracket as a professional living in Nebraska who is living well above the poverty line.

It would be interesting to see an illustration of how regressive Federal tax brackets are when you consider the state-by-state cost of living differences. Some of the most populous states have the highest cost of living.

<comment><USER3> My first thought was "why doesn't the cleaning lady move?" And that lead to a thought experiment: what if everyone moved to Nebraska for a more comfortable life. Prices would go up. The people would experience a Nebraska winter and be willing to have less 'real' money and move back to California. And that's right where we are in the real world. You can't balance just part of life (taxes) when there are other imbalances (benefits of place) that can't be balanced. If costs/life style were comparable in cost between CA and NE, there would be a massive influx into CA. Economics is awesome - it explains so much!

<comment><USER4> The benefits of California are vastly overstated.

Almost nothing about the climate (I _like_ winter) or culture appeals to me, yet I'm pulled towards there to succeed in a tech career. This is simply network effects in action, except that unfortunately this network has been built in one of the locales in the US with some of the most aggressively anti- growth policies possible. Things would be much better if tech was centered around somewhere like Chicago or even New York.

<comment><USER5> Thought experiment: Would developer salaries in general be as high as they are today if we hadn't happened to choose an area with such high cost of living area as our nexus, with the consequential requirement to pay so much to attract talent?

That is, would a guy like me (or you) be able to pull in multi-six-figures a year to telecommute to the Bay Area from out in the sticks if there weren't so many poor souls physically there keeping rates up for the rest of us?

What would things look like if the tech hub were in Ft. Worth TX instead (or Scott's Bluff, Nebraska), with hundreds of miles of flat empty cheap land in all directions to keep housing prices down near zero.

<comment><USER6> The cost of living follows salaries of the tech sector, not the other way around.

Even the tech hub was not on a peninsula, you'd still have an expensive center and poor people would have to suffer a long commute like East Bay people do in the Bay Area.

Just look at London or Paris: it's not cheap despite not having the same geographical problem. It's not as expensive as the Bay Area because there are many people with various jobs, not just highly paid tech workers.

<comment><USER7> It's a combination of demand and city-policies. San Francisco has a lower population density than queens, but the same housing price levels as Manhattan. If the city got its act together and built up a housing density that's on par with Manhattan, cost of living would fall dramatically.

Good governance matters. It makes a huge difference in cost-of-living. The way SF is run, it's almost as though people _want_ the cost-of-living to be as high as possible.

<comment><USER8> > The way SF is run, it's almost as though people want the cost-of-living to > be as high as possible.

Surely you're joking? Of course that's what they want. SF people, home owners, are voting on SF policy. Do you want to lose half the value of your house?

<comment><USER9> > SF people, home owners, are voting on SF policy. Do you want to lose half > the value of your house?

Its more complex than that, as SF people voting on policy also include renters, and even if rent control insulates them from directly paying for price _increases_ , market price declines save them money (and both increases and declines affect local costs of other goods, so they affect CoL even if they don't directly affect existing renters rent payments.) But, renters are largely people who have decided the trade-off of quality for cost at the current price _does_ make sense, so they are less likely than outsiders who might _want_ to move to SF if it was more affordable to make trade-offs that impact the qualities that they prefer in SF for lower prices, even if it would reduce their cost of living.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> There is a vast difference in the value of $100 within states too. For example, western New York state (eg, Buffalo area) is far cheaper than NYC in terms of median home prices, supermarket prices, etc. Similarly, Richmond VA is far cheaper than the Northern Virginia / DC suburbs area.

<comment><USER11> Tax Foundation here. This is a great point and we have a county-by-county breakdown in the works. We will publish that next week, so stay tuned!

<comment><USER12> You might also consider weighting the state average by population. Populations tend to cluster in cities so I think you'll see the "experienced" cost go higher.

<comment><USER13> Nope. Houston housing costs are less than California cities. Indeed, the median home cost in the whole state of Texas is 1/3 of California.

</comment></comment></comment></comment><comment><USER14> The crazy thing isn't the real value of $100 spent on private goods and services. It's the value of $100 spent in taxes.

Federal tax receipts in California, Illinois and New York subsidize Florida, Texas and much of the south.[0] The hypocrisy of fiscal conservatism is that it is enabled by giant high-tax Democratic states.

[0] [https://wallethub.com/edu/states-most-least-dependent-on- the...](https://wallethub.com/edu/states-most-least-dependent-on-the-federal- government/2700/)

<comment><USER15> Other comments have good points, but the main thing IMO is that tax receipts are low in the midwest / south is that what they produce (food) is incredibly cheap now days.

The US produces a ridiculous surplus of food with a tiny portion of the population because we're incredibly good at doing it cheaply. High volume and low cost == low prices == low tax revenue. It doesn't mean low real-world value or produced value.

You could "fix" it by trying to balance revenues by playing with tax rates, but all you'll accomplish is making food expensive in CA/NY, shifting dollars around, and making everything more complicated than it is now.

There are plenty of other reasons tax revenue registers in NY/CA; anyone -- from anywhere in the country -- with a retirement account is funneling it through some NY mutual fund, and entertainment is still mostly produced in CA. If NY didn't exist, it would be managed somewhere else; it's not that NY is producing magical value nobody else understands. It's just most convenient.

If Hollywood stopped producing movies... we'd all be a bit more bored temporarily, but it's again going to get bought from somewhere else.

<comment><USER16> You realise food is cheap because of taxes/subsidies. If we had to pay the true cost of food it would be expensive. And the politician that did wouldn't be liked much as well.

<comment><USER17> Can you show the calculation where subsidies to food are greater than the cost of regulation of food?

Sugar is particularly regulated, with the government strict quotas and tariffs on imports, and domestic allotments to produces. If you produce more than your allotment, it is illegal to sell in the US. The whole scheme is to hold prices high to support famers (agribusiness really), not low.

<comment><USER16> Sugar is regulated and the price is kept high for farmers. But not for the consumers which is my point. Farmers are given subsidies in all crops so they can sell them in the global market at below market rates, not only for the population, but to compete with out countries doing the same thing.

</comment></comment></comment></comment></comment><comment><USER18> I see that the tax foundation says Georgia is $108, but there is a huge caveat. I've lived in Georgia the past 8 years. I've lived in both middle of no where and currently live in Atlanta. The value of $100 is vastly different between those two areas. Georgia is mostly rural, except for Atlanta. My old apartment, 1 bedroom a bath, I can still rent for $575 a month. Lakeside view with a pool, 10 minutes from work. Anyone reading this, most likely will be moving to Atlanta. The COLA is still cheap, but understand the caveat.

Meanwhile, I grew up in Florida, where it says $100 is pretty much $100. There are lots of major cities in Florida and cost of living is more distributed.

</comment><comment><USER19> I'd be more interested in the real value of $100 of _income_ in each state, which would account for differences in state income tax. On such a map, WA would look significantly better, and states such as CA and NY would look even worse.

</comment><comment><USER20> Nice! I would be interested in a more granular version of this to be able to compare different cities.

<comment><USER21> Then you're in luck, because the article links to the original data, which includes this. See the links in the right hand side bar.

[http://www.bea.gov/newsreleases/regional/rpp/rpp_newsrelease...](http://www.bea.gov/newsreleases/regional/rpp/rpp_newsrelease.htm)

</comment></comment><comment><USER22> I'm not suggesting a cause/effect relationship here, but there is an interesting rough correlation between the map in the article and the red/blue political map...

[https://en.wikipedia.org/wiki/Red_states_and_blue_states#/me...](https://en.wikipedia.org/wiki/Red_states_and_blue_states#/media/File:Red_state,_blue_state.svg)

<comment><USER23> Funny I saw a definite "you get what you pay for" correlation - with the "highest dollar value" states tending to align with the states with the lowest measures in health, education and other quality of life indicators. [https://en.wikipedia.org/wiki/List_of_U.S._states_by_educati...](https://en.wikipedia.org/wiki/List_of_U.S._states_by_educational_attainment) [https://en.wikipedia.org/wiki/List_of_U.S._states_by_life_ex...](https://en.wikipedia.org/wiki/List_of_U.S._states_by_life_expectancy)

<comment><USER24> > Commiefornia: 80% High School Graduation Rate

Correlation not found

</comment></comment></comment><comment><USER25> Can someone please explain why NJ is such a fucked state? Is it just because of the population density, is it the years of government corruption? I just don't get how cost of living here is so nuts and how our economy feels so..."poor"?

</comment><comment><USER26> It would be nice to compare cities to countries. For example, I think Nevada vs Las Vegas would be radically different.

<comment><USER27> I was thinking the same thing and also on the cities to state level.

My though is that dense urban cities (e.g. New York, LA, San Francisco) are going to have a lower value due to the high cost of real-estate and also rural areas where a lack of population density leads to distribution inefficiencies in material goods raising their cost. States like Ohio and Georgia with large suburban areas have a balance of cheaper real-estate and high enough population density lower the distribution costs of material goods.

</comment></comment><comment><USER28> Does anybody know of a comparable study for Euro countries?

<comment><USER29> Not exactly the same but you can find comparative prices for 100€ in EU here: [http://ec.europa.eu/eurostat/statistics- explained/index.php/...](http://ec.europa.eu/eurostat/statistics- explained/index.php/Comparative_price_levels_of_consumer_goods_and_services)

</comment></comment><comment><USER30> Shouldn't one of the states act like a baseline? I couldn't find anywhere, where $100 was worth $100. That seems kinda odd to me.

<comment><USER31> $100 is worth $100 on average

<comment><USER32> And the average person or state doesn't exist.

</comment></comment></comment><comment><USER33> If I squint just a little this looks very similar to this: [https://en.wikipedia.org/wiki/Political_party_strength_in_U....](https://en.wikipedia.org/wiki/Political_party_strength_in_U.S._states#/media/File:ElectoralCollege2012.svg)

</comment><comment><USER13> This is a helpful graphic, but I think it's incomplete -- and substantially understates CA's COL. That's because it measures what AFTER TAX dollars buy (as I understand it). Hence income tax free states have an even bigger "bang for the buck" advantage. I prefer to use the Missouri COL comparison of states: [https://www.missourieconomy.org/indicators/cost_of_living/](https://www.missourieconomy.org/indicators/cost_of_living/)

</comment><comment><USER34> Oh look, all the places I will never move to.

Oregon is no longer the bargain it once was. Though I am sure I could move to a cheaper area.

Living internationally is where its at. Numbeo.com gets down to the details (thought their interface is due for an overhaul). [http://www.numbeo.com/cost- of-living/compare_cities.jsp?coun...](http://www.numbeo.com/cost-of- living/compare_cities.jsp?country1=Spain&country2=United+States&city1=Malaga&city2=Portland%2C+OR)

</comment><comment><USER35> Note that DC is the most in the low direction, at ~$84.

<comment><USER36> Cities in general have higher costs, so it's not too surprising that the only entry in the data that is just city is going to have the highest costs.

</comment></comment><comment><USER37> I wonder what this would look like applied specifically to housing. I suspect that people in expensive areas are paying significantly more (both absolute and as a percentage) on housing. In some ways, this is just a side effect of that phenomenon---housing prices eventually trickle down to everything else.

<comment><USER38> Relatively speaking. I live in the mountains of North Carolina and the housing prices are double that of Charlotte. Everything is custom build, there aren't prefab houses. Labor is relatively the same, but food and gas cost's more. You can't really go by state. You'd have to go by county with multiple variables to get a real picture.

</comment></comment><comment><USER39> Like others note, using states as your unit of granularity here because nullifies any useful conclusions you could take from this.

This is basically just a map of "how many dense cities does each state have?" and tells you nothing more interesting than that.

A per-county map would be much better.

</comment><comment><USER40> The correlation between their map and the 2012 electoral college map is striking! The light-colored (expensive) states are the blue states and the dark-colored (cheap) states are the red states, more or less.

</comment><comment><USER41> They need to break the states up too. Southern California $100 is a lot different than a Northern California $100. $100 living along the coast is a lot different than $100 living inland.

<comment><USER42> SoCal and NorCal aren't as different as you'd think - having lived in both places and as one who spends a lot of time between them, I doubt there'd be much noticeable at all.

Now Central Valley life can be a lot cheaper than coastal life - no doubt about that. I have a vacation home in he Central Valley and _everything_ we do here and for the home is cheaper. Painting, repair work, labor, gas, auto and boat repairs, everything.

<comment><USER41> I have lived in both and along the coast and there is a difference between North and South.

</comment></comment></comment><comment><USER43> Now I'm curious about Puerto Rico and other territories. Anyone know the basis of the research?

<comment><USER34> numbeo.com is quite good: [http://www.numbeo.com/cost-of- living/country_result.jsp?coun...](http://www.numbeo.com/cost-of- living/country_result.jsp?country=Puerto+Rico)

</comment></comment><comment><USER44> Know your source. The Tax Foundation:

[http://www.sourcewatch.org/index.php/Tax_Foundation](http://www.sourcewatch.org/index.php/Tax_Foundation)

<comment><USER45> What are you implying? The article is basically a graphical representation of some math done on publicly available figures. If the figures or math are wrong, or if the graphical representations are disingenuous, explain why. But to just say "yeah, well it's The Tax Foundation so whatever" is a cop out to avoid actually discussing the point the article attempts to make.

<comment><USER44> I'm implying that knowing the source of a study or policy paper is useful context.

I'm not making further judgement.

I am highlighting SourceWatch (and similar resources) as useful and pointing out their existence.

</comment></comment></comment><comment><USER46> so the places with low population densities tend to have lower cost of living. Not really breaking news.

</comment><comment><USER47> How can most states be above average?

<comment><USER48> I'm in no means an expert in this, but I think it's because of the relative sizes of the economies in each state [1]. California and New York are very large in terms of GDP/GSP and will therefore move this weighted average accordingly.

[1] [https://en.wikipedia.org/wiki/List_of_U.S._states_by_GDP](https://en.wikipedia.org/wiki/List_of_U.S._states_by_GDP)

</comment></comment><comment><USER49> How does this affect online pricing? Assuming something bought online has the same price for all shipping locations, this would mean that people in expensive states are more likely to save money by shopping online?

<comment><USER19> Much of the difference is in land/home values, but, yes, it would mean that people in states like Hawaii would save more by shopping online. (This of course assumes that the online store doesn't pass along the increased transportation cost in the form of higher shipping.)

<comment><USER50> Alaska and Hawaii are pretty much outliers. Essentially everything is expensive whether it's at a local store or because of shipping surcharges. (Although Amazon Prime still covers both states--which I imagine is quite a big win for residents of those states compared to historically.)

</comment></comment></comment><comment><USER51> This is kind of like saying why should you buy a $200 mechanical keyboard when you can get a perfectly serviceable model for $15?

Its not even false.

</comment><comment><USER45> > _In other words, by this measure, if you have $50,000 in after-tax income in > Mississippi, you would have to have after-tax earnings of $68,000 in the > District of Columbia just to afford the same overall standard of living._

I imagine it is much easier to earn $68k after taxes in DC than it is to earn $50k after taxes in Mississippi.

<comment><USER52> And that is why people move to cities (yes, I know that MS has cities). What point are you trying to make?

<comment><USER45> Assuming the $68k in DC / $50k in MS is the benchmark, my point was that while it's much cheaper to reach that benchmark in MS, you're probably much more likely to reach it in DC. So a random person in DC is probably more likely to earn $68k or more than a random person in MS is to earn $50k or more.</comment></comment></comment>


The size of the shadow banking system estimated according to power law - <AUTHOR> https://medium.com/the-physics-arxiv-blog/5e1dd9d1642 <comment><USER2> The approach seems to be along the following lines:

1\. Fit a model (in the case Zipf's law) to a semi-related field (the size of non-financial companies).

2\. Find that the model doesn't fit reality in the field you're interested in (financial companies).

3\. Posit that your model is correct and reality is wrong, and invent over $30 trillion worth of invisible activity to account for the difference.

Alternatively... your model is just wrong?

<comment><USER3> This is an obvious and accurate criticism. But it's also worth considering that large financial firms are both the best equiped and most motivated to obscure their transaction activity, so I don't think it should be dismissed trivially.

The question is: how could one find evidence that justifies or refutes the idea that the scale of firms does in fact fit zipf and that measure is being obscured at the tail?

Also, I'd point out that the original post did not differentiate between financial and non financial companies in the way you do between points 1 and 2, so your characterization of what fields are related or "semi-related" is your own creation and not part of the original claim.

<comment><USER4> > The question is: how could one find evidence that justifies or refutes the > idea that the scale of firms does in fact fit zipf and that measure is being > obscured at the tail?

Well, I think to find that evidence, you would have to take a measurement of the amount of _shadow banking_ from one of these firms. Objectively, you can't do that, since there is no agreed upon definition of what the term shadow banking means.

</comment></comment></comment><comment><USER5> This is off topic, but does anyone else have a "This Week's Top 5 Posts" section smack in the middle of the article?

Is that something new Medium are automatically injecting? If so, I don't like it. I do like the idea of being pushed to more content _after_ I've finished reading, not _during_ the middle.

<comment><USER6> In my increasingly reflexive editing of website CSS:

\- I disable _ALL_ nags. If there's a modal, slider, fixed bar, or fixed social element, I nuke it.

\- If it moves, I nuke it.

\- If anything is labled "viral", "tease", "nag", or "social" there's really good odds it's getting nuked. Really, stuff like that just makes it too easy, and I may add some wildcards to my default CSS.

\- Unless it's a premier site (e.g., NY Times, the Guardian), I'll nuke the "Recommended", "Trending", "Most Viral", etc., columns.

\- I'll strip any 3-column (or otherwise multi-column) format into a single column of body text, generally about 40-50 ems (about 850-1000px) wide. The LH column generally becomes a full-width header, the RH column a full-width footer. If they survive at all.

\- Bump all fonts to 15pt (body text), 14pt (usually for overview pages), 10-12pt for most navigation elements.

\- Backgrounds to to white or very close to it, text to black or very close. #fffff4 background, #222 foreground, is about as much as I'll push either.

I've noticed that the result is _much_ more soothing reading experiences, with far fewer distractions. It literally affects my ability to focus and peace of mind. Had my stylesheet manager (Stylebot) blow up on me a week or so back, having to deal with The Web As Published ... just drove me nuts. Most site's design simply sucks.

I've got over 500 sites styled to my preferences (many are just very quick and light changes, a few more involved). The results are ... mostly pretty nice.

Among my favorites:

Buzzfeed: [https://plus.google.com/photos/104092656004159577193/albums/...](https://plus.google.com/photos/104092656004159577193/albums/posts/5890225669560306434?pid=5890225669560306434&oid=104092656004159577193)

Unbuzzed: [https://plus.google.com/photos/104092656004159577193/albums/...](https://plus.google.com/photos/104092656004159577193/albums/posts/5890225669560306434?pid=5890225669560306434&oid=104092656004159577193)

A 3-column to 1-column design:

3 cols: [https://plus.google.com/photos/104092656004159577193/albums/...](https://plus.google.com/photos/104092656004159577193/albums/5896495055961929169/5896495071944854994?pid=5896495071944854994&oid=104092656004159577193)

1 col: [https://plus.google.com/photos/104092656004159577193/albums/...](https://plus.google.com/photos/104092656004159577193/albums/5896495055961929169/5896495062245958898?pid=5896495062245958898&oid=104092656004159577193)

And this seems to kill the "Top 5 posts" bits:

blockquote.pullquote, blockquote.pullquote + p, blockquote.pullquote + p + p, blockquote.pullquote + p + p + p, blockquote.pullquote + p + p + p + p, blockquote.pullquote + p + p + p + p + p, blockquote.pullquote + p + p + p + p + p + hr { display: none; }

<comment><USER7> Have you considered maintaining a [paid] chrome add-in with all this stuff in?

<comment><USER6> Hrm.

Some of my edits are pretty rough. Most aim to please one key customer: me. And I do virtually no compatibility testing for devices, browsers, or even display settings.

Still ... could be useful. If only because exporting and uploading the stylesheets would give me a backup.

What would you pay for this?

<comment><USER7> It would have to be an ongoing service, a sort of "Clean Web Subscription" that got updates. Believing that there's a human curating it is what makes it valuable I think. I'd pay an annual fee of $30 for that. I may be an outlier though.

</comment></comment></comment></comment></comment><comment><USER8> One thing about Power Law distributions -- they work really great except at the outliers (ie, top 10). You can't always say that because incumbent #10 is x, #1 is 10x, especially when dealing with organizations that are huge conglomerates of different services, branches and sub-companies.

Look at Market Caps of companies (something that we have public signals about): [https://en.wikipedia.org/wiki/List_of_corporations_by_market...](https://en.wikipedia.org/wiki/List_of_corporations_by_market_capitalization#2013)

The difference between #1 and #10 is <2x, and #3-#10 all vary by just a few percent. Power Law distributions are great, but they tend to fail at the top end.

<comment><USER8> I just wanted to present a counter argument to my reasoning: What if the list of companies (which is publicly traded companies) is missing the real head -- the non public companies that are either government owned or are governments themselves? If we look at the list of companies by revenue: [https://en.wikipedia.org/wiki/List_of_largest_companies_by_r...](https://en.wikipedia.org/wiki/List_of_largest_companies_by_revenue) it does look like the real head of the Power Law distribution is missing.

<comment><USER9> The original paper seems to take the Global 2000 and then re-orders them by asset size, not revenue:

"we find that financial firms dominate the top tail of the firm distribution by asset size"

[http://arxiv.org/pdf/1309.2130v2.pdf](http://arxiv.org/pdf/1309.2130v2.pdf)

</comment></comment></comment><comment><USER10> The study takes on-balancesheet assets as a proxy for company size and fits a model to it that largely works. Fine so far.

But the implication that this works at the top end is either that the companies are hugely mis-stating their on-balancesheet assets or that there are other companies, much bigger than FNMA, that have balancesheets with multiples of the GDP of the States.

Neither of these explanations are plausible even if you believe that they're all lying, thieving scumbags. The numbers are just too big.

However, there is another explanation. That on-balancesheet assets isn't a good proxy for size for big financial institutions. And all of a sudden the problem is solved because on-balancesheet assets isn't a good proxy for size for big financial institutions.

Apart from anything else many of them run huge assets and liabilities off- balancesheet. BTW that doesn't make them secret, they hit the accounts just the same, it just means that the proxy in the study won't work.

And for what it's worth, this is wholly orthogonal to the size of Shadow Banking, how much stuff is hidden in the Caymans etc etc. Fascinating and worrying subjects in themselves.

</comment><comment><USER11> I've got a fun mathematical model supporting controversial assertions about banks. Yes, I could test those assertions against fact by reading the contingent liabilities sections of large banks' financial reports. But accounting is boring. And, let's face it, we all know that I'd just deal with negating evidence by arguing "accounting trickery" is just hiding Truth.

</comment><comment><USER12> _" The shadow banking system is vastly bigger than regulators had thought, say econophysicists who have developed a powerful new way to measure its hidden impact"_

I had to read this a couple of times before I realised what was wrong, but apparently "Econophysics" really is a thing:

 _" Econophysics is an interdisciplinary research field, applying theories and methods originally developed by physicists in order to solve problems in economics, usually those including uncertainty or stochastic processes and nonlinear dynamics. Its application to the study of financial markets has also been termed statistical finance referring to its roots in statistical physics."_ [1]

[1] [http://en.wikipedia.org/wiki/Econophysics](http://en.wikipedia.org/wiki/Econophysics)

<comment><USER13> George Soros knows a lot about financial systems. Strong evidence is the fact that he's rich and shares his perspectives, which are better than mainstream perspectives at interpreting market behavior particularly during major crises. My favourite quote of his is the oft-repeated notion that _Classical economics is based on a false analogy with Newtonian physics_. He generally spends some time either side of that comment making it really clear: there is no mystical market equilibrium of supply and demand, only the behavior of individuals.

</comment></comment><comment><USER14> "On the Forbes Global 2000 list of the world's largest companies, the first non-financial firm is General Electric, which ranks 44th."

This is even scarier than the apparent fact that econophysics is a real thing. I'd like to know the history of the Forbes Global 2000 to get an idea over time of how dominant financial companies have been. I have a hunch they have always had a part (JP Morgan Chase anyone?) but surely this is a new phenomenon to be so dominated by companies that don't actually produce much.

<comment><USER15> If something seems scary/spectacular/surprising it's usually a good idea to do some fact checking. In this case it's very easy, since the authoritative source (Forbes) is available online.

GE is 4th, not 44th. It seems to be a typo.

[http://www.forbes.com/global2000/](http://www.forbes.com/global2000/)

<comment><USER16> If you sort by Assets it is 44th alright.

<comment><USER17> sorting by assets is a weird way to look at things. Given that banks manage people's money, you would expect them to dominate in assets.

market value, profits, number of employees all sound like better metrics to get the "size" of a company. if by "size" you mean the influence it has over the world.

<comment><USER18> Err...

Banks manage other people's money, meaning they ought to lead in _liabilities_.

Bank assets primarily are the loans they extend to customers, such as mortgages and car loans -- in theory anyway, since they paper it nowadays to avoid carrying part or all of the risks. They extend these loans by leveraging the money in your CDs and savings accounts. In a sense, they borrow against the money they owe you.

<comment><USER19> They lead in both, because after all to be solvent their assets have to be at least as large as their liabilities.

</comment></comment></comment></comment></comment></comment><comment><USER20> See the "Inside Job" (2010) documentary movie - (at least in USA) there were attempts to regulate the now unregulated banking activities. The banks' lobby was just too powerful and in result not only the banks remained unregulated, but the regulation system was reduced in staff and weakened.

<comment><USER21> Or, people could read about how and what the shadow banking system is instead of watching a popular, slanted - albeit good, documentary. The "shadow banking system" aren't banks, that's the point.

People underestimate just how regulated the banking system is, despite all the talk about more regulation.

<comment><USER20> "People underestimate just how regulated the banking system is, despite all the talk about more regulation."

I agree with you about underestimation, but I think that the surplus of the regulations that we're talking about here is more to keep the member circle small, therefore it is in the benefit of the incumbent banks. So there is: 1. strong regulation if you want to get in and 2. weak regulation (at least for some banking activities) for keeping you in check afterwards.

</comment></comment></comment><comment><USER22> What happened to labeled axes? Is it not cool anymore to tell people what BOTH axes are supposed to be?

</comment><comment><USER23> It may be that the scale-free preferential attachment model doesn't hold in the generality claimed. The modeling technique of Doyle and Willinger may be more relevant [1].

1\. Walter Willinger , David Alderson , John C. Doyle. Mathematics and the Internet: A Source of Enormous Confusion and Great Potential. Notices of the AMS. Volume 56, Number 5. May 2009. [http://www.ams.org/notices/200905/rtx090500586p.pdf](http://www.ams.org/notices/200905/rtx090500586p.pdf)

</comment><comment><USER24> Banks are generally regulated, they have to have certain amount of assets to work. They cannot have the same distribution as normal companies do.

</comment><comment><USER25> econophysicists? Wtf?

If you want to avoid the term "economics" because most people associate it with "made up financial bollocks based on applying completely inappropriate models from unrelated disciplines", then why include the "econo" part.

Also, you appear to be trying to be modelling a financial phenomenon by applying a completely inappropriate model from an unrelated discipline.

<comment><USER3> Derivatives trading is now dominated by quants with a background in the methods of computational physics. Aspects of gauge theory, etc are clearly and directly relevant. Don't reject this idea out of hand because it is unfamiliar to you. It may be proven a fad eventually, but that's not at all obvious now.

As a concrete example: the Higgs result depends both on machine learning in the form of decision trees to minimize the data, as well as monte carlo simulation to generate theoretical data for comparison to what is measured. Experimental particle physics is very much now an exercise in machine learning and probabilistic simulation.

<comment><USER12> Physical laws _exist_. Objects in the real world _have_ to follow those laws. I accept (I think) your point that ML and simulation can be used to "discover" those laws in some inductive sense.

What I'm having a hard time with is the idea that markets necessarily _have_ "laws" that are "discoverable" in the same way (which is what I think you're saying). They might do, and clearly some smarter people than me believe that they do, but where is the justification for this?

@contingencies mentioned [1] a George Soros quote about classical economics being based on a false analogy with Newtonian physics and markets are just the behaviour of individuals. Observations of particles tell you the behaviour of the particles, but the behaviour is fully determined by underlying physical laws. I don't see how this is true of behaviour of actors in economics, except in the obvious respect that the bodies of traders and the hardware of trading systems are subject to the laws of physics. What am I missing here?

I also wonder if there is a danger of building automated trading systems that are based on ideas from econophysics, and using their resulting behaviour are proof that the ideas were correct in the first place.

[1] [https://news.ycombinator.com/item?id=6481663](https://news.ycombinator.com/item?id=6481663)

<comment><USER3> > I accept (I think) your point that ML and simulation can be used to > "discover" those laws in some inductive sense.

Yes, we use these methods to confirm that reality matches some candidate law.

I personally don't think it's very meaningful to talk about physical laws existing in some independent platonic sense, but the debate over the principle of induction and the epistemology of science is a side story here IMHO.

Ultimately what we care about is that a model successfully predicts reality in the future. I think a useful conceptual division is: with some models, the internal structure of the model gives us understanding of the physical behavior. Most of what we call "laws of physics" are in this category. We can look at the algebraic expression and understand symmetries, constants, conserved quantities, etc. A second type of model is purely algorithmic and utilitarian: it gets the right answer, but the internal structure of the model doesn't provide any comprehensible information or understanding. Most ML algorithms fall in the latter camp.

What muddies the issue is that experimental physics tries to validate models of the first kind, but now generally requires methods of the second kind. We're forced into these methods because of the complexity of what we're studying (particle physics can involve thousands of Feynman diagrams), the noisiness of our measuring equipment, and the rareness of the events we're searching for.

When I say physics methods are very relevant to wall street, I'm talking about the latter methods more than any attempt to find fundamental theorems.

</comment></comment></comment></comment><comment><USER26> Under the broad meaning of Shadow Banking System, everyone speculating in any of the securities market would be a member of it. In my opinion this is very misleading and not really targeting the core feature, which is being a borrower and a lender of higher forms of money (reserves or deposits, not securities).

<comment><USER26> Why has this been downvoted? Can anyone share their view on Shadow Banking System definition?

</comment></comment><comment><USER27> TIL: econophysics is an actual thing.</comment>


Ask HN: I'm a senior in high school looking for a CS internship - <AUTHOR>

Dear HN,
I am a senior in high school who would love the opportunity to intern as a software developer this summer. From the searching I've done, it seems like every company wants someone currently pursuing a degree in university. I am still waiting to hear back from colleges before committing to one. I am familiar with C, C++, and Linux, which is shown by my GitHub page: https://github.com/TheBananaWhale
Do you have any advice on where to look? Is it worth trying to get an internship before I start college? <comment><USER2> As someone who had an internship coding an working on circuits in high school, I can say that it was one of the best experiences of my high school career. I was fortunate enough to have channels in my community to facilitate landing the gig but I worked there for 9 months and learned a tremendous amount.

I would start by figuring out what companies are local to you and seeing if you can make a connection through friends or family. You can always try cold calling a recruiter but typically positions for kids in high school are exceptions and so won't be advertised.

</comment><comment><USER3> I did a high school internship (that the high school set up) for CAD work. After doing it for two years, I decided that wasn't what I wanted to do at all. Saved me a lot of money because I opted not to make it a college major. You should check if your high school could help you find something.

My last internship turned into a position, but to be considered, you had to be either recently graduated from college, or in a college program.

You should look locally, but know that larger companies may require you to be actively enrolled in a program to consider you.

</comment><comment><USER4> I interned at a 'Techstars' startup last summer. It was a great experience (I'm also heading to college next year).

My advice would be to contact 5-10 startups which you'd love to work for (a nice email with the stuff you've made and why you want to work there).

Lastly, try to schedule a lunch with the founders before you commit to an internship, it really helps both parties.</comment>


Google Will Eat Itself - <AUTHOR> http://gwei.org/

<comment><USER2> Dup from 19 hours ago: <http://news.ycombinator.com/item?id=2628328>

Many comments there already.</comment>