"Did You Eat the Whole Cake?" On Learning Estonian - <AUTHOR> https://deepbaltic.com/2019/09/12/did-you-eat-the-whole-cake-on-learning-estonian/ <comment><USER2> I love the list of vowel-heavy words. You can also construct some similar words in Dutch. Or at least you used to before 1996, when a spelling change introduced hyphens into many of these:

      zeeëend - sea duck, or Velvet scoter (nowadays "zee-eend")
      koeieuier - cow udder (nowadays "koeienuier")
      zaaiuien - seed onions (still legal I think)
      kraaieëieren - crow eggs (nowadays kraaieneieren, I think)

We also have the opposite:

      angstschreeuw - cry of fear
      slechtstschrijvend - worst writing

These are fairly normal words. With a bit of creativity, you can make some that are much worse:

      borsjtsjschrokkende - borscht gorging
      haaiaaioorkonde - shark petting certificate
      weggooiooien - throw-away ewes

<comment><USER3> I shuddered when I saw what a 4 letter word got mutilated into

originally (russian) it was Борщ

And coincidentally Borš in estonian.

And in dutch it became borsjtsjsch? How? Why? Why do you need 8 letters to transmit a single phonem?

No wonder dutch can construct these weird word combos.

Edit, nvm, apparently last sch belongs to next word. Still creepy

<comment><USER4> Because there's no close sound to щ /ɕː/ in many European languages.

Some languages approximate it with /ʂ/ (think "sh" in English). For example, Romanian: borş.

Most languages approximate it with a combination of /ʃ/ ("sh"), /t/, and /tʃ/ ("ch"):

\- "sh" and "ch",

\- "sh" and "t", or

\- all three, "sh", "t" and "ch".

Or an approximation thereof.

In Swedish: Approximation for /ʃ/ is "sj" /ɧ/. Approximation for /tʃ/ is "tj" /ɕ/ which yields "borsjtj". Though they could've gone for just "bortj" :)

Dutch is "Borsjtsj": Approximation for /ʃtʃ/ (sh t ch) with sj /ʃ/ and t

German is "Borschtsch": Approximation for /ʃtʃ/ (sh t ch) with sch /ʃ/ and t

etc. etc.

<comment><USER5> My Russian teachers always used the old "fre[sh ch]eese" example for us English speakers.

<comment><USER4> To my ear "shit/sheet" are closer to щ :)

</comment></comment></comment></comment></comment><comment><USER6> > _When I asked our teacher how children learn which case to use, her rather > wonderful reply was that "it comes with mother's milk"._

As a native speaker, I think the real answer is experience. You pick up most of the patterns simply by using the language. I think that's how most people learn English as well. English grammar rules are mostly a mystery to me and many of my peers from school. Estonian grammar is slightly less of a mystery, but that's because it is a mandatory subject even in high school (ages 16-19).

 _About the title:_

"Kas sa sõid koogi (ära)?" \- "Did you eat the (whole) cake?"

"Kas sa sõid kooki?" \- "Did you eat cake?"

"Ära" in the first sentence has the meaning of "nothing left" or "gone". "Ära" doesn't have to be included, but in practice, it almost always is. It's similar with the abessive ("without") case. The example given in the article is "riik on ilma juhita". The word "ilma" means "without" and can be left out. The abessive case of "juhita" is enough - "riik on juhita" \- but in practice "ilma" is often included.

All in all, people generally don't care too much if you get the grammar just right. Mostly right is good enough. Many native speakers even have problems with words such as "kelle _-ga-gi_ " vs "kelle _-gi-ga_ " (the first is correct: -gi/-ki is always at the end). Some grammar rules can even result in words that many native speakers will say are incorrect, eg "keni" \- the partitive plural of "kena" (neat). People are simply more used to the -id/-sid partitive plural "kenasid".

<comment><USER7> From personal experience I think English speakers have troubles approaching synthetic languages in general because the English language is very analytic. I lurk in a few language-learning forums and I try to help anglophones learning French from time to time, they're often overwhelmed by the conjugations and to a lesser extent the grammatical genders which are omnipresent in French grammar.

On the other hand I've been studying Russian pretty intensely over the past year or so, so I can definitely empathize with the frustration of having to learn a complex case system.

I think the important thing is: it doesn't really matter if you get it completely right (or even mostly wrong). As long as you build simple sentences a native speaker will be able to figure out what you're trying to say. If somebody asks me "tu parlons anglais ?" it's definitely going to sound very wrong to me but I'll probably be able to figure out what you're attempting to communicate.

I think it's worth it to actually learn the declension/conjugation tables to familiarize yourself with them but you have to face the fact that you'll probably never learn to use them correctly just by memorizing rules. Eventually as you encounter them IRL you'll start to develop a feel for it.

I wonder if the average English speaker would be able to explain why they say "if I were you" and not "if I am you". Yet few get it wrong.

More generally I find that the main difficulty in learning languages is, by far, vocabulary. You can often get by with a surprisingly low amount of grammar if you understand what the words mean.

<comment><USER8> > If somebody asks me "tu parlons anglais ?" it's definitely going to sound > very wrong to me but I'll probably be able to figure out what you're > attempting to communicate.

If web browsers had feelings, they would possibly often experience this with the HTML soup being thrown at their parsers. "Hey, I think I know what you're attempting to communicate!"

</comment></comment></comment><comment><USER9> Wonderful article, exposing some of the oddities of this non-Indo-European language in the middle of Europe, without becoming too technical.

If you're interested in an entertaining overview of all of the world's languages, I can recommend the course _Language Families of the World_ by The Great Courses - 34 byte-sized (half hour) lectures.

[https://www.thegreatcourses.com/courses/language-families- of...](https://www.thegreatcourses.com/courses/language-families-of-the- world.html)

<comment><USER10> Certainly looks interesting, but in an era where I've been conditioned to expect free content $335 is a bit steep to indulge in a casual interest.

<comment><USER11> Maybe it's a price anchoring tactic to make the subscription plan ("The Great Courses Plus" \- which includes this) look like good value.

<comment><USER9> Yeah, plus they have period revolving discounts, where you get it for 80% or more off.

I must say I started listening to The Great Courses way before Udacity, Coursera, EdX, and so on, and back then it was really a good and competitive source (and, at discounted prices, reasonably priced) to get a variety of professionally produced audio courses. Nowadays of course there are more alternatives.

</comment></comment></comment></comment><comment><USER12> As a Swedish speaker, I get this deeply rooted feeling of righteous indignation about not understanding spoken Danish. I'm only half-joking.

As a Finnish speaker, I don't get that feeling with Estonian. The written form is different enough to warrant a deeper look before I start whining about how annoying I find language studies in general.

<comment><USER13> As a Finnish speaker, to me Estonian sometimes sounds like I'm having a stroke. Like I'm supposed to understand what is spoken but the words just don't resolve into their meanings.

<comment><USER14> As an Estonian speaker, I feel the exact same way about Finnish. I guess it's universal :)

<comment><USER15> No it isn't :) As a Finnish speaker with some exposure to Estonian language, I can understand written and spoken Estonian better than any other language I've had such little exposure to. I'm not very good in speaking Estonian and I can't write it at all.

<comment><USER12> Out of curiosity, how did you go about picking up this fluency in Estonian? Are there any good sources of particularly easy Estonian to get started? I'm semi-interested.

Granted, I'm also super lazy, but Finnish and Estonian should be close enough to just have some threshold to get over until it's pretty smooth, I guess?

Fwiw, I'm native in Swedish, but I speak and write Finnish daily.

To go on about my earlier comparison, I can really visit the site for a Danish or Norwegian Bokmål newspaper like Politiken ( [https://politiken.dk](https://politiken.dk) ) or Verdens Gang ( [https://www.vg.no/](https://www.vg.no/) ) and start reading with a feeling of just moderate confusion. Academic text would require some extra effort. The "voice in my head" or mode of reading doesn't really even change, because I've never attempted to communicate in either Danish or Norwegian per se.

For Estonian Postimees ([https://www.postimees.ee](https://www.postimees.ee) ) I'm stuck pretty much immediately. I can pick up words and some context, but that's it.

At my first job, at an internal IT Service Desk for a certain Nordic energy company, I dealt with Norwegians and Danes. But it was a business context where everyone just seemed to agree to just make a polite effort at having a maybe slightly "clear in a Swedish way" common Scandinavians language, because switching to English would feel a little silly. I guess some of the older dudes working in the field somewhere in Norway might not have spoken English.

<comment><USER15> I'm not fluent at all. I had several long vacations there as a child, visit every now and then and have some exposure to news and culture. And I'm pretty good in getting a superficial understanding of a language, good enough to read or listen to news and understand the gist of it. I can handle several languages I have never formally studied.

<comment><USER12> Mm, I referred to 'fluency' as a sliding scale, which I think is a thing.

But anyway, there we probably have it, your vacations as a child. No lifehacks can beat exposure to language at a young age, I guess. SAD.

Waiting for the day when some drug can trick the brain into some semblance of a kid's language learning skills.

</comment></comment></comment></comment></comment></comment></comment><comment><USER4> I always find it strange how language materials insist on presenting agglutination as cases. I was lucky enough to not have it when I studied Turkish, so for me the

\- -da is "in", not "locative case"

\- -(y)a is "towards", not "locative case"

etc.

I personally find that insisting that students learn proper case names greatly hinders the learning process.

Finnish and Estonian are also agglutinative, and I feel abject horror reading about "essive", "comitative", "abessive" and other cases. Stahp! :)

<comment><USER16> > Finnish and Estonian are also agglutinative

Finnish and Estonian are synthetic languages now. While the Proto-Uralic language ancestral to them both was more agglutinative, the system of consonant gradation (and subsequent loss of consonants) as it developed in these languages means they are no longer very agglutinative.

In a purely agglutinative language you only have to memorize one stem for a word and then you can add all case endings to that one stem. In Finnish, you have to memorize at least two stems (strong and weak stems). In Estonian, things are even more complicated.

<comment><USER17> Current limitations of English centric software language parsing means it's annoying to type machine generated or parseable text in Finnish.

For example pinging people in Facebook comments doesn't work with correct Finnish.

English: "this would be useful to @Pekka"

Correct Finnish: "tämä olisi hyödyllistä @Pekalle".

Machine parseable but incorrect Finnish: "tämä olisi hyödyllistä @Pekka :lle"

It would probably be possible to make a library to understand the transformed forms of words.

</comment></comment></comment><comment><USER14> An interesting thing I've noticed is that a lot of the younger generations in Estonia use English a lot when expressing themselves, especially when talking to their peers. I constantly hear everything from single English words in sentences, to fully English sentences used in-between Estonian ones.

I'm sure this is something that's happening everywhere (because of things like Hollywood and most of the internet being in English), but with the population in Estonia decreasing anyway, I wonder if this is an indication that the Estonian language will be dying out relatively soon.

<comment><USER16> Young people across Europe use lots of English words or even whole sentences. It tends to be a fad though, a way for young people to show off how trendy they are. Once my own peer group in Romania all reached their thirties and were no longer so interested in showing off, everyone generally dropped the English admixture and went back to speaking fairly "pure" Romanian.

Granted, Estonian may be in a different situation due to demographic factors.

<comment><USER6> I don't know if it's really a fad. I use plenty of English words when speaking Estonian to somebody that also speaks English well. Sometimes the language switches every other sentence or even mid-sentence. Some concepts are just easier to put forth in English.

I suppose part of this is that a lot of tech vocabulary in Estonian either doesn't exist or is quite awkward.

</comment></comment></comment><comment><USER18> The "total object" and "partial object" distinction exists in Finnish as well. One famous example is the verb _naida_ , "to marry": _nain hänet_ (total/nominative) means "I married them", but _nain häntä_ (partial/partitive) means I married only a part of them, or in more colloquial terms, "I fucked them".

</comment><comment><USER19> Great article showing some of the complexities in Estonian, really appreciated this breakdown.

I recently worked on a data visualization platform for the trade statistics office in Estonia ([https://data.stat.ee](https://data.stat.ee)) and many of the nuances from this article came up while we were working on the bilingual version of the site.

For example adding prepositions in front of country names, required us to write a custom function taking into account the normal "rules" and exceptions.

Here is the case for _Estonia_ \+ _preposition_

From Estonia: Eestist

To Estonia: Eestisse

With Estonia: Eestiga

Between … and Estonia: Eesti ja … vaheline

And now here is the case for the _UK_ \+ _preposition_. (notice the extra _g_ added in the _from_ , _with_ and _between_ cases)

From United Kingdom: Ühendkuningriigist

To United Kingdom: Ühendkuningriiki

With United Kingdom: Ühendkuningriigiga

Between … and United Kingdom: Ühendkuningriigi ja … vaheline

</comment><comment><USER20> My favourite Estonian pun is a pink bench. Pink means bench in English :)

</comment><comment><USER21> A wonderful introduction to the Estonian language and some of its quirks. I have minor quibbles, although my relationship to the language is as a non- native historian of the country, not a Finno-Ugric linguist or local. I always felt (rather unscientifically, of course) as though Estonian traded some of the agglutinative regularity that characterize Finnish and Hungarian for greater vocabulary familiarity for Germanic speakers. Despite what the author writes in her first endnote on foreign loan words, I've found a knowledge of German to be quite helpful when trying to guess the meaning of a word (or to extemporaneously construct one) using a calque from German.

<comment><USER16> > I've found a knowledge of German to be quite helpful when trying to guess > the meaning of a word (or to extemporaneously construct one) using a calque > from German.

The same can be said of Hungarian. In spite of Hungarian being a Uralic language and utterly opaque to the neighbouring countries, so many of the verbal derivations (especially since the 19th-century language reform) are calques from German. I found it easier to memorize certain Hungarian verbal idioms by storing them mentally alongside their German models.

</comment></comment><comment><USER22> On the subject of weird language constructs there's also the following correct Swedish sentence:

 _Bar barbarbarbarbar bar bar barbarbarbarbar_

This translates to _' naked naked-barbarian-bar-barbarian carried a naked naked-barbarian-bar-barbarian'_, a worthy competitor to the well-known _Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo_

</comment><comment><USER23> I never thought about the langugue they speak in Estonia

TIL there is a langugue called Estonian :)

<comment><USER24> There aren't many countries in Europe without their own language. Switzerland, Austria, Cyprus, Belgium, and the microstates -- can't think of anything else?

<comment><USER25> Switzerland has Romansh as one of its four national languages, though I'm not sure if that's the context you were intending. [https://en.wikipedia.org/wiki/Romansh_language](https://en.wikipedia.org/wiki/Romansh_language)

<comment><USER24> Interesting, I somehow had missed the existence of this even though I've lived a couple of months in Switzerland. Thanks for the link!

<comment><USER25> My pleasure!</comment></comment></comment></comment></comment>


Surprising importance of spontaneous order and noise to how we think - <AUTHOR> http://nautil.us/issue/68/context/why-the-brain-is-so-noisy <comment><USER2> A very similar effect is harnessed in current machine learning models and is called "dropout". The basic idea in context is:

\- you take your reference data and reserve a random chunk of it to figure out how good you're doing, this data is _only_ used to grade performance -- the rest is for training.

\- you notice that as you train on the training data, you start improving your performance on the evaluation data, but only up to a point: then as you train more, the neural net starts getting worse on the evaluation data.

\- this problem is called "overfitting", your neural net is now trying so hard to get the "details" right that it is losing accuracy on the "big picture".

\- so we create a circumstance where the neural net cannot properly do overfitting because those "details" cannot really be resolved, by just dropping nodes at random out of our neural network as we're training it. It will have to pick up a level of redundancy in the nodes and the associated noise of losing nodes should stop the later stages from really having a "focused view" of the training data, such that it can overfit that data.

If you're interested see e.g. Adrian's Medium post about it, here: [https://medium.com/@bingobee01/a-review-of-dropout-as- applie...](https://medium.com/@bingobee01/a-review-of-dropout-as-applied-to- rnns-72e79ecd5b7b)

<comment><USER3> Dropout is computationally expensive though. It does address overfitting (by doing something like training an ensemble of weaker models, none of which will individually overfit, but all of which capture different "views" of the dataset), but it's not without its drawbacks, and people seem to be using it less these days.

<comment><USER4> Qua? I was under the impression that it was just going into architecture with less remark - ubiquitous, rather than dying. It's a great step towards building sparse models, as well, which is important for client side deployments.

</comment></comment></comment><comment><USER5> > When a new image is sufficiently different from the set of training images, > deep learning visual recognition stumbles, even if the difference comes down > to a simple rotation or obstruction.

I'm about as far from an ai expert as you can get.

When I see and recognize a school bus, it seems that object remains a school bus to me until there is very significant evidence otherwise, whether it is ahead, beside, tipped over, or behind as referenced in the example.

It would seem ai on a single image is problematic, and needs classification over time to gain "confidence" instead of a single attribution.

Edit/additional thought: It also seems to me that I know and accept that it's a "bus" before I know it's a "school bus" while another person might immediately recognize a "school bus" and then think "that's a type of bus." How wonderful to think of how those arrangements of hierarchies leads to differing opinions and creative abilities in humans.

<comment><USER6> "It would seem ai on a single image is problematic,"

It means that whatever it is that these latest models are doing, it still isn't what we are doing as humans.

What exactly that difference is... well, if you could confidently and even more importantly, _correctly_ tell me, in a way so detailed and correct it was implementable, you'd be able to become very rich.

<comment><USER7> Well for starters, humans work on continuous video streams rather than still images, so there's a ton more information there. Even when we're identifying a still image, we're looking at a video stream of an object showing a still image (which is why a photo can look "exactly like the real thing" but we're never in any doubt that it's a photo and not the real thing.)

</comment></comment></comment><comment><USER8> I once built an evolutionary algorithm to evolve the neural network of a virtual robot. When I saved to disk and tried to rerun it produced a different result. Turns out the performance of the network was sensitive to floating point precision errors. I added some slight noise to the input values which made it much more robust.

</comment><comment><USER9> Brain is lot more complex. Trying to equate Brain to machine learning or neural network is not correct. The key ingredient of the brain is the reptile brain which is based on fight or flight and highly optimized for survival. Everything uses this part. In order to build similar, we have to build a neural network based on this survival instinct.

</comment><comment><USER10> The most important sentence is the last sentence:

 _There's still a considerable gap between real intelligence and so-called artificial intelligence_

My Tl;Dr on the article is that it's a lot of maybe, we don't know, we don't know why, we are not sure, maybe.

But I certainly agree with the final sentence.</comment>


Ask HN: How do you negotiate to get $100k/year as a SE? - <AUTHOR> I'm a software developer for more than 5 years now, the majority of the projects I worked web software and currently working for an outsourcing company working with US clients. I'd like to know how I'll be able to negotiate to get >= $100k/year of salary, despite me located and based in SEA? Thanks! <comment><USER2> (Personal capacity.)

SEA = Southeast Asia in this context?

Try to get hired into finance or directly by a US company which considers software crucial to their business, and give the company cause to believe they are reasonably in competition with other companies in that set during your recruiting process, so that they pull out their non-local pay scale. My ambient impression from knowledge of a few hiring markets in Southeast Asia is that $100k is either relatively high or prohibitively high on the local scales; this is materially untrue of software engineering for companies in the target set these days, _but_ they are happy to economize at their non-US offices _if_ they think they can get away with it. (Reporting, not endorsing.)

After that, standard advice for salary negotiation applies, but that's probably both the most impactful single thing you can do _and_ about all that is necessary to hit $100k, since that's effectively below the floor for starting salaries in software engineering in the set of companies described above.

(At least some companies in that set hire engineers based in Southeast Asia.)

</comment><comment><USER3> If you want to make $100,000 from anywhere you need one skill, sales. Learn how to sell. Read and watch everything you can on selling.

I recommend you start with "How to Become a Rainmaker: The Rules for Getting and Keeping Customers and Clients" by Jerrfey J. Fox.

Also learn how to stop charging hourly for your services and see [https://jonathanstark.com/](https://jonathanstark.com/)

</comment><comment><USER4> I have worked with several client organisations who use SEA based outsourcing companies. The main reason for their choice is the lower costs compared to using companies with staff based in Western Europe and USA.

So if you are already working for an outsourcing company that has clients in the USA, then you are most likely working for one of the more respected companies and thus probably earning near the upper end of the scale for your location. I doubt that your management will want to pay you more than they have to, no matter how good you are.

I am assuming that you mean USD100,000 p.a. not HKD. For that level of salary you will probably need to travel to countries where they do pay that much for your level of expertise. As @Annatar suggests, 5 years experience with web software, you probably won't be able to call yourself a senior SE in some markets.

</comment><comment><USER5> You need to be a senior programmer: that means 20 - 35 years of experience. Provided you have that kind of experience you will easily be able to command that kind of a salary, although only in Western Europe and the United States. Writing from experience of working in both.

</comment><comment><USER6> That's basically table stakes for a senior engineer no? At that point you should ask for a bit more salary and a better bonus structure or RSUs

<comment><AUTHOR> I actually think that we're underpaid since we live in a country where labor is cheaper even though we get more and more experience, we can't compete to get the same compensation as to the Western part of the world. How do you negotiate something like a raise that much even though there are these circumstances?

<comment><USER6> Where is home? Maybe you compete for other things like better working hours, more work from home time, more vacation?

<comment><AUTHOR> Philippines, I do work as a remote dev. Yes, we do have flexible hours but I still grind 40hrs/week.

<comment><USER6> You might be able to maximize your earnings going independent and becoming a contract worker.</comment></comment></comment></comment></comment>


Discuss link broken? - <AUTHOR>

The discuss link in the story below is inactive.  Is this a feature?<p>http://news.ycombinator.com/item?id=3585561 <comment><USER2> Sigh. You can't add new comments to old threads (presumably this prevents various kinds of attacks).

<comment><AUTHOR> This is what I thought. I'm surprised Squadmail (link in opening post) has posted their product more than once on HN and has gotten nearly no comments or upvotes.</comment></comment>


Technical recruiting: what we can learn from the NFL - <AUTHOR> http://www.bravenewtalent.com/item/2777745/technical-recruiting-what-we-can-learn-from-the-nfltraditional

<comment><USER2> :)</comment>


Fitbits seem to be haiving issues with time - <AUTHOR> https://twitter.com/tiernano/status/953511144693616645 <comment><AUTHOR> was woken up 2hrs earlier than usual this morning by my fitbit thinking it was 6am, when it was only 4... phone the fitbit is connected to shows correct time, so does everything else in the house... other people replying to the tweet seem to have the same issue. bit more digging here... [https://twitter.com/tiernano/status/953533936034033665](https://twitter.com/tiernano/status/953533936034033665)</comment>


Ask HN: How much to charge for our API? - <AUTHOR> We have been working on an algorithm for a while, and later realized that it can turn into a great marketing campaign (sort of "creating a story for your product" type of thing). We pitched the idea to a big company and they are very interested in using it to sell more products (their annual revenue is around $250M). They told us that they prefer use our service as a per-use API with a maximum cap. But we are not sure how to go about pricing. Our questions are as follows:
1) Should we price the API calls based on the value of the product? (each product that our service will help to sell costs $100 on average)
2) Is it better to price the API calls based on 'usage' or based on 'products that ended up sold'?
3) Is it common to request a lump-sum amount up-front (e.g. $50K) in addition to API costs that we will charge later? (since we will tailor our product based on their website and needs, we actually need to spend time and effort to make our service work on their end)
4) How to go about the maximum cap? (basically if the API gets used so frequently, they want to know the maximum amount they will ever get charged) Should we value it based on companies revenue? ($250M) or are there other factors we should consider?
So to sum up, its not really just the cost of our API call, but also the cost of using our idea and our algorithm. Any other suggestions are also highly welcome. <comment><USER2> A few different things here.

1: Unlikely, most often this leads to complicated pricing structures, and would you really be willing to take less from a store using it to sell... Candy vs a Store using it to sell say, Ovens.

2: It's likely better based on usage.

3: That is likely integration cost + contracted usage / time-frame.

4: Figure out what your infrastructure can support. Get estimated traffic from the types of sites you want to be a part of. Is the traffic on every page, or just the 'thank you' page. Do comparisons. Chances are you'll need adjustments (either on the API itself, or the inbound traffic) to avoid performance issues. Multiply that by the number of customers you expect to get a rough approximation of the total calls and the max ballpark of each customer. Recommend coming in somewhere below the Max throughput as the targeted baseline cap (figure out if cap is daily, weekly, monthly, quarterly, etc, and how those differences affect your inbound traffic).

Some things to think about...

Does the API have security on it that needs to be configured?

Do you have a customer facing Dashboard that proves to the customer that your product works via API analytics (how much traffic they've used, etc)?

Do you have a customer facing Dashboard that allows the client to make any changes to the configuration of the data return, or is that all done in the URL signature.

Can the requested Signature and the Dashboard have different configurations, if so, which wins?

Do you have a Analytics integration package that proves to the customer that your product works?

Do you have an idea of how your product would work in a A/B test scenario?

Do you intend to prove value real time, and if so - does that mean your API needs to account for 'hold out' scenarios to establish a comparison baseline?

Do you have your API documented for Developers, including sample requests and same responses to build their integration around?</comment>


Why Aaron Swartz was right - <AUTHOR> https://twitter.com/jonathanmarvens/status/484803047106039808

<comment><AUTHOR> I work in the same building as this kid. Hes a really smart kid but I don't understand some of the things he says in this series of tweets and I think he's full of shit. Would love to hear other opinions</comment>


Show HN: Damocles – An Erlang library for distribution testing - <AUTHOR> https://github.com/lostcolony/damocles

<comment><AUTHOR> This is something I've been working on in my spare time for a bit in preparation for a project I'm about to start at work.

While there are libraries out there for testing certain distribution behaviors across multiple nodes (such as Aphyr's Jepsen), and utilities/libraries that will monkey with traffic headed outwards (Crapify, Comcast, etc), I didn't see anything that gave me the control I wanted to be able to programatically drive multiple adapters, that I might make easy use of in automated testing.

I always like being able to run my tests locally, without having to rely on shared network resources. To that end, this basically lets you spin up a bunch of local adapters, and start dropping/delaying packets between them, without affecting eth0, and without affecting 127.0.0.1/localhost (so running tests shouldn't interfere with anything else, network-wise). You'll need to get some IPs not in use on the network, but everyone running the tests should be able to use those concurrently since they're not being exposed.

This lets youeasily test simple stuff (one node disappeared, and we try a write to a node, what happens?), to completely insane stuff (nodes 1 and 2 can't talk to 4 and 5, nor the reverse, everyone can talk to node 3, but 3 can't respond, and there's a 200ms delay between 4 and 5. And we try a write to a node. Or whatever), all on one box, in a repeatable, automated way.

I eventually want to expand it to work across remote interfaces, so that it can be used at load, but this is a start.

A command line hook is included; all functionality is callable from the command line through it.

<comment><USER2> Great work, thanks for sharing. Nice Dialyzer specs too!

I see you used the tc command. It is an often forgotten gem but really useful in testing failure scenarios in networked (distributed systems).

For everyone else:

[http://linux.die.net/man/8/tc](http://linux.die.net/man/8/tc)

</comment></comment><comment><USER3> I've been thinking of doing something like this for a while. However I want to be able to get datapoints on all the requests going though the proxies so that I can run integration tests on a bunch of distributed systems working together, and then be able to induce failures as well.

The other thing that I think would be great would be to add generative testing to this, to help with testing. Although im not sure how you would model the system, as I havent done much generative testing.

<comment><AUTHOR> Well, I'm not actually passing anything through proxies. Instead, I'm creating new loopback interfaces, lo:0, lo:1, etc. Presumably any tooling that monitors an interface could be set to watch those (in fact, you can create the interfaces on a machine, and leave them up in between runs, telling Damocles about them using 'register_interface'. That way they stay consistent and can be referenced by other tooling).

You can actually use this to test multiple systems working together as well. This is completely agnostic to what's running on the interfaces. As long as everything can be run on one machine (at least until I find the time to do the work to get it working remotely), you can set up multiple interfaces each of which runs an instance of one app and then another set of interfaces that each run an instance of another app (etc), configure the apps to talk to each other however you like, and then start monkeying with the connections between interfaces using this.

I'm not familiar with generative testing but in general, my goal wasn't to provide a testing framework for distributed systems, but instead a library with higher level primitives that could be used to easily compose pretty complex failure scenarios.

<comment><USER3> Ahh I see. So all the connecting up of what listens on which interface is done somewhere else. This just created failures at the network level in some scriptable manner. Everything else is up to you.

Makes sense.

What I was thinking about was actually setting up a proxy so you could mess with or record specific requests, and then if all the services in a system were wrapped like that then you would be able to do some really cool integration testing.</comment></comment></comment>


Ask HN: Where can I learn about Linear Programming - <AUTHOR>

I recently feel like I've been stagnating, writing code that requires no thought (sort of like writing longer versions of FizzBuzz). I want to do something a bit more challenging, and feel like Linear Programming will be a great thing to learn.
If anyone can tell me about some Linear programming resources, I'll be very happy. Also, if you have any suggestions as to other topics that I can look into, those are welcome as well.
Thanks. <comment><USER2> The book _Quantitative Methods for Business Decisions_ By Curwin & Slater would be a good start.</comment>


LibreSSL – More Vague Promises - <AUTHOR> http://www.tedunangst.com/flak/post/libressl-more-vague-promises <comment><USER2> Since it was mentioned, what was the rationale for IETF to use an incompatible implementation of chacha20-poly1305? My google-fu wasn't strong enough to pull up leads.

</comment><comment><USER3> "How more work can there be?"

Missing a word.</comment>


Android Fragmentation - <AUTHOR> http://www.avc.com/a_vc/2012/08/android-fragmentation.html <comment><USER2> While its good to see a big Android supporter like Fred Wilson acknowledge the fragmentation issues, I think convincing any developers out there that this fragmentation is a good thing is going to be a really hard sell.

Beyond just the OEM customization, the whole Android install base is spread out among a number of different (often older) versions - this is not something you see with iOS where Apple does a great job of getting their install base to upgrade quickly to the latest major version.

<comment><USER3> it might make sense to just focus on the google and samsung phones at first

</comment></comment><comment><AUTHOR> Hmmm, not sure if I agree with Fred Wilson on this one. Fragmentation is a real pain to deal with. I guess if the fragmented parts aren't too numerous and they have tons of users then you just deal with it.

As a developer, Android presents a challenge: it's a pain to develop for but it's too big to ignore.

<comment><USER3> yes, exactly</comment></comment>


Famous Last Words: Three Memoirs of Mortality - <AUTHOR> http://www.slate.com/articles/arts/books/2017/07/nina_riggs_the_bright_hour_and_other_memoirs_by_dying_authors_reviewed.html <comment><USER2> disclaimer: I've read (but am aware of the premises of) none of the books the article mentions, so I might have misunderstood the message/tone of the article.

> Like all real literature, it aims above all to tell a truth, however roughly > it goes down.

I wish this statement and its ilk were less common; in my opinion it's either a platitude or false. For example, another plausible raison d'etre: to share an observation. It's not hard to think of others. Now you could say there's necessarily some higher truth encoded in any meaningful observation. But, in my opinion, that line of reasoning ends with the quoted statement being a platitude.

This point could be pedantry, but I make it because I get the feeling the article's author would say Pausch's "the Last Lecture" is not "real literature" because it's purpose is mostly to make people feel good.

</comment><comment><USER3> If you liked When Breath Becomes Air I'd also recommend Do No Harm by Henry Marsh - it's similar in style (maybe like the book Kalanithi would have written at the end of his career). It still touches a lot on mortality. Here's a great chapter from it: [http://lithub.com/aneurysm/](http://lithub.com/aneurysm/)

I'd also add Hitchens book "Mortality" as a suggestion too.

<comment><USER4> BBC has a documentary on Henry Marsh called The English Surgeon. It's also a great watch.

</comment></comment><comment><USER5> Stewart Alsop's _Stay of Execution_ is a good deal older (1970s), but I remember it as very well done.

<comment><USER6> Elsewhere in the fertile field of morbid Conservativism, there's good rumination to be had in the shady corner that is Book III of the _Alan Clark Diaries_. Despite being likeable and witty, the man's an aristocratic snob, a bigot, a serial womanizer - and his miserable end, retold explicitly, renders him palpably futile. The last chapter is written by his observant wife, as he lies wrapped in sheets gurgling and voiceless.

I pose it as if I enjoyed it, but I didn't - it was a terrifying read. Makes you wonder, you know, what to do with your own life (which is to plug the gap with a platitude). Maybe Alan Clark would have been saved if he'd regularly procrastinated on _Hacker News_? Fingers crossed.

</comment></comment><comment><USER7> I was expecting "Here, hold my beer."</comment>


A Grimoire of Functions - <AUTHOR> http://fredrikj.net/blog/2019/05/a-grimoire-of-functions/ <comment><USER2> Seconded! Fredrik is a hero.

Favorite line:

"Euler's formula... Discovered around 1740 by the Swiss Sorcerer Supreme, Leonhard Euler, this formula can transmute exponentials into trigonometric functions and vice versa. A magic formula indeed!"

Indeed! (And a mathematical IDENTITY... as are many others in this "Grimoire"...)

Great work, Fredrik!

<comment><USER3> > _Discovered around 1740 by the Swiss Sorcerer Supreme_

And earlier published in 1714 by the Englishman Roger Cotes.

</comment></comment><comment><USER4> See also Abramowitz and Stegun, of which the latest edition is [https://dlmf.nist.gov/](https://dlmf.nist.gov/)

<comment><USER5> They have a really nice changelog.

</comment></comment><comment><USER6> This is a delightful read. It has the neuroticism of someone whom I admire quite a lot and yet has the urge of curiosity. Thank you.

</comment><comment><USER7> The "rewrite engine" ambitions for Fungrim seem potentially misplaced. It would probably be better to contribute to an existing open source state-of- the-art CAS like Fricas, than to make another one. The article tries to justify this in a weird way:

> Computer algebra systems can be viewed as having three components: a way to > represent mathematical objects, algorithms for manipulating such objects, > and user interfaces around this. The various components are usually > entangled with each other without cleanly separating different levels of > abstraction, so there is a lot of incidental complexity.

So? That is no reason to make another CAS, as opposed to helping refactor Fricas.

> The algorithms are incorrect by design, typically for efficiency reasons. A > good example is ignoring special cases and assumptions on the variables when > simplifying symbolic formulas (some examples given above). Another example > is pretending that floating-point numbers are the same thing as real > numbers. Calculating nonrigorously is not necessarily the wrong approach; it > all depends on the application. It's sometimes better to do something fast > and simple (but possibly incorrect), especially if the result can be checked > afterwards. Sometimes you really want every step to be correct, even if it > takes longer time and requires more complicated user input. For better or > worse, current computer algebra systems are largely designed around the > "fast and simple" principle, with optional correctness added more as an > afterthought (and often not fully supported).

WTF! Only if one uses Wolfram Alpha exclusively!

Another comment [0] by fdej (the author) I just found by accident has some related misconceptions:

> However, nothing open source currently comes close to Mathematica for > symbolic and symbolic-numerical calculus (expression simplification, > symbolic integration, solving PDEs, etc.).

Try Fricas.

> Also, one of Mathematica's main strengths is providing a lot of > functionality in a neat package where diverse components are very easy for > the user to glue together, with automagical interfaces where the user > doesn't need to know what happens behind the scenes

"Automagical" interfaces suck, they will inevitably break when you stray just a bit from the intended path. It is better to have a typed language like Fricas has.

[0] [https://news.ycombinator.com/item?id=17694483](https://news.ycombinator.com/item?id=17694483)

<comment><USER8> All good remarks.

> The "rewrite engine" ambitions for Fungrim seem potentially misplaced. It > would probably be better to contribute to an existing open source state-of- > the-art CAS like Fricas

For the record, I have contributed to several existing open source computer algebra systems (not Fricas though). I honestly don't think anyone has found the right foundations for computer algebra systems yet. Nor do I even think that there can necessarily be a "one size fits all solution" that works better than a multitude of specialized systems for different tasks. Fricas gets some things right where others don't, but it has its own weaknesses. I think it's valuable to explore different approaches. This possible future direction for Fungrim is more of a research project, and in any case, I think it's complementary to much of the work that is already being done on other open source computer algebra systems.

Sorry that this is still a bit vague. I have more specific issues in mind, but I don't have time to elaborate in this HN comment. I hope to do a better writeup in the future!

> WTF! Only if one uses Wolfram Alpha exclusively!

Or Maple, or Maxima, or SymPy, or Sage, or... even Fricas. Despite its type system, Fricas has the same problem as the rest with overzealous simplification. One of the very first examples in the Fricas book, "integrate (1/(x^2 + a),x)" gives a result that isn't valid when a = 0... And that's without even digging to look for something more subtle that will be more likely to catch the user off guard.

> Try Fricas.

I admit that I haven't tried Fricas in a long time, but I'm also confident that I wouldn't need to look hard to find examples. Here is just one quote from the Fricas book: "FriCAS can compute some forward Laplace transforms, mostly of elementary functions not involving logarithms, although some cases of special functions are handled." Mathematica, on the other hand, handles a huge number of Laplace transforms involving logarithms and special functions (largely using Meijer G-function methods in combination with internal lookup tables and many different heuristics). How well does Fricas support definite integration with piecewise functions or functions involving branch cuts? Mathematica has improved a lot in this area in the last few years (even if it isn't perfect). Of course, I'm sure Fricas has its strong areas too, and Mathematica is terrible in some areas.

> "Automagical" interfaces suck, they will inevitably break when you stray > just a bit from the intended path. It is better to have a typed language > like Fricas has.

Oh, I do agree completely about the problem with automagical interfaces that work until they don't :) Nevertheless, even "automagic" that breaks down at some point is great for discovery and prototyping. I think in many cases, we can do a better job providing interfaces that scale smoothly between automatic and user-controlled.

</comment></comment><comment><USER9> This is awesome. I have just started [https://nixpulvis.com/math/10-overview](https://nixpulvis.com/math/10-overview) as a way to index notation and LaTeX syntax for myself. I'll be sure to read though this from time to time. I was going to start adding D3 and all that, but now I'm wondering if I can just incorporate this. I too am using MathJax.

</comment><comment><USER10> Great idea. It might be more useful if it were compatible to and integrated with jupyter notebooks which has become widely used. See [https://www.jupyter.org](https://www.jupyter.org) for more information.

</comment><comment><USER7> > [Regarding CAS-es:] Another example is pretending that floating-point > numbers are the same thing as real numbers.

What kind of lousy CAS does not support arbitrarily precise numbers? For example Fricas/Axiom/Scratchpad gives arbitrarily precise results.

<comment><USER8> Arbitrarily precise is not the same as exact. There are many possible examples to illustrate why this matters. Here's one: let's say that you compute floor(f(x)) where f(x) falls right on an integer n. If you evaluate f(x) in floating-point arithmetic, you probably get n +/\- epsilon and not n exactly because of rounding errors. This makes the floor function go to either n or n - 1, so you sometimes end up with an error of 1 no matter how precise the floating-point numbers.

<comment><USER11> Right, and with a little extra effort, you can make an arbitrarily small error in one of your terms give a result that can be anywhere from negative infinity to positive infinity.

</comment></comment></comment><comment><USER12> Very nice! But is Python the right choice for this? It seems like the macro rewriting abilities of Lisp (especially Racket) would be a perfect fit for this project. And he's already using S-exps, so why reinvent the wheel?

<comment><USER13> Lisp macros are meant to rewrite S-expressions that represent source code in controlled, predictable and well-behaved ways, whereas S-expressions in Fungrim represent data and they can be transformed in generic and aggressive ways to deduce something useful.

Different purposes, with different needs and only a limited technology overlap (the theory of how to maintain program correctness through transformation is related to the theory of how to squash hidden assumptions and subtle type mismatches in algebraic formulas).

<comment><USER12> I think you misunderstand the genericicity of macros in Lisp. Macros take one or more S-exps and return and S-exp. nothing more, nothing less.

There's no reason why macros need to generate valid code. Though in Lisp, it's pretty easy to make any code you want valid.

</comment></comment></comment><comment><USER14> Not a bad TeX reference either, great stuff!

</comment><comment><USER15> Awesome! This guy's a hero.

<comment><USER8> Cheers :)

</comment></comment><comment><USER16> a grimoire never sounded so cool</comment>


Five Things to Take Away from Re:Invent 2019 - <AUTHOR> https://redmonk.com/sogrady/2019/12/06/reinvent-2019/?ck_subscriber_id=512837517 <comment><AUTHOR> This article is centered around AWS but there is a nice chunk near the end talking about the expansion of GitHub and Microsoft's role in that. GitHub Actions is really a step forward and is something that might threaten AWS, as well as others (CircleCI, Jenkins, etc.)</comment>


Ask HN: Which IoTs/smart home devices are useful and aren't gimmicks? - <AUTHOR> I'll admit upfront that I'm skeptical of IoT devices and anything that fits into the "smart home" category. I think the majority of it just isn't that useful and adds a lot of complexity and failure modes to stuff that would otherwise just work.<p>Do any of you have any recommendations for products/configurations in those categories that are actually useful and that don't have issues with complexity or security? Thanks! <comment><USER2> I find smart thermostats to be remarkably useful, and the primary smart device of mine I interact with. That being said, I recommend something like Insteon, Z-Wave, or ZigBee based hardware, be they lights or thermostats or otherwise.

Avoid having a lot of network-connected devices. One smart hub is one device you have to be worried about, instead of wondering if any given light bulb might be part of a botnet.</comment>


Who's In Toronto, Canada? - <AUTHOR>

<comment><AUTHOR> So, who's in TO? Anyone want to chat, or meet (at a pub, say)?

Going to the original startup school, and watching the YC dinner on Justin.TV has re-inforced how important it is to have a local support network. Even if you're not working on the same stuff, there's an energy that's motivating and focusing.

Whether you're planning to apply to YC, planning to start something locally, or already on the way, let's meet and share the energy.

You can contact me at spameatinggrin at gmail dot com.

<comment><USER2> I live about 15 mins away from Pearson Airport, Toronto. Just curious, what do you think about the startup environment in Toronto? Where is all the action at?

<comment><AUTHOR> I think it depends on what you want to do. This is a great place for film / video. It's getting better for games, and of course there's financial services. There also seem to be a bunch of mobile companies here, too.

There's a very healthy ecology of small software companies, and a lot of web companies. There's a bit of defence and aerospace around the airport, but I think Montreal and Ottawa are bigger than Toronto, for that.

Kitchener's not _too_ far away, and they're a major powerhouse with U of Waterloo, and RIM.

ATI just got acquired by AMD, and Alias by Autodesk, so there should be some newly minted angel investors around. I haven't sought financing, so I don't know what that's like. Having been through some private-equity as an employee, the funding definitely seems to be out there.

I think we're building a pretty good tradition, but it's going to take time. The seeds are there though, looking back over the last 20 years (god, I'm getting old!). There's nothing like the unholy Stanford-VC-startup triad yet.

So, I'm not really sure what to say. It's probably harder to get funded here than in CA, and you obviously won't get the same buzz as being YC-funded. On the other hand, if you want to work at a startup or small, hip shop then there're lots of places here.

EDIT If I ever move out of my home-office, I'm going to try to set up next to the U of T comp sci and engineering buildings. That's the nice thing about a down-town campus.

</comment></comment></comment><comment><USER3> I live in Miami, but may move to Toronto. was just there last week. The volvo- sized chunks of ice falling from CN Tower really endeared me. :)

<comment><AUTHOR> Hehe. The snow in my back yard's almost completely melted. Looks like the raccoons and cats left me some "presents". Mmm.</comment></comment>


Tim Cook, Stop Annoying Your Customers - <AUTHOR> https://www.bloomberg.com/gadfly/articles/2016-09-07/dear-tim-we-need-to-talk-about-not-alienating-customers <comment><USER2> If someone is price-sensitive to the point of wanting $1 headphones, why would they pay for a premium phone like the iPhone? His thesis doesn't make sense. And you get lightning headphones in the box.

<comment><USER3> What about someone who already owns a set of, say, Sennheisers; where cost-to- consumer actually reflects the manufacturing costs?</comment></comment>


Quantum Simulations with Julia - <AUTHOR> https://github.com/bastikr/QuantumOptics.jl <comment><USER2> The project home page is nicely designed and has some lovely examples as well: [http://qojulia.org/](http://qojulia.org/)

</comment><comment><USER3> QuTip is a necessary tool in Python, so I'm glad to see something in Julia as well. Excited to see where it goes.

</comment><comment><USER4> Looks great. I have a bunch of ad-hoc Matlab functions for quantum optics, which I should port to one of these more polished libraries.

<comment><AUTHOR> Me too. I'm stumbling across this from hacking some old fortran stuff into a quantum tomography simulation.

Thinking of trying it out here instead.</comment></comment>


Where the Real Skyscrapers Are (Hint: North Dakota) - <AUTHOR> https://medium.com/re-form/where-the-real-skyscrapers-are-hint-north-dakota-76b33694c99e <comment><USER2> Here's a video of a tower worker climbing a 1700' tower that really gives me sweaty palms: [https://www.youtube.com/watch?v=gDYK0zaQuZs](https://www.youtube.com/watch?v=gDYK0zaQuZs)

I've also been impressed with the amount of high quality content on Medium over the last several months. It seemed like a glorified blogging platform at first, but there have been a number of really interesting stories like this one and the one about the inventor of the Aeropress.

[https://medium.com/backchannel/first-alan-adler-invented- the...](https://medium.com/backchannel/first-alan-adler-invented-the-aerobie- now-he-s-created-the-perfect-cup-of-coffee-c5e94ccc538e)

<comment><USER3> At first I thought, "oh, i'm sure there's a line you can attach to your harness to keep you from falling..." but then he gets outside the tower, free- climbs 140 feet, finally positioning himself on the very top of a 1,700 foot tower, with zero support and a tiny piece of foothold, all the while carrying 30lbs dead weight [to say nothing of wind], whereupon he finally clips in his harness.

Jesus fucking christ.

<comment><USER4> Yeah, i was kind of gripping the table almost unable to contain shouts of "attach yourself already" :/ The worst is the final 20 seconds when he dawdles and fumbles the clip with two hands, simply standing at the top looking down :/ But i guess from his point of view that's easiest, he's not climbing any more...

<comment><USER5> I've heard those videos caused a lot of flack for the workers, due to the lack of safety equipment use.

</comment></comment></comment></comment><comment><USER6> "After a tower project gets FCC approval, it takes about a year to complete. While the KVLY tower cost $50,000 to build in 1963, a typical 2,000-foot tower today costs about $3 million."

I'd love to see the full breakdown as to why exactly. That seems grossly unnecessary.

Wolfram says $50,000 in 1963 dollars is $393,000 in 2015 dollars. So there's an almost 10x increase on top of inflation. [http://www.wolframalpha.com/input/?i=%2450000+in+1963+dollar...](http://www.wolframalpha.com/input/?i=%2450000+in+1963+dollars)

<comment><USER7> The price of technological goods such as TVs and computers have been going down constantly, and this means inflation of non-technological goods will be higher than the official inflation rate.

The price of gold is around $1150 in 2015, and $35 in 1960, a 32 times increase.

The price of a KVLY tower increased 60 fold from $50,000 to $3m.

Compared this way, 60x vs 32x is much less extreme.

<comment><USER6> I like your line of thinking, but gold seems like a really poor point of comparison. I tried to find inflation adjusted prices over time for things such as energy and steel and had difficulty. There's lots of easy data on oil but anything else is tricky. Hrm..

<comment><USER7> You can compare other goods like college education, property, medical care besides precious metals. Things that take a lot of skill and labor, and where technology have little to no downwards pressure on the price.

</comment></comment></comment></comment><comment><USER8> North Dakotan checking in. I've seen this tower many times, from the road and up close, it's pretty uninspiring. Hard to get a sense of scale when it's so thin and transparent. Also, you can't see the view from the top (legally). What fun would the Space Needle, Empire State or Burj Khalifa be if you couldn't see what the view is like from the top?

<comment><USER9> Empire State _Building_. Empire State is New York State.

<comment><USER10> I think from the context it's fairly obvious what he's referring to.

</comment></comment></comment><comment><USER11> The tower in Dallas, NC (on the map) is less than a mile from my childhood home. It sits in the middle of a cow pasture -- I grew up catching bluegill and catfish in a cow pond a few hundred yards from the tower. Never realized that it would be the 4th tallest building in the world if it were a skyscraper. Thanks for sharing.

</comment><comment><USER12> One thing the OP missed is the fact that Tokyo Skytree is also a broadcast tower. Just a bit better on the aesthetics than the American ones.

</comment><comment><USER13> Here's the tallest structures in the USA of which many are towers:

[http://en.wikipedia.org/wiki/List_of_tallest_structures_in_t...](http://en.wikipedia.org/wiki/List_of_tallest_structures_in_the_United_States)

</comment><comment><USER14> Interesting, especially since the replacement for the WTC claims its height using a similar structure mounted on top.

The experts decided its the tallest building in the US based on that non- building "architectural element", which could later be extended to re-capture the "title", for whatever that's worth.

[http://www.usatoday.com/story/news/nation/2013/11/12/world-t...](http://www.usatoday.com/story/news/nation/2013/11/12/world- trade-center-tower-willis-tower-largest-us-building/3504515/)

</comment><comment><USER15> Can you imagine witnessing one of those towers when they fell down?

<comment><USER16> Or climbing that ladder all the way to the top?

</comment></comment><comment><USER17> What's the limit on structures of this type? It seems unlikely that the radio towers we have now are the best we can do; it sounds like they just stopped making them bigger because there wasn't anyone that wanted a bigger one badly enough to justify the cost.

<comment><USER18> Legally (in the U.S.), the FAA restricts the height of towers to about 2000', with rare variances granted.

Technically, we don't need supergiant towers, now even less than ever before. High frequency broadcasters (TV) need line of sight to their customers, but -- economically -- TV transmission also has huge energy costs (wide bandwidth), and you're fighting the inverse square law the whole time. So in the low- populated areas where you might reach more people at the margins, it just doesn't make economic sense. In denser markets, the next city will be better served by a local station.

Also economically, the general rule of thumb is that the cost of a tower increases as the square of its height.

But I think you were asking about the physical limit. These masts are typically made of galvanized steel tubes. Pipe OD size and wall thickness (and therefore compressive strength) will vary, but that is the ultimate limiting parameter.

The galvanized steel guy lines are in tension, but it's easy to add more of these as height and footprint increase.

<comment><USER19> Just the first part is key. The FAA won't let you. The article mentions that there are many towers close to the 2000 limit placed by the FAA.

<comment><USER17> Other countries besides the US could build one, if they had a compelling use case. I was mostly just wondering what the physical limits are on this kind of thing.

</comment></comment></comment></comment><comment><USER20> Calling what happened in 1997 "a freak ice storm" is a bit of an understatement. Most of a state without power and only one of these amazing towers still transmitting. I do wonder if there is video of the aftermath of the fall.

<comment><USER21> On December 10, 1989, two of these 2,000 towers collapsed near Raleigh, NC. It was as the ice was melting after a really bad ice storm.

Pictures of the aftermath: [http://www.oldradio.com/archives/warstories/WRAL.htm](http://www.oldradio.com/archives/warstories/WRAL.htm)

</comment></comment><comment><USER22> These towers are usually "disqualified" from height records by using the weasel words "free-standing structure" or "self-supporting" or similar.

In the photo, you can clearly see the numerous guy wires holding it up.

</comment><comment><USER23> I may be old school, but a 3MB (mostly images) web page seems a bit over the top.

<comment><USER24> For a large display (iMac), it's glorious, but agreed, on a MacBook Air (let alone an iPhone), it's overkill.

There should be a way to determine the resolution of a target device, and just send the appropriately sized images.

<comment><USER25> I think that people are starting to realize that a one-size-fits-all approach to images isn't really feasible anymore. If you serve gigantor images that look good on retina or 4K screens to low res devices, you just wasted a lot of bandwidth and tanked your page load time for no reason. The opposite is also true: 640x480 images don't look super great on 4K screens.

You can resize your images to a few key sizes, but it can quickly get out of hand to support all the various permutations of size and format for every potential user (and who knows if most of these objects will ever even be served).

There are some solutions to the problem out there, such as the one from my employer: [http://www.imgix.com/imgix-js](http://www.imgix.com/imgix-js)

Basically though, to do responsive design with image heavy content, you need to make the images responsive as well.

<comment><USER24> "You can resize your images to a few key sizes, but it can quickly get out of hand " \- This shouldn't be a problem. Just resize dynamically as needed and cache. Resizing on the server is a pretty fast operation, and you only need to do it once for any given form factor.

</comment></comment></comment></comment><comment><USER26> Is there a resource somewhere to see how do they build a tower so tall?

<comment><USER27> I was wondering the same thing... If you've ever seen a construction crane go up, it's the same process apparently. Check out this older video: [https://www.youtube.com/watch?v=2KvjwdXA89w](https://www.youtube.com/watch?v=2KvjwdXA89w)</comment></comment>


Jack Dorsey: "It's Really Complex To Make Something Simple." - <AUTHOR> http://techcrunch.com/2011/01/11/jack-dorsey-charlie-rose/

<comment><USER2> I think that Square is an exceptional business enabled by a novel piece of hardware, their headphone jack card reader.

It would seem - then, that the card reader and payment service could actually be seen as a platform play.

Platforms are technologies that are useful themselves - but enable far reaching, broader use cases in ways, that at times, can be unforeseen.

Square could enable a range of cottage industries by providing other applications built on their solution.

We have the ability for mobile payments, as it were, and thus we should see a need for dead-simple mobile business management apps; inventory, supply chain, vendor management, invoicing, product lists etc.

This leads me to believe that Square is a platform that through its deployment applications can be built upon it that will change the way commerce can happen on the individual level.

Further - it would seem that there is also a great opportunity for sales distribution here as well. A product distributor could reach out to and enable a mobile sales force providing all these applications to their sales force in the field on a single device - as the merchants sell product, it can be tracked in real time and supplies replenished.

This could work very well in connected, yet less-developed countries such as rural Philippines, China and other parts of Asia.

Couple this with prepaid charge cards -- and the ability to LOAD cards in the longer term, and there are some significant opportunities that can be built using square alone.</comment>


Two Generals' Problem - <AUTHOR> https://en.wikipedia.org/wiki/Two_Generals%27_Problem <comment><USER2> Given HN's recent obsession with Bitcoins, it's probably worth noting that the "double-spend" problem is a variation on the Two Generals problem (or, more accurately, the Byzantine Generals problem). Regardless of your views on Bitcoin as a currency replacement, it's hard to deny that the block chain is a rather clever solution to this problem.

Of course, it's not exactly a _breakthrough_ solution. As noted in the article, it is possible to increase the confidence of coordination at the expense of speed and resources. In this case, requiring the "commander" to commit limited resources (computational power to generate hashes) and a majority of the "lieutenants" to agree on one commander's solution, then chaining multiple decision events together, the result is actually fairly robust.

Obviously, for most use cases something like the block chain is completely impractical. You wouldn't want to have to wait an hour for each commit to your web app's database. That said, it will be interesting to see if future variations on this theme make their way into other applications.

<comment><USER3> You may be interested in looking into some of the research that has been done in other areas on Byzantine consensus. Depending on the system model, solutions that are much more efficient than the Bitcoin block chain (but have other tradeoffs) exist. For example, check out Castro and Liskov ([http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.127....](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.127.6130)). Personally, I think some people in the Bitcoin community have been a bit dishonest in claiming that bitcoin's protocol solves a consensus problem which wasn't solvable before.

Lower bounds on Byzantine consensus in any practical system model exist which makes it very attractive, for efficiency reasons, to solve a subset of that problem for things like database replication in trusted systems. Lower bounds for consensus vary from f+1 (for f faults) to 2f+1 to 3f+1 depending on what kinds of faults you admit. Still it's very interesting to think about.

</comment></comment><comment><USER4> "Whenever I go the a conference and I discover that there will be a presentation about Byzantine fault tolerance, I always feel an immediate, unshakeable sadness, kind of like when you realize that bad things happen to good people, or that Keanu Reeves will almost certainly make more money than you over arbitrary time scales."

\-- James Mickens

[http://research.microsoft.com/en- us/people/mickens/thesaddes...](http://research.microsoft.com/en- us/people/mickens/thesaddestmoment.pdf)

<comment><USER5> Holy crap, that guy's an amazingly witty writer.

<comment><USER6> He uses standard techniques. Just compare to Dave Barry.

<comment><USER7> Best HN comment ever.

</comment></comment></comment></comment><comment><USER8> For a little bit more, here [1] is a link to the lecture notes from the Distributed Systems course I took that covers this. Also covers the related Byzantine Generals problem. If you find this interesting I'd recommend taking a look through the rest of the course's lecture notes too. They're very descriptive and written really well.

[1] [http://www.andrew.cmu.edu/course/15-440-f13/index/lecture_in...](http://www.andrew.cmu.edu/course/15-440-f13/index/lecture_index.html)

</comment><comment><USER9> This reminds me a bit of the "A Biological Solution to a Fundamental Distributed Computing Problem" paper that I believe came up here, especially the probabilistic approach with (implicit, synchronized) clocks in the last paragraph of the wiki page.

I made a little simulation of the former algorithm here

[http://jberryman.github.io/fly-mis/](http://jberryman.github.io/fly-mis/)

</comment><comment><USER10> The solution: a catapult.

Fire the message over Army B with a catapult.

<comment><USER11> Right! Forget the slow, unreliable upload. Overnight a USB drive.

<comment><USER12> I worked at one place where it was easier to take the server from a remote location, transport it to a central hub, upgrade and install new software, and transport it back then to download the updates and remote install. This was a while back, but bandwidth in some places was horrible and dropped often.

<comment><USER13> See also: [http://en.wikipedia.org/wiki/Sneakernet](http://en.wikipedia.org/wiki/Sneakernet)

</comment></comment></comment></comment><comment><USER14> Attacking Generals and Buridan's Ass, or How did the chicken cross the road? [http://muratbuffalo.blogspot.com/2013/04/attacking- generals-...](http://muratbuffalo.blogspot.com/2013/04/attacking-generals-and- buridans-ass-or.html)

<comment><USER15> Wait... A 2013 blog entry about the Two Generals' problem which repeats several times that there's an "impossibility result" in an unreliable distributed network for that problem and not a single mention of Bitcoin.

I thought that Bitcoin's major technical feat was _precisely_ that it showed that you could solve the Byzantine Generals' problem (which is a generalization of the Two Generals' problem).

So which is it? Impossible result or not? Or does the article talk about something else (but then a mention of Bitcoin, what Bitcoin solves and why it doesn't apply to what's described in the blog entry would have helped)? I'm confused now...

As a sidenote I'd say that writing about either the two generals' or the Byzantine Generals' problem in 2013 without mentioning Bitcoin even once is a bit weird.

<comment><USER3> > I thought that Bitcoin's major technical feat was precisely that it showed > that you could solve the Byzantine Generals' problem (which is a > generalization of the Two Generals' problem).

This sort of thing is extremely sensitive to the definition of the problem you are solving, and the system model you are solving it under. Impossibility results like Lynch and Gilbert's CAP result and the FLP result prove that certain problems are impossible to deterministically solve in finite time in some kinds of system models. Change those system models only slightly, and you get things like Ben-Or's algorithm ([http://dl.acm.org/citation.cfm?id=806707](http://dl.acm.org/citation.cfm?id=806707)) which depend on a random oracle that is not available in FLP's system model.

So it is with Bitcoin. Not only is the definition of consensus different (uniform consensus vs. probability-one convergence on consensus among non- failed processes), but the system model is different. As I understand the bitcoin protocol, a random Oracle is needed, and liveness is only guaranteed against some forms of byzantine behavior (but I may be wrong about that, I'm not an expert in this area).

> As a sidenote I'd say that writing about either the two generals' or the > Byzantine Generals' problem in 2013 without mentioning Bitcoin even once is > a bit weird.

I don't think Bitcoin was a substantial advance in distributed systems theory. It's got a lot of interesting properties for sure, but I'm not aware that there's anything theoretically new there.

</comment></comment></comment><comment><USER14> This formulation is also with "undetectable arbitrary message loss". If you can detect at the sender when a sent message is lost or succeeded (for example like collision detection in Ethernet bus, or wireless collision detection), the result doesn't apply. Because then the parties would know which round was successful and they can stop.

So smoke signals would work :-) Assuming bidirectional visibility (no clouds etc.)

</comment><comment><USER16> I don't understand this problem, I don't think. If it was so imperative that your generals attack at precisely the right time, why would you not have coordinated this attack long before the assault? Is that not a solution?

<comment><USER17> It is meant for things like a network, but to make the analogy work I will provide a scenario.

The imaginary Red Army, and the Imaginary Blue army were both being attacked by the Green army. After waging a long war the Red army and Blue army arrive at the capital of the Greens. The Red and Blue Armies have never met, and are only united by their hatred of the Greens.

The only way for the Greens to be defeated is for the Reds and Blues to attack on two fronts simultaneously.

<comment><USER16> No. I get point of the problem, but what I don't get is how it would be considered unsolvable. I mean it seems like it should be solvable through protocol, which this link doesn't seem to explain how it can't.

<comment><USER18> In a network, "be in the same place" is not a real solution. Try laying out a protocol in terms of messages passed, and you'll see the issue.

My attempt at an explanation: At any given point, one general has less information than the other. When I send a message I learn that I've sent a message, but I don't know that they've received the message. When I receive a message, I learn both. If there is any specific threshold of knowledge required to go, one general must pass it first.

</comment></comment></comment></comment><comment><USER19> This brings back memories of learning about TCP

</comment><comment><USER20> could solutions to this be used for settlement purposes I.e bank transfers etc.</comment>


Open Sourcing Peloton, Uber's Unified Resource Scheduler - <AUTHOR> https://eng.uber.com/open-sourcing-peloton/ <comment><USER2> Is it just me or is every big release from Uber just a custom rewrite of an existing technology? It seems their engineering department has a large not- invented-here attitude. I could be wrong - they're certainly large enough to have custom requirements that aren't met with what's on the market but the pattern is just becoming suspect.

<comment><USER3> I think you are right.

I think that what is going there is also a bit political. They started to grow their Engineering department so fast that they need to justify the headcounts now. So each team is trying to invent new projects all the time. Anecdotally, this was partially confirmed to me by a friend working there.

I said this before, but I still cannot understand why a service like Uber need so many engineers in the backend (multiple thousands). It is a complex distributed application, but nowhere near the scale or complexity of a Facebook or Google.

<comment><USER4> >I said this before, but I still cannot understand why a service like Uber need so many engineers in the backend (multiple thousands). It is a complex distributed application, but nowhere near the scale or complexity of a Facebook or Google.

Thank you so much, I thought I was going crazy. I understand the demands of running a service on the level Uber has, but well, for instance I can't imagine what kind of computational workload / infrastructure requirements would make developing your own resource scheduler a reasonable option - for a Taxi app? With non-essential (to the core product) machine learning?

Forgive me if I'm ignorant, but what exactly does Uber engineering team do?

edit: On their blog I was able to find that they namely "forecast rider demand", from a relatively small [0] article - that is, comapred to the article [1] about what essentially is "just" data visualization, which doesn't help my confusion much.

0 - [https://eng.uber.com/neural-networks/](https://eng.uber.com/neural- networks/) 1 - [https://eng.uber.com/maze/](https://eng.uber.com/maze/)

<comment><USER5> Those cars generate a _lot_ of sensor data. (Tb per drive?). Id imagine that data needs to be made actionable and seperated into training and simulation sets pretty quickly. Mapping is a massive problem to automate.

<comment><USER4> Oh right, the self-driving cars. Well that's starting to make sense now.

</comment></comment></comment></comment></comment><comment> a-wu I've worked with Mesos pretty extensively before and when Uber first announced Peloton last year I was intrigued. Peloton seems to be a wrapper around Mesos that allows for running smaller, unique jobs without having to write a Mesos framework for each. Writing a Mesos framework for every small job you have can get annoying when you just want to define how your job should run and don't really care for the resources or task allocation of the job, and it seems like Peloton solves this on Mesos. It's similar to YARN but not limited to Hadoop. It would have been useful for the project that I worked on because it was more geared for our use case and shifting from Mesos to k8s would've been a huge engineering project.

<comment><USER6> Mesosphere created dcos-commons to reduce the friction of getting frameworks up and running.

[1] [https://github.com/mesosphere/dcos- commons](https://github.com/mesosphere/dcos-commons)

</comment></comment><comment><USER7> Just call it (Uber) customized mesos. I find this article somehow deceiving and boring. I am pretty sure I can run this peloton thingy with most Mesos API calls.

<comment> a-wu It pretty much is, but the appeal (at least to me) is abstracting away all the framework writing. It seems like it's easier to run small, unique jobs on it similar to something like YARN.

</comment></comment><comment><USER8> What exactly is Uber running in its clusters?

\- Route Optimization

\- Demand Forecasting

\- Rider Hotspot Prediction

This post doesn't exactly tell us the true nature of their workloads (other than the crude categorization - batch, stateful, stateless), nor does it talk about the inflection points where off-the-shelf solutions don't cut it anymore and such customization is required. I mean some _before & after numbers / graphs_ on resource utilization would have really helped.

</comment><comment><USER9> At first glance I thought the "resources" here might be Uber drivers, which would be interesting.

Is this thing/Apache Mesos abstract enough to allow for such a use?

<comment><USER10> The best thing I can think of that fits what you're asking is an Actor framework (which abstracts the compute and message passing between objects for you).

Think Akka or New Orleans, etc.

</comment></comment><comment><USER11> This sounds a bit like Two Sigma's Cook scheduler for Mesos. [https://github.com/twosigma/Cook](https://github.com/twosigma/Cook)

</comment><comment><USER12> Immediate reaction: Well, now there's Peloton, the fitness tech company, Peloton, the self-driving truck caravan company, and Peloton, the cluster scheduler...

<comment><USER13> Where do all these names come from? Is it Latin or Greek or something?

<comment><USER14> Peloton is a somewhat common French word for "ball" that became common in English as a sports term for a grouping in a bicycle race, by way of the Tour de France. [https://www.merriam- webster.com/dictionary/peloton](https://www.merriam- webster.com/dictionary/peloton)

<comment><USER15> Small detail, that's not what it means in French, it's basically used the same was as in English (a group of cyclist, group of race car drivers, etc.)

It is however pelota, ball, in Spanish.

</comment></comment></comment></comment><comment><USER16> My first reaction was "this just sounds like Mesos?". And it's cited in their page (which on first read I thought meant they were trying to act as a single pane of glass for Mesos/k8s/etc.):

[https://eng.uber.com/peloton/](https://eng.uber.com/peloton/)

In the OP blog post though, they assert "to our knowledge, there is no other open source scheduler which combines all types of workloads for web-scale companies like Uber."

And then, when you dig...it's just Mesos. They built a framework for Mesos. So, that's cool. But _man_ , the puff piecery borders on dishonesty. I mean-- Singularity has existed, and is implemented at very large scales, for a while. I'm sure Peloton is a fine scheduler, but there's a lot of huffing-one's-own- farts in the documentation here.

<comment><USER17> Singularity isn't a scheduler. It's generally invoked by traditional hpc schedulers like moab or slurm.

edit: clearly I'm thinking of a different Singularity than you.

<comment><USER16> Yeah - on a phone but I'm referring to the Mesos framework by HubSpot.

</comment></comment></comment><comment><USER18> You mean not a bike company?!

</comment><comment><USER19> Peloton should make a framework called Uber in response.

<comment><USER20> The term "peloton" obviously predates the company of the same name...

<comment><USER21> The term "uber" obviously predates the company of the same name...

</comment></comment></comment><comment><USER22> Would people use open source stuff from a morally questionable company? Especially when its just a re-write of existing technology posted to a different github repo?

 _EDIT_ People get so up in arms about Google and Microsoft working with China and the military, but Uber has done some horrendous stuff on their own. Just curious where people think the line is OK to be.

<comment><USER23> Using their open source stuff doesn't fund them to do bad things.

<comment><USER24> Arguably it does indirectly - e.g. being able to say that you're responsible for Popular Tool X might help your brand, make it easier to make sales, etc</comment></comment></comment>


MIT's depression-detecting AI might be its scariest creation yet - <AUTHOR> https://thenextweb.com/artificial-intelligence/2018/09/05/mits-depression-detecting-ai-might-be-its-scariest-creation-yet/ <comment><USER2> A demo would have to come with a billion disclaimers, but could be incredible. I kinda need a demo for catharsis reasons...</comment>


A Race of Two Compilers: GraalVM JIT versus HotSpot JIT C2 [video] - <AUTHOR> https://www.youtube.com/watch?v=lunJmMBkqLo <comment><USER2> As a side note, GraalVM is quite usable for real world stuff nowadays. Here's an example of running a Clojure web service with Graal that provides JSON endpoints, talks to the database, and does session management: [https://github.com/yogthos/graal-web-app- example](https://github.com/yogthos/graal-web-app-example)

The same app can be run on the JVM or compiled statically using Graal. The JVM version takes around a 100 megs of RAM, and has a significant startup time. The Graal version weighs in at 7 megs, and starts up instantly.

</comment><comment><USER3> I got quite good results running some Sudoku solving code in GraalVM (FastR). It was faster than an R/Rcpp hybrid:

[http://bootvis.nl/fastr-sudokus/](http://bootvis.nl/fastr-sudokus/)

Later I ran into some errors which are supposed to be fixed in the development branch but I haven't tested.

</comment><comment><USER4> From what I know, GraalVM EE (Enterprise Edition) _does_ do loop vectorisation.

This will lead to an interesting problem if they want to replace C2 with Graal. Are they willing to regress performance for some open-source-only users, even if it's a performance win for others?

<comment><USER5> Contrary to popular believe, automatic loop vectorization is not as important to most Java workloads as one might think. It gets a lot of visibility as it causes significant peak performance differences in micro-benchmarks.

In the end, you should not trust standard benchmarks and definitely not micro- benchmarks. Do perform tests with your own workload.

<comment><USER6> But this is only true because the Hotspot JVM just isn't meant to be used for high performance code. In theory with a good enough JIT, it should be possible to achieve high performance even in Java. One day Java might have value types and then these optimizations will be able to shine.

</comment></comment></comment><comment><USER7> Just in time compilation, I think it should be called "continuous profile guided compilation" instead, describes better the awesomeness that happens...

<comment><USER8> JITs do more than just profile-guided optimizations. Their secret weapon is speculative optimizations that mean they don't need to work hard (and often fail) to prove the soundness of certain optimizations. They're allowed to guess and be wrong.

<comment><USER9> Wait, ignore _soundness_? How does that work/provide advantage?

<comment><USER10> For example a JIT compiler can, based on the runtime information + inference, discard multiple branches of a function that are effectively pointless and possibly even just inline the results if given those inputs the function is deterministic.

Of course, the function it compiled will actually not work with any different but still valid arguments, but that's not really trouble since the JIT compiler will simply evaluate that the already compiled version won't work for those as the function is called and compile a new version of the function for the new types just before. A pure ahead-of-time compiler wouldn't be able to optimize so aggressively since it would lead to an exponential explosion of possible inputs combinations most of which will very likely never happen.

<comment><USER9> I put it in a bit more detail in a sibling comment but to me, there's no 'guessing about soundness and being wrong' in that particular scenario.

<comment><USER10> I guess it depends on the perspective and interpretation of soundness. If a JIT-compiler AoT compiles your entire program but infers that your function that implements logic that works for every number (as you defined it using the Number interface within the rules of it's type system) will only use 32 bits integers, then it will compile code that effectively does not hold up to the property that was established. The fact that it will stop execution the moment it reaches an invalid path and correct it doesn't change that.

<comment><USER9> It could be and that would be uninteresting. But it's not hard to come up with a (contrived, limit-casey) optimization approach that does actually make guesses about soundness.

Let's say you wanted to optimize a short instruction sequence with a small domain of inputs. You could try to generate all (or at least, zillions) of similarly-sized possible instruction sequences and check them for soundness and performance. Now you're really making soundness guesses. Do real JITs actually make that sort of soundness guess (not that kind of attempt at optimization, obviously)?

<comment><USER8> All the time, and BTW, I didn't say JITs sacrifice soundness but that they don't require _proof_ of soundness. That's different as I'll show.

Let me give you two common examples: virtual calls and branches. A JIT will speculatively devirtualize and inline a virtual call at a particular callsite if it has only encountered one or a small number of concrete instances, even if it can't prove that those are the only instances that can be encountered at that callsite. This is still sound because the JIT will emit a trap that will trigger if an unknown target is ever encountered, in which case it will deoptimize the compilation, go back to the interpreter and then compile again under new assumptions. Another example is branch elimination. If a JIT only ever encounters the program taking one side of a branch, it will only compile that branch (and introduce a trap), even if it can't prove that only that side will _ever_ be taken.

<comment><USER9> Thanks! I did (eventually) figure it out, I initially misread it as something like:

1\. Jettison soundness

2\. ???

3\. Performance profit.

Which seems like witchcraft, then again JITs are full of witchcraft. But it's also not what you wrote. I've now come to understand the two chief weapons of the JIT remain surprise, fear, ruthless efficiency and an almost fanatical devotion to the Pope.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> Here is the same speaker comparing C2 and clang: [https://youtu.be/0yrBuPiGk8I](https://youtu.be/0yrBuPiGk8I)

</comment><comment><USER11> What's the current status of Graal's Python implementation in terms of reaching actual usability? The README on its GitHub repo [1] doesn't inspire much confidence but whenever I check that wording hasn't changed.

I've been following Graal for quite some time, both as a former PLDI guy but also for my day job. I work in bioinformatics software (mostly cancer genomics research) and our group has a ton of (mostly legacy) code in Java and R, but most of the newly-minted grads coming in lean towards Python.

As one of the guys pulling all this stuff together, the Graal "polyglot" multilingual VM concept is of tremendous interest as you can imagine. It would be great to be able to package the legacy stuff interoperably with the new stuff no matter the language, even setting aside the bonus of better performance. But it has basically no practical use to us without Python (+ packages!) due to the direction and language inclinations of the group.

Is there anything new happening on that front? Or anything we could do to help it along? Is there more a detailed status page anywhere? Any sense of when this might land in a truly usable form, or what's the hold up?

I'm a bit surprised that the progress with R (with packages) is so far along but the progress with Python (with packages) seems stagnant (at least according to that README). No offense meant to the team, but that's the appearance. Is it the GIL?

[1] [https://github.com/graalvm/graalpython](https://github.com/graalvm/graalpython), which calls it "early-stage experimental" and "very likely that any Python program that requires any packages at all will hit something unsupported".

<comment><USER12> I had decent luck with Nuitka[1] as long as the project is 100% python. The executables are large but have been mostly portable IME (some glib problems can arise though).

Largest project I compiled was only ~1000 lines but used external deps of pymysql, jinja2, ldap3 along with the stdlib's shutil, tempfile, pathlib, and the base os lib without issues. It takes ~30 minutes to compile on a decently powerful machine though (8650u and 32gb of ram). Most of this time was spent on pymysql and jinja2's compilation.

[1] [https://nuitka.net/](https://nuitka.net/)

<comment><USER11> Thanks, but I don't see what that has to do with Graal or multi-language interoperability which is the key thing here. We have substantial code in Java, R and Python that could all benefit from being able to call one another from within the same process.

An alternative Python compiler by itself frankly buys us very little. Perhaps Jython, if it weren't targeting 2.7.x.

</comment></comment></comment><comment><USER13> A bit OOT, has anyone tried Scala on Graal? How's it?

<comment><USER14> It seems pretty good. Twitter has some talks on this where they claim signifiant performance improvements: [https://www.youtube.com/watch?v=PtgKmzgIh4c](https://www.youtube.com/watch?v=PtgKmzgIh4c).

I've heard that the Twitter JVM team has a road show where they've talked to some other large Scala users about the performance improvements. Initially, people are highly skeptical of the claims, but after trying Graal on their internal workloads, they generally see similar results.

Here's a paper which has some explanations for why you might expect Graal to improve Scala performance: [http://aleksandar- prokopec.com/resources/docs/graal-collecti...](http://aleksandar- prokopec.com/resources/docs/graal-collections.pdf)

</comment></comment><comment><USER15> Is it clear what the licensing is/will be for Graal?

Edit: GraalVM CE describes it on the download page [https://www.graalvm.org/downloads/](https://www.graalvm.org/downloads/)

<comment><USER16> they don't really clearly list the differece between the CE and the EE version

what is "Improved performance and smaller footprint"? If i use the CE, does it produce worse code somehow?

</comment></comment><comment><USER17> I'm curious if GraalVM can do the same rockstar party tricks on Java 11? He uses Java 12, but doesn't go into huge detail why he chose that rather than the LTS.

<comment><USER8> 12 is the current JDK version. OpenJDK has no notion of LTS (e.g. you will see no mention of 11 being an LTS on the project page: [https://openjdk.java.net/projects/jdk/11/](https://openjdk.java.net/projects/jdk/11/) nor any special designation compared to JDK 12's page), and all versions are equal.

Java's LTS means something that could perhaps be quite different from LTS in other projects. LTS is a service offered by companies to _arbitrary_ JDK versions of their choice (e.g. Azul offers extended support for versions other than those Oracle does); there is nothing special about the development, testing, effort or focus put into those versions. In addition, people can choose to maintain OpenJDK update projects, as Red Hat does for 11. Anyway, JDK 12 is simply the current JDK version, and there is no reason to use an old version for a technical discussion -- there is nothing more stable about it, or any other technical difference -- even if companies offer extended support for it. (I work on OpenJDK at Oracle)

<comment><USER18> > OpenJDK has no notion of LTS

But two of the main maintainers, Oracle and Red Hat, absolutely do.

> LTS is a service offered by companies to arbitrary JDK versions of their > choice

And that arbitrary version is 11.

It's technically accurate but substantially disingenuous to suggest that 11 is not Java's current LTS.

<comment><USER8> > But two of the main maintainers, Oracle and Red Hat, absolutely do.

BTW, Oracle contributes ~90% and Red Hat ~5%.

> It's technically accurate but substantially disingenuous to suggest that 11 > is not Java's current LTS.

Either way that has little significance here. It is not the most popular version of Java in use today (that would be 8u2XX) nor is it any more production-ready or stable than 12. You can say that we're interested in results for the current version of Java or in the most popular one. I don't understand why it would be particularly interesting to discuss JDK 11, which is neither.

<comment><USER19> I'm really not sure where your confusion is coming from.

The post itself is interesting on two fronts: as a new emerging technology but also as a leverageable tool. As an OpenJDK developer I understand your interests are more about the former.

For many of us are using Oracle or RedHat LTS builds, we are running either 8 or 11 for "reasons". It's pretty natural to know if these new changes to the platform apply to a given version without asking.</comment></comment></comment></comment></comment>


Drake: robotics planning, control and analysis toolbox - <AUTHOR> https://github.com/RobotLocomotion/drake/wiki <comment><USER2> Curious if anyone else is currently taking the class (Underactuated Robotics, Russ Tedrake @MIT), or has taken it previously? Through edX or at MIT? Any comments?

I have not completed it (on edX, and not sure if I will due to constraints on my time) but I think I would recommend it.

Also: Russ says they are running Drake (in MATLAB) directly on their Atlas robot (an android robot from Boston Dynamics, now owned by Google.) As they say, this is borderline crazy but seems to work and allows them to make rapid changes to their control code. If I recall correctly he said their control loop was in excess of 300 cycles per second, is this right?

Anybody out their want to comment on this or chime in with a comparison to another robot control system?

<comment><USER3> I took this class last Fall at MIT and it's really really good. I would recommend it to anyone interested in this topic.

Also it's great that all their code is open source. It's amazing what they can achieve with trajectory optimization.

I was also impressed that they were able to run MATLAB code live. Although to be fair, they are calling efficient commercial solvers implemented in C++ or other languages under the hood.</comment></comment>


Status Report: Liars and Outliers - <AUTHOR> http://www.schneier.com/blog/archives/2011/10/status_report_l.html

<comment><AUTHOR> "It's a language that's more intuitive. We already know about moral pressure, peer pressure, and legal pressure. Reputational pressure, institutional pressure, and security pressure is much less of a stretch. And it puts security back in a more sensible place. Security is a mechanism; trust is the goal."

Repeat that last line: "Security is a mechanism; trust is the goal."

What other mechanisms do you see helping improve social trust?</comment>