Farewell oil, hello tech: the world's 5 most valuable companies in 2006 and 2016 - <AUTHOR> https://www.weforum.org/agenda/2016/09/farewell-oil-hello-tech-the-world-s-5-most-valuable-companies-in-2006-and-2016/ <comment><USER2> Oil never was that valuable. It was propped by a OPEC cartel.

If you go back in time and watch any press release from the saudis when asked about the idea of "peak oil" \- they always said it was nonsense.

It was too easy to earn money just from the oil production capacity in place that they saw no reason to increase it.

The reason why the party was over for the OPEC countries was because of a petroleum engineer from texas who didn't even get rich or famous from completely changing the norms in international trade and global geopolitics [1] even spider man cannot pull something like that off.

The reduction in oil prices is doing wonders throughout the developing world. India for example had to spend huge amount of tax-payer's money to provide support to poor families. In a single year they saved 50 billion dollars [2] ! It might not seem that much, but a dollar in india can get you much further then a dollar anywhere else ( especially if its taken from saudi pockets ) ( Modi for example needs 100 billion dollars in investment to kickstart a process that he thinks will allow india to reach china's level of capacity in manufacturing )

Open economies is good - and tech was always the king, from the invention of the printing press. Its just investors who were dumb to think that paying for a monopoly is a good idea.

[http://www.npr.org/sections/money/2016/08/17/490375230/oil-3...](http://www.npr.org/sections/money/2016/08/17/490375230/oil-3-how- fracking-changed-the-world) [1]

[http://www.news18.com/news/business/global-crude-oil- price-c...](http://www.news18.com/news/business/global-crude-oil-price-crash- who-gains-the-most-in-india-1186520.html) [2]

</comment><comment><USER3> Going back to 1996 shows Intel, IBM, Softy and G.E. among the largest firms...so hello tech, again?

Source: [http://etfdb.com/history-of-the-s- and-p-500/#1996](http://etfdb.com/history-of-the-s-and-p-500/#1996)

Site appears to have market cap data back to the 1980s. Interesting shifts over the years (Philip Morris even appears "in the pack")

I realize the article might be just a WEF staffer fill piece, but it's unfortunate to see the WEF using market cap as the proxy for valuable companies here.

This might get downvoted as too pedantic: but oil and tech companies valuations use different accounting methods. Similar to stock and flow issues - they are ~incommensurable. e.g. depletion of oil reserves (regardless of measurement and spot prices) is value realization and is not an apples to apples comparison to tech companies future growth/innovation value _creation_ assumptions.

</comment><comment><USER4> Oil is certainly 'high tech'. These firms employ literally thousands of engineers.

<comment><AUTHOR> Is the product hi-tech though? If we use the _" Five types of software"_ as a comparison, I'd rate engineers in oil as #2, internal. [0]

[0] [http://www.joelonsoftware.com/articles/FiveWorlds.html](http://www.joelonsoftware.com/articles/FiveWorlds.html)</comment></comment>


Ask HN: What languages and technologies/tools are you using right now? - <AUTHOR> Hi guys, I'm writing you because I would really like to know what languages, libraries and technologies/tools are being used now days.
Why you say? Well, you see, I'm curious. I dont really like those "graph" showing which languages are popular, I want it directly from you!
Thanks ;)
PD: would be nice for comparison with my past question 4 years ago) <comment><AUTHOR> \- PHP 5.6.x

* Composer [https://getcomposer.org/](https://getcomposer.org/) * Zend :( * Codeception testing framework [http://codeception.com/](http://codeception.com/)

\- AngularJs 1.4.x

* composer asset plugin (to avoid bower) [https://github.com/fxpio/composer-asset-plugin](https://github.com/fxpio/composer-asset-plugin) * Shrinkpack [https://github.com/JamieMason/shrinkpack](https://github.com/JamieMason/shrinkpack) (to avoid npm issues) * AngularUI [https://github.com/angular-ui](https://github.com/angular-ui) (Router, Bootstrap) * Bootstrap 2 & 3 * Ckeditor 4.4 * Lodash [https://lodash.com](https://lodash.com) * Moment.js [http://momentjs.com/](http://momentjs.com/) * Numeral.js [http://numeraljs.com/](http://numeraljs.com/)

\- Swagger [http://swagger.io/](http://swagger.io/) \- Git LFS [https://github.com/github/git-lfs](https://github.com/github/git-lfs) \- Algolia \- Gulp :( \- Ruby 2.3 * Rake [https://ruby.github.io/rake/](https://ruby.github.io/rake/) * Guard [https://github.com/guard/guard](https://github.com/guard/guard) * Data Anonymization [https://sunitparekh.github.io/data- anonymization/](https://sunitparekh.github.io/data-anonymization/)

\- MySQL 5.7 (AWS RDS) \- Chef 11 \- AWS Opsworks \- AWS CloudWatch (Logs, Metrics and Alarms) \- Ubuntu 14.04

Look forward to : \- Upgrade to PHP 7.1 \- Upgrade to Chef 12 \- Upgrade to Ubuntu 16.04

</comment><comment><USER2> \- WordPress

\- Node.js

\- MySQL

\- Redis

\- RabbitMQ

\- React.js

\- npm

\- Browserify

Two really important pieces in my toolkit I made (and probably am the only person using them) are:

\- cjs-noticeboard

A javascript pubsub implementation I use client and server-side.

[https://github.com/Akamaozu/cjs-noticeboard](https://github.com/Akamaozu/cjs- noticeboard)

\- cjs-task

Take a task and easily chop it up into smaller distinct steps.

[https://github.com/Akamaozu/cjs-task](https://github.com/Akamaozu/cjs-task)

Been pretty bad at properly explaining why I use these tools i made myself. I'll put in more effort into doing that soon.

</comment><comment><USER3> Question from four years ago: [https://news.ycombinator.com/item?id=4886390](https://news.ycombinator.com/item?id=4886390)

</comment><comment><USER4> \- PHP 7 + WordPress + memcached + sometimes React + sometimes ElasticSearch. Once in a while Gearman for big tasks.

\- node.js. One project testing out AWS Lambda.

\- One project trying out React Native.

\- AWS, Azure, WP Engine, Cloudflare, Ansible, and a bunch of other hosts and technologies I haven't had enough coffee to remember.

Sound interesting? We're always interested in talking to engineers: [https://10up.com](https://10up.com)

</comment><comment><USER5> \- I use Browserify(1) because it allows me to use CommonJS(2) modules and bundle my JavaScript painlessly.

\- Beyond the pretty standard debowerify(3) and hintify(4) transforms I use babelify(5) to help me sprinkle fancy ES6 features here and there.

\- Node-based watchers and the npm client itself are all broken in many ways and very bad Unix citizens so I've stopped using npm for anything beyond actual package management and I will almost certainly replace the npm client with Yarn(6) very soon. The venerable make(7) or the much much younger but very dependable modd(8) are infinitely better than all that Node-based crap on _every_ front.

\- I use tape(9) for tests (works like node-tap(10) with a little fewer dependencies) but I _love_ the look of ospec(11) even if it's not TAP(12)-compliant.

\- Some JS libraries I've used recently: route-parser(13), pikaday(14), vidage(15), etc.

\- I don't use any Sass/SCSS-specific tool or library.

\- I use Vim(16) for editing text and either git itself (with with(17) when I remember I have it) or tig(18) for versioning source code.

[1] [http://browserify.org/](http://browserify.org/) [2] [http://wiki.commonjs.org/wiki/CommonJS](http://wiki.commonjs.org/wiki/CommonJS) [3] [https://github.com/eugeneware/debowerify](https://github.com/eugeneware/debowerify) [4] [https://github.com/ansis/hintify](https://github.com/ansis/hintify) [5] [https://github.com/babel/babelify](https://github.com/babel/babelify) [6] [https://yarnpkg.com/](https://yarnpkg.com/) [7] [https://www.gnu.org/software/make/](https://www.gnu.org/software/make/) [8] [https://github.com/cortesi/modd](https://github.com/cortesi/modd) [9] [https://github.com/substack/tape](https://github.com/substack/tape) [10] [https://github.com/tapjs/node-tap](https://github.com/tapjs/node-tap) [11] [https://github.com/lhorie/mithril.js/tree/rewrite/ospec](https://github.com/lhorie/mithril.js/tree/rewrite/ospec) [12] [https://testanything.org/](https://testanything.org/) [13] [https://github.com/rcs/route-parser](https://github.com/rcs/route-parser) [14] [https://github.com/dbushell/Pikaday](https://github.com/dbushell/Pikaday) [15] [https://github.com/dvLden/Vidage](https://github.com/dvLden/Vidage) [16] [http://www.vim.org/](http://www.vim.org/) [17] [https://github.com/mchav/with](https://github.com/mchav/with) [18] [https://jonas.github.io/tig/](https://jonas.github.io/tig/)</comment>


Stochastic Optimization for x86 Binaries [video] - <AUTHOR> https://www.youtube.com/watch?v=aD9mZDJzb58

<comment><AUTHOR> The part related to floating point optimization is really cool.</comment>


YouTube Defines "Hate Speech" For Purposes Of Ad Revenue Eligibility - <AUTHOR> https://arstechnica.com/business/2017/06/what-is-hate-speech-on-youtube-video-site-offers-clarity/ <comment><AUTHOR> The definition of "Hateful Content" (one of the three categories of "Hate Speech" that YouTube defines) strikes me as quite broad. I wonder how this will be applied in the following circumstances:

1) Disparagement of majority-race individual for being insensitive or lacking understanding of racial issues. For example: "celebrity X needs to check his white privilege. He clearly doesn't understand the struggles of black and brown people and his opinion on police shootings is therefore irrelevant"

2) Disparagement of majority-religion individuals for their beliefs around LGBTQ issues. For example: "We should boycott Chick Fil A because the owner is a homophobic right-wing bible-thumper who donates his profits to anti-LGBTQ organizations."

3: Disparagement based on age. For example: "The voting age should not be lowered because young people lack life experience necessary to make informed decisions"

Each of these statements disparages a person or group of people based on one of YouTube's protected classes. But many people would argue that they are very different from statements like "America has too many Muslims" or "Mexicans are lazy." Would YouTube treat these statements the same? According to their policy, would they have grounds for treating them differently?

<comment><USER2> > Each of these statements disparages a person or group of people based on one > of YouTube's protected classes

That's dubious. The third isn't reasonably describable as disparagement, and the first two are doubtful; more importantly, perhaps, the description of the categories admits of at least two plausible different readings, one of which (in which the entire list is modified by "associated with systematic discrimination") would seem to _not_ cover socially-dominant groups (such as those in the first two examples) as protected at all.

> But many people would argue that they are very different from statements > like "America has too many Muslims" or "Mexicans are lazy."

Well, yes, even aside from the target feature, they are self-evident structurally different.

> According to their policy, would they have grounds for treating them > differently?

Almost certainly. If you want to compare analogous statements targeting different features, you need to start with analogous statements.

<comment><AUTHOR> Interesting perspective. The statement about youths is certainly borderline in terms of disparagement — it's designed to show that there's a fine line between descriptiveness and disparagement in some cases. It is an indisputable fact that young people have less life experience than older people; it is a logical leap to say that younger people ought not be able to vote because of this.

The last two statements, which target ethnic/religious minorities, were structured differently to be crisp — but I think that if you create analogs to 1-3 using these groups, the result would be the same:

1b) "Black celebrity X shouldn't try to compare her life experience to that of poor urban whites — her opinions are uninformed and irrelevant."

2b) "Hijabs create distractions in an education environment and should not be worn in public schools" (as is the case in France)

3b) "Old people are more likely to have dementia and should have to take a cognitive test before allowed to vote"

These are structured like the original 1-3, and I think each would elicit a very negative reaction from many people (including those who work at YouTube). Do you think the policy offers a rationale for treating 1b-3b differently from 1-3?

<comment><USER2> > Do you think the policy offers a rationale for treating 1b-3b differently > from 1-3?

Potentially (assuming, _arguendo_ , that all are viewed as being disparaging, which I still think is dubious in all 6 cases—if they aren't, then there is no basis for different treatment and none are actually problematic under the policy), and I already explained the reading that the policy language admits that supports this in the grandparent post.

<comment><AUTHOR> I guess we have different senses of what is considered disparaging. Appreciate seeing your perspective!

</comment></comment></comment></comment></comment><comment><USER3> And The Internet turning into Cable TV marches on.

The real TV is for old people to watch soaps, but we finally cracked how to take the raw and unfiltered branding of the Internet and constrain the content into our boring safe-for-advertizer risqué.

People should never stumble upon things that might upset them. People should watch our boring, artificial representation of mundane acts, and feel that they're outrageous because we presented it as such.</comment>


Five-Dimensional Data Storage - <AUTHOR> http://www.technologyreview.com/computing/22673/ <comment><USER2> Very cool, though I don't like the misuse of technical terms just to sound more important. It's not actually fully 5-dimensional... each 'dimension' has only 2 or 3 states. The first 2 dimensions are limited by the size of your media. The third dimension is based on your laser focal plane resolution (think 'depth perception' in a way) and is currently limited to two layers in optical discs. The there's the color (in this example, 3 colors) and polarization (here, 2 orientations). So it's 12 times more dense than current mainstream technology, with the added benefit of a tighter placement of data on in the plane.

The disruptive technology here is really the use of nanoparticle gold instead of etched polystyrene, and the fact that there is the polarization element. Unfortunately these points are kind of glossed over and lost in the fancy media mumbo-jumbo.</comment>


Evidence that Google skews search results - <AUTHOR> http://www.wired.com/2015/06/google-wu-study/

<comment><USER2> _> The team then randomly displayed one of the two sets of results to 2,690 subjects. What they found was that users were 45-percent more likely to actually click through the search results on the second version, which included only organic results. What that means, they write, is that by prizing OneBox over organic links Google is serving up less useful search results, which is therefore damaging to the end user._

Or that users obtained the information they needed from OneBox and, therefore, were not required to click on a link - making the search results _more_ useful. Search should be defined as "obtained the information I was searching for" and not "clicked through to a website."

Counterpoint example:

Searching google for "define 'word'" gives me the definition of the word in OneBox without me having to click to dictionary.com or merriam-webster or any other similar online dictionary. The result? A better search result that doesn't require an additional click-through!

E:

 _> The problem with all this is that, generally, even though they are the ones being harmed, consumers don't care all that much—at least not enough to change their habits._

Am I the only one to find fault with this claim? If users don't give a shit, has it ever occurred to these people that maybe, just maybe, users aren't being harmed and therefore... don't give a shit? If Google didn't provide the information that people wanted when they searched for something, they would do this amazing thing called _use another search engine_.</comment>


Is TDD dead? - <AUTHOR> http://martinfowler.com/articles/is-tdd-dead/# <comment><USER2> TDD is not dead, but hopefully the practise of foisting rigid, prescriptive, quasi-religious software development methodologies on creative individuals is.

TDD, like all of these things - eXtreme Programming, agile, has good points and bad. Listen to what these people have to say, and then take what works best for you.

To me, TDD is about writing my code and my tests at the same time, not test first. As soon as I write just enough code to do something, I'll write a test to see if it actually works. I'll then expand on that test and perhaps add a few more as I refactor that piece of code.

This means I still get the benefits of exploratory programming, unconstrained by test first, but I also get the security of knowing that my changes didn't break anything. It feels very natural to develop this way, as the code base grows, your confidence in its quality grows with it, rather than the inverse (lots of passing tests). Also a fast test suite which runs on file save is _addictive_ , I cannot imagine returning to F5 Driven Development now.

<comment><USER3> I can't state how much I dislike the quasi-religious part of TDD. Trying to establish a logical argument of trade-offs with someone who "believes" in TDD is from difficult to impossible.

The arguments always end up in line with "You just have to be better/be more confident/etc etc", or in more blunt term, "I'm better programmer than you, that's why TDD works for me but not for you".

</comment></comment><comment><USER4> The embedded debate about "hexagonal" architecture is unfortunate.

I think one of the very common sticking points about refactoring and testability is that when trying to demonstrate techniques, you have to pick simple examples to isolate the techniques.

When in reality, those simple examples don't _need_ those levels of abstraction or indirection. DHH's gist is a perfect example of demonstrating something that - by itself - doesn't need to be refactored in that manner.

But it completely misses the point. Those refactoring techniques are essential for when your codebase is 100 or 1000 times more complex. (Jim W. mentions this explicitly in his youtube video.) But the problem is you'd never show that complex a codebase in a demonstration, because the mental load in understanding the prerequisites (before getting to the actual refactoring) is too large.

I think another common sticking point is that people talk about refactoring to make a codebase more testable. And that is a worthy aim and all, but making it more testable isn't the end goal. Making it more testable forces you to separate your concerns/layers, and _that 's_ the goal. The reason that is the goal is because as your code grows, you still want the product to be easily changeable so it can be quickly responsive to customer's needs. And if you need to switch out your data repository implementation on short notice due to your site being suddenly popular, that'll be easier/faster if that implementation is separated out.

<comment><USER5> > Making it more testable forces you to separate your concerns/layers, and > that's the goal.

This exactly, I've made this exact point many times over the years. testing is never the goal.

</comment></comment><comment><USER6> I really enjoyed Gary Bernhardt's response to this discussion: [https://www.destroyallsoftware.com/blog/2014/test- isolation-...](https://www.destroyallsoftware.com/blog/2014/test-isolation-is- about-avoiding-mocks)

You should read the whole article, but to motivate:

> This post was triggered by Kent's comment about triply-nested mocks. I doubt > that he intended to claim that mocking three levels deep is inherent to, or > even common in, isolated testing. However, many others have proposed exactly > that straw man argument. That argument misrepresents isolated testing to > discredit it; it presents deep mocks, which are to be avoided in isolated > testing, as being fundamental to it; it's fallacious. It's at the root of > the claim that mocking inherently makes tests fragile and refactoring > difficult. That's very true of deep mocks, but not very true of mock-based > isolation done well, and certainly isn't true of isolation done without > mocks.

> In a very real sense, isolated testing done test-first exposes design > mistakes before they're made. It translates coupling distributed throughout > the module into mock setup centralized in the test, and it does that before > the coupling is even written down. With practice, that preemptive design > feedback becomes internalized in the programmer, granting some of the > benefit even when not writing tests.

</comment><comment><USER7> Inside DHH's arguments is also a general argument against what I call frameworkitis - the desire to overgeneralize an application in the vague hope that the code will get used in a different way in the future. In my middle career I was prone to that but in the last 5 years or so I've been trying to resist that impulse. I now try to write the application and if another similar application comes around I might pull out reusable portions. On the whole this approach has been a win for me.

Not that Beck or Fowler are advocating anything different but I think DHH is the one with his finger on the pulse of current development trends.

<comment><USER8> I'm a full YAGNI convert as well. And I've developed a strong belief that some amount of duplication is almost always better than the _wrong_ abstraction.

<comment><USER9> _some amount of duplication is almost always preferred to the wrong abstraction_

This. This concisely expresses something I've learnt in the last few years but have struggled to write down. Generalising is great until you generalise in the wrong direction, then it creates extra complexity in all the other directions.

<comment><USER5> I agree, see my reply to the post you were replying to.

</comment></comment></comment></comment><comment><USER10> It seems like there is a general realization is that TDD is useful for increasingly narrow contexts to create business logic where there already exists a generally accepted architecture.

In other words, the controllers of typical web apps which AFAICT is where the technique originated and the source for a lot of the introductory examples.

It's not a silver bullet, but what is?

<comment><USER11> > It seems like there is a general realization is that TDD is useful for > increasingly narrow contexts

There is? I haven't seen this "debate" take place anywhere other than the Rails community. I work on open-source and I can't imagine releasing a library where its APIs are not unit-tested. That's a different context than the one you mention. And I don't think I'm alone, every library I run into has automated testing.

<comment><USER12> Unit testing does not equal TDD.

<comment><USER13> Sure, but I usually write tests when I have a good confidence in the API of my class, even if that API isn't implemented. I tend to have more biases and assumptions if I write the tests later, so I find this works better for me. So, unit testing is often a subset of TDD.

</comment></comment></comment></comment><comment><AUTHOR> Some interesting conversations between DHH, Kent Beck and Martin Fowler.

DHH can be quite outspoken at times but I think he's right to have this discussion and to have it in public so we can all learn something.

<comment><USER7> I agree, I was negatively biased against him based on what I'd read in the past but he comes across as the one with the interesting ideas and passion around them. Beck and Fowler deserve respect but I think DHH is making a very valid point.

</comment></comment><comment><USER4> I'm interested in how Martin and Ken say they don't actually use a lot of mocking.

Let's say you have production code where A will call B, and B will call C. Basic layer separation. It might be a controller calling a service method, which calls a dao.

If you want to test B, then in my mind, that means you want to write a test that would call B (similar to how A does). And then since you don't want C confusing things, you'd create a mock of C (returning a canned result), and assign it to B.

So... how would you test B without mocking C? Are Martin and Ken simply content to let B call C, which would imply that it is an integration test rather than a unit test? Or is there some other design technique they're using?

<comment><USER5> When you test with mocks, most of the time you're not testing reality, you're testing your theory of reality.

Think of it like this: How many times have you deployed an app in production, or against production data, only to have it crash and burn because something somewhere in the data wasn't what you expected? No one would really be surprised by it happening, I think most of us have experienced it a time or two in our careers.

Here's the thing: No seasoned developer is going to recommend that you write a migration process, create a DB with purely dev created test data, and if it passes, immediately pass it into production/into the hands of customers. Most experienced dev's would tell you that's dangerous, and you really need to test it against actual production data first.

In this case, the dev created test data is a 'mock', and the production data is 'reality'. In this context, no one would disagree that the dev created test data is not enough in order to ensure proper functioning.

But many people will tell you that a mock object is enough to ensure proper functioning. This is not true, and the investment required in order to make it true tends to be a lot higher than people expect for the same reason that battle tested code tends to be more complicated than most people would expect is necessary to solve the problem.

That doesn't mean that mocking doesn't bring value, but because the cost of making that mock accurately mimic reality, it means you should reserve it for those times where the alternatives tend to be even more prohibitive.

This is why you'll see a lot of recommendations for using mocks to do things like simulate failed connections at specific points in a protocol handshake. Getting the connection to break at the correct time using an actual network is prohibitively difficult. Getting a mock to do it consistently at the right time is a lot easier to do.

These are the sorts of situations that mocks were originally created to solve. The issue is that somewhere along the line people started abusing mocks in situations that did not warrant them either because the cost of actually getting it correct is too high, or because they didn't actually pay the cost resulting in a false sense of security.

I don't think any reasonable developer will tell you that mocks are worthless, but I also don't think any reasonable developer will tell you that you should be mocking a whole lot.

<comment><USER4> Well, you're more explaining why not to mock, rather than what they mean by not mocking... but I have a hard time agreeing with this, for properly designed code. Referring to my earlier example of B calling C, what if B's business logic is complex, but makes no outbound calls except to C, and C (for simplicity's sake) is also complex, but returns only a boolean result?

Mocking has real value there because you can control what C returns when testing B. You might write tests for if C returns true, if C returns false, if C returns null, or if C returns some general exception. There, you're testing that B behaves correctly, given all possible responses from C. At that point, you've controlled the interface between B and C, and limited the ways that C can respond. Because of that, you can be sure that if your test for B fails, it's because of a problem with B.

If you instead don't mock C, then an integration test that tests both B and C (since B is actually calling C) becomes a lot more complex - probably more than twice as hard to nail down a bug.

The trick there is that by focusing on the unit test, you've written a clear, simple interface between B and C, when you might not have otherwise. I think that's an example of how mocking can really help, and how it doesn't lead to the nightmare of mocks-calling-mocks that a lot of people seem to criticize. And rather than being only useful in cases like protocol handshakes, it seems frequently useful, such as in wanting to mock a dao layer when testing a business logic layer.

At any rate, that makes it sound like by not mocking, they're merely doing multi-level integration tests. I just would have expected them to be more focused on unit tests, which to my mind requires mocking if you have a multi- layer application.

Jim W. made the same point about "not mocking" in his youtube video though... and in the same breath said that he _does_ , however, mock at the boundaries. Which makes me wonder if this is all just a terminology confusion. Because for instance, if I'm testing a public method in business logic, I'm not going to mock the private methods that it calls in that same business logic layer - that just seems crazy to me. And yet I would still call that a unit test, since it's still limited to one layer, if not to one single-layer-of- abstraction method. I will mock the dao layer in those cases, though, because that strikes me as a completely separate layer/concern. But I don't know, maybe someone else would say that I "don't mock all that often".

<comment><USER5> I didn't post that to give you something concrete to attack, I posted it because you claimed to not understand what they meant when they said they don't mock that often.

I will respond to 1 point you made because I feel it's the point you're not understanding.

> Referring to my earlier example of B calling C, what if B's business logic > is complex, but makes no outbound calls except to C, and C (for simplicity's > sake) is also complex, but returns only a boolean result?

Theory and practice. In theory, the only thing you need to rely on is the return value. In practice, that code also relies on the runtime characteristics of the code it's calling, and this includes things like unexpected slowness, as well as unexpected exceptions (depending on the technology you're working in).

This is the entire point about a mock being a theory of reality, and getting that right is difficult.

</comment></comment></comment></comment><comment><USER14> In my opinion,this debate has a lot to do with the language used,the tools ,the IDE...

Java/C#/... can be heavily assisted with the right IDE,(test suit generation,mock/stub generation...).So the cost of testing can be light,no matter how big the codebase is.

In JS,or ruby.One actually has to write everything. Therefore,one would be less tempted to write heavily decoupled code with a huge graph of function constructors/classes. While ruby has a lot of sugar for a wide range of things,that's not javascript case.

So in my opinion,the cost of testing and managing test suits is higher in languages such as javascript or ruby,therefore one is tempted to write as little code as possible,and sacrifice decoupling and sometime testability.

In one hangout,DHH comes with a snippet of code,highly decoupled and testable.But does ruby encourage this kind of architecture?because it's more code to test and more testsuit to manage(and more tests to rewrite when refactoring).

So isnt it more a language /tooling question than a TDD/mock not TDD/mock question?

</comment><comment><USER15> I'd really like to hear what Hansson, Fowler, and Beck have to say, but not so much that I'm willing to sit through videos. The summaries are good (if you click the "more..." links) but I'd really like to read transcripts. Are there transcripts?

</comment><comment><USER16> I once talked to someone from GitHub who said that the way he sells people on TDD is to show them the tests running automatically when they save a file. I thought this was a pretty neat idea, and have used it on a couple of projects. If you write Go for example, you can run fswatch . "clear && go test" in your project's folder and get this result.

<comment><USER17> "TDD" is not "lots of little tests running all the time". You can (and I try to) have the latter without doing the former.

</comment></comment><comment><USER18> "TDD creates bad architecture".

What is good architecture? I subscribe to the idea that we are doing software engineering and as such there are some "generally" understood principles such as SOLID, the Law of Demeter, Cyclomatic Complexity, etc that provide objective measures of "good" architecture (I apply SOLID all the way up the architecture hierarchy not just on classes).

What I've noticed is that TDD results in code that scores well against these measures, while code that scores well is easy to test (i.e. after writing the code).

Therefore, I think the argument that TDD creates bad architecture is false.

About 25 minutes into the talk we get to the crux of DHH's complaint and it is that Hexagon is an alternative to the Active Record (which he created) and the only reason Hexagon exists is to allow TDD. Hexagon requires throwing away the really useful code that is Active Record.

Hexagon appears to be an attempt to introduce sound software engineering practices (SOLID etc) into the Ruby world (with what success I do not know). Active Record and Rails in general is really useful if what you want is what it does, but sometimes its not. The implied claim that Hexagon is a bad architecture is false. The claim that Hexagon only exists to facilitate TDD is false.

"Mocks returning mocks returning mocks"

I use mocks. Fowler and Beck said on the whole they don't use them which genuinely surprised me. They cited examples of code where the test actually enforced implementation rather than purpose. I think that's probably how I wrote tests for the first few years. Code that results in mocks returning mocks returning mocks is code that is violating the Law of Demeter. Its bad code. It happens to be really hard to test, and it happens to be _really_ hard to write tests first that way. Universally I've only ever seen tests like that when the tests were written after the code. TDD doesn't produce code like that because its easier to refactor it rather than keep digging that hole.

Mocks returning mocks returning mocks is a symptom of _not_ doing TDD.

"My mind works differently... I have to write code first"

Spike. Problems that I don't know how to solve I spike first (I write code with no tests, or with tests only as drivers of execution). That's easy. The hard question is, "Now I have all this code, I have to throw it away and TDD it?" That's pretty hard to stick to in a business environment. I choose to write tests-after for all those pieces of code that _already_ meet SOLID metrics, and rewrite the code (using TDD) for the pieces that don't. The pieces that don't are very difficult to write tests for after, and they also happen to be the pieces where I find bugs (for example, I'll cut and paste a bit of logic and find its wrong for one set of inputs).

"All code should have full coverage of automated tests"

All three agreed that this is the case. Fowler: "If you have a full suite of tests I don't care how you got it [TDD or not]". I don't know about you but I'm still fighting this battle. I also have to deal with teams that have a "full suite of tests" and 80% test coverage, but where every single one of those tests simply executes code. No actual "test" occurs. Indeed, in particularly memorable test, I managed to delete 70% of the lines of code and all the tests passed (including deleting the one line that was the main purpose of the method). Approximately 90% of all the tests were complete garbage: they reported success as long as the code didn't throw an exception.

<comment><USER5> > I subscribe to the idea that we are doing software engineering and as such > there are some "generally" understood principles such as SOLID, the Law of > Demeter, Cyclomatic Complexity, etc that provide objective measures of > "good" architecture

Cyclomatic Complexity is not a principle, it's a specific measurement of _potential_ complexity. Law of Demeter is a _guideline_ SOLID is an object oriented-centric set of principles.

These are 3 _completely_ different things.

You then go on to state using these makes software more testable. Since TDD can only exist in "testable" code, TDD is good design because these measurements/guidelines/principles are good design.

It's a non-sequitur of epic proportions.

I guaran-goddamned-tee you I can write a piece of software with less cyclomatic complexity than the Linux kernel, that is not nearly as well designed. Conflating those two things is what it means to be PHB.

CC is simply a measurement of a specific type of complexity, and like all measurements, it means jack shit without context. For example. the measurement of 8 inches. Is that number high, low, or normal? Or put another way, are we measuring a mans penis, a mans leg, or a mans hand?

 _this_ is why so much software turns to shit. This right here. The process by which you make decisions. If it's a good process, you'll have a tendency to make good decisions. If not, as illustrated by the post above, you will have a tendency to make bad decisions. A series of mostly good decisions will result in acceptable to good software. And the opposite results in unacceptable to bad software.

You want to learn how to write good, stable software? Learn to examine your thought process, and be explicit in your attempt to make good decisions. Be willing, and able, to identify bad decisions, _why_ they're bad, and what you should have done instead.

What you did was believe the conclusion (TDD is good design) and then constructed the argument for it. DO NOT DO THIS. This is the stuff of bad decisions and software design failures, and this will happen regardless of which process you subscribe to.

<comment><USER19> I'm kind of shocked by how hostile your reply is! Not even 'passionate' \- it's condescending, presumptuous, and isn't especially constructive!

The only part of this post that I think really deserves a good response is

 _You then go on to state using these makes software more testable. Since TDD can only exist in "testable" code, TDD is good design because these measurements/guidelines/principles are good design. It's a non-sequitur of epic proportions._

That's not really what he said. He was saying that essentially there is a sort of design isomorphism - like a mathematical dual - between code test-ability and other desirable properties like compose-ability, low coupling, re- usability, and maintainability. The argument in favor of TDD says that building code out of a TDD process that requires testing means making design decisions that have this nice property of building highly compose-able code.

<comment><USER5> hostile? or blunt with a bit of dramatic flair. You'll take your pick based upon whether or not you agree with my position, that has nothing to do with me.

> That's not really what he said.

It is what he said, here's his wording, verbatim

_What I've noticed is that TDD results in code that scores well against these measures, while code that scores well is easy to test (i.e. after writing the code). Therefore, I think the argument that TDD creates bad architecture is false._

You're interpreting things that were not said.

The entire argument is a non-sequitur. It doesn't even matter if you agree with him, an honest evaluation of what he stated would result in the conclusion that the conclusion absolutely does not follow from the argument.

The only thing that can consistently produce good design is good decision making. CC doesn't tell you anything without context, SOLID can be misapplied and is very OO-centric, and the Law of Demeter isn't even a principle, it's a guideline. It's like saying 'prefer composition over inheritance unless it's a clear win'. Ok great, but that doesn't actually result in a good design, it's cautionary guidance on what tends to be the better decision.

 _none_ of these things necessarily result in good design, and none of these things are _required_ for good design. Hell, even the idea of 'good design' is nebulous and changes from 1 project to the next, and over time w/i the same project. Good design in a mobile app where energy is of the utmost importance is not the same as good design in a scientific application where correctness and verifiability are of the utmost importance.

The problem here is that you have yet another person coming to a conclusion and then working backwards in order to justify the conclusion. This is the sort of thing that consistently results in bad design __regardless of how many acronyms you follow_.

There is absolutely nothing in those ideas that intrinsically results in good design, or even intrinsically avoids bad design. That too was a part of DHH's point, one that a lot of people seemed to miss.

The conclusion does not follow because the conclusion came first. You mischaracterizing me as angry doesn't change that, but it is another indication of a flawed thought process (that the validity of the argument somehow stems from me being angry or not). Which brings us full circle back to the sort of thought process required for good decisions.

</comment></comment></comment></comment><comment><USER20> This is a strange debate these days. For me TDD made me a much better programmer. I'm not sure how I used to work without it. It saved me on many occasions. It does lead to better quality (from my experience). Frankly I don't see a single downside. Seriously.

Is it hard ? Sure.

<comment><USER21> The most significant downside is opportunity cost; all code is eventually throwaway code, and if you are doubling your authoring burden by writing code and tests on a throwaway prototype, you are iterating half as fast as you could be.

But in my personal experience, most programmers err _heavily_ on the side of failing to test thoroughly something that will become a long-term solution, not on the side of over-testing their throwaways.

<comment><USER22> It sounds like you write your tests based on the code you write rather than its expected behaviour.

You can write a good set of tests when you know the initial requirements, and not all requirements iterate at the same rate as the code. As you learn new requirements, you write new tests. The old ones don't go away (nor do the early requirements if they're good), even if you completely replace the implementing code.

<comment><USER21> > You can write a good set of tests when you know the initial requirements

Throwaway prototypes are part of the requirement discovery process.

</comment></comment></comment></comment><comment><USER23> Coming from the furthest thing considered open-source - all we have are TDD evangelists - we have 0 implementations. The one die hard TDD I've seen is so unproductive it's painful to the business. Is this just something that's enjoyable to kick around?

</comment><comment><USER24> I don't know that gist just seems contrived. I'm totally not a rails guy so I have to ask: is that the only way you can implement that in rails land?

In (most)JS and/or python I would just monkey patch the save method and/or constructor on the employee object to get the isolation and branch coverage.

I would probably need to do some of the same crazy abstraction for Java though.

<comment><USER25> Nothing crazy required public EmployeeController(EmployeeMapper mapper){}

public Employee create() { ... mapper.save(employee); return employee; }

Done.

All you do now is create a mock employee mapper in your tests, and you can verify just what you want in the Controller instead of anything else

You've got * Single responsibility classes (Controller manages employee creation business logic; Mapper is a dumb database wrapper; If you want to go even further, use a Repository instead of a Mapper to introduce one more layer of abstraction between the data model, that way you can slide in caching or in-memory replacements, or what have you at will) * Clean dependencies. There's nothing hidden about what this class needs to get it's work done

<comment><USER24> So the "damaged" class is not really damaged. It's what you'll wind up with if you started with a small rails project and you had to scale it as it got popular? hmmm.

</comment></comment></comment><comment><USER26> Much like Agile, there's good concepts in there but it seemed particularly prone to people taking things to far too far. (maybe you could even use the word "too extreme")

Guice is a representative programmatic example. Reasonable idea but once you trickle it into your code, it becomes a cancer quite quickly.

</comment><comment><USER27> I clicked on this article thinking it was about Telecommunications Devices for the Deaf, wondering what the alternatives are now that installed payphones with TDD consoles are not to be seen anywhere. I must look into the smart- and feature-phone alternatives.

</comment><comment><USER28> Nope. I know people who are still using it.

I believe it comes down to the team. Who is on it, how the team works together. If there are a lot of people on the team who enjoy working that way, they'll use it. If not, they don't. It lives on.

</comment><comment><USER29> If TDD is the best thing on this planet, why have we to discuss about it? Why did not everybody use it?

Answer: obviously, it is not the best thing on this planet.

Like "use the right tool to solve a problem", use TDD if it solves a problem.

<comment><USER30> "If TDD is the best thing on this planet, why have we to discuss about it? Why did not everybody use it?"

That's a pretty easy logical fallacy. For instance, I've encountered developers who don't use version control. That doesn't imply that you shouldn't use version control because everyone doesn't.

<comment><USER31> if you use an IDE with local history support like PyCharms or Webstorm, you can view every single save and every line of code you changed. It works just like git diff.

<comment><USER32> Now what happens when you are working with one other person using only local history support? What about ten people? What if someone is a bozo programmer and screws up your code accidentally?

<comment><USER33> Do not generalize here, what if he is working alone?

</comment></comment></comment></comment></comment><comment><USER34> Don't miss this part of the video. [http://www.youtube.com/watch?v=JoTB2mcjU7w&t=29m20s](http://www.youtube.com/watch?v=JoTB2mcjU7w&t=29m20s)

</comment><comment><USER35> "living" vs. "dead" seems like the wrong dichotomy to me.

How about dialing back the religiosity and ask "is TDD useful?"

</comment><comment><USER36> One reason Ruby makes it needlessly difficult to do do great software architecture and sane TDD is the lack of contracts or strong parameters.

It is easy to take for granted that when you have a function like this:

function foo(int x, string y) { int count = x * 100; print y + " " \+ count.to_string(); }

...that if you send in a string for x or an int for y the function will blow up. And if you are using a compiled language, it will yell at you before you even attempt to run the code.

In ruby the equivalent code would run just fine, even with generally bad input and it is totally up to you (the programmer) to police yourself to write code that doesn't send across bad data.

A ruby programmer (and probably the same goes for PHP and Python), will end up writing a fair amount of tests that simply ensure that the boundaries are enforced properly and sanely in the code.

There is a lot of love about ruby, but if you are going to write safe, dependable, bug free code it is up to the programmer to enforce boundaries, do validations, and so on and so forth and then write tests that validate that they are still correct over time.

Part of the reason this argument isn't so much happening in other languages is in part that there is maybe not as strong of a culture of TDD in other languages, but also simply because some of the problems don't exist in Java, Scala, C#, etc. because of a type checking compiler and stronger boundaries between components.

A lot of the things that we consider "good architecture" are simply the way we name things, the way we put files in folders, and where we place boundaries in our code. We could write everything as a single file that ran sequentially and jumped to different locations (which is largely what happens anyway), but to make things reasonable and understandable we use patterns like MVC, or composition based functional programming, or MVVM, or various OOP patterns, and so on to express program logic and create logical, sensible boundaries in our code.

If you see the world through the lens of naming things, file structure, and conceptual boundaries, you will see this argument about TDD is more about how strong the boundaries in our code should be and where they should live, not whether TDD as a tool is a good thing or a bad thing.

Rails MVC sees the world as 3 tightly coupled things - model, view, controller. And in that tightly coupled world(along with weak ruby boundaries), TDD is painful and probably not worth your time. DHH is right about that 100%.

If you don't structure your project as Rails MVC does, TDD can be a very pleasant way to build your project and have confidence as you change it over time. Ruby is not the best tool for that job, but it can be made to work. In general, the weaknesses of ruby's boundaries will mean this argument will keep coming up over time and will never be solved.

</comment><comment><USER37> TDD is good. I think "Test-First" is dead. Few places I've been at have been able to do test-first.

Small groups trying to deliver on aggressive schedules waste too much time with test-first.

Having a competent leader and skilled team that can follow SOLID principles removes the need to be test-first.

TDD is not dead. Test-first probably is.

<comment><USER38> Aren't those both the same thing? (Not being sarcastic; I'm genuinely confused.)

[http://stackoverflow.com/questions/3192090/test-driven- devel...](http://stackoverflow.com/questions/3192090/test-driven-development- vs-test-first-development)

<comment><USER39> The way I was taught is that TDD is writing a test, making it pass, repeat - the red-green-refactor cycle. Test first could include "write 200 tests that all fail, then write all the code to make them pass". Somewhat substantiated by this answer: [http://stackoverflow.com/questions/334779/is-there-a- differe...](http://stackoverflow.com/questions/334779/is-there-a-difference- between-tdd-and-test-first-development-or-test-first-prog)

</comment></comment></comment><comment><USER40> Is TDD dead? No, it just smells that way. Like all cargo cults, only the clueless follow it religiously. You can spot them a mile off - with a book they bought off of pragprog, their whole test before breakfast mantra, and usually by their crappy brittle test forged code.

</comment><comment><USER41> In my experience, if you are a competent TDD practitioner, you have a leg-up on those who are not. Sometimes a big leg up.

Yeah, it takes a long time to become competent.

</comment><comment><USER42> Betteridge says No</comment>


Closures, Objects, and the Fauna of the Heap - <AUTHOR> http://duartes.org/gustavo/blog/post/closures-objects-heap/ <comment><USER2> Good article, but the depiction of stack vs. heap as "the price is you must now remember to free() memory or take a performance hit on a garbage collector [...] stack and heap: performance vs. flexibility." isn't quite so cut-and- dry. You can always grab large chunks of memory and perform allocation out of it yourself with decent performance and some determinism or pre-allocate space on the stack before calling a function to allow it to pass back a limited amount of dynamic data. Memory allocators are designed for good performance in most cases, but given a more specific case, you can usually do something better yourself :-)</comment>


The Noteblock – Roses will wither, but the Blockchain is forever - <AUTHOR> http://www.thenoteblock.com/ <comment><USER2> I wonder what happens when someone adds copyright-infringing material into the blockchain. Seems like governments would be all too ready to consider all Bitcoin users to be pirates, and they tend to be less understanding of the concept of something being unremovable from the Internet.

<comment><USER3> Umm, good point. I don't have an answer for that. But considering it will cost you about 60c to store a 100 byte payload, I am not sure what copyright infringment material you can put in there. Probably none.

So for now, I will say I don't feel like this is of a concern

<comment><USER4> Many illegal numbers can be stored for a trivial amount of money.

[http://en.wikipedia.org/wiki/Illegal_number](http://en.wikipedia.org/wiki/Illegal_number)

</comment></comment></comment><comment><USER5> Isn't this just needless blockchain spam? Seems like the blockchain is getting huge enough without spammy messaging transactions like these.

<comment><USER3> Well, This was inspired by the upcoming v0.9 release which makes OP_RETURN (allows you to create immediately prunable output to attach message bytes) a standard transaction output in Bitcoin. And when does get released we will move to use OP_RETURN instead of creating messaging outputs.

So back to your question, do I think this is spammy? No, because this project will never generate enough transactions to matter in the slightest bit to the size of the blockchain. However, I feel the project is important to demonstrate a point which is this:

Bitcoin as a programmable currency protocol has enormous potentials beyond just sending transactions. And OP_RETURN (when it does become standard) has important implications for smart contracts, using Bitcoin transactions to prove / transfer ownership. The objective of this project is to raise awareness in that area. And make developers realise the potential in a fun way on Valentine's Day.

At most, I would say using hash160 output to attach messages is not an elegant way to do this compares when OP_RETURN does become standard. But I believe this awareness that we generate is far more important to Bitcoin development.

<comment><USER5> It seems dangerous to encourage people to use the Bitcoin blockchain for messaging at all as even if this implementation has little traffic, future work may only encourage more messaging.

It seems if people want to use blockchain technology for messaging, they should create a _new_ blockchain and not clutter Bitcoin's.

This chart is scary enough without throwing new use cases into the blockchain (e.g. messaging) that have nothing to do with Bitcoin transactions: [https://blockchain.info/charts/blocks- size?timespan=all](https://blockchain.info/charts/blocks-size?timespan=all)

<comment><USER3> I must say I disagree with you although I do see your point.

I don't believe that people will use the Blockchain for messaging. It will cost you 60c per message. I certainly wouldn't use it for messaging.

I think the implication is being able to use OP_RETURN (which btw does not add to the size of the blockchain because it is immediately provably prunable) and people will be able to attach payload onto a transaction. This will be important for this currency to be far more powerful beyond just sending money around.

</comment></comment></comment></comment><comment><USER6> As I understand it there is no fee for a 1BTC transfer. Is there anything stopping me from sending 1BTC back and forth between two wallets, as a means of recording permanent messages?

<comment><USER3> This is actually not possible with this particular system right now. Because you will need to send to particular outputs for messaging.

But if you do it with OP_RETURN then yes, you should be able to do that.

</comment></comment><comment><USER7> Neat site! The dialog box stays open after clicking "How it Works" though. Thought I'd let you know (assuming you're the creator of this) Firefox 24.3.0 on Linux x86

</comment><comment><USER8> I like how they judge contributions by amount of money people send. Growth hacking right thing.

</comment><comment><USER9> Looks cool! How are you actually embedding messages into the blockchain?

<comment><USER3> hey, that's a good question. Each standard Bitcoin address is a derivation of the 20 bytes `hash160`. This allows us to embed an arbitrary 20 bytes message in each output. When you put together a message, we build and propagation a transaction where your message is matched by the hash160s of the outputs.

You can see more here: [http://www.thenoteblock.com/how-it- works](http://www.thenoteblock.com/how-it-works)

<comment><USER10> Please don't do this, as it pollutes the UTXO set and makes it more expensive to run a full node, which reduces decentralization. Please use an OP_RETURN, provably unspendable output if you want to embed data in the blockchain. As a bonus, you can embed 80 bytes, rather than 20.

<comment><USER11> Not all miners are relaying OP_RETURN transactions. The plan is the use OP_RETURN once v0.9 is released.

We were hoping it would've been released before 14th Feb, however that's not the case.

<comment><USER10> BTCGuild now mines OP_RETURN - so there's nothing preventing you from using it.

EDIT: As does eligus (they have always accepted nonstandard transactions).

<comment><USER3> We were aware that some miners did mine OP_RETURN.

We do have plans to move this. We previously decided against because we felt it might take too long for the transaction to get confirmed and wanted to at least wait til there is at least 50% pool market share that mines OP_RETURN.

On second thought, we might move to OP_RETURN in the next week, I will certain evaluate this a bit more tonight.

The backup plan is still to move to OP_RETURN once v0.9 is adopted or more miners start accepting OP_RETURN

<comment><USER12> Please move to OP_RETURN ASAP. Right now, you're making money off the storage of 1000s of people like me running bitcoind, with no ability to remove your messages.

If people are really wanting something to be stored forever, is an extra hour or two really going to matter? BTCGuild is accepting OP_RETURN, along with several small outfits like Eligius, so it won't take days on end to confirm.

<comment><USER3> \-- you're making money off the storage of 1000s of people like me running bitcoind

Not making any money actually. Project is open sourced too

\-- OP_RETURN

Will probably move on the weekend.</comment></comment></comment></comment></comment></comment></comment></comment>


An LSM-Tree-based Ultra-Large Key-Value Store for Small Data [pdf] - <AUTHOR> http://www.ece.eng.wayne.edu/~sjiang/pubs/papers/wu15-lsm-trie.pdf <comment><USER2> This looks like an interesting finding. Unfortunately, the trade-off for this type of efficient small data storage is real:

 _> Note that LSM-trie uses hash functions to organize its data and accordingly does not support range search._

Range search, while not directly applicable to all data sets, is an important feature of the LSM data stores compared (LevelDB & RocksDB).

The authors acknowledge this and say:

 _> There are techniques available to support the command by maintaining an index above these hash-based stores_

So, don't plan on using an LSM-Trie for a direct replacement for your LevelDB or other LSM-Tree based projects that rely on Range searches without considering the additional complexity of building and maintaining an index to perform those Range searches.

<comment><USER3> For folks curious how you might build up a range index atop a hash store, here's a general scheme: [https://www.eecs.berkeley.edu/~sylvia/papers/pht.pdf](https://www.eecs.berkeley.edu/~sylvia/papers/pht.pdf)

</comment></comment><comment><USER4> Of potential interest, the code itself: [https://github.com/wuxb45/lsm-trie- release](https://github.com/wuxb45/lsm-trie-release)

</comment><comment><USER5> Every time I come across a new KV store, I go out looking for an userland implementation from the Acunu guys: [http://arxiv.org/abs/1103.4282](http://arxiv.org/abs/1103.4282). So far, none seen yet.

I'd like to see Stratified B Trees kick LSMs' butts. Or the other way around. I can't code this yet. But I can hope that somebody's already on to this.

Need to mention Percona's take on this space too: [https://github.com/percona/PerconaFT](https://github.com/percona/PerconaFT). The patent notice makes it scary tho.

</comment><comment><USER6> I haven't read the whole paper, but "LSM" is "log-structured merge"; I guess they merge runs over time to keep write amplification down? As eyan points out, Twigg's (Acunu's) "stratified B-trees" may have obsoleted the whole copy- on-write-B-tree family of data structures.

<comment><AUTHOR> This is a form of LSM tree which sacrifices range queries to gain at most 5x write amplification over 5 levels. That translates into a significant increase in throughput compared to LevelDB and RocksDB.

</comment></comment><comment><AUTHOR> This paper targets datasets of 1-10 billion keys, keeping write amplification down to at most 1x per level (i.e. 5x over 5 levels), and in-memory indexes to a minimum.

Here are the slides: [https://www.usenix.org/sites/default/files/conference/protec...](https://www.usenix.org/sites/default/files/conference/protected- files/atc15_slides_wu.pdf)

</comment><comment><USER7> I tried LevelDB with high hopes, but it is ungodly slow. The claims did not line up with reality in any way.

<comment><AUTHOR> This is very different to LevelDB. But LevelDB's claims are actually valid, provided your dataset is not massive, and you understand how your workload interacts with its compaction and write amplification. This paper presents a form of LSM-tree which solves both those issues (massive dataset, low write amplification).

</comment></comment><comment><USER8> Interesting, will try to add that into my project.

</comment><comment><USER9> Now that we know that "number of atoms in the universe" is not a good extremely large number [1], maybe we can use "number of key-value-stores" as the new reference?

[1] [https://news.ycombinator.com/item?id=11588918](https://news.ycombinator.com/item?id=11588918)</comment>


The Shinkansen Story [video] - <AUTHOR> https://www.youtube.com/watch?v=T3LLgzO_PrI <comment><USER2> Really cool! I like it that the Japan generally is not given to the BS-till- you-make-it attitude of SV.

The spiritual successor to the 60s pioneering project, the Chuo Shinkansen, which is the result of proven technology developed over 30-40y continues to chugging away slowly and silently, at an estimated cost of $80B to connect Tokyo with Osaka (in '2037).

Musk's Hyperloop's on the otherhand is what you get from SV. BS vaporware, which would magically connect LA and SF at a mere $6B, with technology which hasn't even been invented yet! Even the current Shinkansen costs twice as much as this valuation (in India!).

<comment><USER3> Agree that Hyperloop is ... hype. But you otherwise paint SV with too broad a brush with your "BS-till-you-make-it" comment.

<comment><USER4> I would go even further to say it's not even broad enough to reflect the general culture in the US.

</comment></comment></comment><comment><USER5> Just rode the Shinkansen last month from Kyoto to Tokyo. Very enjoyable trip, except (like everything in Japan) at my height and size (190cm/6'3") it was a tick too small. Even in the green car.

That, and the annoying foot rest which takes up most of the space in front of you whether you need it or not were the only negatives. Awesome train (although the styling of the modern noses doesn't appeal to me at all) ride. Super smooth.

If you're going to be riding, don't dawdle getting off. You're expected to be at the door ready to get off when the train stops, if you wait, you'll be encountering passengers getting on, and the train doesn't stop long at the station.

p.s. Please don't get offended that I said Japan was a tick too small for me. It doesn't have to fit me, I don't live there. I was just making an observation. :)

<comment><USER6> I really enjoy being in Asia because finally everything is the right size for me. I'm a bit small, and I didn't notice before how everything in the west is slightly too big for me. A minor annoyance, sure, but what a relief when things finally fit!

<comment><USER7> I wonder what you really benefitted in Asia in that regard. Having more roomy space doesn't sound bad and I'm sure clothing won't have shortage of varying sizes.

<comment><USER6> One simple thing: Chairs are usually just a few cm too high. As a result, all the weight of my lower legs is on the part just above the knee and sitting normally is quite uncomfortable and I usually just.. don't, and cross my legs to sit fully on top of the thing. (In formal settings, this is not acceptable however.) I recently flew with China Airlines (the Taiwanese one) and lo and behold, I could finally sit in an airplane normally. Not recommended for tall people, but very much so for shorter ones.

Workspaces in the kitchen or workshop are also just a few cm too high to work comfortably. In public transport, I often cannot reach the straps to hold on to from the top without being in very very uncomfortable stretch. Of course, if they were lower, other people would hit their heads, so it can't be helped.

It's these small things that are hard to set up in any other way (one size fits all) and someone will be inconvenienced at any setup. I'm not saying we should change things here to accommodate people my size. It's just really nice when things simply.. fit.

</comment></comment></comment></comment><comment><USER8> The Shinkansen is just so awesome. GF an I toured Japan last October and we did all inter-city travel by the Shinkansen, using a Japan-Rail pass, so you can just walk through the entry gates. The punctuality is also unparalleled. On a different train, we arrived well in time for departure, only to find the train was delayed! Oh noes! The guy at the counter walked up and off the walkway to our platform to personally excuse himself for the 15 minute delay. Of course, the train was exactly 15 minutes late.

I wish Europe would have this level of high-speed rail. I'm starting a remote job next year, traveling to the office twice month. That trip will cost me ca. 6hrs one-way but with a Shinkansen, maybe 3 hours. The continent would be so much more close-knit with such train service.

<comment><USER9> Can anyone explain unlike other countries how has the Shinkansen not crumbled like the trains in California or New York. Did they bet on the right technology early on and/or are they just better at upgrades ?

<comment><USER10> > Can anyone explain unlike other countries how has the Shinkansen not > crumbled like the trains in California or New York.

Crumbled in what sense? Ridership?

<comment><USER9> BART Muni Caltrain all have a lot of delays and breakdowns. Some of them are also not safe to travel in (but this is besides the technical points.)

<comment><USER11> European countries have an interest in cutting down on the number of cars on the road. The Paris climate accord means that we all have to drastically cut CO2 emissions. Also European cities are suffering from air pollution.

This means that governments invest very heavily in trains and other mass transit. Trying desperately to keep cities livable and functional.

</comment></comment></comment></comment></comment><comment><USER12> Anecdote: I was just in Japan and w/ a train-loving buddy, and he found the fastest in the country for us to take a ride (The Hayabusa line between Tokyo <-> Sendai).

Super fun! It tops out just over 200mph for a few stretches (obligatory screen cap from whatever GPS speedometer app I happened to download at the moment)[1]

We rode coach up to Sendai and First Class on the way back after a long day of wandering around. 1000% worth the experience if anyone is considering the adventure as part of a Japan trip.

[1] [https://m.imgur.com/a/WBtSgJ6](https://m.imgur.com/a/WBtSgJ6)

<comment><USER13> If you know you're traveling to Japan and want to ride the Shinkansen, then definitely get a JR rail pass. They are good for a week (upto 3) of travel, but can pay for themselves in 1 or 2 rides. They're even good for travel within Tokyo on Yamanote and Chuo lines (and if your flight arrives in time for you to validate for the Narita Express or Haneda Monorail).

There are lots of places on-line to purchase, but the ticket must be delivered/picked-up outside of Japan and you must have a foreign passport/residence.

Also, the pass is only valid on the Kodama/Hikari (echo/light) trains, but not Nozomi (hope) which have fewer stops.

The real advantage of Green/1st class is that you can more easily book last munite at busy times without fear of the train being full.

<comment><USER6> Actually, they have a trial right now where you can buy the JR passes inside Japan. The cost is a little higher compared to buying abroad. The foreign passport and short-term visa requirements still remain.

</comment></comment></comment><comment><USER14> small self-deprecating anecdote:

after I landed in Narita for the first time, I rode the Narita Line into Tokyo.

I, coming from the US, was absolutely amazed at the cleanliness and super-fast speed of the train, which I assumed must be a bullet train because we were traveling so much faster than I had ever traveled by train before.

Imagine my surprise when I found out it was just the regular train line, and the bullet train is a whole different beast...

<comment><USER15> "Maximum speed 130 km/h (80 mph)" (Wikipedia)

That's slower than the airport express trains for several European airports I'm familiar with (London Heathrow, London Gatwick, Stockholm, Oslo), which is unexpected. However, it's probably a lot cheaper than any of them.

<comment><USER16> Narita Express is among the expensive train in Japan though. Narita Airport to Tokyo station cost JPY2820 (~$25).

130km/h is the maximum speed any train can operate on a line with level crossing designated by Japanese law. But in Japan a lot of train can maintain near maximum speed throughout its journey, unlike what I have seen in the Europe.

</comment></comment></comment><comment><USER17> Imagine if the US had leaders obligated to resign when projects ran overbudget.

<comment><USER18> Politics needs more skin in the game overall.

</comment></comment><comment><USER19> What love about the Shinkansen, at least Tokyo <-> Osaka is that it leaves every 5-10 minutes so I never make reservations. I just casually go when I feel like it, shop outside for a snack, then go in and get a ticket at a machine and hop on the next train.

This is something I can't do with planes as well as you have to book in advance, get there early, check in, get inspected, line up to board. Filling plane seems to take 15 to 30 minutes. Filling the train takes 2-3 mins. And of course the train drops me off in the center of town, not 30 to 90 minutes outside of town like most airports.

There's a couple of seasons where getting a seat or getting 2 seats together might take more planning but most of the time taking the Shinkansen feels no different than taking any local train or subway in terms of prep which makes it super easy to use. It's a little expensive, $240-$350 round trip (free seating->reserved seating->1st class) but I've still made impulse trips for various events deciding on the same day that, yea, ok, let's go!

I will say China's bullet trains are amazing as well but booking/tickets are not nearly as smooth.

</comment><comment><USER20> Both Mustard and Brilliant look really cool. In fact in this video I wasn't sure at times whether the footage was CGI or real, testament to how good rendering is these days.

<comment><USER21> I think it's rendered in a game engine, possibly Unity. The whole video appears to be put together in a game engine.

</comment></comment><comment><USER22> Here's another nice video about Shinkansen origin story from a video series I enjoy ("Begin Japanology"): [https://www.youtube.com/watch?v=tHCl8z47bL0](https://www.youtube.com/watch?v=tHCl8z47bL0)

</comment><comment><USER23> The next generation Shinkansen will begin testing around May next year (2019). The 'Alfa-X' is expected to travel faster than 400 km/h (360 km/h with passengers)

note: The fastest commercial train in the world is Shanghai's maglev train (431km/h) which uses magnetic levitation.

\- [https://plus.google.com/117929795458007747856/posts/iVaesHMc...](https://plus.google.com/117929795458007747856/posts/iVaesHMcLav) \- [https://www.jrailpass.com/blog/alfa-x-shinkansen-bullet- trai...](https://www.jrailpass.com/blog/alfa-x-shinkansen-bullet-train)

</comment><comment><USER24> There are a couple of high def web cams of busy Japanese lines just outside stations, that stream live on YouTube (with good sound, as well). (Unfortunately, the recent typhoon took out the one by the Italian Park in Tokyo.)

I've found I can watch those for hours. I've thought of putting a TV in my office and just leaving one or the other on as background.

They also remind me a bit of one or two domestic market contemporary Japanese films that have featured trains prominently.

I guess it's probably mostly just a one-off amalgam of experiences that exists in my head. But enjoyable.

P.S. You watch one of these, and -- as a foreigner -- you realize just how many of the high speed trains are running each hour, each day.

<comment><USER25> Can you please link to these? I'd love to say my kid would enjoy these, but who are we kidding? I'd love them too.

<comment><USER24> Yeah, I didn't, in my first comment. I guess I was a bit afraid if they got popular they'd "go away" or something -- even though they are "buffered" by YouTube.

\--

This one is still working:

[https://www.youtube.com/watch?v=H8LY9oqqzXM](https://www.youtube.com/watch?v=H8LY9oqqzXM)

looking SE towards the north end and east side of Shin Osaka station:

[https://goo.gl/maps/LqVgRHNWu7H2](https://goo.gl/maps/LqVgRHNWu7H2)

\--

The following one has been down since the typhoon. I haven't tried reaching out -- I don't make comments on YouTube -- but if it's a matter of the cost of replacement, I wonder whether the person it belongs to would be interested in donations to pay for its replacement. I rather enjoyed it.

[https://www.youtube.com/watch?v=0F-Aw9t1VC8](https://www.youtube.com/watch?v=0F-Aw9t1VC8)

Located across the street and tracks (and the other street) from Italy Park, looking to the SE:

[https://goo.gl/maps/2154UmjBYu72](https://goo.gl/maps/2154UmjBYu72)

\--

P.S. For the Osaka location, once in a while you'll see a commercial jet's shadow approach and cross the forefield. 2 pm local time, or sometimes an hour either way of that, has seemed to be the time for that, this fall.

<comment><USER25> Thanks!

</comment></comment></comment></comment><comment><USER26> Crazy to know this train appeared so early.

I'm also quite curious about the recent US regulation that was dropped about wagon weight. Could not understand how this regulation made sense.

</comment><comment><USER27> It seemed interesting, but narration is very slow. :(

<comment><USER28> 2x speed doesnt help?

</comment></comment><comment><USER29> Aw.

Some of the Shinkansen technology came from BART, which was being built around the same time. Active suspension, all wheel electric drive with electronic controls, and automatic train control all started with BART. Now everybody has those features.

<comment><USER16> Can I have source? According to Wikipedia BART began construction in 1964, the year Shinkansen open. Sanyo Shinkansen, the extension to Hakata, open in 1972, months before BART open.</comment></comment>


Honey, I killed the superbug - <AUTHOR> http://www.theaustralian.news.com.au/story/0,25197,25653017-5013404,00.html <comment><USER2> I can't find any mention of this in medical news sources; it sounds interesting, but the fact that it's being reported primarily by general interest media leaves me a bit skeptical.

<comment><USER3> I've read many things about the usage of honey to treat wounds, the most impressive that I'm aware of is its use to treat necrotic tissue. They can use it on some of the largest wounds because the honey is naturally antiseptic and allows a prolonged open wound to not catch infection, and they use laboratory bred maggots that are completely sterile and insert them into the wound. The whole thing is then bandaged up and the maggots will clean the wound of dead tissue better than any surgeon could and for much, much cheaper. After about a week the maggots should have removed all the dead tissue and your body can completely heal the wound, sometimes with very little scar tissue proportionate to the wound. I believe one of the most common uses is on motorbike fall-off accidents due to people losing huge chunks of their leg and stuff.

It's an extremely impressive treatment and exceptionally cheap and successful compared to the alternatives.

<comment><USER4> A year or two ago, there was some reporting on its effectiveness (presumably backed by a study, but I no longer recall) in limiting further peripheral tissue damage and the need for amputation, in diabetics suffering from that problem. Applied topically; not ingested.

</comment></comment></comment><comment><AUTHOR> Get raw honey. The processed kind loses these properties.

You can tell the quality, just turn over a jar you'll see a water bubble float up. The faster it rises the lower the quality. The raw type has none of that, just the good stuff.

Humans have been eating honey for the longest time. It's probably the best sweetener.

<comment><USER5> Good advice in picking a good raw honey, and I agree that honey is a great choice for a sweetener. BTW, farmers markets and local produce stands are great places to find good raw honey cheap.

<comment><AUTHOR> For convenience, it's at Whole Foods Market too. But I'm unsure of the quality.

I agree about artificial sweeteners, they can cause problems. They seems to trigger psychological effects that cause overeating [1]. The super sweet taste might signal, _binge_. Humans never ate highly concentrated super sweet foods. Fruit was seasonal. But even the fruit we eat today has been bred for sweetness for centuries.

[1] [http://www.purdue.edu/UNS/html4ever/2004/040629.Swithers.res...](http://www.purdue.edu/UNS/html4ever/2004/040629.Swithers.research.html)

</comment></comment></comment><comment><USER6> Honey is hygroscopic. It sucks the water out of bacteria and kills them. But anything hygroscopic does that, honey is not special in that way.

</comment><comment><USER7> The article seems to conflate antibiotics (which kill bacteria living _inside_ you) with topical treatments (which kill bacteria on you). You can't replace antibiotics with this.

</comment><comment><USER8> As far as I know all honey does this, not just Manuka honey.

<comment><USER9> To some degree, but the more voracious kind is based on honey made by bees feasting on Australian Tea Tree .. which is, itself, a pretty wonderful source of all kinds of protective substances.

<\-- grew up in Australia, was constantly healing my scrapes and bumps with Tea Tree oils and Manuka honey .. glad its all finally getting some science.

<comment><USER10> My family used to do similarly, even though "modern" medicines like neosporin were popular then, but IIRC bark and sap from the Manuka tree is supposed to be good, too.

+1 for raw honey being better, but it's pretty hard to get it in big cities (without paying ridiculous prices!).</comment></comment></comment>


Notes from a 1984 Tech Talk About Programming Languages - <AUTHOR> https://codesmithdev.com/notes-from-a-1984-tech-talk-about-programming-languages/ <comment><USER2> Between your giant header and the "we're online, talk to us" chat thing at the bottom, I'm literally left with only half my screen to read the article.

I feel like I'm trying to drive a bus from the back seat.

<comment><AUTHOR> I reposted just to the video. Sorry about that. [https://youtu.be/Jtvgf_CyiS0](https://youtu.be/Jtvgf_CyiS0)</comment></comment>


New PostgreSQL point releases for 8.0 through 8.4 - <AUTHOR> http://www.postgresql.org/about/news.1203

<comment><AUTHOR> An important note from the bottom of the announcement:

"The PostgreSQL Global Development Group will stop releasing updates for PostgreSQL versions 7.4 and 8.0 after June of 2010. We urge users of those versions to start planning to upgrade now."</comment>


The MOS 6502 and the Best Layout Guy in the World - <AUTHOR> http://research.swtch.com/2011/01/mos-6502-and-best-layout-guy-in-world.html <comment><USER2> >The most amazing part about the whole process is that they got the 6502 right in one try. Quoting On the Edge: Bil Herd summarizes the situation. "No chip worked the first time," he states emphatically. "No chip. It took seven or nine revs [revisions], or if someone was real good they would get it in five or six."

In some ways (and I'm speaking in a general sense) situations like that actually make me more nervous than when I know there's a problem. I get this uneasy "there's no way it _really_ went that smoothly" feeling that can be hard to shake.

Then again, my personality is to approach most things in life iteratively, so that probably plays a part as well. Great read either way.

<comment><USER3> That's the sensation of working with someone who's _incredibly_ good.

And, BTW, the 6502 was a work of art. Simple, elegant and fast (even at 1 MHz), it ran rings around the Z-80's you found in more expensive computers of the time. Plus, it was delightful to program.

<comment><USER4> _Plus, it was delightful to program._

Simple maybe, but not delightful to program. I don't know of any assembly language that is delightful to program.

BTW, I knew someone from junior high & high school who could write code for the 6502 using a hex dump with amazing speed. You might have heard of him: Randy Linden.

<comment><USER5> As Doug McIlroy (inventor of pipes, diff) once said of punched cards, "It's the kind of thing you can be nostalgic about, but it wasn't actually fun."

[http://research.swtch.com/2008/04/computing-history-at- bell-...](http://research.swtch.com/2008/04/computing-history-at-bell- labs.html)

<comment><USER6> Punch cards are different: there's substantially delayed gratification.

When you're a poor college student as I was, entering opcodes in a monitor/debugger you wrote in BASIC because you couldn't afford an assembler, then yes, assembly was fun.

Fun, that is, until you had to hand-calculate negative jump offsets. Don't remember why, but for some reason I seem to think that the MC6809 I was running on made it difficult to do so.

<comment><USER7> There must have been something special about the 6809 - I recall doing my Digital Design course on that Chip. And yes, we didn't have an assembler, everything was entered via opcodes either. It was a very enjoyable experience for someone like me who wasn't a gear head, and got to play around with SB555s, NAND Gates, and lots, and lots of wirewrapping.

</comment></comment></comment></comment></comment></comment><comment><USER8> From Jordan Mechner's diary of the development of Prince of Persia (POP was originally coded in 6502 Assembler. It took him four years). Reading Jordan's full diary will take you at least eight hours but it is well worth it.

> We chatted for an hour about peripherally related topics. Broderbund, > corporate America, the rat race, capitalism, freedom. I was seducing him.

At the critical psychological moment, I remarked: "You know, all my clipping is done on the byte boundaries."

There was a pause

<http://jordanmechner.com/old-journals/page/33/>

April 3, 1989

</comment><comment><USER9> The whole 'Reverse Engineering the 6502' talk Michael Steil gave at CCC congress is on YouTube. I posted this earlier in a separate topic, but it didn't pick up.

Clickable links to the 6 parts:

<http://www.youtube.com/watch?v=HW9AWBFH1sA>

<http://www.youtube.com/watch?v=bBE4KHKzhKc>

<http://www.youtube.com/watch?v=tRBo7O_blVo>

<http://www.youtube.com/watch?v=H_15RtVbqGU>

<http://www.youtube.com/watch?v=N9DYmlprCKA>

<http://www.youtube.com/watch?v=eZOUuqc4pk8>

</comment><comment><USER10> Intel Core 2 - Yorkfield, 45 nm process technology, Number of Transistors: 820 Million

MOS 6502, Number of Transistors: 3510

So in theory, a chip with 65536 MOS 6502 cores each with 64K of internal RAM (4Mb cache) could be made.

<comment><USER11> Sure, if you completely ignore all of the extra circuitry for the on-chip network and cache coherency and everything else you would need. Transistor density is far from the limiting factor in the number of cores we can wedge into a single system.

<comment><USER12> If you assume that there is no cache (only on-die memory) and that memory is not shared between cores, things become much simpler and scale more linearly. Core-to-core communications and plenty of other details remain to spend man- years ironing out, but it seems like it would be possible to approach 64k cores or at least 16k.

<comment><USER13> Might be able to peel out the BCD stuff from the 6502 to free up a little additional space and approach communication between cores kinda like the "handshake bus" GreenArrays chips use: [http://greenarraychips.com/home/documents/greg/PB003-100822-...](http://greenarraychips.com/home/documents/greg/PB003-100822-F18A.pdf)

</comment></comment></comment></comment><comment><USER14> I loved the 6502. Around that era I programmed the SC/MP, Z80, the 8080 and the 6800. Although the Z80 was more powerful, the 6502 holds a special place in my heart as it was the first CPU I worked with and I loved the simplicity of the instruction set.

My crowning achievement was a multi-threaded kernel for a CNC punch. Since the stack was at a fixed memory address and there was no PUSHA, I had to change threads (in response to an IRQ) by sequentially pushing the registers on to the stack and then swapping the stack with a block copy. It worked! Crazy :/

I loved reading this article - thanks for posting. Awesome stuff! Makes me want to code my own circuit emulator :)

<comment><USER15> Ditto that. My first programs were in assembly on the 6502, and I'm thrilled to see it getting this attention. I'm just sorry that I don't still have the KIM-1.

<comment><USER16> I was planning on buying a KIM-1 when my dad surprised me by buying an Apple II. I didn't end up writing anything in assembly on the Apple II, though. My first assembly language was IBM-370 in college. I wish now I had started with the KIM-1, though.

</comment></comment></comment><comment><USER17> Here it is in all its javascript goodness: <http://www.visual6502.org/JSSim/index.html>

</comment><comment><USER18> The CCC congress yielded some great talks this year.</comment>