How Google Dominates Us by James Gleick - <AUTHOR> http://www.nybooks.com/articles/archives/2011/aug/18/how-google-dominates-us/?pagination=false <comment><USER2> If any hacker reads this long piece, I'd welcome a one-paragraph summary.</comment>


Non-Recursive Make Considered Harmful: Build Systems at Scale (2016) [pdf] - <AUTHOR> https://ndmitchell.com/downloads/paper-non_recursive_make_considered_harmful-22_sep_2016.pdf <comment><USER3> One should start with, "Recursive Make Considered Harmful" [0] to know where this is coming from.

The best architected use of make is in the FreeBSD build system [1,2] If you want to experience "a system" please give FreeBSD a try.

[0] [http://aegis.sourceforge.net/auug97.pdf](http://aegis.sourceforge.net/auug97.pdf)

[1] [https://www.freebsd.org/doc/handbook/makeworld.html](https://www.freebsd.org/doc/handbook/makeworld.html)

[2] [https://www.freebsd.org/doc/en/books/porters- handbook/portin...](https://www.freebsd.org/doc/en/books/porters- handbook/porting-samplem.html)

</comment><comment><USER4> Another interesting paper is "Build Systems à la Carte" [https://www.microsoft.com/en- us/research/uploads/prod/2018/0...](https://www.microsoft.com/en- us/research/uploads/prod/2018/03/build-systems-5ab0f42d0f937.pdf) which explains characteristics of some build systems "static vs dynamic dependencies; local vs cloud; deterministic vs non-deterministic build rules; support for early cutoff; self-tracking build systems; and the type of persistent build information. ... We show that we can instantiate our abstractions to describe the essence of a variety of different real-life build systems, including Make, Shake, Bazel, and Excel, each in a dozen lines of code or so"

<comment><USER5> ... which is by three of the same authors.

</comment></comment><comment><USER6> Is it really worth blaming the tool? Has reimplementing make for the 50th time really improved things?

The fact is, building software that requires 500 dependencies and 500 sub- steps and 500 configuration options is going to be complicated. It's complicated in the same way that implementing an operating system is complicated. There's no way around it. The complexity is there because it's inherent in the problem.

But it doesn't have to be. Instead of spending 300 hours implementing Shake, or Rake, or Bake, or Cake, or Jake, or Take, why not spend those hours cutting down the complexity at the source? Trim your dependencies. Stop putting so many sub-steps and configurations into your build systems. Cause it's the build systems with the 500 dependencies and 500 the sub-steps and the 500 configurations option that are harmful; not the tools

<comment><USER7> > Has reimplementing make for the 50th time really improved things?

Considering the authors did so and find it to be an improvement in terms of maintainability and usability... I'm going to say "yes". Do you think you know more about the project than they do?

I used to work on GHC. The build system is complex. Hadrian is quite an improvement in power and expressiveness (and is now capable of doing things we wouldn't have been able to implement easily with Make, since extending the prior system was too hard).

> The fact is, building software that requires 500 dependencies and 500 sub- > steps and 500 configuration options is going to be complicated. It's > complicated in the same way that implementing an operating system is > complicated. There's no way around it. The complexity is there because it's > inherent in the problem.

I get the feeling you're going to use this random truism as a springboard to make suggestions despite the fact you've never been involved in the project?

> But it doesn't have to be. Instead of spending 300 hours implementing Shake, > or Rake, or Bake, or Cake, or Jake, or Take, why not spend those hours > cutting down the complexity at the source? Trim your dependencies. Stop > putting so many sub-steps and configurations into your build systems. Is > that the sane way to do things?

That would be nice if everyone had endless time and everything was always done exactly perfectly up front. It would also be nice if you could work completely on your own and never have to interact with any other software in the world.

Binary tarballs, source distributions, upstream library dependencies, cross compilation, thousands of tests, tracking all dependencies _correctly_ (this one alone is ridiculously hard), autogeneration tools (to save errors on tricky parts). Feature detection at compile _and_ runtime (because your users work on some old CentOS machine and no `pthread_setname` is not available), profiling builds, running documentation generators, handling out-of-source builds, handling relocatable builds. I can just keep listing things, honestly. All of these -- more or less -- come back to your build system.

In fact, GHC goes _quite_ out of its way to expressly use as few non-Haskell dependencies as possible. Why? Because the ones it already has are often burdensome and complex, and we have to pick up the slack for them for every user. Nobody using your project cares if Sphinx or their rube-goldberg Python installation (spread over 20 places in /usr) was the reason doc building failed; your build failed, that's all that matters. You've still got to figure out what's wrong, though, for your user. And not wanting new dependencies has been a common reason to reject things -- I myself have rejected proposals and "features" to GHC on this basis alone, more or less. ("Just use libuv!" was a common one that sounded good on paper and never addressed any actual issues we had that it claimed to 'solve'.)

As a side note, it really just amazes me the amount of people who immediately see any amount of non-trivial work in some project and immediately question "well, why don't you just do <random thing that is completely out of context and has no basis in the projects' reality>". Seriously, any time you think of this stuff, please -- just give it like, 10 more seconds of thought? You'd be surprised at what you might think up, what you might think is possible. It's not the worst thing in the job, but being an OSS maintainer and having to deal with analysis' that are, more or less, quite divorced from the reality of the project is... irritating.

<comment><USER8> > Considering the authors did so and find it to be an improvement in terms of > maintainability and usability... I'm going to say "yes". Do you think you > know more about the project than they do?

Every single self-described make replacement project makes the exact same claim, verbatim. Yet, when these projects start to see some use in the real world... Queue all the design shortcomings and maintainability and usability problems.

We're about 4 decades into this game. Perhaps this time everything is different. Who knows. Odds aren't good, though.

<comment><USER9> I think that being the build system for the Glasgow Haskell Compiler - <USER2> is the most commonly used compiler for Haskell - counts as "some use in the real world." I downloaded the source, did `git ls-files | xargs wc -l > wc.out`, then `grep "total" wc.out`, summed the totals, and it comes to 1051451. That's an overestimate in lines of code, as there's certainly documentation in there, but there's about 620,000 lines of Haskell.

<comment><USER8> Great, someone managed to get a build system to work for a project. That's nice. I'm sure there are a bunch of cases where even hand-written makefiles are being used to the same effect. Does that mean that any of those tools are free from any design issue or maintainability problem?

<comment><USER9> That's a good question! I think a good way to figure that out is to publish a paper about what they did, how it solved their non-trivial problem, and then invite others to try to use their tool and techniques to solve their problem.

In other words, you're asking a question that criticizes the mechanism that would answer that question. It's a legitimate question, but a poor reason to disregard what they did.

</comment></comment></comment></comment></comment></comment><comment><USER10> Functional build systems are a great idea. The main problem I had implementing such a system was the file system. It's tough to capture all inputs and outputs, even with the best of intentions.

The other problem is integration. You can't expect all 3rd party projects to suddenly adopt your build system, so eventually you have to invoke `configure`, `make`, `cmake`, `pkg-config`, `xcode`, and so on. While you can satisfactorily capture most of these inputs and outputs, it's non-trivial to do it completely, at some point you have something that works and it's good enough.

<comment><USER11> I mean... Nix and Guix build entire systems off of the idea of capturing every single dependency in the build system, and they work in production

<comment><USER12> Do they even capture things like the C++ headers that your compiler uses? I think that's what he meant. When you compile a C++ file the compiler goes off and reads all kinds of files that are totally unknown to your build system.

Simpler languages like Go don't really have this problem, but they also have sane build systems so I don't know why you'd need CMake or whatever.

<comment><USER13> This is a long-ago solved problem, and the solutions are only getting better. GCC has the -M family of options which are explicitly designed to feed the header dependencies into the build system, and these flags have been around since the dark ages. More modern systems like Bazel will sandbox the build so if you try to #include a file that's not listed as a dependency you'll just get a compiler error. If you want, you can specify the compiler itself as a dependency.

Hermetic builds are the way to go for a ton of reasons.

<comment><USER14> It is a solved problem, but your compiler does not in fact implement an adequate solution with that mechanism.

* [https://news.ycombinator.com/item?id=15044438](https://news.ycombinator.com/item?id=15044438)

* [https://news.ycombinator.com/item?id=15060146](https://news.ycombinator.com/item?id=15060146)

</comment></comment></comment></comment></comment><comment><USER15>
        While we have demonstrated that our approach works,
        we have not yet implemented all features of the build
        system, and hope to do so over the next few months

This is a pretty major caveat. Almost damning in its significance, honestly. There are plenty of "works, but haven't quite implemented all of the old features" projects littering the world. I love that there are learnings here, and those should be seen as the most important artifact of any project. I do wish there were paths to get those learnings back to the old systems, though. :(

<comment><USER16> Table 1 in the paper looks promising, though. Let's hope their estimation is correct:

 _We implemented 5 a new build sys- tem for GHC from scratch using Shake and our build abstractions from §5. The new build system does not yet implement the full functionality of the old build system, but we are currently address- ing remaining limitations; nothing presents any new challenges or requires changes to the build infrastructure._

<comment><USER17> Well its in place now for ghc 8.6 ref [https://ghc.haskell.org/trac/ghc/wiki/Status/Apr18](https://ghc.haskell.org/trac/ghc/wiki/Status/Apr18)

[https://github.com/ghc/hadrian](https://github.com/ghc/hadrian)

Its mostly complete, and as someone that has used ghc's make system and likes make, this build system is miles better and isn't a chthulian horror.

</comment></comment></comment><comment><USER18> _To validate our claims, we have completely re-implemented GHC's build system, for the fifth and final time._

Famous last words.

<comment><USER19> Build, version control and package management: three problems perennially in search of a definitive solution.

For those who are sure we already have a definitive solution to one or more of these, the problem is in persuading everyone else, especially those who think something else is it.

<comment><USER20> Here are multiple examples of "definitive" build systems, by your definition. Many are over a decade old, and haven't been standing still. These days, they generalize to many, many languages, and are working towards byte-for-byte reproducibility of the build artifacts:

[https://reproducible-builds.org/who/](https://reproducible-builds.org/who/)

<comment><USER21> So this is a dumb question, but isn't the question of reproducible builds made trivial by vendoring dependencies? Why is it treated like this holy grail when it should be dirt simple? I must be misunderstanding something.

<comment><USER22> It's not a dumb question, but unfortunately, we live in a dumb^H^H^H^H interesting world.

Check out all the links under this heading: [https://reproducible- builds.org/docs/#achieve-deterministic-...](https://reproducible- builds.org/docs/#achieve-deterministic-builds)

Many legacy systems capture all sorts of nondeterministic values -- from build date (it might be a desire to be "helpful", but breaks reproducibility) to accidentally depending on the order of inodes on your files system!

All of these problems are solvable. It _should_ be dirt simple. It's just a "small matter of programming" :)

</comment></comment></comment></comment></comment><comment><USER23> Meh.

Linux and BSD build systems deal with most of these issues usually with wide support of a variety of recursive makes. Though RPM, DEB honestly suck and never really tried to solve issues automatically. Still drives me nuts that packages are tainted by the 'gold' systems they are built on. The complexity of build systems means very few minds are up for it and most solutions are naive and end up with tons of patchy exceptions and work arounds.

ROCK Linux supported cross compiler capabilities and auto-detection of build parameters and dependency library tracking. (I was working on automated dependency ordering and QEMU based full cross builds, before I got a real job.) It was very robust and outside package developers breaking their own builds it worked solidly. No idea what cool things T2 Linux got up to after ROCK, but maintaining a fresh build system is hard. Build systems are always going to be fragile systems with complexity. The paper seems to be a survey of what they learned vs. definitely having any solution.

</comment><comment><USER24> I'd like to issue a blanket ban on the "considered harmful" thing, as well as sharing print-optimized PDFs on the web for people on computers to read.

Both things are archaisms that can be easily avoided, and in the particular case of this article, the 2nd part of the title works just as well as the title.

<comment><USER19> I am not generally in favor of bans, but if it came to that, I would first like to see a ban on complaints about the style and format of interesting material made freely available by the people who have already put considerable effort into creating it.

<comment><USER24> Style and format are important tho, would you like your docs written in haiku, or as a single three hour light opera video?

Embracing an archaic grandstanding pose and an academic look when much more easily grokkable formats are available just doesn't help get your message across that well. And when there's one of this roughly monthly I'd personally love to at least see less of them.

</comment></comment></comment><comment><USER25> Dropping in with my make horrorstory. I once had the pleasure of interacting with a 20k makefile (builds any one of 15 or so projects for any platform because "splitting up the makefile would lead to code duplication since a lot of the makefiles would be similar") I'm told that makefile is over 40k lines today only a few years later.

<comment><USER26> > _" splitting up the makefile would lead to code duplication since a lot of > the makefiles would be similar"_

So, they've never heard of templating?

</comment></comment><comment><USER27> Sorta reinventing the wheel?

[https://nixos.org/nix/](https://nixos.org/nix/)

<comment><USER28> One could say the same thing about nearly the entire nodejs ecosystem. Reinventing the wheel, and usually with lower quality to boot.

Sometimes it's okay to make something with redundant functionality.

</comment></comment><comment><USER29> My colleague @ejholmes wrote a cool tool that borrows heavily from `make` called `walk` ([https://github.com/ejholmes/walk](https://github.com/ejholmes/walk)). It's written in Go and using a graph for dependencies so that tasks at the same horizontal level in the graph may be run at the same time. We use this at remind.com to significantly speed up our multi later AMI build times.

</comment><comment><USER30> I've only used make a very little bit... but pretty much anyone can understand it for basic use.

This looks like you're writing source code in another language that follows a kind of template, which you then compile and run, and then does stuff and is extremely complicated.

Seems like a failure to me? Shouldn't something "better" be equally simple or simpler?

<comment><USER31> Example from the paper of Makefile code used GHC:

$1/$2/build/%.$$($3 _ osuf) : \ $1/$4/%.hs $$(LAX _ DEPS _ FOLLOW) \ $$$$($1 _ $2 _ HC _ DEP) $$($1 _ $2 _ PKGDATA _ DEP) $$(call cmd,$1 _ $2 _ HC) $$($1 _ $2 _ $3 _ ALL _ HC _ OPTS) \ -c $$< -o $$@ \ $$(if $$( ndstring YES,$$($1 _ $2 _ DYNAMIC _ TOO)), \ -dyno $$(addsu x .$$(dyn _ osuf),$$(basename $$@)) ) $$(call ohi-sanity-check,$1,$2,$3,$1/$2/build/$$ * )

I suspect it easier to master Haskell than this.

<comment><USER15> This just feels like a bloody straw man, though. Poorly written file in X not as good as well written file in Y. News at 11. :)

Substitute "Clever" for "Poorly", if you'd prefer.

<comment><USER5> Would you like to show us a well written version of that rule?

<comment><USER15> First, I probably couldn't even do a well written version of the rule in any language. :) Not exactly my strength.

Second, sometimes the best trick you can do to make a build system cleaner, is to change the system it is building.

Though, as I stated elsewhere in this chain. I don't necessary mean to dismiss this effort. Just showing me gymnastics that are required to do something that most people just don't care to do, doesn't endear me to the ill suitability of the context. (Fun metaphor, actually. The fact that my house isn't designed to allow easy gynmastic practices in the living room is not a criticism of the house or of gymnastics. Showing me that a cartwheel will get you hurt there is not really showing me anything relevant.)

</comment></comment></comment></comment></comment><comment><USER20> It is interesting that none of their examples of why make is considered harmful involved writing an idiomatic make file.

In fact, all of their examples of how bad make is involve the use of non- standard makefile generators, or non-standard extensions to make itself.

It is sort of like arguing that JavaScript is harmful, and then showing snippets of asm.js code to talk about how "tragic" the language syntax is.

Alternatively, I could argue that no one should use C because C++ template metaprogramming is too opaque, and, as further proof, my use of a non-standard preprocessor I implemented leads to 10,000's of lines of deeply nested macro invocations.

(There are all sorts of problems with make, but I'm not convinced the authors actually understand them in enough detail to improve on it.)

</comment><comment><USER32> This looks like a good stab at avoiding all the pitfalls of Make while still providing the same or better capability.

</comment><comment><USER33> It looks great but most of make-like tools are not functionnal. So yeah functionnal everything is better but nothing is functionnal right now.

<comment><USER34> Make itself is largely a functional/declarative language. Beside the recipes (which aren't Make but actually shell, though you can use Haskell if you want), the only non-functional feature I can think of are the global variables, which most makefiles generally don't modify in a non-lexical manner anyway.

<comment><USER33> I was talking about tool like rake in ruby and scons in python. they are not declarative at all. But most important is that if what your are building needs you to set env variable ect... I don t see how it could really be done in a functionnal way(without side effects).

<comment><USER35> Stick the ENV in a state Monad, and run the process with that ENV as the context?

<comment><USER33> There is no real monad in ruby. That why I feel like the article is missing the point.</comment></comment></comment></comment></comment>


Premium Pricing Just Doesn't Last - <AUTHOR> http://blog.atomicinc.com/2015/03/02/premium-pricing-just-doesnt-last-2347/

<comment><USER2> I believe engineers over-rate the cost/feature-consciousness of consumers and under-rate brand consciousness.

People do want to shop on price, but they want a discount on the brand they know. That's why they'll buy a laptop for $20 less on Amazon than at Best Buy, but they probably won't buy a non-name brand laptop.

I don't even think his examples are compelling. There were action cameras before GoPro, there are alternative cameras on the market now. Some have different form factors, some are cheaper, some are better. At the end of the day, GoPro is plastered all over every action sport competition and their daily video email gets millions of views. That's harder to replace than the actual tech - especially when the product itself needs to be redesigned every 18 months.

Also note that "come in under the incumbent's price" is a bad startup strategy. The incumbent has economy of scale & bulk pricing advantages the startup can't match. The only way to do it is to build a different product that is so much cheaper to make that you can squeeze the same margins in a lower price tier. If that's possible, then it means GoPro has shot over what the market needs. In that particular case, I don't think it is true today. It might be true someday.

In AWS's case, I actually think the competitive play is on the higher end. Everyone is trying to undercut AWS's price (generally by giving it away for free). I feel that the last 20 years has shown trying to get under Amazon or Walmat's price is a bad strategy. How can anyone get under their price at smaller scale without taking a "strategic" loss or having 0 margins? I'd rather compete on the high side by offering better docs / UI & UX / support and not be competing completely on the price of hardware.

<comment><USER3> > I believe engineers over-rate the cost/feature-consciousness of consumers > and under-rate brand consciousness.

I agree. They also tend to discount the value of an ecosystem around various high profile electronics goods. Apple and gopro are both amazing examples of that, there are hundreds of accessories for every iPhone and every gopro camera. Then there's the value of a known workflow, of positive word of mouth and, finally, a feeling of community. People would much rather ask a friend who owns a product about some issue than hit the Internet to find a solution.

GoPro and Apple both also tend to bring killer features to market first. Be it TouchID or 4K 30fps for $600.

<comment><AUTHOR> OK, but who pays for 4K 30fps at $600 other than pros?

<comment><USER4> People who read that it was awesome online. Or the same people who buy $4000 DSLRs when the $800 one has every feature they would ever use.

</comment></comment></comment></comment><comment><USER5> Tech changes faster than almost any other industry, which is great for premium brands like Apple as you can keep redefining the minimum baseline performance.

Remember computers in the hey-day of the PC? Every two years brought a huge tangible improvement. Programs made for new computers just wouldn't run on old PCs, or if they did would be unusably slow. That ended sometime around the release of the Core 2, except for niche users (gamers, developers, etc.)

It's the same thing for phones right now. A phone that is more than a few years old just feels like a piece of crap to use; it's slow, the screen is poor, and it doesn't support new features. Good times for manufacturers! (Though we are very rapidly getting to the end of this cycle, IMHO).

Same thing applies to AWS, brought up as an example in the post. The base price is not really the issue -- AWS prices fall slower than Moore's law and it is still more expensive that bare metal. It's the new toys you get every year that make AWS attractive. AWS keeps providing more and more pieces of your infrastructure at attractive price points with less admin overhead. At some point they'll have reimplemented everything the modern company needs and then they too will be subject to more serious competition.

<comment><AUTHOR> I never considered that point. Do AWS prices fall slower than Moore's law, and do they fall slower than the prices of the underlying equipment?

I have managed a goodly number of AWS implementations and considerations ("should we do it or not?"). AWS was cheaper in most cases not because of bare metal vs AWS, but because of the sheer efficiency of scale, and not having to hire staff and expertise to manage all of the infrastructure.

Hmm... you just inspired a new post. Thanks, noelwelsh!

<comment><USER6> > AWS was cheaper in most cases not because of bare metal vs AWS, but because > of the sheer efficiency of scale, and not having to hire staff and expertise > to manage all of the infrastructure.

Once you get to a certain scale, you're hiring admins either way. The only question is whether you're hiring AWS admins or you're hiring bare metal sys|network|linux admins.

<comment><USER4> AWS admins are cheaper and you need fewer of them. With AWS, you don't have to stand up and maintain an entire layer of infrastructure management tools; all you really have is config management.

You definitely have to hire a lot less staff with AWS, ESPECIALLY when you're in that "in-between" phase where you're starting to build out a tech ops organization. You don't have those step functions where you have a huge need for some vendor-specific technology knowledge, but not enough work to justify a full time employee. AWS provides an abstraction layer that you need to have anyway: the difference is that you don't have to _BUILD_ that layer, it was there from day one.

<comment><USER6> > AWS admins are cheaper and you need fewer of them.

I have yet to find an AWS Infrastructure Admin job for under $120K/year (SF, LA, CHI, Boston, NY were the locales I looked in). I _have_ seen quite a few traditional sysadmin jobs under $100K/year.

> You definitely have to hire a lot less staff with AWS,

This has traditionally been true of Linux admins in general. I'm not seeing that much of a difference in the marketplace between Linux admin and AWS admin compensation packages at the moment, except AWS admin positions are usually offered by startups and included equity you wouldn't get at a typical enterprise position.

Disclaimer: I do DevOps/AWS/Infrastructure/Etc.

<comment><USER4> Yeah, I started out my career as a system admin and the AWS infrastructure stuff always made me fear my job was going to disappear. In reality, it's just made people aware that the skillset most system admins have were closer to those of a software developer anyway. As everything moves to the DevOps model, I find that under DevOps, "AWS Admin" as a job goes away, and your developers all just know a lot about more about AWS since that's part of the underlying stack.

AWS is a lot easier for developers to conceptualize as well - they're just logical objects that can be created/destroyed like programmatic objects can. Used to be that if you needed a new server, you'd have to talk to your sysadmin and get a VM provisioned, keys issued, etc. but with AWS all you have to do is issue a method to a class in your framework of choice (since AWS has SDK libraries for most languages/frameworks), and the VM is created, provisioned and integrated into your application like no work needed to happen.

</comment></comment></comment></comment></comment></comment><comment><USER7> There are some counterexamples.

Smart phones are still driven by bling and snobbery so if you have a $900 phone you get $900 of snob value, an $1800 phone gets $1800 of snob values, etc. Same for "luxury" cars -- if you have a Cadillac it had better be your second or third car because you'll still need a ricer to get to work for the days your car is in the shop.

There is a vibrant market for cheap phones and a vibrant one for expensive phones but nothing in between because the main value expensive phones has is as a positional good.

Similarly, there is a lot of hating on tablets because tablets aren't connected with the cell phone plan boondoggle economy, but rather they are driven by economics and people just don't want to pay hundreds and hundreds of dollars. It isn't such an exciting biz to be in precisely because it is competitive.

</comment><comment><USER8> Eh I think this analysis is pretty shallow and misses some key points.

A. Is the product/service something approximating a commodity?

If the answer is yes, then certainly premium pricing is not sustainable. In the case of cloud services it's not quite a commodity, but ultimately if the [cloud] service meets the customers technical and service/support requirements the main differentiator is going to be price.

B. How high are the switching costs?

Once a customer has invested in a certain product or product ecosystem, how expensive is it for them to switch? With Apple vs Android the costs are not necessarily high, but it takes a decent amount of work to transfer media and adjust to new ways of doing things. For some customers that may not be worth it.

C. How big of a factor is branding?

See Apple, GoPro, Amazon, BMW, Mercedes, Coach, etc. With a physical product the brand is often a such large component of the buyer's decision that even if there is a cheaper similar product it[the cheaper item] doesn't even get consideration. A Hyundai Genesis has many of the same luxury features as a BMW 5 series but is much cheaper. But, probably 90% of the time, the person considering a 5 series is never going to even set foot in the Hyundai dealership.

In the IT context, the choice might be between a well-known software vendor and a much-cheaper alternative. But enterprise customers typically are going to go with the name-brand option, even if they don't need all the bells and whistles it provides. Many shops don't need something like Salesforce but they pay the premium just the same.

D. How price sensitive is the target customer?

Depending on the customer and the market, sometimes the customer doesn't really care if there is a cheaper alternative. They're happy with what they're doing and the price difference might be a rounding error to them so they just stick with what they've always been doing.

So yes, premium pricing can't last for some scenarios but that's hardly a rule.

</comment><comment><USER9> But where does Apple figure into that argument? I would say that their premium pricing was and still is worth the difference.

<comment><USER10> Arguably Apple is still only serving the high end of the market. They are in a unique position where the consumer does not do the math on the true cost of the iPhone. I also don't see a whole lot of college kids with rMBP's. Besides, who is to say that they are not going to run out of their premium price runway?

<comment><USER11> Probably depends which college you go to, but Macs are pretty popular on a lot of campuses: [http://askmeboy.com/wp-content/uploads/2014/08/college- class...](http://askmeboy.com/wp-content/uploads/2014/08/college-classroom- laptops1.jpg)

<comment><USER12> As far as the U.S. goes, around 1/3 don't go to college, and 1/3 go to community college. I doubt many of those kids are going for the MBP. I can't tell if you're using this as evidence for or against apple being a premium brand.

</comment></comment></comment></comment><comment><USER13> I would say the argument doesn't have anything to do with premium pricing and everything to do with lock-in.

In the case of AWS, prices are cheap. But once you've built your software to use all of the proprietary AWS services, it's not economically feasible to switch to another cloud provider. You're locked in.

This works similarly for Apple. You do pay more for the product, but it's not trivial to switch to a cheaper Android device after plugging into the Apple ecosystem.

GoPro clearly can't continue to win technologically (what comes after 4k?), and they don't have an ecosystem to lock you in.

<comment><USER4> > In the case of AWS, prices are cheap. But once you've built your software to > use all of the proprietary AWS services, it's not economically feasible to > switch to another cloud provider. You're locked in.

I would actually say that this is not a huge issue with AWS. Amazon tends to either use open protocols, or the industry adopts Amazon's open protocols (see S3). It's also their philosophy as a company that they want to win by helping their customers win. The folks at Amazon drink the kool-aid: from the top down, they legitimately believe that this attitude is their core competitive advantage. Bezos has been known to fire people for making moves that were profitable, but customer-hostile. He sees it as undermining the long-term brand value of Amazon in the pursuit of short-term profit.

Amazon's whole strategy here is really interesting: in markets where they can achieve economies of scale, they are totally willing to help lower their competitors' cost bases. That's because their competitors are now contributing to the scale that allows Amazon to succeed, and also helping Amazon to lower their own cost base. The ultimate idea is that if you always give the customer what they want -- even if that means helping your competitors -- you will be at the forefront of any industry you want.

<comment><USER14> Great points, but I would (and have - news.ycombinator.com/item?id=911252) argue that prices are not cheap, at least relative to costs. They are spending to develop some great services over top, but the perception that they are charging anywhere close to their marginal costs is purely successful PR.

Also, subtle point but I would add that any firing would be for moves _perceived as_ customer-hostile. Just looking into the EC2 Reserved Instances and the supposed secondary marketplace I see some less-than-consumer-friendly practices that seem to be just fine so long as nobody takes note.

<comment><USER4> Like you, I doubt that they are charging anywhere close to marginal costs because that's simply not how you run a successful business ( _especially_ a capital investment business like AWS). Price is determined by the market, and Amazon charges a price they think the market will support - no more, no less. If you think you can put a product out there at a lower price than Amazon, you're always free to try. It was a genius customer retention strategy to continuously drop prices without customers even having to ask - it's basically the opposite of what telecom companies do and it makes Amazon's customers love them.

As far as customer-hostility goes, you're right, there are some customer- hostile practices that happen just due to the nature of some of these B2B markets. But they're largely restricted to "expert" markets where you're expected to know a little bit more and prevent yourself from getting ripped off. There is an element of subtlety to this, but the idea is that you shouldn't be trying to leverage your customers' ignorance in exchange for a big pay day. I'm sure there are small instances of individual marketing managers getting away with things that are against the "Amazon code", but by and large those practices stop as soon as the product is big enough to make it on executives' radar.

<comment><USER14> Agreed and especially like your points around expert markets about the executives' radar.

Just a to build a bit on the pricing point: Outside of perfect competition, price is not determined by the market. It is determined by executive or committee or some agent of a company subject to the market, specifically 1) observed and expected competitors' prices and 2) the company's estimate of the demand curve composed of all possible customers & workloads.

<comment><USER4> Right; I meant only that cost has no bearing on the price set by the company. The two things you mentioned (competitive landscape and demand forecasts) are both ways of measuring the market.

Marginal cost only plays into the equation when the marginal cost of the product is higher than the maximum price/pricing structure the market is willing to bear over the time horizon you're considering. Amazon just has a longer time horizon than most companies.

</comment></comment></comment></comment></comment></comment><comment><USER15> I disagree. GoPro is not Premium. For me GoPro is cheap for what it does.

I had bought Sony and Canon DSLR in the past. THIS IS PREMIUM. Several thousand dollars each.

My Gopro does things NO OTHER CAMERA in the thousands category could do. In fact , the people at Canon, Nikon, and Sony disable features on the hundred dollar category so it does not compete with the thousands.

E.g You want instantaneous electric controlled shutter on your NEX X Sony camera? Sorry, you need to pay thousands of dollars. You want uncompressed HDMI video output? Same answer. They have an HDMI port!!, they simply disable it!! They provide you with crappy analog output.

With my GoPro, I can do it all way cheaper, smaller, lighter, great video, and rock solid. Connecting it to a HDMI grabber provides amazing quality. You could stabilize it with cheap motors as it is so lightweight. Control via the Web?,mobile phone?with my own apps? no problem with the GoPro.

I love this camera so much, and hate the others. If Gopro decided to make a DLSR, I would buy it. The other companies could burn in Hell, they had abused so much.

<comment> saiya-jin good for you to be such a fan, for one I am not, although owning their devices.

Overpriced products, some botched updates (yes, most companies screw up more than once, but that's not an excuse), not really caring that much for customers (cases oficially usable under water were only included in Hero 3, which is roughly +-5th generation, till then they were just waterproof & completely out of focus).

Image quality is a joke compared to recent cell phones. Price wise their top model is ridiculously expensive, without delivering anything special (ie better picture quality).

Battery life is bad, and no sign of improvement (apart from PR talk every single release). THis is exactly the type of device needing long battery life, since it's often in environments where switching battery is impossible/very hard, and often being exposed to cold.

They are chasing 4K video and ridiculous frame rates, but picture quality was never paramount (problem is, competition is worse). This is niche interest, most people don't need nor want 1080p / 120 fps. For nice slow-mo movies, you need anyway much higher framerate (at least 240 fps).

They have good marketing, no doubts. As for products, not so much. I repeatedly end up with sundown images/videos being one ugly orange- oversaturated mess, that cannot be fully corrected in post processing (and why the hell should I? Even my 4 year old phone can cope with it automatically).

I was recently considering whether to go for top of the line GoPro or buy small but good camera. Usages - underwater, climbing & mountaineering side camera. GoPro lost on all but small form factor, and I bought Canon S110 with underwater housing. It costed me same, picture quality is completely differetn level, and form factor is sometimes actually better with Canon.

If you look around, there is quite a lot of people disappointed in them. But as I said, they have good marketing.

<comment><USER15> I am quite emotional because the people at canon and Sony had f*cked me so many times in the past.

The Gopro has solved most of my problems. I am a geek and use my cameras for crazy things like 3d recognition of spaces, with my drones, robotics.

The GoPro is the camera that gets in my way the least of all the cameras I had tested.

Image quality for me is quite good. But nothing replaces basic knowledge of video making that probably would solve most of your issues.

When you say they have good marketing... well their videos are made by professionals. Any pro with any camera could do marvels, but with some cameras takes more effort than with others.

</comment></comment></comment><comment><USER16> As it says, "If there is one truism in the technology market, it is that premium pricing just doesn't last."

It definitely has one aspect of a Truism [1] - no-one really looks at the underlying assumptions. The biggest assumption in much of the tech industry is that you're aiming for mass market - the consumer tech brands like Facebook particularly have that assumption - other people follow sometimes without understanding the implications.

If that's your focus, and you have limited differentiation your pricing has to reflect that over time replacements will come in and undercut you. That's the GoPro example he's using, where the parameters of his comparison are the "components of the hardware camera".

What he's missed is that Apple and plenty of others (think every car brand) have shown that brand and real differentiation can prevent new entrants from being able to provide an equivalent that is good enough. The point is that the customer has to believe that product A and product B are comparable - and that the parameters may not just be the bits or atoms.

For example GoPro's brand speaks to "action sports" and being active, they have software and communities that people want to be in. It's perceived as something that 'active people' use and has brand cachet. So the 'replacement' has to capture all that value, not just the hardware costs. Frankly, they've absolutely eaten the lunch of much bigger hardware companies who have far more economies of scale.

The AWS example is even less clear cut. Since Amazon is a retailer we can assume their strategy is mass take-up, and we know that their general pricing strategy is for people to "believe" they are cheapest. But, we also know that Amazon practises differentiation, we've seen that with Kindle and Amazon Prime. If you look at the AWS services you can see they are aiming to provide a highly differentiated service aside from "simple" IaaS. We also know that as a retailer Amazon uses extremely sophisticated pricing strategies. At this point I would think that Amazon is continuing to drive down the cost of basic IaaS due to economies of scale which makes it very difficult for competitors to get started, while keeping the cost up of 'value-added' services which provide differentiated service and make the whole service a lot more sticky. They will probably do Amazon Prime like things with their whole portfolio over time.

There are plenty of businesses where the strategy of being cheapest is right, but it's not the only approach.

[1] [http://en.wikipedia.org/wiki/Truism](http://en.wikipedia.org/wiki/Truism)

</comment><comment><USER17> Upvoted, not because I agree, but because I would like to see discussion on this. I just bought 2 Mobius Action cams for $70 each and they are awesome. On the other hand someone doing any sports worth recording probably won't even think about buying anything else but the best accessories.

</comment><comment><USER18> You can add Beats to the list - a low barrier industry ripe to undercut a premium priced brand, but it has not done so.

There is an emotional factor, where customers associate their own identity and how they see themselves with the brands. That is what allows premium pricing to last.

</comment><comment><USER19> As long as it's just a better product, there will always be those willing to pay a little more. Even if the product is part for part the same as a competitor's, better customer service might still warrant a premium price. Not having to go through the hassle of dealing with a horrible RMA process for example, is worth the extra money to me.

The nice thing about customer service, is that its monetary value is very hard to quantify, making objective comparison difficult. So there's always room to charge extra.

</comment><comment><USER20> The same holds for many things. For instance, software licenses. If your license is restrictive, then no matter how good your product, eventually your competitors will catch up with a "good enough" product that has a less restrictive license.

Therefore, open source licenses will eventually converge to permissive, like MIT or BSD style licenses.

<comment><AUTHOR> @amelius, I haven't though of software from that perspective. App servers, Web servers, databases and a lot of middleware have gone down that path.

Have you written it up? I would like to see a write-up on that.

<comment><USER20> No I haven't, but feel free to do so :)

<comment><AUTHOR> I wrote one. [http://blog.atomicinc.com/2015/03/11/licenses-as-premium- pri...](http://blog.atomicinc.com/2015/03/11/licenses-as-premium- pricing-2387/)

Feel free to post to HN if you like (or if you dislike and then comment :-) )

</comment></comment></comment></comment><comment><USER21> Many premium brands have been around for years. Rolex, Coach, etc. Sure, trends change but there are always going to be people willing to pay extra for the status of having a premium brand.</comment>


Fat Models – A Django Code Organization Strategy - <AUTHOR> http://redbeacon.github.io/2014/01/28/Fat-Models-a-Django-Code-Organization-Strategy <comment><USER2> Fat models are slightly better than having that crud in your views, but they're still an antipattern. Models should describe the relationships between entities in your business, not be the junk drawer for stuff that doesn't go anywhere else.

Fat models are already a code smell, but passing the request object to your model should really set off alarm bells. You've now made it extremely messy to use your models outside of a web context, making it impossible to unit test them, and pretty much wiped out the purpose of the controller as an encapsulation layer.

Domain-driven design (DDD) provides a lot of patterns that help to keep from either having model code in your views, or these types of junk drawer models.

Basically, DDD as it applies to MVC webapps is: If it's a complex query of some sort, create a query repository that returns model objects rather than a "fat" static method on the model itself. If it's a business transaction, create logic objects that encapsulate the biz logic, and service objects that applies the logic to the models. The controller serves as the broker between requests, services, and entities. You end up with much more testable and reusable code this way.

<comment><USER3> The article precisely says you should not pass the request to the models.

<comment><USER4> That doesn't make it any less of an antipattern. The "fat model" pattern is essentially "if you can't figure out where it goes, put it in the model." In the article's example, the credit card processing code is wrapped up into the quote model. I can't think of a single situation in which this is a good idea.

<comment><USER5> The "fat model" pattern is simply organising your logic and relationships into the model layer. How well you organise it is up to you. If you just dump things from the controllers into random parts of the model then that's your choice but there are plenty of ways to design it well.

<comment><USER6> I'm closely involved in a big system that uses Catalyst (perl's nearest equivalent to Django). We spend allmost all of our time in Model classes (MyDomain::Model::Whatever, not MyWebApp::Model::Whatever that's just a dumb connector). It works well and I get the shits every time I have to deal with code that doesn't separate this stuff out properly. Also works on the front end using modern javascript.

<comment><USER7> I have most of my logic in the views (because I didn't now any better when starting my project). I know the Django way suggests fat models (well I know that now). But is it honestly that big of a deal? I know in my app that any complex logic will be in the views. What will I gain by following these best practices?

<comment><USER6> Yeah, try having a fat controller or view, on a rapidly developing codebase with poor specs which change mid-project regularly while avoiding getting fired.

<comment><USER7> Sounds very like my work.

Yeah, schema changes are a pain in the arse, but I don't see that moving my code from views to models would have a great deal of impact (though I am gradually moving the code that way when it does need changed).

<comment><USER6> A better answer is, that you get to decouple the functionality of your application's business logic from the web portion. This is achieved trivially purely by applying disciplined coding standards. Removing the web dependency makes the code much much easier to write automated tests for, and it gives a sane development path when making minor or major changes to feature sets.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> I write my code as if there will be multiple user interfaces. Usually there are not multiple user interfaces, but thinking as if there will be has a positive impact on the code. Thinking as if there will be multiple user interfaces means putting as little code as possible into the views, since there will be more than one. Some of those user interfaces might not even have Django views and templates, such as a service exposed via Zero MQ. This naturally leads me to putting as much logic as I can either into the models, or sometimes into other modules which in turn use the models. About the only code that goes into views then is the glue that looks things up and gets things ready to be passed into a model.

</comment><comment><USER9> I can only shake my head in disbelief when I heard Rails or Django developers touting their framework and saw their actual codebase (ever see FatCRM codebase back in the days? it has been improved significantly these days, but it used to be ... hard to read).

I even shake my head faster when I read the split in opinions in Rails world: the Rails DHH and the DCI/Service Rails.

Moreover, I keep shaking my head when the Django and Rails community voiced their complain over their so-called bloated framework just because they want an API service.

Last but not least, I shake my head again whenever I heard that Django and Rails is not like NodeJS.

C'mon guys... you guys speak a lot of bad things about JAVA yet you've become Java Enterprise Edition.

I suppose JavaEE (especially 6 and 7) does work better in terms of writing maintainable code.

Stuff like Service pattern, Repository pattern, Data mapping pattern, DTO, Value Objects, Mocking are pretty much "common" in most Java projects I've seen or worked on.

I felt that no matter how cool Rails and Django are, when it comes to medium- to-large codebase, Rails/Django are definitely either following the patterns that pretty much common in JavaEE or may not be so comfortable to use anymore.

</comment><comment><USER10> I honestly think this guy doesn't get it.

The proper way to separate concerns in Django is through applications, which have their own URL routes, models, and views (although they can invoke any other thing in another app).

Through Class-based views you can define base views for common behavior for templates and basic logic, and the built-in views are usually good enough to contain most patterns and easy enough to customize for any other use case you might think of.

A good rule of thumb is that an application should generally have less than a dozen views (I'd say 5 or 6 is enough already). Since each app has its own forms and models file, adding utility methods in an extra file contained within an app is reasonable.

I've used this to manage Django apps with over 50.000 lines of custome code (not counting several dozen extra apps and admin sites with their own customization) and it has never been a problem.

Fat models are just a massive problem. If Python supported extension methods then perhaps you could load your own extensions when performing an action, but a proper design of forms and views that aims for maximum simplity has never failed me.

</comment><comment><USER11> I would stick with no classes being "fat". Breaking up fat classes into many smaller classes leads to code being really easy to understand, test, and develop. I tend to stick with the Single-Responsibility-Principle and classes do not become unmanageable. Some people argue that many small classes leads to complex code, but I have not found that to be true in any case I have come across.

<comment><USER12> Ah, but there be monsters in that sea too. I used to subscribe to the many- focused-models camp, but depending on the relationships you're trying to model, that can become damn-near unmaintainable as well.

I've actually begun building out more "wide" models, that is, models with a great many fields set to allow null/blank, because mixins and abstract base classes are very hard to maintain 3-4 years out. Don't misunderstand me, there's a place for those, but it's easy to get lost on road to abstraction.

Of course, the lesson is that if you're representing complex relationships in code, the implementation is going to wind up complex to some extent.

<comment><USER5> There's another option besides mixins and base classes, composition. You can break your fat model into lots of smaller classes that you aggregate into your original model. The big advantage over using mixins is that the scope is kept much smaller. Keeping scopes small in general is always a good way to scale up to a large codebase (functional languages get this pretty much for free).

<comment><USER11> Yes, I generally stick to composition. "High level" classes end up using a bunch helper classes to perform specialized tasks. I find that I do not often find the need for an abstract base class or mixins, but I will use them if it makes sense.

</comment></comment></comment></comment><comment><USER13> Not sure I agree with fat models being a "rarely seen, alternative code organization strategy".

It's a natural state which most django codebases tend towards. There's a natural Views use ModelForms use Models data flow which leads to bloat somewhere in that chain as you add code. Fat models are easiest to test and the natural place to put most things, but eventually massive models get unwieldly and you start putting things into helpers. The core thing the author seems to have conflate util functions that have access to a request and helpers that have no connection to the request/response cycle.

<comment><USER14> In my experience, fat models are the most common approach to organising code in Django projects. As another comment stated, it's better than having fat views, but it's not ideal.

</comment></comment><comment><USER15> "Fat" abstract models is the way to go if you have logic that operates on the data. It's not even fat. There were fields assigned values in the example. That is exactly what model methods should do.

"Fat" abstract forms is the way to go if your forms have crispy layouts, placeholder initializers, or other stuff that that directly has to do with forms.

Views can be easily split into categorical files and shouldn't contain much logic other than bringing models and presentation together.

Sounds like thinking about what you're doing is the way to go to not have code get out of hand.

</comment><comment><USER16> Maybe you guys can help me apply this to an actual Django project I'm working on.

So I made an "Order" model to track orders. But now I have to add a lot of logic regarding things such as what can be ordered together, and which users can order what.

Where you keep that logic? It seems to make sense to me to put it right in the model, no?

<comment><USER17> Personally I would make models a package rather than just a module (since you sy you have a lot of order related logic). I would create one or more modules in this package, presumably one of those for orders.

Then I would actually create the base logic which clearly belongs to one Order instance as methods on the Order class.

If you have something like "can x and y and (...)" be ordered together I would make an unbound function in your orders module which takes an iterable (Lists etc.) of Orders and works on it.

Note that I am by no means a Django guru or something like that.

</comment></comment><comment><USER18> There is no "one right way", but when thinking about code organization and architecture I find inspiration from the "clean architecture"[1] and similar approaches. The dependency rule can really turn your world upside down!

[1] [http://blog.8thlight.com/uncle-bob/2012/08/13/the-clean- arch...](http://blog.8thlight.com/uncle-bob/2012/08/13/the-clean- architecture.html)

</comment><comment><USER19> Does anyone else feel weird knowing that when you call that model method it may or may not call out to a background worker task?

Why not just throw all of your celery tasks that are related to an app inside of a worker.py file or something else?

If you import from service.py or worker.py then it becomes obvious the thing you're doing is happening in the background.

</comment><comment><USER20> Is this article pointing out the most common way of organizing code in Django as an advice?

I'm such a failure at marketing ... sigh</comment>


How Marc Benioff of Salesforce.com Became The Most Valuable CEO Of All - <AUTHOR> http://blogs.forbes.com/stevedenning/2011/04/14/how-marc-benioff-of-salesforce-com-became-the-most-valuable-ceo-of-all/ <comment><USER2> Salesforce.com is the COBOL of Web 2.0. I find their success story riveting.</comment>


How I wrote Game Programming Patterns - <AUTHOR> http://journal.stuffwithstuff.com/2014/04/22/zero-to-95688-how-i-wrote-game-programming-patterns/ <comment><USER2> This was a great post. I enjoyed the book, and it's always interesting to hear about a writer's process. Everyone tackles the big challenge in a different way.

The only detail I disagree with is the one about telling your friends and family what you're doing as a way to keep motivated. A few years ago I read a paper about the psychological effects of talking about goals. [0]

The takeaway was that, for most people, sharing a goal provides a sense of satisfaction and self-validation. While it is pleasing, the sensation makes us significantly less likely to accomplish goals such as writing books, losing weight, etc.

Again, I enjoyed the post very much and don't mean to nitpick. I just think this is another interesting perspective to consider when you're setting out to tackle a big goal.

[0] Unfortunately, my memory is failing me, but if anyone knows of the paper - or an article or paper along these lines - please feel free to contribute.

<comment><USER3> > _Unfortunately, my memory is failing me, but if anyone knows of the paper - > or an article or paper along these lines - please feel free to contribute_

I believe it's normally referred to as the "Substitution effect", there's a paper by Gollwitzer _et al_ called "When Intentions go Public"[1] that might be the one you're thinking of?

[1] [http://www.psych.nyu.edu/gollwitzer/09_Gollwitzer_Sheeran_Se...](http://www.psych.nyu.edu/gollwitzer/09_Gollwitzer_Sheeran_Seifert_Michalski_When_Intentions_.pdf)

</comment></comment><comment><USER4> Awesome post. I'm finishing my third book, and totally recognize the agonizing over whether the book will ever get done!

On not using a conventional publisher: my first two books were with mainstream publishers (Cambridge University Press and Princeton University Press). For my third book I decided to self-publish online only, because that makes it possible to integrate animations, interactive Javascript, and so on. Four publishers have asked about publishing paper editions of the book, but it's hard to imagine going back to the printed page as the primary target medium. Having the web as the primary target is so much more powerful.

<comment><USER5> Michael Nielsen as in "Quantum Computing and Quantum Information"? If so great book.

<comment><USER4> Thanks!

<comment><USER6> I'm just replying to you so I can say to people that I've replied to you :)

Nielsen & Chuang is one of the bibles of my universities physics dept.!

</comment></comment></comment></comment><comment><USER7> For anyone wondering about the book itself: It covers a lot of very practical, nuts-and-bolts challenges in structuring a game's codebase. It doesn't succumb to the temptation of architecture astronautics.[1] It's more like, "I'm about to type out a class that handles keyboard input. How should I write it?" For that reason, I found it to be one of the most helpful references on game programming I've yet discovered.

[1] [http://www.joelonsoftware.com/articles/fog0000000018.html](http://www.joelonsoftware.com/articles/fog0000000018.html)

</comment><comment><USER8> This was a fantastic read! Congratulations on finishing your book.

Ever since John Resig wrote about Write Code Every Day [1], I've fallen into the habit of writing some code on a new side project of mine. Every day, at least 30 minutes of work. Not saving up things to do for the weekend makes it a lot less stressful and has increased my productivity significantly. And whatever I've been working on is constantly on my mind instead of put aside until the weekend or some other period of free time. I'm definitely reaping the benefits of not "breaking the chain."

[1] [http://ejohn.org/blog/write-code-every-day/](http://ejohn.org/blog/write- code-every-day/)

</comment><comment><USER9> Thanks for taking the time to write this post. It was incredibly motivating to me. A weird side effect is that I now have a scanner in my Amazon shopping cart. Once it arrives, I will start a chain where I doodle something every day, scan it and post it online. I am going to read your book so I can learn more about programming patterns in general—I'm not so interested in programming games yet. Reading the first few chapters, I can tell I'm going to learn a lot.

Also, you can put your own animal drawing on your cover now—and you get to choose which animal!

I'm looking forward to reading new posts about how you need to call your bank every few days to make more room in your account due to all the book sales. Congratulations on getting to done.

<comment><AUTHOR> > It was incredibly motivating to me. A weird side effect is that I now have a > scanner in my Amazon shopping cart. Once it arrives, I will start a chain > where I doodle something every day, scan it and post it online.

This is awesome!

I used to draw (hence the hand-drawn illustrations in the book), but it's one of the many hobbies that didn't make the cut after I had kids. As someone who spends way too much time staring at a screen, there's something really therapeutic about actually working with my hands in an analog medium.

<comment><USER9> I started out as a graphic designer as well, then gradually moved into programming. I have four kids (evenly spaced between 2 and 11), so I know how you feel about extra time for fun hobbies like drawing. (we are a few weeks away from our first dog too.)

Your post reminded me that I shouldn't be spending all of my time in code working on client projects. Yes, it pays the bills, but I should (need to) do a little drawing each day to keep that part of me alive. It's not as big as writing a book, but it will be fun to see how long I can keep the chain going.

I also agree with your comment about screens vs. paper—I have one of those fancy Wacom drawing tablets, but it doesn't come close to the feel of a nice sharp 4H pencil or Pilot v5 pen on copy paper.

Thanks again for the inspiration.

</comment></comment></comment><comment><USER10> I had been wondering if writing a book on GitHub in markdown was a legitimate idea or just based on my inexperience with other writing tools. I still don't know, but the author's work has convinced me that's how I want to do my next writing project.

<comment><AUTHOR> Yes, I really loved the workflow I ended up with. Putting it on GitHub has been hugely helpful. I merged four pull requests and closed twenty bugs this morning. How awesome is that?

<comment><USER10> It really is wonderful to work on documents with git and markdown is the perfect pairing of text format. There seems to be a huge opportunity waiting for someone to make a GitHub marketed to non-techie writers.

<comment><USER11> if you already know git (or github), it'll work for books.

but i doubt it's the best change-tracking system for them.

too much irrelevant crap, and it doesn't cut to the heart of those things that actually need to be done for a book.

plus, most writers will never slog through learning github.

to make a system that is writer-friendly _and_ does what a book-writing change-tracking system ideally _should_ do, you'd have to hack github so much that -- in the end -- it would have been order-of-magnitude simpler (and better) to program a purpose-built system instead, from scratch.

and yes, i told loren this from day one.

he didn't believe me then.

but if you ask him now, i'll bet he won't be quite so sure.

and by the time he's done, i have no doubt he will agree.

yes, writers need a version-control system.

but no, it is most definitely _not_ github.

-bowerbird

<comment><USER12> What you say is interesting, but how much time would he have lost before he could launch if he had to write a system from scratch?

On the other hand when using something ready like git allowed him to launch quicker. If he gets traction he can always write a new system later, if not then he has saved up on a lot of time.

<comment><USER11> perhaps you meant penflip's loren, not the original poster?

if so, then yes, what i'm saying exactly is that _loren_ could have saved himself time and trouble, in the long run, by coding a purpose-built system, instead of hacking github.

what it would've cost him, however, is the huge "jumpstart" he received by proclaiming his goal as "github for writers".

the kneejerk response to that, from the tech circles, was something that created huge word-of-mouth and mind-share, not to mention a great starting user-base, of technoids.

but that was months ago, and now there's some sense of "whatever happened to penflip?" and when he goes out to sell the system to writers, the "github" label will not be a benefit at all. (if anything, it will be a liability; but i think in general writers are fully unaware of it.)

all of this is my opinion. you, like loren, may disagree.

-bowerbird

</comment></comment></comment></comment></comment></comment><comment><USER13> Wow, this is the best insight into the struggles of writing a book I have read. I started a book and quit on the first real hurdle. This post really stirred up something in me - about finishing something for the first time in my life. Thanks for sharing.

</comment><comment><USER14> This is extremely inspiring. I've been on a streak ever since I read John Resig's post and your comments about your book. I'll be writing code for my side project every day, at least 30 mins. I don't know what it is ... But I'm really freaked out of breaking the chain, it keeps me really motivated. I love it.

</comment><comment><USER15> This book looks really cool and I've always been interested in reading more about game programming. Is there a way to grab this as a pdf instead of only- online content?

<comment><AUTHOR> Not yet, I'm working on that now.

<comment><USER15> awesome thanks for the answer! I subscribe to your ebook mailing list, would be awesome to have a update about the pdf through that funnel!

<comment><AUTHOR> > I subscribe to your ebook mailing list, would be awesome to have a update > about the pdf through that funnel!

That's one of the main reasons the list exists. :) The day the books are ready, you'll know.

</comment></comment></comment></comment><comment><USER16> > The last game I worked on, Henry Hatsworth in the Puzzling Adventure

Ah, you're that guy! I bought HHitPA because I read a reddit post by you (presumably) about it.

</comment><comment><USER17> Great post and insight. I'm excited to start reading your book.

</comment><comment><USER18> Just wanted to say nice work. I was browsing GPP the other day and I found it to be a valuable reference, and not just for game programmers.

On a different note, welcome to the Pacific Northwest!

<comment><AUTHOR> Thanks! Moving to Seattle was weird: I felt like it was the place I'd always wanted to live even though I barely even knew it existed until a few years ago.

</comment></comment><comment><USER19> Reading it now. Thanks for sharing. I have a couple of book projects I've been pecking away at for a couple of years. This will help.

Please consider changing your site so it can be resized (two finger zoom) when viewed on an iPad. The font is on the border of being small. I often do my reading late at night. When tired I find it easier to make pages fill the screen side-to-side in landscape mode. Thanks.

</comment><comment><USER20> This is somewhat off-topic, but if the author is still reading these comments: in the chapter on bytecode, it seems that an obvious alternative was overlooked. Don't implement your own little language. Take one that's already there. Lua, Python, JavaScript, there are lots of them! Why not?

<comment><AUTHOR> I think it mentions Lua and a few others. I think the chapter is equally applicable to those too. It's as much about how to implement _your_ language as it is about how _those_ languages were implemented.

(In particular, I learned much of what I know about bytecode VMs from Lua's implementation.)

<comment><USER20> Yes, you mentioned them. But what surprised me is that you never suggested using them, instead of implementing your own. I understand that sometimes implementing your own is the way to go, but surely sometimes it makes sense to just use Lua?</comment></comment></comment>


European Commission fines Mastercard €570M - <AUTHOR> http://europa.eu/rapid/press-release_IP-19-582_en.htm <comment><USER2> You just gotta love Margrethe Vestager. She keeps up the good fight against corporate greed and crime. Hopefully she'll be re-elected as Commissioner this year (not unheard of, considering the awful Goettinger was also re-elected a couple of times, although in different roles).

Also now I know why the recent news about Mastercard stopping service providers from auto-renewing your payments without your consent felt so "strange" (like why would they willingly hurt their own revenues like that _strange_ ). It was most likely a strategic positive PR story to come out ahead of this news.</comment>


Ask HN: Better to get another bachelors or go for an MA in an unrelated field? - <AUTHOR> I graduated with a bachelors in English and Philosophy. Thinking about going back to school and pursuing aerospace engineering. Is doing the math and science pre-reqs outside of a degree program and then just going straight into a MS program possible, or am I going to have to go back and get a BS again? <comment><USER2> The Masters program should list which prerequisite courses you need, and you might be able to take them at a local community college. You'll definitely want to schedule a chat with an admissions counselor though, as each school is different. But they can tell you definitively, before you do all the work for nothing.

</comment><comment><USER3> I recommend to read some articles about student debt and take that also into consideration.

<comment><AUTHOR> yeah the debt is why I'm so skittish about going back to school. I was fortunate enough to come out of undergrad w/o debt the first time, but didn't have enough foresight to use those four years to hone technical skills rather than read Derrida lol</comment></comment>


Is Eurostar's direct Amsterdam service quicker than flying? - <AUTHOR> http://www.independent.co.uk/travel/news-and-advice/eurostar-london-amsterdam-direct-train-service-faster-flying-plane-a8212391.html <comment><USER2> If you came to the comments for a quick yes/no: The question is not answered in the text or video.</comment>


Am I Being Boring? - <AUTHOR> http://www.drmaciver.com/2009/10/am-i-being-boring/ <comment><USER2> I do this all the time! Usually I get this funny feeling when I am writing a paragraph which is unclear/going all over the place. I keep the idea I am working on in that instant in my head. Then I delete the entire paragraph and start again.</comment>


Ruby 2.1: Out-of-Band GC - <AUTHOR> http://tmm1.net/ruby21-oobgc/ <comment><USER2> It would be nice to get these patches in core as part of ruby 2.1.1 so others don't need to patch & recompile ruby from source.

Other that, anything that improves performance is a welcome change.

<comment><USER3> I think Aman already proposed those changes to ruby-core - [http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby- core/...](http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/59728)

He is a member of ruby-core too, so I guess if enough people in core like these patches they should go right in. However I don't see OOBGC patch there in that list.

<comment><USER3> I have not been reading carefully, just noticed that OOBGC stuff is part of - [http://rubygems.org/gems/gctools](http://rubygems.org/gems/gctools)

</comment></comment></comment><comment><USER4> Very nice to see improvement in Ruby's GC. Back in Ruby 1.8 days, I read the source of the GC implementation and I was less than impressed. I should take the time and revisit that code to see what kind of improvements have been done.

<comment><USER5> In that time frame, the interpreter has been entirely re-written, and we've gone through three or four different garbage collectors.

So yeah, it should be quite different.

</comment></comment><comment><USER6> MRI Ruby seems to be making a lot of progress in the implementation details recently. Good to see.

<comment><USER7> Anecdote time!

I've been hacking on a little scripting language[1] lately. To see how its performance compares, I run a few benchmarks[2] against other similar (dynamically typed, bytecode compiled) languages: Lua, LuaJIT (interpreted), Python, and Ruby.

Like others, I had internalized "Ruby is slow" through osmosis. But the version of Ruby I happen to have on my machine is 2.0.0.

In my little benchmarks, it turns out Ruby is one of the fastest. I haven't compared to 1.8.7, but I'm guessing that was much slower.

[1] [https://github.com/munificent/wren](https://github.com/munificent/wren) [2] [https://github.com/munificent/wren/tree/master/benchmark](https://github.com/munificent/wren/tree/master/benchmark)

If you want the gory details, here's the results of running them against Lua 5.3.2, LuaJIT 2.0.2, Python 2.7.5, and Ruby. (Yes, I should try against Python 3. I will.):

                              score time   wren score relative
        binary_trees - wren   3193  0.31s
        binary_trees - lua    1366  0.73s  233.72%
        binary_trees - luajit 6256  0.16s   51.04%
        binary_trees - python 1376  0.73s  231.96%
        binary_trees - ruby   3003  0.33s  106.34%

        fib - wren            3078  0.32s
        fib - lua             2785  0.36s  110.52%
        fib - luajit          6900  0.14s   44.62%
        fib - python          1331  0.75s  231.37%
        fib - ruby            3548  0.28s   86.77%

        for - wren            6080  0.16s
        for - lua             1990  0.08s   50.71%
        for - luajit          5914  0.02s   13.24%
        for - python          2825  0.35s  215.25%
        for - ruby            6595  0.15s   92.19%

        method_call - wren    4707  0.21s
        method_call - lua     1674  0.60s  281.17%
        method_call - luajit  4221  0.24s  111.53%
        method_call - python   767  1.30s  613.69%
        method_call - ruby    3061  0.33s  153.77%

As you can see, Ruby fares quite well.

<comment><USER8> " _dynamically typed, bytecode compiled_ "

I haven't followed Ruby development, but the last time I heard, Ruby 1.8 and prior was a tree-walking interpreter---simple to implement but not fast at all.

<comment><USER9> This changed in 1.9. Ruby is now compiled into a stack machine based bytecode. Ruby 1.8 used the AST internally, you're right there. This made e.g. method calls very slow compared to the new byte code representation.

</comment></comment></comment></comment><comment><USER10> We've now added support for the Ruby 2.1 Out-of-Band GC in Phusion Passenger: [http://blog.phusion.nl/2014/01/31/phusion-passenger-now- supp...](http://blog.phusion.nl/2014/01/31/phusion-passenger-now-supports-the- new-ruby-2-1-out-of-band-gc/)

</comment><comment><USER11> As someone woefully uninformed about these things, why can't GC be implemented as a separate thread, maybe with a lower priority than the primary interpreter? Would a separate thread not be able to count references to objects or something?

<comment><USER12> Quoting ko1 ([https://bugs.ruby- lang.org/issues/8339#note-11](https://bugs.ruby- lang.org/issues/8339#note-11)):

        > Parallel tracing needs an assumption that "do not move (free) memory
        > area except sweeping timing". Current CRuby does.
        > For example: "ary << obj". Yes, the CRuby's memory management strategy
        > (assumption) is different from normal interpreters.

<comment><USER13> Isn't memory only shuffled around during compaction? Why not mark/sweep in parallel, then stop-the-world at compaction time?

<comment><USER14> In a nutshell, generational GCs works by scanning live data. Thus, anything not scanned is garbage. While the application is running, it also allocates new data.

If you allocate an object somewhere --after-- that area has been scanned, the GC will treat that area as garbage, and then you will have memory corruption, leading to segfaults and other nice things.

While parallel mark/sweep is certainly doable (Java does this) it's not easy to get right.</comment></comment></comment></comment>


The Web Centipede: Understanding How Web Communities Influence Each Other - <AUTHOR> https://arxiv.org/abs/1705.06947 <comment><USER2> The emergence of mainstream social networks like Twitter and Facebook makes it easier for misleading, false, and agenda driven information to quickly and seamlessly spread online, deceiving people or influencing their opinions.

There has always been misleading information. Local TV news promos always tell you to stay tuned to hear about x at 6pm only to find out x really meant y.

What we see now is the decentralization of media and it's influences.

<comment><USER3> Hopefully the fact that deception is now obvious will cause people to re- evaluate the way they use their trust networks. I hope in the future that people use trust graphs as a way to estimate the reliability of a claim they see from an unknown party.

<comment><USER4> A trust graph is essentially an ad hominem though.

What if we instead could provide the tools to establish a soundness graph. Evaluate arguments on their own merits rather than their source.

Even Arguments based on false premises can be sound. Just as arguments can be based on true premises and still be unsound. Helping people identify which is which should at least raise the quality of disagreements.

<comment><USER5> _Even Arguments based on false premises can be sound. Just as arguments can be based on true premises and still be unsound. Helping people identify which is which should at least raise the quality of disagreements._

When its premises are false, an argument is always unsound. But it can still be valid. A sound argument is one that is both valid and has true premises.

About builiding a soundness/validity graph, I've dabbled with building a webapp for that (though the graph is more implied than visual). It's still very basic, but if someone has ideas where exactly it should go or how it could be engaging to a community of critical thinkers, please contact me.

[https://arguably.herokuapp.com/](https://arguably.herokuapp.com/)

[http://github.com/gregoor/arguably](http://github.com/gregoor/arguably)

<comment><USER4> Thanks for the correction. Will certainly have a look at the app

My own thinking is that it has to be less of an app and more of a protocol/federated thing augmenting existing channels. Think something like a github bot doing automated reviews with an SO-like community of meta-data authors annotating news articles and such

</comment></comment></comment></comment></comment><comment><USER6> > Our results indicate that alt-right communities within 4chan and Reddit can > have a surprising level of influence on Twitter

It's also common for 4chan users to have accounts elsewhere, just to force memes or opinions. It's super easy to create multiple Twitter accounts and just start posting away.

You can't do this on 4chan with the same level of effectiveness. You can't target a user directly, and there's a lot of obscure culture that you will have to understand to properly communicate and not be outed as a shill by the community. Plus, there's no incentive to post for people who are used to Facebook, Twitter, Tumblr, etc. You're not gaining followers, you'll never get upvotes, you'll never get likes. I believe this is why the other networks will never influence 4chan the same way.

Plus, people are just plain offensive on 4chan, which can be scary if you're not used to it. I remember when there were raids on Tumblr with gore images, and Tumblr users responded by going to 4chan and posting back gore. They only ended up traumatizing themselves.

</comment><comment><USER7> 'centipede' is a rather pointed choice of title. So it's a centipede... of humans... with BS news flowing constantly between them...

<comment><USER8> It comes from a very old image that floated around on 4chan as long as 8 years ago[0]. In a funny way it depicted how memes and information is spread between the various networks online. I think lots of people from 4chan have understood for a long time how these large digital networks interact with each other and ultimate they later realised how to exploit this over the years.

[0] There are various versions, some include Facebook etc: [https://i.imgur.com/aAlOTA8.png](https://i.imgur.com/aAlOTA8.png)

<comment><USER9> One of the authors also used a similar image when sharing the research on twitter.

[https://twitter.com/jhblackb/status/866702758132559876](https://twitter.com/jhblackb/status/866702758132559876)</comment></comment></comment>


Ever get hassle with customs? Denied. Detained. Deported. - <AUTHOR> http://trickaduu.com/2012/12/08/londumb-part-i/

<comment><USER2> No matter what you are planning to do.. "Tourism, I am here to see the country (see friends, relatives etc)" I almost made that mistake before but remembered to shut up just in time.

<comment><USER3> Lying, even when the lie is "conventional" can cause problems as well; many immigration agents are very good at sniffing out little inconsistencies.

I once spent about three hours arguing with immigration officials in "the back room" after a friend decided that "tourist" was a better answer than the real reason (going to a wedding, staying with my family)—but that didn't quite match up with other details ("where are you staying?" "how long are you here?" ...). [Note this was also prior to 9/11, and they probably err on the side of "grill" more often these days.]

The guy that did most of the grilling in that case (and who seemed pretty sharp) ended with this advice: "next time just tell the truth... it's a lot easier for everybody."

I think that's pretty much true.

<comment><AUTHOR> Hard to say really, didn't work in this case. Another friend of mine has been told before by passport control to just say "tourist" when she gave an answer that wasn't "I'm here on holiday" whereas customs then grilled her minutes later on why she said "tourist" when she was there for a wedding.

<comment><USER3> The thing is, though, once you're in the back room, they can fairly quickly determine if you were lying, and if you _did_ lie, that itself will be held against you. Unless you have a very good explanation for why you lied—and do a lot of groveling—just the lie can be enough to deny entrance (even if the truth is innocuous).

Explaining a non-lie can be annoying as well, but not having lied is a much better position to start from.

<comment><AUTHOR> During the second interview in the back room it was fully explained why my friend was there. The woman in charge then told her "It doesn't really matter what you tell me now anyway, I'm going to base it off what you said the first time around." Even after I got to explain the thing again on the phone to the woman later she said if that's the case she made a mistake but there was nothing she could do about it at that point as she "just stamped the paperwork so no reversing it now."

</comment></comment></comment></comment></comment><comment><USER4> As a UK resident, I feel a pang of embarrassment and sadness and anger as I see how non-residents are treated when going through immigration control. Heathrow immigration is the worst. It's depressing, oppressive, cold and ugly. These are probably deliberate choices.

On several occasions, I've seen customs and immigration officials behave aggressively, usually fidgeting with night-sticks (truncheons) in a way that suggests that they are itching to beat someone.

<comment><AUTHOR> The immigration officers were acting like they just uncovered a criminal, power trip took off.

</comment></comment><comment><USER5> Here's how it works.

There are two kinds of people in the world: people who crave power, and people who don't. The people who don't aren't the problem.

There are two kinds of power-craving people in the world: people who act on their craving, and people who don't. The people who don't aren't the problem.

There are two kinds of people acting on their power craving in the world: people who try to join the military or the police, and people who don't. The people who don't aren't the problem.

There are two types of power-craving military or police cadets in the world: the ones who stay there, and the ones who don't. The ones who stay there quickly get their antisocial tendencies beaten out of them by their instructors, in the vast majority of cases, so they're not the problem.

There are two kinds of power-craving academy dropouts in the world: the ones who realise that power trips aren't for them, and the ones who don't. The ones who realise aren't the problem.

The rest become airport security guards. THEY are the problem.

</comment><comment><USER6> OTOH, this summer while crossing the canadian border in Madawaska (ME) to enter the US, I was "stopped".

For some reason the US immigration officier in Puerto Rico had forgotten to add a date to my entry stamp. I was stopped for this technicality, and while getting some paperwork for me in my car the custom officers found I had prescription drugs.

I was worried, but for no reason at all - they figured out the mistake that happened in PR, and checked the drugs official prescriptions matched (antibiotics and some pretty stong painkillers), and then I was cleared to go _with a smile_.

TL;DR : the US custom officer were friendly and efficient, and fixed a mistake.

I wouldn't dream of that happening in Europe, or (god help me) the UK.

<comment><USER7> These customs anecdotes are a dime a dozen. I've had a customs official arguing with another official because he wanted to 'nail me to the wall' for no particular reason _AND I'M A GOD DAMN US CITIZEN_.

Customs officials are given a wide amount of powers, and little to no oversight. At this point, I feel like a US customs official could rape an entire squad of 13 year old cheerleaders live on air over the public airwaves and receive a tax break for the trouble. </bitter>

<comment><AUTHOR> The fact they act on a whim is beyond dodge.

</comment></comment></comment><comment><USER8> Very interesting read. My girlfriend is American and I am from Ireland, we have had a few close calls.Luckily nothing has happened yet!Heathrow is notoriously bad for deporting people. Just wondering, how did you get bumped up to premium? I have a long flight coming up and trying to see if there are any tricks to getting bumped up for free.

<comment><AUTHOR> Same boat as you. Never had any close calls before though, went to Ireland recently via Heathrow and had no problem, gave the exact same info. All seems to be down to the officer in charge and the mood they are in. Show up early and dress well for a chance to get bumped up.

</comment></comment><comment><USER9> I have often wondered whether UK Immigration Officers have a quota to meet...

<comment><USER10> I would be curious to know if some ethnic groups are being targeted so that the statistics show that they're deporting a 'correct' amount of people from each group.

<comment><AUTHOR> This was suggested by one of the people I spoke to afterwards. They can now say that they don't just stop a certain kind of group or look... "We stopped a blonde American girl last week."

</comment></comment></comment><comment><USER11> How could she be so dim? She's a _singer_ and she's going to _sing_ while she's here. Of course they are going to prick up their ears when they hear that.

<comment><USER5> That's Stockholm Syndrome talking. You're standing up for evil. Be ashamed, and change your tune. Really.

<comment><USER11> Gracious no. I'm the last person to stand up for bully-boy customs officials. But in this case a _professional_ singer is coming here to _sing_. Sorry, but I really, really think they are doing their job correctly for once.

</comment></comment></comment><comment><USER12> Wow. Can't believe that immigration officers go to such devious lengths to try and catch out unsuspecting, innocent people.

<comment><AUTHOR> Pretty nuts, entrapment style.</comment></comment>