Test-induced design damage - <AUTHOR> http://david.heinemeierhansson.com/2014/test-induced-design-damage.html <comment><USER2> I suspect my disagreements with DHHs last couple of blog posts have more to do with what each of us has seen in the wild than with any actual disagreement in principle.

For instance, in my experience I more frequently encounter places that have way fewer tests than necessary and there is no consideration about how to verify requirements at all.

Further in my experience, the GUI and database layers are the least interesting parts of the systems I work with. They truly are parts that can and do get swapped out with some regularity.

I suppose if I worked mostly on systems where they were in essence gui's on top of databases with little logic in the middle, I wouldn't want to isolate those concerns either. It would be more trouble than it is worth.

For instance, when I write small unixy command line utilities I very rarely test anything in unit tests. What would be the point? I can easily define the entirety of the specification in example tests that utilize the utility as a black box. I still do it first though...

<comment><USER3> Is swapping out the database layer so common you really need a complicated abstraction like the Repository model he mentioned? We've been running an app for 3 years and have never dropped ActiveRecord or Postgres.

<comment><USER4> Spinning this around on you: I worked on a project where swapping data stores around wasn't common _because_ of tight coupling with AR. We considered swapping data stores around for different and evolving use cases and never did largely because we had so much code that we felt was tied up so tightly with AR that it would have been very expensive to swap. I think it definitely decreases flexibility, but that it's pretty hard to know if you want more flexibility ahead of time.

Pretty much all of these pattern discussions seem to be this way to me - "just do it the simple way! YAGNI!" versus "crap this one time I _did_ need it and it was difficult to change by then! Maybe I should design things more flexibly from the start next time!". It's pretty easy to get burned going either direction, and depends a lot on things like what the project is, what organization is building it, and the level of success it ends up having. The closer a project is to a simple-CRUD, small team/unproven-company, prototype with limited success, the more sense YAGNI makes, and the further from each of those criteria a project is, the more it makes sense to design for more flexibility.

<comment><USER5> > It's pretty easy to get burned going either direction

Quite true, though I'd argue that YAGNI is still true as a probabilistic maxim. You'll make the "will I need it" decision many thousands of times in your career. If you follow YAGNI consistently[1], it will help you more often than it hurts, and you'll come out ahead in the long run.

[1] But nobody is saying you should ignore concrete evidence that you _will_ need something later. That's its own cargo cult. If there's good reason to believe YAGNI doesn't apply in a particular case, don't follow it in that case.

<comment><USER4> I think this is a dangerous line of thinking, but I suppose I wouldn't modify it very much. What I would say is that YAGNI should perhaps be _weighted higher_ , but that the probability of it being wrong in particular cases should be considered carefully.

</comment></comment></comment></comment></comment><comment><USER6> Can all this be summed up with "Avoid all dogma, even this."?

Any time we buy into a dogma at the expense of rationality, we lose. This has been demonstrated throughout history in human interactions with each other (via religion, politics, legal systems), the development of science and technology (see Galileo, Copernicus, the 19th century US doctors ignoring germ theory and killing a president).

Sometimes we create dogmas to try and move things away from bad ideas towards better ideas. Dijkstra's "Go To Considered Harmful" was one such effort. Gotos, as used at the time, were fucking terrible. They were used instead of higher level expressions like if/then/else, for, do/while, function calls. But the (at the time I was in college, early 2000s) refrain was tired and wrong (or misapplied). Sometimes, in some languages gotos can, in fact, be very useful, so long as their use is chosen deliberately and with care (see the C idiom of using gotos to jump to error handling/reporting code in functions).

In the end, nearly every development process runs the risk of becoming a dogma. Avoid that. Study the process, practice the process, and reason about where the process should actually be applied. And we already know that the answer isn't "everywhere and everytime".

</comment><comment><USER7> I feel like MVC is being treated as like the one true pattern to design your web app with and that's just simply not true. Rails is MVC and maybe that's all it should be for what it is intended for. Other projects might not be a great fit for such a simplistic view of the world and maybe that means Rails is not a great fit for projects that don't fit into the MVC abstraction.

In my experience MVP, MVVM, super thin Sinatra API's, hexagonal architecture, functional programming, and other sort of weird approaches fit certain projects much better than the standard Rails MVC approach.

Also, not every project is a web app and there are plenty of times where various testing approaches make a lot more sense than they do in Rails. It's too bad that a whole line of thinking about software quality is being disparaged because it isn't a good fit for Rails as DHH sees it.

TDD is a useful tool in the right context. Maybe that context isn't Rails.

It seems unwise to be telling a lot of smart people who care about software quality to "get off my lawn" so to speak, but I've never run a successful OSS project as big as Rails, so I probably don't have a clue about how to lead a community as big as Rails is.

<comment><USER8> I think one important thing to keep in mind is that TDD is not necessarily synonymous with "software quality". In some cases it's a very useful tool to ensure the quality of your code, but it's not even the stated goal of TDD, and a focus on TDD as the "one true path" to software quality ignores that some things are more effective (not necessarily simpler) to test using more of a "test later", integration-focused testing approach.

I agree that there's projects where a simplistic MVC approach doesn't completely fit. That doesn't mean that every software project needs to be built to the standards of the most complex software, or even that aspects of a project that _do_ require this complexity can't be solved with a more straightforward, simple MVC approach.

At the end of the day, I think the main message I get from DHH's recent series of blog posts is that treating anything as a silver bullet, or a universally beneficial pattern is harmful - and this is equally as applicable to MVC for everything itself as it is for a complex, hexagonal architecture.

<comment><USER9> Bad programmers will write bad code no matter the methodology, pattern, language, tooling, or best practice.

<comment><USER10> That's not useful though. What makes programmers bad? In some cases, at least, it's the methodology, patterns, or best practices they use.

<comment><USER11> More often than not it is lack of any methodology, patterns or best practices.

</comment></comment></comment></comment></comment><comment><USER12> The overriding message here is one of pragmatism. TDD, like a lot of methodologies before it, became gospel and people began practicing it in a dogmatic fashion without thinking about the best way to apply the principles to whatever problem is at hand. The spirit of TDD is that you have a safety net of tests to protect you from making changes in class A and breaking something in class B. If those are acceptance, integration or unit tests, great. If the code is cleanly organized and readable, great. Don't let zealots on either side of the aisle convince you to do anything beyond what makes sense to solve the problem that is sitting in front of you.

<comment><USER13> Well said. Methodologies should be treated as patterns and as such you need to understand what they aim to achieve, the trade-offs in terms of risks and benefits and most importantly how to adapt the pattern to what you are doing. You open yourself up to problems when methodologies and patterns are adopted without thought.

</comment></comment><comment><USER14> Does anyone know of a public example of a Rails application that does testing in the way that DHH says is good?

I'm tired of the talk talk talk talk talk talk of "proper" testing in Rails, yet the examples always seem to be hidden away behind company firewalls. I've only seen a couple Rails apps with Rails-Way test suites, and they were nightmares that took many minutes to run. But I have seen dozens of Rails apps written by opinionated Rails devs with strong views about what proper testing was... and the apps had no tests at all.

<comment><USER15> If your tests take 10 seconds, how much did you really test?

The point it sounds like he's trying to make is that if you say things like "they were nightmares that too many minutes to run" you may be approaching testing from the wrong point of view. He sounds like he wants to say "let the tests take 5 minutes" and I agree with him, thats what CI is for. Commit your code, mark the issue your fixing, let CI tell you if its done or not, take your pomodoro break, coffee break, etc, then sit back down, and pick back up with your test results on the CI server and repeat the cycle, a 5 minute test suite is NOT A BAD THING...

If you think 5 minutes is terribly long spare a thought for us deployment engineers... my test suite involves building and tearing down entire VM's or PXE booted machines and depending on what software is being built and tested through deployment can take an hour or more.

<comment><USER16> > If your tests take 10 seconds, how much did you really test?

I see your point, but time-to-test is a horrible proxy for quality of tests. Business logic isolated from external systems can run _incredibly_ fast, so ten seconds worth of testing can mean an awful lot in that case. The nature of TDD basically _demands_ that you structure your code that way to remain productive. Otherwise it's like using a text editor that takes five minutes every time you try to save a file.

That's my inherent frustration with this argument. Both sides aren't arguing for their methodologies, they're arguing _against_ the byproducts of each others methodologies.

<comment><USER17> Excluding the acceptance tests (written with capybara and spinach) the test suite for my current client takes less than three seconds on my machine, including the run time (excluding the phantomjs boot time) of the suite of ui- exercising JavaScript tests, and they are nearly comprehensive, testing every component contract from the client to the backend. The ATs run in a little under a minute, covering the major integration points. There is very little mocking in any of the suite, and no direct database access.

Testing hurts when you do it poorly or naively. I know because I've done it both ways, and when I find something harder than it ought to be I invariably find some point of coupling beneath the surface. When my design is good, my tests are fast and easy. If you listen to DHH you're going to have problems testing. Not because you have to when writing software, but because he's already made decisions for you which are bad or highly coupled. Don't fall for the straw man. There are better ways to do it.

</comment></comment></comment></comment><comment><USER18> Maintaining giant test suites and trying to keep them running fast is why I am so glad to not be using Rails anymore. Dynamic languages don't scale well for me because the testing is difficult to scale. With Yesod (a Haskell web framework I help maintain) I have a fraction of the need for unit tests. The compiler already gives me the equivalent of 100% code coverage for catching basic errors. I can focus efforts on testing application logic and integration testing.

<comment><USER4> Are you able to find consistent work building web applications with Haskell? It seems that many organizations are reticent to build on those sorts of technologies (not quite sure what else I'm including - maybe OCaml?) for very rational reasons - it is hard and expensive to find employees who are capable of being productive in them.

<comment><USER19> There seems to be a very common perception that it is hard and expensive to find employees who can be productive in less mainstream languages, but I rarely see evidence to back that up. I can't speak for Greg, but in my experience, hiring is hard in _every_ language, but not materially moreso in Haskell.

<comment><USER4> It's interesting, I meant my question to be about finding _companies_ using Haskell or willing to hire contractors that build projects for them using it, but of course I did take it in the hiring direction, so it's my own fault. Rephrasing: Is it easy to find work using Haskell, despite the perception (whether deserved or no) that it is difficult to hire for, which may limit the number of companies willing to build in it?

<comment><USER19> It is very interesting, because the dominant narrative seems to be both that it's hard to hire for and hard to find jobs in, which can't both be true without massive communication inefficiencies, which I'm fairly sure don't exist. My experience has been that it's somewhat harder to find a job writing Haskell than it is to find a job in a mainstream language, but that it is still very doable.

<comment><USER4> This doesn't seem inconsistent at all. It's a classic feedback loop - it is hard to hire because most people don't want to invest in becoming experienced in a technology that not many companies are using, and it is hard to find a job because companies don't want to invest the resources in using a technology that not very many potential employees are experienced in. Very similar to social network chicken/egg problems. Seems to usually be solved by either investment from one or more large companies with an interest (Sun, Oracle), or a "killer app" (Web browsers, Rails, college students for Facebook). My little theory here is woefully inadequate to explain Python's and Go's success, so maybe another way out of the trap is "lots of people just really like it". Not sure if that will or won't work for Haskell...

<comment><USER19> My point is that "hard to hire" implies more jobs than candidates, whereas "hard to find jobs" implies more candidates than jobs. They can't both be true. You can say things like "there are few candidates" or "there are few jobs" in an absolute sense, by comparison to other languages, but that actually isn't very relevant.

This is an oversimplified model because it doesn't take into account engineer skill level, which actually does seem to be the primary problem. Companies want skilled engineers, but it's hard to become skilled without having a job in the language first. So we end up with several companies trying to hire seniors, and several juniors looking for jobs.

<comment><USER20> Both "hard to hire" and "hard to find jobs" are satisfied by a disjoint set of job locations to candidate locations.

That is, there could be plenty of X jobs in City Y, but that does little for the X candidate in Z city. Flip candidate/job as desired.

<comment><USER19> That's true, and there does seem to be a disproportionate number of Haskell jobs in Singapore, but overall I don't think it has a huge effect.

<comment><USER20> More likely you are just seeing a fairly common network effect, where once you are in it is easy to see many connections.

So, if there are some fairly good quantitative treatments of this, I'd be interested. I suspect it isn't too shocking. Probably more than the parent poster and friends think. Probably less than you do. :)

<comment><USER4> I actually really don't have much of a hypothesis at all. I asked my original question out of pure curiosity, and my reply was along the lines of it seeming very plausible for there to be both a shortage of candidates to hire and a shortage of companies hiring. But I also think the opposite is plausible. I think so far the most specific answer to my initial question of "is it easy to find Haskell work?" is "yes, in Singapore it is".

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER21> My least favorite example of when tests damage an API design is with dependency injection. There seems to be very little need to resort to dependency injection as a way to create a good API with easy to understand architecture many times, but dependency injection gets abused because it makes testing easier. You can supply your API with mock and stub classes at every turn if you use dependency injection everywhere, but the consequence is a more difficult to use API that requires the programmer who uses your API to understand more arbitrary and unnecessary implementation specific details.

For example, maybe I just want to open up an encrypted TLS TCP socket to a server. From a user perspective this could be really basic, you provide a library with an API that you provide with server address, port and handlers. It could be as simple as a few lines of code. But the dependency injection version of this would require maybe require creating an SSL Factory, which requires a 509x certificate provider, which requires a certificate storage locater. Then instead of an address you must provide it with an ipaddress factory method and a protocol factory which requires a list of available protocol implementors. Then 200 lines later you want to actually manage your connection and you must provide a connection manager and a byte buffer which itself involves tons of cruft.

Sometimes dependency injection is like a person walking around with their organs hanging outside of the body. When two people want to make babies they don't have to know low level biological mechanics of how sperm sends signals to a ready to be fertilized egg. They don't have to read and learn pointless documentation. They just insert the thing and everything usually works although under the hood it is maybe one of the most complicated processes in biology. That's how an API should work: making complicated things simple.

<comment><USER22> Could you not have the best of both worlds by writing a facade over the public-facing API? I have used this approach in the past and it works well. The facade just handles grunt work of wiring the myriad of types together, as you described. It can be verified by blackbox testing or simply by eye (as it should not contain any logic beyond instantiation).

<comment><USER23> While there are valid use cases, this sort of thinking is why apps are no more responsive today than they were 20 years ago... everything is running through 20 translation layers.

<comment><USER24> While that is a risk, it's not a necessity. Many of the classes in the C++ standard library are designed in exactly this way without runtime performance issues. All of the wrapping and indirection get inlined at compile time.

</comment></comment></comment></comment><comment><USER25> Maybe the real problem is that we have crappy tools for hexagonal-oriented architectures; especially Rails. Classic Rails style dictates that ActiveRecord is Good Enough for your domain logic. This creates a sort of framework lock-in: inheritance is one of the strongest forms of coupling there is, especially when you inherit from classes you do not control. The framework superclass is a likely to be a relic of current-gen frameworks that we do not tolerate in the future.

The technological way out is to use a Data Mapper pattern ORM to isolate the domain logic and the persistence. But this approach won't catch on, because Rails devs have tasted the simplicity of ActiveRecord and aren't about to do more work to get the same result.

It is telling that many language communities eventually head towards amalgamating a collection of really good libraries in a low-coupling manner. This is still a fringe movement in Ruby, currently.

<comment><USER8> If ActiveRecord allows you to do less work for the same result, at least for some less complicated applications, isn't that a good thing? I think that's DHH's whole point - we shouldn't be pursuing some mythical perfectly testable architecture where it doesn't make sense. If you can write clearer, more concise code for less effort that doesn't fit into the purely separated, easily testable TDD approach, is that really such a bad thing?

</comment></comment><comment><USER26> In Java, the most obvious example of testing affecting the design of a class is the necessity of avoiding private methods in order to facilitate testing. While there are ways around this -- reflection, PowerMock, probably others -- they all tend to be ugly and hackish.

This has an effect upon the design of classes, because the easiest path is simply to make private methods package private. This is frequently not the ideal design, and taken to its logical extreme means that you will have no private methods.

I think unit testing is important, and do use it. The line for me, though, is similar to DHH's here: when the drive for unit testing affects the design of the software, that's when I tend to become less enamored.

<comment><USER27> > In Java, the most obvious example of testing affecting the design of a class > is the necessity of avoiding private methods in order to facilitate testing.

IMO, this is unnecessary and a failure to understand the _point_ of unit testing: unit testing is testing the _public interface_ of the unit-under-test in isolation from other components, so there is no reason to avoid private methods to facilitate testing since private methods are, _ipso facto_ , not part of the public interface of the unit under test, they are called by methods in the public interface and tested by testing the methods which they serve. Making private methods public and directly testable makes unit tests _more brittle_ and refactoring _more expensive_ , which is exactly the opposite of what you should be striving for with unit testing.

<comment><USER28> I disagree. For example, a method to generate all the permutations of a sequence is easy to get wrong and should be tested whether or not the library using it exposes it.

Testing an internal method by itself, instead of indirectly through the public API, gives you the same scope reduction benefits that testing a unit instead of the entire program gives (but less pronounced).

Personally I think the solution is to scope unit tests into the thing they are testing. So tests of a private method would be scoped to that method. That way your decisions about what to test aren't constrained, though they can be guided, by what is visible.

<comment><USER29> > For example, a method to generate all the permutations of a sequence is easy > to get wrong and should be tested whether or not the library using it > exposes it.

There are three possibilities here:

1\. If your language or common utility libraries have a permutations() method, you shouldn't be rolling your own permutations() method because one exists in libraries. 2\. If you're in an environment that doesn't have built in permutations() you should group these kinds of very generic functions that are hard to get right in to some sort of utility module (in which case it would necessarily already be public). 3\. If you're in a language that doesn't have built in permutations() and permutations() is in the class which uses it, you have a very generic function on a more specific class, where it has no business being, so it should be moved to a utility class.

In all three cases, the solution isn't just "make it public". If you find that you're just making something public to unit test it, this usually points to a much larger problem with your design.

<comment><USER28> 1\. Agreed. (Assume it doesn't.)

2\. Why am I making my library's utility methods public? It's a frob library, not a generic utility method library. I don't want clients depending on my utility methods. I don't want to support a separate utility library just to avoid testing private methods. I would prefer not to take on an external dependency for a single simple method. Having it private and tested is the best tradeoff here.

3\. Agreed.

</comment></comment></comment></comment></comment><comment><USER30> > the simple controller is forbidden from talking directly to Active Record > [..] This is not better.

It is. The controller layer should be as dumb as possible, it shouldn't contain your (entire) application logic. It's a matter of single responsibility if anything.

Also, I find it very sad that we're still discussing the usefulness of the active record pattern. Other than convenience, it has none. It's a pain to maintain an application that uses it once it reaches a certain level of complexity.

And not just because of testability, it's a pain in the ass to replace/fine tune certain queries if you're calling active record methods in your controller.

<comment><USER4> Honest question: what pattern for database access is better, and what are the best tools for said pattern? I don't necessarily mean just in Rails, but everywhere. ActiveRecord-the-ruby-library is incredibly mature and convenient to use, and for better or worse, encourages active-record-the-pattern. The repository pattern seems nicer in theory to me, but in practice, the best tool (in ruby) to implement it with still seems to be ActiveRecord, and then I find that I'm mostly delegating to the underlying AR object (because it already does everything!) and wondering what I've really gained. I was hoping DataMapper2/ROM[0] may have been a more straightforward but high-functioning replacement for AR, but it seems there has been no progress on it for quite some time.

tldr; I'm wondering how you actually _do_ this. Firstly, in Rails, but other acceptable answers are "other technologies do it in this other way, which is better than how Rails does it for these reasons".

[0]: [http://rom-rb.org/](http://rom-rb.org/)

<comment><USER30> I'm not using Rails (nor Ruby for that matter) so I can't comment on that part, but I found the repository pattern to be really useful. Using it with an active record is something I've seen other people do, and at least it gets the active record calls out of the controllers. It's not an optimal solution of course, but IMO it still beats plain active record.

<comment><USER4> Are you willing to say what language you're using the repository pattern in and what tooling you've found useful in supporting it? I think my use of DAOs in Java struts projects was somewhat repository-pattern-esque, but regardless of the frustrations I have with ActiveRecord, it is _way better_ than that was, so I'm really curious what, if anything, is better than either of them.

<comment><USER30> I'm using PHP and Doctrine 2.

<comment><USER4> Thanks! I'll check it out.

</comment></comment></comment></comment></comment></comment><comment><USER3> I echo his sentiment. Integration testing, especially when you have a JS frontend, makes much more sense. I never saw the point of controller tests and making sure a controller assigns variable @widgets with [widget] and all that nonsense. An integration test will identify all those problems and then some.

<comment><USER31> I agree. Instance variable assignment is an implementation detail, IMO it's more useful to describe their behaviour with a request spec for example.

<comment><USER3> Yes, exactly. We use request specs to test regular HTTP requests to regular pages or our API. And then we use feature tests with Capybara and PhantomJS to load the app, click through it, and make it loads as it would in a customer's browser. This covers most of our application. And then we use regular unit tests for anything not customer facing, such as background jobs. But these unit tests make up only a small fraction of our test suite.

</comment></comment></comment><comment><USER32> Decoupling is one of the fundamental tenants of development, and provides far far more benefits than merely TDD. If he thinks decoupling is about TDD he's missed out on architectures that can be easily fixed when bugs show up by isolating causes in changed code, being able to extend without modifying core code (the open close rule) and managing regressions in general. How do you scale software to a team of developers without decoupling?

The only argument I've ever seen against decoupling is performance, and it's rare that argument makes sense in all but the most real time of applications.

<comment><USER33> I don't think that DHH is arguing against decoupling but rather he is arguing that certain decoupling practices that are encouraged by TDD interfere with the readibility of the application.

<comment><USER34> I've never seen a Rails application that suffered from too much decoupling. If someone out there wrote one of those, call me because I have a job for you. 99% of all things written in Rails are monolithic messes of business logic encrusted with persistence and response handling. So ridiculously coupled that you don't talk to objects by themselves, but bring in a whole family of resources to models and controllers that violate almost every SOLID principle.

The whole "You're not gonna need it" argument works until you actually do need it. Which, unless you're not doing a good job, is going to happen. Then you have no discriminated interface to pry your application blocks away from each other and can't persist a model without dozens of unintended side effects.

There's no readability penalty to decoupling. The more you decouple the less you need to read to understand an application.

</comment></comment></comment><comment><USER35> I don't understand why he has to equal TDD with the mockist approach to TDD without clarifying he is talking about the mockist approach and not TDD in general. Pivotal Labs for example is obviously a huge proponent of TDD, but has been historically been opposed to hesitant towards true, isolated, heavily stubbed and mocked unit tests.

That makes me wonder if he just doesn't have a differentiated enough view of TDD or if he omitted that on purpose to get more attention. I am also not sure which answer is would be more disappointing.

</comment><comment><USER36> " but does so by harming the clarity of the code through — usually through needless indirection and conceptual overhead"

This argument feels a bit thin and unsubstantiated for the general case. I can see his criticism of hexagonal design applied to Rails, but he's using that as a straw man to attack TDD. I think he could better criticise the limitations of TTD by directly examining applications of RGR and other TTD pronciples.

<comment><USER31> It's not really a straw man. Driving your application with tests at the unit level doesn't make a lot of sense, in the case of a web app at least. The BDD approach makes more sense to me. It's how I work and in my experience tends to inform design a lot better.

<comment><USER36> I think saying hexagonal design is bad therefore TDD is bad qualifies as a straw man.

Person 1 has position X. Person 2 disregards certain key points of X and instead presents the superficially similar position Y. The position Y is a distorted version of X.

Here X = TDD is good. Y = hexagonal design could be good for rails Y =/= X

I'm no TDD zealot, I just think DHH's argument here is weak.

</comment></comment></comment><comment><USER37> I was unaware that people were actually trying to unit test controllers. That to me just seems like a recipe for endless frustration. Mock out a web request? Please don't.

Everything I've ever read about Rails refactoring indicates that your controllers should be skinny, implying they don't need to be tested, push all complex logic out into helper functions, lib classes or models and unit test those.

<comment><USER38> I found myself in that exact situation recently, struggling with finer points of Capybara, trying to nail down critical behavior in my controllers . . . then I realized, why had all of this crap crept into my controller in the first place? I refactored a ton of things into the models where they really belonged and ended up with much tighter code and better reuse of functions. Now testing is way easier - the only thing my controllers are really doing now is routing after certain conditions and serving up error/success messages.

Sometimes I feel like Steve Martin when I'm getting more sophisticated with a framework . . . I've got a googlephonic stereo with a moonrock needle, but maybe the problem is the shocks in my car: [https://www.youtube.com/watch?v=Cjjsz14hL48](https://www.youtube.com/watch?v=Cjjsz14hL48)

</comment></comment><comment><USER39> TDD is supposed to affect design in good AND bad ways. It is not true that TDD claims to have the best design, but the more testable. The first time I read about TDD it basically said testability > clarity.

A succinct code that you dont know if its doing the right thing is worse than more verbose code that you can easily verify what it does.

I do think that specifically with Rails, tests get so plentiful that they take long to run and it threatens the whole process. And the weight of testing models/controllers/integration is something that has bitten me before. Particularly, doing less integration and more model, because integration tests can be flimsy and slow an order of magnitude more.

Since my first web programmer job, in 100% of my projects tests grew so big it took them minutes to run, making me nostalgic about the speed of Java tests I had for my first programming job.

<comment><USER40> > The first time I read about TDD it basically said testability > clarity.

Now you scared me. I never tried TDD, and if that's a required tenet, I never will. This is completely upside-down.

Tests can not verify that a program is correct.

<comment><USER39> Only proof can verify a program is correct, and that is so cumbersome and expensive is done on rare critical operations.

Tests give you the ability to know how a certain code behaves in specific circumstances.

Clarity makes it easy to understand the general case.

So if the clarity of the general case is a little worse to be able to test the outlier cases as well, the TDD philosophy would welcome that trade. Or at least, thats how I understood it.

A brute and unpolished example of clarity vs testability:

A)

      def complicated_algorithm(input)
        mod_input = Math.root(input)
        mod_input = input / mod_input
        ...
      end

      def division_in_complicated_algorithm_test
        input = 1
        input.should_receive(:/).and_raise(ZeroDivisonError)
        Math.root.should_receive(input).and_return(0)
        complicated_algorithm(1)
      end

\</comment></comment></comment><comment></comment><comment>-----

B)

      def complicated_algorithm_testable_version(input)
        mod_input = Math.root(input)
        mod_input = divide(input, mod_input)
        ...
      end

      def divide(a,b)
        a/b
      end

      def divide_test
        assert_exception ZeroDivisionError, divide(1,0)
      end

Overall, the point of doing tests first is that you dont hack them together doing complex dependency injections, because naturally to save work you do the easy test..at some expense of the final code.

In the above example, the top example is less verbose, having one full less method, but the test is more complex and fickle, because it was done to verify the above code.

Its not a fantastic example, we could argue that the tests try different things, and that division is too silly to put into a different method. The point is that the above happens when you write code first , test later, and the bottom one happens the other way around. TDD advocates for testability over clarity.

<comment><USER40> I'll argue that clarity makes it easier to understand the code. All cases of it. Not just the general case.

Yet, I can see how one'd want to sacrifice a small bit of clarity to gain a big amount of testability. Thanks for the example.

</comment></comment><comment><USER41> I really identify with some of the points he's making, they're observations I've made myself so it's nice to see someone with his clout bringing them up.

I wonder about the design thing though - our code is in some ways a document of the circumstances surrounding it. Does it make sense to have it conform to some Platonic ideal, which we corrupt when we alter it to make it more testable? I'm really not sure about this, but I doubt it. Code ultimately needs to work in a given set of ways and that's our primary concern with it. Making the code "pure" (or just "easy to read" if you like) is a service to other developers who come along later. So, the tradeoff is testability for intelligibility. I can imagine a lot of scenarios where that tradeoff is a rational one.

<comment><USER11> We were having discussion about a collaborating groups architecture. The guy said something couldn't be done, because it would involve coupling two components. Again that is a nice ideal to aim towards, but surely functionality comes first.

</comment></comment><comment><USER42> Kent Beck response: [https://www.facebook.com/notes/kent-beck/rip- tdd/75084019494...](https://www.facebook.com/notes/kent-beck/rip- tdd/750840194948847)

<comment><USER43> That was a terribly thought through reply. It sounds like the sort of bullshit you hear on sunday morning political shows. "Let's pretend I didn't hear my opponents points, and throw back my own talking points which fail to address his points.".

</comment></comment><comment><USER44> I don't really see DHH giving any arguments as to why designing for tests leads to poor design decisions. I suppose I can buy the argument that there are cases where this isn't true, but I can't think of any and he's not giving any. I would argue that Angular is a good example of how designing for testability creates good design decisions.

Secondly, I don't buy the idea that you should focus on integration tests over unit tests. Integration tests are important, but they're also the most expensive tests in terms of maintenance. Unit tests you can run with every code submit. You can run them multiple times per code submit. Integration tests take too much time for this to be practical.

In all, I'm tired of people making decisions based on what they're _against_. DHH is just being negativistic and defining his code design strategy around being against TDD and test-driven design. That's ok. But what design strategies does he _support_? He starts giving more information about that at the end, but I'm still left scratching my head and wondering what design philosophy he's actually advocating rather than what design philosophy he's bashing.

<comment><USER45> _I don 't really see DHH giving any arguments as to why designing for tests leads to poor design decisions._

It results in pointless levels of abstraction that aren't used to abstract anything in real code, but destroy readability and screw up static analysis tools. It also results in over-splitting of entities to the point where they don't represent anything remotely similar to problem domain. Finally, it encourages "old stuff plus this addition" kind of design. (For example, using a switch statement to cover 7 different cases for days of the week, rather than using a math formula.)

</comment></comment><comment><USER46> Black box (functional) testing is the way to go. I created a flow style of testing, which allows "Fast & Thorough Testing". This is a javascript & jasmine extension, but the concept can be applied to other languages.

[http://briantakita.com/articles/fast-and-thorough-testing- wi...](http://briantakita.com/articles/fast-and-thorough-testing-with-jasmine- flow/)

The nice thing is the testing does not have a large effect on the implementation, so you have the freedom to change the implementation without the tests failing.

The test suite scales since edge cases can be grouped together into a single flow. This removes extraneous runtime burden of having to recreate the same context for a each individual edge case.

I find that I don't need to be performing TDD as often.

</comment><comment><USER47> I certainly do agree about integration tests being important. I've also started moving towards using a live database for my tests. I set up a postgres database by copying over a master copy to a temporary directory and running a postgres daemon from there. It takes ~100ms and with fsync turned off it makes for snappy tests. If it starts getting to be slow I can always move it to a ramdisk.

Here's a library I wrote for golang which wraps it all up in a convenient package:

[https://github.com/surullabs/ghostgres](https://github.com/surullabs/ghostgres)

</comment><comment><USER48> This may be a naive comment, but is DHH simply being defensive about a perceived movement away from Rails i.e. "decoupling from Rails"?

* edit - of course, he could be defensive and right, they aren't exclusive

</comment><comment><USER10> I don't see why TDD proponents make a big deal about not touching the database. It's as if they haven't heard of SQLite's in-memory option, in which the database is just another data structure in RAM, which is all that their extra layers of objects are. True, with that setup, you're using SQLite for tests and, say, PostgreSQL in production. But is that any worse than using your own mock objects in tests? What am I missing?

<comment><USER49> SQLite is not <your actual rdbms>... it's SQLite. A database that does not enforce column types and lacks most of the advanced features of a real rdbms. Using SQLite will work great right up until the point where it won't and you get your fingers burnt.

If you separate your concerns properly you won't need to mock the database layer either. Mocking is just one part of the trifecta of good testing, along with Stubbing and Faking.

For most things it would make more sense to _fake_ the database layer or _stub_ the database layer in your "logic" layer.

However, if your application makes heavy use of the rdbms then you should test that layer too: In your integration tests and not your unit tests. Most places that interact with an RDBMS treat them like black boxes and not like a business layer of its own. You really need integration tests to ensure things like constraints and your business rules are captured properly... most people never bother.

The real problem with TDD and its methodologies isn't TDD itself; it's people shoehorning about 10% of what proper testing should be into two narrow groups: stuff that you can do with "unit tests" and "things we can mock." There's a lot more to it than just those two things.

</comment></comment><comment><USER50> Kind of off-topic, but it is nice to read a thoughtful post that isn't over- the-top flame inducing. Less emotional rhetoric than some recent TDD discussions.

<comment><USER51> Seriously? It's full of snarky swipes at "true believers". I don't think DHH has made a sincere attempt to engage with the other side; this is knocking down a strawman again.

</comment></comment><comment><USER52> I would have a repository layer even if I wouldn't write any tests. I don't want an ORM to interfere with my "business". So this could as well be a post against ORMs. But still I think author is right. TDD does enforce an architecture, and that architecture doesn't have to be the best. TDD zealots often run around, preaching their cult and blind out any drawbacks, as if there were none.

</comment><comment><USER53> I wonder if this isn't a maintainability issue in disguise.

I have never had a problem with unit tests or int tests. As a rule I never use mocks, and everything fits into one of those areas. Either you have real data sources available (such as an in process db) or you make it a module that can be easily unit tested.

It's clear he is against TDD first, and looking for reasons second. I feel other factors are at play.

</comment><comment><USER54> >> … it's a mistake to try to unit test controllers in Rails (or similar MVC setups). The purpose of the controller is to integrate the requests from the user with the response from the models within the context of session.

Well said … Now, if somebody from Salesforce.com could understand this and stop forcing their customers to write these useless tests for the controllers.

</comment><comment><USER55> One thing I like about Java programmers is that they realize everything a class depends on needs to be passed to that class's constructor. There's really no way to avoid it. Change concrete instances to interfaces, and you have a nice testable class. Write an integration tests, write a unit test, they're both easy.

<comment><USER56> s/Java programmers/good Java programmers/.

</comment></comment><comment><USER57> DHH is arguing from the perspective of a Rails developer working on a Rails application. It's no small kingdom but to discredit TDD as a practice for all software developers is short-sighted. There are enough counter-examples of the benefits of TDD in my own experience to make the claim invalid as a universal truth.

</comment><comment><USER58> Test... good. TDD ... sometimes (often) good.

But unit tests can lead to an overly abstracted design that harms the quality of the code.

Test what you can with unit tests but don't compromise your code to do so when there are other ways to achieve a suitable level of testing

</comment><comment><USER59> Curious why DHH is in the title. This doesn't seem to happen with posts from others well-known in the tech community.

<comment><USER6> Because it's in the title on the site, however it should be removed and treated like a "\- NY Times" suffix.

Just checked, it is now removed here.

<comment><USER59> Ah, didn't notice this was on the title on the original site. Thanks for pointing that out.

</comment></comment></comment><comment><USER60> Rails, rails. rails. Everything is about Rails. David is a good guy, but I'm starting to wonder if he's ever built a moderately complex system, involving integrations, message queues, several data stores, a handful of third-party libraries and APIs and deployed on more than 3 machines.

TDD is a tool to manage complexity. It's an advice, not a recipe. Like any technology - it isn't a substitute for thinking.

<comment><USER61> Everyone has their own definition of complexity. I make no mince about a decade worth of developing Basecamp is where I draw my primary experience from.

That system is small by web scale standards -- only 70 million requests/day, 1.5 terabyte of DB data, half a petabyte of file storage, two data centers, and about 100 physical machines -- but probably still larger than 97% of all Rails apps.

Also, plenty of data stores (memcached, redis, multiple MySQLs, solr), many 3rd party libs, job servers, integrations, and more.

So no, it's no Facebook or Yahoo or Google. But it also isn't a toy system, except in the sense that we're still having so much fun playing with it.

<comment><USER60> I am not dismissing Basecamp. I am just saying that in a large portion of the software world, applications are waaaay more complex than normal rails apps. And in that context, TDD makes sense if only to manage complexity. Even if you are not Facebook, but say Airbnb. If their tests are not fast enough and they cannot trust them to make decisions, they wouldn't be able to deploy in a reasonable time. And when slow tests lead to infrequent deployments, that's when the real problems begin. (Airbnb is an arbitrary choice which came from the top of my mind, not anything specific)

My gut feeling is that >50% of software development happens in those complex apps and not rails apps. So dismissing TDD is just yet another extreme viewpoint, which many people will unfortunately take for granted.

<comment><USER62> AFAIK Airbnb uses Ruby and Rails to some extent. An actual job offer lists it as a requisite: [https://www.airbnb.com/jobs/departments/position/2192](https://www.airbnb.com/jobs/departments/position/2192)</comment></comment></comment></comment>


Why a Great Programmer Is Worth Fifty Good Ones - <AUTHOR> http://blog.tomevslin.com/2006/05/why_a_great_pro.html

<comment><USER2> There is no evidence that such a 50x ratio exists, at least, not in a meaningful, actionable sense.

The traditional statement concerns a 10x programmer, and between the best programmers and the worst.

The comparison to Michael Jordan is irrelevant. Is there a 50x difference between a good and a great commercial airline pilot? Do we measure authors by productivity?</comment>


My weekend project: Motivate your team at the moment something good happens - <AUTHOR> http://appevent.io/

<comment><AUTHOR> I'm really not a designer at all so to the guys behind screen leap, just remember that imitation is the sincerest form of flattery. If this takes off, I'll hire a UI person. :)

</comment><comment><USER2> Neat, kind of like a push, instant, throw away yet private chat room.</comment>


Who Buried Paul? (1999) - <AUTHOR> http://ludix.com/moriarty/paul.html <comment><USER2> His other classic lecture is The Secret of Psalm 46:

[http://ludix.com/moriarty/psalm46.html](http://ludix.com/moriarty/psalm46.html)

</comment><comment><USER3> Well for a few years it could have been fairly argued that the Paul's talent had died, until Band on the Run was released at end of 1973.

<comment><USER4> > _until Band on the Run was released at end of 1973_

Yes, and then it couldn't be argued anymore, it was a confirmed fact.

<comment><USER3> You sound like a lot of fun.

</comment></comment></comment><comment><USER5> This takes a while to get to the point, but it's a nice little thesis: worth a read for game devs and the like.

</comment><comment><USER6> And surprisingly someone is actually taking people back: [https://www.storywars.net/stories/7245](https://www.storywars.net/stories/7245)

</comment><comment><USER7> I'd highly encourage anyone to watch the full video rather than just read the transcript. It's an hour, and I think it was well spent.

Wonderful conspiracy theory, and a wonderful conclusion from all of it.

</comment><comment><USER8> It was basically a conspiracy theory. And a lot of people who today make fun of conspiracy theorists probably believed that one back then.

Careful what you make fun of, and carry a mirror.

<comment><USER4> > _And a lot of people who today make fun of conspiracy theorists probably > believed that one back then._

A lot of people who "today make fun of conspiracy theorists" weren't even alive back then.

<comment><USER8> And a lot of us were.

</comment></comment></comment><comment><USER9> Humans are natural clue hunters, we love finding meaning in patterns we want to find.

</comment><comment><USER10> I'm 20 minutes into the video right now and I can connect the dots to one of the most talked about games of 2015, FNAF, Five Nights at Freddy's, the one that made reaction artists like Markiplier insanely popular.

How does such legends begin?

Well, first they begin with a rumor, a rumor of it being real. With the launch of Game I & II, fans started speculations of it being a real story - about a real pizzeria, with real kids involved, and a psychopath who might have murdered those children, or a pedophile.

Then, began the hunt for the plot. Fans began searching for clues in the game play, like the dates on the salary slips, the posters on the walls and artifacts in the halls, the mini games with in the game. Some even sought the protagonist. Some dug of news archives to look for actual events in past that involved pizzeria, kids and murder.

And after v2 launch, some suggested that version II was actually a prequel to version I as the dates were old and the minigames & the game's story line suggested this too. Some even went & tallied the hourly wages during 1960s to validate this.

Fans are fans and they do just anything.

People discussed the game characters and the protagonist. And then began the hunt for the relation between the hotel manager, the purple guy, the kids who died, the kid who wept all the time, the spring lock suits andthe ever- returning-persistent security guard who refused to give up & returned, perhaps with a different name in the sequel. Was it a rebirth? Or did he simply changed identity? Who knew?

Had Paul answered the phone call while he was away on family vacation at some unknown remote scotland country side, there wouldn't have been any space for the frenzy. Since the game creator intentionally lived a life deviod of attentions, fans & their questions went unanswered. And this is the key.

Fans were waiting for the launch of v3 as it would give more clues to who actually was the purple guy. But they were betrayed once again. The game maker, Scott Cawthon, made the plot even more confusing. After the version III, people couldn't place it anywhere. Some said it was a prequel, some said sequel, many said they were confused.

And then came the v4 where fans saw a panting boy running across the house in panic. And some noticed the flowers & suggested it was a hospital & since the kid is in fear, they thought it was a case of schizoprenia & a case of a challenged 4 year old boy.

In the absence of the SSOT (single source of truth), fans began speculating, collecting & piling evidences. They began their own versions of conspiracy theories. The game became "the most talked about game on youtube" by the end of 2015 as fans began making videos with their version of the theory.

Finally, Scott came out of his hideout & made a public comment that this game was not what people thought, and what if all of it was just a dream? And this comment was unhelpful and an insincere attempt to answer the questions of the fandom.

This observation can be carried to more interesting plots of businesses, like stock market trends, venture capital funding, or app usages. Insider trading simply takes the fun out of Stock markets, and hence, it is illegal. Rumors fuel investments. A large group of people with a lot of money without the ability to think what to do about it, often hit with the FOMO, jump into radicalism & make zillion dollar investments. And while the generation X is clueless how to use snapchat, millenials are obsessed with it.

Inquisitiveness is a common trait among primates & us, hominidae aren't excluded from this. All you need to do is, to keep a veil of mystery. As programmers, or physists, or mathematicians, we don't like facts, we love to solve problems & arrive at facts. Fact is a destination where the road ends. Fact finding is a journey where you explore the unknown, meet the unmet, see the unseen and that's what makes life beautiful.

</comment><comment><USER11> Excellent presentation skills.</comment>


Quantifying the Adpocalypse - <AUTHOR> https://mule.substack.com/p/quantifying-the-adpocalypse <comment><USER2> The article suggests that the stock market hasn't priced this in yet. I don't think that's true.

Facebook and Google are both down more than 20% from their recent highs (along with the rest of the market, of course).

No one thinks that a recession is any kind of existential threat for either of them - they both have plenty of cash and fat profit margins.

And if anything, both companies are positioned to emerge from the pandemic in a stronger position than they started - with fewer competitors, more users, less expensive competition for talent (* whimpers *), and acceleration of long-term societal trends that benefit them.

Investors are not going to take their eye off the medium-term future of these companies because of an expected few poor quarters for very well-understood and temporary reasons.

<comment><USER3> your margins go away if you don't have revenue (no ads) and increased variable costs (more users hitting your servers) with constant fixed costs (rent, salary, healthcare).

<comment><USER4> If you have sufficient reserves to last it out, then a year or two with no margins or negative margins should not mean a huge decrease in company valuation. If you lose 20% of your next 10 year profit because you lose 2 of 10 years, then that that justifies a 20% drop in share price but not a 50% collapse.

<comment><USER3> if you believe that, then I'll trade you 20% of my expected 10 year income for 20% of my current discounted net worth (which is theoretically what a company's market cap should be, a discounted claim on future distributions).

revenue today (or in the shorter term) is always more valuable than revenue 10 years out. put it a different way, what if there's another "act of god" in a few years? the probability of an extreme event occurring within a discrete timeframe increases as your timeframe gets larger, thus your certainty in revenue projections should decrease as you project further out.

</comment></comment></comment></comment><comment><USER5> Adding one more piece of data to the story, I've run a large website (Social Blade) since 2008 where the primary source of revenue has always been advertising. It was only last year that we started focusing more on paid subscriptions to cover the natural drop in ad revenue that was happening anyway.

I don't have the data yet from one of my two ad providers, but the one that does provide me real time data I can see a 35% drop in ad revenue this month compared to Feb while having about a 31% bump in traffic.

<comment><USER6> According to Fark.com, there is unprecedented drops in Ad revenue. Companies have cancelled mid-quarter.

[https://www.fark.com/blog/DisseminationMonkey](https://www.fark.com/blog/DisseminationMonkey)

</comment></comment><comment><USER7> Tangentially, one positive side-effect of the pandemic lockdown I've seen is a dramatic drop in the quantity of spam in my physical mailbox. I generally get 5-20 pieces of junk mail a week, but in the 3.5 weeks that the Bay Area has been on lockdown I've gotten maybe 7-8 pieces total.

<comment><USER8> Though it seems organizations have completely disregarded CAN-SPAM.

</comment></comment><comment><USER9> This looks like data for the public network markets. In-house and larger corporations don't quantify on actual ad revenue. Rather, they find any sale by an individual that saw and ad and say that a portion of the sale is attributed to advertising. This is the "attribution" model, which is pure hand waving.

</comment><comment><USER10> It would be good to compare Amazon advertising spend as well.

This article is also missing the fact that we're in an election year in the U.S. The ad spend from that alone could counter a lot of the losses from covid 19.

<comment><USER11> Political ads are very small percentage of revenue for FB [https://www.google.com/amp/s/techcrunch.com/2019/10/30/zucke...](https://www.google.com/amp/s/techcrunch.com/2019/10/30/zuckerberg- political-ads/amp/)

<comment><USER10> According to Zuckerberg, in defense against accusations that he was being greedy by allowing them on FB.

I'd be more interested in an independent or consumer-based third party analysis validating his numbers, since he doesn't mention PAC spending.

<comment><USER12> Trump spent very little in his first election campaign, mainly relying on saying outrageous things and tweeting (one estimate was that he added $2 billion to the value of the Twitter brand.)

How little? Enough that all the Silicon Valley advertising companies missed their quarter, either unaware or wilfully ignorant that he wasn't spending money.

<comment><USER13> He wasn't trying to win.

</comment></comment></comment></comment></comment><comment><USER11> I am unsure you can even rely on analyst estimates to determine what investors are expecting from FB/Google when they report earnings soon. Will anyone be surprised if FB comes out and say they will experience a 20% decline in revenue in Q2?

<comment><USER14> Yes, sell side analysts expectations are basically expectations of the big participants in the market unless there is disclosure / news from the company or price action on the stock that can influence the outlook.

<comment><USER11> The estimates are just absurdly wrong though. Ask any investor and most will just laugh at those estimates. 99% of estimates for all companies haven't dramatically been decreased yet. So I would take these estimates with a grain of salt.

<comment><USER3> yeah that's the point of the article.

</comment></comment></comment></comment><comment><USER15> the author is suggesting that ad spending online correlates with increased sales. is that the case? seems a big part of ad spending is simply part of burning VC money regardless of outcome.

<comment><USER16> A successful marketing exec once said "We know that half our marketing budget is wasted. We just don't know which half."

<comment><USER17> [https://en.wikipedia.org/wiki/John_Wanamaker#Legacy](https://en.wikipedia.org/wiki/John_Wanamaker#Legacy)

</comment></comment></comment><comment><USER18> Anecdotic, but I've read several recent reports about people being kicked out of Google Adsense and Amazon affiliate program around the web.</comment>


Aliens in the Valley: The History of Reddit - <AUTHOR> http://mashable.com/2014/12/03/history-of-reddit/

<comment><USER2> I was interviewed for this article (as well as many of the other early reddit folks), and I think it turned out well. It's one of the few articles about reddit where the author actually spoke to any of us, and it shows in the depth and accuracy.

<comment><USER3> Yeah, my only gripe is his implication that Chris failed to pull off the punk look.

<comment><USER2> My only gripe was that he didn't use any quotes from the two hours I spent on the phone with him. :)

</comment></comment></comment><comment><USER4> Reading this, I'm struck by a few things that I believe are telling about the tech sector's culture.

The paternalistic/maternalistic attitude of Paul and Jessica is more than a little cloying. Frankly, it seems like they enjoyed the position of power granted them by the cash in their wallets.

And second, the focus on dysfunctional behavior as an aberration instead of the norm. See how the term 'dysfunction' is used to describe Yishan's departure, Twitter's board, Reddit in general.

Things rarely go 'professionally' when humans are concerned; I'd argue, in fact, that Yishan found a quite professional avenue to quit, in that it did _not_ become a shouting match over a core issue.

I think it a mark of corporatism to adhere strongly to the facade of professionalism and punish or otherwise shame those that prove it a lie. Meanwhile, cultural leaders within the Valley pick favorites and call them "Muffins" but are, presently, secure from social fallout.

There is an in crowd and an out crowd, in other words.

<comment><USER5> > The paternalistic/maternalistic attitude of Paul and Jessica is more than a > little cloying. Frankly, it seems like they enjoyed the position of power > granted them by the cash in their wallets.

In a setting where ideas change rapidly and companies start overnight, having a hunch about people even through failure might be completely justified. "You've produced garbage so far, but I'm confident that you'll make something good" is much more reasonable in the startup world than it is in, say, large- scale retail. The investment that PG makes in a prospect is really, really low; if "Muffin" doesn't produce, it doesn't really matter to him.

If this paternalistic attitude didn't work, Y Combinator wouldn't be doing it. To use a sports metaphor, they're drafting a football team rather than a baseball team. Baseball players can be studied statistically because the college game is similar enough to the professional game that you can study prior data. In contrast, football is a crapshoot; the college game is so different from the pro game that there's barely a correlation at all between good college players and good pro players. So is choosing startups to fund.

If the Muffins end up making money, that's a great indication that PG & Friends having an "in crowd" is producing results.

If they're incorrect, they're opening the door for a more rational company to do the exact same job as them without Muffins and beat them.

<comment><USER4> As far as I can tell your response is off topic.

I never said it didn't produce results, I said that it had cultural effects.

>If this paternalistic attitude didn't work, Y Combinator wouldn't be doing it.

Your argument seems essentially "it exists, therefore it's fine." Minimal competence also produces results. That doesn't mean the flaws of an approach are mitigated or nonexistent.

My overall point has nothing whatsoever to do with results. What I'm saying is something about ugliness and ugly people and how our culture treats them despite the fact that we are _all_ ugly people.

Unfortunately for Paul and Jessica, they are in public view quite often; in this article we get a glimpse of their ugliness. But they also consistently accept the rewards given them because of this exposure. Good/bad for them, depending on your viewpoint of that tradeoff.

I dwell on them because the disparity between how our culture punishes and rewards 'professionalism' and lack thereof.

This is about the lie of professionalism. Perhaps you're saying professionalism doesn't matter if, even during its absence, results can still be had? I don't see how that counters anything I've said.

</comment></comment></comment><comment><USER6> I really enjoyed this in depth look at Reddit's history. I never realized what an early member Aaron Swartz was, and found a lot of the Conde stuff pretty revealing. The competition and eventual victory over Digg also has some nice anecdotes to take away. And I'm impressed by this kind of journalism coming from Mashable, it'd be nice to see more of this.

</comment><comment><USER7> I'm surprised that no mention was made of the big rewrite. A lot of us early users were lisp/Graham followers and intrigued by the fact that the site was written in lisp. I seem to recall the rewrite being a big deal.

<comment><USER8> I too remember it being pretty harrowing at the time (if only because it felt like we were alienating [har] such a large chunk of our community), but in fairness it happened barely 6 months in. It all happened in a couple of weeks of quick work and I seem to recall we even recycled the db schema.

We did a post-acquisition python-to-python rewrite -- clean out the cobwebs, switch frameworks, and redo the database schema -- which was _much_ harder, longer, and more painful to deploy. Though in that case it was one of those drastic changes whose primary side effect was that _no none notices a thing_.

</comment></comment><comment><USER9> This article only touches lightly on what I believe to be Reddit's primary weakness, namely the inability to elect moderators for various subs. If someone is able to do a Reddit-like site with a fair means for nominating and electing moderators they will have a good chance at eclipsing Reddit's success.

Reddit could also implement this themselves, and given the sometimes questionable actions of some moderators (e.g. when all Tesla-related stories were being banned from /r/technology ) this would help to legitimize the nominal democratic nature of the site.

<comment><USER10> Keep in mind giving mods power incentivises mods to build, grow, and.. well... moderate their little corner of the internet on reddit's platform. You take that power away and good luck keeping mods working hard for you for free.

<comment><USER9> That power would still be there, it would just be checked by the possibility that they might one day lose it.

</comment></comment></comment><comment><USER11> This article really struck a chord with me. It reminds me of a lot of the issues we've had running Neocities ([https://neocities.org](https://neocities.org)), which gives people free space to make their own web sites.

When you operate an open platform on the modern web, the first thing you quickly learn is that you will end up being embroiled in a lot of random controversies you never expected to have a role in. We've had doxxing issues, got caught in some "Gamergate-esque" drama, and even had a DDoS site (using jQuery, pretty ingenious actually) put up that ended up taking down the President of Mexico's web site (you can't make this stuff up). That little stunt got me a few threatening emails from, literally, the Federales (Mexican Federal Police). And of course, there's the usual litany list of pill spam (arbitrage between other countries and US citizens who pay full price is, somewhat hilariously, one of our biggest spam problems) and whatnot.

The biggest epiphany I've had over the course of the year is on the true nature of "freedom of speech" on the internet. You start to learn very quickly that not all speech is free or equal. It may be under the Constitution, but if you're a startup and a multi-million dollar corporation or university (yes, we've had issues with Universities too) shows up and has high-powered (and bored) lawyers to waste your time and money harassing you, it doesn't matter if their complaint is legally valid or not. Your last resort of defense becomes public opinion. If you can't get public support for defending someone's freedom of speech, it doesn't matter what the Constitution says. As Thomas Jefferson once wrote, "the boisterous sea of liberty is never without a wave".

I'm not trying to sound dismal or depressed. Neocities has been exhausting at times, but I'm having a lot of fun. I tend gravitate to stressful startups, and (I blame this on my previous work in political activism) tend to prefer them. Neocities overall has been going very strong, and it gives me a lot of joy to check out the new sites every day and see people learning how to write HTML. It's worth it. And I think, with knowledge and experience, the issues are manageable.

Tying this back to Reddit: Controversies aside, I think Reddit has done a great job over the last few years, and I want to congratulate them for the growth, and salute them for weathering the "boisterous seas" of being a tolerant, open platform. I don't know Yishan Wong personally and can't vouch for him on a personal or operational level, but I feel that Reddit has done a good job under his leadership, and I hope that people don't unfairly pigeonhole him as a failed leader because of the difficult nature of their work. In perfect honestly, having context for some of the issues they've had to deal with, I may have done far worse in his position.

This article really makes me want to meet the Reddit team. I bet there's a lot I could learn from them.

TLDR: Running an open platform is tough. Cut them some slack.

</comment><comment><USER12> I was hoping it would mention the growing problem of astroturfing and "viral" marketing.

<comment><USER13> Post OC to reddit, get upvotes, create the OC and go to jail.</comment></comment>


Put the "Ph" Back in PhD - <AUTHOR> http://magazine.jhsph.edu/2015/summer/forum/rethinking-put-the-ph-back-in-phd <comment><USER2> > We need big thinkers

We need some, but there aren't anywhere near enough big-thinking jobs for all these PhDs.

> The success of science and technology will always lead to new ethical > dilemmas. How do you spot them and try to work with society before they > become problems? A better appreciation of ethics is likely to address what > has become an epidemic of retractions in the scientific literature.

All the recent retractions have nothing to do with scientists being unaware of ethical dilemmas raised by their work. They're to do with perverse incentives, p-hacking, and occasionally deliberate dishonesty.

<comment><USER3> _> We need big thinkers We need some, but there aren't anywhere near enough big-thinking jobs for all these PhDs._

Yeah, man, jobs! It's all about jobs! Jobs jobs jobs! Steve Jobs! Jobs jobs!

How about this? The world needs big thinkers in ways other than their capacity as economic actors. Einstein worked at a patent office. The work he did there (for the patent office) was not his primary contribution to the world at that time.

<comment><USER4> Yeah, I think the author forgot that:

1\. Being smart isn't a product, and most people won't consider it a skill

2\. Thinking big isn't a product, and most people won't consider it a skill

3\. Thinking big is usually reserved for top level people, of which there are less than five

You know what they call the big thinker who is looking for a job with his shiny new PhD? Unemployed.

<comment><USER5> _> You know what they call the big thinker who is looking for a job with his shiny new PhD?_

In technical fields (which is what this article is about)?

The ones who choose to stay in academia and are lucky enought to find a position are called "Professor".

The unlucky or disillusioned ones -- as well as the (majority) of PhDs who never even intended to go into academia at any point during thier studies -- are called, variously, "Engineer", "Analyst", "Consultant", etc. and typically start in the low six figures (and not even in the bay area / NYC). They take a marginal hit to their lifetime earnings, but in most cases make decent salaries and get to work on more interesting problems (obv. interesting is subjective).

 _> Unemployed._

The unemployment rate for STEM PhDs is ludicrously low (FYI: if you google this, be aware that the Slate article by Jordan Weissmann is IMO purposefully misleading. See Table 1 of [http://www.nsf.gov/statistics/infbrief/nsf14310/](http://www.nsf.gov/statistics/infbrief/nsf14310/) and compare to his graphs. E.g., his graph seems to indicate that the unemployment rate for CS PhDs was over around 15% in 2001, but the NSF tables makes it clear that the rate was _0.9%_ , which is even more than what most economists refer to as full employment. In fact, in CS, the increase for 0.9% to 2% is almost entirely attributable to there being a lot more retired CS phds now than 10 years ago (for rather obvious reasons). So the stats he's using to tell his story are, at very best, unintentionally but absurdly non- representative of the actual state of affairs. It's like asking someone the day after they quit their job if they've started their new job yet and, when they say "hell no, taking some time off to recharge", assuming they're unemployable and/or don't have something lined up.)

</comment></comment></comment></comment><comment><USER6> I think this article mixes together and perhaps confuses a few different things.

Stated problems:

\- scientists are not equipped by their training to be able to tackle "big" challenges such as meteorites and pandemics.

\- "hyper-competitiveness" for grants

\- extreme specialisation means that people can't communicate.

And the solution is to learn about:

1\. epistemology 2\. quantitative skills 3\. ethics

I certainly don't disagree with the value of learning about these things. But while epistemology and quantitative skills might help make the scientific process more rigorous, I don't see that it will change competitiveness or the inability to tackle big challenges or deter specialisation.

And while ethics is always a valuable thing to think about, I don't think it helps scientific rigor or any of the author's stated problems.

Overall I found the piece a bit muddled.

<comment><USER7> Quite muddled.

</comment></comment><comment><USER8> Sure, let's "put back" something that was never there in the first place. Philosophiae Doctor is just the name of the degree. Where did it come from? Let's ask wikipedia ( [https://en.wikipedia.org/wiki/Doctor_of_Philosophy](https://en.wikipedia.org/wiki/Doctor_of_Philosophy) ):

> In most of Europe, all fields other than theology, law, and medicine were > traditionally known as philosophy, and in Germany and elsewhere in Europe > the basic faculty of (liberal) arts was known as the faculty of philosophy.

<comment><USER9> Philosophy is also responsible for the scientific method and things like the null hypothesis. There's a _strong_ tendency for 'hard science' folks to look down their noses at philosophy, but the truth is that the fundamentals of their research is squarely due to the philosophy. Philosophy is responsible for scientists developing the methods of rigorous observation.

Source: I once looked down my nose at philosophy, then I started doing first- priciples on the methodology I was taught. Turns out that 'philosophy of science' is a field unto itself.

<comment><USER10> You've got it backwards. It's not like a bunch of philosophers sat round and came up with the scientific method and until they did that there was no science. Scientists experimented and tried to justify their conclusions (sometimes badly or incorrectly), which lead to the philosophy of science. Karl Popper was a recent philosopher, not a forming giant of science, he rationalized the scientific method with falsifiability after the fact, after Einstein's Theory of Relativity, for example.

It's also easy to get the two mixed up as Philosophy basically used to be the nascent 'science'.

It's also worth reading this before you get too attached to philosophy:

[http://www.paulgraham.com/philosophy.html](http://www.paulgraham.com/philosophy.html)

Ultimately present day philosophy, last I looked and certainly since Wittgenstein, is all about playing with word meaning and almost upfront about it, but pg argues in reality it always was like that.

<comment><USER11> You are both right and wrong. Philosophy (asking questions) and science (testing questions) are depending on each other.

<comment><USER12> > Philosophy (asking questions) and science (testing questions) are depending > on each other.

Only in the instances where philosophy asks questions science can test.

Questions like "What is The Good?" and "Is it morally right to do this thing?" are not those kinds of questions.

<comment><USER9> Philosophy also discusses things like how we consider a answer to be robust; stuff we now consider pretty ingrained. It's not all religiose has-no-answer pondering.

It's just that most of the 'ingrained' stuff has been settled now, and people are more interested in writing about new, unanchored stuff. The same is true of science - few professional researchers bother retreading settled ground. How often do you see an article by modern professional scientists that examine the basics of the electrical resistivity of copper?

</comment></comment></comment></comment></comment></comment><comment><USER13> The Senior Executive Provost of Whatever sees your department as nothing but a cost center. Literally. I agree completely with the author, but I'll add one more thing: we need big ideas to overcome the market-fundamentalist ideology that drives university administrations today.

</comment><comment><USER14> I had a friend/college from Russia, he told this _is_ the case there. During the thesis defense there is a section reserved for Philosophy (of life, meaning etc). Any Russians care to elaborate?

<comment><USER15> Can't elaborate on the thesis, but prior to enrolling for a PhD program you indeed have to pass a philosophy exam. This is usually considered the most difficult. Passed it in summer.

<comment><USER16> I remember taking this. I had to answer something about socialism or marxism or whatever (I remember having to name the prominent socialist thinkers). This was in the 2000s, by the way. I got 3 points out of 5. I passed the other exams, math and English, perfectly, and that was not enough. I only got accepted due to personal recommendation of my scientific advisor. Since then I'm of the opinion that "Ph" is complete bullshit.

<comment><USER15> I prefer to think of "Ph" as a tradition.

<comment><USER17> I would love a Doctorate of Tradition. Would never expect to be hired, though.

</comment></comment></comment></comment></comment><comment><USER18> I find it odd that one's exposure to and encouraging of bigger-thinking is somewhat reserved until the PhD level and that, after a long while of not exposing or encouraging such, our systems had ever expected people to suddenly think big once at the PhD level.

It may be more productive to have the expectation of bigger-thinking introduced earlier and let that expectation be realized more fully at the PhD level.

<comment><USER19> This 100x.

By the time you're getting your PhD, the college graduate should have read the classics, done at least 1st year level calculus, biology, physics, ethics, philosophy, and 2-4 years of a foreign language.

High school and undergraduate is the time to be broad-minded. PhD is for very focused work (with the big picture in mind).

Personally, I read the classics and learned Greek and Latin (high school), then in college took 8 courses of philosophy, 2 of Spanish, 2 in economics, 4 in business/entrepreneurship, and a smattering of other interesting things (art, music, etc...) while getting a Physics major.

Then I went and spent 6 years learning more about one thing than anyone in the world.</comment></comment>


A comparison of 6 USB Stick Microcontroller Dev Boards - <AUTHOR> http://www.retroprogramming.com/2011/12/usb-stick-microcontroller-dev-boards.html

<comment><USER2> I can also recommend Micropendous, it's really well-designed. Teensy is also great for slave-only applications.

      http://www.pjrc.com/teensy/

</comment><comment><USER3> Is there any kind of gpio board intended for real-time/thetered use, without code reloading or writing my own communication layer?

<comment><USER4> I used to sell a couple, waaay back when. These days I think Phidgets might be what you are looking for.

I have a few designs (Bluetooth & RS232 based) that I customize for my clients. I may be able to help you, but I should warn that my prices are not exactly hobbyist-friendly :-)

<comment><USER5> Maybe you could consider open-sourcing one of your designs ?

It could attract future clients by showing your proficience. And one design would not endanger your paid work in any way :)</comment></comment></comment>


Ask HN: What is your developer workstation? - <AUTHOR> I'm currently running with a 2015 Macbook Pro. My shift from academia to the real world comes with a fund to buy new toys. What is your current set up and what would you change? <comment><USER2> My main workstation is an Alienware Aurora. For the most part it works great. The thing I don't like in the end is that internally it is filled with custom junk that is difficult to upgrade/replace. Support from Dell tends to be craptacular.

The one thing I would likely also change is that at the moment I have three monitors spread about landscaped, horizontally. If I was going to start with all new hardware I would get at least one monitor potentially all 3 that had the option or swiveling into portrait mode.

For what it is worth, I am currently considering a workstation from Boxx.

</comment><comment><USER3> I currently use dell XPS with very less applications. I mostly interact with FireFox and Bash terminal. I usually use Sublime text for editing and writing code. My machine is powered by Regolith linux which offers i3 configured out of the box instead me doing it myself. Regolith is powered by ubuntu. Lastly my machine is very simple I wouldn't say it's powerful it uses intel i5 processor with 8gb ram

I hope this helps and if i get a chance to switch I would go with ThinkPad x1 Carbon

<comment><AUTHOR> Thanks, that's really helpful to know!

</comment></comment><comment><USER4> Just your average decked-out "gaming" PC. An approximately $3000 AMD/Gigabyte with a smaller-sized 4K monitor.

[https://secure.newegg.com/wishlist/md/7519915](https://secure.newegg.com/wishlist/md/7519915) (I listed all the parts here, but I didn't buy them all here)

I don't really game.</comment>