How Urbanization Can Be a Friend to Birds - <AUTHOR> http://aeon.co/magazine/society/how-urbanisation-can-be-a-friend-to-birds/

<comment><USER2> Not really surprising especally when you consider how much food people leave out on a regular basis. However, this ignores the support infrastructure needed to enable these suburban spaces. Food, water, energy, raw materials, manufactoring, and shipping are all out of site and out of mind, but sill impact birds who cover large areas.</comment>


Software Testing Anti-patterns - <AUTHOR> http://blog.codepipes.com/testing/software-testing-antipatterns.html <comment><USER3> > Writing tests before the implementation code implies that you are certain > about your final API, which may or may not be the case.

How does this myth continue to persist?

Writing the test first has nothing to do with knowing the final API. When I write tests first I am _looking_ for an API. The tests help guide me towards a nice API.

I personally find TDD in this manner works best when taking a _bottom-up_ approach with your units. I start with the lowest-level, simple operations. I then build in layers on top of the prior layers. Eventually you end up with a high-level API that starts to feel right and is already verified by the lower- level unit tests.

Although this style of development has become less prevalent in my work with Haskell as I can rely on the type system to guarantee assertions I used to have to test for. This tends to make a _top-down_ approach more amenable where I can start with a higher-level API and fill-in the lower-level details as I go.

<comment><USER4> > I start with the lowest-level, simple operations. I then build in layers on > top of the prior layers.

I was discussing this with a coleague this week: starting by the lower level details vs starting from the more abstract/whole api. I prefer to start by writing a final API candidate and its integration tests and only write/derive specific lower level components/unit tests as they become required to advance on the integration test. My criticism on starting with the bottom-up is that you may end up leaking implementation details in your tests and API because you already defined how the low level components work. I have even seen cases where the developer end up making the public API more complex than necessary due to the nature of the low level components he has written. Food for thought!

<comment><USER5> Your point of view resonates with me.

Over the years, I've learned to test "what a thing is supposed to do, not how it does it". Usually this would mean to write high level tests or at least to draft up what using the API might end up looking like (be it a REST API or just a library).

This approach comes with the benefit that you can focus on designing a nice to use and modular API without worrying on how to solve it from the start. And it tends to produce designs with less leaky abstractions.

Of course YMMV.

<comment><USER6> I just updated our "best practices" documentation to include the recommendation that tests be written against a public API/class methods and the various expected outcomes rather than testing each individual method.

I think the latter gives an inflated sense of coverage ("But we're 95% covered!") but makes the tests far more brittle. What if you update a method and the tests pass but now a chunk of the API that references that method is broken, but you only happened to run a test for the method you changed?

I like to think I'm taking a more holistic view but I could also be deluding myself. =)

<comment><USER7> Why doesn't your best practices document include a requirement to run the whole test suite?

<comment><USER8> In some contexts there may be valuable tests that take long enough to run they should be run out-of-band. That said, I don't see where the parent says they _don 't_ run the whole test suite.

<comment><USER7> > now a chunk of the API that references that method is broken, but you only > happened to run a test for the method you changed

<comment><USER8> Ah, yeah, not sure how I missed that. Maybe they meant the test for the broken method isn't run because it doesn't exist, but I very much agree that that's not the most natural interpretation.

<comment><USER6> Sorry, this wasn't totally clear and I thought "run all tests" was implied. What I was trying to get at was the difference between "we have a suite that includes individual tests for every single separate method so we have great coverage" vs "we have a suite of tests that run against public APIs that still manage to touch the methods involved". The former may test "everything" but not in the right way, if that makes sense.

I should have said "you've only written a test" rather than only having run a test.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> In small companies where there is no time to "waste" on tests, my view is that 80% of the problems can be caught with 20% of the work by writing integration tests that cover large areas of the application. Writing unit tests would be ideal, but time-consuming. For a web project, that would involve testing all pages for HTTP 200 (< 1 hour bash script that will catch most major bugs), automatically testing most interfaces to see if filling data and clicking "save" works. Of course, for very important/dangerous/complex algorithms in the code, unit tests are useful, but generally, that represents a very low fraction of a web application's code.

<comment><USER10> Yep. There is a kind of goldilocks zone:

\- Company starts out, lots of change and RFCs, testing at a shallow depth (high-level tests wherever possible) helps a ton

\- [Goldilocks zone] Company and team settles down, predictable workflow, "not changing the world" anymore, unit tests help

\- Project goes into legacy, needs to be split/ merged/ refactored/ rearchitected/, again unit tests don't help. Back to shallow tests and stranglers

Only very small part of programming today is doing hardcode datascience and algorithms. Rest are all just plumbing from A to B to C. I _always_ argued in teams to focus on higher level tests, but then, its easy to get shouted down because of "TDD". Sadly, very few people seem to focus on objective rather than process.

<comment><USER11> Yea exactly, it's often painful to see when all you need is an API level test which will anyway cover all the use cases instead of the low level unit tests written for the sake of TDD.. leaner the code more maintainable it is

</comment></comment></comment><comment><USER12> I don't understand why people are so enamored of the distinction between "unit" tests and "integration" tests. That dichotomy has never seemed useful to me. In my mind the relevant distinction is speed. There's one fast test suite that you run before every code commit, and another slower suite that you run every night on a cloud instance or testing machine, and review the results in the morning. The fast/slow distinction is roughly comparable to the unit/integration distinction, but I don't have any problem with testing a DB interaction in the fast suite - and in fact consider it to be a good practice - if the interaction is fast enough. In general the fast suite should check as much as possible within the constraint of needing to be fast.

<comment><AUTHOR> >The fast/slow distinction is roughly comparable to the unit/integration distinction

I think you answered the question yourself. Speed is also shown in the table that defines the types of tests.

You can name the categories foo tests and bar tests if you wish.

Anti-pattern 1 - is companies that have only the "fast suite" you are talking about and anti-pattern 2 is companies that have only the "slower suite".

<comment><USER13> I'd prefer that all testing be done with live data in a real database ... maybe not at the very beginning but as soon as practicable. I came into a project recently with so many issues around differences between real data and assumed data, and unrealistic assumptions about real data that it's been hugely painful, with lots of back and forth between Dev and QA and Ops. With Docker now there's no excuse for not setting up soon in a project's lifecycle a realistic environment (infrastructure topology-wise) with real data on development and QA laptops.

With this you still have fast and slow perhaps, but everybody can run all but the most extreme scaling tests.

<comment><USER14> That's E2E test. What's interesting with unit tests is that you can run them locally, offline. They are fast enough so that you can run them and see if you broke anything. If your tests take too much time to start or relies on a live env (which may be down or broken), you'll spend more time for maybe nothing.

<comment><USER13> With Docker I can run my E2E "offline".

<comment><USER15> If you're touching real data, your test isn't offline.

<comment><USER16> Not true. You might be touching local sample of real or realistic data. It does not need to be static either.

The performance of such a test is still a problem.

</comment></comment></comment></comment></comment></comment></comment><comment><USER17> Got to quibble with Antipattern #9 about test code - there's certainly a place for extracting common patterns from test code (factories, complex setup) but too much DRY in tests can make for hard-to-modify spaghetti. Sometimes it's easier to just repeat a few lines of code in the name of clarity.

<comment><USER18> Similarly, overly elaborate test code can itself be buggy. A specific instance of this anti-pattern is when the test code duplicates the same logic as the code under test, logic which turns out to be incorrect.

In test code, I am a lot more tolerant of straight-line repetition in the name of simplicity, in a manner that would be worth abstracting were it in non- test-code.

<comment><USER19> That's a sign that you need to write tests to test your tests. It's part of treating your test code as a first-class citizen. I always write a test suite for my test suite.

Just kidding.

<comment> squeaky-clean I know this is a joke, but I do have a few "don't panic" tests that do extremely simple stuff, like just run a test and return True immediately, connect to db, connect to mock db, etc.

Sometimes I do something stupid like put an extra comma somewhere without noticing, or update my db driver library without also updating my mock library. I think we've all had that moment where tests fail or it doesn't compile because you thought you copy-pasted some text into an email, but it got pasted into your editor window.

So when I see that literally every test has failed, I know it was some really dumb but likely simple mistake on my part. If 180/195 tests pass.... Shit.

</comment></comment></comment></comment><comment> V-2 _" I have seen projects which have well designed feature code, but suffer from tests with huge code duplication, hardcoded variables, copy-paste segments and several other inefficiencies that would be considered inexcusable if found on the main code."_

I agree with "code duplication" and "copy-paste segments" (although I fail to see how they're two different things. It looks like example duplication to me?)

I _don 't_ agree with hardcoded variables. In tests, they're okay. Tests are _not_ production code. In this case I side with Misko Hevery - <USER2> [https://youtu.be/jVxmk-tVo7M?t=2m54s](https://youtu.be/jVxmk-tVo7M?t=2m54s)

Tests _should_ hardcode values, and (ideally) not contain _any_ logic, not even of the simplest sort. That's the point, that's how we can reliably confront them with production code without risking bugs that fall off off our radar because the same faulty logic leaked from production into tests. Tests should be kept all dumb, all naive, all hardcoded.

That's the thing which proves to be the most difficult to convince fellow developers about, as they're typically used to mechanically transferring over all the usual best practice from production code into tests.

"Ooh, this is so hardcoded" \- "good". "This method name is so verbose" \- "good; will we ever call this method _from code_?". "This could be private" \- "it's a class _with unit tests_ , how will this code possibly be called from some other class while it executes? Visibility modifiers are waste of space here". And so on

</comment><comment><USER20> I don't agree with Anti-Pattern 10 - Not converting production bugs to tests

I used to think this was a good idea too, until I saw the real statistics on a project on this.

This project (50+ developer team) tracked all bugs, and also if they were regressions or not. Almost 0 regression occurred of bugs that were fixed before.

All testing needs to consider return on investment. The reality, at least for that project, was that testing time was best spent elsewhere.

<comment><USER21> Doesn't that just mean that the tests were effective? The goal of doing a test for each bug is to prevent them from happening again. Unless you meant that this project didn't apply this particular advice but was getting away with it just fine?

I personally believe that this advice is one of the most important one actually for a very specific reason: it leaves a very bad impression to client when a bug keep happening again and again after being fixed. Unfortunately I've lived this experience when I was just starting out in my career on a similar project (around 45-50 developers) with basically no tests at all. It wasn't fun explaining to the client, even if they were internal to the company, that the bug we fixed last month had to be fixed again.

<comment><USER22> > _Doesn 't that just mean that the tests were effective?_

Only in the sense that:

\- What's this?

\- A device that keeps tigers away.

\- But there are no tigers in Los Angeles!

\- See how effective it is?

> _The goal of doing a test for each bug is to prevent them from happening > again. Unless you meant that this project didn 't apply this particular > advice but was getting away with it just fine?_

No, he means that they had code to test for the presence of fixed bugs, but nobody reintroduced said bugs and triggered that bug catching test code ever. So even if they didn't have the code, the end result would have been the same.

<comment><USER23> How would you know? If code is failing tests, it doesn't leave my laptop.

<comment><USER22> Because parent said they tracked the test suites.

Perhaps the tests don't run on the devs laptop, but on an integration system (we have such a setup).

(And of course you can make test suites report failures centrally, whether they tests run on a laptop or not).

<comment><USER23> They said they tracked _bugs._ A test failure during development doesn't generally go in the bug tracker.

</comment></comment></comment></comment></comment></comment><comment><USER24> A small quibble with #8 (no manual testing) - I can't recall the number of times a program has passed all the tests the _creator thought to write_ , only to fail when someone actually looked at the output.

Automated testing should not be the end of the testing. It should be the beginning. Manual testing should be the last step, even if every manual test is immediately automated - there's always more to test.

Also, we should be sure to include combinatorial and fuzz testing in that pyramid as well, since skipping them leads to someone coming in with AFL and exploiting the hell out of your app.

<comment><USER25> Agreed. The number of times that manual tests have unearthed completely unthought of situations is huge. A good QA tester will do weird and wonderful things that no developer ever thinks of doing, never mind testing for. Saying you shouldn't manually test is like saying you shouldn't shower. Sure, you can get away with it for a time, and it saves time! But eventually someone's going to point out you smell bad. Having a customer have to tell you your product is broken (smells bad), because you didn't know, because you didn't bother testing, is seriously not what you want.

</comment></comment><comment><USER26> >> Anti-Pattern 2 - Having integration tests without unit tests

I strongly disagree with this one. I think that unit tests only make sense when the project's code has really settled down (not likely to change in the future) and you want to lock it down to prevent new developers on the team from accidentally breaking things.

Unit tests severely slow down development. I've worked on projects where it takes 2 to 3 days to update a single property on a JSON object on a REST API endpoint because changing a single property means that you have to update a ton of unit tests. The cons of unit testing are:

\- It locks down your code, so if your code is not structurally perfect (which it definitely is not for most of the project's life-cycle) then you will have to keep updating the tests as you write more code and more functions around.

\- It encourages you to use certain patterns like dependency injection which might make sense for some (e.g. statically typed) programming languages but are unsuitable for other (e.g. dynamically typed) languages because they make it difficult to track down dependencies.

\- It only makes sense for parts of the project that have strict reliability requirements and where any downtime/failure in that part of the code would result in some loss of business. It's important not to underestimate the maintenance cost of unit tests. More unit tests means much slower development (cuts productivity to half or sometimes even a quarter of what it was without tests for small teams), which means that you need to hire many times more developers to get the same productivity that you could get from a single developer. Sometimes it's OK if a part of the code breaks in non-critical parts of the system; especially if you have some kind of user-feedback system in place.

<comment><AUTHOR> > I've worked on projects where it takes 2 to 3 days to update a single > property on a JSON object on a REST API endpoint because changing a single > property means that you have to update a ton of unit tests

You just described anti-pattern 5. Did you read the full article?

<comment><USER26> Anti-pattern 5 is contradictory to the rest of the article...

>> Tests that need to be refactored all the time suffer from tight coupling with the main code.

What is the author proposing? To write unit tests that are only 'loosely coupled' to the code that they are testing? In my entire career, I've never seen a single unit test case that matches this description.

If it's loosely coupled with the internal code then by definition, it's called an integration test.

Anti-pattern number 5 is basically the author admitting that internal unit testing is a problem in terms of productivity but then they fail to offer an actual solution which doesn't contradict the rest of the article.

Sometimes your code needs refactoring, you need to change the fundamental structure of how some objects interact with each other and when that's the case, unit tests actually discourage you from making the necessary changes of pulling the whole class definition apart (thereby invalidating all the unit test cases for that class) and moving the code to smaller or more specialized classes.

<comment><AUTHOR> >In my entire career, I've never seen a single unit test case that matches this description.

That is not an argument. The fact that you have been doing something your entire career does not make it correct

>If it's loosely coupled with the internal code then by definition, it's called an integration test.

That is your own definition. The article defines an integration test right at the start. It is ok if you have your own definition but that does not mean that everybody has to agree with you.

>Anti-pattern number 5 is basically the author admitting that internal unit testing is a problem in terms of productivity but then they fail to offer an actual solution which doesn't contradict the rest of the article.

The article has an example and shows both the problem and the solution. The solution is to make your tests not look at internal implementation. What more could I do there?

>unit tests actually discourage you from making the necessary changes

you are just describing again what anti-pattern 5 says.

>What is the author proposing?

I am the author, so I know what I am proposing, that is for sure.

<comment><USER26> >>In my entire career, I've never seen a single unit test case that matches this description.

> That is not an argument. The fact that you have been doing something your > entire career does not make it correct

I've worked for many different tech companies in my career (both startups and corporations) and the vast majority of these unit tests were not written by me.

Also I've worked on many open source projects. Same story.

<comment><AUTHOR> >Also I've worked on many open source projects. Same story.

Let me put this way. I am writing an article on how to keep your body healthy and provide a list of common mistakes.

Anti-pattern 5 is "you should stop smoking".

And your argument is "I have been into too many companies (startups and corporations) where people have been smoking all the time. So anti-pattern 5 is wrong."

Is this more clear?

</comment></comment></comment></comment></comment></comment><comment><USER27> I had hoped that this would be a catalog of concrete code examples of common or easy to make mistakes, along with concrete prescriptions for what to instead. Instead it turns out that it is mostly a long winded collection of opinions passed off as insights along with a few general principles that basically amount to _good tests are good_ and _bad tests are bad_. Who reads this and is any wiser as to how to write valuable tests? Writing tests that add value is hard, is a skill that must be acquired. In my experience, most developers who have given up on testing have done so, because they'd been sold the false notion that testing is trivial and always adds value. What they've found instead is that _testing is difficult_ (because they lack the skill), and that the tests they did write, were of questionable value. No wonder they gave up on it. What these people need is not another article talking in loose terms about the wonders and virtues of testing.

<comment><AUTHOR> >of concrete code examples

Examples of what? Python? Ruby? Java? C++? You cannot please everybody. The article is written in a way that touches all developers. And judging by the feedback I got it has succeeded in this way.

>that basically amount to good tests are good and bad tests are bad

And the article helps people to understand which tests are good and which are bad in the first place. If you are already an expert on the subject then maybe you are the wrong audience.

>Who reads this and is any wiser as to how to write valuable tests?

If you think you can do better then by all means I am expecting your article on the subject

>What these people need is not another article talking in loose terms about the wonders and virtues of testing.

Please write the correct article then.

</comment></comment><comment> UK-Al05 I thought we got past the idea that a unit is a method or class.

The original interpretation means a unit of behaviour. When you start thinking like that, intergration tests become less important. But still important.

The idea that a unit in unit testing was a class was a misinterpretation.

<comment><USER4> Totally agree, and this misinterpretation got crystallized into your tools that often recommend/generate tests that are named TestSubjectClassNameTest->eachMethodTest. We have to pay attention to that ;)

</comment></comment><comment><USER28> "Anti-Pattern 2 - Having integration tests without unit tests"

The first project I worked from after graduating had this problem when I came onboard, all the tests were slow as all hell and 80% of each test did the same thing (clearing the entire database and seeded in new test data, for _every_ test). I felt like I made a mistake becoming a programmer because of how gruesome this anti-pattern is when you work with big Java applications. Then we hired a senior developer three months later who promptly started breaking up the tests after checking with the team if that was ok. The productivity of the entire team increased by many orders of magnitude!

<comment><AUTHOR> Yes this is exactly what I was talking about. The anti-patterns are mentioned in order of appearance, so this is very common.

Don't let problems like these disappoint you. There are companies that don't suffer from any of these anti-patterns

</comment></comment><comment><USER29> Really enjoyed reading this article as a Sunday afternoon long-read. Well structured and covers a lot of things I have experienced but haven't formed into such a clear description. Also really appreciate the realistic examples!

<comment><AUTHOR> Thank you! Yes it took me a long time to write and thinking about good examples is always hard...

</comment></comment><comment><USER10> Superb article (some small nitpicks but they are ignorable). I would like to add just one more anti-pattern:

 _Focusing too much on fast feedback_

Nope. One should get priorities straight. The #1 purpose of tests is safety. The "sleep well in the night" test. The "deploy with eyes closed" test.

Performance of the test suite, while "nice to have", is _not the core objective_. If push comes to shove, if it comes down to safety vs performance, safety should just win hands down as a principle.

Doing a bunch of mocks for speeding up 80% of unit tests? Great, but its a borrowed debt, and must be balanced out with 20% of higher level tests.

<comment><USER30> I find this a bit of a strange statement.

I started with Extreme Programming in 99 and grew and evolved through all the refinements to the process around testing. It's always been about providing safety. Fast has always been about not running silly tests that take too long

Fast Feedback is a core objective. It is not in competition with safety, safety is an integral part. Safety is the feedback you are expecting. We want to know about that safety fast.

A popular term that came around in the early 2000s was "Brain Engaged", meaning you needed always to be aware of why you were doing things and not following blind rules. Meaning you need to know the purpose of going fast.

The whole point is to go as fast as possible safely.

Some of the biggest challenges is how to get things quick while maintaining safety. Kind of makes no sense to have fast tests with no safety.

Now you mention mocks, and I have seen people mock in very strange ways that devalue tests.

I like the general guidance from Kent Beck "I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence"

</comment></comment><comment><USER31> "Linux command line utility ... In this contrived example you would need:

        Lots and lots of unit tests for the mathematical equations.
        Some integration tests for the CSV reading and JSON writing
        No UI tests because there is no UI."

As with all command line apps, the command line is the UI and requires testing ( input parameters, error messages, etc... )

<comment><AUTHOR> You are right. I will change the wording to say GUI instead of just UI

</comment></comment><comment><USER32> In cases where manual testing is the only option, is it an anti-pattern to let the testing be done by the person who developed the code?

As a developer, I feel that others should test my code. Is that reasonable?

<comment><USER33> Why should other test your code?

<comment> ben-schaaf Same reason you don't review your own code, or pair program by talking to yourself.

<comment><USER34> This seems a bit of a false generalization to me. For a long time I've been the sole developer on multiple large-ish applications. So of course I review my own code, and I write my own tests. And yes as another commenter mentions: if you don't pay attention this can lead to tunnel vision, and yes it can be hard to be consistent about it.

I discovered those drawbacks fairly early, so quickly made it a habit to be super-strict and critical about pretty much every single line of code I write (both with regards to style, functionality, how it adheres to the standard practices etc).

What I also do consistently is review older code: if I have to add a function to existing code, it happens more often than not that I'll re-read and review the surrounding bits. Often to come up with better code. If time permits, else I take note and go back to it later.

In practice it turns out that if I haven't touched code for months, reviewing it is almost the same as reviewing someone else's code, i.e. starting with a clean slate. Which is also why I'll sometimes write something, commit it, but not yet merge it to production. Then a week or so later I'll come back and review every bit again.

<comment> ben-schaaf Oh, I completely agree, though I wouldn't call writing code carefully and methodically a "code review" if that makes sense.

I personally find that me a couple months down the track is pretty much another person anyway. In a sense you're not reviewing your own code, you're letting it be reviewed by future you. But if you're in a team with multiple developers, having someone else review your code would be more efficient.

<comment><USER35> I tend to review my code with a reviewer hat, not with the author hat, before I open the PR. Find all the places where I'd call myself out for lazy names, poor abstractions, missing docs, etc. then go in and fix those. Repeat until satisfied, then open PR.

</comment></comment></comment></comment></comment></comment><comment><USER36> These are exactly the posts I need to level-up my game. More abstract, top level views without going into detail on setting up system-x to make it work, i can figure that out myself! Any more of these types of articles you guys and girls recommend?

<comment><AUTHOR> Yes I wrote it because it is my main complaint. The internet is full or articles that deal with a specific topic that is very narrow and never give the big picture away.

Frankly the only good source of such material is books (especially the initial chapters where they set the stage for everything)

If somebody else knows where these types of articles exist, I would also like to know where they are.

<comment><USER37> Great article. I don't agree with 100% of your points but testing is a field where everyone has their own opinion shaped by their own experience, and I think you've covered every approach (along with its pros and cons) exceptionally well.

OOC, how long did this take you to write?

<comment><AUTHOR> I started in November. You can see the full history here [https://github.com/kkapelon/kkapelon.github.io/commits/maste...](https://github.com/kkapelon/kkapelon.github.io/commits/master)

Most of my time was spent on thinking the examples and trying to structure the content. The actual writing was very easy when I knew in my mind what I wanted to say (as is always the case with technical writing)

</comment></comment></comment></comment><comment><USER38> Unit tests is a tool to help you write the implementations and verify it works like you intended to.

You could do the same thing by "F5"-ing it, that is running the full app, but the problem with that it is slower and above all NOT repeatable later, without significant setup time.

Running the test, sets breakpoints and debug implementation by running the unit test, is by far the fastest way to develop, while also having atleast some assurance that the code you are writing is "correct"

</comment><comment><USER39> Reminding me of too many headaches at work.

We have all integration tests, and no unit tests. I and several others have pushed for unit tests, but with little success. Our full test suite takes 10 hours to run. We have split it up so we can test what we think is the portion we are modifying, but we're never 100% sure something in the rest of the suite isn't dependent on our changes.

I disagree a little with his complexity multiplying for his anitpattern about no unit tests. In theory, yes, you would multiply them. In practice, it is rarely the case you need to test all combinations - real code rarely looks like his diagram. In our experience with our project, the final number of branches we need to test is much closer to adding than to multiplying.

I very much agree with the "don't test internal implementation". If the primary reason your tests fail is because you refactored or made API changes, your test suite is not robust.

Am having to live with flaky tests right now. Horrible. Our team doesn't want to prioritize fixing them.

One antipattern he left out: Making unit tests a 1:1 match with code, and insisting that a unit test should not test more than one funciton. I know the community is split, but I am very much in the camp that "unit" should not be tied to a function. Don't make it that granular.

</comment><comment><USER40> So we have a REST API created using node and most of the job happens on Postgres and sql files.

Basically, 95% of the methods depend on database [and it's current state].

How can I unit test this?

I've given up on unit tests. They wouldn't make any sense. E2E tests are helping us a lot but I had several attempts to have unit tests but they don't make any sense at all.

<comment><USER41> Those methods are still data in, data out tho right?

So test that with mocked data.

Unit tests should be able to run with no network connection, no other service, just your test file + data.

Otherwise at that point you're doing integration testing, not unit tests.

<comment><USER40> Let's say I have a `SomeEntity.get` function.

Basically it gets an array of id's and calls the database and fetches those entities.

~10 lines of Javascript code.

40 lines of SQL. Which also depends on the state of the database.

So what do I gain by mocking the database? Making sure those 10 lines of javascript work fine? In a vacuum? They are the 10 easy lines anyways which are covered by my E2E test.

What matters is the SQL and how it behaves in different states of the database.

<comment><USER41> And that's why you test the DB with an integration test (and likely a functional test too).

For your unit test you'd just mock the dB call and make sure that function called the "db" and returned the faux data. That's it.

No one said unit tests had to be complex or test the whole stack. Quite the opposite actually.

</comment></comment></comment></comment><comment><USER42> Most of this article I agree with, such as the need for BOTH unit and integration testing, the need to focus on automation, and the need to turn regressions (bugs) into new tests.

I also agree that "Paying excessive attention to test coverage" is not good. However, I completely disagree with much of its supporting text. If your test code coverage is only 20%, then by DEFINITION your tests are awful. That would mean that 80% of your code is completely untested. I agree that for many programs 100% code coverage is not worth the effort, because those last few percentages cost more than their benefit, but that doesn't mean that such low coverage makes sense. Most organizations I've worked with recommend at least 80% statement coverage, as a rule of thumb. I haven't seen any studies justifying this, but this essay doesn't cite anything to justify its claims either :-). You'd want much higher statement coverage, and also measure branch coverage, if software errors are serious (e.g., if someone could be physically harmed by an error). You should focus on creating good bang-for-buck tests first; code coverage is then a useful tool to help identify "code that isn't getting well-tested at all." It's also useful as a warning: 100% coverage may still be poorly tested, but low coverage (say less than 70%) means the program _definitely_ has a _terrible_ test suite.

This statement is misleading: "You can have a project with 100% code coverage that still has bugs and problems." That's true for ANY program, regardless of its testing regime, because any testing regime can only test an astronomically small fraction of the possible input space. A program that just adds 2 64-bit numbers has 2^128 possible inputs; real programs have more.

<comment><AUTHOR> >That would mean that 80% of your code is completely untested

That is not always a bad thing. Maybe depending on the application this 80% is trivial and never breaks. It is explained in anti-pattern 4 that you should start with the critical code first.

>Most organizations I've worked with recommend at least 80% statement coverage, as a rule of thumb

This number is making MANY assumptions.

I would demand different code coverage from an application that runs on a nuclear reactor and from an application that is used as point of sale in a small pizza restaurant.

</comment></comment><comment><USER43> Refreshing article. We hear so many religious arguments about one side or the other in these days, we really need more of these.

I think there's a deep issue that causes all the misunderstanding, the elephant in the room : the definition of a "unit". Words have a meaning in a certain context : if people don't mean the same thing when using the same word they're doomed to misunderstand each other forever. Just ask 5 different people what is a unit and you'll have at least 3 different definitions. The most common one is : in OOP, a unit is a class.

From my experience, a unit should be defined as a much higher abstraction level than that. A better definition would be : "a set of use cases that belong to the same module". In other words, unit tests should be we written in a language as close as possible to your domain language. Or : "test your use cases, not your classes". When you do that, you usually write tests for a few major classes that use all your other classes that are just implementation details. This leads to tests that are far more reliable and easy to maintain, because they have very low coupling to the rest of your application. Typically, this means testing classes at the very edge of your app, classes that directly communicate with the end-user, usually services or something like that.

Let's say you "unit test" a simple car with a steering module. It has all sorts of internal complex mechanism that IMO you generally don't need to unit test directly. What you need to know is if the business value is correctly delivered to the driver, ie :

\- when he turns the steering wheel left, does the car turn left

\- when he brakes, does the car stop

\- when he presses the gas pedal, does the car accelerates

\- etc

Under the hood there are dozens of other classes that perform actions that you don't really need to care about when you test. They will be tested indirectly anyway, because they're used by the high-level classes you do test. I think many people blindly try to unit test almost all the classes they write and it leads to code duplication all over the place and all sorts of other problems that make projects fail, people angry and think unit tests are bad.

<comment><USER44> I have observed the same thing. I have seen way too many test written with the class = unit philosophy that look more like tests of the VM that is running the tests than they look like tests for the code that was supposed to be tested.

</comment></comment><comment><USER45> > Anti-Pattern 5 - Testing internal implementation

I run up against this the most. The internal state often does not matter. What is important is the bevhavior. Refactoring internal state should not break a bunch of tests.

<comment><USER46> Then you will probably like this: [https://www.youtube.com/watch?v=URSWYvyc42M](https://www.youtube.com/watch?v=URSWYvyc42M)

<comment><USER45> Blast from the past; I had forgotten about that one. Thanks

</comment></comment></comment><comment><USER30> One thing that seems to be not really commented on is "Play Testing / Exploratory Testing" ( and semi related, "dog fooding"). This is the manual test process where someone actually uses the software and looks for problems. Problems are then captured as a new automated test case. This is about finding undesirable unforeseen consequences ( sometimes you get desirable ones! though if it is desirable, you will want to capture it in a test also ).

</comment><comment><USER47> I have also noticed that many of the frameworks for writing unit/integration tests are so flexible that developers will easily develop a mini-framework that makes their specif tests able to be expressed in a declarative manner. The downside is that their setup and system isn't really usable for other tests.

I think devs need to keep it simple, even it leads to more code, so that a wider range of tests can be written with the same abstractions.

</comment><comment><USER48> I personally don't like the word 'antipattern' because it sounds like a TDD religion term.

None the less the article does a good job covering lots of the reasons writing tests will result in a bad outcome. Much like the reasons developers dislike certain architecture patterns, it's not the pattern that's at fault but the application of it by individuals and teams

<comment><AUTHOR> I thought about using "common pitfalls". You think this would be better?

</comment></comment><comment><USER49> Great read, and many points I'll consider, particularly the first anti- pattern. I consider that writing too much code that interfaces with other services (which in turn leads to more integration testing) can come from suffering NIH, as it's likely that that code might just be repeating code already provided by the api/library of that service.

</comment><comment><USER50> I understand that if the test hits database, then its an integration test.

But what if a whole load of the logics are in stored procedure? Surely those needs test too.

Of course, one can make an argument that depending too much on stored procedures is an anti pattern....

<comment> atom-morgan The stored procedure should be tested within the domain of the stored procedure. The consumer of its result just tests its ability to use the result it should be given (fulfilling its end of the contract).

</comment></comment><comment><USER51> This website is unreadable on iPad, the font size changes every time I scroll.

<comment><AUTHOR> I have an iPad 1 and it shows just fine. Can you post a screenshot/video somewhere with the problem?

<comment><USER52> Here it is on a 5th gen iPad with iOS 10.3.3: [https://youtu.be/5gqbL_fhy0U](https://youtu.be/5gqbL_fhy0U)

Works fine with Reader View, though.

</comment></comment></comment><comment><USER53> Anti-pattern 14 - generalizing all testing in the industry to e-shops and websites

<comment><AUTHOR> I am open to other suggestions. I have really tried to spend a lot of time to another example and cannot seem to find any.

The other alternative would be a loan approval application.

Remember that the definition of a developer is very broad nowadays. You have hackers working with C/Assembly on firmware, all the way to AI/ML with high level languages.

Do you have another suggestion for a good example?

</comment></comment><comment><USER54> I can't say I agree with this. Partly because it's too generic. I'm of the opinion that you should focus on testing what you deliver. If you're building a framework or library, then unit tests are probably the way to go. If you're developing a micro service that's primarily going to be called by other services, built by other teams for example, you should probably focus on integration tests. If you have an external API, you should focus on e2e tests, as your clients won't know or care that you have several individually well tested microservices behind your API or if it's a monolith. They only care that a given API call has the expected result. The pyramid thing is pretty, but also meaningless IMHO.

<comment><AUTHOR> You are just repeating __exactly __what I am saying as anti-pattern 3

<comment><USER54> Upon re-reading, I think I may have misread large parts of your article.. Sorry about that, these are actually great tips unless you misunderstand them ;-)

at first I took your comments about "the shape is _not_ a pyramid" as there being something wrong with that and that the shape should be a pyramid.. reading a bit more carefully I now see that's the opposite of what you're saying...I blame everything on english being my second language O:-)

<comment><AUTHOR> It is ok. English is my second language as well.</comment></comment></comment></comment>


It took Kotlin 2034 seconds and C 0.5 seconds to find first 10,000 prime numbers - <AUTHOR> https://bitbucket.org/long_cheng/prime_number_sieve <comment><USER2> I guess Swift is quicker as C.</comment>


Pandora's Guide to Mobile App Platforms - <AUTHOR> http://gigaom.com/2010/06/28/pandoras-guide-to-mobile-app-platforms/

<comment><USER2> Really reminds you of the nightmare that was the Old World of mobile development. Especially that pink RAZR story, which really hits home how much developers were at the mercy of fickle handset manufacturers. Of course, if they actually tried to sell their app they would be beholden to the carriers too, which is its own world of hurt.</comment>


Amazon unpacked - <AUTHOR> http://www.ft.com/intl/cms/s/2/ed6a985c-70bd-11e2-85d0-00144feab49a.html#slide0

<comment><USER2> An old rule for writing news articles: use your best anecdote/quote/detail in the lead, use your second best at the end:

> _Angi Cooney, who runs C Residential, the biggest estate agent in Rugeley, > thinks the nature of employment is changing permanently and people should > stop pining for the past. It's "bloody great" that a company like Amazon > chose to come to "this little old place", she says fiercely, looking as if > she'd like to take the town by the shoulders and give it a shake. "People > expect a job for life, but the world isn't like that any more, is it?"_</comment>


Fast LSTMs in PyTorch - <AUTHOR> https://lernapparat.de/fast-lstm-pytorch/ <comment><USER2> Fantastic article about writing fast DL code.</comment>


$99 accessory turns your phone into a laptop. You can play Pokemon Go on it - <AUTHOR> https://www.techinasia.com/superbook-turns-smartphone-into-laptop-for-99-dollars <comment><USER2> This is a laptop shell with an 11.6" screen, a keyboard, and a built-in, supposedly '8 hour' battery.

Given that the only other option in this space is the long-discontinued Motorola Lapdock [1], this sounds very useful.

[1] [https://www.amazon.com/Motorola-Lapdock-100-Smartphones- Pack...](https://www.amazon.com/Motorola-Lapdock-100-Smartphones- Packaging/dp/B005P58B5G)

</comment><comment><USER3> Seems super useful - but it comes to $139 with the shipping charge to India. Wondering if that's too much to pay for a laptop shell.</comment>


Google Trends for VR products seem to decline - <AUTHOR> https://www.google.com/trends/explore?q=psvr,playstation%20vr,oculus,htc%20vive <comment><AUTHOR> Also interesting: Enter a product from another consumer electronics category such as 'iphone' and you see how VR and other categories relate to each other in terms of search volume.</comment>


Things that correlate - <AUTHOR> http://tylervigen.com/

<comment><USER2> An effective way to make people understand that correlation does not imply causation and we can be very easily fooled by randomness.</comment>


Alexa, how many dogs come to work at Amazon Seattle? 7k - <AUTHOR> https://www.latimes.com/nation/la-na-amazon-dogs-seattle-20190621-story.html <comment><USER2> "First-time owners in their 20s or early 30s, accustomed to instant gratification and one-day Prime delivery, often expect immediate cures for animal ailments, she said." Unfair to call the concern about your pet's health and wishing for their faster recovery akin to instant gratification from online shopping IMO.</comment>


Use Your Microwave to Measure the Speed of Light - <AUTHOR> http://io9.com/5526055/use-your-microwave-to-measure-the-speed-of-light

<comment><USER2> That was a let down.

This is not: <http://www.darylscience.com/downloads/DblSunset.pdf></comment>


Show HN: Refinator, online lecture notes and PDFs with prerequisite lists - <AUTHOR> http://refinator.herokuapp.com/ <comment><AUTHOR> I haven't worked on this in a few months, so I figured I'd just show off the current state of the project since I'm unlikely to be able to work on it in the next couple of months.

This is a _really_ simplistic proof-of-concept for something I've often wished to be able to use myself, although I should really support real books as well. Among other things, I've often felt that prerequisite and "read alongside" lists (ideally based on (topic, sophistication) pairs instead of being specific books as it currently is) would be _great_ to have. The about page[0] goes into more detail on this.

Currently, I plan to integrate with MathOverflow/MSE's tag wikis, and maybe allow rep to transfer over in some sense from there by allowing you to link your accounts.

Any edits or additions would warm my heart. Contributions to the code[1] are welcome too! :)

This has also been posted[2] to /r/math on reddit.

\--

[0]: [https://refinator.herokuapp.com/about](https://refinator.herokuapp.com/about)

[1]: [https://github.com/mrkgnao/refinator](https://github.com/mrkgnao/refinator)

[2]: [https://www.reddit.com/r/math/comments/5jx53e/refinator_lets...](https://www.reddit.com/r/math/comments/5jx53e/refinator_lets_you_search_for_online_lecture/)</comment>


Please Jobhop as much as Possible - <AUTHOR> http://angryasian.posterous.com/please-jobhop-as-much-as-possible I started this rant over at help a startup out, but had more to rant about so its on my blog now. Basically a real world response to Calcanis and Suster <comment><USER3> The whole concept of loyalty towards a corporation is off. You can have loyalty to family, to friends, to a country but never to an economic entity whose only purpose (on paper) is profit. I think that professionalism and well-defined internal rules should be the only things expected by employers.

<comment><USER4> In this day and age, yes, you are largely correct, but I don't think it was always so. I get the impression that at one time, "loyalty" between employer and employee actually meant something in both directions; the company might take some losses to hang on to people and/or help them out. Perhaps more so than would happen today.

I'm not sure that's such a bad thing, yet I wouldn't say that what we have today is bad either, I suppose they're just two equilibriums. The problem more likely lies in the transition from one to the other: employees that expect to be treated with loyalty and are summarily dumped with a few years left before retirement, or employers who invest a lot in employees and expect to see them stick around because of it.

This is pretty off topic, but I'm not particularly a fan of 'loyalty' to countries either. Most people happen to be born in a particular one; at least a company is something that you likely chose.

<comment><USER5> _In this day and age, yes, you are largely correct, but I don't think it was always so. I get the impression that at one time, "loyalty" between employer and employee actually meant something in both directions; the company might take some losses to hang on to people and/or help them out. Perhaps more so than would happen today._

The chocolate maker Cadbury would be a good example of this.

</comment></comment></comment><comment><USER6> I'm sorry - It was TOTALLY my fault that I started my career during the Dot Com days and worked through two bubbles. My fault that I worked for people who couldn't keep their companies above water despite working 70 hour weeks for them. I'm glad to know that it will prevent me from working for someone like Mr. Calacanis because of his narrow world view.

</comment><comment><USER7> I feel somewhat passionately about this, and I've always thought this job- hopping debate was extremely skewed.

There are people who want loyalty and dedication above all else -- both from themselves and from their workers and employers. Then there are people who want challenge, risk, excitement, and travel.

From what I've seen, if you want to learn meta-tech (and not just tech in your neck of the woods) you need to get out among a bunch of different industries and job situations and start learning.

Or put a different way: as one of those people, I get approached all the time for full-time jobs. The conversation goes like this "We see that you have experience working in X, Y, and Z, and you have industry experience in A, B, and C. You also have been in G, H, and I job positions. We are desperate to have somebody with these experiences and skills, but all we find are people with a lot of experience in just one of these areas. Would you consider a full-time job?"

My reply is this: if I were the type of person to consider a full-time job, I would not be the type of person who had all those things you want. Full-time jobs are stability -- stable platform, stable work environment, stable job position, stable insurance, stable retirement. Things may start off chaotic (as in a startup) but the goal is to reach stability.

I've been parachuting into situations where the building is on fire and saving the baby. It's like startup work all year round -- and for dozens of different highly-rated companies. And by the way, the rates are great too.

Do that for a while. Learn what you like and what you don't. Then start looking for stability. Don't sell yourself out too soon in the name of loyalty and stability.

EDIT: And I know a lot of folks from those companies who only wanted stability and loyalty and ended up on the street after ten years with very little in the way of marketable skills. Don't fool yourself: IT is a risky and always- changing business. Stability and long-term jobs are an illusion.

<comment><USER8> Just out of curiosity, what exactly do you do? Do you do consulting?

<comment><USER7> I do work-for-hire, yes. I did so much stuff that now people pay me to help them learn how to run teams like a startup and not like the IRS.

And yes, I miss the code-monkey and hands-on PM work. That's why in my free time I still code as much as I can. And why I like hanging out on the net with you lug-heads so much (smile)

So now instead of the building being on fire, they have 50 teams that are taking 4 times longer than industry norms to deliver functionality. How do you fix that without shutting down the place, firing a lot of people, or causing more harm than good?

The thing you run into with this type of work is that _everybody thinks they are unique_ , and they are, to some degree. But also there are a lot of similarities. For somebody without a big breadth of experience who is just reading a book and trying to apply it, it's not so clear what is unique about their situation and what isn't. So there's more to it than training. You have to have a lot of hands-on experience watching what works and what doesn't. Call it strategic technology management consulting.

Actually it's a much bigger fire, but the sense that you are delivering something of immediate value is lost. I've found that as you get better and better at delivering solutions, they give you fuzzier and fuzzier problems, many times with no clear deliverable. Moving from delivering stuff on-time and under-budget to fuzzy-world is not easy.

My job used to be the sharpest guy in the room. Now my job is to make a hundred other guys be the sharpest guy in the room. I don't light the light; I turn the brightness up 40%.

And there is no way you can be a full-timer at BigCorp for 20 years and do what I do.

</comment></comment></comment><comment><USER9> I recently had a conversation with my 16 year old son who was having girl problems. Part of my advice was about playing the field. i.e. "What are the chances the best woman for you, in all the world, lives 1/4 mile away?" Similarly, what are the chances the best job for you is your first, or second, or third? Slim.

Playing the field with employers should be the standard mode of operation. This possibly benefits employers even more than employees. What is worse for an employer than an entrenched employee, who hates his job, doing the bare minimum to stay employed? Surely the answer isn't the guy who was super productive for a year and then moved on.

<comment><USER10> "Surely the answer isn't the guy who was super productive for a year and then moved on."

I've been that guy for my whole life, and not a single employee has tried to convince me to stay with better perks, none.

A couple of times I stayed and got not even enough to compensate for inflation, so fuck em, I moved on.

If you want better perks, every year you have to move on.

ps. But you have to be good, and I mean real good to play the diva card.

<comment><USER11> > _"I've been that guy for my whole life, and not a single [employer] has > tried to convince me to stay with better perks, none."_

Immaterial. You never take the counter-offer.

An employer who refuses to reward you until you're at the point of looking elsewhere is not worth remaining at. Further, you'll wind up on the short-list of problem employees and in the event that they do need to reduce headcount you will be disproportionately likely to lose your job earlier, thus disadvantaging your next job search.

There's no upside to taking a counter-offer beyond immediate convenience.

<comment><USER10> Absolutely, I agree with you 100%.

It is not the counter-offer which counts, that in my book is too late. It is the end-of-year reward which should be AUTOMATICALLY adjusted per inflation plus at least a 5% increase if you overperformed. So I always expect a 10% incease WITHOUT asking for it, or else I'll move on.

On my side, I always overperform more than 10% so I deserve what I expect. The employer? not much, they don't care. They will never care. Their book forbids 10% increase for every employee.

Key point, not EVERY employee deserves 10% increase. Only overperformers. HR guys need to understand that and learn to differentiate and reward accordingly, which they don't. They are also collecting their winnings every month.

ps. I am not talking about food a la carte, or gym towels. I am talking about money and vacation time. Everything else is a welcomed addition to the perks.

<comment><USER11> Ah, I misunderstood; I definitely agree.

</comment></comment></comment></comment></comment><comment><USER12> It's important to remember that Calacanis and Suster are writing advice from a selfish standpoint. _Of course_ they want the people they hire to keep working for their companies no matter how shit the conditions are. If I were them I'd be writing about how the best way to get ahead was to come and work for me for free.

It doesn't mean it's actually useful advice for someone who wants to get ahead.

<comment><USER13> Yeah, Calcanis had this weird turn of phrase in his original blog entry "set his career back 5 years to get his salary ahead by 3 years". WTF does that even mean?

<comment><USER11> It means Calcanis is thinking in terms of seniority and "years of service" being the keys to career advancement and compensation. The decade's-old management mindset that gives us things like SLOC and butt-in-seat quotas.

<comment><USER14> That's one way to interpret it, but consider a company that pays you to sit in a broom closet, doing nothing, all day and forbids you working on your own code at home. Even if you draw a great salary, your skills after three years on this job may be equal to your skills five year earlier (i.e. you didn't learn anything but forgot stuff/your skills became outdated).

Clearly, some jobs teach you more than others...

<comment><USER11> Absolutely. I wouldn't have taken issue with Calcanis' comment if he'd simply framed it as choosing salary over personal development.

My criticism has everything to do with his use of "career" where he should have said "skills" or "personal development" and his use of "years", when time spent has absolutely no objective relevance, as reinforced by your own comment: 1 year at firm A is not equivalent to 1 year at firm B.

So the implication that there's an objective salary advancement/year or career advancement/year rate is nonsense. It's detritus from the failed management schemes of an older era.

</comment></comment></comment></comment></comment><comment><USER15> Don't leave a job until you are sure that they are the problem and not you.

When I was younger I used to think that my boss just didn't get 'it' -- whatever 'it' was. As I hung in there I started to see that I had been sold a bill of goods in school.

You get paid for the job you do, not for the job you _think_ you do.

When you are sure you are doing what you thought you were and still are not getting commensurate pay; that's when you leave.

</comment><comment><USER16> My granda used to tell me you should always try and look at things from a different perspective (which, taking him literally, prompted me to climb the rock garden to look at our house). Working for different companies gives you perspective on ideas that you won't get from working for just one company. The methodology du jour won't work in every situation, it takes perspective to know that. So I agree with point 4, "The 30 year old with more than 6 jobs in that time, probably has a lot more to offer", and a lot of that comes from perspective.

</comment><comment><USER17> Great Article. After college I did what I thought I was supposed to do and work for 2 large corporations for the first 7 years of my employed life. I found the pay below average for the market- especially if you are some one who increases your skillset at a higher level than your peers.

I noticed most of the employees were there to float along with the tide and do enough to stay employed, while very few had aspirations of making changes and doing good things. After that I started "job hopping"- Starting with a contract gig, after 3 months of that went to a medium sized company, and have now ended up at a small company of 100 or so employees. With each subsequent jump I have increased my pay by 15-30%, and learned skills I would _NEVER_ have picked up had I stayed at the "big company".

As mentioned in the article, jumping around from job to job made me realize what I wanted to get out of my employer. Most importantly, I have 20-30 new contacts that I feel I could tap into should I ever become unemployed. Most of the time its what you know _and_ who you know...

</comment><comment><USER18> Having also graduated high school in the middle of the dot.com bubble, graduated undergrad in the middle of the dot.com bust, and been hustling ever since, I am also a job hopper. I don't have much trouble landing work because I have a particular work ethic.

It's a simple heuristic. If I can leave my employers in a good state, I go ahead and job hop. I stay until the project is done, and then I feel ok leaving or staying and take on the next project. Very often there is no next project lined up, so I am compelled to look around.

I think if you have a habit of abandoning jobs in the middle of projects, leaving your employers holding the bag, you _are_ a flake and a risk to hire.

<comment><USER19> Good work ethic. But so what about the projects that never seem to finish? You know, the ones that can be done in 6 months and end up taking 5+ years.

I am working on a project right now(in healthcare) that I thought I could finish in a year's time. After every meeting, I think to myself that I'd be lucky if I can finish it in a decade because of bureaucracy and incompetence. What then? I am not going to stay until the project is done, I am mortal after all!

<comment><USER20> At that point, the good work ethic is to say "look, this isn't working. What can we do to part on good terms?". It's a lot better than saying "screw you guys, I'm going home!" :)

</comment></comment></comment><comment><USER21> This post is great and perfectly refutes their points. They want unconditional loyalty to a low-paying, high-stress job that might sell out or go bust any minute. And for startups in the VC game, they give drip-drops of equity that will never amount to anything even if they did get a big buy-out and then might be asked to move to another city or just laid off. All for the "opportunity" to work there? I don't think so. Maybe I'll just go work for a company where I can work 40 hours a week with an extra 50K a year and build a startup of my own on the side.

</comment><comment><USER22> Jobhop yes, but please be careful--you might just hop yourself right out of the market.

My resume resembles Swiss cheese. I have a whole bunch of short-term jobs, and holes where I've taken off a year or more to pursue my own ideas (I'm "freelancing" during that time).

Fortunately I was able to get another job after the last break, but I didn't seem to be getting as many callbacks as I'm used to. This, coupled with my age, tells me I need to stay put for a while. And I hate that.

<comment><USER23> I doubt it. We are desperate for anyone who can answer our interview questions correctly. We would not care at all where you have worked, how much time you have taken for yourself, etc. Just know _something_ about programming.

<comment><USER24> Agreed. My resume looked the same for a time, and I had no trouble landing interviews and getting offers.

That said, I've been with my current employer for three years now, and I have no intention of leaving, thanks to their flexible work schedule.

</comment></comment></comment><comment><USER25> Love your job but do not fall in love to your company.

On the other hand, where I live, one must switch job to get a better/increase pay. Yes, sad.

</comment><comment><USER26> I guess I have a younger point of view, but it pains me to see people stay with a single company all their lives, it just seems like such a waste. They've dedicated their lives to making someone else richer is what it seems to me.

The last company I worked for went under, and when I was searching for a new job, I interviewed with the corporation Conn's (similar to BestBuy/Fry's). I met one well paid executive there who had worked there for the last 25 years. While well compensated, he looked beaten down, way older than he really was, and unhealthy.

The point of life is not dedication to one or multiple companies. That just sounds horrible to me.

<comment><USER27> _but it pains me to see people stay with a single company all their lives_

When you realize that to most people "it's just a job" it makes more sense. Work should not be your life. All I ask of my job is that it pay me well and be pleasant. If there's more, that's icing on the cake, but I look beyond my time at work to find meaning in myself.

<comment><USER19> Agreed. However, it does become your life when you work 16 hours a day. Simply because you really don't have any life left to live. Except if you consider sleeping time to be a vacation.

</comment></comment></comment><comment><USER28> When interviewing engineering candidates I look for competence and curiosity/passion for technology. Implicitly, I also look for ability to communicate (why did you come up with a solution that you did on the whiteboard?). Given that I only have forty five minutes, I simply _can't_ afford to waste time asking about their life story. I'll ask em about their projects and why they're interested in this position, but I don't need more than a sentence about why they've switched jobs in the past.

Why did they have six jobs before thirty? Who knows? May be they switched career orientation (I've started in operations and moved into software development, that required a "hop"), may be they didn't know what they were looking for, may be they had a family situation.

Likely, if they worked at a single company for ten years, that's irrelevant _if_ they manage to have the desired level of competence and have the desired level of passion and curiosity.

Are they leaving companies since they were fired (rather than downsized or left voluntarily) or put on a performance plan? That's to be checked through their references. Are they hopping purely for increased compensation? Then just refuse to meet outlandish salary demands (out of proportion to their skill level).

It's true that a bad looking resume may not make it past an HR filter. However, I've been swamped by recruiters (agency _and_ company HR) _at all times_ even after my resume had shown I've only been a few months at my company. Furthermore, when you're over thirty you'll likely have a network of coworkers who will be able to refer you to jobs bypassing the HR filter.

Rather than worry about what a blogger thinks about you and your generation, worry about learning and increasing your competence level. Make multiple companies fight over you (because you have a rare talent and excel at it) instead of fighting to get into a company.

</comment><comment><USER29> When I started in IT field (support/sysadmin), I was advised that I should hold my positions for 2-5 years, and avoid job-hopping, cause it looks bad on the resume.

Fast forward 6 years, I quit my job, why? I started in tech support. I've worked night shifts, weekends, crazy schedules when someone was a on a leave or vacation. No compensation for it, crappy salary, little or no bonuses, night time work and weekend work was not paid according to the law in my country (no-one to complain about it, unfortunately). I kept working there because there was no better offer available, the team I worked in was made of great people, there was some promotion plans for me, and I had a loan to pay off. I got promoted. OK, so now I'm a sysadmin, better salary, but still no compensation for working over-time, doing half-month rotation on-call standby if something brakes (and it did), etc. Please mind that the better salary was not a good one, just better. Anyways, one of my fellow sysadmins gets called into active service (obligatory in my country), and I get stuck with constant on-call duty, same salary and again, little or no bonuses. This is where I start looking for options, and I after getting accepted at few other places, I decide to stay where I am, why? Better tech, more complicated work, challenging, etc. A year later I discover that no social service, pension plans and med-care payments were made by my company for me or other people for the last 5 or 6 years ( in my country, the company pays for it, and it isn't quite easy for you to check that on a monthly bases), and I get net income, so I actually don't care about my gross income. I raise hell, and after a few clashes with local and upper management I decide to get the hell out of there.

So, in the end, my two cents are: Loyalty, to the company, friends, team at work, family, girlfriends, etc. has to be something that is deserved on their part, not expected from day one. Dedication and commitment should not be confused with it.

Anyways, I'm now freelancing, and working towards my degree in soft. engineering.

<comment><USER30> I come from a roughly similar background, I started out fresh out of high school in tech support, though had done some programming on the side.

I enjoyed programming more though, so gravitated towards that, and so glad I did. Most of the people I knew from my time in the support / admin trenches have similar stories to tell, apart from some exceptions doing large-scale work for Google, eBay, and telecoms.

On average I think programmers are compensated better, and generally enjoy a better work environment.

When you're seen as working in a cost center as sysadmin and network admin often is, you'll always have the spectre of downsizing/outsourcing hanging over you.

Just don't work permanently at a consultancy (ugh). If you must work permanently, do so for 2-3 year stints, and at product-focused places or places where software engineering isn't just doing crappy line of business apps.

</comment></comment><comment><USER31> This should be taken with a grain of salt. Yes, change jobs enough to get lots of experience and pay raises. Don't get stagnant somewhere.

But imagine you never stay more than 6 months! Potential employers may think "is it worth our time to train this person before he/she leaves? And is he/she too fickle to accomplish anything?"

</comment><comment><USER32> When older people complain about supposedly whiny, entitled, self-centered and disloyal young people, exactly one work comes to mind: Projection.

We came of age while society fell apart. We're in our 20s and won't be able to own a house or have children until our mid-30s, due to the still-absurd housing costs, absent job security, and lack of a decent health care system.

The Boomers were born into an affluent, forward-looking society in which it was embarrassingly easy to advance. You could be a pot-smoking hippie dropout at Woodstock in '69 and a CEO by '75, whereas we have to start racking up internships (unpaid, in most industries) in high school. They inherited a society that would go forward and achieve great things so long as no one came along and fucked it up (as the Reaganites did). We inherited a giant, potentially intractable, mess.

<comment><USER33> This is like a dispatch from Bizarro World.

My parents' Carter-era mortgage was for 15%+, mine signed last year (well before my mid-30s!) was for 5.5%.

The rest of my counterexperience is just anecdote, so I'll spare everyone the details and just ask:

Do you have even a single shred of evidence backing up your nostalgia for things you didn't actually experience?

<comment><USER13> It's more relevant that the Boomers had 10 workers per retiree whereas we have 3 workers per retiree. The Boomers took the next two generations prosperity and used it all up themselves.

<comment><USER34> Sorry, I may be missing your point, but how do you blame the Boomers (of which I'm at the tail end) for this situation? Shouldn't you be blaming the "Greatest Generation" (parents of the Boomers) for setting up a system that depended on maintaining a high ratio of workers to retirees?

<comment><USER35> Well its the boomers that "didn't reproduce enough" (I'm not serious about this one).

However the boomers are the ones that deregulated the shit out of the world, bankrupted the pension funds, built houses and luxuries on credit (they didn't invest much into infrastructure). They are also underqualified and sitting on many positions from which they are screwing their children (that's at least how it works in my country), while simultaneously taking it out on us - <USER2> supposedly being lazy and incompetent - to create families and build homes.

While we stand on streets - well educated and eager to take on challenges, but we're unable to because boomers don't allow us the privilege. And I mean quite literally - don't just whip around mortgage rates. I don't know about USA - but I'm certain that it's a lot harder and costlier to start and run a business in a lot of the world nowadays than 40 years ago.

<comment><USER34> I like the idea that Boomers haven't been reproducing enough (although with 3 children I feel like I've done my part).

As a tail-end Boomer, I don't see current 20-somethings as "lazy and incompetent", but I know that my peers and I discuss the fact that Gen-Y seems to be motivated by different things than we were. In the context of the original discussion about job-hopping, I don't have any problem accepting that members of Gen-Y have different attitudes toward tenure in a particular job. However, as I mentioned in comments for a related posting, the issue from the employer's perspective isn't really whether they agree with your values, but whether they perceive hiring you as a good investment.

The costs of hiring someone are such that most companies want to be able to rely on the individual to stay in the role for at least 2 or 3 years. Furthermore, your value to the company increases rapidly as you gain experience in their particular business - they don't want to see that value walk out the door. Other posters in this discussion have suggested that the answer is for companies to make the job more interesting/rewarding, so that people will stay. I don't disagree with that - creating an environment that attracts and retains the best people is a critical success factor for IT organizations that I've worked with. However, when making a decision to hire, the employer can only consider the candidate's past track record, not some possible future where either the company or the candidate change behaviour.

I think this discussion may really be two separate questions: (1) is it reasonable for companies to discriminate against perceived "job-hoppers" when hiring, and (2) should companies adjust their practices to retain good employees? My answer to both would be a qualified "yes", although from some of the comments it seems like there are some people who would not stay at a company for more than a year or two regardless of the company's retention practices.

<comment><USER36> As my peer said...

I know that myself and most of my peers would love to be loyal to something, we are eager to learn... We love to work hard and play hard. We're willing to learn the ropes - but we have no intention of working for the same company for all our adult lives - because we have seen many cases on our own eyes how that kind of attitude is less then beneficial.

But as mentioned elsewhere our bullshit tolerance levels are low.

I agree with you that costs of loosing productive people who understand context of their work is expensive. I don't agree with you that businesses care (at least not the ones I saw myself). Well maybe they care - but not enough to actually invest anything or seriously commit to solving the issue for themselves.

It's all just talk - yes they'd like if people would stay longer, but they "can't afford" to actually improve living and working conditions for their workers.

Just mgmt business as usual - improving sales and market position is hard, cutting expenses is easy, leave a trail of broken businesses behind on your path to glory.

</comment></comment></comment></comment></comment></comment></comment><comment><USER37> One of my employer offered me _loyalty_ bonus.</comment>


Literate Jenks Natural Breaks and How the Idea of Code is Lost - <AUTHOR> http://macwright.org/2013/02/18/literate-jenks.html

<comment><USER2> Very interesting read. As someone who struggled through the implementation of Jenks[1] a few years ago in PHP, I can attest to the fact that it was extremely difficult to find any programming of Jenks that was understandable and clear. I was able to find some examples of code, but nothing could explain what was actually happening and, as you said, there were no comments or poor comments in those examples.

What I was able to find, were a couple of different explanations of the mathematics. Instead of trying to decode some crappy programming implementations, I spent my time teaching myself the math to understand what the representation of the process[2] actually meant. From there, I was able to understand the iterative process involved and dream up a working solution.

Instead of concluding programmers should be better or knowledge should be more free, I came to the conclusion that I just needed to understand the problem and the math behind it better.

Could it be that maybe the assertion regarding the problem of availability is actually just a problem of lack of understanding or desire for proper comprehension? I'm not accusing the author of laziness, merely wondering if a different conclusion and solution could be arrived at with an alternate point of view on the problem.

[1] - <https://github.com/randomdrake/jenks> \- feel free to check out my implementation. I wrote it a few years ago, so it's probably not the greatest, but it's commented, and worked fine and fast.

[2] - <http://randomdrake.com/jenks.gif>

<comment><AUTHOR> Hey - I should have mentioned your implementation - I stumbled upon it and was like "whoah, this guy actually wrote this from scratch" :)

In this case, it's a combination - the algorithm Jenks arrived at is not just the math-implemented, but a clever solution that (afaik) has not been expressed in pure-math terms.

<comment><USER2> Heh, glad you found it entertaining.

This project was very unique and fun for me.

Interesting that you mention the "combination." When I was able to find the aforementioned image that showed the Jenks method, I knew I had to understand what it meant and what the symbols were. I found it very cool that the mathematics were simple and it was the process that made the method work so well.

Basic exponents and algebra were all that were really needed, but the magic was in the method; much like good programming.

The insight into the power of elegant processes wrapping simple mathematics is something I've repeatedly experienced in my programming career. That moment when you hit run and all the data comes out how you wanted it to. It brings upon the realization that your result would only be possible with a true understanding of the process you were implementing. A simultaneous victory and confirmation of comprehension is a good feeling.

Beauty in the bytes.

</comment></comment></comment><comment><USER3> Here's my implementation of Jenks in Numba: <https://gist.github.com/sklam/4979921>

It uses numpy array instead of list. Doing so without Numba is a lot slower because numpy array indexing and operating on array scalars are slow.

</comment><comment><USER4> Fantastic article. I like to think of this phenomenon with a positive spin though.. I can use the Jenks algorithm without understanding anything about how it works, just plug it in and go. And with a few test cases I can even port it to a new language without really understanding it. I admire Tom's work in digging in and doing it right, but as a journeyman programmer I like that I can just use it without really understanding it. Sometimes cargo cults work.

</comment><comment><USER5> _> If you're a coder, consider whether the abstraction of software can be misused to mask ignorance of basic principles._

One of the problems with Computer Science and Programming, is that such a phenomenon works most strongly within the field.

I also love this quote the author included: The lack of interest, the disdain for history is what makes computing not-quite-a-field. – Alan Kay

</comment><comment><USER6> I do love the docco style documentation, but I tend to think of 'literate' as a specific thing, <http://en.wikipedia.org/wiki/Literate_programming> which I don't think this quite matches.

</comment><comment><USER7> > In basic benchmarks, it's 12x faster than a Python implementation

Oh well, my friend, why didn't you use Numpy?

<comment><AUTHOR> Numpy would be faster, and so would PyPy as I point out. I'm comparing unoptimized implementations on purpose and not trying to incite some kind of language-speed-flamewar.

<comment><USER5> _> I'm comparing unoptimized implementations on purpose and not trying to incite some kind of language-speed-flamewar._

Think about this sentence for a few minutes, then come back so we can all have a good laugh at ourselves and the fragility of human pride.

EDIT: Downvoted? This was not meant with any kind of meanness at all. No matter what choices an author makes with benchmarks, someone will complain. The situation is so catch-22, a laughter is the only effective self defense.

</comment></comment></comment><comment><USER8> For those who care, here's a previous implementation in Python that I've used: [http://danieljlewis.org/2010/06/07/jenks-natural-breaks- algo...](http://danieljlewis.org/2010/06/07/jenks-natural-breaks-algorithm-in- python/)

</comment><comment><USER9> This is awesome. I feel like I owe you a couple of beers or something.

</comment><comment><USER10> Thanks for working this out of course. But, if the old code produced nice- looking plots, what can be said of the new, except you like the way it reads better?

</comment><comment><USER11> This deserves a "Documentation FTW" gold star.</comment>


Show HN: Syllable – A Knowledge Catalogue - <AUTHOR> https://www.getsyllable.ca <comment><AUTHOR> I made a knowledge catalogue to curate personal as well as organizational learnings.

It's simple, searchable and shareable.

Try it out for free! :)</comment>


Qualcomm fined €1bn by the EC for paying Apple to guarantee chip exclusivity - <AUTHOR> http://www.euronews.com/2018/01/24/qualcomm-fined-almost-a-billion-euros-by-the-european-commission <comment><USER2> [https://news.ycombinator.com/item?id=16221924](https://news.ycombinator.com/item?id=16221924)</comment>


Astrologers are not pleased with Amazon Prime's "shopping horoscopes." - <AUTHOR> https://slate.com/human-interest/2019/04/amazon-prime-astrology-shopping-horoscope-terrible.html <comment><USER2> >actual astrologers Ha.

1\. No. There is no measure of an 'actual' in fraud.

2\. Who cares what any self-identifying swindler thinks?

3\. Wow amazon, that is next level low.</comment>