The State of Agile Software in 2018 - <AUTHOR> https://martinfowler.com/articles/agile-aus-2018.html <comment><USER2> This may vary from region to region, but my biggest issue with the present Agile trend is the cargo-culting of SCRUM. Sprints, standups, reviews and retros, planning poker, all followed blindly, but to no effect. Worse yet, the dreaded term "anti-agile" is used to shut down any other way of operating.

It has almost become a religion and any attempts at advocating planning discipline or detailed designs (even in domains where it's called for) are quickly labeled and rejected as "waterfall". Some project managers trained in agile methodologies have trouble grasping some basic common sense project management practices such as dependency tracking and capacity planning.

I recently gave up on all buzzwords and boiled the whole thing down to 3 things:

1\. Deliver things in small, testable chunks

2\. Start with a best-effort plan, but refine weekly based on how things went

3\. If you're blocked or your work is at risk, let others know ASAP

<comment><USER3> The irony being that agile was meant to be a rejection of blindly following a ONE TRUE WAY. A major point was to periodically review your processes and adapt them to better suit your situation. The idea being you don't get locked in to a process that's no longer working and even have some freedom to experiment.

<comment><USER4> But I'm telling you people!, the earth revolves around the sun. \---Burn him!!!!

We're currently swallowing the Scaled Agile Framework pill and I honestly think that it's cause maybe 3 serious episode of depression and burnout in 7 person team.

We failed a sprint a while back. No one knew you could do that until we did.

So at least we're learning.

<comment><USER5> SAFe is actually fine if you choose an appropriate level and follow the practices. It's ridiculous how people read that SAFe is "tailorable" and then promptly do something that is in no way Safe.

Every single day i tell my coworkers to RTFM

<comment><USER6> SAFe is probably the worst offender of what the article is focused on.

I can't think of a more anti-agile statement than 'RTFM'. A more productive statement might be 'Update The Fucking Manual'.

</comment></comment></comment></comment></comment><comment><USER7> After doing software for almost 30 years now, I honestly believe Agile has been the most destructive thing to happen to the industry. Software has become buggy shit that's shoveled out the door as fast as possible. It's not just your development team, it's all development teams.

Windows 10 update deleting your files? Is it the lack of testing, poor code quality, too many features or not enough engineers?

Ultimately it's going to boil down to some cargo cult scrum process of reviews, retros, and bs that we are all familiar with, and Windows 10 is buggy. Android is buggy. iOS is buggy. Nobody cares about the software. They blindly follow the process.

<comment><USER8> > Windows 10 is buggy. Android is buggy. iOS is buggy.

Let's not pretend bugs didn't exist in the pre-agile/scrum era. Bugs have been around as long as software has been around.

<comment><USER9> I think it's safe to say that bugs are way more predominant in software that used to be reliable. It's not that the software was completely bug free, it's the quantity and quality of the bugs now being presented.

<comment><USER8> I don't think windows used to be more reliable. It used to crash all the time. Do you remember the blue screen of death[0]?

[0] [https://en.m.wikipedia.org/wiki/Blue_Screen_of_Death](https://en.m.wikipedia.org/wiki/Blue_Screen_of_Death)

<comment><USER10> But I don't remember any BSOD wiping all my documents just like that.

<comment><USER11> I mean windows 95/98 didn't even use a journaled filesystem... a BSOD could wipe your documents...

</comment></comment></comment></comment></comment></comment><comment><USER12> I've worked at four companies that have claimed to do agile development. Not one of them has deeply embraced the principles of doing so. Generally they have added a few easy practices, like stand-up meetings and sprints. They've kept large-scale chunks assigned to specific developers, defining both the work to be done and the time to do it, and substantial design docs. Unit testing tends to be spotty.

We're not living in Agile World. We're living in Fake Agile World.

I have some reservations about true-blue agile, but I'd be willing to give it a shot. Let's do Fundamentalist Scrum once. I'd be up for it. At least I'd know whether it works or not.

</comment><comment><USER13> > And yet what I'm hearing so much is the Agile Industrial Complex imposing > methods upon people, and that to me is an absolute travesty.

I find it depressing that Fowler sees the same problem I do. I have yet to find a shop that practices "agile" in a way that's actually consistent with the Agile Manifesto. What I've been seeing instead was micromanagement and cargo cult. All power to make decisions lies with management and if there are developers involved at all, it's the top one or two who are de facto part of management.

<comment><USER12> Agile methods do have a bit of a problem with the concept of authority. They really really want to believe that at least within a development team, everyone is equal. That's not a great fit to actual dev teams in companies where there are junior devs, senior devs, architects, and managers.

<comment><USER14> A senior dev in an Scrum team is supposed to do the same work as the junior, except faster!

Then they'll wonder why the experienced people quit.

<comment><USER15> > _A senior dev in an Scrum team is supposed to do the same work as the > junior, except faster!_

Where do Scrum principles/rules/... say that? (I have only passing familarity with, but I haven't seen much that went into more detail than "the development team organizes itself to complete the tasks", which leaves the internal structure open)

<comment><USER14> It's just my experience at multiple companies doing Scrum.

</comment></comment></comment></comment></comment><comment><USER16> This might just be the best, well-balanced talk on how agile has gone wrong, and ways to combat the decay.

I've seen a lot great teams and poor teams operate. A common set of components in great teams: Technical excellence, freedom coupled with accountability and giving a shit about the quality of their work.

Poor teams: lack of discipline, not caring and poor technical skills.

I've also seen great teams fall apart due to external influence. That influence was agile gone wrong, imposed from the top. Freedom: gone, but accountability remains. Technical practices valued by the team: deprioritized by the process. Giving a shit: demotivation followed by quitting or being fired.

</comment><comment><USER17> I started working with agile since day one and it was fun back then. Why? Because it made projects so much more interesting. We were learning how to do things differently and it was pretty cool. Unfortunately, it became more popular and upper management started looking at the benefits of Agile. "Tell me one reason why we should adopt agile in our company?". Unfortunately, a good answer to that was the ability to track the progress and a large increase in delivery and productivity. Teams hate those words but management love them.

That's what agile became today, tracking progress with constant status updates, charts and metrics that tell a team "clearly you did a really bad job this time" or "you're late". Management doesn't even have to do the dirty job anymore. Meaningless meetings, less power for engineers, no creativity (ain't got time for that!), etc.

It went from a cool way to do things to a perfect tool for factories. No offense to factory workers.

<comment><USER18> And the illusion of getting things done, because tickets! makes it seem like progress is happening even if churn is all that's happening. Last contract (I left) was 6 years into churning the same awful pile of spaghetti. I came on board to help break the logjam. Micromanaging Agile manager put me on fire fighting, just like all his other reports, just like the last 6 years of churning had done. Left after 6 months and it was obvious they were going nowhere.

</comment></comment><comment><USER19> My problem with current "Agile" beyond the complaints in that talk is it can mean absolutely anything. Nothing is falsifiable, and the snake oil industry views that as a feature.

Most of that revolves around the way retrospectives are handled and the weird way in which "continuous improvement" is viewed. Agile sees this as continuous process improvement, instead of say, people improvement, team improvement, or software improvement. This process for creating process (aka "bureaucracy") has led to retrospectives evaluating things like burndowns, "What about our process went well?" or "What about our process didn't go well?", etc. Scrum has basically a built-in bias to destroy itself: "We've iterated into this super process-oriented way of doing work, and we did it in small steps.... _Agile!!!_ "

I once made a comment that neither I, my company's customers, nor their customers were paid in story points. I still think that is the crux of why Scrum is so broken.

Agility will become more pronounced in software teams when the feedback mechanism ("retrospective") is based upon 1) qualitatively evaluating product success, and 2) quantitatively measuring _something_ that is relevant to the product or business being built and not a peripheral concern like how much work was done or how it was done. Work done, predictability, and even lateness have very little to do with product success outside the realm of contracting.

<comment><USER20> Focusing on story points is not Scrum, so why blame Scrum? Rather it's clearly delineated that a Scrum team product owner's main job is to maximize the value delivered through development to the customer. 50 story points means nothing unless the PO has prioritized work that's going to yield the most value to the client.

Scrum isn't broken. It's literally all laid out in a 16 page document. Nowhere will you find mention of story points.

Stop blaming the framework for someone's bastardization of what said framework is meant to achieve.

</comment></comment><comment><USER21> >We have to recognize that and fight against it because some people have said, "Oh, we're going to 'post-agile', we've got to come up with some new word," \- but that doesn't help the fundamental problem. It's the values and principles that count and we have to address and keep pushing those forwards and we might as well use the same label, but _we 've got to let people know what it really stands for._

The heart of the problem he's describing here stems from the fact that those principles were originally quite vague:

* Individuals and interactions over processes and tools (does this mean prefer meetings to writing tests and creating a CI pipeline?)

* Working software over comprehensive documentation (don't write documentation? do write documentation? Why not both? Moreover even the most waterfall of companies I've seen have never actually prioritized documentation over working software)

* Customer collaboration over contract negotiation (never did encounter a situation where this principle could even be applied... does this phrase use contract negotiation as a metaphor or is he talking about literal contract negotiation?)

* Responding to change over following a plan (he actually complains in this article that he never meant "don't plan"... idk)

Contrast that with, say "develop iteratively, in small chunks and, where feasible write the test first and release frequently and talk regularly with the customer" \-- those are specific principles.

If you create a vague set of principles surrounding an idea that resonates deeply with people then the emergence of a "priesthood" that doesn't necessarily agree with your original intentions is somewhat inevitable, no?

<comment><USER22> Yes, the values are too vague. The twelve principles are better for conveying what Agile is trying to be.

The "responding to change" point means in practice that it's better to deviate from a plan than to not have any plan to deviate from. The plan has to be adjusted all the time. For that to be possible, there has to be a plan.

The twelve principles are very basic, common sense. Common sense is not common though.

<comment><USER23> Well said!

</comment></comment></comment><comment><USER24> Around me agile has turned into 2 week mini-waterfalls, and everything else seems a very distant past.

<comment><USER25> I've been in a project like that for a while and it was actually pretty good, we got stuff done on a regular and predictable cadence.

<comment><USER24> Not saying it is that bad, just confirming that agile hasn't lived up to the promises as most companies aren't willing to go all way in.

Which is understandable, because according to my experience it is very hard to fit all departments into an agile concept, specially in companies which aren't focused on selling software.

Then you also get stuff like 100% unit test coverage for beautiful management reports, which blows up development costs, as some teams spend more time writing mocks and trying to find ways to test UI code than actually writing product code.

<comment><USER20> You can lead a horse to water, but you can't make them drink it. I don't know why anyone would blame the water for a dehydrated horse.

<comment><USER24> Maybe the water is tainted and staying dehydrated is a matter of survival.

</comment></comment></comment></comment></comment><comment><USER26> This article really hit home for me living in London. Having spent 7 years working in San Francisco and the valley, the degree to which "project management" and the use of buzzwords in the greater London tech scene are divorced from their original uses drives me absolutely bananas. And what's worse is that faux-agile has taken such a strong hold that _I_ seem like the crazy person when I tell them that's not how software is developed in California.

<comment><USER27> I live in the U.S. but I'm working remotely with a mainly U.K. based team that's doing "agile". The way they do it, at the end of a sprint all your tests have to pass, but it's normal for the trunk (yes we're using SVN) to be totally broken until then. That's all a sprint means to them, a block of time when it's okay for things to be broken.

I think this company got into it's current predicament by buying a ticket tracker that uses the word "agile" and never learning anything about it beyond the words in the tracker. Needless to say, the current process isn't working very well.

</comment></comment><comment><USER28> Perhaps we need to question the basic assumption that these ideas and values are worth a damn. The C3 project that started this whole circus twenty years ago completely failed[0] after if its primary business subject matter expert burnt out. This stuff only works when you have really good people committed to each other and their goal. That just isn't something you can expect to find in most of corporate america. If things are so bad that you are bringing in a process consultant, its almost a surety that agile cannot work for you.

[0]: [http://wiki.c2.com/?WasChryslerComprehensiveCompensationSucc...](http://wiki.c2.com/?WasChryslerComprehensiveCompensationSuccess)

</comment><comment><USER29> I've always wondered what makes Martin Fowler such an authoritative voice in software?

<comment><USER30> Fowler has been working for many years at a consulting company (ThoughtWorks), and thus has had insights into many, many software development projects.

Then he has started writing blogs, books, giving talks etc, which has increased his reputation.

I don't agree with everything he writes, probably because we have very different perspectives, but nearly everything technical he writes seems to be very solid, and I've never seen any bullshit from him.

I respect that, lots of other developers do that too.

<comment><USER21> He's definitely no bullshit but he's not very good at contextualizing the stuff he writes. Instead of writing "we did X in Y situation and it worked out great, but it probably wouldn't work well in Z situation" he just says "X is awesome, everybody should be doing it".

And that's the story of how I came to work on a team that maintained ~16 unnecessary microservices.

</comment></comment></comment><comment><USER31> That's unfortunately roughly what I am seeing.

It is pattern of management pushing on teams and then team members themselves declaring "agile" without willingness to actually understand what and why they want to achieve this way, without willingness to understand what Agile really entails and why it works. Being "agile" is a political issue now in every major corporation. Everybody "is" "agile" because not declaring "being agile" puts you against your management who themselves don't understand and don't care and wouldn't recognize Agile way of thinking if they stumbled on it. But that's just what you have to do because you already have scores of other things to fight for and you can't fight every war so you decide this is just one of the things corporations do and you will just try to do your job.

This is really problematic because it completely undermines efforts of people really understanding and trying to introduce Agile way of thinking. Since everybody else already "is agile" and since the effects can only be seen through participation of everybody in the way of thinking and only after a longer period of practice, the management is very unlikely to ever observe any Agile team and and observe positive results of that team as flowing from them working on being Agile.

Effectively management is never going to learn Agile because the setup is such that it severely impedes any efforts to introduce and promote Agile.

</comment><comment><USER32> Agile is like a religion. There is not a single consistent definition and any criticism is easily deflected along the lines that your organization did not embrace Agile on all levels of management. The uncomfortable truth is that Agile is internally inconsistent - you can not build an Agile methodology that will not contradict itself and that will not contradict the real world. A perfect example is the INVEST mnemonic for backlog items or the notion that Acceptance Criteria is against Agile.

One Agile coach ($$$) recently told me that the fact that I have contractual requirements and legal requirements in AC that I need to test for is a symptom of my customers not being Agile and the government not being Agile.

</comment><comment><USER33> I like to sneakily endorse people on LinkedIn with the 'Waterfall' skill tag.

</comment><comment><USER8> I've always had a problem with the scrum concept of team velocity i.e. assigning story points to tasks and then measuring how many story points a team delivers each sprint. The idea that this yeilds any predictive power is flawed. It's based on the assumption that the team is static. As soon as someone leaves the team or someone new joins, all the historical velocity data becomes useless and you need to recalibrate. The reality is that teams are dynamic. They grow/shrink/change all the time.

<comment><USER20> This is up to the team to help other people understand that velocity/productivity is more than likely decrease as team dynamics change. Even if more people are put on a product, the velocity is expected to decrease as those people absorb new responsibilities. Same story for losing people on a product.

</comment></comment><comment><USER34> I always knew that Scrum wasn't as agile as some people want to tell you, but this was the first time I read about 'Dark Scrum' [1].

In my opinion, Scrum is a good tool to kickstart the process of becoming an agile organization, but too often the process languishes there and generates a lot of frustration.

[1]: [https://ronjeffries.com/articles/016-09ff/defense/](https://ronjeffries.com/articles/016-09ff/defense/)

</comment><comment><USER35> This a lovely read, and rings true on so many levels.

Dark scrum huh? Nice.

I feel like there are a lot of people who will read that, about technical excellence, about teams choosing their own ways to work, about needing to constantly _change_ and feel uncomfortable about it.

...but, face it. If you do, you're not doing agile, you're just calling it agile.

</comment><comment><USER36> "... much of what is done is faux-agile, disregarding agile's values and principles." This. It feels so easy to get caught up in methods and tools, to not make a proper commitment (because it is tough to change the way you work) and to land in a sort of semi-agile state that no one likes.

</comment><comment><USER37> agile is a manifesto that I don't think should have ever become whatever people think it is today , its sort of become this Scrumish thing with consultants that try to sell faux zen like wisdom. Some of the silly thought experiments that were getting done on the email lists about how "extreme" we could take things that were done around the early 2000s have now become weird dogma. It's just a mess. I think part of the problem is that some of the core philosophy that people like Kent Beck were trying to capture and communicate was actually quite nuanced and subtle that it simply got lost in the noise of the new "Agile Industry" that quickly latched onto blunt and concrete rules about how to do things and a lot of things seemed lost and diluted over time.

</comment><comment><USER38> Is there a problem in what we might call a meta-framework like agile in that it requires the "good people" that Martin Fowler talks about when the reality is that most of our teams are at least partially average?

People have to learn examples of its implementation but are either not knowledgable enough or trusted enough to start doing things their own way, which causes the rigidity that everyone is talking about.

That and the fact that tooling is not something that most people can knock up in a few days so we relay on the software and its somewhat rigid processes which means we aren't free to innovate fully anyway.

It's a bit like complaining that most musicians only do music the way they learned at college and do not adapt to the "principles of music".

</comment><comment><USER39> Related: "The Tragedy of Craftsmanship"

[https://blog.cleancoder.com/uncle- bob/2018/08/28/Craftsmansh...](https://blog.cleancoder.com/uncle- bob/2018/08/28/CraftsmanshipMovement.html)

</comment><comment><USER40> This whole discussion echoes the DevOps discussion and its definition. What DevOps is and what it means to different people is still hazy and can be twisted into anti-DevOps quite easily.

<comment><USER30> It's even worse with DevOps, IMHO.

Agile at least has the Manifesto of Agile Software Development, which you can go back to, and see if a given process adheres to the ideas in the manifesto.

With DevOps, you have basically nothing foundational that people can agree on.

</comment></comment><comment><USER41> So it's probably an unpopular opinion on HN, but I've seen SCRUM implemented in 3 companies during my career and it has been an improvement for each one of them.

</comment><comment><USER42> my gigantic org is transforming to agile pretty well to be honest. wish there was some positivity in here. biggest problem is lack of technical excellence but people understand the point of agile pretty well and we are working through what makes it difficult to "be agile"

<comment><USER43> If everyone is adapting to it this may work. What I have seen in companies is that the devs suddenly have to do scrum but everybody else in the company does business as usual.

In my current company we did scrum training a while ago and not even the dev managers showed up. And it shows when you talk to them. They have no idea what agile really means other than sprints and "commitments". So instead of being an environment where everybody respects each other it has turned into a pressure cooker for the devs.

<comment><USER42> yeah it's complete buyin from senior execs to business leaders, to business side employees who are tasked with being product owner. no such thing as a tech lead, organization got flattened out etc.

<comment><USER43> That's how an agile introduction should be done but almost never does.

</comment></comment></comment></comment><comment><USER44> We're doing agile in a mechanical engineering firm. A multiyear project. The first thing we'll deliver will be two years from now.

I struggle to put into words why this is such a bad idea because its so blatantly obvious.

Imagine designing and building the golden gate bridge as an agile process.

<comment><USER45> Agile process doesn't necessarily mean agile delivery. But I might worry too.

</comment></comment><comment><USER46> "this realization that people are operating at the best when they choose how they want to work"

But what if how they want to work is "Dark Scrum"... or some kind of Agile religion, or waterfall.

</comment><comment><USER47> Many aspects of this talk (around the lessons to learn from Agile and failed Agile) can be applied more broadly to most functions and industries, not just software development.

<comment><USER24> Given the age of software industry we are quite late to the party, hence relearning history.

I still look forward to proper quality assessment and responsibility like in other industries.

</comment></comment><comment><USER48> [https://news.ycombinator.com/item?id=16772788](https://news.ycombinator.com/item?id=16772788)

</comment><comment><USER8> > ... the reality is troubling, because much of what is done is faux-agile, > disregarding agile's values and principles.

This is a problem with "agile" and scrum. If the process isn't working for you, it's your fault for not doing it right. And no one is ever doing it right. A truly "agile" team is one that operates in some impossibly perfect Zen-like state that no team achieves in practice.

Maybe it's not you. Maybe it _is_ the process.

<comment><USER20> Scrum isn't a process, it's a framework to utilize your own processes. That's like blaming the framework of the house for a bad paint job.

</comment></comment><comment><USER49> Agile Coach here.

FIRE YOUR AGILE COACHES

The monstrosity that agile has become is an insult to the courageous pioneers who spoke out and swam against the current (at the time) to save software engineers from an overreaction to the growing complexity of computer systems. The result has been an overreaction to an overreaction.

1\. MOST COACHES ARE MUPPETS

Most Agile coaches today have a serious lack of engineering and leadership experience. They are merely the sum of the Agile classes they've attended and the book or two they may have skimmed. Most agile coaches do not understand the theoretical underpinnings of their field and therefore can only parrot statements from other "agile leaders". Most can't write code, have never managed anything as large as the groups they coach, don't understand software development in general, thus you would be a fool to take advice from them.

Here's an example. A friend told me a story this week about a new coach who came in and pronounced that what they needed was "big room planning". He proclaimed this without having any context whatsoever. What is one to do when all one has is a hammer? This is not an anomaly, it's par for the course.

2\. YOU'RE BETTER OFF IF YOU DO IT YOURSELF

Today enough has been published about the dangers of approaching complex problems with the tools for solving merely complicated problems. You don't need a Priest of the All Mighty Agile to warn of you of Eternal Damnation and prescribe the One True Way. Every "tool" any agile coach can use to help you understand the challenges to production flow (such as Value Stream Mapping) can all be found in digestible articles on Medium. If you're having trouble finding these my email is in my bio. If you, as a leader, take the time to understand your chosen field of Software Engineering Leadership, and then... you know... actually _lead_ , your organization will subsequently obtain the necessary habits to continue to change and improve as your company grows and changes. The value of this cannot be overstated. Most companies calling me for help today have anemic leadership and that's their problem. They don't need an expert in software delivery performance to help them skip some of the boilerplate, they need Extreme Ownership and true leadership. Instead they continue suckle on the tit of Agile Coaches 5-10 years into an "Agile Transformation" when they should be up on their own two legs walking and eating with a fork. Agile Coaches have, if anything become an enabler to the worst corporate cultures and mindless bureaucracies software engineers have ever had the displeasure to work under.

3\. YOU CAN'T SHIP BECAUSE SOFTWARE DOESN'T MATTER

Lastly, the reason most companies who struggle to increase quality and simultaneously increase delivery performance (aka Business Performance in 2018) is because they don't care about software. It is true, no one starts a bank to have an IT department. However some companies have awoken to the fact that in 2018 TECHNOLOGY PERFORMANCE IS BUSINESS PERFORMANCE. If you are not one of these companies you (personally) have two choices, go and work for one, or lead your company to become one. The alternative is... a steady paycheck? Even in the most laggardly industries, leaders are waking up to the fact that getting technology performance right is the only competitive angle. When these clients call me the things we accomplish are breathtaking. Sadly, they are a tiny minority. Most companies call coaches so a VP can show he's checking all the boxes, so they can continue with the status quo. Some financial institutions may talk about the danger of "Fin tech startups" but when you have enough in assets that you can coast for 50 years, or you have billions in infrastructure that a competitor with all the money in the world couldn't replace inside of a decade, the reality is SOFTWARE DOESN'T MATTER. You can suck as much as you want because the challengers are so incredibly far off it really doesn't matter. These places make Agile Coaches lots of money and they're also the ones who fill the conferences. Technology performance doesn't really matter at these companies. This is why you (who work in these companies) struggle to have a sane build pipeline, adequate test coverage with good tests, communication across teams, and you get nowhere. You're swimming against a tsunami. It's exhausting, don't waste your time. Find somewhere that actually sees the leverage technology can provide them and go work there. You'll be a hero.

</comment><comment><USER50> Four weeks ago my team changed from not-really-scrum-we're-just-pretending some some sort of kanban-like process, which was the first real change after I stuck an item on the wall during a retro which just said "Are we doing Scrum? Should we be?"

Today we had a retro in which we talked about how things are going, and we're all more comfortable with how the process fits our actual workflow, and how it's opened up some other changes that have really helped us with getting useful stuff done, delivered and in the hands of people who appreciate it.

And this came from within the team, entirely, with the support of our team lead.

It's not perfect, but it's good, and we feel able to change things. That's why the other changes happened - because we were reminded that we have the power to change things, and the company actually genuinely believes that. There are some things we can't change, or not so easily, because they cross team boundaries or require money to be spent that we can't convince someone to authorise, but we have more freedom within the team than we had believed that we had. Largely I think because most of us are fairly new to the company and had long ago forgotten the idea of self-organising agile teams because the very concept was a joke most other places we've worked.

I suspect we'll change things in the future, and we do have some concerns at the moment about how we split up big bits of work like implementing a completely new system component into cards of a suitable size and self- containedness and various other issues around definitions of done, but we'll work it out. For now, we're apparently more productive, we certainly feel more productive, and we're all happier because we don't have the entirely artificial pressure to ship something every sprint. That's not how the software we maintain works, and not how it should work either. We just ship stuff when it's ready, because that's what makes us and our stakeholders happy.

What Fowler's describing is something I've seen so many times. Scrum-adherence is in itself something of a cult in some places, and it's done without any mindfulness for why you choose certain processes, the purpose of certain things and whether Scrum was a good choice for you or not.

And I think honestly that's most people's experience of "agile" software development, because people have turned it into an industry of its own and with that comes the books, the rules, the certifications, and the legions of people who read a bit about it and decided that'd be great and it'd tick the trendy "agile" box without having to disrupt the company too much.

Bleergh.

Oh and something else my current employer does that I've seen literally nowhere else? A no-blame culture.

We actually have one.

I wish everyone could try it.

</comment><comment><USER51> Much, much better transcript version [https://martinfowler.com/articles/agile- aus-2018.html](https://martinfowler.com/articles/agile-aus-2018.html)

<comment><USER52> Ok, we changed to that from [https://www.infoq.com/presentations/agile-2018](https://www.infoq.com/presentations/agile-2018). Thanks!</comment></comment>


Fixing the worst law in technology  - <AUTHOR> http://www.newyorker.com/online/blogs/newsdesk/2013/03/fixing-the-worst-law-in-technology-aaron-swartz-and-the-computer-fraud-and-abuse-act.html

<comment><USER2> Good god, this is stupid.

 _The damage was trivial, yet he is threatened with two hundred and fifty thousand dollars in damages and up to twenty-five years in prison._

Really? The prosecutor is seeking the maximum statutory penalty for this? Somehow I doubt that.

 _All the Administration needs to do is to rely on the ancient common-law principle called the "rule of lenity." This states that ambiguous criminal laws should be construed in favor of a defendant._

BY A COURT. The rule of lenity is something that is supposed to guide the behavior of _judges_.

This is lousy journalism, which misleads the audience in an attempt to pander to their sensibilities. There is no way the author of this piece ran it past a lawyer.

<comment><USER3> Well, most of my legal knowledge comes from watching The Wire, but I think you're too critical of the OP's point, though he obviously could've stated them clearer:

> _Really? The prosecutor is seeking the maximum statutory penalty for this? > Somehow I doubt that._

The penalties listed are what's defined in the statutes. Keep in mind that the case at hand was just revealed. The prosecutors won't be saying what penalties they seek until the bargaining and sentencing phase.

> _2\. BY A COURT. The rule of lenity is something that is supposed to guide > the behavior of judges._

You're correct that this is a guideline for judges. But the OP is apparently referring to "the rule of lenity" in how 13 federal judges so far have rejected the DOJ's interpretation of the CFAA. To the OP, this is proof that the law is so vague that it needs to be changed, or else "the rule of lenity" will constantly be invoked.

<comment><USER2> OK, but any journalist writing on legal matters should know that the statutory maximum is not a good guide to the likely sentence in the event of a conviction. To mention the statutory maximum without any qualification is sensationalism masquerading as cold fact.

By that standard I could say 'X has been arrested for littering - he _could_ face the death penalty!!' Well, it's possible - until X has been charged with something specific, who am I to say he won't get hit with a capital murder charge or somesuch?

I don't agree with you about the how the OP is using the rule of lenity. He specifically says the executive needs to change its enforcement on that basis, whereas changing the law is something that would be done by Congress. If he can't say what he means then he has no business reporting on legal topics.

<comment><USER4> >To mention the statutory maximum without any qualification is sensationalism masquerading as cold fact.

If may be misleading, but strictly speaking it's factually accurate. That is the maximum penalty -- and as long as it is, people who don't like it are going to report it.

If you want them to stop, amend the law so that minor offenses don't fall under the same section as major offenses. For a law to carry felony penalties, it should require something in the nature of profit motive and nontrivial actual harm. Then some fool who vandalizes a website for fun can be convicted of a misdemeanor and pay a fine or go to jail for a couple months while we still get to throw the book at people who make fraudulent credit card charges or engage in industrial espionage, and you won't see any more stories about how the former class of people could be jailed until death for the computer equivalent of disregarding a sign that says "don't walk on the grass."

<comment><USER2> They're only reporting _half the story_. I don't want them to stop reporting the maximum penalty, I want them to put it in its factual context. But most won't, because if they said 'the statutory maximum is 25 years but most convictions draw sentences of less than 5 years,' people would not think it's that big of a deal.

As a matter of fact, the mean sentence under the CFAA (as of 2008 EDIT: typo. This is as of 2000.) is only _6.8 months_. Minor offenders tend to get probation, and the heaviest sentences fall on those who interfere with the administration of justice or commit industrial espionage - and they face an average sentence of 17.3 months.[1]

1\. <http://proc.isecon.org/2000/128/ISECON.2000.Nwokoma.pdf>

It's a bit pointless demanding reform of the law without any context for how it is actually working in the real world.

<comment><USER4> >if they said 'the statutory maximum is 25 years but most convictions draw sentences of less than 5 years,' people would not think it's that big of a deal.

I imagine people convicted of manslaughter also don't typically serve anything like the statutory maximum. But we calibrate our expectations and sense of proportionality based on the maximum because that's the number that gets published and compared to penalties for other offenses, and actual sentences are often set in proportion to the maximum, so changing the maximum has the practical consequence of changing the result in typical cases.

>As a matter of fact, the mean sentence under the CFAA (as of 2008) is only _6.8 months_.

If you get convicted of a felony and sentenced to probation, you still get a felony record. I don't think we should be making felons out of people who have done something that deserves to be punished only with probation. The maximum penalty is extremely relevant because that's what makes it a felony, even if it bears little resemblance to what happens in the typical case.

>It's a bit pointless demanding reform of the law without any context for how it is actually working in the real world.

By the real world you mean the court system, but what about the real real world? The maximum is what you read in the statute when you're trying to decide whether to break an unclear law for a good reason. It probably has more influence on the behavior of people in the real world than the unpublicized actual outcomes do.

<comment><USER2> _But we calibrate our expectations and sense of proportionality based on the maximum because that's the number that gets published_

Well, this is the problem - it's often the only number that gets published, because it makes for a better story than qualifying that sensational number with the rather less exciting reality of sentences administered.

You can't go complaining about the length of the maximum sentence and then waving aside the much lower sentences that are actually handed out as if they didn't matter.

 _I don't think we should be making felons out of people who have done something that deserves to be punished only with probation._

We're not, generally. Not every offense prosecuted under CFAA is a felony; in many cases defendants are charged with a misdemeanor instead. This was mentioned int he link I gave you earlier, and it's treated in much more detail here: <http://www.justice.gov/criminal/cybercrime/docs/ccmanual.pdf>

This is why I'm grumbling about the inaccuracy in reporting. You see the big maximum sentence as if it were the only thing that mattered, and you develop a really inaccurate idea of the law as a result.

<comment><USER4> >Well, this is the problem - it's often the only number that gets published, because it makes for a better story than qualifying that sensational number with the rather less exciting reality of sentences administered.

I think it's more of an issue that the truth is complicated and a journalist can't cover a story in the detail you might like without losing too many readers to "TL;DR" or spending more hours to research it than the editor has assigned for the story. The typical sentences handed down are by far not the only important thing left out of articles like this. The things missing from these articles could span volumes of books. And each one of them needs to be put into context. If you discuss the typical sentences handed down then they ought to be compared to the sentences handed down for other similar offenses, or to the offenses that have similar typical sentences, and a thorough treatment would go down the rabbit hole into plea bargaining and sentencing guidelines and the whole works. But given limited resources and limited column inches, instead we end up with heuristics like using the maximum penalty as a proxy for the severity with which the law treats the behavior.

Whenever there is a popular push to change the law, most people aren't going to understand all the details. People don't have time to learn everything about everything. At some point the layman will have to defer to someone trustworthy (like the EFF or the ACLU) to do the legwork and then adopt their position on the strength of their reputation. _They_ need to get down into the weeds and make sure they understand what's going on, but they aren't the ones getting their information from popular media, they're the ones reading the statutes and the case law and engaging in discussions with scholars and legislators etc. Articles in The New Yorker aren't meant for those people, they're meant for the man in the street who needs to be made aware that something is going on and the people who have done their homework are saying it's a problem and we should do something about it. That way a critical mass of people willing to support doing something about it can be achieved, even if it isn't possible for all of those people to each be individually aware of every nuance of the issue. If someone wants to know more then the information is available. Anyone so inclined can look it up. But most people won't; I don't see any obvious way around that. And an article that skims over some of the details for the sake of brevity and approachability is a lot more likely to be read by those people, so that at least they know that something is happening and maybe some subset of the currently-oblivious can be inspired to learn more.

>Not every offense prosecuted under CFAA is a felony; in many cases defendants are charged with a misdemeanor instead.

I'm not sure how that changes anything. That some cases are prosecuted as misdemeanors or in an otherwise unproblematic fashion doesn't mean we can't do better in the cases that aren't. And if many of the cases prosecuted as misdemeanors could under the law have been prosecuted as felonies but for the grace of the prosecutor, that still points to a problem in the law.

<comment><USER2> Look, I get it, you want to change the law and I agree with you in many regards about the necessity of doing so. But you can't dumb your way down to success over the long term, and misleading your readership is neither effective nor ethical.

<comment><USER4> Well I'm open to suggestions. How do you get a million instances of Joe Bloggs to appreciate the subtleties of plea bargaining negotiations without causing them to change the proverbial channel?

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER5> Perhaps we should make a site who's terms of service explicitly bars use by members of congress. Once some member of congress goes to the site, we try to get them put in jail, at which point they'll change the law.

<comment><USER6> Good idea, but I don't think it is possible for a private citizen to bring a prosecution. You could report it as a crime to the local police / FBI, but then they would probably file charges against you for wasting their time.

<comment><USER7> It's all in how you approach it. If your goal is change then the press is your best option. So, send a vary nicely worded letter to the FBI and the press along with a video and logs and the FBI will simply add you to a few watch lists...

</comment></comment></comment><comment><USER8> The law of any overly generalized and overly complex law: the law is no good.

Don't try to repair it. Rip it out and establish clear, concise, use-case specific laws, when necessary.

</comment><comment><USER9> That law is pretty bad, but I wonder whether it is the worst law in technology. Can anyone think of a contender?

<comment><USER10> The Mickey Mouse Act springs to mind... <http://en.wikipedia.org/wiki/Copyright_Term_Extension_Act>

<comment><USER4> DMCA 1201 is definitely on the short list too. But I don't think either of them is strictly as bad as the CFAA. At least with the DMCA you have some general idea when you might be violating it. Nobody even really even seems to know exactly what the CFAA prohibits, and even the narrow readings make felonies out of stuff that really ought not to be that serious.

</comment></comment></comment><comment><USER11> Does Matthew Keys deserve to mentioned alongside Aaron Swartz? Swartz's actions seem somehow more... noble. Both of them "abused" their access, but Swartz did so to make a political point about scientific knowledge, in the hopes of improving our society. I can't discern what Keys's motivations were in this case.

It's possible however that this makes Keys a "better" defendant, in that the law is clearly unjust even in his case.

</comment><comment><USER12> The author of the article has overlooked something ancient and obvious. Jury nullification (<http://en.wikipedia.org/wiki/Jury_nullification>) is exactly for laws such as this one. It's been part of English-based common law for many centuries. If the Swartz case had reached trial and I had been a juror, the chances of me nullifying the "worst law in technology" would have been near 100%. I'm sure most of us feel the same way. If I'm right, the problem is not as bad as the author makes it seem.

</comment><comment><USER13> '... President Obama ... the ultimate enforcer of the law'

Um what?

<comment><USER14> That's the role of the executive branch, and President Obama is the head of the executive branch. It's weird to see it phrased that way, but it's not incorrect.

<comment><USER2> Yes, but he can't just rewrite the law by making a speech, as the author suggests. He's charged with the faithful execution of the laws, not their interpretation.

If you read court decisions where a law is criticized by a judge or justice as unjust or unfair (but not necessarily unconstitutional), they _never_ say that prosecutors should stop bringing such cases to court; they say that changing the law will require _Congress_ to act.

To take a current example of the dichotomy, the Obama administration has filed a brief with the Supreme Court asking it to overturn DOMA (Defense of Marriage Act), as it's the administration's belief that the law is unconstitutional - but in the meantime, the administration is still implementing DOMA as written, because until the SC rules against it (if they do) it remains the law of the land. If two married gay people file a joint tax return, the IRS will send it back with an apologetic letter and tell them they have to file separately and so on.

<comment><USER15> Not sure if you pulled your example from the politifact article, which was the top Google hit on "enforcing doma", or not... but it also mentions specific examples of where DOMA is NOT enforced because the administration has decided it has more authority than is Constitutionally and legislatively prescribed.

[http://www.politifact.com/florida/statements/2012/apr/09/ver...](http://www.politifact.com/florida/statements/2012/apr/09/vern- buchanan/vern-buchanan-says-obama-will-no-longer-enforce/)

<comment><USER2> I hadn't read that. However, in the counter-examples given, which all involve immigration, the AG is using his explicit statutory discretion to halt or defer proceedings on individual cases.

</comment></comment></comment></comment></comment><comment><USER16> Good to see positive change come from tragedy.</comment>


Ask HN: What do you use for authentication and authorization? - <AUTHOR> I am currently starting work on a new app/website. Currently planning to have 1 BE API set to start, probably graphql (which will be user data/information and need to check with the auth server about being protected). I will also have many client apps (web, mobile, potential partners) that will need to make queries to that BE. Do you usually roll your own authentication or use something like auth0/fusionauth/gluu/etc? This product is going to need to be secure as it will be in the healthcare space (so think oidc). <comment><USER2> Hard to say without more concrete details, but if I had to reply in broad strokes:

\- For web, user/pass login exchanged for plain session cookies. Should be marked httpOnly/Secure, and bonus points for SameSite and __Host prefix [1]

\- For web, deploy a preloaded Strict-Transport-Security header [2]

\- For api clients, use a bearer token. Enforce TLS (either don't listen on port 80, or if someone makes a request over port 80 revoke that token).

\- If you go with OpenID/Oauth for client sign-ins then require https callbacks and provide scoped permissions.

\- Don't use JWT [3]. Don't use CORS [4].

Again these are broad strokes - if you gave more information you'd get a better response.

[1]: [https://developer.mozilla.org/en- US/docs/Web/HTTP/Headers/Se...](https://developer.mozilla.org/en- US/docs/Web/HTTP/Headers/Set-Cookie)

[2]: [https://developer.mozilla.org/en- US/docs/Web/HTTP/Headers/St...](https://developer.mozilla.org/en- US/docs/Web/HTTP/Headers/Strict-Transport-Security)

[3]: [https://en.wikipedia.org/wiki/JSON_Web_Token](https://en.wikipedia.org/wiki/JSON_Web_Token)

[4]: [https://developer.mozilla.org/en- US/docs/Web/HTTP/CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS)

<comment><USER3> What's the difference between a Bearer Token and JWT? I thought they were related?

<comment><USER4> A bit of misinformation in this side thread.

A JWT token is composed of three parts: a header, payload, and signature.

The problem is that people can put sensitive info in the payload.

None of it is encrypted, it's only signed with HMAC.

Unless you're keeping track of the tokens, once a token is issued it's valid until it expires, due to it's stateless nature.

You can use a JWT as a Bearer token, but since it's only base64 encoded, you can pull out that payload data.

A truly opaque Bearer token will be meaningless to anything other than your server.

Play with the debugger here to see what I'm talking about: [https://jwt.io/](https://jwt.io/)

</comment></comment></comment><comment><USER5> Healthcare is a dangerous sector for a security novice. Please make sure you are familiar with HIPAA [0], including your obligations when handling health information and the nature of possible sanctions. Handling health data at all is risky. Sharing it with partners is something you probably shouldn't even consider before you can afford a serious legal team.

OpenID is a mechanism for one website to assert a user's identity to another website. OAuth is a way to let a user delegate access to some of their data on one site to another site. Neither have any particular affinity with the healthcare space, and they are not things you sprinkle on for extra security.

[0] [https://www.hhs.gov/hipaa/for- professionals/security/index.h...](https://www.hhs.gov/hipaa/for- professionals/security/index.html)

<comment><AUTHOR> Makes sense. I am sure I misworded, and got turned around a bit. Much of the documentation with fhir talks about oidc. Which seems to be in place if you are doing much more sharing of your data. These things as you mention are probably beyond what is necessary initially and could be added at a further date. However using a service or an open source project that can allow to scale to that size is an interesting proposition.

<comment><USER6> If you are handling any kind of medical data about people, then you cannot think about security at a future date and your life will be difficult from the start.

<comment><AUTHOR> Did I say security? I said sharing at a future date.

</comment></comment></comment></comment><comment><USER7> I would echo a bunch of the comments in this thread. I am only posting this because I own a healthcare startup and your question put the fear of god in me. I would strongly advise that you build a different app if you do not already possess the knowledge required to do this. If you have the means hire a professional and you will be better off. You need to get this right from day 1 or people will get hurt, you will be fined into oblivion and/or possibly thrown in jail. There are also numerous other security concerns that you should be worried about since Auth is just the tip of the iceberg.

I will add the following details which are specific for healthcare companies and a bit of inside baseball. Typically people use an email address as the lookup field for a user account. Since email addresses are considered protected health information under HIPAA, I would highly suggest you use usernames instead (possibly auto generated to be safe). HITRUST includes some details about password rules for their certification process. No one will question you if you follow their rules (I think it's 12 characters minimum 1 Upper/Lower/Symbol/Number each). Use a banned password list, you can find one here[0]. You are going to want to set up some manner of 2 factor authentication (I would recommend U2F or TOTP) for all accounts with manual code backup. OAuth and OpenID are goddamn nightmares. You need to own and manage this process entirely by yourself.

[0]: [https://haveibeenpwned.com/Passwords](https://haveibeenpwned.com/Passwords)

</comment><comment><USER8> Professionals.

Hire an expert.

If you can't answer these questions yourself (which is fine - it's specialized knowledge separate from the skillset needed for building a useful application), you are lacking _critical competence_ for coding anything handling health information.

The security minefield is much much bigger than the login page.

<comment><USER9> Classic hacker news. Ask for technical advice, get called incompetent.

<comment><USER10> If they asked a bunch of doctors how anesthesia worked, they were just about to go perform surgery at home, you'd expect the doctors to warn that it was a bad idea, no?

<comment><USER9> I suppose there may be a distinction between asking "how does anaesthesia work?" and "should I perform surgery at home?".

<comment><USER11> "A is to B as X is to Y", compares A and X, not A and B. It puts A in the context of B, as X is in the context of Y.

the comparison is between "how does anaesthesia work?" (A) and "how does auth work?" (X), relative to "about to perform surgery at home" (B) and "about to implement a service containing medical information" (Y).

The point is: he's about to do something big, and is asking a basic question. The real problem is not the basic question (auth) but the context he's doing it in.

If a nurse in training, in a classroom setting, asked about anaesthesia, it'd be fine. If they're a doctor, about to operate on a live patient, it's different.

<comment><USER9> I like the analogy in your final paragraph. I see hacker news as the nurse in the classroom setting.

</comment></comment></comment></comment></comment></comment><comment><USER12> Before writing any code you should seek to deeply understand the problem space of authentication and authorization. HIPAA compliance is primarily an authorization problem, not an authentication problem. That is: both are important, but the unique set of challenges within the scope of HIPAA have to do with _authorization_ of read/write access to data, not authentication.

Authentication asserts identities. Authorization asserts capabilities. This shifts and compartmentalizes the problem somewhat. Almost all interactive applications need to support robust authentication, but most applications do not require the sophisticated authorization restrictions HIPAA demands.

Whatever it is you choose, you should:

1) Use a mature, reputable library;

2) Use a library which provides the simplest possible interfaces for solving your needs in the most turnkey manner;

3) Engage with a reputable consulting firm specializing in HIPAA compliance and application security.

I would also recommend reading through as much information about Aptible's architecture and design ethos as possible. They have done an excellent job of navigating this problem space.

</comment><comment><USER13> I roll my own using well supported libraries for the languages I work with. These libraries handle the gory bits and pieces where it's easy to make mistakes.

It's a split between using passwordless logins, or standard password authentication depending on who the target audience is.

I would never in a million years ever think about using a service like auth0. It's not just a huge privacy issue but now a critical component of your app depends on a third party service. Also I know of a few sites who use it and the user experience is really bad. It seems like every other time I access a site that uses it, I have to goto a third party auth0 screen to re-enter my login details (which are already auto-filled out by the browser).

Your user authentication flow is a very unique aspect of your site and it's also one of the first things your users see.

You should have full control over it because if your user's first impression is a slow loading non-intuitive user auth system that bugs them to login every few days they're probably going to look for a competing service. I know I would.

<comment><USER14> So... what are those libraries?

<comment><USER13> That depends on what language / framework you use.

With Flask I use Flask-Login. For Rails I still use Devise usually and with Phoenix I just use Plug.Session directly.

</comment></comment></comment><comment><USER15> For authentication I use Auth0 on the free tier, with a passwordless setup that uses Google OAuth and Microsoft OAuth and allows fallback to emailing a code to a user. We store nothing more than the email address. The great thing about Auth0 is the separation it provides between the authentication layer and the web app, and how if you go down the SaaS route you can allow people to bring their own Auth0 accounts and configure their own bespoke authentication.

For authorization you are going to have to implement your own solution once you have an authenticated session. What someone can do always depends on your app and the functions you provide and so there is no nice third party solution to this. In my case I store the map of users to roles and what a role can do in a PostgreSQL app and cache there the answers to "which users are in a role" and "what can a role do"... user permission and roles changes are infrequent but flush the cache and so take immediate effect.

<comment><USER16> How do you implement that cache invalidation, assuming a multiple app server environment? Is it something like a separate redis server?

<comment><USER17> Yes, exactly: you use a fast data store such as redis or memcached.

As a user performs activities, this may involve a scenario requiring escalation or revocation of authorization roles and corresponding permissions. Invalidate cache at this moment. Lazy cache updated authorization info upon next request.

(I authored Yosai)

</comment></comment></comment><comment><USER18> Ory Hydra

[https://github.com/ory/hydra](https://github.com/ory/hydra)

Open source OAuth / OpenID connect server

The docs, API and Docker images make it really easy to start developing against. Then the Docker images and database migration tools make it easy to deploy into our production infrastructure.

Also evaluating the other Ory tools like Keto, a policy engine.

The hackability of these is very attractive over closed services like Auth0.

<comment><USER19> Dex is another OSS option:

[https://github.com/dexidp/dex](https://github.com/dexidp/dex)

</comment></comment><comment><USER20> Since it's in the healthcare space, have you considered hiring a consulting team? Or at least a firm? What other Hacker News readers use for authentication is largely irrelevant since we don't usually work on HIPAA- compliant applications.

<comment><USER21> _What other Hacker News readers use for authentication is largely irrelevant since we don 't usually work on HIPAA-compliant applications._

I think this is a rather sweeping, and wrong, generalization.

Though the chattering masses on HN may not be HIPAA devs, that doesn't mean there aren't HIPAA devs here. I'm one of them.

Every time someone brings up an obscure topic on HN, there always seems to be a group of people who specialize in that topic that come out of the woodwork and have fascinating insights.

The largest number of commenters on HN seem to be Googlers, Facebookers, and one man bands. But there are plenty of, for example, Apple devs on HN. They just choose to keep the S/N ratio high.

</comment></comment><comment><USER17> It depends on what type of system you plan to support. If it's for a hospital setting featuring many different types of actors, roles, and constraints, this requires a greater level of sophistication.

 _Beware, authorization is an Alice in Wonderland rabbit hole where one may fall far deeper than one expected to._

A few years ago, I ported Apache Shiro from Java to Python, resulting in The Yosai Project: [http://yosaiproject.github.io/yosai](http://yosaiproject.github.io/yosai)

It was a grueling but rewarding experience.

I honored Shiro in name and license, open sourcing everything and using Apache 2. I went even further than Shiro by adding two factor authentication workflow using totp and including starter modules for caching, data store, and integration with the web app I was using (pyramid).

If you choose to use python, or even just want something to learn from and reference, check out Yosai. I put a lot into this work to make it useful for others, entirely on my own.

I spoke with Tobias (podcast init) about the project some time ago: [https://www.podcastinit.com/yosai-with-darin-gordon- episode-...](https://www.podcastinit.com/yosai-with-darin-gordon-episode-120/)

</comment><comment><USER22> If someone else needs to use your API, please use OAuth/OIDC. I've seen countless hand-rolled (and insecure) authentication schemes. It's almost always a nightmare. If you plan on connecting this to multiple apps and partners, rolling your own will create a hellscape for everyone

If you use OAuth, anybody can connect using a standard library and "flow" across tons of languages. And you don't need to worry about screwing up the most security critical part of your API. It will save you countless headaches.

Some people bash OAuth because of JWT, but this overblown. Storing permissions in a token is the only sane way to do things if you end up with multiple services down the line (you will).

The whole revocation debate is a bunch of noise about nothing. Make the expiration interval fairly short, and if that isn't enough you can make a cache of revoked tokens that only needs to live as long as your expiration interval. This is still orders of magnitude more efficient than not storing permissions in the token and just as secure.

If you need to revoke all tokens in a breach scenario you just change your signing key. I recommend using SHA256 based signatures rather than public/private key since even though public/private is theoretically more secure, calculating signatures is quite a bit of overhead. If your backend is using a fast language like java/C#/Go, the majority of server CPU will be signing tokens.

Read about OAuth and ignore the haters. The design is well thought-out, secure, and efficient. If it was really that bad you wouldn't see most of the tech giants migrating to and using it. There's a lot of people that don't like it because they don't understand it well enough.

</comment><comment><USER6> I was in a similar position, we started using Okta and eventually migrated to AWS Cognito. Rolling your own auth is a recipe for disaster unless you know what you are doing and really need to. Also, be prepared to be fairly locked in once you choose an auth provider, especially if you choose one that is fairly integrated into your ecosystem.

<comment><USER23> We use Lambda authorizers, how do you use cognito

<comment><USER24> You can create a custom auth flow using lambda and cognito; you can return a series of challenges and create a stateful flow using session tokens which results in a set of access, identity, and refresh tokens.

Alternatively you can use the auth code flow baked into lambda; if you have premium support make a case and someone can walk you through it :)

</comment></comment></comment><comment><USER10> Edit: just read other comments pointing out the healthcare thing. Hire a professional.

Honestly, authentication is not that hard. There are many ways to do it, all with valid trade offs.

What you need to know is what your AUTHORIZATION story will be. Can anyone who can hit your API receive all data? Otherwise, you either need some kind of stateful access control or some kind of bearer token granting certain kinds of access. If the latter is simple enough for your use-case, then JWT, despite it's naysayers, might shine for you. Otherwise you can just use about anything since you'll be looking up what they can access in a DB anyway.

</comment><comment><USER25> The open source ory ecosystem ( [http://github.com/ory/](http://github.com/ory/) ) might have what you're looking for, but it's definitely for advanced usecases. I know a lot of people that worked with Auth0/Okta/AWS Cognito but got so frustrated by downtimes, bugs, and complexity that they moved away. But it is an option for rapid prototyping although I'd keep a "replace it" somewhere in my milestone planning. Another possibility is Keycloak which is very enterprise / java fullstack and quite complex to understand.

Most advice in the comments is pretty bad though. Stuff like "API Clients need bearer tokens" is completely backwards and pushed by marketing people from companies (Auth0, Okta, ...) that misuse open protocols (OAuth2, OIDC) as a way to legitimize the closed source saas approach they took. Along the lines "if it looks complex it looks secure because most people have no idea". It's actually very easy to use cookies (httpOnly, secure) with API clients and you're saving yourself so much complexity with refreshing tokens and all that stuff.

Yet another possibility for super rapid prototyping is: [https://github.com/bitly/oauth2_proxy](https://github.com/bitly/oauth2_proxy)

edit:// I forgot KeyCloak, but it's also for advanced enterprise use cases (SAML, OIDC, Realms, ...) and (from what I've heard) with a steep learning curve and heavy.

<comment><USER19> I think the distinction is that, if you intend to have a publicly accessible API, tokens are preferred vs cookies. For your own mobile clients, doesn't matter

<comment><USER26> Why are tokens preferred over cookies for a publicly accessible API?

<comment><USER19> There are all sorts of cases where managing cookies is annoying when interacting with an API, like via curl. There might be other reasons as well, but making consumption easy is probably reason enough

</comment></comment></comment></comment><comment><USER27> "This product is going to need to be secure as it will be in the healthcare space (so think oidc)."

Considering most EMRs (or at least the ones with which I've interacted) don't go very far beyond a username/password combo, you're probably fine keeping things simple.

Generally, when in doubt, I'd strongly recommend using some existing auth library instead of trying to roll your own. It's not clear where exactly in the healthcare world your site will fit, but if you're aiming for hospitals, any hospital worth their salt is going to be using Active Directory or something similar, so you'll probably want to find something that can support offloading user identification in that direction (and fall back to username/password if the org doesn't yet have AD).

I don't know your specific jurisdiction, but at least in the US, as long as you're encrypting all your data (both at-rest and in-transit) and aren't doing anything egregiously stupid (plaintext passwords, single shared password for everyone, literally selling patient data on the Dark Web, etc.) you should have a pretty hard time violating HIPAA, and you'll already be on-par with most extant medical systems. Any further hardening on the authentication front (e.g. specific session management strategies) will just be icing on the security cake.

If you haven't already, I'd suggest reviewing NIST's guidelines for system security; most official HIPAA reference materials point toward NIST guidelines, and most hospitals will tend toward that direction as well.

</comment><comment><USER28> If your plan is to connect other services then I'd suggest using LDAP for central authentication.

It can easily be connected to any API without much "glue". And most common open source services already support it as auth backend.

It's also easier to audit than any custom service you might concoct on your own because auditors already have experience with it through Active Directory.

<comment><USER29> LDAP is really overlooked by many and I think people are surprised by the amount of software that offers LDAP support.

<comment><USER30> i've stopped being surprised a few years ago that it's usually in the 'call us' price tier :)

</comment></comment></comment><comment> shocked-pikachu I have been using JWT[1] since it's initial push as a mainstream authentication method. There's plenty of libraries that support it and make it easier to use[2]. There's even 0Auth[3] that is a company based around offering authentication services. Simply check for a valid token in the Authorization header, check out the introduction [4].

The library I've linked brings up a good point to make sure that you know the difference between decoding and verifying the token, but after that it's fairly plain sailing.

[1]: [https://jwt.io](https://jwt.io) [2]: [https://github.com/auth0/node- jsonwebtoken](https://github.com/auth0/node-jsonwebtoken) [3]: [https://auth0.com](https://auth0.com) [4]: [https://jwt.io/introduction/](https://jwt.io/introduction/)

</comment><comment><USER31> We use a local version of UAA ([https://docs.cloudfoundry.org/concepts/architecture/uaa.html](https://docs.cloudfoundry.org/concepts/architecture/uaa.html)) he behaves like a google or facebook authentication where the user concedes the permission to the apps using OAuth 2.0 protocol.

We have 10+ web clients, 2 mobile apps and one internet gateway. Spring Security make it very easy to integrate everything.

[https://oauth.net/2/](https://oauth.net/2/) Auth0 is the best documentation to learn about oauth: [https://auth0.com/docs/protocols/oauth2](https://auth0.com/docs/protocols/oauth2)

The best part of using Oauth is that you can change your authentication server (UAA, Auth0, etc) without changing your app code (only configurations)

</comment><comment><USER32> If I do a MVP with React.js, my go to solution is Firebase. [0] Authorization is possible as well. [1] If I need to scale this MVP, I would eventually migrate to a self-hosted backend solution. In my case, it would be Node.js with a GraphQL interface that enables authentication with JWT [2]. Alternatives could be Passport.js and Auth0.

\- [0] [https://www.robinwieruch.de/complete-firebase- authentication...](https://www.robinwieruch.de/complete-firebase- authentication-react-tutorial/)

\- [1] [https://www.robinwieruch.de/react-firebase-authorization- rol...](https://www.robinwieruch.de/react-firebase-authorization-roles- permissions/)

\- [2] [https://www.robinwieruch.de/graphql-apollo-server- tutorial/](https://www.robinwieruch.de/graphql-apollo-server-tutorial/)

</comment><comment><USER33> I'd advocate against using third party services here. In general third party services and healthcare comes with a lot of hurdles and costs which you need to evaluate carefully. Even log data stored on a 3rd party server that might have some PII can be a violation of U.S. law unless they are a contracted BAA following all the same security and legal rules required of healthcare tech.

Overall just start with strong security in mind that will meet the U.S. healthcare security rules/laws, even if you aren't in the U.S., the basic principles are just focused around strong security. People can debate the specific methods, but I will argue using almost any third party service has potential problems for you with compliance. Yes, I agree and understand those services specialize around auth usually, and for most companies that is fine and even for some more fringe areas of healthcare that is fine. But take another viewpoint, that is because those companies secure so many disparate third parties their attack surface is huge compared to your own, and a vulnerability at their end may force you to do public disclosures. That alone isn't a sole reason not to use them, but do consider all the factors.

Also, in the U.S. you will likely (depending on specific type of product) need to deal with HIPAA and other similar acts (HITECH/TRUST etc). None of these are actually all that complicated if you take them into account early, although going back and adding them later can be a struggle.

Basic principles is secure everything, have timed (short interval) token expiration, have a global token expunge, and always err on the side of reauth over pass thru. Also, if you have many backend services, do not rely on a proxy authentication service to pass off requests. Force all services to validate the authentication of each request. Yes this is "expensive" in terms of extra cycles but it minimizes the risk considerably. Lastly, store trace and audit logs of everything you can imagine, all requests.

</comment><comment><USER34> Check out [https://github.com/sysgears/apollo-universal-starter- kit](https://github.com/sysgears/apollo-universal-starter-kit)

Fullstack JS, Apollo GraphQL, numerous client types, batteries included. Has auth implemented plus a number of other things you will find useful in the graphql world.

If you are in healthcare, you may want to find an auth system like DEX that can talk to several backends like LDAP & Active Directory, as each hospital or institution will likely want to reuse what they currently have implemented. You'll need to hook into them. (This is on our roadmap)

</comment><comment><USER35> I'm using a selfmade system atm as as excuse to learn the subject for a MEAN app, but I would use OKTA or some similar service if I was 'playing for keeps.' Just boring client side sessions using Mozilla's client-sessions library with bcrypt for passwords.

I'm finding it hard to get information above the 'follow these steps to use this library' level, yet beneath 'here's how to make a cryptosystem from scratch'. I'm about to read some RFCs unless I find a better intro resource.

</comment><comment><USER36> All new apps I'm building are based on password-less auth [1] & bearer token. NoPassword is just so much more comfortable than having to remember passwords. And boy, people are really bad at coming up with passwords - you might have seen it this Christmas at your parents'. [1] [http://notes.xoxco.com/post/27999787765/is-it-time-for- passw...](http://notes.xoxco.com/post/27999787765/is-it-time-for-password- less-login)

<comment><USER37> This works only on devices for which you simultaneously have email access. For instance Netflix on a home gaming console would fail miserably using this schema. See also [https://www.troyhunt.com/heres-why-insert-thing-here-is- not-...](https://www.troyhunt.com/heres-why-insert-thing-here-is-not-a- password-killer/)

</comment></comment><comment><USER38> Truth about side-projects is most never get to the finish line. I'd like to encourage you to get there, and for that, you'll need to use a third party for auth, at least till you finish your API and seek feedback/scale. You can always roll your own after you've validated the viability of your API.

You can choose firebase or AWS Cognito. They offer readymade sever and client SDKs for a wide variety of targets. Firebase is always free, and AWS free for 50,000 users

They offer authorization based on roles

</comment><comment><USER39> Roll my own. Passwords are stored as bcrypt hashes. Just use plain old cookies to store session IDs.

<comment><USER14> What about localstorage for storing a token instead of a cookie?

<comment><USER40> what about caring about things that matter?

</comment></comment></comment><comment><USER41> Devise, if you're building a Rails app.

</comment><comment><USER42> Why do all API development tutorials always say "Never roll out your own custom auth"? I always found it weird

<comment><USER43> What they said..

\- Your authentication problems are _not_ unique to you;

\- The effort of implementing standards (whether it's front end like OAuth, OIDC, or SAML or back end like hashing) is a pain in the butt and easy to make bad choices;

\- If your project is successful or as your requirements change over time, now you have to figure out how to add MFA, password resets, internationalization, address security audits, etc, etc.

Doing it yourself means you have 100% responsibility for everything when that is probably not your main skillset or really what you want to spend your time doing anyway.

 _Disclosure: I work for one of the companies mentioned in this thread._

<comment><USER42> Hey man, funny thing, I just completed your 3 courses on Lynda.com on the REST API learning path yesterday. I did the Design one, the Validation and Authentication one and the OAuth/OpenID one.

Good stuff.

Also, I have a question for you, is there a good place to reach you?

<comment><USER43> Thanks and great to hear. My email is in my profile. Feel free to drop me a note.

</comment></comment></comment></comment><comment><USER44> We use SAML set up for our customers' identity providers (most of the time, Office365 Azure AD or Google Suite), from a NuGet package.

From there, it's a bearer token encoded in JWT format for ease of debugging. From another NuGet.

We also support username and public key authentication for our SFTP server.

We do support username and password if necessary.

<comment><USER30> and what do you use for authorization?

<comment><USER44> Custom rules on top of ASP.NET middleware.

</comment></comment></comment><comment><USER45> We use Truedentity ([https://www.truedentity.de](https://www.truedentity.de)) which is pretty robust with smartcard and palm vein pattern to make access secure. It uses EJBCA as backend for PKI and it's possible to configure redundancy with it.

</comment><comment><USER46> Ignoring the body of your question and answering just the title, I'm developing [https://accountd.xyz/](https://accountd.xyz/) as a system for mixing and linking identities (email, twitter, github, trello, indieauth for now) and using that for [https://trackingco.de/](https://trackingco.de/), [https://listhub.xyz/](https://listhub.xyz/) and [https://sitios.xyz/](https://sitios.xyz/).

In theory anyone can use it, it works automatically and you don't have to register or anything like that, but I don't know if people are interested in it.

</comment><comment><USER47> I wonder why macaroons [1] are not even mentioned in this discussion yet.

They have all the upsides of cookies, but also can be narrowed down to be handed to third parties (good for APIs), caveats, and have a standardized and implemented [2] verification scheme.

I wonder why they don't see wider use. Do they have significant downsides?

[1]: [http://hackingdistributed.com/2014/05/16/macaroons-are- bette...](http://hackingdistributed.com/2014/05/16/macaroons-are-better-than- cookies/)

[2]: [https://github.com/rescrv/libmacaroons](https://github.com/rescrv/libmacaroons)

<comment><USER48> Macaroons have their own issues, see my previous comment here: [https://news.ycombinator.com/item?id=17879403](https://news.ycombinator.com/item?id=17879403)

If you've got questions about them I'm happy to answer.

</comment></comment><comment><USER49> I'm mostly a backend developer (batch processing, systems integration etc) and it still amazes me there's no turney solution for different languages. I work in a large enterprise so we hook into our SSO for any kind of Auth for our web apps, though I haven't personally had to deal with this since my college days. If I remmebr.NET had something out of the box.

Anyways, when I dabbled a few years back, I uses stormpath but they closed down. Les Hazelwood, the creator or Apache Shiro works at Okta now... Which seems to provide enterprise security.

Can anyone comment on their experience with Shiro, or Okta in general, and if it would help OP.

</comment><comment><USER50> U2F + a password or biometric.

What I wish is Apple and co would give the consumer choices. I am a low threat target; I don't need to enter a pw to unlock the SIP. So presenting a fingerprint + a hardware key would be a huge security improvement and a reasonable defense against 99.9999999% of the threats I face. Unfortunately, Apple assumes we're all being targeting by physical attacks from the NSA, decreasing my convenience level for a non-existant threat.

</comment><comment><USER51> [https://www.apereo.org/projects/cas](https://www.apereo.org/projects/cas)

[https://www.freeipa.org/page/Main_Page](https://www.freeipa.org/page/Main_Page)

[https://www.rcdevs.com/](https://www.rcdevs.com/)

</comment><comment><USER52> I'm in the same boat (SaaS app in the health sector) and opted for Auth0 after using it a fair bit in the finance sector with work. I'd say start with that and once your app outgrows it then you can roll out a solution of your own but I'd suggest you don't try to reinvent that particular wheel when you're starting out.

</comment><comment><USER53> Auth0 for authentication, custom authorization.

</comment><comment><USER54> I use meteor and graphql with accounts package and Alan's: roles packagr. It's pretty easy using meteor.

</comment><comment><USER55> We did the bakeoff and AWS Cognito is the by far the cheapest. It's not as well documented but supports Open Id Connect just like all the other 3rd party auth solutions. The downside is that Cognito does not support a Resource Owner Password Flow (not recommended anyhow. see RFC6749).

</comment><comment><USER56> Echoing the others telling you to hire a professional. If you have the money Okta has a professional services division that will set everything up for you. Use them. If you don't have the money us Auth.0. Don't try to handroll everything you will regret it.

</comment><comment><USER57> If you have the budget for it just use auth0 or similar. You will likely spend less time wiring it up than coming up with a solution yourself.

Going off your question only, this will also likely end up being the most secure implementation for you (relying on a third party service)

</comment><comment><USER58> > healthcare

> OIDC

Nooope. Not a good idea. OIDC is pretty damn complicated to implement as a server. And it doesn't help at all with anything around revocation etc. To make that possible, you have to add a load of extra work and you basically lose most of the benefits of using JWTs.

First, as others have said, if you don't know what you're doing on this, you have no business trying to secure health data. Bring in someone who does know what they're doing and then pentest it aggressively.

Second, I'd suggest you'd be better keeping it super simple. Just have a token in a table, refer to the token by its ID and then attach a 32 byte crypto random to it which gets checked before the existence of the token is acknowledged. Compare it with constant timing. If you don't know what that is, again, you shouldn't be doing it with health data - best to learn on a project with less sensitive info.

</comment><comment><USER59> imho github.com/casbin/casbin is the go to solution when it comes to authorization, it supports ACL, RBAC, ABAC, it's optimized for performance, it's pretty language agnostic, backed by tencent

</comment><comment><USER60> You can use meteor's healthcare track, specifically designed for healthcare apps: [http://clinical.meteorapp.com](http://clinical.meteorapp.com)

</comment><comment><USER61> All our users have client certs, so using then makes sense to me.

</comment><comment><USER62> I have heard of keycloak , open source auth platform

any reviews ?

<comment><USER63> Currently integrating Keycloak as auth service for multiple existing projects. It's very advanced, many use cases and configurations are possible. We are in the middle of the migration but so far it hasn't been hard to integrate.

It's a rather large dependency so I won't recommend it for a single project with straight forward auth requirements.

</comment></comment><comment><USER64> There are seriously dangerous attitudes in this thread regarding what constitutes token invalidation. Hire someone skilled at secure auth architecture. You should have at least 2-3 senior engineers who are very well versed in this if this company is serious about doing anything related to protected patient healthcare data. One person's opinion should never end up being the only input for super critical decisions like you are trying to make.

</comment><comment><USER65> Well, we build and use Scatter ([http://get-scatter.com](http://get- scatter.com)), which is both a desktop app and a JS and C# library. This allows you to use the EOS or Tron blockchains and their account systems. Our users have local access to remote accounts on those networks. It's pretty amazing software, and it is really fun building something totally new.

</comment><comment><USER66> Using anything other than your own libs for your specific needs probably will lead to over engineering.

</comment><comment><USER67> I'm using Keycloak for authentication and Vert.x for aauthorization besides the main use case.

</comment><comment><USER68> in the healthcare space really helps. great detail for this question. honestly, not many people are good at asking questions.

you need an expert, someone that doesn't need to ask this question.

</comment><comment><USER69> Okta.

</comment><comment><USER70> Email or text message sent to customer and they just input the number they were sent to match.

<comment><USER71> This is called two factor auth. SMS is not considered a secure method of transport (mostly due to porting). It _only_ solves opportunistic password compromise via password dumps.

<comment><USER72> In reality it solves far more than that for the average system for the average user. No on in reality is having their mobile number socially engineered to get into startup xs system. It's a good place to start until you are at a scale where you would have dedicated security engineers to work on the problem.

<comment><USER73> > No on in reality is having their mobile number socially engineered to get > into startup xs system.

The problem is that you can't know whether that's the case at start, at some point in the future, or never. You'll only find out that your guess was wrong when you're breached which is never a good time for any service, particularly one covered by HIPAA. Besides, why intentionally implement a known-to-be- insecure second factor method? This is brand new code; there's no need to incur technical debt from day one. Which leads to:

> It's a good place to start until you are at a scale where you would have > dedicated security engineers to work on the problem.

Except that day never comes for a variety of reasons. "Users already expect it so we can't remove that." "There are several dozen other security audits/features/fixes we need to make, let's prioritize those first." "What? SMS is insecure? Weird, never knew that."

Also, if we don't start making the decision now, before the second factor is ever implemented, to move away from using SMS as that second factor for the reasons it is known to be broken, when do we start?

</comment></comment></comment></comment><comment><USER23> Once we lost the user account DB at a startup to database data loss and some other day, someone logged into mongodb and dropped all the tables and backups were not test before, so they did not work.

After that we no longer store these details.

We've been using amazon cognito with lambda authorizers.

<comment><USER74> That sounds like the problem is how you keep your database permissions and backups, not anything to do with auth.</comment></comment>