Show HN: Axiom – No-code Browser Automation - <AUTHOR> https://axiom.ai/ <comment><AUTHOR> Hi all,

We built a way for non-technical people to automate work in their browser. It's in beta, and we've just made it public: [https://axiom.ai/](https://axiom.ai/).

After running a consultancy, we noticed lots of small tasks that never got automated. The cost was too high with existing consultancy-focused solutions. So lots of small repetitive tasks built up.

Recently though, RPA has become a popular technology for automating big, repetitive processes.

RPA (Robotic Process Automation) is a way to automate using the user interface of applications, rather than APIs. If you've ever used an Excel or Emacs macro, it's the same principle. Companies like UiPath and Automation anywhere have had great success deploying RPA for enterprise use-cases (e.g. processing 1000s of invoices for a multinational)

But we believe RPA also has potential as a no-code paradigm for smaller use- cases (e.g a customer support person doing data-entry).

That's because most people intuitively understand UIs, but not APIs.

So we thought, what would happen if you took RPA technology and packaged it up in a no-code Zapier-like SaaS? This way, all the small tasks too small for developer time can get automated.

Our current users are mostly non-technical and in roles like sales or e-commerce administration. They have repetitive extract, transform, load (ETL) tasks, like:

-Gathering data for sales enrichment, then inputting into their CRM

-Connecting and exporting data from e-commerce systems like Amazon and Shopify, where APIs are limited.

UI-based automation is not just a way to fill gaps in APIs. We've found it is a more intuitive and faster way for many non-technical people to automate.

Along the way, we've also discovered many new niches (like sports betting funds) that we didn't know existed.

Axiom is not built for large-scale data-scraping or automated testing, though. These require specific features we do not yet support (e.g. IP switching, or assertion logic).

We don't touch your data. All data-processing and execution occurs client- side, on your machine. We only store the code for execution. For data storage, we use your Google drive account.

Our beta only supports Google Chrome (this is just an MVP. We want to expand support to FireFox and other browsers with time and resources).

I'm sure the HN audience has seen how challenging browser automation can be, particularly with the complexity of modern JS-heavy web-apps. We'd love your feedback, experience and thoughts!

For example:

Some users have said the permissions being requested by our app are off- putting. We're listening and adapting. We're removing permissions + improving communication. To reiterate, we do not store any data processed by bots on your machine: [https://axiom.ai/privacy-policy](https://axiom.ai/privacy- policy)

</comment><comment><USER2> As someone who implemented and runs RPA in the public sector I would be interested if you made a couple of things come true.

No one offers an easy way to let employees build and share "personal" bots. Part of this has to do with non-programmers having a much harder time understanding things like loops than we imagined they would. Another problem is maintaining business process logic and knowledge as employees come and go. And lastly there is the security thing. In an enterprise setting you probably won't want to run this in someone else's cloud, you want to run it through your IT operations in your own onsite/cloud/whatever to both utilise your local accessing rights but also to make sure data never leaves you.

Maybe we're not who you are targeting, but we've yet to find a product that actually lets us let employees build small bots in a way that fits into our IT operations, and this is quite a big market in my country.

Of course you'll be late to the table, and not be what the big consultant agencies like EY are partnered with. But what they are currently selling isn't actually what we need.

So good luck, I'll certainly add you to our "keep a look out" list.

<comment><USER3> Virvar - out of curiosity - why is it that you would like to let the users build those bots? Intuitively, users just don't have the skills, patience, the technical mindset, the understanding of the IT landscape (imagine some non- working SSO - which user is going to know how to get that to work!). Basically "what's wrong" with doing it 'the normal way' \- which is to have an IT team (in-house or external like EY) do the work?

<comment><USER2> We operate 300ish IT systems of various size, from handling vacation time between leaders/employees to full fledged SAP solutions. We have 10,000 employees but only 5 developers of which 2 are also operations engineers developers. If an employee needs a little bit to automate some small workflow, to save them a few hours a week, that's way below the scope of where we would get involved to do an RPA process. But that doesn't mean we don't want people to automate those tasks. In the ideal world we wouldn't need RPA, because much of what it does is things that should work smarter in our current systems. But the enterprise world is far from ideal.

We didn't use one of the big consultant houses like EY. We didn't because we have plenty of business process people, project managers and so on ourselves. And that's typically what the big consultants mainly offer, they offer it along with tech consultants that are often from some partner. We started by gathering info on what was available, as well as what we thought we needed and we drew on experiences from other cities. Many places had gone with the big consultant agencies and gotten this big package on the business end, and one or two processes build with the tech consultant, and then their project stranded because the business part doesn't actually build RPA processes.

We decided to go with a small local startup, where we bought hours and "open" consulting. We did brainstorms with them, but then we build things ourselves and had them review it, and we put a much larger focus on learning the technical parts - and well - we are now the leading city on this area if you compare the 96 cities that aren't our two largest cities. So from my anecdotal view, that is far better than using the big consultant agencies.

I named EY, and that may give them bad publicity here, but EY was actually the best of the big package offers that we didn't decide to go with. Which is why their name stuck with me, so it's a little unfair if this makes them look bad.

<comment><USER3> Thanks Virvar, may I follow up (and I am really just trying to understand here, not to criticise or advocate one way or another): From what you said, I understand that, essentially, 'the kind of steps or tasks that employees would want automated' are too small for someone central to bother to look at. So it's an issue of economics - it's not worth the attention and not worth the cost. I fully understand.

On the flipside, you yourself mentioned a) the skill level (loops) and b) maintenance and - dare I add - say governance / standards.

I guess it comes down to the tradeoff of [not having tasks automated because it's not worth it for 'the experts'] vs [having a prolific ungoverned set of automations deployed by users with insufficient skill level perform them (kind of like excel macros)].

So given the skill issue, and given that users struggle with things like loops etc. - does that mean that they'll basically just be able to implement 'trivial automations that don't involve complex paradigms'? Or how can non- technical people, fundamentally, crack it and develop more elaborate (and therefore more powerful) automations?

This is what I'm grappling with - I see so many no code tools out there, but at the end of the day, you can only do very limited, not so valuable, automations with them. Curious to learn your thoughts there.

<comment><USER2> This is the issue. The "no-code" + "no real oversight" tools don't really work for us, but we wish they did. ;)

<comment><USER3> Alright -thanks for the clarification!

</comment></comment></comment></comment></comment></comment><comment><USER4> Happy Axiom customer here. Its main selling point: absolute ease of use. Like a point-and-click selenium. Our analysts , who are doing quick analytics or analytics without the need for a long-term engineered solution, are very satisfied with it. I find it powerful enough for production use (it works with IG and even FB to some extent).

Only downside I can think of: pricing is a bit opaque, outside the "desktop" license.

<comment><AUTHOR> Hey, thanks!

Yeah, so pricing is opaque because in all honestly, we're still figuring things out.

We're unlikely to lift pricing and annoy users. If anything, we're extending the free trial for most users, and may introduce a cheaper tier too.

<comment><USER5> Cheaper tier would be great. I can think of some things I could use this for that are not 50$/mo things. Scraping personal finance, automating personal blog etc. but I definitely understand a startup needing to figure out pricing

<comment><AUTHOR> Cool, by way of a quick survey on HN, how much would you pay in a cheaper tier?

<comment><USER6> You're not asking me, but if home users are a target market then it would have to be $1 - $3 dollars per month. I could imagine scraping local events websites, banks, maybe online auctions, etc. When home users subscribed to one or two services, we could get away with $10 monthly. Now, with home users accustomed to $1 apps and subscribing to a dozen different services, $3 is about the maximum before signups drop significantly.

<comment><USER7> I would say it is impossible to make any business model work charging $1 to $3 per month.

<comment><USER8> $3/mo is $36/year, most templates on themeforest and other platforms are around that price and they are selling fine. Considering the code is running on the user's side, the marginal cost for low-tier customers is probably minimal.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> Hi, interesting. Does this run in the browser or on the backend?

"All your bots live on you computer and process data in your web browser. We store the steps of your bot and data"

It seems the scripts are stored on your servers? Why? Does this mean that you can see my code or is it encrypted?

How does this compare to the more established web macro recorder tools (browser extensions) like the open-source kantu? [https://github.com/A9T9/Kantu](https://github.com/A9T9/Kantu)

<comment><AUTHOR> Good questions!

This currently runs only on your local machine. We do have a cloud version coming soon, but running your bots this way will be optional.

Yes, the code only is stored, not the data it processes. We could encrypt them so we can't read, but it would prevent us updating your scripts in future for new versions of axiom.

It does mean we could run your code, but this is not something we do, unless you want us to run your script in our cloud.

UI.Vision and long-established macro-recorders like iMacros target developers; this is a complete no-code approach.

Our tool is primarily built for non-technical people. We also have and are building up a bot sharing system (i.e. templated bot app store). It will be subject to manual vetting for quite a while.

<comment><USER10> In a previous company we were scraping customers web pages to load customer and product information because the number of systems and teams involved to connect to their CDP would take months to get up and running. A few problems we had to solve was the web pages changing underneath us and some pages not being formatted the same. We ended up having to detect those changes server- side to alert us to update the scraper. How do you all deal with these use- cases?

Also, we tried to leverage: [https://www.diffbot.com/](https://www.diffbot.com/) but the lack-of- accuracy/lack-of-complete-data + cost never justified it's usage.

<comment><AUTHOR> Yes, very good question (I've answered a similar one on detecting whether the page has finished loading).

It's a deceptively hard problem. Essentially, what we do is fingerprint the element. If the page changes, it boils down to how effective our fingerprinting and search algorithms are, to find the element if it has moved or changed.

The algorithms behind that are good enough for most use-cases now, but it's something we're continuously iterating on.

</comment></comment></comment></comment><comment><USER11> I think for anyone who has ever tried to do significant scraping, or even testing, with something like Puppeteer, you can easily see what services like this should exist. The price is surely targeted towards businesses, and I think that's a smart move.

</comment><comment> dan-robertson I saw an old demo from Microsoft research in their programming by example group. The idea was to figure out how to extract (json-shaped) data from websites given a few examples of where that data was by the user. The video on this site reminded me of that. I think this presents a better interface than MSR and by restricting to tabular structures of data (as far as I can tell), I think the system is easier to understand and likely more reliable.

<comment><AUTHOR> We do indeed restrict ourselves to tabular structures of data, both on the web-page level and outside.

Many non-technical people think in terms of spreadsheets and tables. If you give people the spreadsheet/table as the data structure primitive through which you're iterating, they get the concept of a loop straight away.

Couple that concept with teaching them a variable, and they can achieve a lot.

<comment> dan-robertson I think a tabular model is probably a good idea

</comment></comment></comment><comment><USER12> Looks like a great tool, but $50/month is pretty steep especially when I'm the provider of the computing power. Can you explain the rationale behind this price?

<comment><AUTHOR> Yes, we decided to start high, and introduce lower tiers after, because lowering your prices doesn't upset people, but putting them up does.

Many of our users are expecting to pay $50 p/m, but will get a present surprise of a cheaper tier this month, rather than the other way around.

<comment><USER13> AS someone who is the core demographic that will use this: if this works the way it says it does, it could cost $100 and I would still expense that to my company without blinking.

</comment></comment></comment><comment><USER14> I really like the concept, having personally struggled with browser automation scripting in the past for business and personal use.

However, I'm concerned about the privacy and security implications of storing users' automation scripts in the cloud. What if I want to automate processes on sites or applications which contain sensitive data?

In other responses I think you've confirmed that scraped data itself is never sent to your servers, but even the script and its metadata (e.g. names of fields, search terms, IDs of elements on the page etc.) can contain sensitive information.

In several of my clients' environments the use of this tool simply wouldn't be allowed because no third party data is allowed to leave the internal network.

Is there a way to use the product without sending _any_ data whatsoever to your servers?

Also, does this work with browsers other than Chrome?

<comment><AUTHOR> This currently only works with Chrome (still an MVP). We would eventually like to expand support to other browsers with future iterations.

RE: Data leaving the network.

This is a very good point.

If you have a strong business-case you'd like us to look at, but find privacy is the main issue, e-mail me at yaseer AT axiom dot ai; we may be able to investigate what on-prem would look like.

</comment></comment><comment><USER15> > Axiom is quick to use; just point and click, with no need for API > integrations or complicated code. Get bots running fast and spend more time > on the important stuff.

Great and it gets to the point for the end users. But..

> Step 2 of 2 - Install the desktop application.

> 168 MB for the DMG, 427 MB for the app.

I get that this is your first release but is it possible that the Chrome extension can be used on its own? If not is it possible to reduce the size of this app? I say this because my impression after opening this app is that it will be running in the background which will take up more of my Macbook's RAM given that it is Electron.

On top of that, I already have 11 other Electron apps installed and this one is twice as big as three other electron apps I already have and my disk is already complaining about freeing up space.

<comment><AUTHOR> Yes, we definitely want to eliminate the electron app. I agree they are bloated and annoying.

The first route for this is running your bots in the cloud, although that will mean we have to process your data (something we don't currently do).

We do want to release a stripped down version that can run with the extension alone, however, this won't support many of the most common use-cases for axiom, like bulk downloading and uploading files.

</comment></comment><comment><USER16> Sounds like something similar to the PawClaws project:

[https://chrome.google.com/webstore/detail/pawclaws/figeaallb...](https://chrome.google.com/webstore/detail/pawclaws/figeaallbafpdefmmkfipkgdgpknmhfl?hl=en)

[https://github.com/TobiasKundig/PawClaws](https://github.com/TobiasKundig/PawClaws)

[https://github.com/PawClaws/chromePawExt](https://github.com/PawClaws/chromePawExt)

I wonder though: How do you deal with asynchronous sites which may take a variable amount of time for elements to load? Do you synchronize with Angular or try to detect elements or is it just the old stopwatch-and-pray technique?

<comment><AUTHOR> Very, very good question. The long-story short is that I don't think there's an algorithm that can determine the completion of page-load with certainty for complex JS SPAs.

We fingerprint elements, check if the page is changing along the critical path and wait if it is, and look at network traffic.

We can't say with certainty, but we can form a good guess - the accuracy of that guess is always improving.

This is something I think would benefit from open-source collaboration, if we went down that route.

</comment></comment><comment><USER17> Web scrapping is a huge problem and I am happy to seem more usable solution but why do you need these permissions from me:

See, edit, create, and delete ALL of your Google Drive files

See, edit, create, and delete ALL your spreadsheets in Google Drive

<comment><USER18> Google Drive and Google Calendar permissions are not that great. IIRC there is no distinction between individual CRUD actions, so if the app wants to create a file they also need perms to edit and delete.

</comment></comment><comment><USER19> hey @yaseer, Really cool idea, thanks for sharing! RPA is a great, underserved niche for the SMBs.

I've read through your privacy policy but i'd like some more clarification.

specifically, can you talk about exactly what data is sent to your servers? For example, if i use Axiom to login to a vendor site, those log on credentials should be stored only locally correct? Does your team have the ability to access any of the data that would be retrieved from the vendor application? I have a few really solid use cases but I need to cover the data privacy concerns first.

<comment><AUTHOR> Yep, of course.

That's right; Right now, we don't store credentials.

The cookie from your local machine when logged in is passed to axiom locally, and never sent to us.

We store the code for your script on our servers. This is necessary to update scripts for future versions of axiom. We do not store the data this code processes.

We do have a cloud version upcoming that will process data on our servers, but this will be optional, and not enforced.

<comment><USER19> Thanks, for example, you would be able to see the steps of the automation, and the site (full URL) where the automation is targeted, the name of the fields being captured during scraping, and you might receive some data about the success of the step/process. But you do not receive any feedback about whats gathered or returned to the desktop.

Cloud version could be cool for some people but i think it might kill any shot at the industry I serve.

<comment><AUTHOR> Yes, that's exactly right - we would see the URLs, the names of fields (from the DOM), and success states.

Just not the data itself. It's basically what would be stored in a web- scraping or automated testing script.

</comment></comment></comment></comment><comment><USER20> I could see this being useful for tracking COVID data on government websites. Ontario only posts the current days stats for 24hrs until the next day. You could extract it into a spreadsheet with this.

I'm also curious if there are some use-cases for tracking releases of new content on various sites, similar to how CouchPotato combines TV show releases and tracking them on torrent sites.

<comment><AUTHOR> We've thought about this too! It's not really effective with our desktop version.

The cloud version should enable it though.

One of the most common things people want to do is create a series of bots than consolidates data and generates reports; it has general applicability for use-cases outside COVID.

</comment></comment><comment><USER21> The promo video includes "Click Social Media Buttons: This bot only clicks on the social media button if it has not already been liked or followed" [vid]. I can't think of the intended use case; how do you envision this might be used?

[vid]: [https://youtu.be/D3dqsZ1fyCU?t=112](https://youtu.be/D3dqsZ1fyCU?t=112)

<comment><AUTHOR> In private beta, a lot of users were using axiom to automate their likes and follows, to build up an audience on Instagram. They were doing this work manually in any case.

We would not want this to be done at scale, in a cloud, like a nefarious bot- farm. However, at the scale of a bot tied to a single individual's browser, doing repetitive work in growth-marketing and sales, we'd allow this usage.

<comment><USER22> Isn't this a violation of Instagram's ToS?

<comment><AUTHOR> [https://help.instagram.com/1188470931252371](https://help.instagram.com/1188470931252371)

Creating user accounts with bots, and crawling certainly is, but I can't see anything about automating likes (correct me if I'm wrong).

There are many chrome extensions which do specifically automate likes on the Chrome store. Ours is just a chrome extension for general browser automation, where this is just one use-case.

<comment><USER23> "We prohibit crawling, scraping, caching or _otherwise accessing any content_ on the Service via automated means"

...

"You must not create or submit unwanted email, comments, likes or other forms of commercial or harassing communications (a/k/a "spam") to any Instagram users."

<comment><AUTHOR> Okay, I agree this a grey and perhaps questionable area.

This seemed to be something legitimate people/users were doing a lot, so we gave them a building block to make it easier. Maybe we should have thought about it more.

We'll take a step back from that path if our bots become a nuisance on social media. In all honesty, we built axiom with other things in mind (e.g. Repetitive data entry and admin).

<comment><USER24> It's not grey it's a TOS violation. Facebook goes after these types of bots ruthlessly, especially in election season. I'd recommend not poking that bear, you have so many market opportunities!

<comment><AUTHOR> Okay, we'll take this on board!

It was honestly just some user requests we integrated without enough thought, not something nefarious. This isn't our market by any means.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER25> Why is your product called Axiom "AI"? I'm struggling to see any actual AI baked into the product.

<comment><USER26> I had the exact same thought. I would suggest to take out the .ai because it IMO makes you look disingenuous. If your name is already clear marketing BS, then someone may extrapolate that to the rest of your product.

That happened to me, you already lost credibility with me by naming it .ai because, in my mind, you either 1. Have no idea what AI means, or 2. You know very well but you are lying.

I'm not saying either of those are true, but that's what was going through my mind when I first opened the website.

<comment><USER27> I really don't see what the problem is with the choice of domain name.

Looks like a useful tool that I'll definitely have a play with.

</comment></comment></comment><comment><USER28> Gonna check this tomorrow. Lately I had big pleasure using nbrowser node in node red [https://flows.nodered.org/node/node-red-contrib- nbrowser](https://flows.nodered.org/node/node-red-contrib-nbrowser)

</comment><comment><USER29> "we do not store any data processed by bots on your machine" is that right?

<comment><AUTHOR> Yes, 100%.

All bots run on your local machine, we only store the code for execution, not the data it processes.

We've taken steps to be classified as a GDPR-compliant data-processor.

We use your browsers local storage for many operations that other applications store on their server.

<comment><USER30> The sentence is misleading. I think you meant to say -

"All data is stored locally on your machine and not on our server".

<comment><AUTHOR> Ah yes, I've re-read the sentence, this is what I meant to say!

</comment></comment></comment></comment><comment><USER31> Really exciting to see this! My friend and I are also working on a very similar open-source SaaS offering, Puppet [1], so much so that we use the same "no code browser automation" words to describe our service.

When we launch in the coming weeks, I'll be sure to post on HN, but in a nutshell, it's entirely cloud-based (as opposed to having to downloading something), open-source, and at a price point closer to ~$5/m.

Of course, Axios is a far superior product with many more features, but we're targeting individuals who want to automate small parts of their workflows rather than businesses who'd pay $50/m.

I'm glad to see your launch and I'm sure there's plenty of room to play. :)

[1] Very under construction landing page (all hash links) for the extra- curious: [https://puppet.js.org](https://puppet.js.org)

<comment><USER32> Congrats on nearing launch. You're being downvoted because it's socially unacceptable here to use a competitors thread to promote your service. Unless you are specifically mentioned by another commenter or you have a broader, vendor-agnostic insight to provide related to the service, you'll be better off discussing your service in your own submissions about it. Good luck with the launch!

<comment><USER31> Got it, thanks for letting me know!

</comment></comment></comment><comment><USER33> I see this is sponsored by SAP. Is it integrated with their existing RPA offerings?

<comment><AUTHOR> We're a startup from SAP.io's accelerator program, so quite a separate entity.

We received funding/mentorship from SAP via Techstars, but don't integrate with their RPA (right now at least...).

SAP's RPA is more focused on larger enterprise use-cases within their ecosystem, whereas we're looking at SME and even consumer use-cases.

</comment></comment><comment><USER34> How easy is it to customize data output? If I have a JSON Schema I want to conform to that requires collecting data from multiple linked pages, is it possible to do that?

<comment><AUTHOR> It is possible - but you'll need to write code.

Right now we have a custom code-block with a gloriously undocumented API.

I recommend playing around, seeing if you think axiom is useful for you, then dropping our support an e-mail on ai AT axiom.ai and we can talk you through the rest of your use-case. We're pretty responsive.

</comment></comment><comment><USER35> congrats! i saw that you submitted this 27 and 28 days ago. just wondering what you think made this post the successful one? titling? random chance?

<comment><USER36> I'll take this one! The short version is that I invited the repost and coached the founders the same way that I do for YC startups with their Launch HNs, about which see [https://news.ycombinator.com/launches](https://news.ycombinator.com/launches) and [https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...](https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=by%3Adang%20%22launch%20hn%22&sort=byDate&type=comment).

I will come back and add more explanation about what this means, and how it works, because the intention is to open this facility more up to the community.

Edit: here are the details. I helped edit the text with which they were appealing to HN. Then, when the post was ready, I put it in the second-chance pool ([https://news.ycombinator.com/item?id=11662380](https://news.ycombinator.com/item?id=11662380)), so it got a random placement on HN's front page. We're happy to do that for posts that we think will interest the community. If they don't interest the community, they fall off the front page soon enough.

The support I gave these guys was exactly the same as what we do for YC startups who want to launch on HN—with the exception of the front-page placement mechanism, which is different (see links above).

I've wanted to open this Launch HN process up to non-YC startups for quite some time, and have begun to do it experimentally. The goal is to produce threads that the community finds interesting, and also to help startups. The downside with this is that I'm the only person doing it, there is only one of me, and it doesn't have much spare time. If you want to be considered for this experiment, you're welcome to email firstname.lastname@example.com, but please have mercy if I don't reply right away or don't feel that it's a good fit. There's simply no possibility of supporting everybody, even though I would love to.

<comment><USER37> I'd love to hear more about this topic.

As someone who's put multiple projects on HackerNews and seen zero comments/upvotes, I'd love to learn what "HN Optimization" looks like. Since there's (presumably) no way to use black-hat techniques to get attention, in theory a better post is good for both the community and the OP.

<comment><USER38> It almost sounds like he either straight-up placed it on the front page himself, and/or had the founders organize a small upvote brigade of employees and family and gave them the best time to post

<comment><AUTHOR> Actually, we burned ourselves with a friends and family upvote brigade on product hunt (they lopped off a bunch of them suddenly...).

We learned from that and did not do this on HN (Engineering team were then only ones to have the post shared, as they're real HN community).

I think the algorithms to detect voting rings from dud accounts are not hard.

<comment><USER37> What did the assistance look like? Was it primarily advice?

<comment><USER36> I've answered this in my comment upthread. Let me know if you (or anyone) have additional questions.

</comment></comment></comment></comment></comment></comment></comment><comment><USER39> Congrats on launching! yaseer, why does Axiom require a desktop app (vs running purely as a chrome extension)?

<comment><AUTHOR> We found a whole set of technical limitations to using just a chrome extension (file downloads and uploads to name but one). The simplest solution was to include a desktop service, although I agree it's a huge inelegant electron app.

This paved the way for running axiom in the cloud. Once we can do that, you won't need the desktop app.

I hope with time we can have a slimmed down extension-only version too, but this will be more limited in capability.

</comment></comment><comment><USER40> You mentioned sports betting funds - Just wondering if you could elaborate further on the use case here?

<comment><AUTHOR> Collecting and consolidating data + filling in the forms across different sites to place bets

</comment></comment><comment><USER19> also, there's no download link on your tutorials/help page. kind of a growth hack fail.

<comment><AUTHOR> Very good point. There is now.

</comment></comment><comment><USER41> This looks quite similar to Ghost Inspector.

</comment><comment><USER42> Any plans for open source for on-prem use ?

<comment><AUTHOR> Right now it's actually entirely on-premise, as bots run only on your local machine - we don't process data.

Open-source is something we'd definitely consider - there are many hard technical problems with browser automation that would benefit from open-source collaboration.

<comment><AUTHOR> Correction -it's not entirely on-premise. The script storage and login is tied to our server.

This thread has given us some serious thoughts about a fully on-premise version though!

</comment></comment></comment><comment><USER43> Reminds me of Ghost Mouse from 20 years ago. I remember using it to automate Connect Four and win a few bucks from iWon.com.</comment>


Why the second wave of the 1918 Spanish flu was so deadly - <AUTHOR> https://www.history.com/news/spanish-flu-second-wave-resurgence <comment><USER4> Differences in health systems and infrastructure also matter. The Spanish flu hit the world in the days before antibiotics were invented; and many deaths, perhaps most, were not caused by the influenza virus itself, but by secondary bacterial infections. Morens et al (2008) found that during the Spanish flu "the majority of deaths … likely resulted directly from secondary bacterial pneumonia caused by common upper respiratory–tract bacteria.

Morens D. M., Taubenberger J. K., Fauci A. S. (2008) – Predominant role of bacterial pneumonia as a cause of death in pandemic influenza: implications for pandemic influenza preparedness. J. Infect. Dis. 198 962–970.

<comment><USER5> > The Spanish flu hit the world in the days before antibiotics were invented;

IIRC: Antibiotics and germ theory were invented. But the scientific community and the medical community weren't fully on board yet.

<comment><USER6> You are confusing two things. AntiSEPTICS and germ theory were both well established by the Spanish Flu. AntiBIOTICS were not. In other words we knew how to prevent surgical infection but not how to treat a bacteria that had already settled into someone's body.

Louis Pasteur came up with and proved the germ theory in the 1860s. Joseph Lister proved in 1865 that carbolic acid would prevent infection in surgery. Adoption took many years, but by the 1880s antiseptic surgery was widely adopted and Lister was made a baronet in recognition.

The Spanish flu was 1918, decades after the widespread adoption of antiseptics.

The first antibiotic, penicillin, was discovered in 1928. But it did not enter medical use until 1942. This was not a result of doctors resisting a good idea though. Penicillin is created by a bacteria. How do you produce useful quantities of a substance that in concentration kills the very thing that produces it?

<comment><USER7> "Penicillin is created by a bacteria" \- I think Penicillin is made by a fungus.

<comment><USER6> You are correct, my bad. But the production problem is what I described. The fungus is resistant but not immune to its own poison. It therefore proved very hard to produce commercially viable amounts. However once a mass production method was found, it was considered vital to the war effort and we quickly ramped up production.

</comment></comment></comment></comment></comment><comment><USER8> Good article, but this one from Smithsonian is much more detailed. [https://www.smithsonianmag.com/history/journal-plague- year-1...](https://www.smithsonianmag.com/history/journal-plague- year-180965222/)

<comment><USER9> Useful given history.com responded to my request with an empty document

</comment></comment><comment><USER10> _Harris believes that the rapid spread of Spanish flu in the fall of 1918 was at least partially to blame on public health officials unwilling to impose quarantines during wartime. In Britain, for example, a government official named Arthur Newsholme knew full well that a strict civilian lockdown was the best way to fight the spread of the highly contagious disease. But he wouldn't risk crippling the war effort by keeping munitions factory workers and other civilians home._

<comment><USER11> Or in 2020: must keep the stock market up.

</comment></comment><comment><USER12> All interesting but you just can't compare a Flu in 1918 with today.

The list is endless of what we didn't have in 1918, from computers to do DNA sequencing to even testing the way we do today.

<comment><USER13> Also we have some disadvantages we didn't had 1918. Like most people living in dense cities. Or it is harder to force people into quarantine.

<comment><USER14> And supply chains! There are many things that a 1918 city could manufacture in an emergency that a 2020 city can't.

<comment><USER15> example? I guess if you wanted it, the 2020 city can manufacture anything it could in 1918...

<comment><USER16> OK, let's start with a soap bar or a face mask..

<comment><USER17> You can make soap at home, a factory anywhere can trivially make it. And face masks are paper plus string, again hardly difficult to manufacture in any city in 2020. (In the UK most paper was made centrally in 1918, I happen to know this because I live near the site of the former paper mills - [https://www.thepapertrail.org.uk/](https://www.thepapertrail.org.uk/)).

<comment><USER16> Look, _I_ have a lathe at home and can make an engine if I want to, thing is other people can't because they never were interested in that. And I used the simplest things possible as examples.

Realistically, where would you start with these in a typical deindustrialized town today? "A factory" is a poor answer, some towns have none really, and factories are not interchangeable anyway.

<comment><USER17> There are small factories - everything from workshops to industrial estates - all over the place. If anything it's much better now because CNC machines and lathes are cheap, widely available and very versatile.

<comment><USER18> Yes, they're cheap and widely available. That's because we import them from China.

If your plan for resilience against a long-term disruption of global trade has a critical dependency on global trade, you may need to rethink it somewhat.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER19> There's a surprising amount of well-intentioned but misinformed speculation in this thread.

"We don't know" is the best answer to a lot of the questions about Spanish flu. What we do know is that it disproportionally killed young, healthy people who could go from no symptoms to dear in 24 hours. There's a lot of speculation as to why. I've seen medical experts theorize this is likely due too a cytokine response, meaning basically that a healthy immune system goes nuts. Apparently there are other diseases that have fit this pattern.

The mortality rate is estimated at about 3%. Influenza is 0.1%. Coronavirus is somewhere between those.

Best guess of why the second wave was so deadly was due to a mutation that likely happened in Europe.

It's true we have things we didn't in 1918 but we still have surprisingly few tools to combat viruses. Fun fact: only one virus has had a cure developed and that was Hepatitis-C in recent years. To be clear, vaccines != cures.

We also have problems we didn't have in 1918, specifically mobility of people. It's that mobility combined with people being highly contagious while being asymptomatic that makes this particular diseases such a challenge.

<comment><USER20> The mortality rate of Covid-19 seems higher than that.

The around 3% mortality rate is what you get by dividing the number of deaths by the number of infection (past and present) (as of today: 5359 / 140875 = 3.8%).

But this is incorrect as the number of currently infected people is quite high (half the total number of infections) and unfortunately, a few will die in the near future.

If you divide the number of deaths by (recovered + deaths), the mortality is much higher (as of today: 5359 / (70174 + 5359) = 7.1%).

Which seems actually about as high as the Spanish Flue right in between the low and high estimate (20M/500M, 50M/500M). And it's source for worries since our medical systems are 1) much more developed than in 1918, and 2) they are not yet completely overloaded. If it were to spread rapidly and overload our health systems, the mortality rate could rise even more, let's hope it doesn't reach that level.

<comment><USER21> > The mortality rate of Covid-19 seems higher than that.

Sure if you intentionally mislead people by cherrypicking data.

> The around 3% mortality rate is what you get by dividing the number of > deaths by the number of infection (past and present) (as of today: 5359 / > 140875 = 3.8%).

140875 is the number of confirmed and tested infections. The number of actual infections is certainly much higher. So the mortality rate is most likely much lower than 3.8%. After all, most coronavirus infections are mild for the vast majority of people.

> If you divide the number of deaths by (recovered + deaths), the mortality is > much higher (as of today: 5359 / (70174 + 5359) = 7.1%).

Sure if you ignore the hundreds of thousands of infected who "recovered" from mild symptoms.

> If it were to spread rapidly and overload our health systems, the mortality > rate could rise even more, let's hope it doesn't reach that level.

The standard pattern. Try to fearmonger with intentionally misleading stats and then try to come off as a good samaritan. Why do all the coronavirus fearmongering comments all follow the same template?

<comment><USER20> If it was fearmonguering, it was not intentional on my part. Sorry for that.

Indeed I missed the mild cases which are not quantified but likely quite numerous.

</comment></comment></comment></comment><comment><USER4> There is a very good post from univ oxford on this topic:

[https://ourworldindata.org/spanish-flu-largest-influenza- pan...](https://ourworldindata.org/spanish-flu-largest-influenza-pandemic-in- history)

which provides more scientific background and real numbers and compares it with other flu pandemics and the ongoing covid pandemia.

</comment><comment><USER22> _" The virus infected 500 million people worldwide and killed an estimated 20 million to 50 million victims"_

 _" Somewhere in Europe, a mutated strain of the Spanish flu virus had emerged that had the power to kill a perfectly healthy young man or woman within 24 hours of showing the first signs of infection."_

I knew this, but its still utterly terrifying to see it expressed so bluntly.

As the article says, rapid population movement was a key factor in its spread and mutation. We can't take the world back to a time when international travel was rare, so when we're done with the immediate effects of the Covid-19 pandemic, we (as a species) need to get better at rapid vaccine creation. Like an order of magnitude better. Otherwise the next one could be one that takes us down.

<comment><USER23> Can we not? Why can't Australia for example close all borders and call it the day?

<comment><USER24> For one thing, most countries are not self sufficient in basic necessities (not even food!). You could do it for a very short time, but not sustain it. Also, the economy of the country depends on international trade. Modern economies actually don't have a lot of slack. If you shut down trade, quite a few companies will just go under. You would have to back that up with _huge_ subsidies. It would be a massive undertaking.

<comment><USER25> Viruses don't live long enough to survive ocean shipping, so as long as the (tiny) crew is kept isolated, ocean shipping is fine. Within a country, it's not hard to disinfect a truck and keep the driver isolated.

Of course, if the originating factory is shut, then you're screwed - <USER2> that's true regardless of trade.

Also worth noting: it takes a tiny number of people to produce tons of food (calories). Maybe not true for artisanal salami but for bulk basics.

<comment> blaser-waffle The US is the single largest exporter of grain in the world. Not the largest producer, not the largest consumer, but the largest exporter.

And the US farming population is ~1% of the total US pop. Modern tractors and combines are amazing.

Doesn't mean there aren't a ton of bodies involved in processing and transportation, though. You can probably find local-ish wheat in the US, but you're not going to boil and eat the wheat berries for a porridge like it's 1850 (though you could); you're going to buy breakfast cereal.

</comment></comment></comment></comment></comment><comment><USER26> in case you don't read the article, there is an interesting historical anecdote: why it's called "spanish flu"

> Interestingly, it was during this time that the Spanish flu earned its > misnomer. Spain was neutral during World War I and unlike its European > neighbors, it didn't impose wartime censorship on its press. In France, > England and the United States, newspapers weren't allowed to report on > anything that could harm the war effort, including news that a crippling > virus was sweeping through troops. Since Spanish journalists were some of > the only ones reporting on a widespread flu outbreak in the spring of 1918, > the pandemic became known as the "Spanish flu."

</comment><comment><USER27> Once the quarantine is lifted we will have a bunch of non inoculated people, and the corona will spread all over again.

<comment><USER28> The goal isn't to stop those people from being infected, it's to delay their infection. A massive spike of infections would overwhelm the hospitals. The same number of patients, slower, is doable.

</comment></comment><comment><USER29> The article does not answer the question, except for "it mutated"

<comment><USER30> Does it not? It clearly says that countries like the UK knew full well that the best way to combat the illness was to isolate and quarantine people and cities, but decided not to do it to avoid harming the war effort - <USER3> in a lot more deaths. How is that not an answer to this question?

<comment><USER31> I'm watching the UK government decide not to implement 'delay' measures today, ostensibly to protect the economy. It seems there is always a reason not to do the right thing.

I hope they change their mind very soon.

<comment><USER32> (deleted)

<comment><USER33> Has anyone done the math on how long a flattened curve stretches out if you flatten it enough to not overwhelm the medical system?

So for example, if we have 100,000 ICU beds in the US and we somehow make them all available for COVID-19 patients (unlikely), how flat do we need to make the curve to not go above 100k ICU patients, and how long will that flattened curve last? A month? Six months? Two years?

Assuming 70% of the adult population gets it, that's roughly 140 million infected. Assuming 5% need ICU beds, that's 7 million ICU patients. And assuming the average ICU stay is 2 weeks, we'd need to therefore spread our ICU patients over 140 weeks, or almost three years. Is that how long we can expect social distancing to last if things go perfectly?

<comment><USER34> The UK only has ~4000 ICU beds, and these are mostly already full. We've got no chance.

<comment><USER33> Yeah, just to be clear, I think there's no chance in the US either. I just saw something that said we actually only have 45k beds, and they're mostly already full anyway. Plus there's no chance we're going to be able to keep this clamped down for months. We'll be overwhelmed in weeks.

</comment></comment></comment></comment></comment></comment></comment><comment><USER35> [https://en.wikipedia.org/wiki/Information_asymmetry](https://en.wikipedia.org/wiki/Information_asymmetry)</comment>


Server balancing and failsafe - <AUTHOR>

Site uptime is a major issue for everyone. I currently have space on 6 different hosts and i'm wondering if there is any practical way to use them all effectively. They only thing i can come up with is to be redirected to a server that is currently up. Right now I simply have everything mirrored on all the hosts ready to switch DNS if something were to happen. This however doesn't solve much and takes a bit more work with dynamic sites. It would be nice ot hear what others are doing to ensure great uptime. <comment><USER2> round-robin DNS setups and/or use of haproxy are some of the most common approaches in your scenario (limited budget, early growth phases of site).

Round-robin DNS is basically free to implement, and while it doesn't give you 100% uptime, it helps ensure that a problem on any 1 server only affects a small portion of your userbase.

</comment><comment><USER3> Round-robin would be one approach; simple but effective. You could also make use of something like haproxy, which would load balance for you.</comment>


Ask HN: The most interesting/weird interview question you've been asked? - <AUTHOR>

While reading
http://www.bbc.co.uk/news/technology-16346813 I thought it might be fair to source HN and ask about interesting interview questions. <comment><AUTHOR> Question:

Imagine that you have a non-session-persisting load balancer in front of a cluster of web servers (n >= 3). Since this load balancer doesn't direct requests in any persistent fashion, a user making subsequent requests may be directed to any of the servers available in the cluster in a non-deterministic manner. In your application, you need to handle image uploads from users, and as soon as the upload is finished, you need to show the user their uploaded image. This creates a problem for you because after an upload, the load balancer may direct the user to a web server which does not have the users recently uploaded image resulting in a 404 Not Found for the image request and a poor user experience.

In the current system, an rsync process runs every 5 minutes and copies uploaded images around to each server so that every server has every file, eventually. This isn't a very optimal solution because users may not see their uploaded images for up to 5 minutes depending on which server they are connecting to for any given request and it forces every server to have a copy of every image uploaded, which may not be the most scalable solution.

Assuming you could not modify the load balancer, how would you redesign or fix this system so that files are uploaded in a way that it doesn't matter which server in a cluster a user connects to in order to be able to satisfy a request for the file? There are many ways to solve this problem. Please describe one or more ways and discuss the pros and cons of your solution(s). If your solution is very simple (not a bad thing), consider offering a couple alternate solutions so we have a strong understanding of how you evaluate and approach problems.

-

My Response:

Basically that this was poor architecture and didn't make sense. They shouldn't be using their web servers as a file store. They didn't like that.

<comment><USER2> No offense intended, but I wouldn't have liked that answer either, especially as it isn't an answer to the question posed.

It might well have been a hypothetical problem they've had in the past or that they've heard of, or it might have even been that their application requirements evolved into the use case above.

Not answering the question doesn't indicate that you can solve the problem. Right or wrong, businesses arrive at scenarios like this often due to circumstances outside their control.

Either way, I like it as an interview question, as it would effectively filter out people who are either not clever enough to solve it or not motivated enough.

When I hire, one of the best things I can hope to find in an potential employee is a 'can-do' attitude. While that might sound cliche, you can hire pretty much any smart person to point out the problems you have. It's much more difficult to hire somebody who can both identify those problems _and_ come up with potential solutions and is capable of fixing them.

<comment><AUTHOR> I'd rather someone tell me that the problem is flawed and offer me another alternative to avoid future problems -- that would be experience talking. I'd rather have someone FIX problems and not just patch a system continuously hobbling down the wrong path. I'd like not be too far down the rabbit hole and figure out we need a major architecture overhaul.

More research and planning from the start, saves time and money in the end.

<comment><USER3> Agreed. But this isn't even a solution you can't implement for legacy reasons. At any point you can do this, and you can probably do it live by taking it out of the load balancer equation one at a time.

In this case, centralizing storage makes sense.

Pros: less stuff to manage, no more rsyncs, immediate feedback on all servers. Save money by only needing to buy 1/n amount of space (assuming the space required is significant) because you're only maintaining X GB centrally versus X GB * n servers.

Cons: Money up front, increased latency to fetch data, but you're already waiting up to 5 minutes.

</comment></comment></comment></comment><comment><USER4> I was recently asked given a variable amount of parameters passed into a JavaScript function how would you sum all of the parameters. My answer was I would not write JavaScript code like that, and would fire anyone on my team that did. Needless to say I did not get the job, nor did I want it. I have been critical of this type of interviewing for a long time. Trick questions and magic code really provide little insight to the value of a candidate. Further most interviewers do not have the clinical background to even interpret the results as many of these questions are based off of psychological tests. It's a cargo cult mentality and reflects poorly on an organization.

<comment><USER5> I think this is a reasonable question, insofar as the interviewer probably just wanted to hear that you knew how the "arguments" object works. The base2 library, for example, uses arguments in its format function: [http://www.google.com/codesearch#O7TxUbFbPP4/lib/src/base2.j...](http://www.google.com/codesearch#O7TxUbFbPP4/lib/src/base2.js&q=format%20package:http://base2%5C.googlecode%5C.com&l=1542)

Quizzing people on language features seems a tiny bit silly, but I respect it as a method of ensuring expertise in a given language.

edit- I actually don't think it ensures expertise, but it can be a signal of a lack of expertise, so it is a useful method to avoid a false positive on a hire.

<comment><USER4> I have to respectfully disagree, I have built a lot of JavaScript based application, some of them huge code bases for the like of IBM and not once have I had to resort to using the arguments object. I consider myself to be an expert, but should I ever need to use it, I would look up the API because it is not a feature used in the mainstream of development. Understanding technical nuances of a languadge does not make a developer an expert, delivering a clean, maintainable and elegant code base to production to generate revenue does. Now my opinion would differ if it was someone hired to actually work on V8 or one of the JavaScript run-times but requiring mastery of nuances of a languadge when they will be used rarely if at all seems to me to be a filter that can eliminate good candidates in favor of trivia knowledge. Trivia can be looked up, delivery and maintainability cannot, these questions favor trivia over delivery.

</comment></comment></comment><comment><USER6> "If you were a piece of fruit, would you rather be one still on the tree or one that had fallen to the ground?"

</comment><comment><USER7> How many gas stations are in the United States?

<comment><AUTHOR> I like these types of questions, but I've never been given one. How did you respond?

Here's my take:

My first inclination would be to think about all the gas stations in my neighborhood, and all of those I pass as I travel somewhere. The gas stations start to spread out the further from civilization I go, but I've never gone more than a quarter to half a tank of gas before seeing another. (I have a phobia about running out of gas in the middle of nowhere.)

But the difficult part it is thinking about how gas stations are set up. There is definitely a relationship between population density and gas station count. The less people, the less gas stations -- but conversely there seems to be a point where the population is so saturated (SF proper) that gas stations are scarce inside the city.

But maybe I just think that because of the relative distances I'm traveling. Inside the city I'm walking and see less gas stations, but outside the city I'm driving and covering a longer distance in the same amount of time and see more gas stations.

Also, we seldom see huge lines at the gas stations which means they are plentiful enough to service all available commuting cars without significant queue. If there are just enough to service cars without a queue, maybe we can say this (naively):

X number of people in the US. Three people per family, so X/3 = Y number of cars. (A lot of families have more than one car, I know) If each gas station has an average of Z pumps and is utilized constantly at an average of W% through the day, we need at least Y * Z * W% gas stations to service everyone appropriately.</comment></comment>


2015 Linux Jobs Report: Linux Professionals in High Demand - <AUTHOR> http://www.linuxfoundation.org/news-media/announcements/2015/03/2015-linux-jobs-report-linux-professionals-high-demand

<comment><USER2> > Nearly all hiring managers are looking to recruit Linux professionals in the > next six months.

Very hard to believe, unless there are several hidden assumptions, like looking at specific industry sector and ruling out companies based on some criteria. And with those, what other assumptions are there in the report?

The very summary makes whole article uncredible.</comment>


Docker containers should not run an SSH server - <AUTHOR> http://jpetazzo.github.io/2014/06/23/docker-ssh-considered-evil/ <comment><USER2> What is the general pattern of usage with Docker containers? Are they supposed to isolate just one application and do IO via a networks socket?

It used to be you ran everything on a real server. Then it moved to VMs in the cloud. So spawning and managing VM was the the new thing to with a large IaaS,PaaS,...aaS industry around it.

The product would consist of one or multiple VMs. All possibly running multiple applications as different processes.

Then I guess inter-dependencies between applications and OS versions got to be complicated, so the idea was that each application should run in its own lightweight VM (with LXC as long as they share the same base OS kernel at least).

Isn't this just pushing the problems into managing dependencies between more little VMs while also constraining the architecture? It increases the difficulty of synchronizing 10 or 20 separate isolated applications start,upgrade,fail. Maybe all that living on yet another VM guest machine (so conceptually having 2 virtualization levels). Handling more complicated network setup (bridging, firewall rules at multiple levels). Handling effects of disk and other subsystem interacting with each other in strange, sometimes sub-optimal ways.

One idea was that ok, this is good for security. One can build secure containers. But doesn't SELinux do that better? It even has a multi-level security mode. It sure is complicated but it is used.

<comment><USER3> > What is the general pattern of usage with Docker containers? Are they > supposed to isolate just one application and do IO via a networks socket?

The maintainers of Docker _appear to_ strongly believe that 1 application per container is the way to go.

At sufficient scale, they are pretty much correct. You grab Y bare metal hosts and spin up X docker containers for X processes.

> Isn't this just pushing the problems into managing dependencies between more > little VMs while also constraining the architecture? It increases the > difficulty of synchronizing 10 or 20 separate isolated applications > start,upgrade,fail. Maybe all that living on yet another VM guest machine > (so conceptually having 2 virtualization levels). Handling more complicated > network setup (bridging, firewall rules at multiple levels). Handling > effects of disk and other subsystem interacting with each other in strange, > sometimes sub-optimal ways.

A handful of people advocate for immutable role-based containers [e.g. [https://devopsu.com/blog/docker- misconceptions/](https://devopsu.com/blog/docker-misconceptions/)] for that reason. In that use case, its really replacing running something like Xen + Chef, KVM + Ansible, or whatever.

You grab a host machine, you stand up your X containers with Z processes per container on Y hosts.

I think this is _really_ equivalent to the Microservices vs. Normal SOA vs. Monolithic argument. Given sufficient scale/requirements, each makes sense. However, none of them are optimal for all situations.

<comment><USER4> > At sufficient scale, they are pretty much correct. You grab Y bare metal > hosts and spin up X docker containers for X processes.

Can you elaborate why they are correct with this view? What is inherently superior about 1 process per container? What is wrong with having two cooperating processes per container? And why should Docker containers only run on bare metal?

<comment><USER5> > And why should Docker containers only run on bare metal?

The "point" of Docker is to be a lightweight replacement for virtualization in some cases. Layering Docker on top of virtualization, while fine, adds layers rather than removing/replacing them.

</comment></comment></comment></comment><comment><USER3> I think the core flaw with the "no SSHD in Docker, ever" reasoning is it makes a number of assumptions.

To me, for my use, Docker is really a lightweight role-based VM I can put on a provisioned host of some kind [another VM, dedicated server]. In other words, its a really simple way to deploy X identical instances of an entire service/application. If _any_ component in that instance of the application is non functional the container is dead and you route requests to a different container.

The problem with the "one process per Docker instance" logic is you need a great deal more service discovery logic as a result. There are a bunch of projects/methods to simplify this but at the end of the day it adds complexity to operations. Instead of a single health check to monitor, you have X containers to monitor. You have X containers to discover, etc. Sure, this lets you get out of running Supervisor and SSH on a Docker container...but I think it adds alot of application complexity that you can avoid 90% of the time.

SOA makes sense when you have a large deployment but if you are deploying to a cluster of 5 machines...its overkill. Many [likely the majority] of projects are at the scale of a basic cluster for redundancy.

<comment><USER6> I think that's just an issue of tooling and getting used to a different way of working with the services. If you run each of those services as a single docker container and group them on the same server, there does not need to be any more service discovery: You can link all of them. That still gives you the flexibility of modifying the linking without having to modify any of the containers.

The main practical difference vs. running them all in one vm is explicitly documenting dependencies - both _which service_ relies on which package sets etc. (assuming you're strict about what you put in the Dockerfiles) and which part of your system needs access to which other component or needs to share access to which directories.

You can also easily start them e.g. via systemd unit files or via a tool like fleet, and pull the ip/port of dependencies on startup of the container and pass them as arguments. This also makes deploying them as a unit easy, but still decouples it:

If you need to move one component to a different server, or suddenly realize your foobar server needs lots of resources and want to load-balance it across multiple machines, you can easily do so by forwarding a port via stunnel or load-balancing via haproxy or what have you without affecting the actual containers.

As for your health checks, you can still have a single all-encompassing health check if you want - whether it's in one "vm" or a set of separate containers does not really change that in any way, though the bigger the setup, the more additional checks you'll probably want to add. You have X containers to monitor, but you had X server processes to monitor previously. If your use case made it ok to depend on just e.g. hitting a page on a website that depends on all X services being up, then your single old monitor still does the job.

The overhead can be made really minimal and beneficial even for a single server setup, yet help you if/when you suddenly want to scale part of it. Personally, I run about a dozen docker containers on my home server so far, and I expect that to increase several times over as e.g. every web app I experiment with ends up running in its own container, with the docker setup in a git repo as documentation of exactly which project caused me to pull in which additional packages etc., and keeping the host itself as pristine as possible. Makes me lot less nervous about distribution upgrades etc.

<comment><USER3> > I think that's just an issue of tooling and getting used to a different way > of working with the services.

Load Balancer w/ Builtin HTTP Health Checks <-> Application Containers [nginx+php-fpm]

Load Balancer w/ Builtin HTTP Health Checks <-> Container [nginx] <-> Container [php-fpm] <-> Additional health check you have to build out to check that php-fpm is working correctly <-> Additional Tooling to make your process work

For every single service in your architecture you are adding an additional level of redundant complexity because it "helps you scale".

I can spin up as many app containers as I want, as often as I want without a second thought or a moment's consideration. I don't care if I have 1000 of them or 1, as long as the load balancer can handle the connections.

The scale you are talking about is when you need more than just a couple load balancers...and you aren't going to need that for 90% of projects.

<comment><USER7> I think that in the specific case of nginx plus php-fpm, separating things all the way out to one process per container is overkill, and it would be better to put the two in a single container, managed with something like runit or supervisord. Conceptually, the combination of nginx and php-fpm can be treated as a single service, and your operations team (or youu when wearing your ops hat) would probably prefer to think of it as such. There are two tricky parts though: getting both underlying processes to log to the container's stdout/stderr, and getting them both to shut down cleanly in response to a SIGTERM to the supervisor.

<comment><USER6> I disagree. It's a perfect example where one of the components is likely to scale differently than the other, and where the components are reusable separately but less reusable together.

As someone constantly wearing my "ops hat", I much prefer to see them treated separately for that reason.

</comment></comment></comment></comment></comment><comment><USER8> Hrrm. I want to agree, but as far as containers go outside of purely docker, I just don't buy these arguments.

It makes perfect sense to use industry standard, encrypted communications with proven cryptography when creating vast numbers of systems. Yes, you don't have to. No, that doesn't mean it's a bad idea.

I'm not sure why things would be any different with docker, and after reading the article I'm unconvinced. If you want to be locked in to docker's APIs, so be it. If you want to be free of them and integrate in other ways, such as proven, portable, secure methods like SSH, that's fine too.

If a management task as simple as deploying a key is so hard in docker (couldn't you just bind-mount a read only _.ssh_ dir?), maybe you should consider alternative methods of container instantiation.

I don't see how granting access to the host is a cleaner architecture... from a security standpoint, it seems the opposite.

<comment><USER9> I don't think I understand your point.

From the Docker Host itself, if you need to manage the state of a container, the intuition is that you need to go into the container (with SSH) in order to do so. But by externalizing your state, you can manage it without the need to enter the container. Assuming your Docker Host is secure, this doesn't make anything less secure just because you're no longer abusing SSHd in order to manage your application's state.

In the case you need to gdb, or strace the process, you can do that from the Docker Host with nsenter. Assuming your Docker Host is secure, you no longer need to abuse SSHd to carry out a debugging task that has nothing to do with needing a secure shell.

Neither of these use-cases have anything to do with the security of SSH.

In the case that you need to do these things from a remote host, the prescribed answer is indeed SSHd to access the Docker Host, at which point you switch to the previously suggested methods for managing state.

"I don't see how granting access to the host is a cleaner architecture... from a security standpoint, it seems the opposite."

Because now you only have to worry about one security layer instead of N security layers for each container you run. The security layer is now actually coupled to the act of granting access to the host, its intended purpose vs granting access to a container so you can manage its state or debug it or whatever.

As far as being locked into Docker's APIs, I totally miss the aim of this remark. Volumes are just paths on the filesystem. If you're talking about the interoperability of standard tools to manage your state, I don't think they will have problems in this case.

<comment><USER8> _the prescribed answer is indeed SSHd to access the Docker Host, at which point you switch to the previously suggested methods for managing state. [...] As far as being locked into Docker 's APIs, I totally miss the aim of this remark._

Yes, you missed the point. Please read the other response to comprehend the difference.

</comment></comment></comment><comment><USER10> I don't get it. What's wrong with putting my SSH public key in the image?

Tbh I'd pay more attention to host key regeneration! Easily overlooked.

Until there's better investigation tooling available, sshd seems a fairly sensible approach.

MAC / SELinux prevents many other poke-inside techniques on enterprise platforms

<comment><USER11> I don't think it's SSHd in particular that he has a problem with. Docker maintainers do not seem to like the idea of people treating Docker containers as lightweight VMs. They seem to want images to be restricted to running as few processes as possible - ideally one. I think they view it as too hard to scale once you start going down the path of running multiple services in an image - I've seen the "pets vs cattle"[1] analogy used for explaining why.

There is an image by Phusion called baseimage-docker[2] which adds SSHd, init, syslog, and cron in an attempt to make Docker containers more like lightweight VMs. But in the #docker channel, I've seen people have issues with it. For example, one person had some /etc/init.d scripts that wouldn't start up (other ones started up fine). Turns out that one of the signals that init was waiting on to start that script was never getting sent (I think it was networking coming online?), and that was just a side effect of how Docker works that couldn't easily be fixed. The Docker maintainers in the channel discouraged using this image for these reasons.

[1]: [https://groups.google.com/forum/#!msg/docker- user/pNaBYJkmnA...](https://groups.google.com/forum/#!msg/docker- user/pNaBYJkmnAA/TsA3b8u5kf0J)

[2]: [http://phusion.github.io/baseimage- docker/](http://phusion.github.io/baseimage-docker/)

<comment><USER12> Pets vs. cattle is about customization vs. automatic setup; I don't understand how it's relevant to whether the containers have no SSH or identical SSH.

<comment><USER13> Pets vs. cattle was the last battle. Now there's immutable cattle vs. mutable cattle; Docker is trying to promote immutable infrastructure and microservices and eliminating ssh is one aspect of that.

</comment></comment></comment></comment><comment><USER4> I am the author of baseimage-docker ([http://phusion.github.io/baseimage- docker/](http://phusion.github.io/baseimage-docker/)) and I work at Phusion. I have the feeling that Jerome wrote this article mainly in response to the fact that baseimage-docker encourages using SSH as a way to login to the container. I believe that the ability to login to the container is very important. Depending on how you architect your container, you might not have to, but I believe that it's always good to have the _ability_ to, even if only as a last resort method.

I had a pleasant conversation with Jerome quite a while ago about SSH and what the "right" way is to login to a Docker container. We were not able to find consensus, but Jerome is a brilliant guy and his reasons were sound. For some time, I considered using lxc-attach to replace the role of SSH. Unfortunately, a few weeks later, Docker 0.9 came out and no longer used LXC as the default backend, and so suddenly lxc-attach stopped working. We decided to stick with SSH until there's a better way. Solomon Shykes told us that they have plans to introduce an lxc-attach-like tool in Docker core. Unfortunately, as of Docker 1.0.1, this feature still hasn't arrived.

Now, Jerome is advocating nsenter. There is currently an ongoing discussion on the baseimage-docker bug tracker about replacing SSH with nsenter: [https://github.com/phusion/baseimage- docker/issues/102](https://github.com/phusion/baseimage-docker/issues/102)

But leaving all of that aside, we regularly get told by people that Baseimage- docker "misses the point" of Docker. But what is the point of Docker? Some people, including Jerome, believe it's all about microservices and running one process in a container.

We take a more balanced, nuanced view. We believe that Docker should be regarded as a flexible tool, that can be mended into whatever you want. You _can_ make single-process microservices, if you want to and if you believe that's the right choice for you. Or you can choose to make multi-process microservices, if that makes sense. Or you can choose to treat Docker like a lightweight VM. We believe that all of those choices are correct. We don't believe that one should ONLY use Docker to build microservices, especially because Microservices Are Not A Free Lunch ([http://highscalability.com/blog/2014/4/8/microservices- not-a...](http://highscalability.com/blog/2014/4/8/microservices-not-a-free- lunch.html)).

Baseimage-docker is about _enabling users_ to do whatever they want to. It's about choice. It's not about cargo-culting everything into a single philosophy. This is why Baseimage-docker is extremely small and minimalist (only 6 MB memory over), flexible and thoroughly documented. Baseimage-docker is _not_ about advocating treating Docker as heavyweight VMs.

<comment><USER14> I don't think your base image misses the point of Docker. Different people use Docker for different purposes, that is normal and a fundamental goal of Docker.

I do have criticism for your _communication_ around that base image, starting with the link-bait blog post "you're using Docker wrong". Your message is that anybody not using Docker _your_ way (full-blown init process, sshd, embedded syslog) is doing it wrong. That is not only incorrect, it contradicts Docker's philosophy of allowing and supporting more than one usage pattern.

My other criticism is that you point out a known Docker bug (the pid1 issue) and use it as a selling point for your image, without concerning yourself with reporting the bug let alone contributing to a fix. Meanwhile many people hit the same pid1 bug and have reported, suggested possible fixes, or contributed code to help implement that fix. If you want to be taken seriously in the Docker community, my recommendation is that you consider doing the same.

<comment><USER4> Hi Shykes, glad to see you replying. Your point about communication is fair enough. I will take a look at how the communication can be improved. However, let me stress that the message is _not_ "you're using Docker wrong unless you're using it our way". I see how it can be read like that, but the real message is much more technical, complicated and nuanced. The message is fourfold:

1\. Your _Unix system_ is wrong unless it conforms to certain technical requirements.

2\. Explanation of the requirements.

3\. One possible solution that satisfies these requirements: Baseimage-docker.

4\. Does your image already satisfy the requirements? Great. If not, you can implement these requirements yourself, but why bother when you can grab Baseimage-docker? And oh, it happens to contain some useful stuff that are not _strictly_ necessary but that lots of.

As you can see, such a complicated message becomes waaay too long and hard to explain to most people. It probably only makes sense if you've contributed to the Linux kernel, or read an operating systems book. If I explained it in a way that's too technical and nuanced, 99% of the people will fall asleep after reading 1 paragraph. So the message was simplified. I apologize if the simplified message has offended you, and I am continuing to finetune the message.

As the for the PID 1 issue: I genuinely thought you guys didn't include a PID 1 _on purpose_ , because running one isn't that hard. Last time I talked to Jerome, he had the opinion that, if software couldn't deal with zombie processes existing on the system, it's a bug in the software. With that response in mind, I thought that the Docker team does not recognize the PID 1 issue as really an issue. So please do not mistake the lack of a bug report as malice.

Later on, you told me that you guys are working on this, and I was glad to hear that.

I get the feeling that you feel bitter about the fact that I chose to write Baseimage-docker instead of contributing a PID 1 to Docker. Please understand that I did not do this out of any adversarial intentions. My Go skills are minimal and I am busy enough with other stuff. This, combined with the fact that at the time I thought the PID 1 issue was simply not recognized, led to me write Baseimage-docker. I would like to stress that I look forward to friendly relationships with you, with the Docker team and with the community.

<comment><USER15> Do you have any links to relevant discussion, documentation and/or code related a (official?) pid 1 process for/by docker? I'm not able to find it quickly and I thought it might be useful if you could share given that you clearly have some context. Thanks!

<comment><USER4> I don't know what the Docker team are working on, but this is the PID 1 process we use in Baseimage-docker: [https://github.com/phusion/baseimage- docker/blob/master/imag...](https://github.com/phusion/baseimage- docker/blob/master/image/my_init) It's a custom system we wrote specifically for use inside Docker.

</comment></comment></comment></comment></comment><comment><USER16> Volumes are convenient! But what do you do if you have a multi-host setup? Multiple volumes, shared volumes, distributed file systems, NFS? What do you use when?

<comment><USER17> Would you not mount nfs volume in each container to share data?

<comment><USER16> ok, then how do you increase server storage with minimum downtime? say you're in a cloud environment, specifically AWS

</comment></comment></comment><comment><USER18> Reading some of the other comments in this thread, I think I'm getting a better idea of why this guy is suggesting that SSH in Docker could complicate things, especially when you are scaling to a large number of servers, but what I still don't quite understand his argument about security updates. Is that in any way SSH specific, beyond the fact that SSH is powerful and it's critical to be up-to-date on powerful things?

If it's a problem to update SSH, then it should also be a problem to update whatever else you have in your Docker container. I guess there's some argument to be made that if you're running a single-purpose Docker container, the updates to whatever service it's running won't sync up with the updates to SSH, so you may drastically increase the number of times you'll have to package the image, but that's just a general argument in favor of single- purpose containers, not anything specific to SSH like the key management issue.

</comment><comment><USER19> Its funny that he uses Docker as a build and installation tool for nsenter. If you look at the installation example from the nsenter github page it shows how you can mount your host's /usr/local/bin inside the container where nsenter will be built. Pretty nifty/hacky way for building software and keeping your system clean.

</comment><comment><USER20> Wow. Given all the hype around docker, I'm really surprised nsenter type functionality isn't part of the core. Glad I'm using FreeBSD where running a shell in your jail is just 'jexec my_jail bash'. Compare that to the rigamarole described in the article. Maybe the docker guys will benefit from just reviewing the tooling for existing systems. I also recommend a look at ezjail-admin.

<comment><USER6> nsenter is part of util-linux, which means it will eventually make it into pretty much every Linux distro around. The functionality doesn't belong in Docker, because it isn't Docker specific - all it depends on is cgroups, and cgroups is a kernel feature, not a Docker feature.

Once it's in there, a "jexec" equivalent would a couple of lines of shell script.

<comment><USER20> Being part of util-linux makes sense. However, it would be so much nicer if I didn't have to look up those two lines.

</comment></comment></comment><comment><USER21> nsenter might be a short term solution, but IMHO, getting a resolution to this issue [https://github.com/dotcloud/docker/issues/1228](https://github.com/dotcloud/docker/issues/1228) so you can just do "docker exec <cid> /bin/bash" is much simpler than the nsenter call he recommends. Plus, he leaves out chroot, so I have a different view of the filesystem than PID 1 inside the container does.

</comment><comment><USER22> As a cloud hosting provider, I don't want to give customers access to the hardware node. Then what?

</comment><comment><USER23> i don't see that wrong to setup the ssh service, if i accept ssh daemon is safe which i do.

<comment><USER6> You need not just accept that the ssh daemon itself is safe, but that:

\- Your key management is safe. \- The process manager you now need to introduce to start sshd and the app running is safe. \- That the ssh daemon is sufficiently protected against abuse. \- That your configuration of it is safe.

If you don't _need_ ssh in every container to do achieve what you need to achieve, why do you want to have to deal with each of those _and_ waste the extra resources of having a bunch of extra sshd's and process monitors running?

(To the last point: Yesterday we suffered an attempt at brute-forcing ssh on a public facing server. We're used to people trying to brute force passwords. But as it happens, it is "easy" to make openssh consume all of your servers resources if you don't block access on the network level in the event of an apparent attack; so if any of those ssh servers are reachable in any way from the outside, you _have_ just increased your attack surface even if your key management and everything else is perfect and they have no way of actually getting _in_ )

<comment><USER4> If you are worried about the attack surface, then SSH - as Baseimage-docker configures it - isn't that much of an issue. By default, we do not expose the SSH port to the public Internet, nor do we install any keys. Unless otherwise configured by the user, you first have to login to the host machine, and then from there login to the container through SSH.

<comment><USER6> While it's great that you ship with secure defaults, to me, if you're going to restrict it to access from the host only, that just makes it more pointless to run sshd in the containers vs. the alternatives presented in the article.

<comment><USER4> Like I already said, there is an ongoing discussion about replacing SSH with nsenter now that nsenter is a viable alternative: [https://github.com/phusion/baseimage- docker/issues/102](https://github.com/phusion/baseimage-docker/issues/102)

SSH was purely chosen because until recently there wasn't a better alternative. lxc-attach stopped working out of the box since Docker 0.9. See [https://news.ycombinator.com/item?id=7951042](https://news.ycombinator.com/item?id=7951042)

</comment></comment></comment></comment></comment><comment><USER24> its all about useability basically use nsenter.

other points in the blog are true but its not what people want. people want nsenter, but they dont know, so they use ssh.

</comment><comment><USER25> Typo: where a special key with force a specific command</comment>


What Arc gets right - <AUTHOR> http://rondam.blogspot.com/2008/02/what-arc-gets-right.html

<comment><USER2> For me, it's most important that Arc's been a lot of fun so far. I get to coerce and zap things. There's not much standing in the way of me doing. It's not a bureaucratic language.

Of course, this is coming from a person who was overjoyed only hours ago when she figured out that Ruby can evaluate strings as code. Maybe it's not pragmatic. But I don't code to be pragmatic.

So far it been a lot of fun. I get a heck out of blurring the line between data and code. I like the hackerly feeling it get when I figure out how to use closures in some crazy way. I like being able to easily write my own languages or language elements, in the flow of writing the larger program. I like that I don't even have the _impulse_ to reach for the docs, like I would with Java or C# or Python or even Ruby. When I think of something cool that I hope the language can do, everything encourages me to just find a way to do it myself. It's an empowering language.

I've only explored a bit now, but I think I see where things are headed, and I like it.

It might be that it's a language for people who like writing languages. I know I do.

</comment><comment><USER3> _"... what Arc gets right ..."_

Release early and often. Users sure are commenting on it for good or bad.</comment>