Serf: A decentralized solution for service discovery and orchestration - <AUTHOR> http://www.serfdom.io <comment><USER2> I'm jumping on a plane right now (a couple hours) but I'd be happy to answer any questions related to Serf once I land. Just leave them here and I'll give it my best shot! We've dreamt of something like Serf for quite awhile and I'm glad it is now a reality.

Some recommended URLs if you're curious what the point is:

"What is Serf?" [http://www.serfdom.io/intro/index.html](http://www.serfdom.io/intro/index.html)

"Use Cases" [http://www.serfdom.io/intro/use- cases.html](http://www.serfdom.io/intro/use-cases.html)

Comparison to Other Software: [http://www.serfdom.io/intro/vs-other- sw.html](http://www.serfdom.io/intro/vs-other-sw.html)

For the CS nerds, the internals/protocols/papers behind Serf: [http://www.serfdom.io/docs/internals/gossip.html](http://www.serfdom.io/docs/internals/gossip.html)

Also, I apologize the site isn't very mobile friendly right now. Unfortunately I can write a lamport clock implementation, but CSS is just crazytown.

<comment><USER3> A meta question if you will - I have often come across situations in work where "if only we had that tool". sometimes I have hacked something together, other times taken it further and tidied it up and released it. But this seems to have a large level of polish

so ...

When did you realise the need for surf

Did you work on it as a main project at some point or is it a side project

When and how did you decide to commit to getting this done

and the big one for me - tools are driven by a need, but often the need keeps coming while the time to build it diminishes. What strategies did you use to keep the plates spinning while building surf?

I think we can all mostly answer the questions - I just waant to know how different your answers are from say mine when I don't release two major OSS projects and you do.

cheers

<comment><USER2> Great questions. I'll answer each in turn.

I want to mention the "polish": I personally don't believe in releasing an open source project without polish. If it is missing docs, its just not complete. If it is ugly, it is not complete. The technical aspects of Serf were done weeks ago. Getting the human side of things done took another few weeks (contracting designers and such).

> When did you realise the need for serf?

The need for something like Serf has existed since I started doing ops. Every time I hit something where I say to myself "why is this so hard/crappy" is when I write it down in my notebook for a future date. I then just think on the idea for awhile and eventually when I feel like I have a significantly better solution than what is out there already, I build it.

I decided to start building Serf when @armon started throwing gossip protocol academic papers at me. I realized he figured it out, this was clearly significantly better, so we started working on it.

> Did you work on it as a main project at some point or is it a side project?

To get it out the door we focus on it for some period of time. After it is shipped it is still what I would consider a "main project" but time is split between various projects.

> When and how did you decide to commit to getting this done?

A few weeks ago. It took about a month to build. Building it is easy. Figuring out WHAT to build... took a long time. I have to say I've had "service orchestration/membership" in my notebook for years.

> What strategies did you use to keep the plates spinning while building surf?

No good answer here, we just prioritize some things over others. Serf was our top priority this month.

<comment><USER3> Thank you - that "building was easy, compared to knowing what to build" put a lot into perspective. And reaching out to external people to build the polish is a surprise, but obvious in retrospect.

I am afraid that for such a helpful and clear answer, you get a mere 1 karma point from me - but thank you.

</comment></comment></comment></comment><comment><USER4> Looks like this might be loosely related to Mitchell Hashimoto [1], who makes Vagrant.

[1] [https://twitter.com/mitchellh](https://twitter.com/mitchellh)

<comment><USER5> Both are HashiCorp projects.

</comment></comment><comment><USER6> The title needs to be improved. "A decentralized, highly available, fault tolerant solution..." for what? The title should include "for service discovery and orchestration".

<comment><USER7> Agreed, these tag lines come off as buzzword soup instead of informative. I would love a small quick scenario describing what serf can help prevent/enable.

<comment><USER8> You might find this useful: [http://www.serfdom.io/intro/use- cases.html](http://www.serfdom.io/intro/use-cases.html)

<comment><USER9> Yes, that page helped me, but the front page should be the hook and it left me none the wiser.

</comment></comment></comment></comment><comment><USER10> It seems serf and etcd are both trying to solve service discovery but are attacking it using different approaches. Which is pretty cool!

Serf looks to be eventually consistent and event driven. So you can figure out who is up and send events to members. This gives you a lot of utility for the use cases of propagating information to DNS, load balancers, etc.

But, you couldn't use serf for something like master election or locks and would need etcd or Zookeeper for that. Serf and etcd aren't mutually exclusive systems in any sense just solving different problems in different ways.

They have a nice write-up on the page here: [http://www.serfdom.io/intro/vs- zookeeper.html](http://www.serfdom.io/intro/vs-zookeeper.html)

</comment><comment><USER11> This would really benefit from a "how does this relate to zookeeper". I think this is an entirely new service, with different technical insides, and trying to provide a higher level solution to what people usually cobble together with ZK.

But I'd be interested in comments from someone knowledgeable.

Edit: I see this is addressed at [http://www.serfdom.io/intro/vs- zookeeper.html](http://www.serfdom.io/intro/vs-zookeeper.html) but it would be nice to have something more "just the facts" rather than arguing the serf is good.

<comment><USER12> In writing that section, we tried to provide "just the facts". If there is anything that seems wrong or misleading in any way, we'd like to know so that the page can be corrected. It is not our intention to say "Serf is good, ZooKeeper is bad". They are very different tools, and we are just trying to highlight the differences. In fact, we believe that the strongest use cases involve using those tools together.

<comment><USER11> I didn't mean to say that it sounded like a sales pitch. What I meant is that it's talking about relative strengths and weaknesses, where what I really need to understand what Serf is is more along the lines of how the API / model differs from ZKs notion of writing to or waiting on locations in the distributed space.

</comment></comment></comment><comment><USER13> The package documentation for the `serf` library[1] looks really exciting. I've been wanting to make a distributed file synchronization tool, and perhaps this would be an excellent library to build it on.

Question: as a relative networking idiot, how does NAT traversal fit into all of this?

[1] - [http://godoc.org/github.com/hashicorp/serf/serf](http://godoc.org/github.com/hashicorp/serf/serf)

<comment><USER12> We designed the `serf` library to be able to be easily embedded, so hopefully it can be of some use. Unfortunately Serf does not make use of any sort of NAT traversal currently. We've open sourced the project hoping to get the community involved, and NAT traversal is something we'd gladly work with the community to get implemented.

<comment><USER10> The first building block, STUN, is implemented over here: [https://github.com/ccding/go-stun](https://github.com/ccding/go-stun)

</comment></comment></comment><comment><USER14> How does security work in a system like this. If this is used in a shared hosting system, can a user inject false messages into Serf with for example PHP?

<comment><USER2> Yes, they can. In the general case its not an issue because usually your nodes are inaccessible by the public, but if you're using a shared hosting environment, this is entirely possible.

We're addressing this in the next release by signing/encrypting gossiped messages. See the roadmap: [http://www.serfdom.io/docs/roadmap.html](http://www.serfdom.io/docs/roadmap.html)

</comment></comment><comment><USER15> This is actually very cool and is something incredibly handy for managing failover/membership.

What I unfortunately don't understand is that there doesn't seem to be a library I can use to take advantage of this in my own application. If I have a program (in Go) am I expected to spin up my own serf process then communicate with it via socket? Is there an option for me to have serf live inside my application?

<comment><USER12> The Serf executable is actually just a wrapper around the `serf` library. That library is designed to be embedded in Go applications. Documentation for the library is available here: [http://godoc.org/github.com/hashicorp/serf/serf](http://godoc.org/github.com/hashicorp/serf/serf)

</comment></comment><comment><USER16> What happens to your cluster when your network experiences intermittent packet loss and your random UDP messages get lost? Nodes just start going down and up randomly? (For those of you going "So what, that's normal", this not a quality of an HA system)

<comment><USER12> I'd highly recommend taking a look at this page: [http://www.serfdom.io/docs/internals/gossip.html](http://www.serfdom.io/docs/internals/gossip.html). One of the great attributes of the gossip protocol is it is very robust to intermittent network failures. Under minimal packet loss conditions (<5%), the rate of false positives should be very low. This is due to a few techniques, one of which is indirect probing, and another is a novel "suspicion" mechanism. In the case of a network partition, the parts of the cluster can run in isolation and will recover when the partition heals. If you are interested, the paper referenced there ("SWIM: Scalable Weakly-consistent Infection-style Process Group Membership Protocol"), is the foundation of Serf. In the paper you can find more details about the behavior of the cluster, false positive rates under packet loss, and partition handling.

tl;dr the systems is in fact designed with network errors in mind, as opposed to handling them being an afterthought.

<comment><USER16> What you're saying is it's designed with the knowledge that it's going to cause false positives, and basically doesn't work well under anything more than minimal packet loss. I think this is probably an important factor to note in the description (and I still fail to see how this is considered highly available or fault tolerant, as described in Intro pages)

<comment><USER12> I think we are maybe just working with different definitions. High Availability for Serf means that it can continue to handle changes in topology and deliver user events in the face of node failures and network problems. However, it is inevitable that there will be a degradation in it's performance given network failures. If there are serious packet loss issues, Serf will mark a node as failed.

I'm not saying it "won't work well". It works as it is designed to. It will be available for operations, it will automatically heal when the partition recovers, and the state will be resynchronized with the "failed" nodes. The system will be in an eventually consistent state, which is expressly documented and is it's normal mode of operation.

If you consider 5% packet loss "minimal", I'm not sure what applications you are running. TCP degrades at over 0.1% packet loss, and most UDP streaming protocols have serious degradation over 5%.

<comment><USER16> I'm still confused. You mention resynchronizing when the "partition" "recovers". First, can you clarify what a partition is? Second, can you define "recovery"? I'm not worried about performance degradation, i'm worried about nodes being marked down when they aren't down.

Please correct me if i'm wrong, but it sounds like this software only works reliably when you have two sets of nodes that suddenly can't communicate _at all_ , and are eventually connected. Sometimes that does happen on a real network, but often the cause of failures is intermittent and undetermined for hours, days, or weeks. In this case, how would this program work? Would network nodes keep appearing and disappearing, triggering floods of handler scripts, loading boxes and keeping services unavailable?

Yes, tcp performance does degrade under packet loss. It also continues to operate (at well over 50% loss) and automatically tunes itself to regain performance once degradation ends. And it does not present false positives.

It maintains its own state (ordered delivery), checks its own integrity, stands up to Byzantine events (hacking), and is supported by any platform or application. Unfortunately, due to its highly-available nature, it will eventually report a failure to an application if one exists. But if latency is more of a priority than reliability, UDP-based protocols are more useful.

If you're designing a distributed, decentralized, peer-to-peer network, that's cool! But I personally wouldn't use one to support highly-available network services (which is three out of the five suggested use cases for Serf)

</comment></comment></comment></comment></comment><comment><USER17> They should add "from the folks who brought you Vagrant" to the top of the homepage.

</comment><comment><USER18> can we see serf as a kind of riak core but written in go?

<comment><USER12> Riak Core provides a superset of the features of Serf. Riak Core uses gossip to manage membership, but it also provides quorums for coordination, and is based around the notion of a hash ring and virtual nodes. You could instead using Serf to build riak core like technology on top.

</comment></comment><comment><USER19> Highly available?

<comment><USER12> Yes, the availability of the system is not tied to any given node(s). Any node (or group of nodes) can continue to operate in the face of failure.

</comment></comment><comment><USER20> This isn't much of a solution when it involves putting a binary on every host. Clearly, the best solution is a service framework at the platform level, not a separate, unmodifiable blob thrown on your hardware.</comment>


Pragmatic Functional Programming - <AUTHOR> http://blog.cleancoder.com/uncle-bob/2017/07/11/PragmaticFunctionalProgramming.html <comment><USER2> This article feels kind of like a tease. It mentions all the usual suspects of better code this, and state isolation that. It of course touches on homoiconicity and that code can modify code. It shows a bit of host language interoperability but it never says why any of this Pragmatic beyond the obvious and has just two code examples.

Granted I love Clojure and it does feel very pragmatic but that's more because of the attention to detail put into the language. Examples include the strong host interoperability, mentioned in the article, but also the collections in the standard library and the argument order of functions to allow for things like the threading macros (-> or - >>) work the way they do. Watch any of Rich Hickeys talks (simple made easy or the speculation keynote) and you will get a feel for the thought put into things.

Having said all of this it is important to note that just because of all the praise functional languages are getting doesn't mean there aren't trade-offs. My most recent problem has been dealing with multithreaded map (via Claypoole) calls leading to hanging http requests (in Aleph) and the nice clean layers of abstraction have to be stripped away to assist in debugging the problem.

The grass is always greener on the other side.</comment>


Should I invest $100K in a startup for 3% equity? – pre money valuation is $20M - <AUTHOR>

I'm stuck in a situation. I started consulting for a start up for a fixed dollar amount per month. This startup is owned by 2 partners - one of them is the CEO. Since 2012 when they started it, they both have invested around $350K total. Up until now, they have not asked for funding. They have a monthly revenue of around $20-25K (mostly consulting revenue) with a $30K monthly burn rate. For the 1st time, they went cash flow positive last month. They don't have any full time employees, but have around 10-12 guys working on and off on temp basis - on 1099. Now, they are looking to get funded. There is an investor willing to invest $2M as seed and asking for 10% of equity and a board seat. They are very interested in that offer and considering it. Their cap table looks something like founder shares - 80% and common stock - 20%.
I have been consulting for them for 4 months now. After 2 months, I realized they don't have the cash to pay me for my consulting fees. However, since I believe so strongly on their product / services / business model, I decided to stick. In return, they offered me to do a cash investment around $100K for a 3% equity (undiluted founder shares). With this cash influx, they plan to start paying back my consulting fees arrears. Also, with the $2M funding, they are offering me to become a full time employee with executive role.
I'm looking at this from a long term perspective. What attracts me is the 3% equity in the company and with my 100K investment, they will start paying my consulting fees monthly. I'm not able to decide if this a good deal for me? Also, what happens if the $2M funding does not come through? Ultimately, how can I protect my investment? With my investment, should I also ask to become a board observer? <comment><USER2> First off, you should retain an attorney and ask that attorney all of these same questions. This is non-negotiable. You will want this moving forward.

If the contract for this transaction is sound (refer to the "Get An Attorney" section) then you are buying 3% of the company at $3.34mm valuation and, if the seed money comes through at the amount and percentage you state, you have 6x'ed your investment. That's not a bad deal. However, there are some caveats. It all smells fishy.

First, this company has been in business for 3 years or so and they have not seen a profit until last month? Their valuation is effectively $0, especially if they are in the consulting space and not the product space. They have no assets. They make no money. They have essentially no history of being successful. I am not suggesting that they have no hope of becoming successful, just that they currently should not be valued at $3.34MM. That would be 13 times annual gross? No way.

Second, we can extrapolate that they are not worth $20MM. You mention that they were cash positive last month for the first time. Let's say that means they made $35k on a $30k burn. Let's also say they can keep at that level, month over month. At a $20MM valuation they are being valued at 47 times annual gross? No way. The investment firm will learn this quickly as they do their due diligence.

Third, let's look at debt. They aren't paying you. How many other 1099 contractors, vendors, and others are they behind payments on? Do they account that debt and those payments into their $30k monthly burn? I will not speculate, but if you are planning to invest $100k into them, you should know for sure.

If you were going to invest $100k into a company like this, I would suggest a much lower valuation. I would also suggest that you get a board spot. I would not be shy about asking for a stepped up equity position under various circumstances (The other investor doesn't come through? More equity for you. You spend more time working for the company? More equity for you. etc). Say 10% equity for $100k. another 10% if the seed money doesn't come through. X% annually as you help them grow the business over the next 4 years.

Now, if it were me, I would never invest in this deal. $100k is enough to invest in my own successful business without the baggage. I would leave, invest in my own company, and sue them for my unpaid work. But, that's just me.

</comment><comment><USER3> Obviously you have way more information about this, but judging from what you've disclosed, it's a terrible investment.

Stock in a company is worthless (literally, as in "worth $0") until 2 things happen: * the company gets sold * the company does an IPO

Those are hard for all kinds of companies but next to impossible for a consulting company.

Consulting companies don't scale they way product companies do.

Revenue of $20k/month is nothing (this is what Google spends on a good engineer per month). They don't make enough to pay you a market rate.

How is this company going to ever get to multi-million dollar a year _profit_ (a condition necessary but far from sufficient for a sale or IPO) ?

You seem to be so pre-occupied with details of the deal that you seem to be missing the obvious thing: investing in this company looks like a terrible idea because there's no way this company is going to be so successful to warrant a sale or IPO.

</comment><comment><AUTHOR> Thanks a lot everyone for an amazingly prompt response. I did leave some important details.

The CEO is a serial entrepreneur - he has sold 2 companies to Autodesk and 1 to IBM in the past. He has enabled investors to realize exit profits in the US$180 million range with his founding team member involvement. He is also a highly rated and a well renowned speaker - this is how he gets business. Also, yes - they have been consulting so far but along the way, they have several prototypes developed (to be evolved to full products dependent on funding). I guess thats where the valuation comes in.

Someone1234: I like your idea of not putting all eggs in one basket. I think he did PMV at worthworm.com. Also, the fact that someone is willing to invest $2M - 10 multiple of the seed amount is $20M. Is this how you typically validate the PMV? And yes, one of my incentives to invest is to get myself paid. Is that a good approach?

kkowalczyk : I agree consulting companies cannot scale that much. But, this is positioned to become a product company (dependent on $2M funding) - in that case, what are your thoughts?

Jeffmould: The CEO has confirmed I will be at the top of the stack. Future funding will not dilute my equity. I like your idea of convertible debt to safeguard my investment. In fact, the CEO did mention this to me - he will take this investment as a loan to be returned to me - maybe he meant convertible debt? Is there anything more I should watchout for?

auganov: Thanks for your honest answer. I don't blame you as I left out the relevant details earlier. But how can I ask for 15% for only $100K when Mr. $2M is asking only for 10%?

lxfontes: Yes, if the $2M seed comes through, they have offered me a full time employment with some common stock but without the founder shares. Founder shares will only be given with cash investment.

<comment><USER4> Thinking about this even more, the company is only cash flow positive because they are not paying you. And if they are not paying you, the question begs who else are they not paying.

Just because the CEO has sold multiple companies and realized profits for previous investors, does not automatically make this a golden egg as well.

Finally, just because the CEO says you are at the top of the stack and future funding will not dilute you, does not mean it will not happen. In fact it is fairly certain to happen if additional, bigger investors come on board. If you want to get an idea of the dilution, Google an Excel cap table example (there's actually a pretty good one on [http://cooleygo.com](http://cooleygo.com)) and play with the numbers a bit.

I would honestly skip the whole deal, push for either them to pay you what you are owed to date or go with a convertible note. I would also talk to an attorney before I made any decision. It just seems to me that the CEO is trying to make it sound like a great deal with no risk, when in essence the risk is enormous.

</comment></comment><comment><USER5> What kind of product is it?

A consulting business isn't really worth what you are proposing -$300k a year with a valuation of > $1m.

If I am understanding you right, if you invest $100k into the company, they would use that money to just pay back your consulting fees right? Why not have them compensate you outright stock and forgo cash compensation? It seems like a waste of time to put cash into the company and then pull the cash back out.

Also if you are concerned with them not raising the funding you could just make your investment contingent on the raise. Basically say to them "assuming that we can get this round done, I'm in for $100k". As an entrepreneur I hate this (makes me think you are betting against the company) but I could be persuaded if you picked up the axe and charged into battle with me. So make sure that if you do this you are busing ass trying to make the round happen.

</comment><comment><USER4> IANAL, or an accountant for that matter, but something does not add up or you are leaving a lot of detail out. If you invest $100K for 3%, then the investor comes on at $2M for 10% you are going to take a hefty dilution from that. Without knowing all the details of the company or who/what they do, I am skeptical that a company only generating $20-25K monthly from consulting revenues is getting a $20M valuation. That seems a bit high for a consulting company that has been in business for 3 years doing less than $500,000 annual sales.

From the sounds of it, they are only asking you to invest so they can turn around and pay you from your investment.

With that in mind, you may want to consider doing a convertible debt and if/when investors come on you have an option to convert to equity or collect what is owed to you. And if the investor does not come through, the company still has a debt to you.

</comment><comment><USER6> If he has had multiple exits worth any considerable cash, why doesn't he just put in the money himself instead of getting investors, especially an investor like you. Seems like he himself is not confident in his idea and doesn't want to risk his own money in the venture.

<comment><USER7> > especially an investor like you

This is a good point, at 100k for a round you're very friends and family sized, and the responsible question is, are you a qualified (legally and financially planning wise) to lose this much money.

</comment></comment><comment><USER8> They're certainly making you a better offer than the $2m/10% individual. That person is effectively paying $200K/1%, as opposed to your $100K/3% deal.

I would say consider this two separate deals, don't conflate them(!):

1) The investment: You're out $100K and have a fairly risky investment in this company that could be worth $0 one day (and no set payback date). $20m valuation is meaningless, it just helps you vaguely judge risk, but not much (also who did the valuation? How confident are you in them? Who is the 2m investor? Do they even exist?).

2) The full time employment: A full time employment offer, when or if, that happens should be considered on its own merit (although that alone sounds almost like a further investment as I imagine they're going to "under" pay you since you're now a stakeholder, or worse expect free work)

The ultimately question you should be asking is: Can you afford to lose 100K outright? Because that is on the table. Could you put that 100k on black on the roulette wheel, and risk a 50/50 shot of losing it? Or worse on a 1/16 or 1/32 shot. Because that is the reality.

To be honest I'd consider taking a 3%/100K if I could afford to lose (completely) 100K (and the company had good prospects). But I wouldn't work for them below market rates, because then the risk has risen too much, and it sounds like you do pretty well anyway on your consulting.

So if I did the investment I would then go and make money elsewhere, so that even if this company burns to the ground, my maximum liability at that stage is "only" 100K, not a penny more.

Working for them for 50% of normal or worse not getting paid reliably (as now) is just more and more investment and before you know it you might be throwing another 100k or 200K after your initial investment just from lost wages alone. Essentially trapping you into more and more risk.

So look at risking it on the company but curtail the level of risk. You shouldn't be in as deeply as a founder, your potential compensation isn't high enough.

</comment><comment><USER9> No.

The $2M is complete bollocks and they need the 100k to keep going until they can find some actual investment. Otherwise they'd be paying your bills.

So tell them you'll consider it once they pay you - and get some kinda signed statement that they do, in fact, owe you for two months consulting.

Frankly, it smells bad and for me that's enough.

</comment><comment><USER10> short answer: I wouldn't do it.

1\. I would ask for my unpaid consulting fees

2\. Ask if they want me to join without the 100k down and take less % on shares

The rationale here is:

\- Low monthly revenue for a long time

\- Not paying contractors is bad

\- How many other people have this 'offer' on the table?

If I were to put 100k of my money somewhere, it would be on a 1-year sabbatical and start my own thing.

</comment><comment><USER11> I would not give $100k to anyone who didn't pay me and didn't tell me until after I had done the work.

You could cut out the middle man and just pay yourself. Anyway 3% isn't a controlling interest and you are entirely dependent on their goodwill in any liquidity event.

</comment><comment><USER12> Investing on the hope that they will use the money to repay what they owe you is pants-on-head retarded.

Option #1 - convert the debt to equity at the proposed valuation.

Option #2 - get paid, don't invest

Option #3 - expect to never get paid - walk away or continue for fun

Don't give those fools money. Working for 2 months without getting paid is enough of a gift.

</comment><comment><USER13> OMG NO.

3% would be a bad deal even for a regular employee considering they don't have money to pay you. If you really want to do it you should ask for 15%+. But please, don't.

Do you even trust these people? Unless we're missing some important information they sound either ignorant or shady.</comment>


Our corporate WebSite runs on Clojure and Google App Engine - <AUTHOR> http://www.hackers-with-attitude.com/2009/09/our-corporate-website-runs-on-clojure.html <comment><USER2> That headline didn't kick me much (well, so you can build a website on clojure, duh!) but while clicking through that site I was fascinated by their writing about the company philosophy.

[http://www.freiheit.com/tag/Jobs#ag9mZGN3ZWJzaXRldGVzdDFyDQs...](http://www.freiheit.com/tag/Jobs#ag9mZGN3ZWJzaXRldGVzdDFyDQsSB2FydGljbGUYJww)

The english translation is horrible but here some highlights from the Jobs- page:

* The day begins at 9:00AM, not later, preferably earlier.

* Release dates are never(!) shifted.

* We believe in the "Broken Window"-theory. Companies degenerate, too. That begins with a paperclip laying around. Consequently every employee must _empty_ their desk before they leave.

Along these lines there's this cultish subtone in all of their writing which is hard to translate. Reading their blurbs gave me the chills like no other company description I've ever read. Definitely has a "zombie B-movie" taste to it.

<comment><USER3> I don't really have a problem with that. I'm self-employed most of the times, but during the occasional "sortof hired" periods I like to take work ethics seriously. And I definitely don't mind a nice quiet clean office - again, not at all something I'd do on my own, but I definitely wouldn't mind such a policy. There is a bit of zen in 5 minutes of cleaning up at the end of the day.

</comment></comment><comment><USER4> Does anybody _actually_ like tag clouds? I've always wondered.

<comment><USER5> I've never even clicked a tag cloud.

<comment><USER4> I clicked a tag cloud once. It wasn't that great.

</comment></comment></comment><comment><USER6> I assume they have a cron job running to keep the site alive - Google App Engine for Java has a nasty habit of putting your app to sleep and then it takes 20 or 30 seconds to wake it up (at least that's what happens with JRuby - I don't know about Clojure).

</comment><comment><USER5> Is it me or is it really slow?

</comment><comment><USER7> I'm curoius why everyone resizes images when they're uploaded.

I think the better way is not in any way changing the uploaded file, instead resizing it when it's actually requested by a client. This way, your pages can ask server for whatever actual size they need in the moment instead of setting image size once in stone. Like, if you've uploaded kitty.jpg, you can then ask for kitty_120x50.jpg or kitty_300x.jpg or kitty_x200.jpg (with missing dimension being calculated to preserve ratio).

If course, resized files are stored along with the original to avoid costly resizing process each time. If there's already a copy with required sizes, it's served really, really fast.

<comment><USER8> I think amazon.com does something like this. I read an article that dissected their product image urls to explain how their image service worked but I can't find it now. If I remember correctly the service could resize/rotate, add sale "stickers," and a bunch of other stuff.

EDIT found the article: <http://aaugh.com/imageabuse.html>

<comment><USER7> Yeah, adding stickers on the fly is also the way to go.</comment></comment></comment>


SublimeCodeIntel - code intelligence and autocomplete for Sublime Text 2 - <AUTHOR> https://github.com/Kronuz/SublimeCodeIntel <comment><USER2> I've been using it for the past two weeks. It's very buggy but quite good when it works. I'm using the most recent version from git.

To give a few examples of the bugs I encounter every day:

1\. If you use it to auto-complete css files and you want to type "text- shadow" the moment you hit the dash the text gets replaced with "text-align". So if you type quickly you end up with "text-alignshadow:". The second time you type text-shadow it does work. This goes wrong for all definitions that contain a dash because SublimeCodeIntel (or the libraries it depends on) considers a dash a separator and separators trigger code completition.

2\. It autocompletes on empty matches. If you start typing and for one reason or another the auto complete window pops up (and it does so automatically) and you hit space, or colon, any other separator what you just typed gets replaced with the topmost match in the completion list (even if 0 characters match).

3\. I think it messes with my undo/navigation buffers, but I'm not quite sure.

4\. The python completion crawler keeps reindexing and reindexing. So in practice the Python completion is nearly useless. It also doesn't give hints for parameter order (which I consider the most valuable hints by far) even though Komodo does (the library is ported from Open Komodo afaict)

5\. ST crashes a few times a day after installing this plugin. Not 100% sure if it's to blame, though.

6\. Unlike Komodo it doesn't index jQuery properly.

These issues aren't dealbreakers though. I just hit escape whenever the auto- complete window pops up.

<comment><USER3> Some of these issues arise because SublimeCodeIntel triggers the auto-complete prompt automatically, which it wasn't designed for. I'm planning to rework the auto-complete system at some point to make it more appropriate for this scenario.

</comment></comment><comment><USER4> This is pretty slick!

Screenshot if anybody wants to see what it looks like: <http://i.imgur.com/BH3yd.png>

<comment><USER5> Is it pretty fast? Any lag?

<comment><USER4> Feels instantaneous to me. Application startup time is still nice and fast too.

</comment></comment></comment><comment><USER6> Very exciting, loving sublime text2. I think once this is stable it will stop any reasons to switch to a full blown ide. I think I need to fork this and add scala support!

<comment><USER7> Still need a debugger :) Too bad MacGDBp is pretty much unusable.

<comment><USER8> Why is that? I use it. I admit it's not "full featured" but it does get the job done.

<comment><USER7> I have issues getting it to work consistently. Also the fact that it isn't tightly coupled with my editor makes it very inefficient for me (needing to open up the file i want to debug in MacGDBp, and go back to my editor to make changes). Perhaps it just works differently than what I'm used to from the IDEs -- any tips or tricks would be appreciated because I'd rather use ST2 + a debugger than a big bloaty IDE.

</comment></comment></comment></comment><comment><USER9> I've been trying out Sublime for a few days, but it will take a long time for me to switch to it from Notepad++; certain things I use a lot (code folding, XML completion, byte editor, almost everything on the np++ Tools menu) appear absent from Sublime, but maybe I am missing a wonderful world of optional plugins. Thoughts? The l33t color schemes and text map are nice and all...

<comment><USER10> Code folding is available since Build 2126

</comment></comment><comment><USER11> This is really nice but wasn't this already linked somewhere on here like a day ago?

I haven't been able to try it out much besides playing around in an old ruby project. It doesn't really seem to do much unless you are coding pure ruby but it's convenient enough.

makes Sublime Text 2 even better and I like that :-)

</comment><comment><USER12> SublimeCodeIntel and SublimeLinter are very buggy. Try opening a semi~big file and try to type something...

My suggestion to SublimeCodeIntel developer is: write your own version!! Code Intel even fails on Komodo.

Sublime Text is just awesome! <3

</comment><comment><USER13> Does every text editor expand until it becomes an IDE?

</comment><comment><USER14> I can now start my Monday very happy.

</comment><comment><USER15> oh man, I JUST switched to sublime this afternoon and was wondering if this was possible... DESTINY!!!

</comment><comment><USER16> <3 Sublime Text 2</comment>


First look: latest Google Android SDK a big improvement - <AUTHOR> http://arstechnica.com/news.ars/post/20080217-first-look-latest-google-android-sdk-a-big-improvement.html

<comment><USER2> Google might be a little to late. The battle will be between apple and google on the hand held front. However apple will need to crank out a new iphone design if they want to compete with the new android phones that will be coming out. Consumers have grown to think of any phone over a year old an old model and inferior model.</comment>


Ask HN: What are your biggest pain points as a data scientist? - <AUTHOR> Some examples, productionizing code, cleaning data, documentation etc. <comment><USER2> 1\. Dealing with biased data

2\. Cleaning/understanding data - nearly all data sets I've used have duplicates, missing data, highly anomalous distributions in some fields that indicate we aren't measuring what we think we are, etc. So a lot of my time is spent figuring out what's going on in the data, cleaning up the issues, figure out what subset of the data is reliable, and then dealing with the biases introduced by what is missing or wrong.

3\. Dealing with people who don't understand or respect statistics and data science. For example, I've been brought in to "do the analysis" on an "A/B test" where a team didn't appropriately randomize their samples, and also hadn't done a statistical power test beforehand so had an underpowered test anyway, so there was just no hope of validating that their change was an improvement.

<comment><USER3> I tried scaling the task of #2 across multiple people recently. It took way longer than had I personally done it myself.

I want to know if there is a way to spread the load for this.

</comment></comment><comment><USER4> Getting the dimensionality correct between layers in an untyped language is incredibly painful. Getting a prototype working on my personal machine and then having to play AMI roulette on AWS to get it running on a GPU is frustrating and expensive. Every time I read the tensorflow documentation I think, "I wish i could pay 500 dollars for a version of this that looked like someone cared about it."

<comment><USER5> You can skip AMI roulette by using AWS Deep Learning AMIs ([https://aws.amazon.com/machine- learning/amis/](https://aws.amazon.com/machine-learning/amis/)) or the new container equivalent ([https://aws.amazon.com/machine- learning/containers/](https://aws.amazon.com/machine-learning/containers/))

</comment></comment><comment><USER6> That other groups in the business want simple, black and white answers to very complex questions.

An expectation that data can eliminate the need for reason and thought is problematic.

I have tried to communicate the reasoning for things like judgemental forecasting but success is hard to achieve.

<comment><USER2> Agreed. Many people don't want to think, they only want to know.

One of the things my team does is build data tools. The number of people who want to take data they have hardly looked at, put it through a tool they don't understand, and rely in important ways on the output, is astonishing to me.

</comment></comment><comment><USER7> Having high expectations by stakeholders on some 'AI' magic when a heuristic will get us 80% there in 10% of the time

<comment><USER8> IME a number of Data Scientists reinforce this thinking. When people think you are are a genius, it's painful to admit that 80% of project goals can be achieved with a simple heuristic.

It's incredible how many "Data Science" problems can be solved with a better dashboard that enables people to look at data in a more useful way.

</comment></comment><comment><USER9> That it is 90% cleaning, and only 10% modeling. Never had a dataset that was ready to use like the ones on Kaggle. Most of the time I get a mixture of Excel Sheets with weird formatting, .csv files and SQL dumps that have questionably encoding, and lots of unnecessary information and missing values.

<comment><USER8> I think Kaggle participants miss out on some of the best parts of being a Data Scientist. Fiddling with data, writing scripts to clean/transform/ingest data, interacting with data owners and subject matter experts, etc. IME that's actually a lot more fun than fiddling with parameters and looking at model performance metrics.

</comment></comment><comment><USER10> Getting lots of data (for deep learning models), cleaning that data, and labeling it.

</comment><comment><USER11> what about playing with different hyperparameters? I always found that time consuming? What do you guys think?

</comment><comment> Iwan-Zotow Data!</comment>


One Day Only: A Chance to View One Map to Rule Them All - <AUTHOR> http://www.smithsonianmag.com/smart-news/one-day-only-chance-view-one-map-rule-them-all-180959391/?no-ist <comment><USER2> Discussion of the map's discovery: [https://news.ycombinator.com/item?id=10440402](https://news.ycombinator.com/item?id=10440402)</comment>


HP ARM netbook, AirLife 100 (android, touchscreen, 3G-always-on) - <AUTHOR> http://www.linuxfordevices.com/c/a/News/HP-Compaq-AirLife-100-on-US-website/

<comment><USER2> I would be quite sad if they delayed the release of this due to the Palm/webOS acquisition, because it looks really nice.

Relevant page on HP's site: [http://h10025.www1.hp.com/ewfrf/wc/document?docname=c0208419...](http://h10025.www1.hp.com/ewfrf/wc/document?docname=c02084190&cc=us&dlc=en&lc=en&jumpid=reg_R1002_USEN#N1200)

There's Flash Lite 3 support, but I don't see a mention of the Market.

</comment><comment><USER3> "...the AirLife 100 is ready to operate on the network of any U.S. carrier that chooses to subsidize the device. "

Does it really have to be subsidized by a carrier to run there?

</comment><comment><USER4> I would love an ARM development machine.

How hard will it be to blank this thing and put Arch on it?

<comment><USER5> My thoughts exactly.

I remember a youtube video I saw recently in which a Lenovo sales guy tells the interviewer (at some trade show) that the software environment for their Skylight ARM-based netbook woulb be entirely proprietary, and that they would even push upgrades to devices in the field. While it doesn't exclude the possibility, I think this attitude makes it less likely that Lenovo will look kindly upon hackers loading their own OSs. Hopefully HP will take a less hacker-hostile approach.

And if anyone could make an ARM netbook with a Pixel Qi screen, that would be out of this world.

</comment></comment><comment><USER6> Anyone know how they deal with apps not designed for the 1024x600 display? Do they upscale? or does Android take care of this automatically?

</comment><comment><USER7> i must say id be very interested in seeing a android netbook.. and id love to play with it, but i cant imagine it being that usefu, i think it would be rather slow and probaby just like a nexus one or a droidl, i hope im wrong because id really like to encourage the use android and the upcoming meego.

<comment><USER8> I'd just rip Android out and install a vanilla Ubuntu ARM desktop.

I am more than happy with Ubuntu running on an Intel Atom. I guess I wouldn't be that much disappointed at it running on ARM.

<comment><USER9> > 16MB of flash storage

You're going to need to upgrade first.

<comment><AUTHOR> It's 16GB.

Flash is incredibly cheap now, you can get 4GB for $12 at the supermarket (that's AUD; pr $10 USD)

<comment><USER9> They must have fixed the typo. You're right of course. I thought it was a little strange to have so little memory.</comment></comment></comment></comment></comment>


Not sure if this means Media Temple will stop storing passwords in plain text - <AUTHOR>

Given that Media Temple's recent security breach was caused by storing plain text passwords (http://michaeltorbert.com/blog/media-temple-hacked/), I'm not sure what their new "Support PINs" accomplish that makes them more secure than a SHA1 hash.
Today's email from Media Temple:
(mt) Media Temple has enhanced the security of your account with a new feature called "Support PINs". This new feature was requested by users like you desiring stronger measures for the verbal privacy of their passwords.
"Support PINs" allow you to optionally generate a temporary pin code inside the AccountCenter that will permit you access to (mt)'s Customer Center. This measure keeps primary passwords verbally and electronically private, and encrypted at all times.
You can learn how to use Support PINs by visiting our KnowledgeBase. <comment><USER2> The Support PIN's are another option for accessing our customer support over the phone without the need to share your Account Center password.

Support PIN's are temporary and expire shortly, allowing them to be a secure method of authenticating a customer when calling in for support.

We've created PIN's in an ongoing security initiative and to address the desires of our customers.

You can simply login to your AC and generate a temporary PIN to share with our support staff.

<comment><USER3> Yes, but are you still storing passwords in plain text, or just removing the idiotic need for your support staff to see them in plain text?

<comment><USER4> passwords are no longer stored in plain text</comment></comment></comment>


How Everything Became the Culture War - <AUTHOR> https://www.politico.com/magazine/story/2018/11/02/culture-war-liberals-conservatives-trump-2018-222095 <comment><USER2> >Politics has always been adversarial. Traditionally, though, we've had a fairly robust national consensus about a fairly broad set of goals—a strong defense, a decent safety net,

What? The "consensus" on the safety net is 70 years old. There are people alive today who were around when senators were denouncing Social Security as vile Bolshevism.</comment>


The Scotch programming language - my Christmas break 2010 project - <AUTHOR> http://www.bendmorris.com/2010/12/intro-to-scotch.html

<comment><USER2> This looks like a really cool project/language. How do I build it? Does it build on OS X?

\--

I'd love to "watch" the project on github, but alas it's stranded on the island of google code. Check out this [1] basho blog entry on why they chose github over bitbucket — most of the reasoning applies to google code as well. I'm not affiliated with github, by the way, just a happy user.

For a project like this, the concept of "watchers" isn't a vanity metric. I've been working on a lisp->javascript language and have found github watchers to be surprisingly motivating and encouraging.

[1] [http://blog.basho.com/2010/11/11/a-few-more-details-on- why-w...](http://blog.basho.com/2010/11/11/a-few-more-details-on-why-we- switched-to-github/)

<comment><AUTHOR> Unfortunately, I don't have a Mac, so my ability to support Mac users is limited at the moment. Scotch should build from source just fine if you install GHC and use Haskell's Cabal tool to download a few extra packages - haskeline and executable-path are the two I can think of at the moment that are needed.

The command to build is ghc --make scotch. Just run the executable that's produced. Let me know if you run into any problems and I'll see if I can help.

As far as Github, never used it, as I've always been a SVN guy - I'll have to look into it after the holidays are over. Thanks for the suggestion.

<comment><USER3> cool stuff! got it working under os x. deps are: haskeline, executable-path and Decimal

</comment></comment></comment><comment><USER4> This is kind of neat. It seems like it'd be useful as a more easily debugged Haskell.

I'd be more inclined to actually play with it, though, if it accepted a superset of Haskell, which allowed arbitrary IO tainting. So I might write:

        do
            pure code
            {- Scotch debug statements
            ...
            -}
            pure code

And then just dump it into the Scotch compiler/interpreter when I ran into issues.

Of course, this would only work if you could guarantee semantic equivalence to the Haskell except in those debug bits.

<comment><USER5> "A superset of Haskell, which allowed arbitrary IO tainting" doesn't really make any sense with Haskell's semantics, unless you're talking about unsafePerformIO, which already exists.

</comment></comment><comment><USER6> Wow. That has some awesome concepts. Keep up the good work!</comment>


Facebook releases Tornado (Python web server that powers FriendFeed) - <AUTHOR> http://developers.facebook.com/news.php?blog=1&story=301

<comment><USER2> Hah, love the change in shade of blue on the graph to match facebook's - interesting touch.

<comment><USER3> I'm not sure why andreyf was being down-modded, the graph on the original blog post at <http://bret.appspot.com/entry/tornado-web-server> uses a subtly different shade of blue. This is a clear attention to details that is worthy of at least a small note in a community such as ours :-)

<comment><USER2> Yeah, I should have explained the context of that remark better. Thanks for fishing me out of downvote land :)

</comment></comment></comment><comment><USER4> Chat demo running live: <http://chan.friendfeed.com:8888/>

Bret's blog post: <http://bret.appspot.com/entry/tornado-web-server>

</comment><comment><AUTHOR> Source: <http://github.com/facebook/tornado/tree/master>

Documentation: <http://www.tornadoweb.org/documentation>

</comment><comment><USER5> <http://news.ycombinator.com/item?id=815681>

</comment><comment><USER6> nice</comment>


New fast and free Advanced Port Scanner released. Try it now - <AUTHOR> http://www.advanced-port-scanner.com <comment><USER2> Nice booth babe.

The page is too slick to trust. And there are any number of open source port scanning tools.</comment>


TensorFlow Tutorial and Examples for beginners - <AUTHOR> https://github.com/aymericdamien/TensorFlow-Examples <comment><USER2> Related: today only, Packt Pub's book, _Getting Started With TensorFlow_ , is the free daily e-book:

[https://www.packtpub.com/packt/offers/free- learning](https://www.packtpub.com/packt/offers/free-learning)

(You need to set up an account on packtpub.com to participate; in addition to allowing downloads, books are available in one's account to be read online).

The book will be available through Thurs., Aug. 31 till 4PM PDT (i.e. midnight UTC), when the free daily e-book changes.

<comment><USER3> keep in mind there is lots of bad reviews about this book

</comment></comment><comment><USER4> As someone who utilizes machine learning but doesn't have a great understanding of neural networks, I've not had good experiences with TensorFlow. Myself and a friend attempted to use it for a project but found a lack of examples and the documentation to be very unclear. Even doing basic things for which tutorials did exist presented us with lots of warnings and errors. Debugging one of these, for example, led us to believe that certain functions were incomplete or abandoned, without clear messaging in the documentation that this was he case.

For those interested in diving into machine learning, I'd recommend not starting with neural networks. I think there's a belief amongst those unfamiliar that a neural net will be 10x better than anything else, but this is not always the case, and you can accomplish a lot with simple functionality found in scikit.

TLDR, glad to see more tutorials!

<comment><USER5> Agreed about scikit-learn. Anecdotally I've found that most "deep learning" and "big data" problems I see in industry could be trivially solved using scikit-learn on a reasonably powerful laptop.

Also if you want to really understand neural networks I can recommend Andrew Ng's new deep learning course on Coursea. You start with deriving the underlying math and implementing a simple (but fully functional) neural network using just basic numpy, so that when you get to tensor flow and the latest cutting edge techniques you actually understand what's happening under the hood and how things actually work.

</comment></comment><comment><USER6> A little rant from someone blocked learning.

To me the irritating point is all the tutorials with MNIST, but I'm not able to find only one explaining how to prep a good dataset to use with it (the one in section 5 is ok, but I feel it is not really practical?).

Neither one about trying to classify something literary from scratch.

Please stop with MNIST sample and character recognition ones...

</comment><comment><USER7> Really excited about how much clear and concise documentation is available here. I've been considering looking to machine learning and tensorflow to at least get the concepts. This is just what I need to push me over the edge and get started

</comment><comment><USER8> Has anyone used TensorFlow for any hobby projects? I would be interested to know what kind and how it worked out. This is a pretty neat tutorial, I'll have to set aside more time to look it over.

</comment><comment><USER9> Nice. Examples with comments are always appreciated.</comment>


Ask HN: Anyone with music projects looking for co-founders? - <AUTHOR> Hey guys,
Anyone here with music projects looking for co-founders?
Let me know :) <comment><USER2> Soon...

</comment><comment><USER3> yep!</comment>


Haxe 4.0.0 preview 2: final keyword, arrow functions - <AUTHOR> https://haxe.org/download/version/4.0.0-preview.2/ <comment><USER2> Haxe, hashlink, heaps... This stuff is just amazing and if it would have been backed by a large corporation, it would be a top 3 ecosystem.

<comment><USER3> I tend to feel that not being backed by a large corporation is a point in Haxe's favour.

It's making the argument that it should be used because it is good. Improvements to the language have to stand on their own merit rather than what would drive the parent coproration's agenda.

</comment></comment><comment><USER4> Haxe has a nice site for discussing upcoming features: [https://github.com/HaxeFoundation/haxe- evolution](https://github.com/HaxeFoundation/haxe-evolution)

The pr page has the list of ongoing discussions: [https://github.com/HaxeFoundation/haxe- evolution/pulls](https://github.com/HaxeFoundation/haxe-evolution/pulls)

The issues page is a sort of staging ground for complex gripes that don't have straightforward solutions yet.

It's a great community, I've been part of it for years and recently joined the compiler team.

</comment><comment><USER5> Looked at their site the other day and saw they got string interpolation backwards from the shell ("" vs ''), how is that possible?

<comment><USER6> IIRC, you can use either. Whatever quote you used to start the string is the one that will end the string.

var foo:String = "foo"; // foo

var bar:String = 'bar'; // bar

var fooquotebar:String = "foo'bar"; // foo'bar

<comment><USER7> That doesn't cover the interpolation part of it, though.

        var z = "This is just a literal value";

        var q = 'This automatically runs Std.format, letting me include "$z"';

</comment></comment></comment><comment><USER8> Anyone used Haxe in a server?

<comment><USER9> There are several ways to do it: compile to C++, compile to the Neko VM, or use Node (there are official bindings: [https://github.com/HaxeFoundation/hxnodejs](https://github.com/HaxeFoundation/hxnodejs)) In development it's Haxe (which is a lovely language to work with) but in production it can be something you're familiar with.

<comment><USER8> I like Haxe. I coded in AS3 for years.

What I wanted to know was if anyone could share their experience when running a Haxe target on a server.</comment></comment></comment>


Was the containers hype an industry/tech media propaganda? - <AUTHOR> "It happened all of a sudden, and everyone now wants to be a part of it. Even the companies that have years of experience in the virtualization world failed to capitalize on this movement that doesn't seem to have a clear starting point"
I am becoming skeptical of the tech media and its role in driving the tech industry, especially the software aspect of it. Every technology or tool that we use, every meetup or conference that we go to is driven underneath with an agenda to sell the product, and not necessarily help engineers work.
I see no point in studying any technology, tooling than the toolings my organizations use because all these new tech is in someway linked to drive adoption of companies driving those technology and not necessarily improve the industry in general.
This is a cause of concern for me, because if the software that I'm using and promoting is low-key being used to drive someone's agenda, that I'm becoming part of that hidden-agenda too. It feels these organization are using us engineers as a medium to promote and drive their idea and it is eating me alive.
What does the HN community this about this? And how should I articulate this problem? <comment><USER2> I worked for a startup roughly when docker started gaining proper momentum but at the time I felt it's still not mature enough to use it in production. We was using (I believe) less secure chroot jails to package our app and, well, it was a bit of a nightmare. I was in need for containers technology and once we decided to move to docker I never regreted this move. Now I use podman for I'd prefer not to have to run containers as very privileged user and I don't see better alternative.

There was good press around docker but I also heard lots of stories about problems from my colleagues who decided to use it earlier than me, but I'd not call this propaganda. Docker solves real problem and that's why it got so much attention.

</comment><comment><USER3> Everything starts as propaganda, simply because only companies (big ones) have the PR budget to push new tech.

Also, tech companies need constant change to survive, since with every new change, all the old systems become legacy, and we have a new wave of spending (this is true for hardware as well as software).

However, this is not necessarily bad, you just need to play this game correctly.

For example, if you are a startup, you can use this change as an entry point when new markets get opened.

As an engineer you can also enjoy it if you ride this wave. For example, become a Kubernetes consultant.

The holy grail, however, is the actually drive this change as a startup. I.e. use the PR budget of the big tech in order to introduce a competing product.

<comment><AUTHOR> Thanks for the response. But the problem is it takes sometime to gain expertise and consultant firms. And it is very difficult to foresee a tech disruption as it is primarily driven by big-tech's agenda rather than of introducing new tools and other firms just blindly accepting it as standard.

<comment><USER3> No problem. So it is actually become easier over time, as the IT field concentrates around Google, Amazon, and Microsoft. Basically look for the new announcements in each major conference, and specifically for new beta services, or open-source projects.

Also, it takes around a year to become proficient in new tech, so you must allocate some of your time (at work or free time) to such experimental bets.

</comment></comment></comment><comment><USER4> The media's job is not to inform you at this point. Their job is to get eyes on their product. That's no secret. However, the logic behind your approach is flawed.

Why? Money is king in every industry. Sure, hype exists. I had one instance where I felt exactly like you, and guess what hot new flavor of the month that was? React. It didn't take long to realize who the idiot was in that situation (me). It was hot because it was the next logical step, and no amount of being frustrated because I had just gotten comfortable with Angular could change that.

Containers aren't some snake oil racket. It's pretty easy to see why if you take off the tinfoil. They are cost and resource efficient, and they provide a large cushion for less than optimal situations. Containers are smaller than VMs and less resource intensive so they allow you to do more with less hardware. Sudden rush of people because Reddit or HN fell in love with you? Provision a few more with almost zero effort knowing exactly what you'll be getting. Working on bug fixes or improving shit? Cool. There's no difference between the local instances and what hits production.

I mean, it's kind of like the folks who get real passionate about only using basic code editors because all the extra tools are "cheating". Time is money, and it's the only thing we can't get back. If containers significantly reduce cost in terms of resources, man hours, or money for you or the company you work for, it doesn't take a rocket scientist to understand why it'd be pushed so hard.

I understand being cynical in this industry, but let's be realistic. Companies are only pushing what they believe will be good for their bottom line. No one's pushing containers because of some shitty agenda, just as they weren't when Node.js was blowing up to the same screams of "this is all manufactured hype".

There are definitely cons to using containers. You're pushing a bunch of shit with root access all sharing the kernel out there, and if a single one is compromised, they're all compromised by definition. That's a legitimate concern, but if I'm being honest, this sounds more like you haven't been in a situation where containers were the proper solution more than a tech industry scam to push an agenda.

</comment><comment><USER5> What is eating you alive? You're likely working for an organisation trying to push their software or services onto other people.

Open source adovocacy has the same kind of ideas from another perspective.

There isn't some magic ideal of an engineer. You solve problems. Those problems are generally driven by the agenda of whoever is paying for it. Unless you're running your own business then it's your agenda you're pushing.

If you don't want to use containers, then don't use them. VMs or bare metal haven't disappeared. Any application that's container deployable should be deployable outside of a container.

"I see no point in studying any technology, tooling than the toolings my organization"

That's a fine perspective to have. Keeping up with technology can by very time consuming, and we work to live rather than live to work.

However if you want to be involved in decision making processes. You should have an understanding of the solutions that you don't want to recommend.

Saying "I don't like how the press have been pushing it everywhere and I think there's an agenda" is not an acceptable argument if you're asked about the possibility of contsinerising your company's application.

<comment><USER6> > "I see no point in studying any technology, tooling than the toolings my > organization"

My personal take on this is that the opposite should be done -- study and understand the fundamental problems tech orgs have and know the tools that solve those problems. That's the way to cut through the media BS vs. valuable news.

Containers became popular because they bring value to tech orgs/teams -- removing the need for a LOT of communication/coordination between infra teams and dev teams where skills are generally different.

</comment></comment><comment><USER7> Following that throught process everything will look like a propoganda.

I suggest you to read and listen to everything (all propoganda) and make your own choice.

For containers, I don't use containers because everyone thinks its great, but because it makes my own life easier.

Similarly I could say functional programming is a propoganda, but I haven't picked it up yet since I don't see that much of pros.

<comment><USER8> _> Following that throught process everything will look like a propoganda._

Almost everything is. In tech industry and tech media in particular. PR is essentially another name for propaganda, because propaganda got bad name at one point and propaganda firms had to rename it to continue those practices. Evangelist and advocate roles are essentially people employed to spread propaganda. Even just the idea of startup growth is deeply tied to propaganda.

 _> For containers, I don't use containers because everyone thinks its great, but because it makes my own life easier._

That's not really true. They help a bit with certain problems, introduce other problems, and container-based technology implementations pile on so many problems and look so incompetent, it's hard to even take any of it seriously.

<comment><USER7> > That's not really true.

Haha, are you telling me my perception of my life getting easier is not really true?

<comment><USER9> He means you're statement isn't objectively true.

(But I would say that containers used right, do objectively make our lives easier.)

</comment></comment></comment></comment><comment><USER10> Yes, this is pretty much how it is. The best solution will be passed over time and time again if there is a better marketed alternative. To make things worse you have management adopting product X because they know engineers want to work on it, and engineers adopting product X because they know there aren't any nasty-arse legacy maintenance projects using it. This is but one of the mechanisms that keeps our industry as the expensive, buggy, under performing mess it is.

To articulate the problem: The tech industry is made of people, and people's actions are based on all sorts of not-all-that-logical incentives. Principally tribalism. We evolved this way and it's not going anywhere.

</comment><comment><USER11> Maybe docker had a little bit of hype? But openvz n lxc have been around for a long time..

<comment><AUTHOR> Was there a reason why they didn't take off like docker did?

<comment><USER12> IIRC, the main problem was UI/UX. Docker containers in essence are a bit like running isolated Linux processes in a chrooted and cgrouped environment. Chroot exists for decades, and so does cgroups. Yet, Docker came up with a UI and UX which, among other things, let you download and run isolated Linux processes by simply running 'docker run <container name>', thus shoving everything well under the rug and practically eliminating the need to pull any arcane trick to get Linux processes to run in isolation.

<comment><USER13> running processes in isolation has been happening for a long time. (jails for instance, have existed for a very long time and freebsd has been used a lot for hosting services in the past).

while i agree docker solved the UI/UX problem, it also introducted a massive security risk, as running unverified binaries in production is very easy for developers. Most shops don't have any private repository in use nor do they have the people (anymore) to build proper processes and systems around running code in production.

<comment><USER12> > also introducted a massive security risk, as running unverified binaries in > production is very easy for developers.

Truth be told, that has nothing to do with Docker and everything to do with how said teams operate.

</comment></comment></comment></comment></comment><comment><USER14> Some news in media are published with money. You send your story and pay some money, then they publish it as a news.

</comment><comment><USER15> Just sit and watch the trends roll in, and fizzle out. Choose your wave. (HN- driven decision-making has existed since nearly the beginning as well.)

</comment><comment><USER16> It was an industry propaganda from companies who knew containers had something to offer over VMs (performance gains) but span those genuine advantages into claims that containers were a revolution rather than an evolution.

People started doing silly things like run containers on top of their cloud providers VMs (creating a second layer of isolation and massive amounts of engineering burden) and hence Kubernetes became synonymous with wasted engineering effort.

<comment><USER12> > People started doing silly things like run containers on top of their cloud > providers VMs

This assertion is simply wrong. Running containers in a VM is not silly at all. It's a very basic and very important feature. I'm actually surprised how, in HN of all places, this misconception is still mindlessly repeated in this day and age.

By firing up a VM you're actually allocating in the cloud the computational resources you need, and by running containers you are using those computational resources as you need. When you need more instances of a service running you replicate a container instance, and if a container instance is idling then you shut it down. If you run close to exhaust the computational resources of your VM instance then you just pick up your credit card and fire up another one to make room to run more containers. You never have to worry about anything because it just works.

And Kubernetes is the one responsible for monitoring resources and managing how container and VM instances are initialized or shut down, and how everything is kept up and running and communicating while you worry about nothing.

<comment><USER16> The full quote is:

> People started doing silly things like run containers on top of their cloud > providers VMs (creating a second layer of isolation and massive amounts of > engineering burden)

Regarding your statement:

> When you need more instances of a service running you replicate a container > instance, and if a container instance is idling then you shut it down.

> monitoring resources and managing how container and VM instances are > initialized or shut down

You're confusing orchestration with isolation. Orchestration has existed far before containers. You are already paying your cloud provider for isolation.

Containers are not needed for orchestration. I'm going to take a guess here and say you have been in the tech industry for about five years.

And regarding your claim:

> You never have to worry about anything because it just works.

No. Running a Kubernetes or Docker cluster is a non trivial task.

Here's a more likely scenario for running your own container farm on top of the platform your employer already pays for from your cloud provider: you just want to learn Kubernetes and have someone else pay you to do it, despite the lack of business benefit.

<comment><USER12> > People started doing silly things like run containers on top of their cloud > providers VMs (creating a second layer or isolation and massive amounts of > engineering burden)

I've addressed the full quote, and it's entirely wrong and made by someone who clearly has no practical experience running containerized apps. If you read my comment you'll check the points that address that misconception.

> No. Running a Kubernetes or Docker cluster is a non trivial task.

Kubernetes is not containers. It's a container orchestration service. That's an entirely different thing. It's like you are confounding application installers with the OS that runs the application.

And it seems you also confused Docker Swarm, Docker's container orchestration system, with Docker. Those are two entirely different things.

And container orchestration services are not a problem. If you ever get the opportunity to learn about containers and container orchestration services, you'll see that Docker Swarm is trivial to use even in bare VMs or even directly in dedicated hardware, once you get the ingress controller thing sorted out. In cloud providers it's even much simpler because ingress controllers are already provided out of the box and you have nifty UIs to configure everything.

And by the way, I appreciate if you stop rewriting your points midquote and after you get replies addressing the points you rewrite, because you're misrepresenting everything you and I am saying with each edit.

<comment><USER16> I wrote a comment then spent a minute editing it, because I am in bed on a phone. _Shrug_

Here's a shorter version of the entire conversation, quoting you:

> By firing up a VM you're actually allocating in the cloud the computational > resources you need, and by running containers you are using those > computational resources as you need. When you need more instances of a > service running you replicate a container instance, and if a container > instance is idling then you shut it down.

You're confusing isolatation with orchestration. You haven't been in the tech industry a long time. You do not need containers for orchestration. You do not need containers for isolation if you are already paying for VMs.</comment></comment></comment></comment></comment>


The Ethics of Saving Lives With Autonomous Cars Are Far Murkier Than You Think - <AUTHOR> http://www.wired.com/opinion/2013/07/the-surprising-ethics-of-robot-cars/

<comment><AUTHOR> Food for thought:

If a weakly-godlike AI system slightly more adept at seeing the moral implications of current actions on future events than you are suddenly instructed you to kill the person next to you by any means necessary, would you obey it?

</comment><comment><USER2> Most of the counterpoints raised have their basis in pure conjecture or assumptions. If everyone held themselves accountable for n-th degrees of freedom outcomes, nobody could in good conscience draw a breath. It would, after all, increase the entropy of the universe.</comment>


Surviving the Age of Humiliation - <AUTHOR> http://online.wsj.com/article/SB10001424052748703612804575222580214035638.html?mod=WSJ_LifeStyle_Lifestyle_5

<comment><USER2> > "But one of these days, someone will launch a comprehensive lawsuit, and a > jury will say that an Internet provider like Google is liable," he predicts. > "Sooner or later, an Internet provider will pay a massive price" for > spreading malicious gossip.

How is that possible? Can juries override safe harbour provisions?

Google has already been sued (in Italy) over something like this, and the outrage about that conviction was fairly universal iirc.

<comment><USER3> Bad litigation trips through courts all the time, it's not a deal breaker on justice, it's par for the course.

 _an Internet provider like Google_

Seriously now, this guy is pretty confused. Just as many litigants, juries and judges must also be when bruised egos and high technology collide in a courtroom.

<comment><USER2> I think he meant to say 'internet service provider', and even if I associate that with companies providing access technically speaking Google is an ISP and things like youtube are 'internet services'.

[http://arstechnica.com/tech-policy/news/2010/02/your-new- isp...](http://arstechnica.com/tech-policy/news/2010/02/your-new-isp-google- launches-1gbps-fiber-to-the-home-trial.ars)

</comment></comment></comment><comment><USER4> But I thought we weren't embarrassed anymore? <http://news.ycombinator.com/item?id=1322721>

</comment><comment><USER5> An interesting thing the article doesn't mention is the effect of anonymity on malicious gossip. The ability to publish widely and anonymously with such ease is unprecedented. Is it not that society's getting nastier, but that it's so much easier to be nasty and get away with it?

<comment><USER6> Surely the ability to be nasty and get away with it encourages nastiness by removing the negative reinforcement towards being "nice", thus making society nastier?

<comment><USER5> I hadn't thought of it that way. I had thought of it as additional opportunity for the same amount of nastiness to manifest itself.

<comment><USER7> Reminds me of a line from King of the Hill, when Minh Souphanousinphone is a bit too blunt with Peggy Hill, Minh apologizes by saying, "Oh, I'm sorry, I said something out loud that I should have only thought."

There's a lot more "out loud" when the cost is so low. I half hope it leads to people becoming more critical about who they let get under their skin, and more critical in what they believe from random sources, but I'm not optimistic.</comment></comment></comment></comment>


Dear YC Company, How To Use Your $150K Uncapped Note To Your Advantage - <AUTHOR> http://blog.eladgil.com/2011/01/dear-yc-company-how-to-use-your-150k.html

<comment><USER2> If I were a YC company suddenly getting $150K, I would set aside at least a small part of it for testing different user acquisition strategies and business models.

Take as little as $15K and make that your testing budget, money you're not afraid to lose learning what works and what doesn't.

Create a few different landing pages testing different price points, appeals, targeting different customer segments, etc.

Or take the patio11 approach and create a few highly targeted mini-sites centered around a very specific niche or customer.

Then test every traffic source out there, carefully documenting the traffic volume, CPC, CTR, conversion rate, CLV, etc from each one. AdWords, Facebook, media buys, buying backlinks, it doesn't matter. You're just testing.

The goal when testing traffic sources/acquisition strategies is to move fast and fail as quickly as possible. And that's a lot easier when you're not constrained by a $10/day budget and worrying about every penny.

Don't worry about getting an immediate ROI on that money just yet- you're paying for valuable data that you can show investors and use to scale your business. Don't think about the money as wasted- every paid click brings you more data you can use to refine your campaigns.

</comment><comment><USER3> Not to put your article down, but I'm sure YC startups have at least this advice, and probably much more (more detailed, more sophisticated and contextually aware of the specifics of the startup and its market) from YC (PG, Jessica, Harj, Garry Tan, Paul Buchheit, etc) themselves... so I'm not sure who this article is targeted at.

<comment><AUTHOR> I had two audiences in mind:

* General interest. I.e. what are the implications to YC companies themselves? The letter format was intended as a fun way to frame it :)

* The YC companies. From talking to a handful of them, I am much less certain than you are that every founder has completely digested the full implications of the note (various ways to use it, and what to think about avoiding).

As you point out, the YC team has some great people on it, who can undoubtedly support their startups in navigating some of these issues.

<comment><USER4> Having raised cap and interacted with a bunch of YC alums (hired 2 and friends with others), I can strongly second Elad's points. Just because they're in YC and exposed to some very accomplished people doesn't imply that they already know these things. They have a lot to think about when they're starting up their companies, so it's hard for them to absorb everything at once, despite being super smart.</comment></comment></comment>


Show HN: Signal – Simple, beautiful and privacy oriented open source analytics - <AUTHOR> https://github.com/astrocorp42/signal <comment><USER2> Signal is a wide known secure messenger ...

<comment><AUTHOR> I know but I also find this is the perfect name for this project (the goal of analytics is to discover signal in the noise, privacy oriented like the messenger app) so I don't want to exclude all names already associated to existing projects and restrict to some weird or badly orthographed names</comment></comment>


GitHub is down - <AUTHOR> Is Github down for you?<p>>> No server is currently available to service your request. >> Sorry about that. Please try refreshing and contact us if the problem persists. <comment><USER2> Yep. GH is down as of a few minutes ago: [https://status.github.com/](https://status.github.com/) (see "app server availability").

For what it's worth, it's trivial to write a status check of your own using the GH API ([https://status.github.com/api](https://status.github.com/api)).

</comment><comment><USER3> The first thing I do when github is down is check hacker news to see if I am the only one. heh.

<comment><AUTHOR> I checked their status page which didn't reflect what I was experiencing, and nothing was on their twitter feed, so I wanted to check if anyone else was experiencing it.

<comment><USER4> They seem to have a seperate twitter for this sort of thing: [https://twitter.com/githubstatus](https://twitter.com/githubstatus)

<comment> joatmon-snoo There's usually a delay on these things.

</comment></comment></comment></comment><comment><USER4> According to @githubstatus they're experiencing major service outages.

</comment><comment><USER5> [https://status.github.com/messages](https://status.github.com/messages) :/

</comment><comment><USER6> Site is down and also pulls and pushes are affected too.

<comment><USER6> ssh connection to github servers is also not working.

ssh -T email@example.com

</comment></comment><comment><AUTHOR> Now it's back!

</comment><comment><USER7> Down in Boston, MA

</comment><comment><USER8> Down for me in Jupiter, FL.</comment>


Entrepreneurship dreams from a village boy - <AUTHOR> Hey Everyone, From my childhood, I have a dream to start my company.Because lack of facility I am not able to pursue my passion. Finally, I decided to drop out of my college to pursue my dream. I knew the power of Internet And decided to use the whatever I have to achieve my goal. If you have spare time, please visit my blog and encourage me to follow my passion.(Am also looking for great mentor ) Thank you.<p>https://villagestartup.wordpress.com/ Regards, Vicky <comment><USER2> I hope there is content on your blog soon.

Good luck.

<comment><AUTHOR> Thank you brudger.</comment></comment>


Ask HN: How to deal with coworkers treating you badly? - <AUTHOR> I work within a small team of 3 devs currently and 1 person managing us without any dev experience.
We have one lead developer and the other 2 just devs.
The lead dev is a very arrogant person, mr know it all type of guy. He previously had a lot of issues within this company because he was failing on deliveries, although he's been here for many many years and they were afraid of showing him the door cause they'd have to compensate him a lot.
Thing is the company and my manager has now got past through the stage where he didnt trust this person and has forgot that him(manager) hand in his resignation cause of this guy.
Now that guys is lead dev with full on trust for some weird reason. I don't mind that at all, I don't care about this company anymore I just do it for the money and because I am in a place atm that I need to get rid some of my debt or I would be looking elsewhere.
Thing is how do I deal with this lead dev? He is a very difficult guy to speak to, I am trying not to speak to him at all, and of course anytime he is trying to tell me anything tech wise I am not listening to him at all. Thats his fault, he brought me to the place where I cant trust him on anything.
How can I deal and get past through this weird situation and deal with the fact that like yesterday he called me a hacker... out of nowhere just because he wanted to show his EPEEN. How? <comment><USER2> Which country is this and its not actually clear what does

"that him(manager) hand in his resignation cause of this guy"

<comment><AUTHOR> UK.

Our manager was very pressured by this Lead dev a year ago and hand in his resignation which was declined by the CEO and convinced him to stay.

The reason for this was due to that lead dev being an impossible person to work with and he was 1 year late on building a website, 3 months at start took another extension for 3 months then another and then a final one.

Thing is they couldn't easily fire him as he had been with the company for more than 5 years and they didn't keep track, they had to implement a proper timetable as to what he does and when etc and it would take 2-3 warnings to fire him... very hard in the UK. And making him redundant was also not possible due to him being in that field for more than 20 years he would be getting a huge sum from the company.

At the end another guy in the team that was also co-lead had to quit he couldn't stand working with that dev as I can't either.

Now a year later the company + the manager has forgot what they went through with this guy and give him the ok to run everything.

<comment><USER2> um you can fire people for breakdown in trust you know and why would him being in the field for 20 years most of which is not with the current employers increase his redundancy.

</comment></comment></comment><comment><USER3> Just leave.</comment>


402: Payment Required - <AUTHOR> https://medium.com/@humphd/402-payment-required-95bc72f06fcd <comment><AUTHOR> Here's an old hacker news discussion about the 402 code in the spec which is fairly interesting: [https://news.ycombinator.com/item?id=7857236](https://news.ycombinator.com/item?id=7857236)</comment>


My Favorite Paul Graham Quotes from Business of Software 2009 - <AUTHOR> http://www.kevinminnick.com/2009/11/my-favorite-paul-graham-quotes-from-bos2009.html <comment><USER2> 21 Trends:

#5 United States

#6 Silicon Valley

#20 Venture funding

all need to move to the "Not Trends".

The US, like a wealthy person whose money ran out but kept spending on credit cards, has just started its "big reckoning" and it will only get worse -- much, much worse.

VC and the Valley are the old model. Technology is moving us away from the large capital requirements that used to be necessary to go from 0 to 60. Prepare for a deluge of Indian, Chinese and European startups because of this.</comment>


Getting it Wrong: Surprising Tips on How to Learn - <AUTHOR> http://www.scientificamerican.com/article.cfm?id=getting-it-wrong

<comment><USER2> Guessing before you know the answer creates suspense: a cognitive receptacle for the answer + emotional engagement. When the answer comes along, you grab it, like an enzyme grabs one of its raw materials, because it _fits_. And it's a relief.

When studying at uni, I took a past exam paper under exam conditions, before doing any study: it wonderfully concentrated the mind on what I needed to learn. (I got the top mark, with a big gap to the next person).

When teaching at uni, I presented a puzzle at the start of the class, that didn't make sense until the material in that lecture had been heard. The entire class came up to me at the end to check what the answer was.

</comment><comment><USER3> Learning through trial and error is so damn useful because that's how our brain is designed to learn.

Whenever we make a prediction, dopamine neurons in our brain start to fire, triggering a feeling of expectation. If we predict right, we get a surge of dopamine and some positive reinforcement. If we get it wrong, the dopamine neurons stop firing which, aside from making us feel like crap, serves as a signal that the prediction needs to be revised. Normal learning operates by consciously acquiring information and hoping it eventually filters down into the subconscious. Trial and error, on the other hand, puts a direct line into our subconscious - it's much more efficient.

</comment><comment><USER4> Sometimes I wish educators studied some Machine Learning and similar things.

Not to detract from the interesting finding, but it sounds to me like someone discovered that in supervised learning it is advisable to use a negative corpus as well :)

</comment><comment><USER5> This seems to reflect the advice often given by hackers to roll your own framework as a learning experience.

<comment><USER6> Not only that, but making mistakes is often the best way to learn how to do something correctly.

People who don't make very many mistakes are usuall also people who don't try to do anything new, so they're not giving themselves a chance to learn to begin with. Learning to make and accept mistakes is one of the best lessons anyone can learn; it helps you to keep your mind open so that you CAN learn.

</comment></comment><comment><USER7> Learning by mistakes is a good way to learn im sure of it. But this article on page 1 covers more about learning where you have a teacher who already knows the material, it would be far harder to use this failing technique if you are creating the material to begin with.

Minus the above I can relate to learning by failing.

For those who use the command line in unix. I hardly can ever remember the command if I grabbed it off google. When I attempt, fail, look it up knowing what I am after I always find myself more often than not still with the correct knowledge.

I do feel this knowledge is sort of common knowledge or more like "yes, that is what i would have said if i ever put it in words". Needless to say it is good to read these types of obvious thoughts as it just aids the life learning process.

</comment><comment><USER8> "In fact, they found, learning becomes better if conditions are arranged so that students make errors."

Well, then, there's no better learning environment than computer programming!

"People remember things better, longer, if they are given ... tests at which they are bound to fail."

No pain, no gain.

I used to wonder why a damaged body part is painful for so long after it has 'made it's point'. After I'd lived long enough to forget several painful lessons, I understood: the longer the pain lasts, the longer the memory lasts.

Makes me think scars are evolution's way of helping us to remember.

</comment><comment><USER9> The approach they outline is nothing new. It's how I developed mathematical proficiency 20 years ago, following the advice in _Mistakes and how to find them before the teacher does_.

[http://books.google.com/books?id=Z4-JbH0Cf0IC&dq=misteak...](http://books.google.com/books?id=Z4-JbH0Cf0IC&dq=misteaks+and+how+to+spot+them+before+your+teacher+does&source=gbs_navlinks_s)

</comment><comment><USER10> In his wonderful book _The End of Education_ , Neil Postman recommended error- detection as a powerful teaching/learning method.</comment>