Crockford - <AUTHOR> https://paulstraw.svbtle.com/crockford <comment><USER2> It seems pretty clear that this is more about a clique that is attacking Mr Crockford in order to obtain more power for themselves.

They can now say - "Hey - you have to listen to us! We Matter! Look at what we forced the Nodevember people to do. We're a moral authority!".

To go from complaining about a lack of diversity, to basically saying 'you screwed up and now need to prove your love to us by uninviting this JS superstar'. It's like the stereotypical abusive boy/girl friend requiring their lover to do something they normally wouldn't do to prove their love.

It's simple bullying, really.

<comment><USER3> Who are these bullies?

<comment><USER4> Social Justice warriors.

</comment></comment></comment><comment><USER5> In cases like these, it's worth thinking a bit about tolerance. Tolerance, simply put, is behaving politely and professionally towards people and organizations that you dislike. Note the "that you dislike" part.

Think you're tolerant because, say, you don't mind being around gay people? Well, do you hate gay people? If you say no to that, then you aren't being tolerant at all by being around them, you're just normal.

Do you hate, say, Neo-nazis? Then you can demonstrate your tolerance by engaging with them politely and professionally if you encounter any.

Apparently, at some point, we not only ran away from being proud of our tolerance, many people are wearing their level of intolerance as a badge of pride. In some circles, it seems to be a good thing to find anybody who disagrees with you in some way, and gather a virtual lynch mob to make sure that the worst thing you can arrange happens to them.

Personally, I don't think Crockford did anything wrong. But if you do, why don't you try demonstrating your tolerance by welcoming him at your conference anyways and listening to his views on technical subjects? If he says something that bothers you, you can calmly and politely explain to him how his statement hurt you, or maybe even just let it go. That's what tolerance looks like.

Can we please get back to making real tolerance something to be proud of?

</comment><comment><USER6> One of my consulting projects includes doing programming for a relatively big millenials-dominated site. Users on that site know exactly what cards to play in what order to get people on staff and/or affiliates fired - it's Crybullying all the way. It doesn't even matter if the claims made are true or not, just the proclamation of being hurt or oppressed is sufficient. The site owners ultimately making these decisions do so because they live in absolute terror of what these people could do if they turned against the site itself or them personally.

We now live in a world where anything is done to appease you if you only cry loud enough. It always used to be the case in personal conflicts that whoever was better at mud slinging had a significant hand up as compared to someone who's less talented, but in combination with the new fad of Crybullying, this is a devastating tactic.

<comment><USER7> Agree. I ran a dating site for 7 years and had to deal with such inquiries on a daily basis ("Buhuu, that guy/girl sad mean things to me, close his/her account").

Most often it turned out the complaining person was the problem.

Worst cases were third parties who requested such actions on behalf of the "victim". They were loud, asked other members to complain too and even showed up in our offices to demand account closings - without any proof or valid reasoning.

I never understood how people handle their real lives without being able to argue or verbally discuss such issues. Even if these include occasional insults. IMHO these are just words - you don't need to cry for higher authorities (site admins, conference organizers or law authorities) all the time.

<comment><USER6> _> I never understood how people handle their real lives without being able to argue or verbally discuss such issues_

My guess is they handle it very well, because they're just using a strategy that works out perfectly for them. This only has to be learned or reinforced once, and it becomes a problem solving strategy for life because it's such an _easy exploit_ to use with our social protocols. It can be used to solve pretty much every issue where they can win by exerting pressure.

Whoever happens to get in their way also often takes permanent damage by being labeled something bad that sticks and this hampers that person's ability to not only resolve the issue at hand but prevents them from being heard in the future as well. Either way, you either acquiesce or you get bulldozed.

</comment></comment></comment><comment><USER8> Crockford did nothing wrong. Fuck Nodevember. If you have huevos or egos or whatever, don't support that conference. He is one of those guys that will email you back if you have a technical question. I know because he already did twice to me. Who can say that about someone so influential?

<comment><USER9> It's another conference scam anyway. They're charging $350 for tickets with unconfirmed speakers and they have an open call for speakers.

Considering the main value is out of the speakers selling before having much of anything confirmed or even finished just screams like yet another conference trying to cash in (they're even taking sponsors, too).

I don't mind conferences where you have to pay mind you but I've only seen the small, borderline scammy conferences where you have to pay before finding out the full line up of speakers.

<comment><USER8> I never got the point of those anyways as youtube is sufficient.

<comment><USER9> I mean sometimes they're good if you can interact with the speakers, ask questions and maybe even network / talk with the other folks interested in the same topics as you. But yeah in general I would imagine YouTube would be better with this.

Now if it's a _free_ conference? Then I'm all for it almost always for it. YCombinator Start-up school comes to mind; I enjoyed it a lot the last time and I'm going to the upcoming one :)

<comment><USER8> That's a good point. I think because I have 3 young sons and am so busy outside of that type of stuff I have a different take on my personal time in terms of spending more if it with geeks than I have to. :)

</comment></comment></comment></comment></comment><comment><USER10> This "X shaming" or "X gender inappropriate joke" is really getting out of hand. You can't say anything anymore. You may hurt someone's feelings, bla bla bla. I'm sick of this. I'm sick of this political correctness that is intoxicating our society. You cannot say anymore: "hey, grow a pair!" because this is sexist? Come on... this is insane! One thing is to respect everybody and another one is to apply censorship to everything. Even though people didn't meant to offend, harass or shame other people.

<comment><USER11> > Even though people didn't meant to offend, harass or shame other people.

That's the thing - people who shame others are not looking at intent behind actions; they're getting offended by shallow interpretations (a sign of insecurity). Any reasonable person looking at Crockford making his "weak" joke would not think his intention was to harass physically weak people.

This kind of shit is also why you see people like /r/the_donald aggressively provoking and fighting back against these so-called "SJW's" \- the pendulum has swung way too far the opposite way.

<comment><USER12> > This kind of shit is also why you see people like /r/the_donald

I'm not sure if you just referred to a subreddit as a person, or referred to a ridiculously famous man by the path to the subreddit about him?

Either way, wtf.

<comment><USER11> I referred to the _people_ on /r/the_donald, as a group

</comment></comment></comment></comment><comment><USER13> There's the old idea of "you judge your actions by your intentions and others intentions by their actions."

I think culturally we've entered into this weird area where we're not even judging people so much by their actions but rather by how their actions make some people feel. There is nothing wrong with being mindful of how your statements and actions are perceived. Communication is always a two-way street where what is heard is sometimes more important than what is said. It's just too bad that we're shifting so far away from being able to understand the intentions of the speaker. Fear of saying the wrong thing never teaches someone why and what is wrong to say.

<comment><USER14> Absolutely. It's great that societal norms of what is acceptable in speech are progressing to be considerate of more and more out-groups. But we need to be mindful that news of these new norms do not reach everyone simultaneously, and we should not tar and feather people for not independently coming upon the insight that a certain element of speech can be hurtful. While we should all seek to understand and empathize with the rest of the world, it's a vast, complex world beyond any one person's direct experience. If someone gratuitously drops the N-word, ostracise them; but if there is good reason to believe the speaker is well-intentioned but perhaps ignorant on some point of concern, the proper response is to discreetly engage in a respectful, open dialogue, not to publicly shame anyone. If the speech is hurtful and dialogue fruitless, direct action may be required, but not before.

<comment><USER15> These aren't norms and they aren't new the subject has changed but this is exactly the level of thought policing that true liberals used to fight and even die for.

This is no different than the puritans or any other group that wish to enforce their own opinions at the expense of everyone else.

<comment><USER14> Asking that white men respect others is not hurting "everyone else." White men are the small group here. Norms of tolerance and mutual respect only ask that white men welcome everyone else to the table.

</comment></comment></comment></comment><comment><USER16> It seems like those calling for removing speakers are often trying to set identity contingencies. That is, a rule like, "if you are Douglas Crockford, then you can't speak at a conference". However, like identity contingencies against people based on gender, race, etc.. this kind of rule is disappointing because it assumes there is something inherently limited or bad about being Douglas Crockford.

What seems better to me would be asking for an explicit behavioral contingency on being a conference speaker. For example, "if you are going to make sexist remarks, then you cannot speak at the conference".

This makes clear that if you think Douglas Crockford is not fundamentally broken as a human, but could learn and become better at following the contingencies set by the conference, then he is fine to speak. It also opens room to work with Crockford to create a plan for him to speak at the conference, such as going over his speech beforehand, or sitting down with him afterward to discuss ways in which he may have violated the conference's standards.

He can't stop being Douglas Crockford, but I don't think that means he can't stop producing behaviors conference organizers find offensive or harmful (if he wants to).

P.s. check out the book Whistling Vivaldi for an interesting introduction to some potentially harmful (or beneficial) effects of identity contingencies.

<comment><USER12> That concept won't work for these people. Making rules like "don't be sexist" means that to prevent someone coming, you have to show that the person was in fact being sexist.

The homepage for Nodevember has a big "code of conduct" section. No idea if it's new or not, but one bit is:

> Do not insult or put down other attendees.

Insult is taken, not given. You can't decide what insults me, the same as I can't decide what insults you. By that token, anyone can claim that anything anyone says is insulting to them.

I find plenty of common nodejs community 'norms' to be insulting to my eyes to read, never mind to my brain having to deal with the mental shenanigans required to try to make sense of the decisions made.

Does that mean I have the ability to say "NPM's dependency management is so poor quality it's insulting" and have any talk about NPM banned?

<comment><USER16> > Insult is taken, not given. You can't decide what insults me, the same as I > can't decide what insults you. By that token, anyone can claim that anything > anyone says is insulting to them.

Agreed, and because the conference organizers wield the power to decide who speaks (or to implement deciding in some fashion), they can choose how the insult they're referring to is measured and responded to. For example, it could be by committee, etc..

I'm disappointed if, regardless of their belief on what constitutes insulting behavior, they didn't put forth at least some effort to see if Crockford could meet their standards.

> Does that mean I have the ability to say "NPM's dependency management is so > poor quality it's insulting" and have any talk about NPM banned?

You can say that, and organize a conference that bans talk about NPM. As long as your were upfront about it, I'd be fine with it--just sounds like the trappings of a bad conference. If you invited a speaker, only to realize that they were planning to talk about NPM (say you hadn't made your contingency clear), then hopefully you would check if they would drop talk about NPM. If you found out that someone had talked about NPM at a previous conference, and so removed them from your conference without doing the above, I would be disappointed in you, too ;).

> Making rules like "don't be sexist" means that to prevent someone coming, > you have to show that the person was in fact being sexist.

I'm not sure if you're saying that, to you, nothing in his behavior appears sexist. I was trying to think of the issue of what to do after / assuming they believed he had acted inappropriately in the past.

</comment></comment></comment><comment><USER17> I'm trying to imagine an alternative timeline in which Nodevember, after having announced Crockford as a speaker and then receiving pushback on Twitter, privately discusses things with Crockford and mutually agree that they should part ways, but come up with a plausible explanation that would minimize the loss of face for both parties.

(for the sake of argument, let's say we agree that conferences have the unilateral right to invite and reject whomever they want for any reason that doesn't violate current federal laws)

1) What would that public and plausible explanation be? e.g. "Due to scheduling conflicts, we regret to say that Mr. Crockford cannot give the keynote speech but will be attending etc.etc."

2) How much time would it take (in days) to plan and then move on this?

3) Is Crockford the type to take this graciously and quietly and, even though he's justified in feeling wronged, doesn't feel the need to retaliate now or in the future?

edit: I think that Nodevember can do what it wants, but that it seriously fucked up for not vetting Mr. Crockford before inviting him and then really fucked up by throwing him under the bus. But that's been thoroughly discussed in last night's thread

<comment><USER18> "it seriously fucked up for not vetting Mr. Crockford before inviting him"

What exactly does Mr. Crockford need to be "vetted" for, with whom, and how exactly did those people gain the authority to decide who gets to speak and who doesn't?

<comment><USER17> Whose authority? The same authority that allows them to create Nodevember: their own. I've never run a conference so I'm only speculating here: when people decide to organize a conference, they have an idea of what ideas and environment and discussion they want to promote, whether it's Nodevember or the American Neo-Nazi Party. I don't know what Nodevember's standards are but they clearly don't allow for Crockford, and as long as it's not for reasons that might violate civil rights, e.g. if Crockford were to rely on a wheelchair and Nodevember wanted to convene at a place without elevators or ramps, they can have whatever standards they want, just like they're free to write and uphold a code of conduct they believe in.

But generally, and again I've never run a conference, if you have standards, you should have them at the time of planning the conference, including when inviting keynote speakers.

<comment><USER18> Sorry, when I said "whose authority" I wasn't referring to the conference runners -- obviously they can choose to invite and disinvite anyone they want. I was referring to these Twitter randos the conference runners are apparently required to "vet" their speakers with.

</comment></comment></comment></comment><comment><USER19> Yeah, I don't get it either. FWIW, and why I'm commenting, "promiscuity" with regards to computer communications isn't even new with Crockford. It's the exact correct term used in the industry to refer to a mode you can put network cards into if you want to inspect all traffic that reaches it, rather than just traffic indented for your workstation [0]. It goes back at least to the early 1990s when I first encountered it, probably earlier.

[0] [https://en.m.wikipedia.org/wiki/Promiscuous_mode](https://en.m.wikipedia.org/wiki/Promiscuous_mode)

<comment><USER12> Oh come on, don't let facts and technical terms get in the way of a good witch hunt.

</comment></comment><comment><USER20> First they came for Brendan Eich and I did not speak out because I was pro gay marriage.

Then they came for Douglas Crockford and I did not speak out because I did not attend JavaScript conferences.

Then they came for me—and it did not matter because I am a C# programmer.

<comment><USER21> Cute. However conflating or equating what Douglas Crockford has done with Brendan Eich is in my estimation unreasonable.

As far as anyone can tell, at worst Crockford has a conversational style that isn't excessively concerned with political correctness or charm. He used words like "gonads" and "promiscuous" in a way that can be misconstrued.

At worst, Brendan Eich can be accused of working to actively work against the aspirations of a community because they are different to him, even though these aspirations don't materially affect him.

<comment><USER20> Gay marriage was illegal in the US until last year. His position was not a fringe position. I am glad marriage equality comes to the US but if we start prosecuting people for their political beliefs, things are going to get ugly real quick. The tide of political direction can turn on a dime.

How about marijuana legalization? It's coming to all of the US states sooner or later. Let's start making a list of people who are against it so we can start a revenge hit later.

<comment><USER21> I completely agree with you. I'm just saying the two controversies, on a relative basis, aren't in the same order of magnitude.

<comment><USER20> Got it.

I don't understand how the word promiscuous become such a toxic word even in the context of human relationship. We are several years into the era of Tinder and other hookup apps and people are still freaking out with the concept.

</comment></comment></comment></comment></comment><comment><USER14> It disappoints me to see how many folks on Hacker News have responded to this incident by condemning the entire concept of standards of conduct and the idea that the developer community should take the goal of inclusivity and sensitivity seriously. There is no question that our community too often can feel hostile to those who aren't white hetero men, and that is a real problem.

On the other hand, there is no question that advocates for inclusivity are sometimes overzealous and careless, and no doubt, some people have even exploited the issue in bad faith for their own aims. These mistakes confuse and alienate would-be allies in the community. While no doubt well- intentioned, Nodevember has hurt the cause.

My plea to those who are tempted to dismiss the whole project of inclusivity in response to this incident is this:

The program needs debugging, but that doesn't mean the program is unworkable or the goals are undesirable. Perhaps we forgot a semicolon somewhere, or maybe we need to do a complete rewrite. But creating a community that is comfortable for everyone is an inherently valuable goal. And it doesn't need to be uncomfortable for cisgendered hetero white men, either. Please don't abandon this goal. Instead, let's engage in a constructive dialogue to figure out how to do better.

<comment><USER22> > But creating a community that is comfortable for everyone is an inherently > valuable goal.

Well, I can never be comfortable in a community where I might have my reputation and/or career destroyed over a reference to testicles that some might deem inappropriate.

The people doing this are vicious and hateful, and they need to be given a taste of their own medicine.

<comment><USER23> Maybe the idea of communities in tech should just go away. People who share similar ideas on tech should just get together to share those ideas without having to setup a bunch of arbitrary rules and gatekeepers.

Some brilliant people might also have offensive ideas. Why not have a system that allows that brilliance to shine through?

The larger problem here is that these are not communities of hobbyists. Saying something silly or having someone misinterpret what you are saying within the community could cost you your living. Crockford will be okay because he is already well established, but what if he wasn't as well established?

<comment><USER24> > _Some brilliant people might also have offensive ideas._

Yup. And some brilliant ideas are offensive.

</comment></comment></comment></comment><comment><USER22> I think the only way to oppose this tidal wave of bullying and shaming and witch hunting is to employ all of the same tactics the cry-bullies employ. People and organizations cave to these bullies because, in their cost-benefit calculations, they see little or no downside to caving, and possible upside. In order to stop this, it has to made _costly_ to cave to them. If everyone who thinks it was ridiculous and cowardly for Nodevember to uninvite Crockford were to make it known that they would boycott Nodevember and their corporate sponsors in the future, they might see things differently.

Also, people who routinely seek to destroy the reputations and careers of people who don't speak and behave according to their strict standard should run the risk of having their own careers and reputations destroyed in response. This form of sweet justice happens once in a while, but not nearly enough.

</comment><comment><USER25> I'm a liberal with three daughters and I'm certainly pro-inclusion but this repulses me. We've grown as a society to over-indulge people with fragile personalities. Yuk.

<comment><USER26> I disagree about the notion that the attack on Crockford comes from "people with fragile personalities", this look to me more like advanced identity politics manipulated at a high level, definitely would take me 10'000 hours to master.

</comment></comment><comment><USER27> Given the context, chicharrones was clearly a mistake or a play on the word cojones. The earnest parsers of speech for fatal psychological landmines would not in a million years be able to make this tiny mental leap, as it requires a sense of humor.

</comment><comment><USER28> A key pillar of today's identity politics is a default state of victimhood in a zero-sum world where a crass rubric of privilege is the only determinant of one's outcome.

Sadly, when people (are made to) believe they are victims, almost any amount of egregious behavior opens up for them.

We should not quietly tolerate puritanical bullying behavior from the far left any more than we did that from the far right.

</comment><comment><USER29> See also [https://news.ycombinator.com/item?id=12422420](https://news.ycombinator.com/item?id=12422420)

</comment><comment><USER30> A point i didn't see mentioned is that the talk about "having the balls" is all about the fear-of-monads being overblown and unjustified, and in this context making unfunny jokes about overblown and unjustified bravado conveys a point -- and does so in a way that no amount of explaining that "it's not so bad really" would. I actually think saying he was being exclusionary in this instance is downright bad faith.

</comment><comment><USER31> As for Mr Straw, glad to hear he made the decision to think for himself, rather than buying into all the BS that the conservatives or liberals / Republicans or Democrats are selling.

All things in moderation, I suppose, and don't be afraid to look at the historical facts and likely consequences of decisions.

Not that facts help you make a values decision about what to do when there is a "zero sum", conflict of interest type situation...

</comment><comment><USER32> Mr. Straw is part of the problem, imo.

There's a lawful vs chaotic dichotomy going on - Mr. Straw started out lawful conservative, but then found out about a different set of laws he could leverage into a position of power.

It's a similar but different power leverage being exercised elsewhere by a new generation. Feeling excluded by more usual measures like achievement or talent, they've taken to using social taboos and bullying to gain power.

It's vaguely annoying to watch, but it's best ignored. The very lack of talent that people who use social manoeuvres to get to the top means the groups they dominate won't be very important or effective, unless it's law making directly. I'll get worried then. I'm fairly sanguine about node.js conferences.

<comment><USER33> Or it could be that he changed his opinions based on new perspectives like any intellectually honest person does.

<comment><USER32> People who self identify as busybodies creating rules of conduct etc. seem to me to be more interested in rules than opinions. Setting the rules, especially when it's around social norms - rules almost for the sake of rules - is a cheap power tactic pursued in organisations all over the world for decades. Thought police / religious fundamentalists exist in every political shade. It's the zealousness of the policing that I have a problem with, not the particular shade.

Just my opinion. I don't care much about the politics. I don't think either side of US politics is wrong or right per se, for example. I'm just very wary of people who have strong opinions that they want to turn into rules.

</comment></comment></comment><comment><USER34> Fundamentalists are immediately, overwhelmingly, correct.

</comment><comment><USER35> How come this story and the previous one have been penalized on HN despite the high number of upvotes?

This story was published to HN only an hour ago but has gathered over 115 points, yet it has been relegated to page 2.

The previous story [1] was published 14 hours ago and has over 540 points, yet is now to be found on page 4.

If the algorithm was operating as usual then both stories would still be in the top half of the front page.

[1] [https://news.ycombinator.com/item?id=12422420](https://news.ycombinator.com/item?id=12422420)

<comment><USER14> Would getting flagged bump it down? It's a shame people don't think this is a worthwhile conversation to be having.

<comment><USER36> How does anyone know that this happens because of flagging? What if the moderator(s) did it? Does HN have a moderation log at all?

</comment></comment></comment><comment><USER37> Political correctness stifles upward mobility and promotes the maintenance of the current 'status quo'.

We're trending towards a future where people can do anything they want as long as they have enough power and don't say anything offensive while doing it.

It's horse shit to be honest; most humans don't really give a fuck about a gendered presentation slide and removing all gender references from everything isn't how you magically gain 'equality'.

Some of the worst humans I know are the ones who love this garbage because it's essentially the current way to bully and silence the people you don't like.

</comment><comment><USER38> Sometimes people encourage me to get into speaking at conferences. There are a number of reasons I don't want to, and shit like this isn't the main one, but it certainly doesn't help.

Everything just feels very tense (to be clear, I think the reasons for this are understandable) but it also just seems kind of thankless.

</comment><comment><USER39> Great dissection. Thanks Paul

</comment><comment><USER40> He said a few things that feel like promoting a non inclusive environment. Which is bad, especially from someone like him, because many people look up to him and probably think it is okay to say such things.

Edit: Also interesting that the HN crowd always confirms the accusations they receive.

<comment><USER8> monads sound like gonads, its elementary humor but there's nothing offensive about it. If assholes were only attached to men, some dumb fuck would complain if he made fart jokes.</comment></comment>


Ikea gives India employees six months paid parental leave - <AUTHOR> http://money.cnn.com/2017/03/15/news/india/ikea-india-parental-maternity-leave/ <comment><USER2> Just wanted to highlight that the Indian Parliament recently passed a law requiring 26 weeks (6 months) of paid maternity leave for all companies with more than 10 employees [1].

Though, obviously, good on Ikea to extend it to both the parents, instead of just the mother.

[1] [http://timesofindia.indiatimes.com/india/parliament- passes-b...](http://timesofindia.indiatimes.com/india/parliament-passes-bill- to-raise-maternity-leave-to-26-weeks/articleshow/57565644.cms)</comment>


Ask HN: Advice for my phone project - <AUTHOR>

Few months ago I tried to build a phone platform where shy people could phone call and talk to members of the opposite sex to get more comfortable in conversation.  Ideally, a person would set an amount from free to x money per minute.  Shy person would call anonymous phone number and be prompted for credit card.  After payment it would call provider cell phone so user does not have their phone #.  After end of conversation, system automatically tallies bill and gives pittance to me and most money to provider.  At that time also, user is prompted to buy emailed mp3 recording of conversation for x money and provider gets most of that money too.
I am very novice and do not have heavy coding experience.  Worked a little bit with visual basic, html, wordpress, drupal, & php.  Tried using Twilio Twimlets, but couldn't get exact functionality I wanted.  Another user on here recommended I use Stripe for payment processing, but couldn't find a novice function akin to Strimlets.  Lastly tried using Zapier and then I gave up.
Wanting to try again.  Are there any more easy-to-use resources HN could recommend for this project?  If anyone knows of an open-source downloadable platform that is similar, maybe I could manipulate that.  Appreciate your time and advice! <comment><USER2> Try learning enough Django or Rails to serve up the TwiML you need. Don't worry about the payment stuff just yet.

You can use ngrok or localtunnel to make it possible for twilio to get your TwiML from your laptop.

You'll likely find that the most difficult part of your project will be getting money to the provider fast enough.

Good luck!</comment>


The Elegance of Deflate - <AUTHOR> http://www.codersnotes.com/notes/elegance-of-deflate/ <comment><USER3> There were a lot of encoders at the time using this general scheme (a few more values to indicate match length or distances). PKZIP won at the time because it was faster, and PK had the name recognition from his PKARC, which was a superfast implementation of SEA ARC (the dominant archiver on PCs at the time).

PK had to stop working on PKARC because of a C&D request from SEA. He wrote the first algos of PKZIP, which were on par with SEA ARC on compression (and with PKARC on speed), but weren't much better. (And have been deprecated since the 1990 if I'm not mistaken).

Then, the Japanese encoders started to rule the compression ratio (and had comparably reasonable compression times) - LHArc, LZAri, don't remember the rest of the names. LHArc or LHA (don't remember which), had basically the same scheme that PKZIP converged on, except it used adaptive arithmetic coding. PK replaced that with static huffman coding, trading a little compression for a lot of speed, and the format we now know and love as "zlib compression" was born (and quickly took the world by storm, being in a sweet spot of compression and speed).

There's another non-trivial thing that PKZIP had going for it - it put the directory at the end, which meant you could see the list of files in the archive without reading the entire archive! This sounds simple, but back then everyone adopted the tar-style "file header then data" appendable style, which meant that just listing the files inside a 300KB zip file (almost a whole floppy disk!) meant reading that entire floppy (30 seconds at least). PKZIP could do it in about 2 seconds.

<comment><USER4> > There's another non-trivial thing that PKZIP had going for it - it put the > directory at the end, which meant you could see the list of files in the > archive without reading the entire archive!

Beginning _and_ the end, which continues to bite us in the ass until today, where we regularly stumble over bugs when one part of a toolchain uses one entry and the rest the other.

<comment><USER5> The directory is only at the end in the PKZIP, there is additional info before each compressed file, like in TAR. It's actually good to have both, as it allows recovering individual files even when there's a corruption in other files or in the directory.

<comment><USER6> Yes, it's possible to use the information to recover a corruption.

The problem is, what happens if the two are different? This can be through accident or maliciously. If one tool uses the first and another uses the second, then you'll end up with different results, and it can be hard to figure out why they are different.

It's like a CD, where audio players interpret the CD different than CD-ROM players. Some anti-copying techniques tried to take advantage of this to produce un-rippable CDs. A problem, however, was that car CD players used CD- ROM drives, so these CDs weren't playable in those cars. ([https://en.wikipedia.org/wiki/Compact_Disc_and_DVD_copy_prot...](https://en.wikipedia.org/wiki/Compact_Disc_and_DVD_copy_protection) )

Or some HTTP header attacks based on duplicate headers. Let's say there's a firewall which allows only access requests for "Content-Type: text/plain". If the attacker includes the header twice, once as text/plain and the other as image/gif, then the firewall might only check the first while the back-end web server interprets the second. This is a silly example; I couldn't remember the real attacks which take advantage of the same mechanism.

<comment><USER4> Zip inconsistency vulnerability: [https://nakedsecurity.sophos.com/2013/08/09/android- master-k...](https://nakedsecurity.sophos.com/2013/08/09/android-master-key- vulnerability-more-malware-found-exploiting-code-verification-bypass/)

Duplicate HTTP header vulnerabilities:

• [https://bugzilla.mozilla.org/show_bug.cgi?id=376756](https://bugzilla.mozilla.org/show_bug.cgi?id=376756)

• [https://splash.riverbed.com/thread/7772](https://splash.riverbed.com/thread/7772)

If a spec is open for interpretation, you can be sure that some software gets it wrong.

<comment><USER5> The "zip inconsistency" linked is not a problem of the Zip file format specification. The goal of the zip format was never to provide a tamper-proof cryptographically secure binary package. Whoever uses this structure for some more security-wise complex demands is responsible to make sure that the security assumptions he needs are respected by the code he implements. One simple method were signing the resulting archive, then no modification is possible, and therefore also isn't possible adding a second entry with the same name by the attacker.

I really don't care about HTTP headers or Reed-Salomon, as they are completely irrelevant to the Zip format designed in eighties for very specific purposes then (being fast on the floppy disk and 4 MHz 8088 processor, allowing to recover as much as possible from the floppy even after the floppy disk sector failure), I just claim, through the whole conversation here, that the Zip format is not a bad format for having a central directory and the metadata outside of it too, and I gave my arguments for that. NTFS also has some metadata on more than one place and it's an intentional design decision solving real problems: in Zip case allowing for fast access of the list of the files in the archive but also easy recovery of the single files when some part of the archive gets corrupted.

I also claim that the Zip format is not in any way "bad" for not making impossible per the archive format design and specification of having the two identical file names in the archive. I can even imagine use cases where exactly allowing that is/was beneficial.

</comment></comment></comment></comment></comment></comment><comment><USER7> Really enjoyed reading this, I think I'd like to look more closely at an implementation to understand it further.

Reading the article, I was reminded of a nagging question I've had in the back of my mind for a bit.

The ASCII table was invented in the '70s, when the cost of disk storage was much higher than it is today. But it's a shared dictionary, a standard that we can all agree upon, something that's already on every computer.

The thing I've wondered about is whether there could be any advantage to creating new generations of shared dictionaries that are domain-specific, and much larger.

For example, in the specific (and over-simplified) case of transmitting large quantities of English text from a server to a client, you could reduce the amount of data sent over the wire if both parties shared a lookup table that contained the entire English dictionary. In that case, you wouldn't transmit a book as a collection of characters, but rather as a collection of words.

Furthermore, it would seem like you could apply the same traditional compression methods like what's described in the article to furthermore reduce the amount of data being sent. Rather than identifying repeating patterns of letters, you would identify repeating patterns of words.

Of course, the obvious drawback is that an English lookup table is useless for transmitting anything other than English text. But again, disk storage being as cheap as it is, I wonder if it wouldn't be such a monumental problem to store many domain-specific dictionaries.

Of course, you'd always want to keep the ASCII table as a fallback. Much in the same way that a conversation in sign language (ASL, specifically) is largely composed of hand gestures referring to words or concepts, but the language still includes the complete alphabet as a fallback.

The thing I don't understand well enough is whether modern compression already incorporates concepts that are similar enough such that a shared-word dictionary would be useless. It seems like the "LZXX" style of compression essentially includes a similar but dynamically-generated sort of dictionary at the beginning of the transmission, and subsequently refers to that in order to express the message itself. Would the gains of having enormous shared dictionaries cancel out the advantage of that approach, only in a much more "wasteful" way?

<comment><USER8> I considered a project loosely along those lines. My idea was a One-Meg-of- Data project. Define a megabyte of data that everyone would have have in a bit-identical form.

The appeal of this to me would be in the code golf style of approach of specifying the actual data. Apart from plain useful raw data, It would include a bare minimum VM that can be used to generate larger outputs from the data. More complex VMs would be permitted provided a reference implementation can be run on the Minimal VM. The code for the reference Implementation would, of course, be included in the one megabyte.

A program using the data could request

* raw bytes

* output from the VM given code and parameters (with perhaps a cycle limit to stop endless loop hangs)

* output from the VM given code and parameters specified at an index in the data.

* output from the VM given code and parameters specified at an index in the data plus user parameters.

The system would be stateless, the VM would only exist for the duration of the data generation call. The same parameters would always produce the same output.

I have doubts as to it's fundamental usefulness but it would certainly make a fun collaborative project at the very least.

<comment><USER9> Not exactly your idea, I think, but perhaps similar:

\- A patent from ibm for some previous dynamic dictionary [1]

\- Brotli: 120kb predefined dictionary [2]

[1] [http://www.google.com.br/patents/US8610606](http://www.google.com.br/patents/US8610606)

[2] [https://en.m.wikipedia.org/wiki/Brotli](https://en.m.wikipedia.org/wiki/Brotli)

</comment></comment></comment><comment><USER10> It's a good system. Though the full DEFLATE spec has quirks like storing literal blocks prefixed by their length and then the ones complement of their length.

The next step in improving a simple compression algorithm is to replace Huffman encoding with arithmetic/range encoding. Huffman encoding proceeds one bit at a time, treating everything as having a probability that's a power of 1/2\. Arithmetic/range encoding uses more accurate probabilities and then packs them together, letting you save a fraction of a bit with every symbol. As an analogy, consider how to encode 3 decimal digits. You could spend 4 bits on each digit, and it would take 12 bits total. Or you could combine them into a single 10 bit number.

</comment><comment> brian-armstrong I really enjoyed this article. It's just enough detail to leave you thinking, "Hey, I should write my own DEFLATE library"

<comment><USER11> Would be fun to do that in Haskell

Edit: [https://github.com/GaloisInc/pure- zlib?files=1](https://github.com/GaloisInc/pure-zlib?files=1). Beautiful they've done just the decompression! Challenge awaits

</comment></comment><comment><USER12> The article mentions LZW. In terms of algorithm elegance LZW should be head and shoulders above 'deflate'. LZW got patented which (aside gifs) screwed up its popularity. 'defalte' allows custom dictionaries that adds some extra complexity.

I'd strongly disagree with _In many people 's eyes compression was still synonymous with run-length encoding_ as arj was extremely popular to the point it was used to cure full stealth viruses (v512 for example). _hard to provide references now but wiling to search some old bbs_

Other than that - fairy nice/nostalgic article.

<comment><USER13> I very recently had to look at an LZW decoder, and it is indeed more elegant than Deflate as described here. However, LZW doesn't have a Huffman post- encoding step, so it does not compress as well as Deflate.

<comment><USER3> Asymptotically it doesn't matter - that's the genius of all LZ schemes (LZ77 =~ LZSS, LZ78 =~ LZW). All the other details just take care of the early pre asymptotic stage.

<comment> TD-Linux No, it still does - LZW assumes flat probabilities for each entry in its dictionary. After a while, both the encoder and decoder will know more accurate probabilities, but LZW has no way to take advantage of that.

<comment><USER3> But asymptotically it doesn't matter because the LZ78/LZW code words represent longer and longer sequences, with the more probable sequences getting shorter codes. The additional letter (per each code word, in the case of LZ78) doesn't matter asymptotically. Practically, of course, it does.

<comment><USER13> How do the more probable sequences get shorter codes?

<comment><USER3> That's basically the objective of every compression algorithm. Huffman coding does it directly by construction, Lempel Ziv do it indirectly - in LZ78/LZW, you get essentially and asymptotically (though randomly, per individual sequence) code length proportional to log_2 prob(sequence), because the sequences added to the dictionary are added by order of probability.

In LZ77/LZSS, it is basically the same, except the dictionary is not constructed explicitly but rather implicitly from the history. In both cases, you need to refer to Lempel and Ziv's proof that indeed, the codeword length converges to log_2 P(sequence), which is optimal by the Shannon Source Code Theorem.

I highly recommend Cover & Thomas book on information theory if you find that interesting and are not afraid of math. And Shannon's original 1949 paper ("A theory of communication") is surprisingly and exceptionally readable - <USER2> it doesn't cover LZ for obvious reasons.

<comment><USER13> Okay, so it's because the most probable sequences are more likely to be encountered first, which assumes the data is more or less uniform. (I.e., it doesn't work as well when the data starts with a large blob of random data, and the more regular data comes afterwards. But I guess that's true for most general compression algorithms. At least LZW has a reset code.)

I'll see if I can find the C&T book. Shannon's paper was part of the course material at uni, IIRC; at least I think I read it long ago.

<comment><USER3> It's not just a matter of encountered first, it's of encountered more often ("encountered first" is a special case of "more often"). But you really need to delve into the proof to figure out why that's enough for optimality. At the very least, _I personally_ don't know of a simple intuitive way to explain that.

It is important to remember what each compression algorithm is optimal for.

LZ77 and LZ78 are optimal for observable markov sources -- that is, sources in which given x recent consecutive outputs (where x is the markov order), the distribution of the next symbol is fixed. While this is rarely ever the case, it is reasonable to assume that e.g. English text is a 10-th order markov model (with respect to characters) or 3-rd order (with respect to words).

The source you described is NOT markov overall, although it might be markov in the tail (past the random blob). Asymptotically, of course, it is markov :)

<comment><USER13> Hmm... I thought that in LZW, once a sequence has been assigned a code, it can never be assigned a shorter code; codes are assigned in order, and they only get longer over time (until a reset is triggered). So the frequency of a particular sequence cannot influence the length of the code assigned to it. I guess I'll read up on it a bit more until I comment further. I do see that it could end up to be optimal in the asymptotic case.

<comment><USER3> You are, of course, correct. However, in the same way that you cannot reassign codes to the letter A-Z and yet you can compress by assigning a _longer_ code to a _sequence_ (the code is longer but is still shorter than the individual codes needed to represent the sequence), then even though you cannot reassign a code, you will get a new code for a longer sequence that includes the shorter sequence, represents more letters, and overall is more likely to occur than the shorter sequence followed by something else.

Assuming no reset, over time the shorter codes fall out of use because the longer ones eventually include everything they represent, extended by all possibilities (with the more probable extensions getting shorter codes by virtue of appearing more often / earlier).

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER14> > This paragraph has five-hundred and sixty-one lower-case letters in

It must have taken some trial-and-error to make this sentence correct, since the number itself adds some letters.

<comment><USER15> Pick a value that's close enough, and tweak everything else until it's true :) This post has one hundred and thirty-two chars in it.

<comment><USER16> This, ten

</comment></comment></comment><comment><USER17> The most clever part of deflate is actually how they store the huffman trees without storing the frequencies but rather the length in bits of each symbol. Combining the length of each symbol with its lexicographical ordering is enough to uniquely specify the actual tree.

Ok, some research showed me that the idea can probably be traced back to Schwartz & Kallick 1964 (can't find the text), popularized by TAOCP vol. 1...

<comment><USER18> The technique is known as "Canonical Huffman" and is quite elegant indeed. It is best understood by considering the codes as binary numbers, then the tree structure naturally appears. There's a very short algorithm to generate the appropriate encoding and decoding tables in the JPEG standard and it involves not much more than shifts and adds.

</comment></comment><comment><USER19> I wouldn't say that entropy coding is "separate" from dictionary / window compression. DEFLATE surely isn't the first algorithm to combine LZ77 and some form of entropy coding, and surely won't be the last.

I highly recommend that everybody go out and build an INFLATE implementation; it's a lot of fun.

Mine can be found at [https://github.com/magcius/libt2/blob/master/t2_inflate.h](https://github.com/magcius/libt2/blob/master/t2_inflate.h)

<comment><USER20> I did this in javascript years ago (I had done snappy first, but wanted smaller data). It's fun and not too difficult.

The deflate side seems a little more complex. I played with it while on vacation about a month ago, but stalled after doing the LZ77 and huffman table generation, because I really just wanted to learn on the LZ77 bits worked, and I don't actually have a project that needs it.

</comment></comment><comment><USER21> Anyone know what's the history behind that setjmp/longjmp as 'poor man's exception' error handling? Since C works perfectly fine without exceptions, why add such a hack? I've seen this in libpng and I think libjpeg has it too (one of many reasons I'm avoiding those libs).

<comment><USER22> C lacks "zero-cost" exceptions, where the code should be just as fast as it would be if no exceptions were involved as long as one isn't thrown. Although they are _much_ more expensive when thrown than checking a return value and using indirection for any logical return values, they are a bit cheaper when you don't expect the exception to happen very often, or don't care about performance if it does (e.g. a corrupt png or jpeg file).

My dream is something like the UX of Swift's exceptions (basically normal C error handling except for it does all the tedious pointer work for you) with the ability to tell the compiler to either implement the exceptions as return values or stack unwinds for a particular code path with the flick of a switch.

<comment><USER23> My big hope in this regard is Rust, which has a very good track record if implementing good, usable, zero-cost abstractions.

If they manage to implement zero-cost futures [1][2], maybe they will also implement sime kind of zero-cost error handling abstraction (either via exceptions, or via some other useful abstraction for that purpose).

[1] [https://aturon.github.io/blog/2016/08/11/futures/](https://aturon.github.io/blog/2016/08/11/futures/)

[2] [https://news.ycombinator.com/item?id=12268988](https://news.ycombinator.com/item?id=12268988)

<comment><USER24> Rust's abstractions are only zero cost if you're looking in the wrong place for your costs. The Result / try! stuff in particular I would be very surprised if it is faster than a good exception approach, for example, particularly the happy path. It's amortised throughout though, while different exception throwing mechanisms have dramatically different performance, so people's intuitions aren't reliable.

I like Rust a lot. Zero costs is not a reason though; my reason is no GC and not as bad as C or C++.

<comment><USER25> Don't forget exceptions aren't free themselves; you need landing pads, etc, all of which are not required.

</comment></comment></comment></comment></comment><comment><USER26> Deflate is so ubiquitous (being a web standard) that I'd like to see a hardware Deflate encoder/decoder.

It really needs to be in the core instruction set of x86 & ARM.

<comment><USER18> [http://www.xilinx.com/products/intellectual- property/1-7aisy...](http://www.xilinx.com/products/intellectual- property/1-7aisy9.html)

</comment></comment><comment><USER27> Using additional bits to encode lengths isn't really that different from having a marker indicating if the payload is a match vector or not. In fact, it could be argued to be less efficient for certain types of data depending on the huffman distribution since all "words" are larger than a byte. It also prevents you from encoding longer matches. I treat the decision as a compromise between the two extremes that "worked" for the majority of types of data "good enough" for some definition of both quoted words. The entire idea works well enough but I wouldn't go out of the way to declare it a marvel of engineering or anything.

<comment><USER28> "it could be argued to be less efficient for certain types of data" This is true of all lossless compression.

<comment><USER27> You're invoking a stronger "technically correct" interpretation of the statement I'm making. Let me qualify. In a blob with many short repeated sequences, the algorithm as presented is better. In a blob with many long repeated sequences, a marking bit is better. Honestly, neither example is that farfetched.</comment></comment></comment>


Geeks suck at marketing - <AUTHOR> https://blog.ungleich.ch/en-us/cms/blog/2018/07/31/geeks-suck-at-marketing/ <comment><USER2> So you are marketing your VM data center by talking about how a specific group of non-marketers suck at marketing?

That's bad marketing.

You need to produce communication that reaches those people, who are interested in VM data centers AND have the power to directly or indirectly make the buying decisions.

Now you only reach people who are thinking about marketing in some way. Not good.

Now, the other thing is, you compare buying display ads to a content marketing campaign. That's what this piece is ultimately about.

Well, guess what, you can pay for adverts or you can pay somebody to write blog posts etc (what you call "talking" in the article), you still have to pay and the challenge is to hold these costs down.

You make a value statement, you say you'd rather apply a means of advertising that you can stand behind for humanitarian reasons. I applaud you for this!

<comment><USER3> I love one of your sentences most (non ironic): "You need to produce communication that reaches those people, who are interested in VM data centers AND have the power to directly or indirectly make the buying decisions."

Who do you think those people are? ;-)

<comment><USER2> Typically the MBA types who maintain the company's ledger.</comment></comment></comment>


C# Source Generators - <AUTHOR> https://devblogs.microsoft.com/dotnet/introducing-c-source-generators/ <comment><USER2> So the main difference with stuff like T4 is that it's written in C# and doesn't rely on the IDE but is part of the language definition?</comment>


Ask YC: Part-time consulting while working on a startup, good idea? - <AUTHOR>

My business partner and I have been working on our startup for sometime now. We do not have funding yet and have been bootstrapping it. I work full-time as a technical architecture consultant, while my business partner is 1 day a week on his job and rest on the startup.
I made a decision to quit my current contract and go full-time on my startup so that we can get it to market fast. Once I notified my client, I was offered an option to work 10 hours per week on the current project and have the rest of the time for my startup.
What are the pros/cons of doing or not doing this? Is this advisable? Should I just put 100% on startup for 3 months?
Thanks for all opinions/advice in advance. <comment><USER2> So, this is exactly what we've done. Briefly:

(1) The comments saying that it's hard-to-impossible to transition from consulting to product are correct. Two problems: (a) it takes more than just discipline to walk away from money --- it takes reckless speculation, and that's painful; (b) consulting hours don't bucket neatly, and don't schedule neatly, and so you're constantly being disrupted.

(2) That said, we now have multiple full-time people working on product, which we're launching soon. The classic consulting model is a pyramid scheme, where people claw their way to "partner" and get fat off profit sharing. In the product-consulting model, instead of setting aside money for partners, you pay full-time devs. This appears to work.

If I had to choose between VC and consulting, and my consulting practice was lucrative (ours is: software security billable hours are expensive), I'd do consulting again, even though it cost us a year. Reasons:

* I'll trade a year for control over my own destiny.

* The year is a false economy, since, for _most companies_ , getting funding takes many speculative months.

* There are things about running a consultancy as a business that translate to running a product business, and running a business is a valuable skill.

* The networking and customer face time you get from consulting is hugely valuable.

* Time to market is simply overrated.

</comment><comment><USER3> Q1) Do you have the resources to survive if it takes 6 to 8 months instead of 3 months? Being done with a product in month 3 doesn't mean sales will start rolling on month 3, day 1. Think revenue date, not launch date.

Q2) Do you have the mental self-discipline to focus on one thing at time... even when you have two things on your plate or are you more productive when you only have one thing on your plate?

Q3) Have you started talking with your future customers yet? If not, talk to them first. Confirm they need what you're building. Do they need it bad and now? Assumptions can hurt.

</comment><comment><USER4> Two little stories: A friend of mine has been trying since 10 years to transform his consulting company into a product company - without success. The projects he has implemented were too expensive in terms of development and maintenance to have enough time apart. And none of the projects was general enough to produce a code-base that could serve as a product for several customers. Another friend implemented a little access-based niche-domain tool during an internship for a big enterprise that he was able to resell to several competitors and even several times within the same enterprise. He is working around 5 hours per week and makes a confort living. Honestly, I think the first story gives the more likely scenario.

<comment><USER2> I'm biased to the security software market, but note that it's a market where companies have constantly tried to play the "productize-the-service" card. What I've observed is, this almost never works.

In fact, taking a consulting approach to product development (working closely with a first customer) also slowed us down drastically, and our product actually has very little to do with our service (what we're launching has nothing to do with reverse engineering software).

There are also some high-profile failures behind our strategy: building a straight-up consultancy to "fund" a full time product group.

But then again, who knows? Building a business is hard. It is not like applying to college, where you clear some "acceptance" hurdle and then you do your homework and everything works out. With the best people, the most credible strategy, and even the right market timing, circumstances can still screw you over.

</comment></comment><comment><USER5> I can only think of two good reasons to do consulting while you're working on your startup:

1) You don't have enough money to survive without the income from consulting. 2) Your consulting gig will help with sales in your startup (specifically, you think the company you're consulting for will buy your product once it is built).

Even if reasons 1 and/or 2 are at play, it still is a distraction. I do consulting for reason 1, but I plan to outmode that with an upcoming angel round. I am not doing all that much consulting these days anyway, but I still feel like it is a hindrance. See a semi-recent blog post about it here:

<http://blog.jabbik.com/2008/03/momentum.html>

<comment><USER6> I can think of one other good reason to consult when you're working on a start-up ---- the consulting project that you're working on is something that will teach you something that you need to know in order to make your start-up successful. There's nothing better than learning on someone else's dime. Especially if the company for which you're consulting has some particuarly smart people or cutting-edge resources that you wouldn't have access to otherwise.

</comment></comment><comment><USER7> Contract work can be a great way to bootstrap the business if you're careful about it. I chose projects that we're high paying and easy to walk away from (corporate training, wrote a second edition of a book, strange enterprise documentation gigs).

Once we had a product we could sell there was a temptation to push it on any person with money. The problem with taking those kinds of customers is that it ties down your product. Keeping an independent stream of income would have given us more flexibility after we'd launched the product.

Also, some people say that having side projects makes it hard to focus on your main project. I found the opposite, having a main project that I loved made it very hard to focus on the side projects.

</comment><comment><USER8> It's a decision you have to make depending on your finances.

If you can work for 3 months on the startup without feeling any financial burden I'd go with that but if these 10 hours of work a week translate into a good chunk of money it may be worthwhile to do it.

A few questions to ask yourself:

1\. Could the 3 months potentially turn into more?

2\. Can you set the hours you will be working to accrue the 10 hours or will the client be calling you whenever they have issues? The first scenario is more preferable.

3\. If you do the 10 hour option you can try to do it the same day as your partner so then you have more time together.

Just my thoughts..

</comment><comment><AUTHOR> Thanks for everyone's input. To answer some questions and comments being posted: 1\. I do have enough funds to last 6 to 8 months on my own if I had to.

2\. I'm not sure if I'm going to learn much on the project I am on because now my role is more advisory, but I will interact with potential service providers for my startup.

3\. I do think that I have issues focusing on two projects at the same time because I like to do a good job on whatever I'm doing. This also means that there is a good possibility of me spending more than 10 hours a week on the project.

4\. We have been talking to a small set of our future customers, and they can't wait for it to launch. And it needs to launch now.

I've made the decision to go 100% on my startup, and not worry about consulting. Someone reminded me how I worked through college and have always regretted it, wishing I would've have just taken a loan and paid more attention, had fun, etc. I don't want to look back at this and say "If I just did it 100%"

Thanks for all your advice and opinions, it really did help.

</comment><comment><USER9> 10 hours per week is not that much, stick with the current project, plus, its good to have some cash flow for ramen :-).

<comment><USER2> 10 hours billable --- 2 out of 5 work days --- can easily wind up being 20 hours of wall clock time, factoring in ramp-up, off-the-clock support, documentation, non-billable research time, and then the overhead of dropping in and out of your product.

Of our original 3-person founding team, excluding all the hires since then, we've found that none of us can keep up with the FT devs, even when we work out one-week scheduling bubbles. This is obviously a classic "flow" problem.

I'm from dev originally, not consulting, and I was surprised by how expensive (timewise) services work is.

<comment><USER9> Services is expensive, but you should be billing for some of those things like off-the-clock support, documentation, research, etc.

<comment><USER2> Depends on the customer, but we never bill for rustproofing. Get used to bending over backwards for people who pay you now; good survival skill. Truism: it's the perception of customer service that people appreciate, not the execution of it.

</comment></comment></comment></comment><comment><USER10> Not if you can help it, unless the consulting directly feeds into your product. I find it's better, mentally, to focus on something 100%, rather than worrying about both your product and your clients problems, unless the latter is really something you can do on autopilot.

<comment><USER2> But you consult. Do you think you're making a mistake doing that? Or is your web page just out of date?

<comment><USER10> I said, "if you can help it", but unfortunately, I can't help it. I speak from experience when I say that it's distracting, and that periods where I've been able to focus 100% on something are vastly more productive.

<comment><USER2> Hear that. If you could choose to take a typical predatory VC round, putting a few mil in the bank and losing control of your board, would you trade positions from where you are now?

<comment><USER10> Well, if we're talking about stuff that's not very likely, I'd be pretty happy to go through the YC program, actually. Barring that, finding a niche product that's sustainable, but not 'get rich quick' material would make me happy.

I saw the bad side of what VC's can be like at Linuxcare, and it's pretty ugly.

</comment></comment></comment></comment></comment><comment><USER11> I would recommend putting 100% on the startup for 3 months. I find that I can get a lot done if I put 100% of my time into anything for 3 months. You'll probably get enough done that people will be able to get an idea about what the product could become. By that time you'll either be able to find investors or realize that it's time to move on (or go back to full time consulting).

There is a reason the yc session is 3 months long. That's probably about all you'll need to prove your concept. Save up the bare minimum you need to survive and go. Good luck either way!

</comment><comment><USER12> Its easy to get consumed by consulting and not make much progress on your own products. With contracting, you must spend time finding new contracts, accounting, and sometimes even legal issues.

I wasted years contracting instead of working on my own projects full time. All my energy was drained by the clients I had.</comment>


Ask HN: What are the best resources to properly learn Scala? - <AUTHOR> Hello, I've started to work with scala and I think it's a beautiful language. that said, I'd like to be better at it. What are some good books to get thst teach about the nitty gritty details of the language and it's use in production? <comment><USER3> I'm the author of a couple of books on Scala: Essential Scala and Advanced Scala. You can find them here: [http://underscore.io/books/](http://underscore.io/books/)

Depending on what your background and what you want to learn, different books will be appropriate:

\- Our books are focused on what we consider the most important concepts for functional programming and how they are implemented in Scala. If you want to know about algebraic data types, type classes, and so on, and how to implement them in Scala or use the implementations in the Cats library our books are a good choice. We emphatically do not teach all the language. If you want to know about OO features like self types and trait mixins don't turn to our books.

\- Odersky's book has a lot of detail. It does a good job of presenting all of the language but doesn't IMO do a good job of showing you how to use the language.

\- FPiS is a very good book that does what the title suggests. My main criticism is it can take a bit long to get to the point and it doesn't talk about libraries you might use in practice like Cats or Scalaz.

\- The Coursera courses I have taken are fairly good, but don't do a great job of talking about the higher-level design concepts.

I haven't read other books, so can't comment on them.

Finally, if you don't have a background in formal CS our free book Creative Scala might be useful. The current draft is at [https://dl.dropboxusercontent.com/u/8669329/creative- scala.p...](https://dl.dropboxusercontent.com/u/8669329/creative-scala.pdf) and the source is at [https://github.com/underscoreio/creative- scala/](https://github.com/underscoreio/creative-scala/)

<comment><USER4> I'm interested in hearing about your team and diversity discounts, it doesn't look like there is any information on them on the site (unless I missed it!)

<comment><USER3> Team discount is 20% off for purchases of ten or more copies.

Diversity discount ... I don't think we have a policy for books, but for training we offer 50% off to people who are under-represented in tech and I'm happy to offer the same for books.

Email me (noel at underscore.io) to arrange either.

</comment></comment></comment><comment><USER5> There is a course on Coursera thaught by the Scala creator: [https://www.coursera.org/learn/progfun1](https://www.coursera.org/learn/progfun1)

<comment> gingerbread-man It's actually a series of 4 courses, 5 if you include the "capstone project." I've watched a few of the videos-- it's pretty decent, but intended for those who don't have much experience with functional programming. The first course I believe is modeled off the Structure and Interpretation of Computer Programs.

I think the answer to this question really depends on where you're coming from. An experienced Java developer will need to focus on different things than someone who has spent the last decade hacking Common Lisp or OCaml.

<comment><USER6> I would agree and disagree.

The first course is the one that's dedicated to Scala the language, whereas the rest are more focused on reactive/parallel programming that happens to be in Scala.

The first course is good, and acts as a good introduction if you've encountered functional programming before.

</comment></comment></comment><comment><USER7> I found _Functional Programming in Scala_ to be the best source: [https://www.amazon.com/Functional-Programming-Scala-Paul- Chi...](https://www.amazon.com/Functional-Programming-Scala-Paul- Chiusano/dp/1617290653/)

<comment><USER8> I would say that rather depends on what OP means by "properly".

If OP already knows Scala and would like to get better at it _and_ one assumes that "Scala as Haskell in the JVM" is more desirable than "Scala as a better Java", then yes, that's a great suggestion.

If OP wants to learn Scala from the ground up, I disagree pretty strongly with starting there. It's a great book, well written, with a wealth of challenging and mind-opening exercises, but I wouldn't recommend that as anyone's first exposure to the language. The second or third, sure, but it purposefully restricts itself to what its authors consider a sane subset of the language.

I happen to agree with their definition, but, in my experience, knowing a programming language is about reading it as well as writing it, and by restricting your learning to a subset of the language, you'll find yourself unable to understand perfectly valid, normal code - code that is part of the standard library, for instance.

I would rather suggest reading Odersky's book while allowing oneself to skip large chunks that go rather in too much details about fairly useless things like the XML API. Or going through scala-exercises ([https://www.scala- exercises.org](https://www.scala-exercises.org)). Or the coursera class - <USER2> first incarnation was pretty good, I assume the new one is at least of the same caliber. They might not make you an expert overnight, but they'll get you to know most of the Scala features you're likely to encounter in the wild, and to be able to read and learn from most Scala OSS projects.

<comment><USER7> I actually learned Scala from this book, and the only languages I had used in production before were Python and Fortran. I don't think learning the functional subset first made it harder for me to learn the object-oriented and imperative parts later on. Plus, understanding the functional subset gives you a better understanding of many of the internals, such as what `for` comprehensions are actually doing.

Of course, if you _just_ read one book and stop there, I'd agree. But most people will continue learning from tutorials, videos, stackoverflow, etc.

I've also seen several other people learn Scala this way, usually without knowing any functional programming before, and really haven't seen any signs of issues wrt not understanding code written in other styles.

</comment></comment></comment><comment><USER9> Depends on your level. Scala is extremely powerful, and so can be you. Just pick progressively more and more advanced books, and you'll be there in no time. Compiler's always got your back. Seriosly, I've been on search for good language for last 8-10years, always been unhappy. 2 years ago I started working with scala, and never looked back. I am happy with it and I can see myself doing it for as long as it's around and there is competetive market.

<comment><USER10> Until you try clojure or even kotlin

<comment><USER9> I mastered kotlin to dev some mobile apps. Having years of C# behind me, it does not impress me much.

Clojure only for fun, I need types. But lisps sounds fun. Can't comment more. Prehaps you're right ;)

<comment><USER10> There is `clojure/core.typed` if you want types or `clojure.spec` if you want specs. What I love in Clojure (and in any lisp for that matter) is that you can imlement language features if you need them and the language itself provides you with many but only if you need it (optional dependencies). This gives you flexibility which you can only dream of if you use scala.

<comment><USER9> I know of clojure typed. But im worried it needs to be first class. I.e i need all my dependencies to be typed, as that is my api and it usually covers large surface.

Id also be careful with your 2nd statement. Scala has macros.

Im happy clojure is on the block, dont get me wrong :) each to their own. Type system is incredibly important to me and my productivity.

<comment><USER10> Scala macros are not like clojure macros. Clojure is homoiconic while scala is not so you really can't compare the two.

</comment></comment></comment></comment></comment></comment><comment><USER11> I would recommend the stairway book and the coursera course. I would also steer away from the some of the other recommendations posted here, including "Functional Programming in Scala", which are basically teaching you how to write Haskell in Scala. I don't think this is a good way to learn Scala because I don't think Haskell-style code makes for effective Scala.

<comment><USER12> >I don't think this is a good way to learn Scala because I don't think Haskell-style code makes for effective Scala.

Why is writing functional code in scala not effective? How would one go about understanding 'for comprehensions' in scala without understanding what flatMap/map is used for in lieu if exception. Or why pattern matching is used.

<comment><USER11> In my opinion, functional programming, like most programming terms, including object-oriented programming, means very different things to different people. For example:

1\. The LISP family of languages encourages a style of functional programming that eschews static type systems, relies heavily on macros and algorithms involving singly-linked lists, and doesn't mind exposing the ability to mutate state.

2\. The ML family of languages, of which I am the least familiar, are statically typed, but are also OK exposing mutable state and do not track any effects in their type systems.

3\. Haskell heavily pushes reliance on Monads, monad transformers, and effects/state tracking through the above.

4\. Proof assistants and similar languages like Coq go beyond tracking effects in the type system; they track values themselves, and essentially require you to prove mathematically that your program is correct.

So, given the above, what do you mean by functional programming? From the talks I've seen given by the creator of Scala itself, I believe he sees Scala as fitting in with the ML family languages moreso than Haskell.

Also, the way Scala's type system and runtime currently work, I don't think it's a good fit for Haskell-style programming. I tend to find monad transformer-based code in Scala exceedingly ugly and difficult to read compared to Haskell. The underlying runtime itself also has little mechanism for optimizing monad-based code, whereas the GHC compiler has probably man- years of effort spent into optimizing code using monads.

</comment></comment></comment><comment><USER13> I used [https://twitter.github.io/scala_school/](https://twitter.github.io/scala_school/) to learn, I found it useful.

<comment><USER14> That tutorial is good. But be mindful that in it there is no explicit distinction being made when using Scala standard library types and Twitter library types.

e.g: Futures used in there are from Finagle rather than scala.concurrent.Future

<comment><USER15> Just to add on to this incase anyone is wondering, Twitter has a library that allows you to use the different futures together ([https://github.com/twitter/bijection](https://github.com/twitter/bijection)) if you need to.

</comment></comment></comment><comment><USER16> I started with Scala a year ago for work. I used a mixture of the resources listed here, but I found this [1] series the most valuable, after the other resources gave a basic foundation.

[1]: [http://danielwestheide.com/scala/neophytes.html](http://danielwestheide.com/scala/neophytes.html)

</comment><comment><USER17> The Artima book by Odersky is the bible, and so you need that one.

For reference, I like [http://naildrivin5.com/scalatour](http://naildrivin5.com/scalatour) although it's 2.8 rather than the current 2.12.

For style guide, I'd go with Li Hiayo's blog, notably the Strategic Style series: [http://www.lihaoyi.com/post/StrategicScalaStylePracticalType...](http://www.lihaoyi.com/post/StrategicScalaStylePracticalTypeSafety.html)

</comment><comment><USER18> I read scala for the impatience, but once you know clojure, haskell, erlang and other languages the scala language seems to be one more language, a better java but nothing that really surprise you. I know that using scala you can make your programs very difficult to read using obscure notations for operators. I like to use scala as a repl for exploring java classes, a little more useful that clojure in this regard.

</comment><comment><USER19> As you are asking for books \- Scala for the impatient ([http://www.horstmann.com/scala/](http://www.horstmann.com/scala/)) \- Programming in Scala ([https://booksites.artima.com/programming_in_scala_3ed](https://booksites.artima.com/programming_in_scala_3ed))

I found those books very good resources. The Scala website lists a few others ([https://www.scala-lang.org/documentation/books.html](https://www.scala- lang.org/documentation/books.html))

Aprt from that I found Daniel Westheides blog a very good starting point ([http://danielwestheide.com/scala/neophytes.html](http://danielwestheide.com/scala/neophytes.html))

<comment><USER20> I learnt Scala from Odersky's "Programming in Scala" and can fully recommend it. I didn't have any functional language experience beforehand.

</comment></comment><comment><USER21> If you want to learn "functional" aspect and not only library and syntax, I would recommend SICP.

[https://mitpress.mit.edu/sicp/full- text/book/book.html](https://mitpress.mit.edu/sicp/full-text/book/book.html)

<comment><USER22> How much of this book is "functional" aspect and how much is everything else? Going through the TOC, it looks like the book tries to start from 0.

Would you recommend any particular sections for someone who's looking to only understand the functional parts?

<comment><USER23> Your comment tells me you should start from the beginning and read the whole thing. :)

Just give it a try. I was hooked after first chapter!

</comment></comment></comment><comment><USER24> Scala Cookbook: [http://shop.oreilly.com/product/0636920026914.do](http://shop.oreilly.com/product/0636920026914.do) (most of it is available online here: [http://scalacookbook.com/](http://scalacookbook.com/))

Once you are done with this, you can pretty much pick any of the projects here and start reading the source code:

[https://github.com/lauris/awesome-scala](https://github.com/lauris/awesome- scala) [https://github.com/adamw/awesome- scala](https://github.com/adamw/awesome-scala)

Reading code is the probably the best way to learn any programming language IMO

</comment><comment><USER14> "Another tour of Scala": [http://naildrivin5.com/scalatour](http://naildrivin5.com/scalatour) was informative for me.

It's a bit dated, mostly targetting Scala 2.8. Current version is 2.12. But should not be fundamentally different.

</comment><comment><USER25> I know this question is mostly about resources in dead tree format, but don't forget there are sometimes local events and meetup groups that can help you learn Scala (and other languages too!).

For example...

If you live in Dallas/Fort Worth and would like to learn Scala then you should know that the local Scala Enthusiasts group is hosting a Scala Essentials workshop on April 7th, 2017. This will be an all-day, hands-on event for Scala beginners. We will start with a language intro and finish up coding a REST server. More details can be found here: [https://www.scala- enthusiasts.com/scala-essentials-workshop-...](https://www.scala- enthusiasts.com/scala-essentials-workshop-2017-04/)

</comment><comment><USER26> I reccomend "Just Enough Scala for Spark" tutorial from Dean Wampler. It was scheduled in Strata NYC, Singapore, and San Jose. Good part: practical. No "weird" or "religion-like" stuff. Just get s __t done today.

It's archived in SafariBooksOnline: check [https://conferences.oreilly.com/strata/strata- ny-2016/public...](https://conferences.oreilly.com/strata/strata- ny-2016/public/schedule/detail/51593) before applying to Safari just in case.

</comment><comment><USER27> If you're looking for books I recently made a list of good books for Scala - [https://blog.matthewrathbone.com/2017/02/14/scala- books.html](https://blog.matthewrathbone.com/2017/02/14/scala-books.html).

Otherwise Scala School is awesome: [https://twitter.github.io/scala_school/](https://twitter.github.io/scala_school/)

When I worked at Foursquare we would send folks to the Scala School website as homework. :-)

</comment><comment><USER15> [http://rerun.me/](http://rerun.me/)

This blog has some good articles on how to work with Futures properly as well as some Akka stuff.

</comment><comment><USER28> I would recommend Introduction to the Art of Programming Using Scala: ([https://www.amazon.com/Introduction-Programming-Chapman- Text...](https://www.amazon.com/Introduction-Programming-Chapman-Textbooks- Computing/dp/1439896666))

It's very practical, however it approaches the language from a completely novice perspective. If you have years of programming experience, it may be a little redundant for you.

<comment><USER29> Same

</comment></comment><comment><USER30> You should check this [https://www.tutorialspoint.com/scala/](https://www.tutorialspoint.com/scala/) & [https://bigdatauniversity.com/learn/scala/](https://bigdatauniversity.com/learn/scala/)

</comment><comment><USER31> I learned by reading "Programming in Scala" by Martin Odersky who invented the language. [https://www.artima.com/shop/programming_in_scala_3ed](https://www.artima.com/shop/programming_in_scala_3ed)

</comment><comment><USER32> I hope you don't mind me asking: Is there a specific reason behind you wanting to learn Scala "properly"?

</comment><comment><USER12> piggybacking on this question. I am really struggling with sbt. Is there is a good FP in scala equivalent for sbt that walks you through feature by feature via exercises and examples .

<comment><USER9> Hi. Sorry I don't have a good answer for you, but check sbt docs, it's pretty comprehenful, but it's not best.

Your best bet is to visit opensource projects and look into their build.sbt's, how they do things -they're usually not big enterprise projects and you can quickly 'get' how they setup what.

Always feel free to visir /r/scala and ask sbt-related question there, there is active community happy to answer both sbt/scala/fp related questions.

</comment></comment><comment><USER33> I liked Scala for the Impatient but I'm pretty novice.

</comment><comment><USER34> What's the market demand for Scala like?

<comment><USER35> Very good on the coast, high paying, fun work. Pretty sparse otherwise. I'm sure there are exceptions, but it does seem like it is growing. And, ironically, the main reason companies cite not choosing Scala (or only dip their toes into Scala) is lack of Scala engineers.

<comment><USER34> Interesting, thanks. I suppose companies on the coast may tend to be more open to new stuff.

</comment></comment></comment><comment><USER36> If you aren't leaning Scala for work or a specific project that requires it , have you considered Kotlin? I have found the language much more productive in my day to day, after initially discovering Scala and digging deeper.</comment>