Practical Text Classification with Python and Keras - <AUTHOR> https://realpython.com/python-keras-text-classification/ <comment><USER2> Neural networks often perform worse than bag of words for text classification. They compared to a bag of words baseline, but they didn't tune the regularization parameter, which is important.

They should probably have also tried tf-idf and n-grams.

<comment><USER3> Definitely not true. Transfer learning a model trained on a corpus then fine tuned on labeled classes is current SotA approach [1].

[1] [http://nlp.fast.ai/category/classification.html](http://nlp.fast.ai/category/classification.html)

<comment><USER4> You mention a single counterexample, but is it "definitely not true"? I think there is a strong publication bias: papers will report when they improve on the baseline, but that's likely not representative for the common case (e.g., limited data, no pretrained model available, no time for extensive parameter tuning).</comment></comment></comment>


Yak Tack – remember words with spaced repetition - <AUTHOR> https://yaktack.com <comment><USER2> Cool tool, btw your post seems to have been killed instantly by HN's server, I vouched it but it might have been too late

</comment><comment><AUTHOR> Hmm. Bummer. Maybe because my account has no cred?</comment>


Stop Obsessing: 7 Areas Where Perfectionism Hinders Good Writing - <AUTHOR> https://blog.hubspot.com/marketing/perfectionism-streamlined-writing-process <comment><USER2> Can't say I disagree with this. The amount of time I've spent worrying about writing the perfect story is ridiculous, and that perfectionism has been a reason I've missed quite a few trends I probably should have written articles about.

That said, you know what makes this even worse?

Having a SEO plugin installed. Having the software yell at you about 'readability scores' and 'transition words' and not using the same word at the start of three sentences in a row will make you even more obsessively picky about your writing than merely being a perfectionist will.</comment>


Ask HN: What is the most rewarding outdoor or physical activity you do?  - <AUTHOR>

I'm trying to find new outlets for exercise and km curious what others enjoy. <comment><USER2> I've recently taken up backyard blacksmithing. I was able to start for around $100 and it has been a great learning experience. Also, I get to play with fire and hit metal with a hammer; it's a great stress reliever (:

</comment><comment><USER3> Hiking and backpacking are fantastic. Lots of areas have hiking clubs you can join.</comment>


Twitter CEO Intervened to Keep Alex Jones, Richard Spencer Accounts Active - <AUTHOR> http://thehill.com/policy/technology/404826-twitter-ceo-personally-weighed-in-on-companys-handling-of-alex-jones <comment><USER2> [https://news.ycombinator.com/item?id=17905679](https://news.ycombinator.com/item?id=17905679)</comment>


The relevance of Second Life and open-ended 3D environments - <AUTHOR>

Hi folks. In the wake of Google's announcement, I just posted the same question on a Brazilian list for workers in "all things web" and it turned out SL is pretty much dead and buried here. I would like to know how alive it is in other places. Any comments? <comment><USER2> <http://www.useit.com/alertbox/981115.html>

Ten years old, but apparently still relevant.

Best part, the part I always remember:

 _Note that 3D works for games because the user does not want to accomplish any goals beyond being entertained. It would be trivial to design a better interface than DOOM if the goal was to kill the bad guys as quickly as possible: give me a 2D map of the area with icons for enemy troops and let me drop bombs on them by clicking the icons. Presto: game over in a few seconds and the good guys win every time. That's the design you want if you are the Pentagon, but it makes for a boring game._

<comment><USER3> Reminds me of the old game designer joke:

Q: 'What would a game look like if a UI designer designed it?'

A: 'A big button that says: "Click here to Win."

</comment></comment><comment><USER4> I don't know anybody that uses SL. Maybe I'm just having a 'nobody I know voted for Reagan moment'. but I think that SL has the best PR/marketing dept in the world and every 'story' on SL is a press release.

</comment><comment><USER5> i tried SL for a few hours and didn't like it. i don't know anyone that uses it.

its dead and buried, at least for me, here in the southeast US.</comment>


Medium Has a New Logo - <AUTHOR> https://medium.design/logos-and-brand-guidelines-f1a01a733592 <comment><USER2> That "M" looks wrong. If the only thin stroke in your wordmark is the first stem, you've got something seriously unbalanced.</comment>


Four steps we're taking today to fight online terror - <AUTHOR> https://blog.google/topics/google-europe/four-steps-were-taking-today-fight-online-terror/ <comment><USER2> "Third, we will be taking a tougher stance on videos that do not clearly violate our policies — for example, videos that contain inflammatory religious or supremacist content. In future these will appear behind an interstitial warning and they will not be monetised, recommended or eligible for comments or user endorsements. That means these videos will have less engagement and be harder to find. We think this strikes the right balance between free expression and access to information without promoting extremely offensive viewpoints."

I cannot help but think that this will not be applied evenly - that some political content will be allowed and some will not.

<comment><USER3> Aha, so big US companies are our police now.

I share your fear. The "kosher" content would be now defined by some faceless person in NY/SF.

<comment><USER4> They aren't installing a filter into the _browser_ they're OPENLY addressing a problematic issue on YouTube, one of many, many video hosting websites. Nothing stops a terrorist from getting a computer, an internet connection, and hosting his own damn video calling for the murder of women and children.

When you call for violence against non-combatants you're breaking the law in every single western country. If there were only one web browser and the company behind it were implementing universal blocking measures _maybe_ I'd agree with you, but honestly I'd have to think long and hard first. Radicalisation is impossible to survive in the long run as the average power an average individual keeps going up.

<comment><USER5> They didn't say they were going to start removing videos with illegal content, they already do that. They said that they were going to start removing videos that don't break any rules, yet the company deems them unsavory. Which is incredibly frustrating since 1) YT has become the center of our changing culture, and 2) not everyone lines up with the PC Californian culture that dominates large multinational corporations.

<comment><USER6> They didn't say they would remove the videos, instead they will display an "interstitial warning and they will not be monetised, recommended or eligible for comments or user endorsements." Which is not even on the level of a shadow ban, as practiced e.g. on HN.

<comment><USER7> I wonder what effect Google's wagging finger and implied scolding from an interstitial will have on people who stumble across a video they like but is branded as naughty.

I find it an interesting question because:

A) Not every video branded as culturally unacceptable will be. Not every video is as bad as the worst-case hypothetical used to justify the content classification.

The landscape of cultural attitudes differ from California-based content minders. The categorization can be flat out wrong, there will undoubtedly be a small percentage of videos that even the minders see as mis-classified.

B) Social interventionist policies can - and often do - backfire.

e.g.: Teens that deliberately seek out taboo. The allure of R movies, M games, Explicit Lyrics, and underage binge drinking can cause them to live a period of their life less well-adjusted than if that content wasn't aggressively filtered from their lives in the first place.

</comment></comment></comment></comment></comment></comment><comment><USER8> In the west terrorism works trough media exposure. Fighting it in media is good strategy. Violence itself is ridiculously low volume per capita.

If traditional media would step in, it would be major win. Instead of having symbiotic relationship with terrorism, where they hype up the terror, they could stick to reporting the facts and situation and not providing non-stop terror-porn entertainment to audience.

Once people realize that terrorism just one small risk among other must bigger risks (like high rise buildings with flammable material) they stop fearing it and the terrorism stops being effective. It will never completely vanish, but it will not attract radicals as it used to.

<comment><USER9> I don't agree that traditional media outlets "hype up the terror". If you compare the coverage of, for example, the recent attacks in London and the fire you're referring to, they seemed to get about equal billing.

On the smaller scale, violence will get more prominent coverage, yes. But that's just a reflection of public interest, and how much intent makes a subjective difference. News outlets like the BBC or the Guardian seem to be far away from "terror porn", and in the case of the BBC, it's obviously wrong to suggest they're doing it for money, seeing as they're not financed by ads.

<comment><USER10> How often do you hear about shootings in South Chicago in world news? 294 people have died there so far this year![1]

Terrorist events should be treated like any other murders and people will very quickly stop caring. They are just hyped up by the media to drive views and the governments love it because it allows them to expand their power.

1\. [https://www.dnainfo.com/chicago/2017-chicago- murders](https://www.dnainfo.com/chicago/2017-chicago-murders)

<comment><USER11> I used to hold this very view but now I don't watch the news. The reason why I don't hold this view is in the very name and nature of the news, it's "new". Everyday commonplace events even horrific ones are not new and therefore are not part of the news. Choose to not watch.

</comment></comment></comment></comment><comment><USER12> > We have also recently committed to working with industry > colleagues—including Facebook, Microsoft, and Twitter

All of those platforms that are notoriously bad at curbing terrorism and great at promoting the most asinine "I can't believe it's not a troll" social justice concerns.

Also, not to mention present day Twitter gets a lot of money from wahabbi- funding Saudi Royals: [https://qz.com/131532/meet-the-people-and-funds-that- stand-t...](https://qz.com/131532/meet-the-people-and-funds-that-stand-to- profit-from-twitters-ipo/)

</comment><comment><USER13> I truly hope these changes will include the US and Allied forces combat footage.

I've seen videos on YouTube contain some of the most shockingly needless violence with complimentary cheering and derogatory language used by British and US soldiers (often directed towards foreign people) one can imagine, disturbing.

It seemed like footage aimed at radicalising people to me.

<comment><USER14> Do you mean to say that the footage should be censored? I think their crimes should be seen be all.

A better stop-gap measure against the radicalization of American terrorists would be to prevent the DoD from recruiting at high schools, at sporting events, in movies and video games, etc.

</comment></comment><comment><USER15> This is a terrifying step for the internet. These kinds of step lay the groundwork for censorship but fail to consider the impact of these changes in the event that the bad guys come into power or gain influence of these management systems.

<comment><USER16> Supporting violence and hate isn't acceptable in any other format, why should the Internet be excluded.

<comment><USER17> While your statement is quite vague it is no less completely inaccurate. "Supporting violence" is subjective and ambiguous. Are action movies and video games supportive of violence? How about war documentaries?

Furthermore, while "hate" speech is not the best use of our ability to communicate it does fall under the protection of free speech - at least in the United States. In some groups hate speech (however it is you describe it) is acceptable.

Just as you are free to not associate with those groups the internet should be free to express the ideas you disagree with.

You don't have the freedom to interfere or disrupt the freedoms of others.

<comment><USER16> Actually just went back and re-read what I wrote, I'm not entirely sure what the point was I was trying to get across now.

</comment></comment></comment></comment><comment><USER18> This scares me in ways terrorism never has.

I'm not sure terrorism has ever accomplished its goal to effect radical change. But a powerful entity ostensibly protecting us from a group perceived as a common evil that's not worth debating?

</comment><comment><USER19> I am vehemently opposed to babysitting society, but violence is not speech, it's action. That much seems fair. I do worry about step 3. It sounds like that's going to silence a lot of minority opinions or screw over groups who post no-so-liberal commentary videos etc. Anything hiding behind an interstitial is just crackpot conspiracy theory.. Google is a negative influence on society bleep. Zap.

<comment><USER20> It's surely better than the current situation of literal neo-Nazis openly talking on YouTube about "white genocide" and how the "Jewish question" must be "solved", without Google seemingly so much as batting an eyelid.

<comment><USER2> I'd rather have the neo-Nazis spewing their bullshit out in public, where everyone can see how stupid it is. Driving something underground only makes it more attractive in the end - after all, if it was wrong The Man wouldn't need to suppress it, right?

<comment><USER21> The problem with allowing neo-Nazis to spew their bullshit alongside the Diet Coke and Mentos videos is that it creates a normative appearance: "this isn't being removed, so there is something tolerable about it, so maybe it's not so bad."

> Driving something underground only makes it more attractive

See, I think you're wrong, and that the secret to attraction is visibility. As an online advertising company would know.

<comment><USER2> But don't the neo-Nazis and ISIS already have visibility? Again, all I think this does is add the allure of the forbidden fruit.

</comment></comment></comment></comment></comment><comment><USER22> This is clearly in response to the call from several world leaders for more participation from google et al. in combating terrorism online.

Theresa May, as well as some Australian leaders have made very general statements that the big tech companies are not doing enough to fight terrorism.

I expect that these statements are merely a prelude some announcement of legislation, seeking even greater government powers and access to googles data.

I imagine law enforcement would rather have detailed identity information on the people consuming the content, rather than simply having the content removed more efficiently.

So while removing propaganda and violent videos from youtube is just going to drive the traffic elsewhere, at least googles action may help resist further government encroachment on our online privacy.

</comment><comment><USER23>
      to fight online terror

      1. identify extremist and terrorism-related
         videos

      2. increase the number of independent experts

      3. a tougher stance on videos

      4. expand its role in counter-radicalisation
         efforts

But I'm not terrified by things that happen online. Least of all videos of events in the past.

And, to be honest, I've seen plenty of NSFW videos.

Let's not kid ourselves. Terrorism has pretty much nothing to do with videos. Radicalization has more to do with the social realities faced on the ground, day after day, as we live our lives. Alienation and the misery of slums, and hopelessness of monotony without progress, are what lead to terror.

The true effects of propaganda are over-stated and over-imagined.

<comment><USER24> Yes, but that's a less useful story for the people and organizations built on gathering ever more power, so the alternative will be repeated until it sticks. Given the number of people in this thread who take the official narrative at face value, it looks like it's working.

</comment></comment><comment><USER25> Democracy has it's flaws, but I don't know any other system that works better for society today. One of democracy flaw is balance between freedom and safety, you can't have both. You can easily resolve this (terrorism) problem in totalitarian regimes (look at Kadyrow and Chechen Republic) but whole society will pay price for that. On the other hand it's easy to criticize any censorship like moves but radical islam ideology spreads like a cancer through Twitter, youtube and other social networks. You can fight terrorists, kill them but you can't kill the ideology. If you can't kill ideology you need to control it's spreading. What tools do we have to do that aside from moderating content?

Lately I was watching some live stream on youtube about UK terrorist attack to get information about what's happening. I've seen what couple of thousand people wrote on the chat below the video. I didn't seen so much hate, intolerance, racism, fascism from so many people at same time in my life. I've closed it because I couldn't read that, the problem is that probably many young people didn't...

This is very hard problem without any satisfying solution. On one hand we don't want to give up our freedom of speech, we are fearful that some evil actor will use it to manipulate our society, on the other hand, ask parent of a dead kid that died in last UK attacks or any other attacks, ask yourself, what you would give up to get your dead child back.

<comment><USER26> Frankly speaking, my observations on the actions and expectations by various "democratic governments" around the world lead me to conclude that the world as we know it is fast moving to "totalitarian democratic regimes", where instead of one totalitarian leader/pack controlling and oppressing people for generations (like in the case of dictatorships), we have the illusion that we have the freedom to vote out and vote in different parties/leaders, who eventually toe the lines of the previous elected governments and end up making things even worse overall. Maybe I was naive and ignorant and all this was happening all along over decades and centuries.

The increasing tilt in the balance of power toward the state, on the balance of power equation between the state and the individual, is quite alarming. It seems to me that world over, we're allowing monopolistic views and policies to grow, to the detriment of common people. Terror and terrorism are very useful for those in power to gain even more power. Terrorists who claim to fight for freedom are winning by showing practically that those in power are continually curbing our freedoms. With such policies, tech companies are just joining forces with "democratic oppressive governments" in making them more powerful.

I do not see a clear and easy solution for these problems.

</comment></comment><comment><USER27> Let's see, this is the same Google/ YouTube that marks dozens of PragerU videos as "restricted", despite three Wall Street Journal editorials protesting that. The same YouTube that has been demonetizing Paul Joseph Watson, Alex Jones, and other conservatives. In summary, Google has long proven that they have an unapologetic Leftist agenda.

<comment><USER28> Remember that in the last election, virtually zero newspaper, magazine (eg The Atlantic) or any other media outlet endorsed Trump in their editorials.

And yet these same organizations assure us they will have clearly defined and wholly impartial standards.

Uh huh...

</comment></comment><comment><USER29> Radicalization is a very real issue as we're getting ever more entrenched in our curated echo-chambers - I'm glad they're taking a stand.

Unfortunately I fear the damage has already been done, and these policy changes will almost certainly push newly-popularized radicals onto platforms Google has no meaningful influence over.

</comment><comment><USER30> I should start an over/under pool on how soon we'll start seeing the "OMG they banned my channel for THIS, and there's no person within 1000 miles I can find to fix it" type of posts.

</comment><comment><USER31> So instead of the rule of law we have reached the rule of google.

Funny development.

</comment><comment><USER32> Interesting that they didn't address specific steps they'd take to protect their content reviewing teams... Sounds like it would be appreciated, especially in the context of the recent issue at Facebook.

<comment><USER33> _especially in the context of the recent issue at Facebook._

I'm possibly out of the loop, what happened at facebook? Are you talking about the fake news fiasco?

<comment><USER32> In short, Facebook accidentally revealed the identities of the moderators responsible for taking down stories related to terrorism, to potential terrorists. The HN story is here. [0]

[0]: [https://news.ycombinator.com/item?id=14572585](https://news.ycombinator.com/item?id=14572585)

<comment><USER33> Wow.. the real-name policy at work.

<comment><USER34> So to speak, yes.

</comment></comment></comment></comment></comment><comment><USER35> That's an excellent news. I was absolutely disgusted by the amount of pro Daesh content on Youtube 2/3 years ago where you could see their videos that became famous for their high production value. Youtube absolutely contributed the spread of fundamentalism no question. And there are still a bunch of hateful Nasheed easily accessible but Youtube is still not removing them because "it's music" ? outrageous ...

</comment><comment><USER36> I'm rather skeptical when terrorism (or Lovejoy) is invoked as a reason for more censorship, hope it turns out well this time.

 _we are working with Jigsaw to implement the "Redirect Method" more broadly across Europe. This promising approach harnesses the power of targeted online advertising to reach potential Isis recruits,_

It is not really clear why is Europe specifically targeted.

<comment><USER37> > It is not really clear why is Europe specifically targeted

If you mean "targeted by attacks" I'm not sure why.

If you mean "targeted with this method" is because most of these kinds of acts in the West happened in Europe and not somewhere else. Also, IIRC, most of the attacks came from people born here, not immigrants, so it wouldn't make much sense to target wannabe attackers directly in the Middle East.

</comment></comment><comment><USER38> From YouTube Hero's Program rules :

[https://support.google.com/youtube/answer/7159025](https://support.google.com/youtube/answer/7159025)

>>> Participation in the Program by some participants may be restricted (e.g., no or limited perks), including persons who are government officials, including (i) government employees; (ii) candidates for public office; and (iii) employees of government-owned or government-controlled companies, public international organizations, and political parties.

that tells much...

<comment><USER39> What does it tell?

<comment><USER38> It tells that Google is completely opaque on who gets hired and who doesn't (because the sentence is fuzzy). Google is a private company so it seems logical that they have the last word. However, given the size of Google, I'd be happy if the ethical argument about what is an acceptable video would come at the table too.

The fact that Google tries to do something about terrorism is absolutely not equal to the fact that it is efficient at that. For us to know that, we must be able to review their results at that (and how do we even define a result in this area :-/ ) as well as their methods (which requires transparency).

</comment></comment></comment><comment><USER19> This is has become another one of those mysteriously-disappeared-from-the- front-page-of-hacker-news posts.

</comment><comment><USER40> I don't believe that there is a problem with terrorism online or that such a thing as online terror exists. Only the most feeble can be terrorised through the screen.

The problem is that there are terrorists among us who are willing to bomb, stab, or run people down with a truck. That there are entire communities where this either receives support or is at least tolerated.

It has very little to do with technology. They don't do it because they saw an ISIS video on YouTube or a post on Facebook. I doubt this is even a minor factor. Hodgkinson watched The Rachel Maddow Show. On cable.

<comment><USER41> I wouldn't say that it's about individuals being terrorized through the screen. I'd say that it's at-risk individuals who are persuaded to radicalize through the internet. By at-risk, I refer to individuals who may be suffering personal issues in their life, and are susceptible to the messaging used by terrorist recruiters.

<comment><USER7> If there is one thing at-risk people respond positively to, it's ham-fisted social intervention that DGAF about them and openly seeks to control their behavior.

</comment></comment></comment><comment><USER42> Will every internet company have to reinvent their own counter-terrorism tools?

That doesn't sound very efficient to me.

Also, isn't this a task for governments?

<comment><USER43> I'm more worried about government getting involved than some loss of efficiency.

</comment></comment><comment><USER44> Hello Big Brother.

<comment><USER45> Do Hacker News moderators count as Big Brother? An website can decide how it wants to moderate itself.

<comment><USER46> Google said they will cooperate with governments and 3rd party organisations, which means they will have a voice on what government 'thinks' is good or bad and they will use that power to impact our political-social-legal regulations and standards in a way google sees it. At some point AI created and provided by google will be creating law and regulations in some cities or states...

</comment></comment></comment><comment><USER47> more and more regulation = less freedom

Simple equation, no way out of it

</comment><comment><USER48> Лайки Крутятся, мемасы мутятся.

</comment><comment><USER49> They say they are fighting "violent extremism online" but how can you be violent online? That's a physical thing. Instead, they outline four points for greater censorship of videos.

<comment><USER50> You cannot be violent online but you can spread lie and propaganda and some people could believe it. Governments have a long history of doing it.

</comment></comment><comment><USER51> Personally I'm opposed to this, assuming google's goal with these steps is to protect open societies. Not because I think you should be able to post content without restrictions, but because I think it serves society better to get desensitised to this type of violent content.

In the context of 'protecting' society:

IMO it's better to protect your society by removing or blocking blatant propaganda, but I don't think that ban should be extended to violent/gruesome content. The less you see of it, the harder it will hit once it gets past the blockade. I consider it a healthy human response to be able to see such images and think "Fuck those <INSERT_GROUP_OR_STATE_HERE> assholes", and then move on with your day. Shielding people will have the opposite effect, since people will not be familiar with the possibility of seeing such things and it will therefore retain it's shock-value/utility for the <INSERT_GROUP_HERE>.

No matter how much (western?)society might fool us, humans can deal with this. We've lived as savages, constantly tested by nature and the cruelty of life, for most of our existence on earth. I'm not saying that we should return to such a state, but we also shouldn't needlessly cuddle responsible adult people either.

<comment><USER52> It's not the blood and gore that we can't handle. It's the psychological manipulation. They're using the same psychological tricks that Trump used, that Colgate uses. That every advertiser uses. But now it's war. This is the new war: the war for the mind of man itself.

<comment><USER16> Are you equating a fair American Presidential campaign to people who cut people's heads off and record it?

<comment><USER52> No. They are as different as can be. But there are many kinds of different. Some are better than others. And neither of those are good.</comment></comment></comment></comment>


Slack is Down - <AUTHOR> https://status.slack.com? <comment><USER2> "Smooth sailing"

Edit: What's the common practice here for status pages? Shouldn't they be using external health checks to auto-update the page?

Edit2: Seems to be working again for me

<comment><USER3> > What's the common practice here for status pages? Shouldn't they be using > external health checks to auto-update the page?

For Ops? Absolutely. Externally? No, not really. You don't necessarily want every conceivable blip or issue to be public knowledge, and you also want someone to provide some humanity to your status communications. There's a difference between "service is impacted" and "service is impacted significantly enough that all users/customers should know about it."

<comment><USER2> Makes sense, but if they're advertising 99.92% uptime, what does that mean if you can't trust that the status page is updated for real outages?

</comment></comment></comment><comment><USER4> Status page is updated. Here's the outage status link: [https://status.slack.com/2018-09/60787986d73e3949](https://status.slack.com/2018-09/60787986d73e3949)

</comment><comment><USER5> I never really think about it, but for someone that works remote you feel very lonely when slack is down...

</comment><comment><USER6> Slack is down, we are also experiencing AWS issues (latency, hard to diagnose right now). Anyone else?

<comment><USER7> I can't connect to Slack either. It says

Server Error

Sorry! Something went wrong, but we're looking into it.

If the problem continues, please check our Status page for updates.

But the status page says smooth sailing. Can't even contact them via [https://my.slack.com/help/requests/new](https://my.slack.com/help/requests/new)

<comment><USER7> Seems that they're up now.

</comment></comment></comment><comment><USER8> Seems to be back up. It was funny, my first instinct was to go on slack and tell everyone about it.

</comment><comment><USER9> And it's back!</comment>


Why we are still light years away from full artificial intelligence - kiril-me https://techcrunch.com/2016/12/14/why-we-are-still-light-years-away-from-full-artificial-intelligence <comment><AUTHOR> Light years is a measure of distance</comment>


The Relative Cost of Bandwidth Around the World - <AUTHOR> http://blog.cloudflare.com/the-relative-cost-of-bandwidth-around-the-world <comment><USER2> For those who don't know the story about the Australian internet industry, Telstra plays a pivotal and depressingly stupid role.

Telstra was a government monopoly in the past and owned + owns almost all the copper network in Australia. From 1997, Telstra was progressively privatized without separating the telecommunications business (phones) with the telecommunications infrastructure (cables). This was an idiotic idea.

There were some regulations that forced the infrastructure to be offered wholesale to competitors but they're still the gateway that the majority of the population get onto the internet. It's also problematic as Telstra are doing worse and worse at maintaining the copper network that is relied upon. If you are an ISP using their network, you need to go through their maintenance fees to improve the line quality for any of your customers. The cost of wholesaling through the Telstra network is also fairly high. Regardless, we've got reasonable competition at the ADSL/ADSL2+ level thanks to this.

Forward to the National Broadband Network (NBN), the government plan to connect the majority of Australian homes to fiber to the node (FTTP). Telstra submit the equivalent of a joke tender for the project -- 12 pages connecting only ~90% of the population when the tender requests 98% -- and then suffer their biggest one day stock fall in history when it's rejected.

Construction of the NBN rolls on without them, promising to replace the copper network with a fiber network that will be the new telecommunications infrastructure used by the majority of ISPs.

However... a new government came to power in Australia. One of their election falsehoods was that FTTP was too expensive and too slow and instead a ridiculous hodge podge (sorry: "hybrid") network of failing technologies would be preferred. Instead of building a complete FTTP network for $73 billion, they're instead going to spend $41 billion on a hybrid network that will need to be replaced only five years after completion according to the chairman of the NBN[1].

This hybrid network relies on the old copper network owned by Telstra. Telstra have previously stated that it is ready to die and will have billions of dollars of maintenance issues.

This is also only a very brief discussion of the ridiculousness.

tldr; Telstra played an important role in screwing up the previous generation of Australia's internet. Telstra is continuing to play an important role in screwing up the current generation.

[1]: [http://www.zdnet.com/its-time-for-turnbull-to-swallow-his- nb...](http://www.zdnet.com/its-time-for-turnbull-to-swallow-his-nbn- pride-7000024263/)

<comment><USER3> $73 billion was the LNP's figure, by the way, which involved some extremely questionable assumptions (such as despite the actual FTTP cost per premises dropping over time as the rollout was progressing, exactly like it did in other rollouts around the world, the LNP's strategic review claims it would have doubled in the future for reasons which were redacted). I think even Fletcher admitted eventually that it would have been much more like the other sceneario, $56 billion, in reality. Still a bit more than Labor's $44.1bn estimates, but nowhere near the LNP's past ludicrous claims (they were saying up to $96bn).

Still would have been worth it at double the price to get rid of Telstra. I can't believe how much the politicians have ruined it.

<comment><USER4> Today in the paper[1] there was a feasibility study by an 'independent' analysis group that said that most homes would only need to use ~14MBit connections - two HDTV streams plus a couple of vidchats. As a result, the crappy alternate NBN was more than suitable.

How on earth can you take these people seriously when they can't recognise that data consumption trends have been rising continuously throughout the life of the internet. I mean HDTV is only 1080p, and we already have 4k monitors available to consumers today - which are more than 4 times the data. One single 4k video feed is already about double everything else they'd budgeted for in their "somewhat extreme" scenario. Zero forward thinking.

[1]"Such a thing as too much speed"... [http://www.theage.com.au/federal- politics/political-opinion/...](http://www.theage.com.au/federal- politics/political-opinion/nbn-cost-benefit-analysis-finds-theres-such-a- thing-as-too-much-speed-20140826-108jtw.html)

</comment></comment></comment><comment><USER5> I think there are a few huge gaps. Russia, they got as many people as Japan. And another even larger hole is Africa, there's well over one billion people in Africa. Compared to ridiculous 22 million in Australia, that's a lot more. And in India, there's even more people than there is in Africa. Roughly US, Europe and Australia combined got same population as India or Africa which is in same ball park with China.

<comment><USER6> In Russia, 2 Mbit / 512 Kbit ADSL costs me about $8 per month; they keep suggesting I make a free switch to a 15 (or 30 or 60, have not checked the actual speeds yet) Mbit cable connection, but I am unsure if I do want to pay extra $3 :) Given the (free) D-Link 2300 running a 2500 firmware is really showing its age sometimes now (after 6 or 7 years of non-stop work), it may be indeed time to switch (not that I need anything more than 5 Mbit downlink, I think though).

This is a state provider, there are many private ones around, and their prices (usually for a 50-100 Mbit downlink via usual Ethernet cable) are $10-20 per month (comparable, that is), if I remember correctly.

The ADSL quality has always been good, btw (esp. given I wired the modem to telephone cables probably 30-40 years old :) ) -- decent pings, stable speed, was down only once or twice (in all the years).

<comment><USER7> I'm curious why you're satisfied with 2 Mbit / 512 Kbit. Do you never use streaming video / music? Don't you get frustrated waiting for large downloads?

<comment><USER6> 2 Mbits means 900 Mbytes per hour -- enough for youtube streams in 480p (and more often than not, in 720p, without pauses to buffer), and ok if software distributions are downloaded (XUbuntu 14.04 was 900 Mb, iirc); music takes a fraction of the downstream capability...

Shortly, people used to 30 MBit downlinks might have forgotten how little most of our (modern-day) media requires -- often a megabit, or two per second :)

<comment><USER8> I am on a 4Mbps ADSL line, I would have liked a little more like 10Mbps. But Stability and Ping time matters A LOT more for an internet connection.

The only real downside is uploading speed. Which meant i dont to enjoy Google Drive or DropBox as much.

Hopefully when G.Fast arrive next year, those who are still relying on copper will get a very decent upgrade.

</comment></comment></comment></comment></comment><comment><USER9> Just as a note, Cloudflare is not alone in struggling with this. I'm very surprised that the status quo has been maintained for so long. I suggest that Cloudflare should look under the process of transit becoming 'Declared', within the Australian Trade Practices Act. More here: [http://en.wikipedia.org/wiki/Competition_and_Consumer_Act_20...](http://en.wikipedia.org/wiki/Competition_and_Consumer_Act_2010#Part_IIIA:_Access_to_Services)

</comment><comment><USER10> Australian here: can confirm Telstra as one of the most 'evil' companies I've dealt with and it's very clear to those of us in the Australian IT sector that they're holding us back.

</comment><comment><USER11> Holy crap Australia is expensive. Thanks Telstra, this is why Blizzard and other companies won't come here.

<comment><USER12> blizzard is actually peered in sydney ;)

[http://monitor.nsw.ix.asn.au/cacti/graph.php?local_graph_id=...](http://monitor.nsw.ix.asn.au/cacti/graph.php?local_graph_id=455&rra_id=all)

<comment><USER13> Interesting set of graphs available through that site. Is the same type of information available for US peering?

<comment><USER14> No, the US exchanges don't provide any public data at all, this is also why they can't be listed in the Wikipedia Table for "Internet Exchange Nodes by throughput".

If you can find some data though, I'd encourage you to add the data to the table :)

</comment></comment></comment></comment><comment><USER15> At my place of work our 50:50 Unlimited (not Telstra) is in the low to mid four figures a month.

For a company with offices worldwide, we have by far the most expensive and have the worst performing connection to our regional datacenter.

The standard response from any supplier is to blame Telstra.

Recently I was porting some ISDN phone lines from an old provider to a new SIP provider. Not only did the two need to arrange and agree the porting with each other, but Telstra needed to work on both of their behalf to facilitate the port. One number took 3 months.

</comment><comment><USER16> Is this the thread where everyone is sharing how much they are paying?

All of my companies are bootstrapped (and bandwidth is the largest expense item), so some of the rates quoted got me excited.

It sounds like there are a lot of knowledgeable folks here (on the buy side), would you mind sharing some info?

I typically buy between 2..5 Gbps per POP in the US and Europe.

What's the best rate I should be able to get, you think? Tend to buy L3 or premium blend (no Cogent or HE). Any insights would be appreciated.

</comment><comment><USER17> Note that CloudFlare never states that they use Telstra for transit, nor what the blended cost would be for transit with the other transit providers, so this is actually, somewhat misrepresenting Australia.

</comment><comment><USER18> No mention of New Zealand. New Zealand is an amazing country, but goddamn, they have some pricey internet access. Wasn't Kim Dotcom planning to finance a new pipe to New Zealand at one point?

<comment><USER19> My experience is that Internet connections are price competitive with the US (and perhaps cheaper).

You can get 100/50 with 80GB of traffic for NZ$110/month, which is US$91.64. You can go full unlimited at USD$115. Amazingly, this is the same price as Comcast Extreme 105 which is only 105/20.

One thing to understand about the NZ market. Since carriers charge per byte, I was generally able to get full speed out of my links during prime time - barring TCP RTT throttling.

</comment></comment><comment> guard-of-terra "Running undersea cabling is more expensive than running fiber optic cable across land"

I always imagined that the opposite would be true.

<comment><USER20> While sharks biting internet cables may be funnier, more often ship anchors and natural disasters cause cables to break.

<comment><USER21> On land there is a seemingly unlimited supply of idiots with backhoes, though.

<comment><USER22> Or just one Georgian grandma with a shovel.

[http://www.theguardian.com/world/2011/apr/06/georgian- woman-...](http://www.theguardian.com/world/2011/apr/06/georgian-woman-cuts- web-access)

</comment></comment></comment></comment><comment><USER23> I'm confused. AAPT, Optus, PipeNetworks, etc all offer transit at significantly lower prices than this for anything I've been quoted with (circa $20-30/Mbps for 100Mbps+).

Is this just because they're offering blended traffic?

<comment><USER24> 50% of the traffic they send in Australia terminates in Telstra-controlled networks and Telstra don't offer peering.

<comment><USER23> Was talking about transit offered by Australian providers.

</comment></comment></comment><comment><USER25> I'm not sure why Africa and the Middle East specifically aren't really mentioned in that article since the prices would be on par with Australia if not more, also due to having only 1 or 2 ISPs and no competition.

<comment> squeaky-clean Because CloudFlare doesn't have any data centers there.

[https://www.cloudflare.com/network-map](https://www.cloudflare.com/network- map)

</comment></comment><comment><USER26> Telstra seems to disagree: [https://twitter.com/Telstra_news/status/504435708098908160](https://twitter.com/Telstra_news/status/504435708098908160)

<comment><USER27> They're correct that at scale the dollar pricing listed is at least 10x too high in Australia. What they fail to acknowledge is that it's also 10x too high in every other region described. So the conclusion remains the same, Australia is 20x more expensive than Europe for no reason other than Telstra's market power.

</comment></comment><comment><USER28> Would it be feasible to have a global federation that played referee to negotiations between citizens and ISP's on pricing for Internet access?

</comment><comment><USER29> Good article, but frustrating they picked some arbitrary $10/mbps as their baseline instead of a more accurate number.

<comment><USER30> Do you know what a more accurate number would be? I'm curious...

<comment><USER31> We pay $1.50 per Mbit with a commit of 200Mbps in NY, LA and Miami. This is for 'premium' blended transit (e.g. Specifically excludes Cogent and HE)

</comment></comment></comment><comment><USER32> Why is peering so much lower in the US? Is that a business strategy on the part of ISPs?

<comment><USER33> From the article: "In North America, while there are Internet exchanges, they are typically run by for-profit companies." ... "In North America the combination of relatively cheap transit, and relatively expensive exchanges lowers the value of joining an exchange. With less networks joining exchanges, there are fewer opportunities for networks to easily peer."

<comment><USER34> Let me get this straight: North America has several for-profit companies that operate continental-wide networks, because building them is cheap, but they peer with each other less often than in Europe. The situation isn't ideal, but it makes the internet "backbone" market competitive, so it works out.

Side note: the situation is very different when talking about last mile ISPs in North America.

<comment><USER35> I always find it interesting that the business network in the US is often ignored. It's quite stellar on a combination of quality and price. I think it's an inconvenient data point for people that only like to bash the US telecommunications situation.

<comment><USER36> People tend to complain about problems and ignore things that are fine.

While a desire to avoid pointing out any positive of a situation that drives them crazy frequently may be part of the reason it isn't mentioned I think it pales in comparison to just forgetting about whatever isn't a problem.

</comment></comment></comment></comment></comment><comment><USER37> Wow, North America came out on top of an Internet comparison for once?

<comment><USER38> No, Europe had more free peering so had a lower effective cost.

<comment><USER39> Just spent a few weeks there and I can tell you home broadband is very fast compared to back home in the US - and apparently my relatives in the EU pay 1/4 as much as me when you add in TV and landline.

</comment></comment></comment><comment><USER40> The TL;DR of the article.

>If Australians wonder why Internet and many other services are more expensive in their country than anywhere else in the world they need only look to Telstra (Note: ~50% of all internet is from Telstra in Australia).

>What's interesting is that Telstra maintains their high pricing even if only delivering traffic inside the country.

>Given that Australia is one large land mass with relatively concentrated population centers, it's difficult to justify the pricing based on anything other than Telstra's market power.

>In regions like North America where there is increasing consolidation of networks, Australia's experience with Telstra provides a cautionary tale.

<comment><USER41> This seems to be a common failure mode in Australia. Grocery prices follow the same pattern—food is extraordinarily expensive, which you assume might have to do with the country's remoteness until you notice that local foods are just as overpriced. Mangoes grow by the side of the road in Queensland, but sell for $3.50 at the supermarket.

The root problem is duopoly (in the case of grocery stores) or monopoly (Telstra).

<comment><USER42> Is it expensive relative to your income?

<comment><USER43> Australia is shockingly expensive the first time you visit there. Normal economics just don't seem to be at play.

It took me a long time to put my thumb on what Australia is like and it wasn't until I watched the show "Fringe" and saw the alternate universe (with the Airships and the Twin Towers) that it finally struck me, it's like an alternate universe America. Everything is _just_ like the U.S., except for some weird little details. Big cars, lots of open land, cowboy analogs (drovers), purpose built capital designed by a foreigner, densely packed East Coast, East Coast vs. West Coast rivalry. And then suddenly it's not: driving on the left, everything is overpriced, egg yolks are a different color, etc.

I remember seeing real estate listing for some _really_ remote housing in some dying gold town somewhere. Like 2 bedroom bungalows with no property. And they're priced at what you'd expect to find in a nearby suburb of Sydney (except the housing in the suburbs around Sydney is ludicrously expensive). While anywhere else in the world people would basically be paying you to take the title so they could get the hell out of there.

The local markets in Australia are just _weird_.

 _note_ I live in one of the most expensive housing markets in the U.S. so I'm used to seeing high prices. But even single bedroom apartments in Sydney suburbs rent out at prices you can get entire houses for in similar geographic regions around my area. You'd think the exact opposite would be true. In almost any other part of the world with plentiful land and low population, you end up with cheap housing, while in Australia that doesn't seem to be true at all.

<comment><USER44> Australia is generally ridiculously expensive, that is true, but there is usually a reason (not necessarily a good one) why "local markets in Australia are just weird."

> I remember seeing real estate listing for some really remote housing in some > dying gold town somewhere.

Without further details, I can't say whether your assessment of "dying" is correct or not, but it is true that towns that have mining operations experience explosive growth (and contractions) in housing prices. Local governments in general don't want to allow a lot more house building, because in 5, 10 or 20 years time when the mine closes down, having all those empty houses causes a ghost town effect where the town really will die (the towns almost always existed before the mines, and they want them to exist after the mine leaves too). That means people pay north of $1 million dollars for a fibro shack. Mining companies will hire the local showgrounds long term, so that they can build a tent city for their workers (who use the showground toilet and shower facilities etc). Local house prices will fluctuate by hundreds of thousands of dollars in days, based upon financial news from the mining companies about whether they are liking to build/expand/close the local mining operation.

Yes, it's weird.

As for Sydney specifically, it is actually very space constrained. With the ocean on one side, and the Blue Mountains on the other side, there actually isn't a lot of room for expansion.

You are correct that in general there is a lot of undeveloped coast line though (inland is basically inhospitable). It's just not where the jobs are.

<comment><USER4> _As for Sydney specifically, it is actually very space constrained. With the ocean on one side, and the Blue Mountains on the other side, there actually isn 't a lot of room for expansion._

Melbourne has vast swathes of open land in all directions, and housing prices are nearly the same.

I asked the question to a friend, and he replied that I just don't get it: the people who already have houses aren't interested in reducing the value of them. Almost all federal politicians have multiple properties. Why would they legislate to do that? So we have negative gearing that will never get solved, lots of foreign purchasing driving up prices, and no-one willing to pull the pin out of the electoral grenade and start dealing with our housing bubble.

<comment><USER45> While Melbourne doesn't have any shortage of land to build out into it is getting very far away from the inner city.

As long as you are happy to drive everywhere and can find a suburban job or are prepared to make long commutes there is plenty of relatively cheap housing to be had.

</comment></comment></comment></comment></comment></comment></comment><comment><USER46> _In regions like North America where there is increasing consolidation of networks, Australia 's experience with Telstra provides a cautionary tale._

But, but, the market is _magic_! Regulation is _evil_. Does not compute!?!!

<comment><USER24> Australia didn't start with a free market in telecommunications and the aftermath of a botched privatisation means that we're left with a distinctly dysfunctional market as a consequence.

<comment><USER46> The USA might have had a free market at one time, but we haven't since 1934, so memories are hazy. The uptick in competition post-1996, which has steadily rolled back for some time now, was mostly a gimmick to funnel investment into the Daughters Bell. TFA is spot-on that concentration is bad for consumers, and our slow roll-back from even the paltry market diversity we once had for data connection bodes ill for us.

I heartily apologize to all who were disconcerted by my caricature of the useful idiots who "believe in a free market in telecommunications", as the most recent email I received from my corrupt Senator put it.</comment></comment></comment>


Tesla driver found asleep at wheel of self-driving car doing 150km/h - <AUTHOR> https://www.theguardian.com/world/2020/sep/17/canada-tesla-driver-alberta-highway-speeding <comment><USER2> Tesla is almost certainly going to get sued at this rate. Calling a feature that needs constant driver supervision "autopilot" should already be treated as advertising fraud, but soon enough someone will die because a Tesla owner didn't read the fine print on what it means, and at that point Tesla has to be just as liable.

</comment><comment><USER3> Previous submissions:

\- [https://news.ycombinator.com/item?id=24508051](https://news.ycombinator.com/item?id=24508051)

\- [https://news.ycombinator.com/item?id=24506649](https://news.ycombinator.com/item?id=24506649)</comment>


Ask HN: (2018) What are some of your favorite Unix command line aliases? - <AUTHOR> By which I mean either aliases or short bash functions.
Examples:
# download a video from YouTube or other major sites, recoding if needed into .mp4 format

  alias yt='youtube-dl --recode-video mp4'
 # empty the pasteboard (macOS only) [edit: fixed description]

  alias pbclear='echo "" | pbcopy'
 # remove formatting from text in pasteboard (macOS only)

  alias pbclean='pbpaste | pbcopy'
 # remove a directory even if it contains .DS_Store files (macOS specific)

  function rmdir() {
      [[ -e "$@/.DS_Store" ]] && /bin/rm "$@/.DS_Store"
      /bin/rmdir "$@"
  }
 # go to the "Feeling Lucky" result for a Google search

  lucky () { open "http://www.google.com/search?hl=en&q=`echo $* | sed "s/ /%20/g"`&btnI"; }
 <comment><USER2> While the git-completion.bash [0] function isn't short, this small addition to my bashrc has made my life much easier:

      if [ -f ~/.git-completion.bash ]; then
            . ~/.git-completion.bash
      fi

[0] [https://github.com/git/git/blob/master/contrib/completion/gi...](https://github.com/git/git/blob/master/contrib/completion/git- completion.bash)

</comment><comment><USER3> some small ones I use.

alias x="xdg-open"

alias update="sudo apt-get update; sudo apt-get upgrade"</comment>


HNNotify Source Available - <AUTHOR> https://github.com/thebuzzmedia/hnnotify <comment><AUTHOR> __UPDATE URL[https://github.com/rkalla/hnnotify](https://github.com/rkalla/hnnotify) __

My apologies for taking so long to make this available after retiring the site. If anyone is interesting, here it is.

It's a Play 1.x app with Mongo 2.6.x sitting behind it.

The interesting bits are the Jobs that run discovering, indexing, reconciling and mailing notifications: [https://github.com/thebuzzmedia/hnnotify/tree/master/app/job...](https://github.com/thebuzzmedia/hnnotify/tree/master/app/jobs)

and the JSoup-based Parser (thank god for JSoup): [https://github.com/thebuzzmedia/hnnotify/blob/master/app/par...](https://github.com/thebuzzmedia/hnnotify/blob/master/app/parser/HNParser.java)

If anyone would like to take this and relaunch the server, please feel free - it's completely self-sustaining... if it wasn't for 2 botched OS upgrades in the ~6 years it was running I don't think I ever needed to do anything with the server. It just ran.

Logs are incredibly verbose to make troubleshooting easier... watch the disk space - if anything does go wrong you can simply kill it and relaunch and it will pickup where it left off.

The only problem I had was running out of disk space and getting blocked by Gmail by too many daily outbound mails... other than that, set it and forget it.</comment>


Ask HN: Do you incorporate before or after validating the project? - <AUTHOR> Hi great community. I have this question for a long time...
Do you incorporate before you validate or after? In fact the question would be... how can you start getting clients to pay you without incorporating? Do they pay on your name? And you inc. after that? How do you get this working and pay your share if taxes? I am from EU, if that makes any difference<p>Thank you very much <comment><USER2> After, unless you need to do something to validate for which you want the liability protections that being a corporation provides. Accepting money for a product that may or may not make it to market is one of those things.

I would have prospective customers sign LOIs before incorporating, but then incorporate before any money actually changes hands, so that all business is done in the corporation's name.

</comment><comment><USER3> I am from the US, and my core rule has always been that the moment someone is going to pay me for it I need to have the business setup. For me, this is a liability concern as the business will insulate my personal assets and protect me with some relative degree of confidence. Depending on the product and your risk tolerance of course this can vary. And I know some people disagree with me, but this is my personal rule.</comment>


ARM touts PlayStation 3 power in a mobile GPU - <AUTHOR> http://arstechnica.com/gadgets/news/2011/11/arm-touts-playstation-3-power-in-a-mobile-gpu.ars

<comment><USER2> To put this in perspective, the original iPhone had Sega Dreamcast level graphics processing.

<comment><USER3> Craaaaaaaazy taxi!

</comment></comment><comment><USER4> I miss the raw numbers of 'polygons per second.' Interesting that ps3 does 250-300, I've never read that anywhere, I guess it matters less and less over time, but it is still interesting.

</comment><comment><USER5> Five years from now companies will be bragging that their phones can go as long as several minutes between charges,

<comment><USER6> If Koomey's Law holds then 5 years from now the batteries should last even longer because of efficiency gains.

[http://www.economicsofinformation.com/2011/09/is-koomeys- law...](http://www.economicsofinformation.com/2011/09/is-koomeys-law- eclipsing-moores-law.html)

</comment></comment><comment><USER7> And? The PS3 design is what, seven years old at this point? I would be shocked if some clever dick couldn't jam that many vector units onto a middling processor and call it good, at this point.

The reason that, say, _Uncharted 3_ looks so great has very little to do with any commonality with this ARM design and much more with sophisticated tooling that is now available.

IOW: BFD.

<comment><USER8> Yeah, you are right. It's totally not a big deal that tablets will soon be able to compete head on with consoles. That totally doesn't change anything at all about the future of consoles.

There are new tablets every year. At the very least this might force Sony and Microsoft to move faster.

<comment><USER9> While I totally agree that the progress of mobile GPUs is amazing and Sony/Microsoft should be scared... this comparison ("we have a GPU that's almost as good in one particular area as PS3") is very much flawed.

The GPU in PS3 is probably it's weakest part. Most of games that push the hardware end up using a ton of SPUs to do work that a GPU would normally do. Image postprocessing, deferred lighting, skinning, vertex transformations etc. It will take a while until the mobile chips can host the power of PS3's SPUs!

Another big point is memory bandwidth. Even if ARM's new GPU could achieve these vertex transformation rates in theory, I'd bet in practice it wouldn't because of much, _much_ more limited memory bandwidth on a mobile system compared to a console. And bandwidth will not rise fast there, since memory subsystem is one of larger energy consumers.</comment></comment></comment>


Large Text Compression Benchmark - <AUTHOR> http://mattmahoney.net/dc/text.html <comment><USER2> That's a good classic, but don't forget to also read his excellent writings of the connections between data compression and artificial intelligence:

[http://mattmahoney.net/dc/rationale.html](http://mattmahoney.net/dc/rationale.html)

Also, his book "Data Compression Explained" was a great help and eye opener for me:

[http://mattmahoney.net/dc/dce.html](http://mattmahoney.net/dc/dce.html)

</comment><comment><USER3> So, most interesting for me are the well known compression utilities:

                    Compression                      Compressed size      Decompresser  Total size   Time (ns/byte)
      Program           Options                       enwik8      enwik9     size (zip)   enwik9+prog  Comp Decomp  Mem Alg Note
      </comment><comment>-           </comment><comment>-                     </comment><comment>----  </comment><comment>-----  </comment><comment>-----  </comment><comment>-----  ----- -----  --- --- ----

      7zip 4.46a        -m0=ppmd:mem=1630m:o=10 ... 21,197,559  178,965,454          0 xd 178,965,454    503   546 1630 PPM  23
      WinRAR 3.60b3     -mc7:128t+ -sfxWinCon.sfx   22,713,569  198,454,545          0 xd 198,454,545    506   415  128 PPM
      xz 5.0.1          -9 -e                       24,831,648  211,776,220    103,692 x  211,879,912   2482    36  660 LZ77 26
      bzip2 1.0.2       -9                          29,008,736  253,977,839     30,036 x  254,007,875    379   129    8 BWT
      gzip 1.3.5        -9                          36,445,248  322,591,995     38,801 x  322,630,796    101    17  1.6 LZ77
      compress 4.3d                                 45,763,941  424,588,663     16,473 x  424,605,136    103    70  1.8 LZW

I think a very good compromise between speed and size still is bzip2, so i will stick to bzip2. Seems to be in the middle between compression time, decompression time, memory and size.

<comment><USER4> Note that there is a huge difference between compression levels in xz (LZMA), while there is almost no difference between bzip2 levels. This benchmark uses the slowest 9th level. If you try lower levels, e.g. xz -3, it may give better compression ratio and better compression speed than bzip2, and, being an LZ77 algorithm, much better decompression speed. Here's an example with different levels: [http://pokecraft.first- world.info/wiki/Quick_Benchmark:_Gzip...](http://pokecraft.first- world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO)

Also, I recommend using parallel versions (pbzip2 or pxz) if you have more than one CPU core: compression is highly parallelizable, so if you have 2 cores, they can half the compression time.

<comment><USER3> I didn't know about the parallel versions, thanks!

</comment></comment></comment><comment><USER5> The charts are horrible

1) The y-axis (time) is logarithmic, while the x-axis(size) is linear.

2) The x-axis begins at 1000, which makes it seem like there's a huge size difference between the slowest programs, which isn't true.

<comment><USER6> 1) What' is so bad about log-linear graphic? My rule of thumb is trying to use an scale were the graph looks like a line. Line are easy to understand, and to apply the inverse transformations to get the true form of the curve and an approximate formula. In a graph it's difficult to distinguish the positive part of 1/x, 1/x^2, ..., e^{-x}, ...

<comment><USER5> Because we don't perceive time logarithmically. 10 sec compression is 10 times worse than 100 sec compression.

So looking at the first chart, lpaq9i and durilca4linux points are right next to each other, while in realty durilca4linux is 530sec slower.

<comment><USER7> I don't see the problem. It's obvious what the chart conveys if you stop to think for a moment (at least speaking for myself, I'm used to looking at graphs with a logarithmic time scale, and didn't think about it). In fact, it's reasonable that a linear improvement in compressed size should require an exponential increase in time and memory.</comment></comment></comment></comment>


Ask HN: A quick tutorial on Web Application Testing? - <AUTHOR>

Can someone give me a quick reference which I can use to develop a proper mindset to test a web application?<p>I dont want tools. I want to know given an app and requirement documents how should I start. <comment><USER2> Are you testing functionality or UX?</comment>


During a Technical Interview, Ask – Why Is This Position Available? - <AUTHOR> https://csswizardry.com/2019/04/tips-for-technical-interviews/ <comment><USER2> Maybe, if you don't care whether you get an offer or not. This is an exercise in gift-horse dentistry for people who need the work.

</comment><comment><USER3> There's a few common sense recommendations, but I think I wouldn't recommend the part on white boarding:

> When the whiteboarding task starts, make a quip or light joke along the > lines of Oh, wow. I hope you provide full-time staff with computers—I don't > like the idea of typing up my handwritten code at home on an evening!. > Something like this should help to make the situation a little easier, while > also gently pressing home the sheer ridiculousness of the situation.

I wouldn't do that. It will highlight your discomfort with the task even though you haven't even started on it. The whiteboard work can go well or not so well. You don't know how it will go, so there is no upside to leaving a negative impression right off the bat. It could actually influence how the interviewer will perceive your performance at the white board.

I also don't see what benefit you can derive from hinting at the ridiculousness of the task that the interviewer is asking you to perform. I doubt the interviewer is suddenly going to say: "yeah it's ridiculous, let's skip the coding part".

> Next, before you begin, forewarn them that this is an uncomfortable > environment in which to 'write code'; tell them that you intend to annotate > any knowledge gaps as you go, thus demonstrating that you know you need to > look into something a little further, but also that you have a vague idea of > what that something else is.

They already know it's an uncomfortable environment. This "forewarning" can easily be interpreted as you making excuses before you've even started. Don't tell them you'll annotate knowledge gaps as you go: just do it _if_ you encounter a knowledge gap.

> Don't ask if you can write pseudo code: tell them that's how you're going to > approach the task. Regain some of the power.

"No sir, we want to see actual code, not pseudo-code."

I think you're actually be doing yourself a disservice by strong-arming pseudo code, as it projects discomfort with actual coding.

Pseudo-code is equivalent to describing the algorithm: for that, you can draw some diagrams and explain the algorithm verbally. It makes sense to get an ack from the interviewer before going into the actual code: "That's how I plan to tackle this. Does it make sense to you?".

Trying to "regain power" from the interviewer may work on junior interviewers, but most senior people won't let you get away with it, and it can easily backfire.

At this point you've already sent 3 signals that you are quite uncomfortable with writing code, before you've even started!

My suggested alternative: act confident. When the interviewer tells you you're about to get to the white boarding part of the interview, just say "let's do it!" with enthusiasm, take a good breath, and pretend you're at the board with a colleague trying to work out a problem. Good luck!

<comment><USER4> whiteboarding is such an interesting cultural issue. I'm only surprised how resistant Developers are to this skill. the best developers I know can think on a whiteboard in a group.

it's not an everyday task, but it's a very good measure of communication skills.

syntax may not be as important, but it is a measure of your Comfort Inn on language or system. asking any Junior developer to be syntax correct I would find inappropriate appropriate, but for a language specific expert I do not see this as a high bar</comment></comment>


Access Dropbox as a network drive on Mac and Windows - <AUTHOR> http://blog.expandrive.com/2013/05/14/introducing-expandrive-3-dropbox-rackspace-openstack-webdav-dreamobjects-and-more/

<comment><USER2> Congratulations. Now, can you explain me, in dumbest possible terms:

a) What _is_ this?

b) Why do I want to use this? What advantage does it give me over using just Dropbox?

c) Does it consume disk space?

d) If it does not, does it have a cloud component? Probably not, but I'm not sure what's going on here.

Your website doesn't really tell anything about your product—you might want to take a look on that.

<comment><USER3> It's software that lets you mount Dropbox as if it were a network drive. It's advantageous to just Dropbox because it doesn't consume disk space. It doesn't have a cloud component; it uses the Dropbox API to interface with Dropbox's servers directly.

<comment><USER4> Also advantageous because there is less latency between saving "locally" and the changes being reflected on the remote server (or client) as compared to Dropbox. Dropbox has to sync up to the cloud then back down to your remote server, whereas this is direct. That latency can be a pain if you're making a lot of small changes often and wanting to see the outcome.

</comment></comment></comment><comment><AUTHOR> It didn't quite make the release but Google Drive and SkyDrive support is coming very soon. As are alerts for changes.

The big win is that you don't need to sync in the entire dataset, and you can access multiple accounts.

<comment><USER5> You also posted '[and soon, Linux]' a while back (late April).

Any word on this? :)

<comment><AUTHOR> Still in development, mostly a packaging issue right now. But it's working! 4-8 weeks out.

<comment><USER6> > 4-8 weeks

I think you mean 6-8 weeks.

<http://sixtoeightweeks.com/>

</comment></comment></comment></comment><comment><USER7> I was an ExpanDrive user for a long time. I used it for painless SSH mounts in Windows. I don't have a need for it these days, but if I did, I wouldn't hesitate installing it again. Recommended!

</comment><comment><USER8> I am an ExpanDrive customer and I am very happy with it. I just upgraded to version 3 and it is very nice to mount drives over SFTP from where ever I am.

All in all the product is polished and works better than OSXFuse and having to open SMB ports, etc for my use.

</comment><comment><USER9> If you just need the local drive part on Windows, I have been using this for some time to make the complete Dropbox folder accessible as a local drive m: on all my computers. Of course you still need to run Dropbox to handle the syncing.

\- create a file containing this: subst m: c:\Users\<username>\My Documents\My Dropbox\

\- put this file somewhere in your Dropbox folder (that way it will get synced to all your other computers and you can use it there, too)

\- add a link to it in the Start Menu->Startup program group on all your computers

Voilà!

<comment><USER10> Thats a nice tip, the syncing to other PCs only works if all your PCs run the same version of windows and you use the same username and directory structure though.

<comment><USER9> Yeah, that's true. You can make it more robust this way:

subst m: %USERPROFILE%\My Documents\My Dropbox\

</comment></comment></comment><comment><USER11> I don't understand what this is. From looking at the webpage it seems to be an interface to a remote file store, but we've had such things for years, such as sshfs and FUSE. What am I misunderstanding?

<comment> SG- It has a nice GUI and is simple to use. It's like not understanding a nice FTP/sftp client because there are free/CLI versions available. It's not for everyone, but it my experience with using it for sftp mounting is that it was great in OSX years ago.

</comment></comment><comment><USER12> How is the sftp/sshfs support on Windows for expand drive? I've used opensource options in the past for Windows and they were all pretty buggy. One that isn't so much would be one worth buying imho.

<comment><AUTHOR> The original version of the product, back pre-2009 was called SftpDrive and was Windows only. In many ways it has long been our core product. But as time goes on more people are using various cloud services and fewer people are using SFTP.

<comment><USER12> Tried out the trial a bit ago and it seems to work fine with a dropbear ssh client for Android so I can mount my Android device. Some clients I've had issues with working with dropbear or not letting me connect with root access (and the root directory, but no issues like that here). Happy with it so far.

<comment><USER12> Update, had some issues with it refreshing the cache. Hitting f5 didn't seem to change file statuses after erasing files (using windows 7). Had to disconnect and reconnect the client to get them to update in sftp.

</comment></comment></comment></comment><comment><USER13> So the implication is that it doesn't consume local disk space if connected as network drive, right? That would be a very useful feature!

</comment><comment><USER14> Let me recommend Syncovery for doing the same job for me for the last couple of years

<http://www.syncovery.com/>

</comment><comment><USER15> I'd love to see the sales numbers for this sort of a product. I just bet it's taking in more than an average HN-grade startup :)

</comment><comment><USER16> Is this only a drive interface to files elsewhere, or is it a local-sync solution too, a-la-dropbox?</comment>


I hate cut-and-paste - <AUTHOR> http://jacquesmattheij.com/I+hate+cut-and-paste

<comment><USER2> For bonus points - add robust code folding to this, especially when the folding persists across edit sessions.

I worked on a large embedded system where all the principal developers used such an editor (there was some occam heritage). One incautious copy and paste of 'a handful' of lines, and you had accidentally replicated a complete subsystem! It made debugging 'interesting'.

Every time I think 'Oooh - code folding - what a great idea', I retrieve those memories.

More generally, any time editor cleverness is interposed between the creative idea and the source code, I get a twitch. It should be possible to work on the source code without requiring anything more than a bare-bones editor - that is all I can assume about what my favourite editor shares with whatever future developers (incl. myself) happen to be using.

<comment><USER3> Yes, that's why in my previous job I simply forced all my team to use Vim, with almost the same vimrc. Before me, some were using Eclipse, jedit, even dreamweaver, that was a mess, also with encodings.

But to complete Jacques' rant, I'd say that if duplication is bad in the part of the code handling logic and actions, it is not bad and often "normal", or even "better" in configuration parts or in content parts (eg templates). Factorizing a list of configs to replace some duplication with logic (loops, conds) can be a very bad idea.

</comment></comment><comment><USER4> Actually cut-and-paste is ok, since the code is moved from one (wrong) place to another (better) place.

Problem is copy-and-paste, which litters codebase with duplication and lead to messy and hard to maintain code.

<comment><USER5> The author meant copy and paste:

> Think about that ctrl-C, ctrl-V sequence next time you are about to use it.

Many people use "cut and paste" as a generic term that refers to either copy and paste or cut and paste. You have to figure out which they really mean from the context.

<comment><USER4> I know what he mean, but that's not what he said. And it is one of the primary reasons why bugs exists, you mean something, but program something else.

</comment></comment></comment><comment><USER6> Great article. Code expands to fully use all available editing tools. If you can see 1000 lines of code on your screen at once, you'll work with 1000 line chunks of code. If you have great tab-completion, you'll make your APIs big enough to make the tab completion useful. If you get a million lines of boilerplate every time you create a file, every file will be more than a million lines long.

What's great about ed is that you keep most of the program in your mind. There is no tab completion. There is your brain. And when you keep the whole program in your brain, you write less code and the code you write is better thought out. I wish we could go back to the days of ed, but all we can do now is consciously resist the desire to work at our tools' maximum capacity.

<comment><USER7> Yeah, but would you be willing to read and develop, say Cassandra, in ed? I, for one, wouldn't. One more thing: would you be able to?

</comment></comment><comment><USER8> _Then came the full screen editors. They were a bliss compared to line editors. Really. If you don't believe me try 'edlin' for a day or so. We'll see how you like that. Then you'll be wondering how come any software got written at all for the first 2 decades of computing._

I have been wondering that, actually.

On the topic of the article, luckily, the Rails community (and python, and others) seem to have embraced this idea, under the heading "DRY principle" (Don't Repeat Yourself). There might be a connection between that embrace of DRY and the fact that Ruby/Python afficionados tend to prefer "simpler" text editors (Textmate, Vim, Emacs, Sublime) rather than IDEs.

<comment><USER9> Worth noting though that this can go too far the other way. I've definitely written things where, in my refusal to repeat myself, I've made a hard-to-read mess.

Here's an example from when I was very first starting out with ruby:

<http://pastie.org/2749772>

This is a pretty hard to read piece of code, despite my rambling comments. All done to avoid copying and pasting the same easy-to-follow method ~10 times.

<comment><USER10> But the problem with this code isn't that there is too little duplication! The trouble wass that your grasp of the language wasn't that good and didn't have any idea of how to organize the code.

<comment><USER9> So, would you have solved it by a more readable form of metaprogramming? Seems to me it's either that, or copypasta.

(Could still be me being a noob, but I can't see some other architectural means of avoiding a choice between either duplication or tricky-to-read metaprogramming in that case.)

</comment></comment></comment></comment><comment><USER11> Although I'm not from the punch card era, I overall agree with the sentiment of this article. I learned to code has a hobby when I was a kid and then took a CS degree. During the CS degree I was indoctrinated out of "bad habits" that I developed as a self-taught programmer. One of these bad habits was being terse. I took my degree during the golden age of createFactoryOfServerProcessGenerators() type things. Now, with more experience and wisdom, I realize I was right all along. A development environment that comfortably fits in your brain makes programming more effective and fun. And maybe I'm too idealistic, but I strongly believe the two are related.

</comment><comment><USER12> Copy-and-paste is bad, I agree, but on the other hand, I hate it. And by 'it' I mean _all of it_. I have never seen a large system that could, for more than a moment in time, be seen as anything else than elephantiasis.

It's shameful and I'm sick of it. I want out. Right now, I'm paid quite well as a contractor to make some enhancements to a project developed by a hundred underpaid developers in a distant land, who did a remarkable job, considering. But it's rat's nest -- they didn't stand a chance. And I can't look myself in the mirror anymore and say this is what I do. It's grotesque. I'd rather run a coffee shop (and indeed I've been talking about it) because at least I could go home at the end of the day having seen a little beauty here and there, instead spending all day staring into the face of the Elephant Man.

<comment><USER13> Beware, you're getting scarily close to burn-out territory if I interpret you right.

<comment><USER12> Thanks for your concern but I'm not sure it's burnout. It's waking up. Here's my process:

1) Sure it's ugly, but look at what it can do!

2) Ok, it's ugly, but with good practices, good people, good management, you can ameliorate the worst of it.

3) Fuck it.

I've decided that this is my last contract. I have a project called Kayia, and my wife has a site called Kongoroo. I'll continue to work on those but otherwise that's it. If I'm not coding in Kayia, I'm not coding.

<comment><USER14> I completely understand your #3. We keep making excuses and rationalizing, for years and years, the fact that the overwhelming majority of software projects are fucked.

It doesn't sound like burnout to me either. I went through something similar. The solution was to admit that I had taken a wrong turn in my programming career, and commit to working only on things I believe in. It was either that or get out of the software business altogether.

<comment><USER12> Thanks for understanding. I really appreciate it.

</comment></comment></comment></comment></comment><comment><USER15> As usual, there is merit to the opposite side too. Don't create some huge abstraction with class hierarchies and virtual methods and stuff, just to avoid cutting and pasting a ten-line function somewhere.

Often it's a good idea to start by cutting and pasting, and then afterwards figure out what ended up being sharable.

<comment><USER16> Agreed. I recall a useful rule-of-thumb that states you should wait until the third time you need a bit of code before packaging it into something reusable. Otherwise, you might be wasting your time.

<comment><USER17> This sounds like a rule-of-thumb for making sure bugs get fixed in one place but not another. :)

Seriously though, this article got me to thinking and I realized that why would one live with the smell of copy-and-paste when it's just so darn easy to write a singly reusable function?

I agree with what you in the sense that there's no need to go from what might be a few lines of copied code to a full-blown library or subsystem. But for a small bit of common code not refactoring that out immediately seems to me a really bad practice.

<comment><USER18> If you factor too early you end up with a function that takes ten parameters. Sometimes cut+paste functionality evolves into truly divergent code, and you have to have a "feel" for when it will happen.

</comment></comment></comment></comment><comment><USER19> I hate people who say cut-and-paste when they mean copy-and-paste. Pet peeve of mine, I guess.

<comment><USER20> ... or yank-and-put

</comment></comment><comment><USER21> I really disagree with the premise of the article that IDEs encourage copy&paste. Full screen editors do. IDEs do the _opposite_ , by making it super-easy to reuse existing code through code completion and docblock tooltips. In a decent IDE, copy&paste is often _more_ work than finding the right method and calling it.

<comment><USER22> Agreed. The ability to select a chunk of code and click "refactor into new method" makes my code a lot neater.

</comment></comment><comment><USER23> My first programs ran on a programmable calculator. I wish I could have kept the habits I had back then. Drawing block schemata on paper before coding seemed tedious but it was in fact valuable. Damn you lord Sinclair and your ZX Spectrum for providing a full screen "editor" which let me skip the design phase. And it didn't really have cut'n'paste in the modern sense.

(Edited) Also: <http://clonedigger.sourceforge.net/> Use it for great good.

<comment><USER24> I'm not sure the original ZX Spectrum editor qualifies as "full screen" as selecting a line and visually editing that line are separate tasks. It has more in common with RegEdit than with Notepad.

</comment></comment><comment><USER25> Does anyone know if there are tools out there that look at a code base and find "similar" snippets of code?

I would expect such a tool to parse the language into AST form and find branches that are the same except some identifiers and a few other details. It is probably intractable in general, but I think it is feasible for most code bases.

</comment><comment><USER26> Yep; we've just had exactly this problem with a contractor. Some functionality needed tweaking, which was done on one page - but on testing we found the exact same functionality hadn't been tweaked on all the other pages.

Huh?

Turned out it was a copy/paste job... which in turn became only the start of the rabbit hole. :)

</comment><comment><USER27> It's gratifying to get something up and running quickly. I do a lot of "copy and paste programming" to get a functional prototype doing stuff. it's helpful when I need to show a non-technical person (e.g. my boss) in a convincing way. ctrl-c + ctrl-v is my best friend

Most of it is throw-away code: just playing with ideas and different implementations. Exploratory programming is cheap these days, and I really think it's for the better.

If it looks like I'm narrowing down on something I'm actually gonna use, I refactor, rewrite, simplify, and delete a lot of code. ctrl-x becomes my new best friend. (I write in natural languages much the same way.)

</comment><comment><USER28> And I hate guns. No wait.. actually I hate people who use guns to kill other people.

<comment><USER29> I understand the analogy, but on the other hand, the barrier to using a gun for killing people is nearly infinitely higher (I hope) than the barrier to using copy & paste to "just fix this little thing here".

So many times, when you need a quick solution for a small problem and you know that you have fixed this exact same problem a few weeks/months ago at a different spot, there's a _huge_ temptation to just go and copy&paste those lines.

By doing so, you have just created debt. What if that code is using some API that you want to change a few years later? Whoever is going through the old code now has to fix up all instances of your copy & paste action.

What if the initial code contained a bug that somebody else fixed? They might not know about the copy pasta. It's very likely that they only fix the initial instance of the code and not all other places where it was pasted to, so the bug partially remains, or, worse, is later classified as a regression (which it's not).

So, coming back to your analogy, hating copy & paste is analogous for hating guns not only for their potential of killing people but also for their potential for causing accidents and for their potential to use them for any kind of potentially non-violent crime.

As the guy who is usually doing the refactorings to our 7 year old codebase, I'm the guy who is suffering from copy pasta and I'm telling you: I totally agree with the original article. I've yet to see a single instance where copy & paste of more than a single line of code didn't cause me non-insignificant amounts of additional work.

<comment><USER30> That is spot on. But blaming text editors and IDEs is a missed shot. If the author blames the editor I'm temped to think that he himself resources to copy paste because he as his editor to blame.

</comment></comment></comment><comment><USER31> There are some greate advice however its impractical in the real world. (I am not defending copy and paste, the DRY principle should be your number one rule.) Most (good) contractors work in this way however its impossible to lay out your solution to its entirety and build a beautiful solution. Without specs being changed on you or you realizing that what you are building is not something the clint wants.

</comment><comment><USER32> But, but... surely what you are suggesting is people be intelligent... why would they do that when they can be bone idle lazy???

</comment><comment><USER33> Right on.

To get code shipped I usually copy and paste if it really saves me time, but make a checklist of optimizations I can do at a later date.

When that time comes, find/replace does the dirty work.

</comment><comment><USER34> This is reminiscent of a classic Yegge blogpost: [http://steve- yegge.blogspot.com/2007/12/codes-worst-enemy.ht...](http://steve- yegge.blogspot.com/2007/12/codes-worst-enemy.html)

</comment><comment><USER35> Good IDE's will help you find code duplication.

</comment><comment><USER36> It's poor carpenter who blames his tools.

What he really hates is his lack of willpower when it comes to doing things the wrong way. Cut-and-paste enabled him to be lazy, but it certainly didn't force him to be.

<comment><USER37> Have you read it all? Because that's essentially what he says, despite the title and first phrase.

 _But like any sharp tool it has the ability to cut you just as easily as it cuts the wood, so you need to apply it with care._

</comment></comment><comment><USER38> copy/paste is one of the cancers of our profession

<comment><USER39> Correction: BAD copy/paste is one of the cancers of our profession.

Like all tools, it has its place, and there are people who tend to misuse it. Just because it's misused by bad programmers is no reason to deprive GOOD programmers of its benefits. You might as well ban screwdriver heads in power drills because lots of clumsy, inattentive people tend to strip screw heads with them. Similar misguided calls-to-arms have been raised over other useful tools such as preprocessor macros and goto.

<comment> loup-vaillant Could you give your estimate of the properly used/badly used ration of copy/paste, preprocessor macros, and goto? Mine would be "too low for me to worry about".

Okay, there _are_ some (corner) cases where they really are a good idea. But "never ever use this Chtulu Abomination" still is a damn good heuristic.

<comment><USER39> Good copy/paste: Anything boilerplate such as manually initialized structures/arrays of stuff, language-specific boilerplate (Java comes to mind here) that is essentially the same repeated shit but you gotta do it anyway, unrolling loops, setting up switch statements. Anything unavoidable (or too expensive to refactor) that's repetitive.

Preprocessor macros: Code generators, compile-time switchable code (such as logging) without filling your source files with #ifdefs

Goto: Managing complex resource allocation/deallocation within a function:

        bool success = false;
        int fd = -1;
        char* memory = NULL;

        fd = open(filename, "rb");
        if(fd == -1)
        {
            LOG_ERROR("Could not open %s: %s", filename, strerror(errno));
            goto done;
        }

        memory = malloc(BUFF_SIZE);
        if(memory == NULL)
        {
            LOG_ERROR("Out of memory");
            goto done;
        }

        for(blah blah blah)
        {
            if(failed to read or whatever)
            {
                LOG_ERROR("It's borked");
                goto done;
            }
        }

        ...
        success = true;

        done:
        if(memory != NULL)
        {
            free(memory);
        }
        if(fd != -1)
        {
            close(fd);
        }
        return success;

I disagree that dismissing a tool out of hand as an "abomination" is a commendable approach. If you take that attitude (or instill it in others), you'll probably never learn the proper use of such a specialized tool, leaving you ill-equipped to deal with the situations those tools handle well.

<comment> loup-vaillant Boilerplate: your language is bad. Switch to a better one, or modify the front-end of your compiler (they probably won't let you, though, and that sucks). I mean it, the cost of boilerplate is really high.

Unrolling loops: I never saw an instance where that was necessary. Plus, the compiler can often do it for you. I know we often use high performance applications (video decoders, 3D games…), but very, very few of us _write_ ones.

Switch statement: the syntax of the construct is heavy, we should lighten it. The rest is hardly boilerplate any more:

      light_switch (expr) {
        1  : i11; i12;  // "break;" is implicit
        42 : i21;
           : i_default; // "default" is implicit
      }

Preprocessor macros: I agree (I mean, I back-pedal), they are more useful than the rest. However, I still avoid them by default, as they make really good foot-guns.

Goto: your example shows exceptions (try…catch finally here). Goto makes much less sense when you have them.

Now my point isn't to never do those things _at all_. Only to think of them as last resorts. The "Chtulu Abomination" metaphor helps me do that.

<comment><USER39> "your language is bad. Switch to a better one"

All languages have boilerplate somewhere. It's unavoidable. Switching languages just because there are pain points is not a solution, because you'll simply be exchanging one problem for another.

"or modify the front-end of your compiler"

This is most definitely pie-in-the-sky. In the real world of real business, you can't do this.

"I mean it, the cost of boilerplate is really high."

Oh, I agree wholeheartedly. But you need to work in the languages that programmers understand today. I'm not going to have nearly as much success hiring smalltalk programmers as I would have hiring Python programmers, for example.

"Unrolling loops: I never saw an instance where that was necessary."

I have, but then again I've been at this for 20 years.

"Plus, the compiler can often do it for you."

You can't be sure until you look at the disassembly. Often, it does it wrong.

"Switch statement: the syntax of the construct is heavy, we should lighten it."

That's not going to happen within the next decade. Meanwhile cut & paste is my tool of choice to cut through the boilerplate.

"Goto: your example shows exceptions (try…catch finally here). Goto makes much less sense when you have them."

But it makes LOTS of sense when you don't have them. And it's still useful in C++ and Objective-C, where you still end up interfacing with C libraries. If you don't know about the goto "poor man's exception", you'll either end up with repeated and buggy deallocation code, or a monstrosity of nested scopes, or a bunch of check-return-and-throw constructs, which in the case of Objective-C will slow your program way down because it implements try/catch using longjmp.

Goto is also useful for breaking out of inner loops when the language doesn't support that (some languages support "break [label], to make it seem less like a goto).

I don't consider these to be last resorts; I consider them specialized tools. Much like design patterns, they're for mitigating some deficiencies in a programming language, but they're only effective if you know how to use them.

<comment> loup-vaillant It seems we mostly agree.

However, I'd like to challenge the belief that modifying the front-end of a compiler is too hard, or unreasonable. Even in the so-called "real world" where screwing up means you're fired.

First, the point is to tweak the _language_ , not the compiler. For instance, we may want to lighten the switch() syntax before GCC does, but we do not want to modify GCC itself if there's a simpler way.

More often than not, there _is_ as simpler way: just write a parser and a printer for your language, so you can do source-to-source compilation by chaining them. Printers are easy. Parsers are almost as easy, except for C++.

Then tweak your parser (lighten some syntax, add some keywords…), do some pre- processing between the parser and the printer (yeah, true macros), whatever.

Now there are some caveats: such a pre-processor may confuse IDEs, and may screw up error reporting (where errors don't track back to the actual source code). I personally don't care much about the former, but to solve the latter, the base compiler need to provide a way to be told where a given line of "source" code actually comes from (very useful for tools such as Lex/Yacc). Unfortunately, taking advantage of this will greatly complicate your pre- processor.

</comment></comment></comment></comment></comment></comment></comment><comment><USER40> I'm confused. Isn't this just a discussion of code duplication, that most try to avoid in programming because it means it should probably be re-factored into something reusable? Isn't it more about copy/paste then than cut/past?

</comment><comment><USER41> cut-and-paste != copy-and-paste.

</comment><comment><USER30> This article is just wrong.

Thanks to eclipse I go days without copy-pasting. May I suggest that if you're copy pasting too much you're doing it wrong?

If we take it to the extreme, why would you need to have the same code in two different places?

<comment><USER42> Ehm, the article describes what you are talking about. So in fact you agree with the article ;)

<comment><USER30> Yes, the problem is exposed correctly. But he blames his tools rather than taking the blame himself. Or maybe he is just trying to blame his coworkers using some euphemism, still lame.

<comment> loup-vaillant Tools have much, much more power over their users than you seem to think. Take subversion vs git for instance. How often would you merge branches with either tool?

We tend to follow the path of least friction. If a tool you use changes that path, and unless (even if?) you consciously fight it, you will change your behaviour, whether this is a good thing or not.

</comment></comment></comment></comment><comment><USER43> > I blame the IDE's

Please, _please_ learn to write correctly. "IDEs", not "IDE's"

</comment><comment><USER44> So I make a tool to make my work easy, then I go and misuse the tool to make my work shitty. Then I go around blaming the tool for my shitty code since it does not hinder me writing shitty code. I never thought a editor was supposed to hinder writing of shitty code.</comment>