Aurora - New MySQL-Compatible Database Engine - <AUTHOR> http://aws.amazon.com/blogs/aws/highly-scalable-mysql-compat-rds-db-engine/ <comment><USER2> (Disclaimer: I work for an AWS competitor.)

To me, there is a very interesting contrast to be had between this announcement and Microsoft's announcement: it feels like Microsoft is discovering the business value of being open at the same time that Amazon is living in the time warp of proprietary everything. Has Microsoft internalized that open source is (or can be) a differentiator in the cloud? Amazon is clearly still oblivious to it -- and it will be very interesting to see if this service generates fear of vendor lock-in...

<comment><USER3> Amazon is way ahead of everyone else in the cloud space. Their closest rivals (Microsoft & Google) are pretty much copying AWS. Most of Azure's services are equally proprietary and I doubt that will ever change.

This just leaves the other smaller players (DigitalOcean, Joyent, Rackspace, etc) who are mostly offering something akin to EC2 and then partnering with other vendors to offer the missing pieces on top (frankly what other choice do they have? They can't get into a race with Amazon/Google on who can build the most no. of services - they will never win that race).

<comment><USER4> ignore OpenStack at your peril.

<comment><USER5> I worked on and around Openstack for 18+ months. I think ignoring it is pretty safe to ignore at this point.

<rant>

There might come along something that is actually good, but Openstack isn't it. The architecture is just bad, it will never be as reliable as something like AWS. (not because of scale, purely because of lack of error handling capabilities)

Reality is these sorts of orchestration systems need to be written by specialists. The vast majority of Openstack was written by people that admit they have no clue about systems level programming. This is what hype does, it forces a whole bunch of bad programming and architecture down everyones throats.

The marketecture and hype machine have done horrible things to Openstack. Not only have vendors riddled the thing with lockin and crap code that needs to be supported, but they have pushed entire projects that should have been shot in the head. _cough_ Ceilometer _cough_

There is security, performance and plain availability problems everywhere, mostly embedded deep in the architecture.

Dumb decisions like "All systems must be Python" when Python is clearly not suited to a large number of the things they want to do is painful. As is the general "not invented here" syndrome and boys club that leads to certain libraries or patterns being pushed over others, usually to the peril of the project due to the 'blessed' thing being incomplete and unproven.

</rant>

I don't intend on returning to Openstack if I can avoid it, unfortunately my skill-set does tend towards that sort of thing so we will see how I go.

<comment><USER6> Why you said that when a lot of large enterprise are very actively investing in ? Can you tell more details ?

</comment></comment></comment></comment></comment><comment><USER7> Reading between the lines, it sounds like they've rewritten InnoDB's storage layer and made it multi-server aware w/ consensus.

</comment><comment><USER8> I think the problem AWS is seeing is how they are being commoditized (e.g. you can just run your database on the cheapest hosting provider), so profits will move towards 0. They will need to add more services like this (Aurora, DynamoDB, etc.) to ensure AWS isn't a commodity (and you can't just easily switch to DigitalOcean).

<comment><USER9> I think that the AWS vendor lock-in won't come from single features such as databases from specific vendors, but rather the ecosystem of Route53, elastic load balancers, S3, EBS, AMIs, Glacier, RDS, CloudFormation templates, SQS, security groups, libraries like boto, auto scaling groups, and everything else besides instance provisioning. Of course you can implement most of these yourself, e.g., HAProxy for ELBs, rabbitmq for SQS, but all that requires application code changes and significantly more configuration management.

<comment><USER10> So far I've been able to avoid vendor lock-in on the code level (besides S3, but the S3 API is simple and has become pretty much a standard for object storage).

On the infrastructure level, the "lock-in" part is not so much the technology, all of that is relatively easy to replace. But it would take me two additional FTE's to configure and manage everything AWS offers as a service ourselves.

But that applies to a lot of things these days, AWS just happens to be a one stop shop for a growing range of commodity services.

</comment></comment></comment><comment><USER11> More product details: [http://aws.amazon.com/rds/aurora/details](http://aws.amazon.com/rds/aurora/details)

and Frequently Asked Questions: [http://aws.amazon.com/rds/aurora/faqs/](http://aws.amazon.com/rds/aurora/faqs/)

</comment><comment><USER12> Ouch. This pricing is pretty rough for SaaS sold on the premise of cloud scalability.

At $200/month for the entry level, their lowest price is many times what the cheapest geo-replicated "SQL engine as a service" from Google or Microsoft is. I'm not sure how the performance differs, but I am guessing theirs are no slouches.

Microsoft offers "SQL Database" geo-replicated for as low as $15/mo., and it scales up from there. Not sure about performance, but it would be apples to oranges (MySQL versus SQL Server) and difficult to compare. I wonder what the TPC numbers are, but apparently the TPC organization doesn't allow publishing that yet.

Google offers "Google Cloud SQL", also geo-replicated, and their cheapest pricing is between $10 and $18 dollars a month.

<comment><USER13> I don't think this is targeted towards those who only need to store a small amount of data and have very low performance needs. If you are are already spending $500/mo on an RDS instance, then it sounds like this would be a great solution. If you are spending $10/mo on a micro instance for your database server, I don't think this is targeted towards you.

</comment></comment><comment><USER14> Does this introduce more latency compared to running a LAMPs stack on EC2 or does it target larger databases? My database is about 300 megabytes.

<comment><USER15> You probably aren't the target audience if your database fits in RAM.

<comment><USER14> You can see notable changes in performance if your database gets fragmented, and tweaking the .my.cnf script seems like a massive distraction from my actual work. Having somebody handle these and improve performance for long queries would be wonderful.

<comment><USER16> With a database that small, performance shouldn't ever be noticeable. Are you running the DB on EBS?

1) If you are having trouble with tweaking my.cnf, give the perl script `mysqltuner` a go. It is pretty good.

2) The biggest improvement would be moving the storage to SSDs. I personally prefer Linode to DO, but both are cheap, and good.

My linode servers can do full text non-blocking backups at about 100MB per second, so depending on your outage recovery requirement, you may be able to get away with a simple cron job.

</comment></comment></comment></comment><comment><USER17> Is a relational workload that requires four-nines-plus availability like Amazon is touting really something an organization would be willing to run in the cloud? Especially since that category includes a lot of personally identifiable data that legally can't be trusted to a third party. Wonder what kind of use cases they're aiming for.

<comment><USER18> people should be roughly as comfortable with this as with operating mysql on EC2 or using RDS, and I think that's quite common.

</comment></comment><comment><USER19> "Aurora" is already the name of a cloud infrastructure management product, an open-souce apache project no less: [http://aurora.incubator.apache.org/](http://aurora.incubator.apache.org/)

Using an existing name for a product in a similar space is just confusing and hurts everyone.

<comment><USER20> To be fair, I just searched for "aurora software" and "aurora open source" and got nothing for that project on the first page of results.

A lot of the good names are already taken; there's going to be overlap, and they're very different products.

<comment><USER21> Name-clashing with an Apache project really surprises me. Also, consider the fact that Aurora stems from Mesos, which supposedly Amazon (cloud; although DB people are doing this one) folks have to know.

</comment></comment></comment><comment><USER22> Why mysql? Why not postgres engine?

<comment><USER23> No one is saying that Aurora is built on MySQL, I'd personally like to know :) It's the protocol (drivers) compatibility what has been claimed so far.

I'd agree with you from a technical perspective it would have been better to offer Postgres compatibility, rather than MySQL. But I believe the intent is to target both a bigger market (as of today) and challenge Oracle.

<comment><USER24> > tables that I create in Amazon Aurora must use the InnoDB engine

It hasn't been claimed, but the article is filled with MySQL comparisons and references. I would not be surprised if it was a MySQL fork.

</comment></comment></comment><comment><USER25> 4 times faster than MySQL on the same platform, how did they pull that off?

<comment><USER11> Here is what they claim in the FAQ:

"Amazon Aurora delivers significant increases over MySQL performance by tightly integrating the database engine with an SSD-based virtualized storage layer purpose-built for database workloads, reducing writes to the storage system, minimizing lock contention and eliminating delays created by database process threads. Our tests with SysBench show that Amazon Aurora delivers over 500,000 SELECTs/sec and 100,000 updates/sec, five times higher than MySQL running the same benchmark on the same hardware."

<comment><USER26> I'd love to see more details (active benchmarking). At the very least, what is the CPU and disk utilization during the benchmark? It'll shed some light on how this was done: a 5x improvement in cycles per query? Or better caching?

People will benchmark this themselves ASAP. If you do, try to include some basic system metrics. The output of "vmstat 10", "iostat -x 10", and some "pidstat -t 1" would be a great start. This may only be possible for the MySQL benchmark, if Aurora is only visible via an API, and the database system can't be accessed directly (?).

<comment><USER11> To get the most comparable hardware setup for the comparison, I think you would want to run both Aurora and MySQL via RDS. RDS does not give you access to the underling EC2 instance, just to metrics via Cloudwatch.

</comment></comment></comment></comment><comment><USER27> Has anyone been able to find specific pricing on this? All of the language I've seen about it has been vague. One of the things that's kept me from using RDS for toy projects is that it basically amounts to running another instance on top of what I already have. For a "real" project, that's a drop in the bucket, but for toys and prototypes, it can be a significant chunk of the cost (so I usually end up just running my own db on the same instance as the web server for those projects).

<comment><USER11> Pricing is at [http://aws.amazon.com/rds/aurora/pricing/](http://aws.amazon.com/rds/aurora/pricing/)

The cheapest is a db.r3.large for $0.29 / hr

<comment><USER27> Ouch. Thanks.

<comment><USER28> Just slightly more expensive than the same instance with MySQL on RDS ($0.24)

<comment><USER29> Yes but "Storage is automatically replicated across three AWS Availability Zones (AZs) for durability and high availability, with two copies of the data in each Availability Zone."

Is that included in the server cost? Or will you actually be paying ~3x the server + storage costs?

<comment><USER30> The pricing needs to be more clear. For example we run MYSQL RDS in production with two AZ enabled, we pay x2 the prices.... so if its only costs a bit more than MYSQL RDS and you get 3 AZ for the same price -- this is good. Moreover, there are costs that come with storage (database size), this thing seems to be smart about being more efficient with allocating space.

<comment><USER31> As i understood this your database storage is replicated to multiple places, but it still runs on one instance. It does not have failover to those replicated places. You'll have to launch multiple instances for that just like RDS. So cheaper, no.

</comment></comment></comment></comment></comment></comment></comment><comment><USER32> I'm interested in transaction support for e-commerce. Magento users can experience significant performance issues on MySQL, and the upper tier are always interested in more RDBMS performance, eg via NewSQL solutions.

So what can Aurora do for that workload? Do the support multi-table transactions and referential integrity across all 3 Availability Zones? Similarly, they mentioned Durability targets; what's their targets for Consistency (ie ACID).

</comment><comment><USER33> This is beginning to feel like how it was when Intel was competing with AMD -- evolutionary improvements at first, which just kept on coming, increasing in size and impact, until AMD just faded out of the high-perf server market.

</comment><comment><USER34> So, this is a Xen VPS (Amazon uses Xen as far as I remeber) with a MySQL 5.6 database, with a custom storage engine that stores data in some Dynamo-based storage. Any code could access it via MySQL protocol (bindings to libmysqlclient.so). Fine.

<comment><USER35> I appreciate your technical insight, but glueing those technologies together is far from trivial. Making it easily to manage and reliable is even harder.

I trust your best intention, but even Dropbox was dismissed on HN as user- friendly rsync.

<comment><USER36> MariaDB enabled Cassandra as a storage engine: [https://mariadb.com/kb/en/mariadb/documentation/storage- engi...](https://mariadb.com/kb/en/mariadb/documentation/storage- engines/cassandra/cassandra-storage-engine-overview/)

</comment></comment></comment><comment><USER37> oh neat, AWS following GCP instead of the other way around.

<comment><USER25> Your profile says you work at Google...

<comment><USER37> I certainly do. On cloud no less. The reality is that Amazon got to market several years before Google, so GCP had some catching up to do. That trend is changing, and this is one indication.

<comment><USER25> If you're going to post your subjective opinion on something involving your employer vs a competitor it's best to include a disclaimer in future.

<comment><USER37> What was the subjective opinion? "neat"? The rest was factual.

<comment><USER25> The subjective opinion was about "following" when Amazon has had relational database services for years, and the latest engine they say was under development for 3 years, not exactly something in response to goog.

Irrespective of that you shouldn't be posting as joe public when in fact you are a Google employee playing cheerleader for Google products on threads about your competitors products. Keep it classy.

<comment><USER37> We're not talking about a relational database, we're talking about a mysql- wire-compatible database. Google BigQuery has been around for a while too.

Adding "disclaimer: googler" to every post I make on HN seems pretty obnoxious to both me and anyone reading. I just didn't feel that "neat" \+ a fact qualified. Clearly opinions on that differ, and I'll probably just post less in the future.

<comment><USER25> It's only necessary in cases where there is an obvious conflict of interest.

<comment><USER37> That's fair. It's certainly not my intention to astroturf.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER38> What is the use of a cloud relational database service? I thought everybody is going for Document Based databases.

<comment><USER39> If your data is fundamentally relational you're better off putting it into a relational DB. [http://www.sarahmei.com/blog/2013/11/11/why-you-should- never...](http://www.sarahmei.com/blog/2013/11/11/why-you-should-never-use- mongodb/)

<comment><USER40> Just want to point out that the headline is sensational and there are many good use cases for document stores.

<comment><USER41> The headline doesn't mention document stores, it mentions MongoDB.

<comment><USER40> Which (as a document store) has plenty of use cases... and even when you read the article it's mostly about how they didn't understand what they were doing going into things... It's not a compelling argument.

"But this stuff wasn't obvious at all. The MongoDB docs tell you what it's good at, without emphasizing what it's not good at. That's natural. All projects do that. But as a result, it took us about six months, a lot of user complaints, and a lot of investigation to figure out that we were using MongoDB the wrong way."

The film/actor/career example they wanted to do could have been solved pretty easily with a document that had an _id for an actors collection and a name... that's just the document paradigm... I'm a huge fan of both postgres and mongodb and I know that's not a popular opinion around HN. I just get tired of seeing clickbait headlines being cited as the criticism within is correct.</comment></comment></comment></comment></comment>


Source Code for IoT Botnet 'Mirai' Released - <AUTHOR> https://krebsonsecurity.com/2016/10/source-code-for-iot-botnet-mirai-released/ <comment><USER2> I bet it can be used for at least two good purposes. First, to scan for vulnerable devices. And second, to shame vendors of insecure devices.</comment>


Using Perl to scrape the web - <AUTHOR> http://ssscripting.blogspot.com/2009/12/using-perl-to-scrape-web.html

<comment><USER2> Mechanize for Perl & Ruby is quite cool.

However, it's not able to execute JavaScript. The only lib I found which does so with a reasonable subset of JS is HttpUnit, in Java. Though it has kind of ugly interface IMO, I use it with a success. Doing it from a Clojure REPL makes it quite handy tool for web scripting.

<comment><USER3> To use Javascript then try the CPAN module WWW::Selenium (<http://search.cpan.org/dist/Test-WWW-Selenium/>).

<comment><USER2> Yeah I considered it but Selenium uses Firefox, does it? And I needed a self- contained script without such dependencies.

<comment><USER3> No, Selenium works with all major browsers.

</comment></comment></comment></comment><comment><USER4> Perl would certainly be my language of choice for screen scraping. And, some people see it almost like stealing. I know of people that look at Google negatively for its news indexing method. I have skimmed through a book (though I do not remember the name) that seemed to claim that Google profits only from the work of others. (I think they added value in their collaboration of information, though.) Nonetheless, you must be careful not to create a backlash (or legal issue) with screen scraping. The irony is that one of the best targets for screen scraping content for your own benefit may be Google itself... however, it almost seems like they encourage it. (They want you to use their APIs instead, though.)

Spidering has been around for a long time, and people act like screen scraping is new. It is really the same that has existed for years. If you are going to do it, though, Perl is certainly the way to go. It is fast, efficient, and robust.

<comment><USER5> Theft requires that the taking denies the current owner access/use to whatever was taken. Copyright infringement appears to be what you're referring too.

Google IMO is more of a symbiont than a parasite.

<comment><USER4> You are correct, and I agree. It is not theft, and Google is really a huge collection of mitochondria... helping the fledgling Internet become something more by combining it with an intellectual powerhouse. I could not have said it any better myself.

</comment></comment></comment><comment><USER6> Perl has been the language of choice for web spidering since 1999, when it replaced REBOL for that purpose. Hint: LWP module made such a dent on the industry, nobody was able to replace it until the last year or two when other things started popping up.

</comment><comment><USER7> This line seems problematic:

      use strict;

Most of the web is messy. Beautiful Soup and its ilk would seem like a better choice for parsing.

<comment><USER8> Ahem.

<http://www.perl.com/doc/manual/html/lib/strict.html>

<comment><USER7> Ouch, should have RTFM. Thanks for the pointer.</comment></comment></comment>


The Blue Tape List - <AUTHOR> https://randsinrepose.com/archives/the-blue-tape-list/ <comment> mark-r The example of the door dents was particularly relevant for me, because it actually happened. When we built our house in 2000, the front door got a lot of dings in it. We mentioned it to the builder, and they sent over a new door but never installed it. We stored it against a wall in the garage. 19 years later and it's still in the garage. The only time it's moved is when it fell over and put a dent in the car. Not sure what conclusions you can draw from that.

<comment><USER2> Perhaps that labor and the experience needed to do a job correctly is often more valuable than the materials needed for the job.

Or maybe it's just about motivation.

<comment><USER3> Certainly. It's a flaw in English (and perhaps our brains in general) that it's easier to discuss objects than services. A Verb in the Kingdom of Nouns and all that jazz.

</comment></comment></comment><comment><USER4> As someone that's recently joined a new team this was a fantastic article to read. I do sometimes get the sense that everything is broken—even though after waiting a few days it seems like super minor things.

One error though:

> he whipped out a role of blue painter's tape

should be "roll"

<comment><USER3> Maybe it was something else that could fill the role of blue painter's tape.

</comment></comment><comment><USER5> It'd be nice to be able to "unlock" this heightened awareness after you've become dulled - any ideas how to do that?

<comment><AUTHOR> In my experience, it's really difficult. Things that have worked for me:

* switch to a different department/team/role

* take a sabbatical (maybe two weeks, but a month is better)

<comment><USER6> How do you switch to a new department in a large org? I've been working on a couple cloud systems for 5 years. Demand is high in my area but I want something more technical but without a pay cut. I use JavaScript day to day and built a lot of REST/SOAP web services over the years.

Most of the internal jobs are .net and I could learn c# (have some java experience out of college) but I don't think my resume will sway the managers.

<comment><USER7> You won't know until you ask. It depends on your org, clearly, but in general large companies are somewhat supportive of mobility, because it means they don't lose you to somebody else.

Find a job you actually care about. Chat with the manager. See what they need. Good chance nothing comes off it, but at least some chance it opens doors for you. Maybe not even immediately, but six months later that manager mentions to another manager "Hey, X cares about Y - aren't you staffing up a new effort?"

Without a paycut might be slightly harder, but usually large orgs operate, for better or worse, on "all software engineers are interchangeable cogs". This is one areas where that actually can play to your advantage ;)

<comment><USER6> If I switch internally I think I can avoid the pay cut. If I look outside I become a junior dev again in X technology, x could be .Net/C# or Python.

<comment><USER8> _If I look outside I become a junior dev again in X technology, x could be .Net /C# or Python._

No, no, no. You're looking at the wrong companies. At the _right_ companies, seniority has nothing (literally nothing) to do with how long you've used a particular programming language.</comment></comment></comment></comment></comment></comment>


Kurzweil: 'Exponential' Change Ahead for Games, People - <AUTHOR> http://www.news.com/Kurzweil-Exponential-change-ahead-for-games%2C-people/2100-1043_3-6231644.html?tag=item

<comment><USER2> I was at GDC and saw this keynote. I'm not sure why it was a GDC keynote. The talk was barely even tangentially related to game development.

The talk was similar to his TED talk but an hour long rather than 20 minutes. <http://www.ted.com/index.php/talks/view/id/38>

<comment><USER3> Games debatably present the best opportunities for deployed AI. The alternatives are backend servers or aggregate knowledge across the web.

Games are the most consumer facing.

<comment><USER2> Yup. Agreed.

Chris Hecker's talk, Structure vs. Style, was much better regarding this subject. His argument is that we don't have a Structure vs. Style decomposition for AI. We have it on the web => HTML(structure) and CSS(style). We have it in computer graphics => triangle(structure) and texture map(style).

Of course, this got the panties of all of the LISPers in a bunch.

</comment></comment></comment><comment><USER4> If Kurzweil is right about even a fraction of the stuff he says, I'm fairly certain he'll prove to be an exceptionally important thinker. Unfortunately, I have serious doubts about almost everything the man says. He often seems to me to be taking the canned shot approach to writing: predict as many big things as possible and support them all as best he can, and see what hits.

In other news, this reminded me a post I saw a few years ago: [http://d-squareddigest.blogspot.com/2005/09/most- important-w...](http://d-squareddigest.blogspot.com/2005/09/most-important- weblog-in-world-what.html)

"As the American economist Tyler Cowen points out, loony extropian and transhuman nanotechnoartificiallinteligenceoboosterist Ray Kurzweil is, because of his wild technoutopian prophesies, "the most important thinker today, if only in expected value terms".

Or at least he was, until this morning when while having my breakfast, I made the following prediction:

"Everything that Ray Kurzweil says will come true, and not only that but we will all get a pony"."

<comment><USER3> Hype aside, he has an interesting approach to invention. He planned on launching a tool to help blind people read. It is a digital camera with a tiny computer. During development, people worked on algorithms on PCs knowing they would be deployed on a portable device not yet available. There is some use to plotting technological growth trends.

I think Cowen's comment is a bit tongue-in-cheek.</comment></comment>


Show HN: Myrtille, open source HTML4/HTML5 remote desktop protocol client - <AUTHOR> Myrtille, open source - zero plugin - browser remote desktop. 1.5.0 available, with new features and improvements! http://cedrozor.github.io/myrtille <comment><USER2> Requires a server-side component...

Really cool project, though.</comment>


Worst Password list of 2015 - <AUTHOR> http://blogs.wsj.com/digits/2016/01/19/qwerty-and-123456-top-list-of-2015s-worst-passwords/?mod=LS1 <comment><USER2> we're making huge progress as i guess all these complex password rules for new passwords have started to show their effect, and we finally started to leave old faithful "12345" behind :)

"For the first time last year, "1234567890," "qwertyuiop" (top row of keys on a standard keyboard) ..."</comment>


What We Are Seeing - <AUTHOR> http://www.avc.com/a_vc/2011/10/what-we-are-seeing.html <comment><AUTHOR> Note to self: Do Fred a favor and don't submit his posts too early in the morning. This is not the first good and relevant post of his I've inadvertently buried.</comment>


Introducing the world's first smart bicycle bell - <AUTHOR> http://shoka.me/

<comment><USER2> thanks for sharing</comment>


Protons not as "strange" as expected - <AUTHOR> http://www.symmetrymagazine.org/breaking/2010/04/27/protons-not-as-strange-as-expected/

<comment><USER2> "What we now think of as the atom – consisting of a nucleus and its electrons – was first described by John Dalton in the early 1800s. Later that century, scientists found that they could strip electrons out of the atom. Then, in the early twentieth century, they found that the nucleus contained even heavier particles, which were dubbed protons and neutrons.

"Fast forward to the 1960s, when theoretical physicists proposed that protons and neutrons have building blocks of their own – particles called quarks. Shortly thereafter, researchers at SLAC National Accelerator Laboratory conducted experiments that demonstrated that quarks really did exist."

Reading this passage I found myself struck by how glad I am to have read the Feynman Lectures on Physics in high school. He does such a great job explaining what made people ask the questions that this author presents in a "suddenly, people thought there were protons, neutrons, and quarks" manner of talking. If you haven't read Feynman's lectures, do so. I don't know that ignorance of physics has the same repercussions of, say, numeracy (<http://en.wikipedia.org/wiki/Innumeracy>), but it certainly provides a healthy background against which to read these types of articles.</comment>


How to hire a programmer when you have no technical knowledge? - <AUTHOR> Hi guys, what should I do if I would like to hire a programmer but don't have technical knowledge to evaluate a candidate? Are there any agencies or services that can help me with that?<p>Thanks! <comment><USER2> With no technical knowledge, you shouldn't hire a programmer.

But the way you write about this, it's as if you are assuming you cannot gain technical knowledge. Unless you are a rock, this assumption is probably false.

So first step, learn about the domain of concern. Including some technical knowledge. You'll need at least enough to decide whether your plans are well considered, and to communicate your intent to whoever you might hire.

Then you could learn about hiring: [http://lmgtfy.com/?q=should+I+hire+a+programmer](http://lmgtfy.com/?q=should+I+hire+a+programmer)

<comment><USER3> Damn, you beat me by a minute! ;)

</comment></comment><comment><USER3> [http://lmgtfy.com/?q=recruitment%20agency](http://lmgtfy.com/?q=recruitment%20agency)

You might want to refine that search to include the area where you are looking to hire.

<comment><AUTHOR> What if I want to recruit him on my own - I don't want to hire an agency to do the hiring for me. I need sb/sth to help me with evaluatuion of candidates.

</comment></comment><comment><USER4> It depends what you need them for, you're leaving out a lot of important details like task and budget. Contact me if you'd like to discuss, it's very likely I can help.</comment>


LinkedIn sitemap.xml - <AUTHOR> http://www.linkedin.com/sitemap.xml

<comment><USER2> It would be helpful if an editor could provide some kind of warning of this in the title.

My Netbook is still trying to open this over dialup, and xPUD has ground to a halt.

<comment><USER3> I agree, it just beach-balled Safari here.

<comment><USER4> I've noticed it doesn't take much to beach-ball Safari, at least for me. OS 10.8, 2008 model MBP, Safari 6.x

Canary kept on chugging at the expense of the whole OS hanging.

</comment></comment></comment><comment><USER5> I've actually been working on my own site's sitemap, and I've figured out a bunch of things:

You can use a sitemap index [0] to split a sitemap into multiple ones. This is beneficial because you can change part of it and not have to have google re- crawl everything, since index updates happen right after a whole sitemap has been crawled. As a result, changes you make to a smaller sitemap get indexed way faster.

I've switched to grouping my bulk pages into sitemaps alphabetically, and then putting important ones (front page, about) into different sitemaps, and specific landing pages into yet another different one.

[0] <http://en.wikipedia.org/wiki/Sitemap_index>

</comment><comment><USER6> They might want to consider using sitemap index files: <http://www.sitemaps.org/protocol.html#index>

Edit: "If you want to list more than 50,000 URLs, you must create multiple Sitemap files." According to 'icebraining' they have 47785 urls.

<comment><USER7> They already have it: <http://www.linkedin.com/robots.txt>

<comment><USER6> Ah, ok, strange that they have /sitemap.xml in addition to that then. I would assume any robot that would understand sitemap also understands that it can be specified in robots.txt

</comment></comment></comment><comment><USER8> Here is one for google plus <https://ssl.gstatic.com/s2/sitemaps/profiles- sitemap.xml>

<comment><USER9> Wow, this is a sitemap of .gz files containing the links to the actual profiles. I did a wget to download all the .gz files. It's still going 10 minutes later.

<comment><USER10> Do you have a script to fetch them all?

<comment><USER9> Here, I wrote about it on my blog: [http://mgakashim.com/123/downloading-and- processing-a-list-o...](http://mgakashim.com/123/downloading-and-processing-a- list-of-all-google-plus-profiles)

</comment></comment></comment></comment><comment><USER11> What about it?

<comment><AUTHOR> An easier scraping?

<comment><USER12> It's against their TOS.

<comment><USER13> The Clickwrap based TOS you mean? ;-)

It's frustrating, at least for me, that the legality here is still so gray (at least IMO it is).

Much of their content is likely in the public domain (facts / basic non- creative information) although there is definitely plenty that is not; the lack of 'black-and-white' is what frustrates me...

Clickwrap - <http://en.wikipedia.org/wiki/Clickwrap>

</comment></comment></comment></comment><comment><USER14> Is this actually for all 150M users? I guess they really want crawlers to go the shortURLs easier...

<comment><USER15> Not even close to 150M:

      $ curl http://www.linkedin.com/sitemap.xml | grep -c '<url>'
      47785

I wonder why were these selected. Probably the most searched.

<comment><USER16>
      user@/tmp > grep lastmod sitemap.xml  | cut -d - -f 1 | uniq -c
      47785     <lastmod>2006

They were probably sitemaping all their users when they started the service, then they have decided to stop shortly after.

</comment></comment></comment><comment><USER9> Managed to get a list of all only the URLs.

<https://dl.dropbox.com/u/8433360/linkedinusersurls.txt>

</comment><comment><USER17> Found in robots.txt link to Sitemap: <http://partner.linkedin.com/sitemaps/smindex.xml.gz>

</comment><comment><USER18> This document just locked up my browser, i7 w/ 8gb ram using chrome.

</comment><comment><USER19> Interesting, thanks for the share.

</comment><comment><USER12> I'm not In the sitemap :(

In case you were looking for me:

<http://linkedin.com/in/pjbrunet> :)</comment>


Lifestyle programming - <AUTHOR> http://successfulsoftware.net/2013/11/06/lifestyle-programming/ <comment><USER2> Strange that this dropped off the page so quickly, I think it's an interesting post.

I'd like to add that it's quite possible to have a high growth company that does not take on institutional investment. My company is neither a Lifestyle business (I also hate that term, I think it's an awesome thing) or a VC-backed startup, but we have lots of users and are making really nice revenue.

The costs have gone down so much, that I think the reason people still think fundraising is required is because they can't afford to live or hire in SF without it, and there is a lot of pressure by successful investors for young founders to make the "big" bets.

My goal is to have an id software, 37signals, MailChimp, Atlassian, GitHub, Campaign Monitor, etc. kind of company. Now _those_ companies are/were fucking cool. Plus, they've actually stuck around for more than a few years!

<comment><USER3> Lifestyle business, micropreneur, microisv, indie. None of them are great terms. I am still waiting for someone to come up with a better term for what I do.

<comment><USER4> I've always been fond of the label "living the dream."

<comment><USER5> Sounds completely unobtainable. Lifestyle at least sounds like a choice that a person could make, and have control over.

<comment><USER6> Most of the people I know who use that phrase are being somewhat sarcastic. Most notably the helicopter pilot from Germany I knew who was getting paid absolute shit because until you make a certain number of flight hours, your options are extremely limited and you have to take a second job to make ends meet. Every time I said, "What's up?", he said "Living the dream, man!" then proceed to complain about all the shit he has to put up with. He loves flying, but the "dream" is anything but.

<comment><USER5> Poe's law strikes again!

</comment></comment></comment></comment></comment></comment><comment><USER7> For those interested in this path, it's worth checking out Rob Walling's book "Start Small, Stay Small: A Developer's Guide to Launching a Startup."

I found this book especially insightful.

<comment><USER8> Ditto, great book. Rob's blog is also full of interesting stuff. (softwarebyrob.com)

<comment><USER9> and the podcast startupsfortherestofus.com

FollowSteph3 reminds me a little of Mike from the podcast. While Rob Walling realizes he needs to hire freelancers etc. in order to sustain the business and make it work in a reasonable time frame, Mike took a few solid years to bring in other people. He wanted to do it all by itself. I've been listening to the podcast for a few years and you can see how Rob built one solid business and just launched another one in the same time frame Mike barely did one.

<comment><AUTHOR> I wouldn't say barely did one ;) [http://www.followsteph.com/2013/08/07/landlordmax-2012-2013-...](http://www.followsteph.com/2013/08/07/landlordmax-2012-2013-fiscal- year-10-year-anniversary-and-a-10th-record-year/)

</comment></comment></comment></comment><comment><USER10> Nice write up.

I wonder if this is something that's more common with certain life circumstances.

For example... I'm married and have 2 kids. I had a corporate job for the first 10 years of my marriage. I traveled a ton and completely missed out on their lives. When I changed jobs, it was a bit of a shock for all of us.

Now, we recognize that it's not what we want anymore. I work for a small software company that is ridiculously low-stress, but even now I wish I could spend more time with my family and be a little more free to take off and go camping with the kids if we felt like it.

</comment><comment><USER11> AMEN: "A man is a success if he gets up in the morning and gets to bed at night, and in between he does what he wants to do." ― Bob Dylan

</comment><comment><USER12> Totally agree - it's a great lifestyle. +1: It works for me too - have been doing it for 7 years now.

<comment><USER13> Could you share a bit more information about how well it works for you ? Are you making an income comparable with a 'decent' corporate job (eg. 100K + a year) ?

</comment></comment><comment><USER14> "But really, how much money do you need? Is money going to make you happy? How many meals can you eat in a day? How many cars can you drive?"

House and decent education for kids is a major expense.

<comment><USER15> Most families manage with combined incomes less than senior software engineers. How big does the house need to be?

<comment><USER16> Their kids typically don't get a decent education. Give me a location in a developed capital and a price bracket.

<comment><USER17> Get yourself a proper country.

</comment></comment></comment></comment><comment><USER18> Are there places where "birds of a feather" congregate? Forums, lists, etc? I'm excited to tap the wisdom of others as I get my business off the ground.

<comment><USER3> Two free forums populated mainly by people bootstrapping their own businesses:

[http://discuss.bootstrapped.fm/](http://discuss.bootstrapped.fm/) [http://discuss.joelonsoftware.com/?biz](http://discuss.joelonsoftware.com/?biz)

<comment><USER19> The bootstrapped.fm link makes me wonder what other wonderful sites the HN population visits, but that I have no clue about.

<comment><USER20> Snap. I've been looking into doing something like this for the past few months, and this is the first of heard of this and lifestyle.io. Guess I wasn't looking hard enough :)

</comment></comment></comment></comment><comment><USER21> So true. Reading posts like these makes me believe more and more what I decided to do by leaving a full time job and working for myself and what makes me happy the right decision.

I have only just started and am about to release my first startup so I'm not at that level yet. I'm hoping to get there someday though.

</comment><comment><USER22> Wow, there are other people like me? I've been doing this for ten years full time. We should all get together or something. Maybe help each other out. I'll be making a move to a different type of programming or going back into the work force because my software is pirated so much these days.

<comment><USER23> Bootstrapped / friends and family only funded companies are the majority. You just don't hear about them much because they don't make for good headlines.

</comment></comment><comment><USER24> Working from home my employer's work has been drying up and I've been bouncing between building my own software or getting a "real job"... Your article (and software) has inspired me to do my own thing.

At least until the next article convinces me to learn more under someone else's time ;)

</comment><comment><USER25> I also like this idea but it just seems difficult to get the business started and making enough money to compensate yourself. Just to be sure, I don't mean working from home for elance comes under lifestyle business.

<comment><USER26> Don't set the bar too high at first. Just aim to make "some money, greater than zero". If it's money you no longer actively have to work for, then congrats, you've got an income stream!

Do it again, and again, and you'll find that 1. You're getting closer to supporting yourself from it 2. You get a better idea of how to do it again, faster, cheaper

</comment></comment><comment><USER9> All the power to you brother.

Do you miss the human interaction one gets at the office?

<comment><USER3> Not really. When I work, I work. And I get to spend a lot more time with my family than I would otherwise. I have plenty of interaction with customers via email, skype etc. Also I hang out online with other entrepreneurs, who are generally very helpful and supportive. Its not quite the same as face-to-face interaction. And I guess I don't meet as many new people as I would working in an office. But I don't have to put up with people I don't like and I certainly don't miss office politics!

If you are very extrovert, it might not be for you. But how many extrovert programmers do you know? ;0)

</comment></comment><comment><USER27> What's utterly missing in the current environment is an avenue for the mid- risk/mid-growth businesses to get financing. Low-risk/low-growth companies can get bank loans, and high-risk companies have VC, but nothing's in the middle.

Something that reliably grows 25% per year isn't slow, ineffective, or useless. Most people would be thrilled to see 25% annual income growth (or even 10%). It is, after all, exponential growth. But that's a space that no one will finance. These companies are too risky for bank loans or personal funds (for most people) but VCs aren't interested in 20-40 percent per year. What I'd like to see out of the crowdfunding trend is a viable method of financing the mid-growth/mid-risk space.

This space is a natural fit for top programmers, most of whom succeed by continuously growing their capability (measured in value-add potential) by 20 to 40 percent per year-- not being goofy and trendy and hoping to be noticed by Sequoia or TechCrunch, which will probably never happen.

Here are some earlier thoughts I had on this issue: [http://michaelochurch.wordpress.com/2013/03/26/gervais- macle...](http://michaelochurch.wordpress.com/2013/03/26/gervais- macleod-17-building-the-future-and-financing-lifestyle-businesses/)</comment>


The Day Einstein Died - Photo Gallery - <AUTHOR> http://www.life.com/image/first/in-gallery/41842/exclusive-the-day-einstein-died

<comment><USER2> Just, wow:

 _So, I get to the building and nobody's there. I find the superintendent, give him a fifth of scotch, and he opens up Einstein's office so I can take some photos._

</comment><comment><USER3> He bribed people with alchohol repeatedly...

And the picture where they have Einstein's brain on a cutting board is a bit... well

<comment><USER4> Social engineering at it's finest. Almost everybody will take some kind of alcohol.

</comment></comment><comment><USER5> I think these are still intensely private affairs.

</comment><comment><USER6> Recommended book: "Driving Mr Einstein"</comment>


Recounting the rationals (2008) - <AUTHOR> http://fermatslibrary.com/s/recounting-the-rationals <comment><USER2> I find this incredibly beautiful. Short papers that introduce a "general" audience to a supposedly well-known mathematical idea should really be seen as a valuable contribution in mathematics.

Granted, I am a pure maths PhD. I would be really curious to know how many non-mathematicians can glean value/enjoyment from reading this paper, and what if anything could constitute a necessary/sufficient condition for achieving that.

</comment><comment><USER3> I still find that the sortest one is this:

take the prime factors of _n_ , the primes with even power go to the numerator with halved powers, the primes with odd powers go to the denominator also with halved powers but rounded up. This is bijective, due to the uniqueness of prime factorization, and of course the numerator and denominator greatest divisor is also guaranteed to be 1, as they not share prime factors at all. So e.g.

45 = 3² _5¹- > 3/5 60 = 2²_3¹*5¹ -> 2/15 2 -> 1/2, 3 -> 1/3, 4 -> 2/1, 5 -> 1/5, 6 -> 1/6, 7 -> 1/7, 8 -> 1/4, 9 -> 3/1, ...

</comment><comment><USER4> pdf: [https://www.math.upenn.edu/~wilf/website/recounting.pdf](https://www.math.upenn.edu/~wilf/website/recounting.pdf)</comment>


What do you mean 'we need more time'? Project schedule estimation in software - <AUTHOR> https://blogs.dropbox.com/tech/2015/10/what-do-you-mean-you-need-more-time/ <comment><USER5> Almost at the end : "Yes, all this sounds like a lot of work". Exactly. And it will most probably still turn out to be wrong, because you will forget to ask some questions, and there will be things that you simply cannot foresee that will influence the time.

Turns out it's pointless to even try to do this. So why waste valuable time to do all this, if you know it's not the truth anyway? The answer is to accept that you will learn things as you go along, and that things take as long as they take, and to rather deal with that. Marketing wants a product by Christmas. Fine, we can do that. They just cannot say what needs to be in that product by Christmas. They can have whatever the last production quality release is at that stage. Instead of taking many weeks or months to try and answer all these estimation questions, we start and tackle the high risk items first. We learn from them, readjust, and as we get closer to the deadline, we keep communicating with all the other stakeholders what we think will be delivered on that date. It's a narrowing cone of uncertainty. One sprint before the deadline we'll be pretty accurate in predicting what it will be. 2 or 3 sprints before, accurate enough to be able to do marketing. At the start : the best we can predict is simply whether we feel we can have something useful at the deadline.

<comment><USER6> Except some of us make a living selling development of projects and solutions to customers that require pretty accurate (cost) estimates prior to even getting the contract.

Crazy world, I know...

<comment><USER7> Absolutely this, clients need to know their costs upfront.

Meanwhile, I've been on the other side, working with a shop that insisted they were agile, and so refused to give an upfront cost.

That project ended up delivering a substandard project where key parts did not work well. Their answer was to tell us they would of course be happy to be paid for another 2 week sprint to fix those bugs...

As a customer it wasn't very satisfying.

<comment><USER5> I don't think working fixed price quote would have given you a different outcome though? The team produced bad quality product, fixed cost or agile won't change that.

Selecting the team to build the product on price is often a bad idea, and that's all fixed cost gives you above agile (the ability to select on price).

<comment><USER7> A fixed cost makes it easier to turn and around not pay (in whole or part) if the product is really not up to the standard specified. It's harder to do that if the cost is not fixed but overrunning and the response is "well we just need more time".

Also fwiw, the team wasn't selected on price.

<comment><USER8> Not pay? Heh. For work performed? OK...

That is why my contracts require payment up front.

<comment><USER9> Sounds like they should have negotiated for a base rate with bonus pay out on meeting QC targets. For heavens sake people, align your incentives!

<comment><USER10> Everyone, on both sides of this river, they need to grow up.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> Analogies like "painting a room" are pretty useless because they are very misleading. Especially when it is an innovative product, a better analogy would be: "painting a room on Mars".

With software development, there are a lot more "unknown attributes" that determine the main part of the effort estimation unless it is a repetitive task.

So, a critical/skeptical engineer would always drive the customer nuts by asking questions nobody can know and the conclusion will end somewhere like it is not possible to give a upfront estimation and stop the project. Or the effort to give the estimation approaches the effort to develop.

<comment><USER12> Before NASA sent their first drone to Mars they made a lot of predictions, a lot of tests, guessed a lot of things that may get wrong, etc. Do some of their missions still fail? Sure. But the amount of success is much much larger because they didn't just sent a drone there and watched what happened.

So even if all you say is correct it's still worth improving one's estimation skills.

<comment><USER13> > Before NASA sent their first drone to Mars they made a lot of predictions, a > lot of tests, guessed a lot of things that may get wrong, etc. Do some of > their missions still fail? Sure. But the amount of success is much much > larger because they didn't just sent a drone there and watched what > happened.

How much effort went into estimating Opportunity's lifetime at 90 days? How useful has that estimate proven to be?

<comment><USER14> That is cherry picking. Did you try to comprehend the post you are replying to? Take into account all of the estimations, not just one failed estimation.

<comment><USER13> Sure, take into account all the estimations. How much value have they delivered? Show me how NASA has got more science done than if they had "just sent a drone there and watched what happened". Because I don't believe they have.

<comment><USER12> The funny thing is despite having the opposite opinion I understand so well how you see it. I wish I could show you my point of view better.

<comment><USER14> That was exactly how I felt. It is a matter of pay grade, I think.

</comment></comment></comment></comment></comment></comment></comment><comment><USER15> [http://qr.ae/ROlwZp](http://qr.ae/ROlwZp) \- I've found this answer on quora to be one of the best explanations as to why software estimates are so off.

<comment><USER16> Digging down into the details is one of the best responses to absurdly unrealistic schedules.

</comment></comment><comment><USER17> Estimates are a tricky thing. Ultimately, I don't think they work in software. We should be looking for a different solution.

Think of it this way - a physician can prescribe a medication and tell you how long you'll be on it, but can she estimate how long it will take before your blood pressure falls within normal parameters? She can tell you what has happened in the past, but suppose you take the medication and your blood pressure remains stubbornly high. Has she failed to meet her targets?

A lawyer may be able to tell you, roughly, how long litigation tends to take, but can your lawyer estimate how long a case will take, what it will cost, what the outcome will be, and the date it will all be wrapped up? If that doesn't happen, did the lawyer fail to meet targets?

Well, in both cases, sort of, yeah. You didn't get the outcome you wanted. But these professions, run by the people who staff them, understand what can be predicted and what can't.

This blog entry uses the task of estimating painting a room as an example, and focuses on breaking the task into subsections. Ultimately, I think the entire approach is deeply flawed, and it all starts by thinking that software development is similar to something like painting a room, just a more complicated version of it.

In my opinion, it's very different. Of course, it's very different from prescribing medication or litigating as well - in fact, I'd say that it's generally considerably less predictable than even these things. I'd say it's closer to writing a novel, creating a film, or recording an album, except that software needs to _work_ , and can be much more mundane at times as well. Ultimately, I don't think the analogies are useful, we need to realize that software is its own thing, and steer clear of too much reasoning by analogy or example.

Unfortunately, the anxiety caused by making estimates under conditions of uncertainty (and possibly duress) is a source of stress and unhappiness for developers, and I do think it's one of the things that drives people out of the field. I actually think this may also be one of the reasons people love creating apps as a hobby but hate doing it for money.

</comment><comment><USER18> The thing that annoys me most about project estimation is when the micro- managing types don't let you even look at the problem space first, an instant answer is expected. Of course the figure plucked out the air could be 'x days' with 'x' being greater than the time one expects internally so backside is covered, yet that is not really a true answer.

At the moment I am putting together a 'simple' feature on a website, where I have been able to explore the problem space first. For the $content there actually needs to be half a dozen date fields that were not in the original brief, I doubt I would have discovered this had I not rolled a little bit of code first and explored 'prior art' in the area.

I need to put together a semi-working prototype to discover such things - particularly when UX is involved. Yet, far too often, the opportunity to do actual preliminary dev work is denied by those that 'know best', preferring a spec. in a Word document (or something else equally 1990's) instead of a rough working prototype in code (with back-of-envelope sketch).

My current feature-being-added will need to be re-factored at some stage, once users have started using it and problems/features overlooked in the original brief catered for. It is not as if the users of my new feature will know that 'actually they wanted it to work this way' until they have something to see. So why we have these silly cast-in-stone processes I do not know.

I also must say that I find the metaphor of painting a room is quite a poor one, I would not need to paint 'an alcove first' to get a handle on how long the whole room would take.

<comment><USER19> > don't let you even look at the problem space first

this is the core of the issue. when one has the time to break down all unknowns and discover them, estimate are quite spot on. but who want to pay for the estimate?

so we are left trying to guess what time a particular unknown is going to take to solve.

the good metaphor here is asking how much time it will take to solve a puzzle magazine, from cover to cover by looking at the title.

<comment><USER20> The point you're making is exemplified in this article which makes an analogy to a hike from San Francisco to Los Angeles:

[http://www.michaelrwolfe.com/2013/10/19/50/](http://www.michaelrwolfe.com/2013/10/19/50/)

from

[https://www.quora.com/Why-are-software-development-task- esti...](https://www.quora.com/Why-are-software-development-task-estimations- regularly-off-by-a-factor-of-2-3)

</comment></comment></comment><comment> work-on-828 As a new grad who has worked a couple different jobs now, some things I don't get about this:

\- What do you do when you don't have a spec and when you ask the senior engineer you are working with questions, they don't respond or push back against having a spec or a plan longer than the very next feature?

\- The author says that estimation is easier to do if you are doing something you have done before. What do you do if you are always working with a new framework or language?

\- How do you break down the steps if you don't know what a project entails until you are finished?

\- What do you do if asked to estimate how long something will take to debug? What if you just started on a project and the error message you are trying to debug doesn't make sense? My approach there is to try to understand the system I am working with. If the system doesn't have documentation and the existing people on the project don't have time to give a walk through of things, I figure that means source diving. However, I've gotten the feedback that I should avoid trying to "understand the universe" when debugging. What can I do to produce accurate estimates of how long it takes to find a bug in a system without spending the time to gain a mental model of the system?

\- This excercise takes a lot of time. What do you do when asked for an estimate in a meeting rather than over email? What

\- What do you do if the thing you are building relies on getting an external API to work and that API is either undocumented or is documented in a foreign language? Assume that the google translation is not making sense and this is your first project and you don't have a budget to hire a professional translator? This sounds like something you would need to get working before you could actually give an estimate.

Why do experienced engineers ask for estimates anyway? Every time I give one I feel like I am lying and I try to warn people "I really don't know how to come up with estimates but I am guessing 3 hours". How do I deal with it when they are upset about it taking a week and a half?

<comment><USER21> For a lot of these, rather than trying to estimate the thing you're asked to estimate, you need to either estimate how long it'll take you to do the analysis you need in order to come up with an estimate, or you need to provide a time-limit on the task after which you'll provide a status update and have a discussion about the next step.

Managers don't really care about time, they care about budgets and risks. In your debugging example, the existence of the bug is a big unknown risk, and your manager needs to manage that. You obviously can't tell how long it'll take to fix the bug, but you can say "Give me an hour to debug it, and after that I should have a better idea of what the cause is and how long it'll take to fix. I might even have it fixed already by that point." That gives your manager a budget (one hour) and risk control (more information in a fixed time), and the option after a small investment to bring someone else in to assist.

<comment> work-on-828 It sounds like you are saying that when I take an hour to work on debugging something, I should spend 55 minutes taking notes on the system and where the problem might be and 5 minutes turning those notes into a coherent email. Is that a reasonable approach?

I certainly prefer to think of debugging as a process of learning about a system rather that trying to try things at random without making a mental map of where I am.

<comment><USER21> I'm saying that when asked for a debugging estimate, you should commit to spending no more than an hour (or other amount, depending on the problem) before stopping to report on your progress. How you spend that hour depends on the circumstances; if you could predict that then you probably already know enough to give a somewhat accurate estimate.

When you stop to report on your progress, you don't need to deliver detailed notes. It could be as simple as "yeah, fixed it" or "figured out the problem, I need three hours to refactor the code to resolve it" or "based on what I'm seeing, Bob would be better at fixing this; I can spend a few minutes to show him how to reproduce it."

"try things at random" is not debugging, so you're right about the process.

<comment> work-on-828 usually when I debug things my method is sort of a mix of linear and binary search where I start at one place, see if the a value is what I expect it to be, then move along to a different part of the code to see if something is wrong there, reducing the search space and repeating as necessary.

Sometimes my method is instead to look at data output at a certain level of abstraction like system calls (which is usually not helpful except for detecting a network hang or a path config whose directory doesn't exist) or database queries or something. That is usually more helpful if I either already suspect I know where the problem is and need to know what model name or file name to `ag` for.

If (Hephaestus be praised) there is actually an automated test framework in place, I'll start by writing a failing test or copy-pasting an existing test and modifying it to fail, then (if it isn't JavaScript), inserting a debugger into a function the test calls.

All of this tends to be pretty successful.

But I've occasionally found myself on a project where I just have a sense that I don't know what is going on or how pieces fit together and I feel a strong temptation to stop and fix that sense of bewilderment. When I resist this urge is when it feels like I am trying things at random.

</comment></comment></comment></comment></comment><comment><USER22> A pet peeve of mine:

Scenario: Finished requirements, analysis, design; completed development and testing; in UAT; Production deployment: 1 week away.

Customer: Hey this field name does not exactly represent what we want to convey. Can we change its name quickly before we go live?

Eager Novice (who is unfortunately in front of the customer at that moment): Oh yeah, that's just a small change in the Customer model. I think we an get this ready and unit tested in 2 hours.

Customer: Great. Lets git er done!

Project Manager (wary of customer and afraid to rock the boat on an already shaky project): Sure we can do that.

Grizzled Veteran/Grey Beard (who for some reason wasn't consulted on the change request): Hmmm....have we done an impact analysis of the change? What are the downstream impacts on code that references this field? What do we need to do to propagate the change to the db? What about the seven or so Web Services that are impacted by this field change? How long does it take to do a regression test? etc...

Net effect: 2 Hrs initial estimate given to customer; 4 Days worth of work in actuality; Production deployment moved down a week..

<comment><USER23> Agree 100%. Golden rule: Never give off the cuff estimates (even if you're sure).

</comment></comment><comment><USER24> Regardless of project/planning methodology, having a ever-growing checklist of less-obvious estimation items is a simple way to avoid forgetting to consider them. The book "Software Estimation: Demystifying the Black Art" [1] by Steve McConnell contains a number of these.

Some examples:

      - Team or company meetings
      - Team member sick days, holidays, vacations
      - Bugs in third party software
      - Maintenance of previous systems
      - Maintenance of build automation
      - <USER2>
      - <USER3>
      - ...

... or your own list of whatever other items you have historically forgotten, or tend to forget when estimating.

In the painting example, the moving of the furniture was something I overlooked, despite having made the effort to carefully think of all tasks. A estimation checklist built up from previous painting projects could have triggered me to consider this task.

[1] [http://www.amazon.com/Software-Estimation-Demystifying- Devel...](http://www.amazon.com/Software-Estimation-Demystifying-Developer- Practices/dp/0735605351)

</comment><comment><USER25> I'm about to being a rant here because this is a larger problem than anybody likes to admit.

Developers are good at development. By and large, they are not good at task management, project management, and certainly not estimation. One of the primary reasons for this is that their plates are always overflowing and they work more efficiently than many people in other professions. They shift priorities on the fly when work in one area cannot be progressed. Development time is social currency. You can make your boss happy by progressing his priority, or you can make someone else happy by progressing their priorities. Not many developers truly answer to a singular master. And few allocate time for learning although we all spend a significant amount of time doing it - <USER4> else do we maintain relevant skill sets?

Can a developer really estimate a project and tell his manager that 3 weeks in the timeline are for 2 unknown and unforeseen problems that require interaction with vendor support? No, because the manager will fight back and say that those 3 weeks are really just zipping up logs and making three phone calls and "if it takes anything more than that, hand it over to me!" Most can't tell their manager that it will take 2 weeks to parse a CSV because the source data is shit and they have too many other things they are working on. So they stay an extra 6 hours in the office and get called a rock-star until the day that they opt to go to their child's school play instead.

The solution to this problem is simple. Take task management and project estimation out of the hands of developers. It's time for the return of the secretary to the workplace. A single $50-80k administrative assistant could easily support, manage the priorities of, and improve the outward facing communication for five developers. With experience doing so, that admin could provide better time estimates for all of them and would be better at negotiating realistic timelines than any of them individually.

People have been coming up with methodologies and software to solve this problem for decades. Developers don't have brains built for project management. Yes, they _can_ do it, but they don't get paid to work in MS Project. They get paid because they know the difference between a bubble sort and a quick sort and when to use either. Conversely, I don't know a soccer mom who doesn't successfully manage a more challenging schedule and competing set of priorities better than every nose-down developer I've ever met. Give appropriate work to appropriate people. Developers have their talents. Scheduling, estimating, and task management are not among them.

<comment> squeaky-clean Going to join you in this rant if you don't mind.

> Most can't tell their manager that it will take 2 weeks to parse a CSV > because the source data is shit and they have too many other things they are > working on. So they stay an extra 6 hours in the office and get called a > rock-star until the day that they opt to go to their child's school play > instead.

Ugh! For real! I had this happen to me recently. I have trouble telling people "No", and pushy PMs who also happen to be my friends means that I often bite off more than I can chew, and have to either "be a rockstar" (which you start to realize isn't such a compliment), or deliver late / fail to deliver.

So I just started being a dick about it, and it works for me. If I say no too politely, it's really easy to convince me to change to a yes. My turning point was when discussing a project that was much more complex than management assumed it would be (I'm compelled to link the relevant xkcd [0]), and I already felt overwhelmed on my current projects.

The discussion was something like "Ok, how about by this date? What about by this date? What if we dropped these features?", I thought for a moment and responded "Ok, look. I can promise you any date you want, and we can shake hands and leave this meeting. But I'm telling you the reality is I can't do it, and it won't be done. If you'd like to take some of my other projects and move them into the _will not be done_ category, we can talk. I'm only saying this because I don't want to agree to a deadline and then be constantly thinking 'Oh well, that's the date I'm getting fired.'"

I'm very lucky that everyone here is really nice, and realized how stressed I must have been to react like that. So we've reprioritized some things and are beginning to outsource some of my more menial tasks that took up a lot of time.

Didn't really intend for that story to have a point, just venting something my non-dev friends don't relate to. I guess the point could be that the problem isn't just caused by bad managers, or bad dev estimates, or bad process, it's a little bit of everything, and the ratios are different for every company or person.

[0] [http://xkcd.com/1425/](http://xkcd.com/1425/)

<comment><USER26> Personal experience here - everything got better when I really started being a dick. I flat out tell them that they can't change reality and things take time; and I will not work extra hours or during the weekend to keep promises I didn't make. I had the same feeling of "when are they going to fire me" but actually it looks I am getting much more respect.

<comment> squeaky-clean Exactly the same here. I think it's not really "being a dick", but just a dissonance between how the different types of people operate. I'm usually so passive that any sort "No, I want my free time after work" feels rude, and I'm probably not very good at being assertive and nice at the same time yet, haha.

The other developers always seemed to get the message that when I said "Maybe... I'm not sure if I have the time for it... I can see...", I really meant "No, but please don't make me say no.", while anyone else hears it as "Cool, he said he would see if he has the time for it" and then follow up with me a few days later "Did you make time for that project?"

I'm making an effort to flat out say "No" more often, but it's nice that other people are also beginning to realize that I only mean "Yes" when I actually use the word "yes".

</comment></comment></comment></comment><comment><USER27> I think also one of the biggest mistakes is making over estimates, or playing too safe. Especially when working with big teams and multiple projects (with a lot of dependencies).

<comment><USER28> Why - has that led to a spate of projects completed early?

<comment><USER29> It leads to what I call "do it right syndrome".

Since I have time, I'll do it right, with a factory class, and extendible configuration package (that I'll write myself so it will be just perfect) and, for optimal speed, I'll use Red-Black trees that I'll have to write my own implementation of since there isn't a standard library ...

<comment><USER30> What's wrong with separation of concerns, extendability, optimal speed, etc.?

More often than not, I observed doing it right the first time is actually a huge time saver compared to hacking some stupid prototype nobody can understand later.

<comment><USER31> Agreed, except in cases where the code will never be maintained again (e.g. game releases on N64). In my experience an "over-estimate" that leads to the so called "do it right" syndrome happens if _someone else_ estimates for a developer. When the developer makes the estimate him/herself and doubles it, you can be damn sure they are going to need all that time and then some to even get it in the ballpark of long-term acceptable.

(That being said, the company needs to have standards for code acceptability in the first place, which doesn't always happen)

</comment></comment></comment></comment></comment><comment><USER32> The tendency to estimate poorly - particularly to underestimate - is one of the key reasons to use a very tight, short-iteration agile process.

If I estimate something will take two days, and I'm off by 50%, it'll take three days. If I estimate it'll take two years, and I'm off by 50%, it'll take three years. And because I can get a lot more practice at two day schedules than two year schedules, I'll improve my accuracy over time, through steady practice.

This takes us back to the triangle of constraints that Kent Beck talked about in the first Extreme Programming book. There are three constraints on projects - schedule, scope, and resources. You can control _two_ of them. Generally, resources are inflexible. So in practice, projects are either schedule-bound (we must hit this date) or scope-bound (we must have these features). A project that tries to do both, without a significant resource surplus, is probably doomed.

Since estimation improves over time with short iterations, it seems better to me to be scope-constrained than schedule-constrained. With steady progress, you can achieve the goals eventually. Good agile processes are much better at saying where you are and where you're going than when you'll get there.

<comment><USER33> I've never experienced estimation improving over time in an Agile process. In fact, since short estimation cycles are just as susceptible to political games as longer cycles, and this dominates the whole process in any organization, the length of the estimation cycle really has nothing to do with the estimation issue.

But, what shorter estimation cycles are responsible for is a decoupling of the overall big picture progress from the immediate tasks being prioritized. This often leads to sprints that are well scoped, and everything in the sprint gets completed close to on time (at least as close as in longer work cycles, but not usually any closer), but, _crucially_ all of the work has to be scrapped because the whole sprint, _conceptually_ , was not right. In my experience, this happens maybe 1/5 of the time in Agile exactly in ways that would be prevented with longer-term planning.

It's very similar to the classical fractal effects of measuring the coast of Britain. By refining your ruler, you merely _think_ you're being more accurate, and a lot of people are doing a lot of performative managerial crap for the sake of the Scrum performance, like a ritual. It _feels_ like you are estimating better, but really because the whole meta-level _concept_ you are working on isn't guided by longer-term thinking, you end up having so much round off error, added up over more and more cycles, that the overall waste is colossally greater than systems that involve some more significant longer-term pre-planning and which work on variable cycle lengths instead of a cookie- cutter, one-size-fits-all approach (which is the core philosophy of Agile, though people try to obfuscate this fact by No True Scotsman-ing it with the Agile Principles).

I also can't miss the opportunity to point out this great article on the political games we play during estimation: < [http://research.cs.queensu.ca/home/ahmed/home/teaching/CISC3...](http://research.cs.queensu.ca/home/ahmed/home/teaching/CISC322/F09/files/EstimatingGames.pdf) >

<comment><USER32> All excellent points. But having done big waterfall, bad corporate agile, and good agile, the problem of work failing conceptually is common to all of them. The question is, how do you recover from it? I think good agile recovers more gracefully from conceptual failures than either waterfall (which tends to throw good money after bad, because admitting failure is not an option), or corporate agile (because corporate agile is usually just buzzword-compliant waterfall).

<comment><USER33> I'm not saying waterfall is a good alternative. Waterfall is just as shitty as Agile, but one benefit is that wastes less of everyone's time with meetings and busywork. But waterfall is equally guilty of a one-size-fits-all mentality, from the opposite view of Agile, and this cookie-cutter property is what makes both methods attractive to bureaucracies.

I'm a fan of common sense. If it's clear in a given team and project context that two week sprints, story points, and frequent meetings are really going to help, then just do it. And when that stops working, just switch and do something else. When architects or researchers are telling you a given project direction needs to slow down for some more intensive piloting research, do it and don't hesitate to violate sacred Scrum principles.

This stuff should be figured out organically, based on the given project and given personnel. Never with a fixed mandate to a single prescribed method or time frame.

<comment><USER32> Continuous improvement of the process itself is part of any good agile process. Corporate buzzword compliance agile is often guilty of not doing that - the "cookie-cutter property".

But importantly, I've found that a refined agile process saves more work than it costs, by figuring out what doesn't need done before doing it, and prioritizing immediate value over future value. It's very important to be able to say "This is good enough for now, but we know it's not good enough for the future". It keeps the perfect from being the enemy of the good. Likewise, it's important to be able to say "Well, I guess that didn't work", and have it be an integral part of the process. Without process, you have no guidance over what wrong turns you took and what those wrong turns cost.

<comment><USER33> I dunno, what you say just reads like a bunch of buzzwords that Agile, even supposedly "refined" or "good" versions of Agile, never prioritizes or delivers. More often, when quality work is produced it's produced _in spite of_ Agile, not because of it, which is a shame because then people falsely attribute that success back to Agile.

I admit though that it's hard to know exactly what you mean just exchanging textual comments like this, so I'm happy to leave it at that.

</comment></comment></comment></comment></comment></comment><comment><USER34> One of the fundamental issues is that upper management sees estimates as bell curves as though 50% of the time a project may finish early. In reality, very few projects on time, let alone early. No one wants to hear about reducing scope but at the same time an MVP that solves the customer problem without feature creep should be evangelized within organizations. In reality, schedule follow more of an exponential function where slippage time goes from bad to worse very quickly as resources/scope/schedule snowball very quickly. Things get scrapped before we see this in practice.

</comment><comment><USER35> I may be cynical at this point but overeager project scheduling/estimation processes and top-down software processes in general are really where software development as a creative and experimental occupation dies.

<comment><USER36> If you can do "creative software development" and get paid for it, count your blessings. And better yet, give a very big "Thank you!" to your manager.

The reality of this trade, regardless of you doing in-house work or a product for the market, is that everything is required for yesterday. There is an "economy of pressure" where every stakeholder pushes everyone else as much as they can in the expectation that they will be pushed as well. If you do not get to experience this, it means someone in your chain of command is very good at pushing others, and that some other part of the overall system is shouldering most of that pressure all by themselves.

<comment><USER35> So if I understand you correctly you're saying that "creative software development" is inherently incompatible with the hierarchical realities of the corporation?

If so, I think you're probably right.

"The reality of this trade [..] is that everything is required for yesterday"

But it just isn't. This is an artificially created pressure for selfish reasons, and should not just be accepted in passing.

<comment><USER37> The game is rigged. Only way to win is not to play.

<comment><USER31> Or according to the parents of this reply, to play under the wing of an expert player.

</comment></comment></comment></comment></comment><comment><USER38> article assumes specs are known. they might be for some kind of brochureware but for any serious work they never are, and in business reqs change day to day. 60% of software features are never used - so that's 60% overspend right there. there's no reason not to do fixed cost agile (personally wouldn't do it any other way). trying to fix scope otoh is just bad business from the get go - a) you might have target outcomes but you don't know exactly what you're going to need to get there b) useful software tends to be ongoing in terms of reqs

</comment><comment><USER39> You could just use data, machine learning and optimal control to make better estimates. It has aleady worked for fighter jets, self driving cars, and automated factories. With enough data it will work for anything.

<comment><USER40> What kind of features would we use? Which prior projects will help predict how long it takes a team of 3 developers (one of them senior with a history of underestimating time, two junior with little track history) to produce a novel photo sharing app where the requirements might or might not change every day?

The problem I see is that there isn't enough data, and the problem space (the number of possible projects and teams) is too large to draw meaningfully on past data.

Unless we're looking to predict specific CRUD apps where the requirements are known up front and a team cranks these out consistently.

Humans have already been flying fighter jets, driving cars, and doing work in factories, so we know it's a learnable skill. We have not yet learned how to estimate arbitrary software production.

</comment></comment><comment><USER41> As long as sales, or its local equivalent, commits to as yet non-existing products and features, it will ever be thus.

</comment><comment><USER42> > The fact that I technically only asked how long the fix would take is > something only an engineer would bother pointing out. -_-

Well excuse me for not fitting your prejudiced profile for a neurotypical extroverted 20-something white male. Yes, I might have needed the clarification between "how long will it take you to paint the room" vs "how long until the room is back the way it was with the walls in a different colour", but you help noone when you talk down to the painter like that. You asked a question and I misunderstood, because we use slightly different language and I have a ton of other things to think about all the time.

Can we stop this "oh let's treat engineers like they're children with mental development issues" already? I'm not some god in an ivory tower, I don't want you to kneel before me, I just want the usual everyday respect you afford to your peers and to engage with me as one professional with another.

This kind of treatment is not ok.

<comment><USER29> Agreed. As an engineer, I can also claim that if I say "we can deliver that March 15th", then I get a argument about why can't it go out with the Feb 28th release and why is it going to take 30 days to do something simple and why ... all because marketing people can't understand delivery cycles.

So, let's just not have that type of conversation at all.

<comment><USER43> Ever talk to a contractor about a home renovation, or even a painter as in this story? If you start questioning why something will take so long, in my experience the usual answer is something like "if you think you know better than me, do it yourself." Especially if they are good at what they do. Most good tradespeople have more work than they can do, and will just walk if they think a customer is going to be a pain in the ass. How often will developers take that approach?

<comment><USER36> Well tradespeople usually work for multiple clients on smallish projects. For _good_ tradespeople, a significant number of those projects come from repeat customers, which means their optimal strategy is not to serve all customers equally, but to prioritize those that are more likely to bring repeat business down the road. A difficult customer is usually not someone you want to do repeat business with, so they get "managed out" pretty reliably.

Most software engineers, on the other hand, work for one big customer at a time (your employer). And though I agree that a bad employer at the end of day is detrimental to your career, you cannot afford to switch jobs on trivial matters, on risk of getting labeled as a grasshopper. That's why you do not see developers "walking".

</comment></comment></comment></comment><comment><USER44> > These are the skills that elevate someone from a good programmer to a great > software engineer

I stopped reading at engineer

<comment><USER45> You really shouldn't.

While the term engineer is loaded with the licensing and professional aspects, and understandably so, the people working with software very often do actually have to perform the task of engineering solutions.

Dismissing a valid dialog on this basis does no one any real good.

Despite the issue you raise, the query and discussion remain valid and high value.

</comment></comment><comment><USER46> _In tech, we spend little time talking about the softer skills like communication, project management, and prioritization._

Well, that's not a good start. Maybe my career has been abnormal, but I've spent a lot of time thinking and talking about project management skills, particularly estimation. It's something my employer expects all non-junior employees to do - more than half my team are certified in scrum or have taken courses in other PM methodologies.

For most of what we do, the hard part is project management. Sure, the techie code stuff is usually more fun, but that's not even half of my daily workload (and hasn't been since well before my title changed from "product architect" to "project manager").</comment>


Rudy Giuliani Says Twitter Sabotaged His Tweet. Actually, He Did It Himself - <AUTHOR> https://www.nytimes.com/2018/12/05/us/politics/rudy-giuliani-twitter-links.html <comment><AUTHOR> "FAIRNESS PLEASE", demands Giuliani after falsely accusing Twitter of conspiracy to alter his tweet which simply contained typo which was automatically turned into a link. Classic.

"Twitter allowed someone to invade my text with a disgusting anti-President message. The same thing-period no space-occurred later and it didn't happen. Don't tell me they are not committed cardcarrying anti-Trumpers. Time Magazine also may fit that description. FAIRNESS PLEASE — Rudy Giuliani (@RudyGiuliani) December 5, 2018"</comment>


Co-founder holds source code hostage - big legal fight over copyright ownership - <AUTHOR> http://hoviblog.blogspot.com/2010/05/tech-startup-nightmare-on-my-street-how.html

<comment><USER2> I can't tell whether this article wants me to sympathize with JustMed, talking as it does about the "bitter irony" of their contractor/employee's standing to sue them for failing to honor employment obligations. These people brought on an engineer, who was apparently solely responsible for developing the core IP of the company, paying him for years only in stock. When they eventually deigned to pay him, they didn't withhold taxes.

The word for someone who performs a core company role in exchange for nothing but equity is "cofounder".

The word for someone who is paid to do work but who is responsible for all their own taxes is "contractor".

It freaks me out that the Appeals court decided that these factors weren't "dispositive" of the engineer's relationship with the company, because (pp) "that's how startups work". The hell it is. If you want people to work for you, you get an employment contract together. If they're employees, you pay them, and you do W2 withholding.

<comment><USER3> _The hell it is. If you want people to work for you, you get an employment contract together. If they're employees, you pay them, and you do W2 withholding._

The engineer in question was married to the founder's sister, and she died. Obviously there was a lot of personal relationship intertwined into the business relationship. In addition, there's a traumatic event in the lives of both the founder and the engineer. This probably meant there was no real contract, thus judges could swing in favor of either one of them. Your comment makes it seem like this is a typical business vs. the contractor case, which it isn't.

<comment><USER4> The IRS doesn't care if there's an agreement, when it comes to determining employee vs contractor. The rules are very clear:

[http://www.irs.gov/businesses/small/article/0,,id=99921,00.h...](http://www.irs.gov/businesses/small/article/0,,id=99921,00.html)

<comment><USER5> Well, they are spelled out, but there is still a lot of unclarity in the execution of these rules. Enough so that some large consumers of contractors make everyone go through another consulting organization.

As an example, the phrase _Does the company control or have the right to control what the worker does and how the worker does his or her job?_ seems a bit open to interpretation. Lets say you are a contractor working on a BFE Java or .net application. The project has its source control rules, its status reporting rules, its code convention rules, and all are likely under the control of an employee of the company in question. Sounds like the company has the right to control in this very common case.

So I argue that the rules may be extensive, but they are not very clear.

</comment></comment></comment></comment><comment><USER6> The company is lucky they aren't located in California. Here, even if you are identified as an employee, you may retain ownership over works you have created unless you have signed a "work for hire" agreement or something similar.

<comment><USER2> This is a great point and is the reason why most established companies have a _standard contractor agreement_ that they _never change_ that has a work-for- hire clause, along with the NDA and IP language, and a formal statement of work that specifies payment schedule and acceptance criteria.</comment></comment>


Instagram may have lost nearly 25% of its users - <AUTHOR> http://www.nypost.com/p/news/business/rage_against_Dh05rPifiXBIJRE1rCOyML

<comment><USER2> daily usage is not the same as user counts

<comment><USER3> Not to mention the NY Post is not the same thing as credible, non sensational tech journalism.

<comment><USER4> If it makes the beloved unviable overhyped startup look bad, it must be because of "sensationalism", "bad journalism", or maybe the numbers are just down because of "the holidays". Or maybe if the article is accurate about the drop in daily users, maybe, just maybe, the user count is still the same?? Sometimes it's difficult to accept the truth.

<comment><USER3> Actually... I think it's because I'm an ex tech journo and am jaded from seeing how reporting like this happens. Not because I particularly care for instagram.</comment></comment></comment></comment>


Linux and programming languages(Java, PHP, C++, SQL, ASP..) quizzes and question - <AUTHOR> http://xquizzes.com

<comment><USER2> Buggy quizzes and not accurate... :'(</comment>


#startups - <AUTHOR> http://irc.lc/freenode/startups/

<comment><AUTHOR> Everybody interested in startups should know this awesome irc channel.

If you are new to irc, here is how you join. First register a nickname by typing

        /msg nickserv register <your_password> <your_email>

Open the email and activate you account. Then type:

        /msg nickserv identify <your_password>

Voila! Welcome to #startups!</comment>


Quantum computer built inside a diamond - <AUTHOR> http://www.sciencedaily.com/releases/2012/04/120404161943.htm?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+sciencedaily+%28ScienceDaily%3A+Latest+Science+News%29 <comment><USER2> Can someone more inclined elaborate a bit on this please?

"As opposed to traditional computer bits, which can encode distinctly either a one or a zero, qubits can encode a one and a zero at the same time. This property, called superposition, along with the ability of quantum states to "tunnel" through energy barriers, will some day allow quantum computers to perform optimization calculations much faster than traditional computers."

I get it, a qubit can store multiple states, but if someone could give a little more on how it does that, and why that means quantum computers will be faster someday, that'd be excellent.

<comment><USER3> I wrote a paper in highschool so take this with a grain of salt basically the idea is have truly parallel programming instead of having concurrent ones the ability of the qubit to solve for multiple states means they can solve for multiple problems at the same time while conventional computing seems parallel whats really happening is that calculation are switching states so fast they feel parallel. If I am wrong someone correct me.

<comment><USER4> Yeah, that's completely off (it would be a more apt description of the distinction between multiple cores and hyperthreading).

The important characteristic of quantum computation is not that it's faster or more parallel than classical computation, but that it's a _generalisation_ of classical computation. The same way that classical mechanics describes a subset of quantum mechanics rather than disjoint phenomena, all current classical CPUs are technically "quantum" CPUs (the best kind of quantum CPUs!).

So, what does this mean?

Well, for all the software we have today, and for almost everything we know how to code today, absolutely nothing; in fact, most of what we can do on classical computers will be slower simply because it will be _quite_ some time before a quantum computer is anywhere near as well-engineered as, say, a Core i7 – especially when you take into account that the properties of quantum mechanics make this engineering not only different but also significantly more difficult.

What we do get from the computational paradigm shift, however, is a broader set of operations which allow for a broader set of algorithms.

Take Grover's algorithm (the one mentioned in the article): [http://en.wikipedia.org/wiki/Grovers_algorithm#Algorithm_ste...](http://en.wikipedia.org/wiki/Grovers_algorithm#Algorithm_steps)

If you understand the notation, what you'll see is a series of linear operations (quantum gates) being applied to vectors (qubits (classical bits are scalars)) in a quantum circuit. These can all be used to describe any operation which would be performed on, say, an x86 core (albeit more verbosely), but can also take advantage of quantum mechanical properties to manipulate non-classical information (which gets complicated to discuss).

For the canonical example of how this can significantly change computation as we know it, see Shor's algorithm, which speeds up prime number factorisation from sub-exponential time to polynomial time, and thus could potentially break RSA encryption if we had a practical quantum computer): <http://en.wikipedia.org/wiki/Shors_algorithm>

Aside from that, I'm sure people much smarter than I am could go on in technical depth about how, despite not _really_ being an advancement of mythical proportions in and of itself, this new class of algorithms would be able to revolutionise areas like search, machine learning, physical/biological/chemical simulation, and so on.

</comment></comment></comment><comment><USER5> I should point out that diamonds can also make very good semiconductors when properly doped, with very large carrier mobility compared to silicon. We just don't know how to make the necessarily complex patters on them.

<comment><USER6> > _I should point out that diamonds can also make very good semiconductors > when properly doped, with very large carrier mobility compared to silicon. > We just don't know how to make the necessarily complex patters on them._

Don't know what you mean by "properly doped": The biggest problem with diamond is that it is a bitch to make n-type. In fact, the conduction band is above the vacuum level!

It is why people try to use diamond as a thermionic emitter: electrons in the conduction band naturally try to leave the material.

<comment><USER7> Doping is the intentional adding of impurities.

Maybe phosphorous could be added among the carbon molecules in the diamond, similar to how silicon is doped with phosphorous to make a better semiconductor.

Maybe nitrogen or oxygen would be a better choice, for the similarity in molecule size - they wouldn't disrupt the diamond's carbon lattice structure as much.

<http://en.wikipedia.org/wiki/N-type_semiconductor>

<comment><USER6> You realize that pretty much every group V dopant has been already measured? The problem is that n-type dopants are deep in energy, far away from the conduction band. So the electron from the donor stays with the dopant atom rather than activating and putting its electron in the conduction band.

> _Maybe nitrogen or oxygen would be a better choice, for the similarity in > molecule size_ [Typo: Dopants are atoms not molecules.]

Many Group V atoms have been tried. See eg <http://phycomp.technion.ac.il/~david/thesis/node17.html></comment></comment></comment></comment>