Why I left Heroku, and notes on my new AWS setup - <AUTHOR> http://www.holovaty.com/writing/aws-notes/ <comment><USER2> We're working on solving these sorts of devops problems in Ubuntu with Juju: <https://juju.ubuntu.com/>

As an alternative approach would be that a Juju charm (script) would handle the initial deployment of a stock Ubuntu AMI and the customization in one step (or with puppet/chef) and then allow you to add new instances based on scale (though currently not automatic). When you have changes to your service you update the charm and just `juju upgrade-charm`.

This would consolidate step 1, 5, and the "ongoing" into one tool and you'd get a cloud-agnostic deployment (openstack cloud or bare-metal).

<comment><USER3> FWIW, I tested juju a few months ago and found it to be buggy and unreliable. Sometimes the instances would connect together correctly, and sometimes they would fail inexplicably. Didn't seem ready for any kind of production use to replace config mgmt tools.

<comment><USER4> If you are looking for another general purpose orchestrator, I'd suggest taking a look at Ansible - <http://ansible.cc/>

Disclaimer: I am the primary author.

<comment><USER5> I have tried ansible and love it. I invested the time (a few hours) to get a script working for my stack, and now I have a 70ish line script that will provision a server or VM (with a shared code directory in the VM), from clean installation to the codebase up and running in its production state, in one command.

New project? I just copy the script, change a few variables (names and packages it needs), and I get deployment for free.

<comment><USER3> Sounds interesting. Any possibility that you could anonymize the script and paste a link to a gist of it? would be interesting to see some real-world Ansible examples.

<comment><USER5> Sure: <http://pastebin.com/KPrm3Zky>

It could use variables a bit more, I think, but there were a few bugs with expanding them, so I didn't use them. I'll fix them later on, though.

<comment><USER3> amazing. that example explains how it works better than 200 pages of documentation :-)

thx!

<comment><USER5> You're welcome, examples are pretty useful for this sort of thing. Plus, after you have a base to work off, it's trivial to extend.

I'll put this up in a post on my blog, maybe it'll help others as well.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> Yet another post about scalability/architecture that goes down under load. There should be some rule for that.

<http://i.imgur.com/UEHzaOL.png>

EDIT: Back online, the site migrated to AWS is _not_ the blog.

<comment><AUTHOR> Heh -- my blog is still hosted on Heroku, and I forgot to up the dynos. Which I suppose is a nice (and ironic) argument for doing the AWS auto-scaling stuff!

The site I was writing about, soundslice.com, has not seen any blips.

<comment><USER7> There's no need to increase dynos yourself. Just use the [Adept Scale](<https://www.adeptscale.com/>) add-on.

</comment></comment></comment><comment><USER8> _> The way we set up Soundslice is relatively simple. We made a custom AMI with our code/dependencies, then set up an Elastic Load Balancer with auto- scaling rules that instantiate app servers from that AMI based on load._

Doesn't sound that simple to me (as a complete sysadmin noob). Somebody should write a book about this.

<comment><USER9> Would you be interested in a blog post about it? I'm a sysadmin/devops with 12 years in, and build stuff like this on a daily basis. I wouldn't mind sharing how the sausage is made.

<comment><USER10> Last week, I literally spun down an EC2 instance and signed up for Digital Ocean because I couldn't figure out how the hell to make stuff work on EC2, but have lots of VPS experience. As a dev and not a sysadmin, it's much easier to go with what you know... but I want to learn.

<comment><USER11> What's the difference between EC2 and a VPS for you? Do your VPSs already have things installed or a GUI? I've used EC2 before as a single server, never scaling. The main difference was installing things that are usually pre- installed (like on Ubuntu's official desktop image). Is that it or is it more about the scaling?

And thanks for the reference to Digital Ocean. Never heard of them before. Seems great, might try using them :)

<comment><USER10> I have no idea, just things don't work on EC2.

> Do your VPSs already have things installed or a GUI?

Nope. I prefer straight-up Arch linux. ssh in and go from there.

To add some concrete-ness to the mix, I was installing ejabberd. When it came time to ping the server... no response. I did the exact same steps on my Digital Ocean VPS and everything went fine. I had done whatever commands EC2 expects to open the right ports...

<comment><USER12> For EC2 security groups you have to open access to both the correct ports _and_ protocols. Ports are a concept at layer 4 of the OSI model, while ping, or more correctly 'ICMP Echo Request', is lower down the TCP/IP stack at layer 3. So when configuring the security group, look for the option to choose Protocols, then enable ICMP :)

<comment><USER10> Thanks. This is exactly the kind of stuff that I could learn from said blog post, and why I think "EC2 == VPS" is false.

The documentation I was following made no mention of this at all.

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> This really resonates with me. I have been having identical issues with Dotcloud, a Heroku competitor.

There's nothing worse than wondering if deploying your stupid one-character typo fix will hang or leave the app in an incomplete state.

I myself am working on setting up salt stack for deployments. I like that all of it is in python, and after a couple of days I've begun to make progress.

I am migrating things slowly: first cron jobs, then internal tools, and at some point, our website. The only thing that really needs "scale" is our frontend, and I don't yet know how I'll manage that, but you know, I guess it is time to learn!

<comment><USER14> Huge fan of salt stack, and I may push Adrian in that direction. The changes we made here were minimal in the way he deploys, and with so many architecture changes behind the scenes, we decided less is more on that front.

<comment><USER15> any link on the advantage of using saltstack for aws instead of AMIs ?

<comment><USER16> Salt is more easily repeatable across heterogeneous environments.

For example, we're using Salt to manage our development, testing, staging and production environments. We couldn't do this with AMIs, since the environments are quite different.

It makes updating quite easy, as you don't need to create a new AMI, just push your update to your release branch, then run state.highstate from your Master.

If you configuration changes, you update the master and then you can push that change across all (or just one) of your environments.

</comment></comment></comment></comment><comment><USER17> On my end I get a complete crash once a week when running with one dyno. The official response for support I got was to upgrade to two dynos.

Apart from the PostgreSQL hosting, I'm not really happy with Heroku. Add to the random deploy breakage I've experienced mid-launch (not to mention it's slow as hell), I'm going to move back to my own servers.

<comment><USER18> Heroku docs do say (and I'm afraid the link to it is escaping me) that if you run only 1 dyno, it will shut down after something like 6 hours of inactivity. So if no one hits your site for 6 hours, the next time someone does it has to boot the dyno up again, which can take enough time for the request to time out.

Not saying this is a great policy, just explaining in case you weren't sure. When you go to 2 dynos they keep things up for you presumably because now you're a paying customer.

<comment><USER19> This can be mitigated by having cron hit your site every few hours. There are some free cron services like <https://www.setcronjob.com/> that do this.

<comment><USER20> Welcome to the cloud, where you need to rely on a third party service to keep your other third party service online. Of course, what happens when setcronjob.com goes down.... well you then use pingdom to check that it is up...... down the rabbit hole we go!

<comment><USER21> Welcome to the web, where you get an easy hosting service like Heroku without paying a dime and then complain about how you need to put some effort to hack the system so you can continue to pay nothing.

<comment><USER17> Just to be clear: I am paying $70/mo for SSL and Postgres, so it's not that I'm paying nothing. However, based on the minimal server requirements for my app (most users are using our client-side Javascript tools), one dyno "should" be enough.

</comment></comment></comment></comment></comment></comment><comment><USER22> If you think puppet/chef is too much complexity you might find saltstack worth some attention. I haven't used it in real anger yet, but I've really liked what I've seen so far. I see they've even got instructions specific to aws now too [1]

Also worth mentioning cloud formation as well [2]. That might make the pain of chef/puppet more of a worthwhile investment!

[1] <https://salt-cloud.readthedocs.org/en/latest/topics/aws.html>

[2] <http://aws.amazon.com/cloudformation/>

<comment><USER23> Ansible is another alternative that is relatively simple.

<comment><USER24> I've been using Ansible for a couple weeks now and it is awesome how simple it is compared to Chef/Puppet (IMO)

</comment></comment></comment><comment><USER25> Just my 2 cents... the problems described here seem to fall into the category of "things didn't work 100% of the time".

I have some bad news for you; nothing I have ever used worked 100% of the time. Doubly so on AWS. Just off the top of my head; just in the last week we have seen 5% of AWS instance act so badly that we had to recycle them, and that is just the easy to diagnose problems. Don't get me started on the IOPS marketing BS that Amazon sells.

It's just the reality of a lot of moving parts and complex systems. From the sound of it, the OP had very little _actual_ downtime, and had to make some end-runs a couple times. Shrug, just life in a high-scale world IMHO.

The goal should not be to bounce around providers until you find the Garden of Eden. The real goal should be to accept failure and build to tolerate it. Now maybe it is easier to build fault-tolerance if you are closer to the metal... BUT I would bet that Heroku has much better experience and tooling to detect and resist faults then rolling your own.

</comment><comment><USER14> Really loved working on this with you, and I'm excited to see what you come up with now that you have a much better understanding of AWS. Congrats on the re- launch!

<comment><USER26> When engaging a super qualified expert like this, say in devops, what levels of interaction are available? For example, is is possible to get the one hour consult that points the way and mentions the land mines to avoid, with maybe some template files, or is it only the complete package where the expert does everything from soup to nuts? Is it cost effective for an early-stage venture to use the rockstar for something like this? Would someone like this only apply after product-market fit?

<comment><USER14> Well, it depends on the application. Most people think about scaling far too early, and the same applies to configuration management. I am a fan of shortcuts, early and often... worry about things when you have things to worry about.

That being said, making some easy architecture choices early on can have an enormous effect on your sanity. It is probably worth it to bring someone in to hint you in the right direction after you have a prototype to show and have already started to make choices. This is basically where I helped Adrian.

If your app is struggling under success, you are ideally in a better position financially than most startups are at the beginning. This is when you bring someone in for longer term engagements, or potentially when you hire someone to work on this full time.

I think you are hitting on something that is really a need in the market right now. When you don't need a consultant, and you need 30-90 minutes of "how do I do this?" Q&A, it is hard to figure out who to talk to.

From my perspective, it is hard to find those engagements. I do a sad majority of my time in the sales side of things. Recently, I spoke to some folks at 10xmanagement and Adrian pointed me to anyfu, both of which are approaching this problem but in slightly different manners.

Lastly, you don't want the soup to nuts guy. This is your app, and if you aren't deeply involved with what is going on with it, you are putting yourself at a huge disadvantage post consulting engagement.

<comment><USER27> I've often gone on Quora or Stack Exchange to answer questions about scaling or stack design, but the questions (and answers) are often very simplistic, or the questions are so specific that I'd need to spend hours answering.

If there was a place I could go to get some cash for my time answering these questions, I'd certainly be more inclined to spend the time to do so thoroughly.

Mentoring contractor start-up perhaps?

</comment></comment></comment></comment><comment><USER28> From the OP:

> I changed the app to use cookie-based sessions, so that session data is > stored in signed cookies rather than in memcache. This way, the web app > servers don't need to share any state (other than the database). Plus it's > faster for end users because the app doesn't have to hit memcache for > session data.

The switch to using cookies for storing session data instead of memcache has tradeoffs. Sure, you no longer need to ask memcache for the session data. But you are also shipping a significantly larger cookie back and forth _on every request_.

If you're storing a lot of data in your session, this could actually slow things down in the long run. [1]

[1]: [http://yuiblog.com/blog/2007/03/01/performance-research- part...](http://yuiblog.com/blog/2007/03/01/performance-research-part-3/)

<comment><AUTHOR> Definitely a great point. Fortunately my session cookies are _teeny_.

</comment></comment><comment><USER29> Sad to see the recommendation to use MySQL - in my experience RDS hasn't been worth the effort - but I'm probably biased since I already invested the time in automating PostgreSQL replication setup.

<comment><AUTHOR> Believe me, I am sad to stop using Postgres. What parts of RDS haven't been worth the effort? I didn't have to make much effort, beyond rewriting my Postgres triggers into MySQL syntax.

<comment><USER9> RDS is pretty expensive for what you get. You can't restore to a running instance from snapshots, you get very little control over the environment, and you can't replicate between geographic regions (only availability zones).

I know you said you're a two man shop, but in this case it may make more sense to leverage other IaaS DB services instead of RDS.

<comment><USER30> I am not sure what you mean by "You can't restore to a running instance from snapshots" cause, well, I've done it a few times.

Could you expand a bit maybe?

<comment><USER9> My apologies. I should've said "to an existing instance."

[http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_R...](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html)

"You must create a DB snapshot before you can restore a DB instance from one. When you restore the DB instance, you provide the name of the DB snapshot to restore from, and then provide a name for the new DB instance that is created from the restore. You cannot restore from a DB snapshot to an existing DB instance; a new DB instance is created when you restore."

</comment></comment></comment></comment></comment><comment><USER31> Two quick Heroku recommendations that wouldn't fix all (any?) of these issues, but would at least make them more palatable.

1\. Change your default heroku error pages. From this: <http://s3.amazonaws.com/heroku_pages/error.html> and this: <http://s3.amazonaws.com/heroku_pages/maintenance.html> to something else (in the app settings tab).

2\. Keep a staging instance up and running with a cloned database so that you can avoid random heroku-created errors in production.

</comment><comment><USER32> I get a 500 when I go to the site. It seems like a bad omen for an article about your hosting setup.

<comment><AUTHOR> Which site? My blog? The blog is still on Heroku, and it sounds like it was overloaded for a while there. I just upped the number of dynos, so it should be fine.

I don't have any indication that soundslice.com (which is the site I'm hosting on AWS) was returning any 500s, but if it was, please let me know.

<comment><USER33> I am curious how many dynos you end up using under traffic for a blog site. Care to share?

Also, how many dynos did it take to handle the load on soundslice, when it was on the front page of reddit?

<comment><AUTHOR> I don't have a definitive answer for the blog, to be honest. I just add four more dynos (to bring it up to five total) and it seems to be fine. I figure it's only for a few hours, so I can be sloppy about it without needing to worry about huge costs. Wish I could be more scientific about it!

As for the number of dynos when Soundslice was on the Reddit homepage, I _think_ was using 10 at that point, but, again, it was a totally non- scientific thing. 10 did the job, but fewer dynos might have been just as fine.

<comment><USER33> Thank you for responding. That's exactly what I was looking for, a ballpark figure for what can get you through a peak in traffic.

</comment></comment></comment></comment></comment><comment><USER34> AWS is great. We use it a work and I spun up extra server capacity for a once a year open studios event.

That being said as someone who's been in the Unix world for a while figuring out which AWS services to use is not obvious at all. It took me some time to figure out the alphabet soup of sevices and I was familiar from work. I ended up using "cloud formation" which builds you a server (LAMP or other) and optional database/loadbalances configuration. Its was that or selecting a LAMP ami (Amazon machine image). They have a lot of documentation but its hard to get an overview of what everything is (S3, elastic storage....) Plus configuring web server/ database servers for best performance can be non- trivial.

I get better speeds from our organizations cheap "shared hosting", during low loads. I was pretty sure the one week of very high loads would have crushed the shared host, thus AWS was perfect.

</comment><comment><USER35> Switched from Rackspace to AWS - couldn't be happier. I think AMZN is really far ahead of the competition and their pricing seems to be as good if not better for many use cases.

</comment><comment><USER36> The OP says he opted not to use Chef/Puppet and went with baked AMIs instead. In my experience, Chef simply takes too long to build a machine. I use it to ensure a repeatable, self-documented machine setup, but then if I'm on AWS I snapshot an AMI for quick scaling. This also lets you more easily use Amazon auto-scaling tools like Cloud Formation, which bring up new machines based on an AMI. But having the Chef script is still great for knowing what's on your box and having "source code" to change it.

One thing I'm curious about re the OP's process: he says he is using Fabric for deploys. Does that mean every time he deploys new code he has to snapshot a new AMI? In that case, why use Fabric at all? I'd be worried about auto- deloying AMIs with outdated code.

Since EC2 instances are "disposable," one approach is to never "update" an instance, but instead you release new code by simply launching fresh instances with the latest code, then destroying the old ones.

<comment><USER14> The baked AMI comes up and pulls, so it doesn't need to be baked each time.

If you take the route you just mentioned, I highly recommend using Netflix's Asgard (<https://github.com/Netflix/asgard>) and check out their recently released Aminator (<https://github.com/Netflix/aminator>). Asgard specifically makes AMI based deploys outrageously easy.

</comment></comment><comment><USER37> How come nobody thinks about Google's App Engine (and/or Compute Engine) in these situations? Is it really that unbaked?

<comment><USER38> It's too expensive and has dangerous vendor lockin. I myself was burnt 2 years ago and vowed never to return to that platform. Now we use combined Heroku and AWS services. However I do miss GAE though, its deployment is smooth, their auto scaling up is great. It's just that I don't want to be locked in any more.

That being said, we still use GAE as a platform for fast prototypes, as a cron service to keep our Heroku instances up + firing up an EC2 instance every night to run a script for 5 minutes. So we only use GAE when there's minimal risk of being locked in.

</comment></comment><comment><USER39> What's the cost difference between Heroku and AWS for you?

<comment><AUTHOR> Ah, I should have mentioned this in the post. The price is going to end up being basically the same. I could have done it even cheaper, but I'm paying extra for multi-AZ stuff (i.e., making the database and load balancer available in multiple availability zones, for failover protection).

<comment><USER40> You should edit this in at the end.

edit: though you'd have to factor in the cost of bugging Scott when you're not his friend.

</comment></comment></comment><comment><USER41> For discerned users who are ready to spend effort, AWS is definitely the way to go. No restrictions, fire up your virtual machine, setup everything as you like and optimize as you need - you have the control button.

</comment><comment><USER42> This sounds pretty great. I haven't used Heroku, but have used Rackspace, AWS and Linode.

What worries me the most with AWS, is that there's virtually no support. I guess you can pay and get decent support, but on the low-end of the scale, it's pretty thin or non-existent. Now, I don't hear of so many issues with AWS that warrant contacting support about, and I don't face any issues myself, but just the thought of something going strange and having nobody to talk to makes me nervous.

In contrast, Linode does not have so many cool features, but their support is there and very responsive.

<comment><USER19> You can just bump up to AWS's $50 paid support as-needed for one month, if you need to. They've been very responsive to us with the three cases we've opened - roughly an hour to resolution each time.

<comment><USER42> Thanks. That makes more sense, and I guess a reasonable price if you consider you don't need this every month.

According to <https://aws.amazon.com/premiumsupport/> \- the developer support is within local business hours though...

I still wish this could have been factored into your hourly AWS costs rather than as a monthly fee. Say, pay extra 10% for every EC2 instance-hour or something and get it included (without the $100 minimum as in the business support that is).

<comment><USER19> It wouldn't be sustainable. The folks trying to build the next Amazon.com on a t1.micro would be paying a buck a month and expecting the same level of support Netflix gets.

<comment><USER42> Yep, you're right. I guess support is one of those aspects that can't be measured per Gb, so there must be some kind of an entry cost. I do wonder how Linode manages to keep this manageable for them though.

</comment></comment></comment></comment></comment><comment><USER43> Can you share approximately how long the process took you? Thinking of going down that road with 2 small basic heroku rails apps 1 web, 1 worker, 1 small postgres DB. very little traffic. Both apps already at ~$170/month on Heroku. These are paid SaaS B2B apps, but they generate so little traffic but the way heroku partitions their services or addons, this really should cost like $30-40/month on a regular "hosting environment"

I know almost nothing from a unix sysadmin perspective, but would invest the time if it is feasible.

<comment><AUTHOR> It took about four full days of work, spread out over a couple of weeks. It would have taken me much longer had I not been helped by Scott (my friend mentioned in the post).

I don't know the details of your app, but $170/month sounds high for that server arrangement. If you set up a few auto-scaling Amazon micro instances, it would give you more capacity and be cheaper. Of course, the tradeoff is that you'd need to either learn how to do it or hire somebody. :-/ Good luck!

<comment><USER43> 2 apps, same profile SSL $20 postgres basic $9 1 free web dyno $0 1 additional worker $36 Scheduler addon usage: $5

$70 for each app(Its actually ~$140.00) (approximately 100-200 business customers might use the apps daily)

And of course, bad billing practices where they dont stop charging you for addons you have stopped/remove and send invoices 1 month late so by the time you notice, another round of billing as occurred incorrectly. Have resulted in me paying $350 and $251) in the past 2 months.

Exact quote from Heroku customer support email, after overcharge of ~$300 in past 2 months

"I'm sorry that the delay in receiving your invoices caused the charges to continue for longer than you would have liked, however please keep in mind that we offer your Current Usage details[ on your account page]"

Yeah, not happy with heroku at all. Even though i can better use my time generating additional sales for the apps, Im so pissed, I plan to burn time move and cancel ASAP...

</comment></comment></comment><comment><USER44> I used to be a big fan of Heroku for a few customer's projects and for small stuff (mostly free hosting) of my own. At least for my own projects, I decided that for the monthly cost of a large VPS (I use RimuHosting) I can run 5 web apps written in Clojure (I mention the language because this is an efficient run time setup, once the JVMs warm up). I am giving up temporary scalability for a lot more bang for the buck. I also like dealing with smaller companies because you get great personal service.

<comment><USER45> Mark, thanks for sharing. Do you use an app server like immutant or does each app run on a separate JVM instance?

</comment></comment><comment><USER46> AWS OpsWorks is another reason to move to AWS. It is not good as Heroku yet! But is it evolving...

<comment><USER47> I had a great conversation the other day with one of the OpsWorks product managers. Been using Opsworks for a new client and so far I'm loving it over Rubber/Capistrano. Rubber was ok, but Opsworks wraps up the entire process just enough (but leaves enough control)

</comment></comment><comment><USER48> Not sure how I feel about these "Being a sys-admin is hard, I'd rather not do it" type posts. This is literally THE CORE OF YOUR COMPANY, your site not working is inexcusable!

I think this is going to be an emerging problem for many startups these days. Patterns of completely unstable products because "ops is hard LOL" followed by a desperate attempt to hire actual sysadmins for their company and excuse after excuse of how "awesome we are for growing so quickly!".

<comment><USER49> This. If you don't want to and/or are unable to properly operate your systems, please stay on Heroko, AppEngine etc., otherwise this is a disaster waiting to happen.

</comment></comment><comment><USER50> Is anyone aware of an up to date comparison of the various orchestration systems?

I have rolled my own which, being totally biased, I do believe is more full featured than most of those mentioned in this thread, as well as more redeployable/generic. (100% pluggable platforms, built to integrate tightly with build services and developer environments, etc.). Limited internal testing at this stage (1000s of VMs instantiated, across three cloud providers (one internal), only two distinct OS deployment environments targeted thus far).

However, it'd be great to have an overview of this area... it's certainly exciting. I am beginning to feel like the DevOps batton has been passed from large companies with massive infrastructure requirements back to the community over this last year, as more developers are becoming aware of the pitfalls of single provider cloud solutions and the rough edges and missing features of existing multi-cloud deployment APIs. We can probably expect great things in this area over the next 12 months.

</comment><comment><USER51> When exactly do you bake a new AMI? On every Soundslice deploy?

<comment><AUTHOR> No, definitely not on every deploy! For a deploy, I just do a "git pull" on all of the production boxes and restart the web server.

Baking a new AMI only happens when there are new underlying dependencies, like a new package from apt-get or a new Python module from pip. In other words, it's rare.

Hope that helps!

<comment><USER52> Is there any reason for not letting pip update things automagically from your `prod_requirements.txt` using a simple fabric `fab update_pip` when necessary?

<comment><AUTHOR> Yes, in fact that's what I do! The issue is I need to account for any new servers that might spin up later (hence updating the AMI).

I could also change the AMI's "user data" to run pip when the instance loads, but I'm not 100% confident it will always run without errors. I feel better doing it manually and baking it into the AMI. Personal taste.

</comment></comment></comment></comment><comment><USER53> I saw your video ([http://37signals.com/svn/posts/3446-adrian-holovaty-talks- so...](http://37signals.com/svn/posts/3446-adrian-holovaty-talks-soundslice- at-37signals)) on the 37signals blog where you mentioned that the server stuff was boring. Now it just got interesting!

<comment><AUTHOR> Ha, I guess you're right! I must say it's become more interesting than I expected.

</comment></comment><comment><USER54> I have always wanted to learn how to build stacks on AWS and use them with my apps. Does anyone know of any good resources out there for this, definitely for a noob like myself? Perhaps a codeacademy-like site or user-friendly book?

</comment><comment><USER55> The notes are nice, but this seems like a highly relevant detail: "I'm lucky to be friends with Scott VanDenPlas, who was director of dev ops for the Obama reelection tech team"

<comment><USER20> At least a bit of the knowledge has now been shared via this article.

</comment></comment><comment><USER56> My company does automatic setup too (<https://circleci.com>). However, we went in a different direction than Heroku, and allow people to nail down the exact version of their platform they want to use (for example, our users use over 30 different Ruby versions). We haven't needed the same flexibility for DBs and libraries, but I imagine that will come.

</comment><comment><USER57> I love Heroku: git deployment, the "dyno" abstraction, Procfiles, buildpacks, putting configuration in environment variables (all the 12 Factor App stuff: <http://www.12factor.net/>).

Is there an open source implementation of Heroku (close to 100% compatible, not just similar ideas) that runs on your own cloud?

<comment><USER58> Clone of Heroku? None I'm aware of. A bring-your-own PaaS? Many. I work with Cloudify myself: <http://www.cloudifysource.org/>

Nice things: It integrates with Chef/Puppet, allows built-in auto-scaling so it's cloud independent... has recipes for many applications and services ready to go: <https://github.com/CloudifySource/cloudify-recipes>

Drawbacks in my mind are that it's based off of Groovy which may or may not be helpful to some people. The other thing is that it doesn't have a direct method to do source/application deployment so you kind of have to roll your own via 'Custom Commands'+Groovy.

<comment><USER59> You can also launch many of the popular frameworks such as MongoDB, Play Frameowrks etc online: <http://www.cloudifysource.org/cloudifyRecipeCatalog.html>

</comment></comment></comment><comment><USER60> I haven't seen a very good reason to go with Heroku over AWS in any relevant scenario. It seems like you're paying more for less.

</comment><comment><USER61> Just wait until AWS East goes down. Not if, just when. But Heroku is in AWS East, too. I am so happy to have gotten out of AWS.

</comment><comment><USER62> This doesn't put me off heroku very much. Seems a pretty meager list of complaints.

</comment><comment><USER63> you can use Cloud 66 to deploy your apps from Heroku right to your own servers with little hassle and no need to do much sys admin work.

</comment><comment><USER64> well, that escalated quickly

</comment><comment><USER65> AWS, is simply amazing infrastructure automation. Learning how to use it, even if your not a sysadmin, is going to be a part of your job as an engineer at some point in the future.

How many times has your boss or your organization said "we want xyz", but we cant hire admins and we have no hardware?

Every time someone hits that brick wall, AWS is right there. Although I rarely use the retail side of Amazon.com, I think AWS is a stroke of genius whose full impact wont be felt for a few more years.

</comment><comment><USER66> very interesting , thanks !</comment>


Samaras expects 'exceptionally important find' at Ancient Amphipolis - <AUTHOR> http://www.ekathimerini.com/4dcgi/_w_articles_wsite1_1_12/08/2014_542093 <comment><USER2> All pure speculation. They haven't even excavated yet, only outlined the walls.

They write so much in 3 different places, including literally admitting hype in the write-up: "There has been speculation that the tomb could contain the remains of Alexander the Great..."

Worse yet, this is old speculation news. Last year, nearly to the day, they (the Culture Ministry) made a similar announcement: [http://www.dailymail.co.uk/sciencetech/article-2401057/Have-...](http://www.dailymail.co.uk/sciencetech/article-2401057/Have- archaeologists-discovered-grave-Alexander-Great-Experts-enormous-marble-tomb- fit-king-Greece.html)

This area is home to many excavation projects; google "site:.edu Amphipolis" and you'll see quite a lot of them.

<comment><USER3> Although you are not entirely wrong, "all pure speculation" is different than "this is a speculation based on some findings, so far".

<comment><USER2> Sorry, my language is harsh because I was addressing the nature of this writeup, which I labeled as "pure speculation." I work in the news industry, so I both experience and see this side of the news industry, and I don't like it. It's the un-informed PR rewrite story.

Way it happens is a general-assignment person (who writes well but has no specific topic knowledge) gets an assignment to write a story. The story originates from a press release. The press release is political: (culture ministry is an office, tasked with promotion amongst things). Politician bypasses actual archeologists and scientists and then peppers the story with speculation designed to boost their office priorities and awareness. "By the way, this is probably Alexander's tomb. Very exciting."

The person writing the story has no training in history, archeology or is generally aware of trends, claims and excavations. No clue what an in-situ find is. They look at the PR, look-up a few background factoids, rewrite the piece and publish it. As you walk by and ask how their morning is, they casually proclaim "did you know pp, they discovered Alexander's tomb this morning???"

I cringe and walk on.

<comment><USER3> Yup, you are not wrong. This particular newspaper (Kathimerini - "Daily" in greek) is known to be government-aligned and is basically jumping on every opportunity to make it look like more and more things are running better in the country.

It is also very much worth noting that the greek prime minister himself, Samaras, said "Please be patient for a few days - for a very important archeological discovery to be made". Alexander's tomb? "I told you so". Someone else's tomb? "This is still an important discovery, blah blah blah".

<comment><USER4> Some Greeks are very proud (even precious) about their cultural history. Check out this response to Dr Armand D'Angour's 'attempt to re-write Greek history' (and dont' forget to read the comments): [http://eu.greekreporter.com/2014/07/23/bbc-attempts-to- rewri...](http://eu.greekreporter.com/2014/07/23/bbc-attempts-to-rewrite- ancient-greek-history/)

On the issue of Alexander specifically, it's important to note how important Alexander is in whole Greece/FYR Macedonia thing. Alexander was a Macedonian, of course, but what was then Macedonia is now part of Northern Greece, meaning the (modern) country of Macedonia can only be called (The Former Yugoslav Republic of) Macedonia. By 'locating' the tomb of Alexander in (modern) Greece, the Greeks further solidify their claim that Alexander is part of their cultural heritage, not (FYR) Macedonia's.

(This is interesting because when Alexander was actually alive, he did everything he could to make himself look more Greek, whereas the Greeks proper saw him as a barbarian Macedonian and wanted nothing to do with him...)

<comment><USER3> I am greek. :D

<comment><USER4> Yes, as I said, it's only /some/ Greeks that feel this way. I hope I haven't misrepresented the situation too much? (Please correct me if I'm wrong!)

<comment><USER3> No, you are pretty much right. Our national identity is our ancient (not just the cultural - more recent poets for example are not that prominent in our daily pride-talk, and more recent events like the civil war are comfortably omitted when we talk about "Greece") history. So there is a lot in stake for us with those matters.

However I would disagree with the "Macedonian barbarian" note you made: Alexander was educated by Aristoteles and he was elected as the main greek general in the war against the barbarians (Persians) by the rest of the greeks.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER5> What hope is there for humanity if war mongering tyrants are at the very least not honored and worshiped. War and organized murder should not be a virtue, it should be a necessity for defense; after which you lay down your arms while still preparing for the possible necessity for defense from ever present psychopathic human demons.

</comment><comment><USER6> This would conflict with the generally accepted story, that the Greek general Ptolemy took Alexander's body with him back to Egypt, where it was used to bring legitimacy to his new regime.

<comment><USER7> He was transported to Memphis, then Alexandria and was laid to rest there and Roman Emperor's used to come and pay respects at his tomb long after his death,

[http://en.wikipedia.org/wiki/Memphis,_Egypt#Ptolemaic_Period](http://en.wikipedia.org/wiki/Memphis,_Egypt#Ptolemaic_Period)

[http://en.wikipedia.org/wiki/Tomb_of_Alexander_the_Great](http://en.wikipedia.org/wiki/Tomb_of_Alexander_the_Great)

The title of this post is way off, it doesn't even reflect the actual article.

</comment></comment><comment><USER8> From what I've read the tomb is said to be 10 times bigger than that of Philippos his father. Now if history is correct and Ptolomy took Alexander's body to Egypt, could it not be so that the tomb was in any case originally built and intended for Alexander?

</comment><comment><USER9> Couldn't the submitter have waited a few weeks until they actually found something? (Also, how did this get so many upvotes?) The article doesn't even quote the archaeologists, but only a government minister, for chrissake.

<comment><USER10> A few weeks? Well, they've been two years excavating as of the date of the article, and so far it looks like they've found a strange tomb. It may be years rather than weeks before they know anything definitive.

</comment></comment><comment><USER11> Unlikely. He was buried in Alexandria, then moved to the desert, then taken to Rome and lost. They might have found his family tomb in Greece however. They found his Dad about 20 years ago with lots of spectacular artifacts.

</comment><comment><AUTHOR> Another link with video [http://greece.greekreporter.com/2014/08/11/the- mystery-of-an...](http://greece.greekreporter.com/2014/08/11/the-mystery-of- ancient-amphipolis/)

</comment><comment><USER12> >>There has been speculation that the tomb could contain the remains of Alexander the Great or his wife, Roxana, and their son, Alexander IV.

TLDR</comment>


In 1991, Congress authorized $650M to make driverless cars a reality (2013) - <AUTHOR> https://www.smithsonianmag.com/history/the-national-automated-highway-system-that-almost-was-63027245/ <comment><USER2> When I was studying computer science at Carnegie Mellon in the '90s, I played bass in a band, and the guitar player was a grad student at the Robotics Institute working on the autonomous vehicle project, the reason CMU was part of the consortium in the article. NavLab 2 was an Army Humvee with a laser scanner mounted on the front, along with some cameras and a bunch of other sensors, and some powerful-at-the-time computers (Sun SPARCstations, IIRC) in back. It was a pretty impressive sight: [https://www.cs.cmu.edu/Groups/ahs/images/navlab_1_5_images/n...](https://www.cs.cmu.edu/Groups/ahs/images/navlab_1_5_images/navlab2.color.small.gif)

It was usually parked in the Field Robotics garage, but if you went out early on a weekend morning, you might catch it driving itself slowly around Schenley Park (which is right next to CMU campus) – without anyone at the wheel, which was really unreal and sci-fi-ish at the time.

I remember one story of a jogger who was surprised to suddenly run across it. The story goes that the he or she screamed and actually fainted. The truck was programmed to stop if it found itself in front of any obstacle, so when the jogger came to, there was the truck looming above.

I have no idea if the story is actually true, but I like it anyway.

<comment><USER3> For anyone else having trouble loading the image: [https://web.archive.org/web/20160920032328/https://www.cs.cm...](https://web.archive.org/web/20160920032328/https://www.cs.cmu.edu/Groups/ahs/images/navlab_1_5_images/navlab2.color.small.gif)

That's pretty incredible - they let it run unsupervised? Or did the jogger just faint for a very short time period?

<comment><USER2> I remember it running autonomously on the little roads in Schenley Park, which were wide just enough for the vehicle but not open to traffic. Unfortunately, I never learned what happened to the jogger.

</comment></comment></comment><comment><USER4> One the problems with government run transportation R&D is that the responsible agencies are fundamentally not in the R&D business. DoT and subordinate agencies like the FAA are regulatory and risk management organizations (with some mandates in inspection, construction and telecommunications). They lack the expertise, structure and know-how to build effective Research and Development programs and to overcome their own institutional risk management culture. For example, the FAA has been trying to roll-out a new air traffic control system for decades and at a cost that regularly far overruns any estimates the Agency can put together.

The DoD, DoE and the Intelligence Agencies have dedicated R&D arms (DARPA, IARPA, ARPA-E, etc.) with long legacies of knowing how to to research and shepherd it into operational development states and/or commercialization. Even other parts of the civilian government has things like the NSF,

The closest the civilian world has for transportation is maybe NASA? But for self-driving cars, the DoT's R&D components are not well organized and seen very much as side-show "Administrations" and are generally new affairs. DOT's version of DARPA is called "RITA" and was formed only in 2005 by strapping together a handful of previous organizations inside of DoT and are focused mostly on, you guessed it, improving safety and studying statistics.

It's no wonder that most of the government research that's actually made progress in this area have all come from DARPA.

<comment><USER5> I think you're missing a few factors. Most critically, DARPA has some success because they have a customer (the DoD) with specific requirements and a nearly unlimited source of funding compared to any other government initiative. The Defense Department probably wastes more money each year on meandering, unfinishable snake-oil projects than has been spent on driverless car R&D by all entities in all of history.

But it's true that driverless car tech is unlikely to come out of direct government funding. Instead, that money should be going to fund quality education at all levels, and to build proven technologies that would solve many of the same problems the driverless-car pipe dream would only be an inferior version of--ie high speed rail links in the middle distance where it's a better form of transport than cars (driverless or not) and planes.

<comment><USER6> Driverless cars are directly coming out of DARPA funding ex: [https://en.m.wikipedia.org/wiki/DARPA_Grand_Challenge](https://en.m.wikipedia.org/wiki/DARPA_Grand_Challenge). Google for example can trace it's program directly back to DARPA.

Companies are more willing to fund after the basic proof of physical possibility shows up. Vs the kind of this might be possible in 30 years projects DARPA gets off the ground.

PS: It's often just as much about people getting experience working on these problems as it is any specific technology.

</comment></comment></comment><comment><USER7> We invented horseless auto mobile but required continuous human attention for steering and managing the machine. With a horse indeed we already had semi- driverless operation in some scenarios.

We are reinventing the "horse" \- sometimes I think.

<comment><USER8> I often remark that people really did want a faster horse.

<comment><USER9> How hard would it be to put an actual horse in control of a car?

You get the speed of a car, and the self driving of the horse (or some other animal).

<comment><USER10> Here's a brief history tracking the progress of developments in that area. I'm not sure where this tech lies on the Gartner Hype Curve though.

Pigeon controls missile - [https://en.wikipedia.org/wiki/Project_Pigeon](https://en.wikipedia.org/wiki/Project_Pigeon)

Horse drives car - [https://www.horseandhound.co.uk/videos/news- videos/buttersco...](https://www.horseandhound.co.uk/videos/news- videos/butterscotch-horse-driving-car-kentucky)

Goldfish controls robot - [https://www.theregister.co.uk/2017/01/26/robot_goldfish_tank...](https://www.theregister.co.uk/2017/01/26/robot_goldfish_tank/)

edit to add one more as I realised that one of my favorite 'most worrying tech developments' fits right in...

Rat neurons cultured on silicon substrate control robot - [https://www.technologyreview.com/s/401756/rat-brained- robot/](https://www.technologyreview.com/s/401756/rat-brained-robot/)

</comment></comment></comment></comment><comment><USER11> > Needless to say, the program didn't deliver driverless cars and automated > highways to Americans. So what was the problem? The legislation didn't > really give the Department of Transportation any direction on how they > should go about the research—only that they needed to demonstrate it by > 1997. But perhaps the biggest problem was that the legislation never clearly > defined what was meant by "fully automated highway system."

Look a lot more like the problem was that there just wasn't any follow through. Prototypes were made, tests were done, proofs of concept made, but $650M is not enough to overhaul an entire nation's infrastructure, and one shortish strip of highway isn't enough to convince people to go out and buy whole new cars.

</comment><comment><AUTHOR> I found this article linked to a video someone posted here: [https://news.ycombinator.com/item?id=18755243](https://news.ycombinator.com/item?id=18755243)

The video is also pretty interesting: [https://www.youtube.com/watch?v=RlZEeIC_2lI](https://www.youtube.com/watch?v=RlZEeIC_2lI)

<comment><USER12> > In the future, which may be 10, 50 years from now, there won't be a driver

@1997

21 years are gone.

> You'll be able to read the newspaper on your way to work.

Yeah. About that...

<comment><USER13> The opening titles are even better:

> Highways of the future may feature relaxed drivers, talking on the phone, > _faxing documents_

<comment><USER12> > talking on the phone

Again, they were not completely wrong...

</comment></comment></comment></comment><comment><USER14> In early 90s, BMW had a working prototype IIRC. Something quite elaborate even though the processing power was limited. Kodak moment.

<comment><USER15> I believe one of the earliest non-guided (so actually autonomous) self driving cars was the Vamors by Dickmanns in the 80s. IIRC it used one or two racks stuffed with transputers to run the machine vision and control software. Later there was the Euroka project based off that.

> ... achieved in 1995, when Dickmanns´ re-engineered autonomous S-Class > Mercedes-Benz took a 1,000 miles (1,600 km) trip from Munich in Bavaria to > Copenhagen in Denmark and back, using saccadic computer vision and > transputers to react in real time. The robot achieved speeds exceeding 175 > kilometres per hour (109 mph) on the German Autobahn, with a mean time > between human interventions of 9 kilometres (5.6 mi). In traffic it executed > manoeuvres to pass other cars. Despite being a research system without > emphasis on long distance reliability, it drove up to 158 kilometres (98 mi) > without any human intervention.

<comment><USER16> I worked the spring and summer of 1991 at Daimler-Benz as an intern on their autonomous vehicle (VITA, a 7-ish meter bus at the time). This was the 2nd generation bus and I think the immediate predecessor to the S-class mentioned in parent comment. We'd sometimes test in Munich or Turin with the other European auto marquees, but mostly tested on an abandoned stretch of Autobahn just outside of Stuttgart.

I worked on the vision system (as in [under the direction and guidance of my boss] wrote all the code to improve frame rate from 5Hz to 10Hz, double the processing resolution (to around 128x96 IIRC), and implemented Kalman filtering for preceding vehicle detection and ranging). System was built in Occam on Inmos transputers. Each was about $15K and roughly the power of a 486 but with fast, reliable message passing. We could auto drive (lane follow and distance keep only) using only vision, but not especially reliably; it would fault and give up every 3-15 minutes, depending on lighting and roadway conditions. Partially wet, drying roadways remained a big problem as I left at the end of the summer.

Really cool internship. Part of [https://en.m.wikipedia.org/wiki/Eureka_Prometheus_Project](https://en.m.wikipedia.org/wiki/Eureka_Prometheus_Project)

VITA image: [https://goo.gl/images/Cz8ZuZ](https://goo.gl/images/Cz8ZuZ)

<comment><USER14> I wonder how fast did these companies ran to their archive department to find notes from that project when Google, Tesla et al started to pitch level5 auto- automation.

ps: Also did you know about the pseudo self driving system made in the US (50s) by embedding a radio wire in the road so the car above could stay in lane ?

<comment><USER16> Didn't know about the radio guidance; will have to google it.(I do know about the 4-course aviation radio nav systems, but am not old enough to have flown them.)

With regards to quickly searching the archives, I suspect all the major manufacturers have had a reasonably steady, perhaps varying in intensity, advanced research group. So, it's probably less searching the archives and rather just asking the grizzled engineers in those groups...

</comment></comment></comment></comment></comment><comment><USER17> I was watching First Man recently, the movie about the first moon landing.

I'm wondering how much the popularity of computers changed how SpaceX was able to develop its software and properly analyze the data. I guess it made things easier, but to be honest I have no idea how hard NASA worked its computers and systems to use data in the 60.

</comment><comment><USER18> I always find it hilarious that they test drive autonomous vehicles in Arizona. Never in Tokyo or Amsterdam.

<comment><USER19> Why? If they don't work in Arizona, they won't work anywhere else.

If AV doesn't work in Tokyo, they may still work in many places.

<comment><USER20> I think this sort of illustrates just how hard iterative development is for most people to grasp. It's similar to how junior engineers frequently aim for the final finished product on the first iteration whereas most experienced engineers will iterate on development when in unfamiliar spaces.

</comment></comment></comment><comment><USER21> No. No. No!

This can't be true!

I like driverless cars. My libertarian friends tell me that nothing the government does is good.

Theeefore this cant possibly be true.

Same thing with the internet.

<comment><USER22> Despite the sarcasm, this is a good point. Which revolutionary technologies have __not __been developed with public money?

<comment><USER23> That the government spends a shitload of money on basically everything under the sun doesn't mean that public investment is always a good idea. The Internet was very largely developed by the private sector; despite public underpinnings of the infrastructure, the private sector would have brought it along in time as well and the result would have been largely the same.

<comment><USER24> The private sector was the phone company, and their vision was quite different. Without huge amounts of federal funding creating operating system support, tools, and operational practice the future might have looked at lot like CompuServ or AOL. I don't know whether they still have it but the old displays at Epcot used to be a fascinating warped-mirror alternative history.</comment></comment></comment></comment>


Why do rich people lie, cheat and steal more than those on low incomes? - <AUTHOR> https://www.thejournal.ie/readme/opinion-why-do-rich-people-lie-cheat-and-steal-more-than-those-on-low-incomes-4647197-May2019/ <comment><USER2> The story about the rigged Monopoly game made me think of the reproducibility crisis. I have a hard time believing that most people wouldn't point out their huge starting advantage when asked why they won. Really? You played a whole game of Monopoly, where you had four times the other guy's money, rolled twice as many dice, and got more money for passing Go, and you think you won because of superior skill? I don't know people who win at fair Monopoly and are under the delusion that Monopoly is a particularly skill based game.

This seems like the kind of result that would be tough to reproduce. This test result is ideologically satisfying and sensational - "The rich are evil, we knew it." The null result, "People realize it when they are massively advantaged in an unfair game" seems unpublishable.

I googled the study and the name of the author with the word "replication". It seems like at least some of the author's research on this subject area, though I can't tell if this is the same Monopoly experiment or not, failed to replicate.

[https://www.nature.com/articles/sdata2016120](https://www.nature.com/articles/sdata2016120)

<comment><USER3> I have issue with "delusion that Monopoly is a particularly skill based game", because it's actually very skill-based, but designed in a way that allows bad players to win a single game. It's like Poker in that respect, skilled players will win more frequently.

</comment></comment><comment><USER4> I think, with the replication crisis any sort of 'experiment' needs to be repeated quite a bit before being published... especially if it aligns with a preconceived bias.

</comment><comment><USER5> The causality runs the other way. Being willing to lie, cheat and steal makes you more likely to become rich.

But if you have any soul you'll be miserable.

Read Tolstoy's Confession.

It's short.

</comment><comment><USER6> very informative, but I think the simple answer: Because they can and they WILL get away with it. I mean the richer you are the more you get away with, heck even murder (ask O.J.). I'm a poor fullstack dev/freelancer (Earn 30-80k depending on pipeline, I really never know some months are great some we have to ask family for $$). I obey all the laws because I can't afford a ticket. I'm poor by choice a bit though, I work for a few startups sometimes taking equity for trade in hopes one of them makes me rich and to build my resume, though I've been at this 3 years and could probably get a full time gig (I live rural so remote is a must though). I also freelance and work on side projects to try and get my own SaaS up, if I didn't have ADHD things might look better. :D

</comment><comment><USER7> Good, Succinct and to-the-point article. Validates much of what many have observed.

The key takeaway: Money => Increased independence from others => Prioritising self-interest over everything else.

</comment><comment><USER8> This is why they should be taxed more :)

</comment><comment><USER9> Not becoming a dick while getting rich is hard. Wasn't that Jesus statement 21 centuries ago ?

</comment><comment><USER10> Punishable by fine = legal for rich people.

</comment><comment><USER11> > Why do rich people lie, cheat and steal more than those on low incomes?

Because a financial fine with no serious criminal penalties behind it is barely noticed by the highly wealthy. A $550 traffic/moving violation and the associated increased car insurance premiums associated with it means two very different things if your annual income is $55,000 or $600,000.</comment>


AlgoExpert – 70 Video Explanations of Popular Coding Interview Questions - <AUTHOR> https://www.algoexpert.io/product?src=hn <comment><USER2> Has anyone tried this? Any feedback?</comment>


Detecting Consciousness (2017) [pdf] - <AUTHOR> https://www.alleninstitute.org/media/filer_public/3e/7a/3e7aabb0-5da7-4915-b4b6-2aa896c8faee/2017_11_howtomakeaconsciousnessmeter.pdf <comment><USER2> "...when they were deeply asleep and therefore unconscious."

Advanced meditators are able to be vividly conscious in deep sleep. Though it's probably impossible to describe, the closest might be "consciousness without content." It's subtle enough that most of us overlook it.

Even something as common as lucid dreaming took centuries to be taken seriously. The community at large insisted that it was a delusion. We are probably years away from being able to make sense of the claim that we can be profoundly conscious in deep sleep. Perhaps that can only happen when more of us are trained well enough to experience it for ourselves.

I'll be eager to see how society integrates these kinds of understandings into end-of-life issues, if I'm still around.

<comment><USER3> > Advanced meditators are able to be vividly conscious in dep sleep.

How can that be known except through self-reporting? At least with lucid dreaming, experiments have shown communication from the subject via eye signals[1]. Has a similar thing being done with meditators?

[1] [http://www.historydisclosure.com/scientific-experiment- prove...](http://www.historydisclosure.com/scientific-experiment-proved-lucid- dreaming-exists/)

<comment><USER2> Great question. I'm not sure the mind has control of the body in that state. But there's an even more fundamental problem: intention is a form of content, so one would by definition be out of the state by expressing intention.

I'm sure to some this sounds bogus or like a cop-out. But experiencing it (or similar states, as can happen with psychedelics) can impact our waking lives in meaningful ways, so I hope it's not brushed under the rug for much longer.

</comment></comment></comment><comment><USER4> I find the wording "detecting consciousness" problematic. They are not detecting consciousness as if they were detecting subatomic particles in a cloud chamber, or pollutants in the water; they are just measuring some neural fireworks that they believe coincide in time with the reported subjective experiences of the patients.

Then there's this:

> The current most promising scientific theory of consciousness [...] is > Integrated Information Theory (IIT). [...] IIT emphasizes the differentiated > and integrated aspect of any subjective experience and postulates that the > mechanism supporting conscious experience in the human brain's neocortex > must likewise incorporate these two attributes.

I don't see any reason why the presumed features of our subjective experience should be "reflected" in the physical working of our brains.

<comment><USER5> > I don't see any reason why the presumed features of our subjective > experience should be "reflected" in the physical working of our brains.

If subjective experience has absolutely no physical correlates then wouldn't that make it strictly supernatural?

It's a coherent position, but it turns consciousness into an article of religious faith.

<comment><USER6> Consciousness is supernatural and metaphysical by virtue of it existing outside the natural and physical world. This is true even if we could prove that some formations of matter correlated or even had a one-to-one relationship to consciousness. It's easy to see that the experience of, say, reading this post exists nowhere in this physical universe. In fact, consciousness is the only supernatural phenomenon that we know of, and we even have first-hand, moment-to-moment proof of it.

<comment><USER7> >It's easy to see that the experience of, say, reading this post exists nowhere in this physical universe.

Never mind easy, I can't see the truth of your proposition at all. Why wouldn't my experience exist in this physical universe? Everything else does and we've never found evidence of a single thing that doesn't. That I experience a feeling of existing in no way at all implies it's not based in physics.

<comment><USER6> > Everything else does and we've never found evidence of a single thing that > doesn't.

Right, consciousness seems to be in its own, unique realm in some way.

> That I experience a feeling of existing in no way at all implies it's not > based in physics.

That must be some trippy physics. It's probably safest to be agnostic about it, but it just seems absurd to me to imply that a mathematical formula can produce feeling or experience. How can you describe the experience of qualia mathematically? I think this is the Materialist view gone a step too far. Still, can't rule out any possibility if we're searching for the truth.

<comment> matt-attack They said the same thing about flesh. It cannot simply be made of the same material the makes up rocks and sand. But it is. They said the same thing about thoughts, that they are the domain of the soul. But we've now seen that they're entirely physical and measurable constructs. Same was said about mental disorders. Turns out they're physical deficiencies.

Wouldn't we learn at some point to stop presuming complicated things to be non-physical?

<comment><USER8> Let's assume it is a physical process. We can then recreate it away from a body. We run the experiment. Something is now experienced. Who is it that experiences?

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> The existence of true unconsciousness is clouded by a seemingly-impenetrable self-reporting problem:

 _From the perspective of recall, unconsciousness is indistinguishable from a gap in memory._

There is no way for a person to determine whether they were truly unconscious or just conscious at a sub-memory level.

(This is why I personally doubt the existence of true unconsciousness. I think it's entirely possible that consciousness is uninterruptible, at least during the lifetime of a conscious organism.)

<comment><USER10> I know this is going to sound bullshit but hear me out. I agree with you, because when I took DMT it really felt like for the first time in my life, my brain had been completely reset. I had to re-remember absolutely everything, including the fact that i existed. This doesn't happen when you wake up, instead it feels like you just appeared in the real world, having just recently experienced things in dreams that you may or may not remember.

DMT was not fun, also

<comment><USER2> It can also feel like one is not exactly re- _remembering_ the world, but re- creating it. As if one is recalling the trick of creating the world.

</comment></comment></comment><comment><USER11> This is fantastic research about detecting consciousness in the human brain specifically, but I haven't come across much discussion about detecting consciousness _in general_ , in arbitrary arrangements of matter, and by _simple inspection_ (by poking it and seeing how it reacts) Poking it is the traditional way (eg behavioral tests), but that limits detection to only those conscious phenomenon we succeed at poking, and simultaneously succeed at detecting a reaction from (eg it doesn't take 10,000 years to react, or emits chemical pheromone signals that aren't detected/recognized, or communicates via micro-range RF, or something).

I call this the "intelligent lifeform sensor" problem, after the implementation from Star Trek.

Consciousness is clearly an information theoretical phenomenon that occurs in matter. There should be some sort of general information-theoretical description that applies to matter (be it cephalopods, or rocks, or fungal networks, or exploding stars, or....)

</comment><comment><USER12> This is about a new technique for detecting consciousness that borrows from computer science. I found the article to be approachable by a layman.</comment>


The Archipelago of Weird (2005) - <AUTHOR> http://siderea.livejournal.com/201206.html

<comment><USER2> _``When I signed up for graduate school, the one thing I was absolutely, positively clear on was that I was sojourning to the Mainland. ... I can 't exaggerate how helpful this mindset has been on a practical level. Really, it's anthropology-fu. By approaching pretty much everything from an "I'm a foreigner" stance has made social navigation vastly easier.''_

I suppose that approach could apply to many other situations, such as software developers who need to work in a company that's dominated by business people.

By the way, the author of this piece has a fascinating history (see the "Profile" link): she's a programmer who changed careers to become a psychotherapist.

</comment><comment><USER3> This is fascinating because I've totally become a mainlander, although in high school and young adulthood I flirted with many island cultures, and everything about this post reminds me of some of my mid-90s irc days and socializing with all manner of gamers, goths, freaks and phreaks.

These days I had sort of been feeling like there are no more islands because of The Internet and all culture is available to everyone at all times. But maybe this is just part of passively slipping into the mainstream and not recognizing how the subcultures are thriving without me.

<comment><USER4> There are a few good thriving subcultures out there, from what I can tell. My main interactions have been with burners (as in burning man) and makers. But some of my friends are also deep into anime, video games, hacking, SCA, and the subcultures that follow those interests.</comment></comment>


Ask HN: What is a solid % for newsletter signup? - <AUTHOR>

On standard site (small business/web app/etc), what would be a good % for newsletter signup?
What about when offering a discount code?
Does HN have any past experiences they could chip in? <comment><AUTHOR> Some of my stats from one of my sites...

215 subscribers/7,086 visitors (3%)

With offering a discount code. Is a "one time only" offer as well, that is presented BEFORE they see any content of the site or prices.

Interesting enough, almost everyone who goes onto the mailing list ends up purchasing.</comment>


Is your New Year's resolution to learn to code? - <AUTHOR> http://newyearofcoding.com?hn

<comment><USER2> Yes, php and JavaScript</comment>