An Important but Rarely Discussed Lesson of the Stanford Prison Experiment - <AUTHOR> http://blogs.scientificamerican.com/beautiful-minds/an-important-but-rarely-discussed-lesson-of-the-stanford-prison-experiment/

<comment><USER2> This is an article talking about the fictionalized movie as if it were the facts of the case.

<comment><USER3> How do we know its a fictionalized movie? Well in the movie all of the prisoners are white.

</comment></comment><comment><USER4> Zimbardo

* old project : "What makes us evil?"

* new project : "What makes us good?" or Hero ? [http://heroicimagination.org/](http://heroicimagination.org/) ( The Bystander Effect ; Mindset; Social Conformity; Adaptive Attributions; Situational Awareness; Prejudice and Group Perception )

[https://www.facebook.com/pages/Heroic-Imagination- Project/32...](https://www.facebook.com/pages/Heroic-Imagination- Project/322165964228)

\---

video: "HOW CAN AN ORDINARY PERSON LIKE ME BECOME A HERO The Heroic Imagination Project - Philip Zimbardo" The Heroic Imagination Project, Philip Zimbardo at Brain Bar Budapest 2015.

[https://www.youtube.com/watch?v=HdAvV3SjNmY](https://www.youtube.com/watch?v=HdAvV3SjNmY)

</comment><comment><USER5> Power does not corrupt. Power only exposes what is already corrupt.

<comment><USER6> Care to defend this? My only conceptions of this being a truism involve either destiny or presuming people don't change, and neither are satisfying to accept on their face.

<comment><USER5> Did you vote me down? I never said anything about people not changing. My post has nothing to do with destiny or the human capacity to change themselves.

In a sense what I'm saying is the more darker conclusion from this experiment is that the "power" given out in the experiment didn't magnify or change anything. The guards in the experiment were already corrupt, the experiment only served to reveal their true nature.

Aspects of Darkness exist within us all. We've all felt the tug of greed, anger, lust, gluttony and hate to certain extent and we've all taken measures to hide this darkness. Power eliminates fear and reveals traits that we would otherwise hide.

I'm not saying everyone is inherently evil, nor am I saying people can't change. My belief is that people can change, some people are more evil than others and that power reveals the true nature of a man.

</comment></comment></comment><comment><USER7> "Maria Konnikova casts some doubt on that conclusion, arguing that the real lesson is the power of institutions to shape behavior, and how people are shaped by those preexisting expectations." "Yes, power corrupts. But power does not corrupt everyone equally.

      As just a middle aged man who has observed many people, including my own behavior; I have honestly found that 99% of the people(men, and women equally) will go along with the crowd/institution as long as it's legal, and involves their livelihood.

My antidodial observations are mainly from work, or educational enviorments. I was in a professional school years ago. (I won't mention the profession out of privacy.)

The first day of class all the students were bright eyed, and filled with optimism. Well as time went by it was pretty obvious that the profession was based on antidodial evidence. I was beyond disillusioned, but kinda knew what I was getting into before paying all that tuition money. At the time, their was still enough bad research which confused the the truth that the whole profession was placebo based, or basically a scam. I noticed all the students, and all, but one Instructor, made up reasons to believe in the program/profession.

These students, and teachers financial/educational lives would have been negatively effected if they questioned the profession too much. So what did my class(around 75 people) do when it became obvious the profession really didn't work the way we told, or believed?

The students did absolutely nothing!

They didn't question the bad science the instructors were spewing. Their was only one student who dared to question the profession, and his eventual livelyhood, as I recall. The smarter students rationalized, or went onto denial about the profession that was based on basically antidodial evidence, and naieve politicians. Not one student walked away from that program--well one, but for other reasons(nervous breakdown--I recall?). A married couple was in my class. Her husband(23?, perfect health) had a stroke on a Sunday afternoon, after getting a treatment from this profession/school. She showed up on class a few days later. Her belief/denial was so strong, I don't think she even dared to link his death with the school's faulty theories?

My point is no one dared to question the school/profession. Their hasn't been a week that goes by that I don't think about the findings of that Stanford Prision Experiment, and my experience at that school.

I have spent over twenty years looking at people in institutions(educational, occupational, personal relationships), and found that most people just go along with the crowd, or institution. Yes, I have found a few Rebels, but true Rebels--I can literally count on one hand. By true, I mean calling the institution's behavior out, or leaving.

The amount of compliance to bad/questionable institutions; I find very depressing. No, there's no utopia, but what people do to other people/environment because they--can, just reminds me how scared/desperate/selfish/malleable, and in some cases just evil we all are?

(This is nothing more than my ramblings on my life. It's purposely vague, because I don't have facts--just antidotal evidence, and observations. I do think about the intricacies of that study almost daily, and its findings hold up on too many of my life situations/events. Sorry, about my writing. I just don't have the energy to edit this. By the way, if you are a true Rebel; you have my praise, and gratitude. I don't recommend fighting the institution if you are poor--you might become homeless? I was no Rebel. I was in the 99%.)

If you have become Institutionalized by our society; I don't see anything wrong with fighting the system in little ways? Just doing the right thing every once in awhile helps? Nothing Big, and Dramatic, but truely doing the right thing in a merky situation might help? Wow--I feel depressed.)</comment>


Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS - <AUTHOR> http://www.allthingsdistributed.com/2012/01/amazon-dynamodb.html <comment><USER2> A feature request (I see that Werner is reading this message thread): for development it woud be very nice to have the Java AWS SDK have a local emulation mode for SimpleDB and DynamoDB. This would allow development while flying or otherwise off the Internet. Similar functionality as AppEngine's local dev mode.

<comment><USER3> Since HN doesn't show vote totals I wanted to chime in and say this would be really useful to me as well.

<comment><USER4> HN doesn't show vote totals, but it does sort posts based on the number of votes (descending), so the more popular posts rise to the top, just like the front page.

<comment><USER5> That isn't how the frontpage works, and if it did, we would rarely have news turnover.

<comment><USER6> Well, yes and no. Upvotes push submissions toward the top of the page, but the ranking algorithm also includes a decay factor so that old articles eventually fall away.

</comment></comment></comment></comment></comment><comment><USER7> This looks like a great service - there are some really interesting ideas here. The provisioned throughput is a great idea. It's interesting to see it runs on SSDs as well, I wonder if its storage is log based like LevelDB. The composite hashed indexes are really interesting as well - I guess they partition the index across nodes, and each node maintains a range of the sorted index. It'll be interesting to see how usable that is in practise.

I read with interest Steve Yegge's mistakenly-leaked email about the oppressive conditions for engineers at Amazon. It's hard to reconcile with the sort of innovation they consistently show.

<comment><USER8> > I read with interest Steve Yegge's mistakenly-leaked email about the > oppressive conditions for engineers at Amazon. It's hard to reconcile with > the sort of innovation they consistently show.

My take on Steve's rant (as an engineer at Amazon) is that a lot of the issues he pointed out are legitimate, but at an entirely different scale than he was pitching them at.

Day to day I work on a product with another couple dozen or so engineers. We build what makes sense for our product, and for the most part we build it in a way that makes sense for us. Sometimes we are under pressure to leverage other parts of the platform, and sometimes that does entail a lot more work. Most of the time, though, it ends up reducing our operational load (because the systems we depend on support products much larger than ours :) and giving us someone we can page when things go pear-shaped.

Amazon isn't the perfect place to work, but it's generally not bad (other than the frugality thing; that sucks as an employee no matter which way you slice it).

<comment><USER7> Interesting, thanks for the perspective - he definitely made it sound like a sweatshop, and that's not the sort of environment normally associated with this kind of innovation.

</comment></comment></comment><comment><USER9> I simply do not see how competing cloud vendors can keep up with this. Most of them are still struggling to provide anything beyond a simple API to start/stop machines.

<comment><USER10> Well, Azure has been offering a similar NoSQL service for a few years now.

[http://sigops.org/sosp/sosp11/current/2011-Cascais/printable...](http://sigops.org/sosp/sosp11/current/2011-Cascais/printable/11-calder.pdf)

<comment><USER9> No. They offer the equivalent of Amazon S3, SimpleDB and SQS, but nothing comparable to this

<comment><USER10> Can you elaborate how this is different and not comparable? Azure's table service offers the same automatic partition management, unlimited per-table scalability, composite keys, range queries, and availability guarantees. The linked paper goes into more details.

<comment><USER11> Besides the points I already complained about before... How about 200ms response times even when performing a query using the Row & Partition Keys. I'm not sure if by composite keys you were referring to something other than the RK & PK because those are the only indexes you get.

<comment><USER10> ATS response times within the Azure data center are pretty impressive in my experience.

Your partition keys can be composite, have a look here:

[http://blogs.msdn.com/b/windowsazurestorage/archive/2010/11/...](http://blogs.msdn.com/b/windowsazurestorage/archive/2010/11/06/how- to-get-most-out-of-windows-azure-tables.aspx)

I agree with your other pain points - in terms of not being able to get counts, secondary indices etc. However, you can easily simulate some of those - maintain your own summary tables, indices and so on. These ought to emerge as platform features pretty soon though. It's not perfect, but its feature set is close to Dynamo.

As for Mongo DB, I guess this service has been built from ground-up to provide the availability guarantees and automatic partition management features. I don't know if Mongo provides those. You could run Mongo yourself on Azure if you wanted to; there's even a supported solution done recently.

<comment><USER11> Hmm, I guess when I think about composite keys I think of ways to indicate a specific field/column as being part of the key. Data duplication along with string concatenation aren't really an elegant way to do it. If I remember right you also can't update the key values once the record has been saved. This is coming from a big SQL guy though :)

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> This still seems a bit expensive to me for an application that would require thousands of writes per second? ie. 5k writes per second is ~$120/day. Using this for performance based analytics for example would seem out of the realm of reason for the moment.

<comment><USER13> Can you explain your use case a bit more? I'm having a hard time imagining something that does ~430M DB writes/day but can't easily afford to pay $120 for those writes.

<comment><USER14> Remember that the throughput is per item, not per query. For instance we have an indexed query that returns ~1500 rows each time. Just doing that query a couple of times per second would create that kind of throughput requirement.

<comment><AUTHOR> The amount of consumed read units by a query is not necessarily proportional to the # of items. It is equal to the cumulative size of processed items, rounded up to the next kilobyte increment. For example if you have a query returning 1,500 items of 64 bytes each, then you'll consume 94 read units, not 1,500.

<comment><USER15> The official documentation seems to clearly contradict you. The pricing calculator doesn't let you specify a value of less than 1KB. Who's right? Or maybe I'm just not understanding what either you or the official pricing doc is saying :)

<comment><USER16> From the pricing page (<http://aws.amazon.com/dynamodb/pricing>):

\---

If your items are less than 1KB in size, then each unit of Read Capacity will give you 1 read/second of capacity and each unit of Write Capacity will give you 1 write/second of capacity. For example, if your items are 512 bytes and you need to read 100 items per second from your table, then you need to provision 100 units of Read Capacity.

\---

Looks like 1KB is the minimum for calculations.

<comment><USER15> Agree, but Amazon's CTO said something different, hence my question.

<comment><USER17> Werner is right. The query operation is able to be more efficient than GetItem and BatchGetItems. To calculate how many units of read capacity will be consumed by a query, take the total size of all items combined and round up to the nearest whole KB. For example, if your query returns 10 items that were each 1KB, then you will consume 10 units of read capacity. If your query returns 10 items that were all 0.1KB, then you will consume only 1 unit of read capacity.

This is currently an undocumented benefit of the query operation, but we will be adding that to our documentation shortly.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER2> For about $15/month for minimal "reserved capacity" charges and $1/gigabyte per month replicated disk space this service looks like it will cure a lot of deployment headaches for a reasonable cost.

Interesting that when I signed up for the service that they verified my identity with a cellphone call, like Google sometimes does.

</comment><comment><USER18> "Amazon DynamoDB stores data on Solid State Drives (SSDs)" This is big.

<comment><USER19> When Amazon notes "SSD," from a client's perspective, it is only marketing. The storage media matter when you are managing your own hardware. The storage media do not matter in SaaS. For example, the storage media could be floppys and you would get satisfactory performance if there was a memory cache. Similarly you could get poor performance with SSD media if the networking layer(s) were slow. Similarly if the storage media were fault-likely CD's that wouldn't matter either, to us, because of the data replication performed in "the service." What matters in this case is the reported and actual latency.

</comment></comment><comment><USER20> I would love the option to add N index (at cost).

My guess is they will add options for additional indexes in the future...everyone needs start somewhere. Even at Amazon's size and scale.

<comment><USER8> > Even at Amazon's size and scale.

At Amazon's size and scale, it's all the more important that you start with something simple with well-understood performance characteristics from day one. AWS doesn't really get a grace period during which they get to fix scalability problems.

<comment><USER21> Exactly, compare this to SimpleDB. SimpleDB started out with an advanced query language that let query and filter your results in all sorts of ways. And guess what? SimpleDB is still limited to 10 GB per domain (aka, database). Want to horizontally scale? The official suggestion is to shard your data across domains. This is a really messy solution because you have to preshard based on estimated database size and resharding is nearly impossible (you'd have to rewrite your entire DB).

AppEngine went the other route and provided a very simple database API at first and all queries had to be range scans over an index. Any query you wanted to perform had to be precalculated by defining a composite index and some things (like inequalities on multiple fields) weren't supported. Over time they've built upon their basic database and added features such as a zigzag merge join algorithm which lets you perform queries that were otherwise impossible with a given set of indexes.[1]

I bet DynamoDB will be going the AppEngine route by starting with a simple, scalable base which can be used to build more advanced query engines and features.

1\. [http://code.google.com/appengine/articles/indexselection.htm...](http://code.google.com/appengine/articles/indexselection.html#Improved_Query_Planner)

</comment></comment></comment><comment><USER22> While putting my data completely in the hands of another company makes me nervous, I have used s3 since its release and have had no problems. Amazons really seems to "get" developer needs. I do wish they would ease down a smidge on thier pricing in general, but otherwise I really feel about them the way I feel about google at this point.

</comment><comment><USER23> [https://img.skitch.com/20120118-etwh2gwu52isnn47fw2jpcir5p.p...](https://img.skitch.com/20120118-etwh2gwu52isnn47fw2jpcir5p.png)

You could buy a lot of Riak or Cassie for that.

<comment><AUTHOR> 5000 reads per sec of 64Kb items, would make you stream 2.5 Gbits/sec using consistent reads and 1Gbits/sec writes, moving close to 1.5TB each hour. At the end of the month you have read well over 800 TB and updates 160 TB... That is a substantial application you have in mind... :-)

<comment><USER24> That may be true for an application with a constant load, but applications with a less balanced load have to provision for their peaks. My company (Malwarebytes) has very irregular traffic (at the hour mark we get very big spikes, but only for a couple of minutes) and it seems like we would have to provision (for this specific app) that peak for the entire hour. I might be misunderstanding the billing for this service though- if we ask for more units for 15 minutes, would the billing be prorated?

This actually hits on my only real issue with AWS in general, which is the hourly billing. We've used the mapreduce service a bit, and having a cluster roll up and fail on the first job is heartbreaking when it has a hundred machine hours associated with it. Obviously that is far, far cheaper than us building out our own cluster (especially with spot instances, which I can't even describe how much I love), but for some of the services smaller billing times would be useful.

</comment></comment></comment><comment><USER25> Here's a brief overview comparison of DynamoDB vs. BigTable:

<http://vschart.com/compare/dynamo-db/vs/bigtable>

The site is also able to compare many other different databases.

<comment><USER26> It seems as if this site is a user-edited wiki, and there are a lot of things that need to be filled in (in case anybody is up to speed on DynamoDB and wants to help). For instance, the Map/Reduce entry was still '?' when I wrote this.

<comment><USER27> And Big Table (or at least GAE) does support transactions.

</comment></comment></comment><comment><USER28> I work for a popular startup that has been privately testing dynamo for the last few months.

It's a fantastic product and even while in private beta has been stable and well supported.

</comment><comment><USER29> 1$/GB/month on SSD and replicated. So basically, 0.25$/rawGB/month if they replicate 4 times.

They are making money on the read/write and are selling the capacity at current cost. Which — knowing the tendency for AWS to decrease the prices very slowly combined with the huge decrease of the prices of the SSD drives in the past months/years — is not a bad strategy to convince us to switch.

</comment><comment><USER30> I posted a comparison to Cassandra here: <http://news.ycombinator.com/item?id=3480480>

</comment><comment><USER18> There are some bugs in signing up for the service. In console, if I go to dynamoDB tab I am asked to Sign Up first. I click the sign up button and I am told "You already have access to Amazon DynamoDB". Repeat.

<comment><AUTHOR> Can do me a favor and drop that in the AWS DynamoDB Forum so folks can look at it? <https://forums.aws.amazon.com/forum.jspa?forumID=131>

</comment></comment><comment><USER31> Question about Composite Hash Keys that someone might have the answer to (or be able to relate to other known implementations):

The composite key has two attributes, a "hash attribute" and a "range attribute." You can do a range query within records that have the same hash attribute.

It would obviously be untenable if they spread records with the same hash attribute across many servers. You'd have a scatter-gather issue. Range queries would need to query all servers and pop the min from each until it's done, and that significantly taxes network IO.

This implies that they try to keep the number of servers that host records for the same hash attribute to a minimum. Consequently, if you store too many documents with the same hash attribute, wouldn't you overburden that subset of servers, and thus see performance degradation?

Azure has similar functionality for their table service, requiring a partition key, and they explicitly say that there is a throughput limit for records with the same partition key. I haven't seen similar language from Amazon.

Whether you scatter-gather or try to cluster values to a small set of servers, you'll eventually degrade in performance. Does anyone have insight into Amazon's implementation?

</comment><comment><USER32> Awesome. Anybody want to sublease a 3 yr RDS reserved instance? :P

</comment><comment><USER33> What's the concern for lock-in here - how difficult with DynamoDB be to migrate away from? Amazon seems to be increasingly catching the App Engine Syndrome, though to be fair, it's been mostly network traffic-style functionality in the past.

<comment><USER15> Lock-in? You'll need to get all your data out and transform it into whatever alternative you pick. I'd say as far as AWS lock-in, this is on the low side of things.

<comment><USER34> The lock-in comes not from the data format, but rather from the quirks of the particular system du jour: API lock-in.

<comment><USER35> Getting your data out will also cost you a boat load of money. Same as putting it in.

</comment></comment></comment></comment><comment><USER36> From the limits page(1) I see you can only have 256 tables per account.

(1) [http://docs.amazonwebservices.com/amazondynamodb/latest/deve...](http://docs.amazonwebservices.com/amazondynamodb/latest/developerguide/Limits.html)

<comment><AUTHOR> As with all the other AWS services you can have your limits lifted upon request.

<comment><USER37> Does that apply for the 50-per-account cap on S3 buckets as well?

<comment><USER38> I'd imagine S3 falls within the scope of "all the other AWS services"...

</comment></comment></comment></comment><comment><USER39> I read through a number of the docs and can't quite find the answer to this question, hopefully someone here can help me out quick.

I already have a bunch of (large-ish, deeply nested) JSON objects defined for my application. I don't really want to go about redefining these since they work great between my various node processes and the front end. I am saving them in a nosql database already, I am curious about switching (to save on devops costs). I only request based on 1 Hash Key (int) and 1 Range Key (int) for all my current get operations.

Looking through the docs/examples I see a lot of this type of thing:

        {"TableName":"comp5",
        	"Item":
        		{"time":{"N":"300"},
        		"feeling":{"S":"not surprised"},
        		"user":{"S":"Riley"}
        		},
        	"Expected":
        		{"feeling":{"Value":{"S":"surprised"},"Exists":true}}
        	"ReturnValues":"ALL_OLD"
        }

The JSON item has a kinda-of 'type syntax' on it. I really don't want to redefine my deep objects, but would be willing to redefine the Hash key and Range key, while leaving the rest of the nested types alone.

Ok, my question: Do my JSON objects need to conform to this 'type syntax' JSON notation in the examples? Or can I save just any JSON object into this database and only annotate the Hash Key and Range Key using this special notation?

<comment><USER40> Their usage of JSON is just incidental to your usage of JSON. They use JSON as a REST transfer format. You can pretty much ignore their JSON if you use one of the high level libraries in the SDK.

You can define a table with 3 fields: yourKey, yourRange, and yourJson. Put your entire JSON data as string in the yourJson field.

</comment></comment><comment><USER41> Pricing: _$0.01 per hour for every 10 units of Write Capacity and $0.01 per hour for every 50 units of Read Capacity._

<comment><USER34> It is amusing that was positioned to try to address SimpleDB's problem of "pricing complexity". These aren't those complicated "Machine Hours", they're "Capacity Units"!

As someone who has grappled in the past and is grappling again with the issues of pricing a database-as-a-service though, this is very much a non-trivial issue. If you have nice scaling characteristics and you want to charge the minimal price, your price isn't going to be predictable. Essentially, what Amazon have done here is to set a cap and then charge you the cap irrespective of actual usage, which seems to be the model that's winning on the cloud.

<comment><USER42> Capacity Units strike me as a big improvement (for the user) over Machine Hours, because it's very clear how a given usage pattern will translate into Capacity Units. I can predict how many Capacity Units I'll need. I've gotten badly burned over seemingly simple queries using unexpectedly high Machine Hours in SimpleDB.

<comment><USER34> My understanding is that DynamoDB's Capacity Units are just a query throttle, and you get charged based on the throttle you set, whether or not you use that capacity. It also looks like you can still have one query that consume many, many Capacity Units (e.g. table scans).

SimpleDB's Machine hours are basically the same units, but without the throttle.

So, from a technical and value viewpoint, it's a huge step backwards (pay for capacity rather than for usage), but I'm learning that psychology is perhaps just as important here.

It seems like what you really want is a throttle with per-query charging, to cap your bill. Probably you'd much prefer not to be forced to pay your cap every month, but I don't think that's being offered.

(Edit: Downvotes? Am I wrong here? If so, please contribute to the discussion and tell me why!)

<comment><USER42> True, it's pay-for-capacity and that's worse for the user. On the flip side, the constant factor seems to be about 20x cheaper (caveat: this is based on my personal experience with SimpleDB; since Amazon doesn't seem to explain how "box usage" is computed, I don't know how broadly applicable my experience is).

The big plus for Capacity Units is that Amazon actually provides a deterministic model for figuring out what you'll be charged for a given query.

<comment><USER34> Ah - that is a fair point, transparency of the "unit". The "box usage" formula was reverse-engineered and shown to be fairly simple: [http://www.daemonology.net/blog/2008-06-25-dissecting- simple...](http://www.daemonology.net/blog/2008-06-25-dissecting-simpledb- boxusage.html)

This is a big step forward in transparency, although I would suggest that SimpleDB's pricing shouldn't have been obscured in the first place.

</comment></comment></comment></comment></comment></comment><comment><USER15> I think this is huge. My first wish, before secondary indexes, is that they don't round up to the nearest 1KB for pricing.

<comment><AUTHOR> noted. (both requests that is).

</comment></comment><comment><USER43> With regard to reliability, how safe would a service like this be to store your data? They mention "back up to s3" but this sounds more like archiving. I'm wondering about backups in the case of a problem, or is the data so spread across all of their physical locations that data loss impossible?

<comment><USER40> SSD are pretty safe in general. Amazon also makes replicate of your data. It's the same if not safer than other NoSQL solution. Definitely safer than those RAM-based NoSQL approaches.

The "back up to s3" part is for archiving data, like periodic snapshot backup of your data so that you can get back the old data in case they are deleted by app/human.

</comment></comment><comment><USER44> Fast, Consistent, Secure & Replicated.

Game changed.

<comment><USER23> Not really if you pay attention to the set of NoSQL DB's available already. They are just hosting their own now.

<comment><USER44> Would be interested in details about any with Strong Consistency, decent replication across fault areas, which get the same level of performance (single digit ms) at a cost equivalent to dynamodb.

Quite a lot can be done with smart use of hugely scalable simple key-value maps.

</comment></comment></comment><comment><USER45> I'm too scared of the lock in. Afaik this isn't like EC3 where you can move a VM off the EC3 stack. The data is stored in a proprietary format so once your in, you're in.

Game changer? I need to see some evidence first on how well it performs and integrates.

<comment><USER46> It's a database, so you can always get your data out again.

<comment><USER34> It's a non-standard (NoSQL) database. Even SQL databases, despite years of standardization and efforts at poaching each other's customers, still have rough edges that make moving between different SQL database products non- trivial. Just because you can get your data out doesn't mean you're not locked in to all of SimpleDB/DynamoDB's quirks, of which there will be a lot more because it's not following a standard approach. Your code will have to go through contortions to work around / adapt to the SimpleDB limitations; effort which might well be wasted or counter-productive on a different system. That's the lock-in with AWS in general: API lock-in, not data format lock-in.

<comment><USER46> DynamoDB and SimpleDB are also a _lot_ simpler than SQL databases. As far as I can tell, DynamoDB is a key/value DB with support for ranges and MapReduce, and not dissimilar to other NoSQL databases like Riak.

There may be instances where large datasets are hard to migrate from DynamoDB, but overall it doesn't look to me like lock-in would be that much of a problem, assuming you have a decent abstraction layer.

</comment></comment></comment></comment><comment><USER34> Very interesting that there's no mention of the CAP theorem here, despite Amazon's heavy reliance on it in the past when marketing their non-relational stuff.

<comment><USER47> I'm very curious about this too. I posted in another thread:

"Doesn't that just sweep the latency tradeoff under the rug, or is flash making up the difference? What about the availability tradeoff? (I like the formulation here, consistency vs availability and consistency vs latency, as opposed to CAP which never made sense as a 3-way tradeoff: <http://dbmsmusings.blogspot.com/2010/04/problems-with-cap-an...>)

<http://news.ycombinator.com/item?id=3480805>

</comment></comment><comment><USER48> I hope this means an SSD option for RDS is coming soon.

</comment><comment><USER49> I wrote an Erlang API for DDB while it was still private. I'm hoping my client will let me opensource it in the near future.

</comment><comment><USER50> Why is there a 1MB reply limit on BatchGets and obviously no limit on Gets? Did anybody find limits on a single item?

<comment><USER51> The items are limited to 64kb so its implicit.

</comment></comment><comment><USER52> Any word on when boto will support this?

<comment><USER53> Sooner than you think

[https://github.com/boto/boto/commit/8ef550ec028b379bcd85e550...](https://github.com/boto/boto/commit/8ef550ec028b379bcd85e550eed564e4c9035d56)

</comment></comment><comment><USER54> Is there a Ruby SDK?

<comment><USER53> Yes (which I find mildly surprising given how long the Ruby library took to support batch ops on SQS)

<https://github.com/amazonwebservices/aws-sdk-for-ruby>

</comment></comment><comment><USER55> Trying to read thru all the hype, this looks like Riak hosted on machines with SSDs, with less features, and a nice billing system in front of it.

Of course for people who want a hosted solution, the fact that it is hosted, is what gives it a lot of value. There haven't been a lot of hosted NoSQL databases, at least on this scale and availability, out there.

But technologically, what's new here? Is there anything here that is really innovative? (not a sarcastic question)

<comment><USER56> For one, it responds quicker than Riak: Riak has (cold) response times of about 300ms, while this service claims single-digit ms response times. Also setting up Riak is not exactly trivial, and using this service outsources that hassle.

<comment><USER23> You can use Riak Smartmachines on Joyent's cloud that would get similar performance for an order of magnitude cheaper than what amazon is charging. If you are seeing 300ms response times, you are not using SSD's and you are not using a similar number of nodes that Amazon is charging you for.

I'm not hating on Amazon, it is a good move for them and they are doing some things that Riak cannot do, but cost and response is not one of them.

<comment><USER56> I both wouldn't bet on Joyent's Smartmachines being a magnitude cheaper and being as fast: On the speed: We run a Riak cluster, and for us the actual response times we get from Riak are, as described, slower than what Amazon promises in its docs.

On the price: If you would get 3x16 GB machines with Joyent, that would cost you 1400$/month. You can get a lot of resources for that with this new AWS service.

I don't have any experience with either Joyent or this new DynamoDB, but I do have some experience with Riak, and from the docs this new service would be a very viable competitor.</comment></comment></comment></comment>


Ask HN: Funding Strategy - Bootstrap or Angel? - <AUTHOR>

I'm at a crossroads with my app and I'm looking for some insight from those who've come before me.
I've spent most (OK...all) of my meager family investment seed money ($3K) on a very basic demo that essentially "secures" the most sensitive IP for my concept and demonstrates the core defining feature of the app.  I know that coding something probably in no way protects a concept, but I felt like I had to have something real to show investors because I'm not a programmer and this is my first project/startup/attempt-at-something-awesome.
My question is what would be the best use of my time moving forward?  Should I keep pitching friends and family for cash to add more power to my demo?  Should I work on developing a great presentation for angel investors?
I feel like the window of opportunity for my app is probably 10-12 months from closing.  If I'm not in the market by then, I'll be toast.  I've been working on the idea for the past 14 months, I know what I want to do with it.  If you were me what would you do? <comment><USER2> If you can't code, and you can get funding, you probably want to try to get funding, then use the funding to attract talent.

Really, you want a technical co-founder, but if this is the first thing you've done and you don't have cash, that will be difficult. A business guy without money is like a technical person who doesn't have completed projects on his resume.

<comment><AUTHOR> Finding a technical co-founder has been surprisingly difficult as I've worked through my network of college buddies. The project is risky and I think my buddies who are also trying to establish their careers are more tentative than I thought they would be about this.

The people who like this project the most are the industry contacts I've pitched to. Essentially I've got three believers in my product: the IP lawyer I consulted with in fall of '09 when this whole thing started, the former coworker who is also a "guerrilla-style" marketer/graphic designer/photographer, and my current coder who I've contracted to do $2K of work to build the alpha prototype.

As far as investors go, I think the recession of '08 shook people's perceptions of Wall St. and made it easier for startups to argue for funding from individuals. I make the case of, "Hey, you lost how much in your mutual fund? This product is a high risk to be sure but the potential return for a small amount of cash is astronomical." I never say those words exactly of course but that's a theme of my financial argument to investors.

Its very very cheap to start a web company and I'm trying to run it that way.

As far as my status of being a nontech guy without money, that's true to a certain extent because I have no formal training. I have been working a lot on developing the marketing and product design end of things. Graphic design has been a hobby of mine for a long time so I've been doing all the initial UI design. I guess that's my "sweat" contribution to the thing so far.

There's always talk of "value" with startups. I would argue my "value" to my startup is in the fundraising, conceptual development, and UI design end of things. I communicate well with investors, I'm comfortable speaking to people because of my teaching experience, I know the difference between selling and educating (thanks to two failed attempts in sales) but I can do both if necessary. I also have talent in graphic design and can create meaningful mockups to guide coding development. People pay for those two functions in the industry. A lot of R&D money goes into developing ideas (Google Wave anyone?) so the conceptual work I've been doing is also not without value.

The only difference is that I don't have formal training so it takes me longer to develop these things to industry standard formats because I don't know what developers need to create a quality product. I'm learning though.

<comment><USER2> I'm not saying you don't have value to bring... just like the programmer who hasn't yet completed a major project, you might be brilliant, but you might not be. The thing is, choosing a bad/useless co founder is nearly always fatal to the venture, so people can be a little conservative.

If some business guy without money approaches me and claims he knows how to raise money, well, maybe he does and maybe he doesn't. How would I know? but if the same guy approaches me and can prove that he's already raised $bignum dollars from various investors, then I at least know he's good at raising money. I mean, that's not the whole of the business guy job, but it's certainly an important part.

<comment><AUTHOR> I hear you. No worries.

I've been running into some dominant themes on HN as I've been participating and learning. This is my first project so usually when I say I can't code, HNers say, "Well you have no resume and can't code, so what value do you bring?"

Since I'm applying for winter session, I want to let the selection people know how I respond to those questions because I'm sure they have similar concerns.

</comment></comment></comment></comment><comment><USER3> Don't throw money on an idea .. way too risky if you're not technical and you don't have a technical co-founder.

That's not how things work.

Programmers can afford to start startups because the biggest chunk of investments comes from their own talent; the fact that they can sit down and hack something.

Business guy + idea + money + employees? Can't imagine that working out very well. Unless the idea is simple enough and the technology is not in itself the main product.

I mean, seriously, you spent 14 months on the idea? How do you know it will work? Sounds to me like you're putting all your eggs in one basket.

This is another reason why programmers can afford to start startups: they can change the idea quickly, and they wouldn't be in a position where they invested a lot of money in the previous idea.

Where as, if you just hire developers to develop your idea, and then it turns out that it won't work very well, you'll be in trouble because you now wasted all that money on it and you don't want to just throw it away and try something new. OK, maybe you will be able to switch your idea once, but you can't do that all the time, you don't have this flexibility.

So get a technical co-founder. That's my only advice.

<comment><AUTHOR> Thanks for the response!

I hear you. The reason I'm pursuing the project is because the product is nontechnical, cheap to build, and has mainstream appeal. There is no rocket science involved here. I'm taking existing tech and combining it in a new way that I think (and everybody says this, I know) could be really exciting.

I am looking for a technical cofounder and I might have one with my current coder. We're just phasing our way through the initial build of the site to see if we work well together. He's had some success on previous projects but we just met a couple of months ago. It's going to take time for us to figure out if we can work well together. Things are going well so far, so that's encouraging.

_______________________

Edit Note: I don't know why the comment above was downvoted. I tried to upvote it but it stayed at 0. No offense taken by me whatsoever.

</comment></comment><comment><USER4> Do you need outside investment to continue development and launch your app or can you bootstrap the launch?

Create a list of wants/needs for your app. The wants is all the features and functionality of the app that you envision. The needs are the ones that get you to a MVP.

What is it going to take to launch this app? Does it require outside sales, an infrastructure build-out (servers, storage, offices), major marketing effort?

Assuming you have a MVP, I would start marketing the app to your target market. Get some traction and even some paying customers under your belt. Get feedback from these customers, reassess your wants/needs list based on that feedback. At the same time, begin working on a presentation for potential investors. You should treat your friends/family investment the same as you treat any angel investor. Make sure all the documents are in place, whether the investment is via convertible or straight out equity. Having your friends/family invested, a MVP, and some traction will make getting angel investment a lot easier. It proves you can execute on your idea and that there is a market for your product.

<comment><AUTHOR> Thanks for taking time to help me out!

I have a marketing buddy who believes the best course of action would be to polish up an amazing presentation of mock ups and present it to some angels because the site will probably be rebuilt after receiving major funding anyway.

I've been operating under the idea that it's better to get something out as inexpensively as possible and see how people respond before getting heavily invested in the concept. That sounds like what you're advocating.</comment></comment>


Cellphones As Radios: Immigrants Dial In To Native Stations - <AUTHOR> http://www.npr.org/blogs/alltechconsidered/2013/11/15/245227755/cellphones-as-radios-immigrants-dial-in-to-native-stations

<comment><USER2> This is a pretty good solution. But not many radio stations on this yet.

I also can't seem to find a directory of radio stations and their phone numbers on their website.

While many low-end phones includes a FM radio, they don't have an AM antenna. This could be pretty useful for listening to talk radio.</comment>


Facebook Isn't Budging on React's BSD and Patents License - <AUTHOR> https://wptavern.com/facebook-isnt-budging-on-reacts-bsd-patents-license <comment><USER2> Facebook has decided that keeping the React patent grant as-is is in their best interest. I respect their right to do so.

At this point, there are two reasons for companies and individuals to avoid React: legal reasons, and philosophical ones.

Legally speaking, it's probably true that the patent grant and possible revocation of it is unlikely to impact the majority of companies who might want to use React. But even the potential uncertainty that the patents file introduces could be a decent reason to avoid React, depending on a company's risk tolerance. I'm not a lawyer, though, so don't count on any of that as legal advice.

The philosophical objection to using React is a valid one too, I think. I don't personally find the patent license too objectionable, but I respect the view of those who feel that it's not in keeping with the spirit of open source and/or Free Software. This, too, is a perfectly fine reason to choose something like Vue, or Angular....or Elm, or Ember. Fortunately, there are plenty of pleasant options to choose from when it comes to front end development these days.

<comment><USER3> I also respect the philosophical objection. My issue is with the practical one. I already find the supposed risk dubious, but it is especially weird to me how many people are just now deciding the risk is too great. As far as I can tell the risk has been unchanged since the introduction of the license.

I suppose teams could have been using it in the hopes that the license would eventually change, and changing course since it's now clear the license won't change. But even that doesn't add up, because if your company is really trying to be careful about such legal risks, you wouldn't have started using it in the first place.

<comment><USER2> I agree with you on the practical side of things. I don't think it makes a lot of sense for companies already using React to drop it in most cases.

I was mainly talking about new companies or new projects deciding to use React because of the potential risk. Even if the risk is tiny, it might make sense for risk-averse teams to choose something else, since there are so many good alternatives available. There are also cases where React is the clear-cut best choice from a technical perspective, and in these cases I'd say the benefits almost always outweigh the potential risk.

</comment></comment></comment><comment><USER3> I'm still very dubious of the _actual_ issue, though. Like what is supposedly going to happen to people who use react.

And from what I understand the license hasn't changed so why has it become an issue _now_? Like the guy in the article who says his teams have to pull it out... hasn't whatever legal danger is being alleged here existed this whole time? But right now it's become an issue? I dunno, it just smells fishy to me.</comment>


The importance of timeouts - <AUTHOR> https://brightinventions.pl/blog/the-importance-of-timeouts/ <comment><AUTHOR> Timeouts are not an exciting thing to talk about. They do not add immediately perceivable value. They are difficult to guess get right and force one to consider problems that are hard to solve. In fact, in my experience, the timeout is only ever considered when our software stops working or is about to. That is an enormous shame since, in my opinion, carefully applied timeouts can vastly improve software resiliency.</comment>


Ask HN: A self-hosted site like HN? - <AUTHOR> I would like to setup a self-hosted site similar to news.ycombinator.com but for cars and mechanics. Is there a free software web project that mimics HN features? <comment><USER2> [https://github.com/lobsters/lobsters](https://github.com/lobsters/lobsters) is similar and actively maintained.

<comment><AUTHOR> Thank you very much :)</comment></comment>


I created a Fitbit clock face in TypeScript - <AUTHOR> https://github.com/Shpota/zeit <comment><USER2>
      return `Viertel vor ${nextHour}`;

There is a mistake right there! This should read

      return `Dreiviertel ${nextHour}`;

(Not "a quarter before 3", but "three-quarters of 3").

:)

[https://i.stack.imgur.com/fNkDo.jpg](https://i.stack.imgur.com/fNkDo.jpg)

<comment><USER3> That may be a regional thing. I hear "viertel vor" far more often.

<comment><USER2> Was meant as a joke, this is a very controversial subjects in the German speaking world :)

<comment><USER4> My love for the German People is adequadely illustrated by the fact that this is, I quote, "a very controversial subject".

</comment></comment></comment></comment><comment><USER5> Nice, will check this out. I'm also currently developing a clockface for Fitbit OS but I did not go with Typescript but Ive been wanting an excuse to start using it so I might rewrite what I've done in Typescript.

I also just want to say that the whole developer experience working on a Fitbit OS clockface or app has been very nice and easy. Kudos to the Fitbit developers!

<comment><AUTHOR> I come from typed world and TS makes a big difference for me. If you haven't tried it yet I would strongly suggest.

Fitbit also provides a types lib which makes it easier.

The only constraint I had is that there is no emulator for Linux and running on a real device doesn't always work.

Let me know once you implement your clock face. I would like to check how others do it.</comment></comment>


George Soros: China is using tech advances to repress its people - <AUTHOR> https://www.theguardian.com/business/2019/jan/24/george-soros-china-using-tech-advances-to-repress-its-people <comment><USER2> Target the data. ML, AI won't do anything without data. Stop the surveillance, stop the threat.

But oh, I guess we need the surveillance to stop fake news, terrorists or nationalists, protect against election meddling, optimize ad penetration, and spy on foreign companies and governments...

</comment><comment><USER3> George Soros is jealous and angry he didn't think of that.....

<comment><USER4> Soros is actually a huge advocate for freedom and privacy of the individual.</comment></comment>


What if doing your best -- at what you're best at -- isn't enough? - <AUTHOR> http://www.tedquarters.net/2012/03/07/the-lobster-pot/ <comment><USER3> This is such a good article in two ways.

First this is an internal dialogue that is a natural offshoot of 'imposter syndrome'. And for me it actually becomes more frequent as I progress further in my career as a dev. This is because it becomes more apparent to me how much I _don't_ know, and I begin to wonder if I'll ever be good enough to attain the level of expertise that I aspire to.

What makes it more poignant for me is that I've pursued something like this before (sports career) and didn't quite make it (understatement), and I'm currently trying to reconcile myself with all the time and effort I invested in the process and how to ramp down from something that was my overriding ambition for so long. The good thing is that I've learned a lot of lessons from the experience and the main takeaway for me was ... if it isn't working, get out ... life is too short and there are many many other things to try your hand at.

The second thing that I liked about it was how it shows exactly why we should have empathy for other developers. I loved this section

 _For every dude I dismiss, every footnote in a roundup post, every guy deemed worthy of mere shrugs or raised eyebrows — all of whom are well-compensated for their work, no doubt — there is someone far away on the other end of the phone coaching him or advising him or loving him who wants desperately for him to succeed and be happy_

because one of the things I try to live by is this

 _Be kind, for everyone you meet is fighting a hard battle --- Plato_

I've had endured a few insults in my life as a dev ("You'll never be an engineer son, maybe a scripter or something, but never an engineer"), by people who thought I'd never make it as a programmer. Many years, later, here I am ... doing a pretty decent job at it, and I'll never forget all the kind people who encouraged me, took a time out to explain something to me ... again, just like I'll never forget the couple of times someone was a complete dick to me, because they thought I had no prospects as a developer.

I guess what I'm saying is, whenever you feel the need to dismiss someone who has no skills ... maybe take a step back and see if what they need is encouragement, coaching and a little bit of direction instead.

<comment><USER4> Speaking as someone who started literally from 0 in terms of developer knowledge a scant 3 years ago, and having been spat on, insulted and otherwise ridiculed for ever believing I could join the cult, I will say this: What doesn't kill you makes you stronger.

I had code with security holes that you could drive a Mack truck through. I learned security. I had sloppy copypasta code. I learned OOP. I had inefficient code. I learned memory management processes and how to structure code better for efficiency.

Could those who spoke down to me have been more helpful? More coddling? Not said anything at all? Sure. But who knows how that would have stunted my growth.

It's good to have compassion, and it's good to care. But if you really want to be the best at something, you're going to have to want it. And if you really want it, when someone beats you down, you're not going to stay down. You're gonna learn how to hit back.

As far as ultimately being where you want to be, sometimes it's simply the luck of the draw, and you have to understand how large a role that luck plays, as well, especially when you're being enchanted by stories of multi-million dollar startup investments.

<comment><USER5> _I will say this: What doesn't kill you makes you stronger._

This saying has always bothered me, Nietzsche or not.

What doesn't kill you, in the literal sense, can often cripple, maim, or psychologically impair you. It could make you hesitant, lose confidence, or refuse to try again.

Whether it's your life, or your professional career, the "failure or luck against adversity makes you stronger" bit seldom matches reality.

Sorry to take that sidetrack, but it happens to be a saying that I wish would disappear.

<comment><USER4> Agreed, it's trite, overused, and arguably obsolete phrasing. My point was more that we often face large degrees of adversity early on when we approach a difficult challenge. If we get up and walk away after we're knocked down in the beginning, we probably didn't want it that badly to begin with.

</comment></comment></comment></comment><comment><USER6> Analogy to startups: \- Filter out all the background noise just like athletes filter out the media. Stop caring about what [insert tech blog here] just posted about your competition, who got what funding, who's presenting at Disrupt. Focus on yourself - the rest is just fluff.

Analogy to personal development: \- If you love what you do and your best isn't cutting it at your role then consider a minor pivot. The best basketball and football coaches at the professional level are usually not the best players and yet they managed to find a role that leverages their love for the game while pivoting away from their physical disadvantages.

<comment><USER7> _The best basketball and football coaches at the professional level are usually not the best players and yet they managed to find a role that leverages their love for the game while pivoting away from their physical disadvantages._

Even players themselves can be in systems that don't work for them and they may not seem like good players until they get into a system that fits them. Minor pivots are a good thing and should be tried.

</comment></comment><comment><USER8> Doing what you're best at in a field with limited places for the best - <USER2> baseball - may set you up for failure.

There's a relevant concept in economics called comparative advantage - <http://en.wikipedia.org/wiki/Comparative_advantage>. Even if you are not the best at _anything_ , there are some things that are "enough" - that create surplus.

<comment><USER9> Another important economic concept that bears on this issue is "positional goods." Baseball abilities are positional goods because the value is almost all in being the best or among the best, and any other level of ability has no value to society. This is not the same as, for example logging. Half the ability in logging is half as useful economically.

</comment></comment><comment><USER10> >Worrying about the above-listed peculiarities of playing the sport professionally and actually playing it professionally are likely, in many cases, mutually exclusive.

There is a fantastic book called, "The Inner Game of Tennis" that helps describe that the ability to reach your max potential is to silence the self- criticizing ego and essentially just do it by seeing and feeling.

Anecdotal? Sure Interesting? Absolutely.

<comment><USER11> If you can find the video of the author, Timothy Gallwey, teaching a middle- aged out of shape lady in a muumuu to play tennis in one hour, it's really inspiring.

<comment><USER12> Here: <http://vimeo.com/6152948>

<comment><USER13> Video was worth watching.

</comment></comment></comment></comment><comment><USER14> Just because someone is doing something, it doesn't mean that is what they are best at. Maybe this particular baseball player would be much better at being a lawyer. But he wants to play baseball--so there he is, struggling in Florida.

How does one know what they are best at, anyway? I try to be reflective, and I ask people whom I trust what they think. Often the answers don't agree, or they reflect the subjectivity of each relationship (e.g. my lawyer telling me I'd be a good lawyer--but how would he know if I'd be a better cook?). Sometimes they are things that I actually don't like doing. So where does that leave me?

</comment><comment><USER15> I guess that's why baseball players are encouraged to remain even-keeled, and not be stuck in a high, or low. Just focus on execution, and forget about the drama, and mental noise.

</comment><comment><USER16> If you're doing your best at what you're best at, but in a field that gives you no pleasure, will success even be enough?

Doubtful.

</comment><comment><USER17> What about doing your best at something you're terrible at? Sometimes no matter how hard you try, you still make a little impact in a big industry.

If you got into your field of expertise with ease, find it lacking in challenges. with little impact, maybe it's time to try your best at something new & difficult and therefore perhaps more rewarding.

Even moving into a field with a wider scope, something critical that benefits others greatly. Success and job satisfaction can be found in many places other than 'being the best' or being in an enviable position.

<comment><AUTHOR> NB that the article is not about "doing your best" but about "doing what you are best at doing."

</comment></comment><comment><USER18> Baseball must be especially tough for players, considering it is mostly just individual contests over and over, pitcher versus batter. On team sports like basketball or hockey you can always rationalize that the rest of your team should have done more.

<comment><USER19> What?? The pitcher in particular absolutely depends on the rest of his defense.

Batters depend on those behind them to move them around.

I think golf or tennis would be a better analogy for your point.

</comment></comment><comment><USER20> It's like a less tongue-in-cheek version of the despair.com poster: <http://www.despair.com/fail24x30pri.html>

</comment><comment><USER21> Why would you even consider this question? I know why you would, but on a deeper level, spending time on this is not healthy or constructive.

Maybe you can't ever be at your best. Maybe you'll always improve, no matter how long you train. Idealistic? You bet your ass. It's taken me further in life than I ever thought possible--and it will continue to carry me.

Don't forget: they don't know who you are. You don't know fully who you are.

</comment><comment><USER22> I'm getting a 403 response.

<comment><USER23> Here's a mirror: <http://i.imgur.com/gZ4xB.jpg>

(well, it's more a simple screenshot of the page rather than a mirror, but you get the idea)

</comment></comment><comment><USER24> good writing for sure but so much set up for so little insight? I you're gonna do that you might as well just get to the point!

</comment><comment><USER25> Go blue ocean. If you're being true to your vision and individuality, it will always be good enough.

P.S. I'd like to qualify "good enough". I've poured blood, sweat, tears and spit into ideas that ended up being completely ignored by the world. I still consider them good enough though. I am still proud of what I accomplished.

You are the ultimate decider of what is good enough.

<comment><USER26> What blue ocean would you suggest a baseball player enter, pray tell?

<comment><USER25> I was simply responding to the title. I didn't read the blog post (misread it as an Ask HN).

Slap my wrist accordingly.</comment></comment></comment>


Brain: Searching for the first PC virus in Pakistan - <AUTHOR> http://www.youtube.com/watch?v=lnedOWfPKT0&fmt=22

<comment><USER2> I found the (non)reaction to Mikko's gift to be kind of odd. In western cultures, a gift like that would show a lot of thought and consideration and probably be quite sentimental. They didn't seem to know what to make of it. I wonder if it's just a difference in gift-giving traditions between Pakistan and the West.

<comment><USER3> I don't know. I thought they appreciated it when I gave it.</comment></comment>


From lawyer to coder - <AUTHOR> https://tech.grammarly.com/blog/from-lawyer-to-coder <comment><USER2> Neat-o!</comment>


Securely check your password in the Pwned Passwords database - <AUTHOR> https://github.com/sashee/haveibeenpwned-sha1-query <comment><USER2> Why not to write a script making a hash and querying HIBP directly, so people could run it on their machines ?

sashee.github.io will be aware of your: password hash without salt, IP, User- Agent, possibly your browser fingerprint.

<comment><AUTHOR> Nope, those are not sent anywhere, only to the HIBP API, you can check the code.</comment></comment>


Meltdown Proof-of-Concept - <AUTHOR> https://github.com/IAIK/meltdown/ <comment><USER3> I'm curious if someone can point me to any source that discusses how the next generation of CPUs that Intel, AMD, ARM might be working on is actually going to address this & the Spectre issue architecturally.. It's great that we have a potentially performance killing fix but the real "fix" or rather, solution, is to alter the architecture. Since I'm not an EE/CE dude... is anyone aware of where such discussions on the WWW might be taking place?

by the way, that PoC was intense. Makes you wonder if the NSA knew about it all along :)

<comment><USER4> > Makes you wonder if the NSA knew about it all along :)

Former head of TAO Rob Joyce said "NSA did not know about the flaw, has not exploited it and certainly the U.S. government would never put a major company like Intel in a position of risk like this to try to hold open a vulnerability." [1]

Who knows if that's true or not, though. Certainly the U.S. government has done exactly that many times in the past (like with heartbleed).

[1]: [https://www.washingtonpost.com/business/technology/huge- secu...](https://www.washingtonpost.com/business/technology/huge-security- flaws-revealed--and-tech-companies-can-barely-keep- up/2018/01/05/82ccbe18-f24e-11e7-b3bf-ab90a706e175_story.html)

<comment><USER5> It's odd to publicly state that they didn't know about it, because now if they don't do the same after the next big flaw comes out, the implication will be that they indeed knew and were quietly exploiting it. I thought that was why they generally don't comment on these things. The less-charitable assumption is that they'll make this claim every time regardless of whether it's true.

The claim that "the U.S. government would never put a major company like Intel in a position of risk" is obviously bullshit. TAO's job necessarily involves exposing companies both in the US and overseas to that kind of risk on a daily basis.

<comment><USER6> It's the type of announcement that makes me wonder if they had the chip makers incorporate it specifically for them to exploit.

<comment><USER7> > It's the type of announcement that makes me wonder if they had the chip > makers incorporate it specifically for them to exploit.

...sorry, _what_?

It makes you wonder if the NSA had chip makers incorporate speculative execution and caching because... timing attacks?

<comment><USER6> No.

It's just that it's highly suspicious that anyone is making any type of mention of it at all.

</comment></comment></comment></comment></comment></comment><comment><USER8> The Project Zero bug report (with PoCs/timeline) was also made public a few minutes ago [https://bugs.chromium.org/p/project- zero/issues/detail?id=12...](https://bugs.chromium.org/p/project- zero/issues/detail?id=1272#c3)

<comment><USER9> I wonder what happened to "This bug is subject to a 90 day disclosure deadline. After 90 days elapse or a patch has been made broadly available, the bug report will become visible to the public." Executive meddling?

Edit: Probably the 'extreme circumstances' bit mentioned in [https://news.ycombinator.com/item?id=16108434](https://news.ycombinator.com/item?id=16108434)

<comment><USER10> I think for a bug this big it is pretty understandable. So far, it seems clear the actions of all involved were in a good spirit of responsible disclosure.

<comment><USER11> Except if you are into *BSD. In that case you might want to label it "selective disclosure" instead of "responsible disclosure".

<comment><USER12> Well, since some of the BSD folks publicly stated that they'd ignore any embargo, that seems like a pretty predictable consequence. And in this case I understand that it took a while to develop workable mitigations. Immediate disclosure might have caused great harm.

<comment><USER13> Tarring all of the BSDs with the same brush is wrong, both in general and here specifically. There's also the matter of both Matthew Dillon and Theo de Raadt discussing this topic months or even years before Google Project Zero made its discovery.

* [https://news.ycombinator.com/item?id=16086047](https://news.ycombinator.com/item?id=16086047)

* [https://news.ycombinator.com/item?id=16074531](https://news.ycombinator.com/item?id=16074531)

* [https://news.ycombinator.com/item?id=16075744](https://news.ycombinator.com/item?id=16075744)

Moreover, the OpenBSD people have made some remarks about how it was commentaries in Linux patches and discussions on _LWN_ that actually let the cat out of the bag this time.

* [http://pythonsweetness.tumblr.com/post/169166980422/the-myst...](http://pythonsweetness.tumblr.com/post/169166980422/the-mysterious-case-of-the-linux-page-table) ([https://news.ycombinator.com/item?id=16046636](https://news.ycombinator.com/item?id=16046636))

* [https://news.ycombinator.com/item?id=16084404](https://news.ycombinator.com/item?id=16084404)

<comment><USER14> The bugs Theo was talking about were unrelated to these ones.

<comment><USER13> I did point to [https://news.ycombinator.com/item?id=16074531](https://news.ycombinator.com/item?id=16074531) . However, it is also wrong to err in the other direction as you have and to say that they were unrelated. Others have already made this point in [https://news.ycombinator.com/item?id=16075744](https://news.ycombinator.com/item?id=16075744) , which I also pointed to.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER15> This was the GitHub repo mentioned in the meltdown.pdf that was 404'ing until now. We have native Spectre replication code too. What still seems to be elusive is the JS-based Spectre impl (probably waiting at least for Chrome 64, though I confirmed via [https://jsfiddle.net/5n6poqjd/](https://jsfiddle.net/5n6poqjd/) that Chrome seems to have disabled SharedArrayBuffer even before they said they would which wasn't the case a few days ago).

<comment><USER16> This is the closest thing to a javascript implementation I have seen: [http://xlab.tencent.com/special/spectre/js/check.js](http://xlab.tencent.com/special/spectre/js/check.js)

from: [http://xlab.tencent.com/special/spectre/spectre_check.html](http://xlab.tencent.com/special/spectre/spectre_check.html)

<comment><USER15> Nice. Reviewing the code, it is as the PDF said where they are just constantly incrementing a val in the shared buffer to get a fairly precise timer. But it seems to be using the timing to determine across 256 indices (99 tries to check) to check cache hits. So just removing this timer is not enough, it just increases the surface area of bytes you have to read and sift through to see if you have other mem? Anyone have a writeup on this?

<comment><USER17> Isn't the high-precision timer required to detect a cache hit or miss-- as in, the side channel being exploited here is in the timing of a cache hit or miss; there's no data leaked directly into Javascript?

That's not to say that removing SharedArrayBuffer (and high-precision performance timers, which were removed a couple years back to mitigate some other timing-related vulnerabilities) is enough to completely eliminate Spectre; there might be other methods that can time accurately enough to reveal information.

(I might be completely wrong here, but this is my current understanding of the situation, at least.)

</comment></comment></comment></comment><comment><USER18> Moritz Lipp's twitter is actually interesting to follow. He is reconstructing images which do not fit into cache. Quite amazing.

[https://twitter.com/mlqxyz/status/950378419073712129](https://twitter.com/mlqxyz/status/950378419073712129)

(I personally do not have a twitter account but was looking for the paper and stumbled upon it, glad I did!)

</comment><comment><USER19> Linux 4.15 and the appropriate modules protect against the attack.

To test, set CONFIG_PAGE_TABLE_ISOLATION=y. That is:

        sudo apt-get build-dep linux
        sudo apt-get install gcc-6-plugin-dev libelf-dev libncurses5-dev
        cd /usr/src
        wget https://git.kernel.org/torvalds/t/linux-4.15-rc7.tar.gz
        tar -xvf linux-4.15-rc7.tar.gz
        cd linux-4.15-rc7
        cp /boot/config-`uname -r` .config
        make CONFIG_PAGE_TABLE_ISOLATION=y deb-pkg

<comment><USER20> I have CONFIG_PAGE_TABLE_ISOLATION on. I roll my own kernel and all that.

Trying the kaslr program right now, it's not figuring out the direct map offset and it's probably already been a minute or two. So it works?

EDIT: After 40 minutes, it has attempted all addresses and did not find the direct map offset.

<comment><USER19> It took about an hour for it to find the offset for me.

I think that the page isolation slows it down, even if it doesn't completely eliminate it.

The second test had something like a 0.05% success rate on my PC, and took over an hour to get a few dozen values read.

After trying this with the new kernel, I started up an AWS instance and ran the tests there. The first test (KASLR) succeeded within a few seconds, and the second test had a 100% success rate (read 1575 values in a few seconds).

<comment><USER20> Basically, the first test (kaslr.c) did not even work for me, and it scanned all addresses and wrapped around and started again.

You probably know this (saw you're the person I replied to initially), but for others reading this to check that it's on, "dmesg | grep isolation" should be able to tell you whether the page table isolation is on after you enable it in the kernel.

Given the other tests require the offset, I think I'm safe? I'm going to run it again just to be sure.

</comment></comment></comment></comment><comment><USER14> libkdump is really clean code and worth a read, nicely wrapping the inline assembly you need to do the flush+reload and keeping the algorithms in pretty simple C. It's worth taking a few minutes to read through it.

This code is from TU Graz; I assume this is from Daniel Gruss's team, who participated in the original research.

</comment><comment><USER21> High-level programmer here. Can someone explain please (already read the ELI5 in previous threads) how does the attacker extract the actual data from the processor L1 cache after tricking the branch prediction and have the CPU read from an unauthorized memory location?

I understood the "secret" data stays in the caches for a very short time until the branch prediction is rolled back, which makes this a timing attack but don't get how you actually read it.

EDIT

So perhaps someone can ELI5 me "4.2 Building a Covert Channel" [1] from the Meltdown paper which is what I didn't understand.

[1] [https://meltdownattack.com/meltdown.pdf](https://meltdownattack.com/meltdown.pdf)

<comment><USER22> Caveat: I am also a high level programmer.

My understanding is that the problem is that the data in the cache _isn't_ rolled back.

You fetch the secret data. You then fetch a different memory addressed based on the contents of the secret data e.g. fetch((secret_bit * 128) + offset) [1] so if secret_bit is 0 it's fetched the memory at offset into the cache, if secret_bit is 1 it's fetched the memory at offset+128 into the cache.

After the speculative work is rolled back, the data that it fetched into the cache still remains. You then time how long it takes to fetch offset and offset+128. If offset comes back quickly, secret_bit was 0. If offset+128 comes back quickly, secret_bit was 1.

_That_ is where the timing attack part comes in: "timing attack" refers to using measurements of how long something took to glean information, not that you need to do it quickly.

[1] In reality you do it on the byte level and use &, but I wanted to keep it to guessing a single bit to make it simpler.

<comment><USER21> > You fetch the secret data. You then fetch a different memory addressed based > on the contents of the secret data ...

I was under the impression that there is no interface to read data from the CPU caches and that the cache is managed by the CPU itself only.

<comment><USER22> Right, which makes it a bit of a tricky attack to pull off. But if you know what you're doing you can do some operation that requires memory address x and be reasonably sure it will end up in the CPU cache. If you then do an operation on memory address x, and it happens really quickly, and you do an operation on memory address x+128, and it happens a bit slower, you can assume that x was in the cache and x+128 wasn't.

<comment><USER21> Yes, I got the part where you can time if memory address X is in cache and X+128 isn't. But how does one read the data at memory address X?

<comment><USER22> You load it into a register. If you're trying to drive it from a high level language, I guess you can do something like an add which will get compiled into instructions to load it into a register first.

</comment></comment></comment></comment></comment></comment><comment><USER23> I have run the first test on several machines, with mixed results, but on my workhorses (ThinkPad x220, Zenbook UX305) the exploit seems to work.

I thought the recent kernel-/firmware-/ucode-patches should have prevented that.

EDIT: The other demos fail, though, as they should. _sigh_

EDIT: For some reason, demo #2 (breaking kaslr) works on my Ryzen machine, but not on the others. :-?

<comment><USER24> Spectre should work on most modern computers. There are no kernel patches in stable to prevent Spectre right now. Only Meltdown is mitigated by KPTI. The new Intel microcode and the kernel code to control it will propagate out in the next couple of weeks.

</comment></comment><comment><USER25> Looks like Intel SGX is at least vulnerable to Spectre attacks too: [https://github.com/lsds/spectre-attack-sgx](https://github.com/lsds/spectre- attack-sgx)

</comment><comment><USER26> First I read about this, so I thought "who's shorting Intel now I wonder", turns out it's the CEO [kinda]:

>"reports this morning that Intel chief executive Brian Krzanich made $25 million from selling Intel stock in late November, when he knew about the bugs, but before they were made public" ([https://qz.com/1171391/the-intel- intc-meltdown-bug-is-hittin...](https://qz.com/1171391/the-intel-intc- meltdown-bug-is-hitting-the-companys-stock-big-time-while-rival-amd-is- soaring/))

I assume he's supposed to now be prosecuted, that sounds like insider dealing? [I'd like to say "will be prosecuted" but ...]

<comment><USER27> as far as conspiracy theories go (i read that some days ago on reddit), he's wont be persecuted because he cooperates with the NSA. refuse to cooperate with them and join Nacchio and Qwest.

</comment></comment><comment><USER28> I am running a razer blade 2017 with ubuntu 16.04 and so far all of the PoCs have worked. I currently have my kaslr offset and I am now testing the reliability. So far it doesn't seem very good with a 0.00% success rate at 60 reads. It did take a while to find my kaslr offset with multiple passes through the entire randomization space so I need to stress my CPU more in order to improve the success rate of having successful branch speculations.

<comment><USER29> I installed the recent kernel release from Ubuntu, but the tests still working fine.

</comment></comment><comment><USER30> Not sure what this means, but while I'm mining Monero on the CPU with xmr-stak the PoC is thwarted.

First, the "Direct physical map offset" comes back wrong in Demo #2. Second, if I use the correct offset, the reliability is around 0.5% in Demo #3 - <USER2> not consistently... after a few tries it did come back with >99%

Basically, screw up your caches continuously.

</comment><comment><USER31> From the papers, these two bugs are also exploitable from ARM.

Does it mean a hacked IOS/Android app can also (in theory) sniff the password enter in system dialog as demo in the video?

       Realtime password input - https://www.youtube.com/watch?v=yTpXqyRYcBM

<comment><USER32> Important to differentiate between ARM the company, the instruction set architecture(s) and the specific implementation of those ISAs. The licensable nature of ARM means there very likely are (possibly undiscovered) implementations of the ARM ISAs floating around which are susceptible to Meltdown.

<comment><USER33> I was under the impression that they generally license the IP cores (or at least some IP blocks) to implement the ISA and downstream vendors don't implement those differently.

</comment></comment></comment><comment><USER34> MacOS is yet to have a patch for 10.12.6 (Sierra) to resolve this.

<comment><USER35> It is patched on Sierra, this was part of the 2017-002[0] security update on the 6th of December.

0: [https://support.apple.com/en-gb/HT208331](https://support.apple.com/en- gb/HT208331)

<comment><USER36> That link shows Meltdown in reference only to High Sierra, not Sierra. What am I missing?

<comment><USER35> You're right, can't believe I missed that.

Edit: See the archive[0] apparently I'm not going mad and it used to say that the patch was applied to Sierra and El Capitan, but Apple has since changed that.

0: [https://web.archive.org/web/20180105102220/https://support.a...](https://web.archive.org/web/20180105102220/https://support.apple.com/en- us/HT208331)

<comment><USER36> Thanks. I thought I was going nuts, too.

</comment></comment></comment></comment></comment><comment><USER37> Does anyone have a link to Linux PoC code for Meltdown that uses speculative branch execution?

I've only seen two implementations: one based just doing the access to kernel memory, catching the SIGSEGV, and then probing the cache. Obviously that could be closed by the kernel flushing the cache prior handing control back t user space after SIGSEGV. Doing that would have no impact on normal programs.

The second is by exploiting a bug in Intel's transactional memory implementation. But I assume Intel could turn that feature off as they have done so in the past. Since bugger all programs use it doing so wouldn't have much impact.

Which means the approach being take now is done purely to kill the speculative branch method (ie, Spectre pointed at the kernel). The authors say it should work, but also say they could not make it work. I haven't been able to find working any PoC for my Linux machines.

So my question is: is there any out there?

<comment><USER37> Never mind: [https://bugs.chromium.org/p/project- zero/issues/detail?id=12...](https://bugs.chromium.org/p/project- zero/issues/detail?id=1272#c2)

</comment></comment><comment><USER38> Can the videos be put on YouTube for convenience?

<comment><USER39> #1 - realtime password input - [https://www.youtube.com/watch?v=yTpXqyRYcBM](https://www.youtube.com/watch?v=yTpXqyRYcBM)

#2 - physical memory leak - [https://www.youtube.com/watch?v=kn0FopiF16o](https://www.youtube.com/watch?v=kn0FopiF16o)

the videos aren't very long, someone should compress it to <10mb as an animated gif and do a pull request to put it in the README

<comment><USER40> >the videos aren't very long, someone should compress it to <10mb as an animated gif and do a pull request to put it in the README

There's no need to use an awful format like gif, just embed an efficiently compressed video file with the <video> tag

<comment><USER39> I completely agree with the sentiment, however github currently does not support embedded video in markdown [1]

Animated gifs do work when embedded, but need to be <= 10mb [2]

[1] [https://stackoverflow.com/questions/4279611/how-to-embed- a-v...](https://stackoverflow.com/questions/4279611/how-to-embed-a-video-into- github-readme-md)

[2] [https://stackoverflow.com/a/46701929](https://stackoverflow.com/a/46701929)

<comment><USER40> Oh wow, that's crazy - especially 8 years since they said they'd look at it! Thanks for the info

</comment></comment></comment></comment></comment><comment><USER41> One of the reason I don't consider the timing attacks that important is that there are often easier ways to bypass ASLR.

<comment><USER42> Are there easier ways to read kernel memory?

<comment><USER41> The point is what reading kernel memory would be useful for.

<comment><USER42> One wonders why /dev/mem was ever read restricted to start.

<comment><USER41> Reading /dev/mem is far easier/faster and typically provides more data than this attack would.

</comment></comment></comment></comment></comment><comment><USER43> The secret program confirms what others have seen, it's not so much "read any physical memory" as "read memory in cache"

<comment><USER35> > "read any physical memory" as "read memory in cache"

You can force values from any memory to affect the cache in a predictable manner which enables you to read all physical memory. See [https://news.ycombinator.com/item?id=16108574](https://news.ycombinator.com/item?id=16108574) or read the paper yourself [https://meltdownattack.com/meltdown.pdf](https://meltdownattack.com/meltdown.pdf)

<comment><USER44> > You can force any memory into the cache so yes it's is read any physical > memory.

Is there a direct method for that or do you mean that you can repeatedly try reading memory addresses until the address that you want to access is actually in the cache prior to your access?

<comment><USER35> The exploit is based on reading values that you shouldn't be allowed to access in speculative execution and then using the returned values to create persistent changes in the cache(they persist even after the CPU detects your illegal access). Those persistent changes are then read via a side channel attack.

So you read any address you want speculatively and then use the result to prime the cache in such a way that you can determine what the value you read speculatively was. This works because modern operating systems map kernel space addresses into normal processes and to make syscalls faster.

I'd recommend reading the paper[0], it's fascinating stuff.

[https://meltdownattack.com/meltdown.pdf](https://meltdownattack.com/meltdown.pdf)

</comment></comment></comment></comment><comment><USER45> These two bugs (Meltdown and Spectr) are really very speculative things. It is like when human beings became aware of astroid orbits they thought that earth is in danger of being hit by one. Now that is indeed a theoritical possibility but what are the chances? These two bugs have been existent for 20 years and there is no known exploits of them. In the GitHub demos also they mention that the demos will work only if "For this demo, you either need the direct physical map offset (e.g. from demo #2) or you have to disable KASLR by specifying nokaslr in your kernel command line." \- So you basically start with a broken system to exploit these bugs.

<comment><USER46> This is literally a PoC. It's too late for the standard "I can't imagine how to exploit this so surely it cannot be done" fallacy. You are looking at an example of how to do it.</comment></comment>


Ask HN: Rate my MVP (Newspaper iOS App for the Guardian) - <AUTHOR>

Link is http://todaysnewsapp.com<p>Based on the concepts of Phil Gyford's Today's Guardian http://guardian.gyford.com <comment><USER2> Sounds great! I signed up.

When will the app actually be available?

<comment><AUTHOR> We're aiming to submit to the App Store in 7-10 days. The iPad app is done, we're working on making it universal and testing on multiple devices.

</comment></comment><comment><AUTHOR> Clickable link: <http://todaysnewsapp.com>

</comment><comment><USER3> Nice design!

How it renders on iPhone/iPod Touch smaller screen?

<comment><AUTHOR> Do you mean the website or the actual app? We're in the process of making it universal but so far it looks great on the iPhone as well.

Will post iPhone/iPod Touch screenshots soon.

<comment><USER3> The actual app.

I'll wait the screenshot, thanks.</comment></comment></comment>