Highly Available Block Storage - <AUTHOR> https://www.digitalocean.com/features/storage/ <comment><USER2> Spun one up and ran some quick numbers on a 100GB volume:

root@ubuntu-1gb-nyc1-01:~# time dd if=/dev/disk/by-id/scsi-0DO_Volume_volume- nyc1-01 of=test.dat bs=1024 count=10000000 10000000+0 records in

10000000+0 records out

10240000000 bytes (10 GB) copied, 58.0655 s, 176 MB/s

real 0m58.248s

user 0m2.608s

sys 0m41.604s

Some quick observations:

* Easy to add one when creating a droplet; by default they let you create volumes with these sizes: 100GB, 250GB, 500GB, 1000GB, 1.95TB; it's also really easy to create your own size.

* You can resize in any increments; took about 4 seconds to go from 100GB to 110GB with no downtime; you obviously need to resize/manage the mounted volume yourself.

* [Edit 1] Deleting the droplet does NOT destroy the volume. Worth keeping in mind when you spin them up/down.

* [Edit 2] Remounting an existing volume to a new droplet was quick and painless.

<comment><USER3> Don't use dd for those tests it's really bad, especially on VMs.

<comment><USER4> What should be used for I/O throughput tests instead?

<comment><USER5> I usually use Bonnie++ when testing disk performance. [http://www.coker.com.au/bonnie++/](http://www.coker.com.au/bonnie++/)

<comment><USER6> The only thing worse than dd is bonnie++. Please, folks, use fio, or at least iozone, with multiple threads and/or a queue depth greater than one.

<comment><USER7> Are you sure? Bonnie++ has byte-wise and block-wise tests. Yes, the byte-wise tests are CPU bound (as expected), but I have not seen that for the block-wise tests on any machine so far?

</comment></comment></comment></comment></comment></comment><comment><USER8> Reminder that just a few weeks ago DigitalOcean rolled over on one of their customers and took down 38,000 websites after receiving a claim of infringement from the NRA against a parody site hosted on surge.sh:

[http://motherboard.vice.com/read/nra-complaint-takes- down-38...](http://motherboard.vice.com/read/nra-complaint-takes- down-38000-websites)

<comment><USER9> Reminder that you have to act on abuse notifications sharpish. You're providing a service, it's on you if you ignore abuse notifies.

"We received notice on behalf of a trademark holder that a customer of DigitalOcean was hosting infringing content on our network. DigitalOcean immediately notified our customer of the infringement, and the customer was given a five day period to resolve the issue. The infringing content was not removed within the specified period even though several notifications were issued. Per DigitalOcean's terms of service, a final reminder was issued to our customer and, when no action was taken, access to the content was disabled. The infringing content was subsequently removed by the customer and all services were restored in less than two hours."

<comment><USER10> That statement is problematic, namely this:

> The infringing content was not removed within the specified period even > though several notifications were issued.

You don't have to remove content under the DMCA, you can also file a counter- notice which gets the content host off the hook and then the matter goes to court[0].

But that also assumes DMCA which, if memory serves, was not in play here. It was a trademark complaint, which DigitalOcean has no responsibility to resolve.

Ultimately DigitalOcean's response, even with that statement, seems at odds with how the law is actually written. The other party also claimed they did respond to DigitalOcean, they just never removed the legal parody material which is their right.

DigitalOcean's understanding of the NRA's rights is more expansive than the law itself. Effectively their trademark policy is to automatically side with the trademark holder, irrespective of fair use[1] (see page 9+).

[0] [https://en.wikipedia.org/wiki/Digital_Millennium_Copyright_A...](https://en.wikipedia.org/wiki/Digital_Millennium_Copyright_Act#Title_II:_Online_Copyright_Infringement_Liability_Limitation_Act)

[1] [https://apps.americanbar.org/litigation/committees/intellect...](https://apps.americanbar.org/litigation/committees/intellectual/roundtables/0506_outline.pdf)

<comment><USER11> It's against DO's terms of service to "use the Services in violation of the copyrights, trademarks, patents or trade secrets of third parties", which appears to be the issue here.

[https://www.digitalocean.com/legal/terms/](https://www.digitalocean.com/legal/terms/)

<comment><USER12> > Effectively their trademark policy is to automatically side with the > trademark holder, irrespective of fair use[1] (see page 9+).

Did you miss that? It's not infringement if it falls under fair use. They were not following their TOS because they did not confirm the content was infringing a copyright

<comment><USER13> Expecting a hosting provider to wade into fair use waters hand-in-hand with you is generally unwise.

The vast majority will not unless you are a large customer with your own legal staff on retainer to provide the appropriate legalese/notices/etc.

<comment><USER12> I don't expect that, but I do expect that a company not immediately kowtow to an infringement request if there is some ambiguity as to whether it's infringing.

That said, the original reporting on this and the statement from DigitalOcean are at odds (Motherboard's update with DO's statement), and since I haven't verified either, I'll retract any specific support for either side of this particular instance.

<comment><USER13> > I don't expect that, but I do expect that a company not immediately kowtow > to an infringement request if there is some ambiguity as to whether it's > infringing.

That requires money for a lawyer to evaluate it. If the customer has their own legal staff that does this and relays that opinion to the host, as well as being large enough to cover any legal costs DO might incur, DO would be fine with it.

You are basically saying you are entitled to using DO's legal staff and financial resources in addition to the hosting you've paid for.

<comment><USER12> > You are basically saying you are entitled to using DO's legal staff and > financial resources in addition to the hosting you've paid for.

No, what I'm saying is that DO _must_ already do this to some degree if they are handling requests, as otherwise I could send letters claiming trademark/copyright infringement for any number of things and get many customers shut down. If they have internal guidelines for what they do in cases when trademark/copyright infringement, I expect they follow those. I also expect that those policies do the minimum legally required of them. That's not because it's cheaper and garners good will from customers (it does), but because to do otherwise is taking sides in a legal situation without being an appointed arbiter of the law. Not only is this excessive, but it's anti-customer.

If DO is doing what they think they must by law, I have no problem with that, as long as that is clearly explained. In the case we were previously talking about, the statement from DO (at the motherboard article) is somewhat ambiguous as to why they did what they did. _Per DigitalOcean's terms of service, a final reminder was issued to our customer and, when no action was taken, access to the content was disabled._ Was the take down required by law, or was DO overly aggressive in handling it? Without a statement as to why, (and I think that given some people's assertion that they went beyond what was legally required of them), their reasoning is somewhat ambiguous, and harder to call into question. If they clearly define they enforced their TOS based on what they believe is a legally required of them, then we can look at the law and their actions and evaluate whether that's true, and if it's not, DO can learn from the experience or be called out as a company that is capricious in their execution of the law.

What it boils down to is that "We received a complaint infringement. We enforced our TOS and shut down access to the content in question." leaves a lot open for assumption. I would be much happier if it was "We received a complaint infringement _and as we believe is legally required of us_ we enforced our TOS and shut down access to the content in question." It's a small change, but it allows customers (and critics) a much clearer view on how DO handles situations like this, and allows for the public to make an informed choice on whether they think DO was correct in their actions (whether they really were legally required to do so). It's subtle, but I think it's a very, _very_ important distinction.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER14> Don't be confused: the article makes the mistake of comparing DOs new block storage service with other companies object stores. EBS is the competitor to this, not S3. Same for gce persistent disks and azure drives.

Unfortunately this means the pricing comparison is just wrong.

<comment><USER15> Ohh

</comment></comment><comment><USER16> I think this might be a mistake. Ever since Joyent's commentary on one of the big Amazon EBS failures in 2011 [1] [2] [3], I've been suspicious of all network-attached block storage. Then again, I haven't heard of any big EBS failures recently; I wonder what changed.

[1]: [https://www.joyent.com/blog/on-cascading-failures-and- amazon...](https://www.joyent.com/blog/on-cascading-failures-and-amazons- elastic-block-store)

[2]: [https://www.joyent.com/blog/magical-block-store-when- abstrac...](https://www.joyent.com/blog/magical-block-store-when-abstractions- fail-us)

[3]: [https://www.joyent.com/blog/network-storage-in-the-cloud- del...](https://www.joyent.com/blog/network-storage-in-the-cloud-delicious- but-deadly)

<comment><USER17> Network block storage isn't inherently broken, the initial EBS implementation was frankly just unreliable.

We've not had anything like those dark days with Persistent Disk. It's still true that having your storage across the network opens you to _networking_ failures taking out your storage, but the gain in durability and maintenance pays for it (in our case, live migration would just be crazy with local spinning disks, we tried it didn't work).

Disclaimer: I work on GCE, and we want your business ;)

<comment><USER18> > in our case, live migration would just be crazy with local spinning disks, > we tried it didn't work

It looks like Exoscale does live migrations with locally attached SSDs.

<comment><USER19> Then they'll run into the same problem anyone does doing that: migrating reasonably-sized block devices across a reasonable network takes an unreasonable length of time.

Been there, done that, don't want to sit staring at consoles waiting for live disc migrations ever again.

<comment><USER18> My guess is that they live migrate VMs only before a planned maintenance. Let's say they use a 10 Gbps network, and they use only 1 Gbps of bandwidth for migrating data, then migrating a 200 GB disk would take something like 1 hour, which sounds OK. Can you share some details about your experience?

<comment><USER19> You're missing a subtlety: it's a _live_ migration. You've also got to migrate the data the VM's writing while the migration is going on. Depending on how your network's set up, it might well be possible for the VM to saturate out the migration process. Plus this won't only be planned maintenance, it'll be to get off dodgy hardware, too, including spannered RAID controllers where you _really_ don't want to risk hanging around. Add in that you're likely going to be moving sets of discs at once (so possibly a couple of TB at a time), rather than individually, and you're very quickly looking at spending a day (or a night) at a time watching in case there's a network blip which means you need to restart any of them. This does not lend itself to a peaceful, happy existence.

It doesn't take doing this very often to make you realise that this is fundamentally backwards: you want the disc data already present on more than one storage server so that if one goes pop, you're not stuffed. Once you've done _that_ , you can make the observation that hardware RAID is no longer necessary, and save yourself a layer of complexity.

<comment><USER18> Thanks a lot for your detailed answer. I understand your arguments, but Google Compute Engine Local SSDs support live migration [1], which seems to prove it's possible, despite the difficulties. Any advice about this?

[1] [https://cloud.google.com/compute/docs/disks/#data_persistenc...](https://cloud.google.com/compute/docs/disks/#data_persistence_on_local_ssds)

<comment><USER19> Yeah, it's possible. Just not pleasant. What they're doing (and how unpleasant it is to operate) will depend on exactly what they mean by:

> Your instance might experience a short period of decreased performance

and how tolerant they are of unplanned reboots. I suspect what it means is that they'll throttle your VM (or maybe pause your IO) so it can't interfere with finalising the migration.

That being said, this is Google. They've probably thrown more man-hours at this than anyone else would think sane. I'll note that this is in the context of automatic migration away from "maintenance events" (whatever that covers) - it sounds like they think they've automated away a lot of the reasons we were having to keep an eye on things, but they're still vulnerable to hardware failure (obviously).

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER20> This is EXACTLY the thing that I need for one of my droplets! I love how there is nothing "special" about it - it's just a disk that you can attach to a droplet. I'm sure that under the hood there's some kind of magic going on, but it looks like it's nicely abstracted away. This is what I hoped block storage would turn out to be - here's a block device, use it like one.

As soon as this rolls out to the region I've got that droplet in, I'm going to pull the trigger on it. I might even spend the effort to migrate my droplet to a supported region just to get this.

<comment><USER21> Same here. My $5/month droplet is sufficient for all my needs, except it's a bit limiting on storage. $2/month for an extra 20GB, doubling my storage, would be great. I don't want to migrate anything and I don't care about more CPU or RAM, I just want more space!

<comment><USER22> Totally agree, this was their main limitation. Glad they have now solved it.

</comment></comment></comment><comment><USER23> I have been asking for non-SSD on DO for a long time now. My heart jumped when I saw the HN title, only to be dashed on the rocks.

What are us data nerds supposed to do? We want to take 10 terabytes, run a batch process on it, keep the 20TB, then continue with about 5GB of working data until the next month's terabyte comes in, then we want to batch through the 21TB. Right now the price slider doesn't even go up to 21TB, and clicking on the "need more storage button" doesn't go anywhere, but I'm assuming it would be $2100 / month which is more than 3x as expensive than vanilla S3.

<comment><USER24> You're looking at the wrong market.

At Hetzner, you can rent a dedicated server with 2x3TB drives for about $25/month ($0.005/GB, to scale to multiple servers, use Ceph) and a few larger machines for under $100. At OVH you can buy object storage for around 1c/GB and rent a few dedicated servers for less than $100/month, or use their cloud.

If you go to the _really_ low end, time4vps will give you a 1TB VPS for 2EUR/month, if you pay for 2 years (and they give you 4x the storage as bandwidth).

I don't work for any of these companies but I have services with all of them.

<comment><USER23> Yeah, but I like the DO per hour billing, design, and API. Other than their insecure-by-default SSH key thing, [https://digitalocean.uservoice.com/forums/136585-digitalocea...](https://digitalocean.uservoice.com/forums/136585-digitalocean/suggestions/9307569-return- the-droplet-s-ssh-public-key-as-part-of-api) that is.

<comment><USER25> How many hours does it take to load 10TB and then reduce it to 5GB? ;)

<comment><USER23> Actually because I use the DO API and lots of small instances that work together, not that long. Right now I use S3, which blows because I hate the S3 cli and API, so I would love a better solution.

<comment><USER26> Not sure if it's _much_ better for you use case, but Backblaze introduced an S3 competitor called B2 a few months ago.

Price calculation is straightforward:

\- $0.005/GB/month for storage

\- $0.05/GB download

\- $0.004 per 10k downloads

There is a base free tier as well.

[https://www.backblaze.com/b2/](https://www.backblaze.com/b2/)

<comment><USER23> Wow thanks for this! I'll definitely use these guys in the future.

</comment></comment></comment></comment></comment></comment></comment><comment><USER27> It's the same price as AWS's General Purpose SSD EBS volumes.

[https://aws.amazon.com/ebs/pricing/](https://aws.amazon.com/ebs/pricing/)

<comment><USER22> Yep.... Digital Ocean is much more competitive for instances though and out going bandwidth.

<comment><USER28> At the cost of significantly worse networking (VPC is _fantastic_ ), no object storage mechanism, no equivalent to IAM for instance permissioning, and no autoscaling (seriously? in 2016? I had to go look to make sure I wasn't misremembering--that they recommend something like DOProxy should make the lot of them faintly embarrassed).

DigitalOcean may have some value at the basement level of compute, but as a professional in this area there is literally no situation where I would use DigitalOcean right now because I value my time. AWS is already laughably cheap, the tooling is overwhelmingly superior, the resources available are better, and you aren't duct-taping together half-solutions and reinventing every wheel. (This is less an endorsement of AWS and more an endorsement of Not DigitalOcean; GCE is more than fine, Azure has some ugly bits around autoscaling that I don't like but you can get by.)

</comment></comment></comment><comment><USER29> This helps me with a nicer deployment setup. I was always keen on 'rebuild from scratch' rather than 'update stuff and hope you're idempotent and have captured all changes' but transitory data was always the problem. Now I can start building a new updated droplet and the only downtime will be that needed to detach and reattach the block storage containing the db etc.

Anyone see a flaw in this? (I know there are other ways to achieve similar benefits - my files could be on S3 and the database could be a separate droplet etc but these introduced various drawbacks and added complexity)

<comment><USER30>
      > Anyone see a flaw in this?

Perhaps not a flaw, but some issues with your setup are implied.

If you're rebuilding from scratch _because_ you're not sure that you can update things, then you're probably in need of a configuration management tool (I'm a big fan of saltstack[1], mostly because I don't like Ruby or DSL's, but there's lots of options out there[2])

If you're worried you're going to lose transitory data, it _sounds_ like you don't have a trusted and tested backup/archival/recovery process in place. So having it stored on a single EBS / DO BS / etc means you're still exposed. If you're rebuilding and rolling data over, in this scenario, I'd be copying, rather than relocating, any precarious data repositories.

[1] [https://saltstack.com/](https://saltstack.com/) [2] [https://en.wikipedia.org/wiki/Comparison_of_open- source_conf...](https://en.wikipedia.org/wiki/Comparison_of_open- source_configuration_management_software)

<comment><USER31> I tend to avoid those config management tools other than for basic bootstrapping exactly because while you can use those too to recreate from scratch, when you don't do that, you leave the door open for undocumented, unknown state, since most of them basically take a system in an unknown-but- hopefully-mostly-consistent state and try to bring them to a known state.

But they'll only be in a known state in that case if your setup is extremely comprehensive.

In practice I've seen too many config management tools where long running servers have ended up in unknown states because changes have been applied, and subsequently changes have either been made outside of the toolchain, or changes have been done to the config in ways that doesn't let the tool know what has changed, or the tools simply doesn't have a way of comparing machine states without comprehensively enumerating everything on the server (e.g. people running Ansible playbooks that adds X, subsequently removing the requirement X from the playbook, and going about without considering whether or not X will interact with Y which they've added later).

As a result, I see rebuilding from scratch as largely orthogonal to whether or not you use a configuration management tool or e.g. build VM images that you replace wholesale, or whatever you do: You should rebuild from scratch regularly, as coupled with a test-suite it's the only realistic way of knowing whether or not you've left anything out of your build process.

My biggest caveat with config management systems is that they tend to end up encouraging live changes to a setup, instead of a build-test-deploy cycle. Sometimes that's necessary, but to me that's a last resort.

<comment><USER32> I actually agree with your overall point but generally speaking if someone is making changes outside of your standardized toolchain you have a human problem. Emergencies aside, you use those tools for a reason. Straddling the fence is almost the worst of both worlds.

<comment><USER31> Agreed. But in my experience, the easier you make going outside the process the easier it becomes to invent excuses for why it is ok. A lot of my job involves making the right thing to do the path of least resistance, because when it isn't, because humans overall tend to be the cause of a whole lot more of the problems than the servers.

</comment></comment></comment></comment></comment><comment><USER33> I like their straight forward pricing. $0.10 USD per GB per month. No IOPS limits.

That said, how do you prevent a rogue droplet from going crazy and hogging up all of the SSD I/O?

<comment><USER34> Unless I missed it, I saw "no need for complicated formulas to determine the overall cost for transactions or IOPs limit," which I do not read as "no IOPS limit." I was looking but could not find any word on performance.

<comment><USER35> I read it as they either silently limit you, or have a free for all and let the traffic duke it out for iops.

</comment></comment></comment><comment><USER25> Setup: find it's name, format and mount it, as if you were adding an SSD to a desktop.

[https://www.digitalocean.com/community/tutorials/how-to- use-...](https://www.digitalocean.com/community/tutorials/how-to-use-block- storage-on-digitalocean)

</comment><comment><USER36> This has been a much requested feature and I'm sure it will be very popular. I'm still reminded of this quote though:

" _He was a bold man that first ran a production database on a brand new block storage service!_ "

</comment><comment><USER37> I love how DO focuses on what matters the most: Inexpensive VMs and scalable block storage.

If I had to pick two, those would be them!

<comment><USER35> Well except being 4 years late to the block storage game? Seems they have an uphill fight. Aws and gce match them on low end droplet price and offer much more. No local ssd but not sure what % of apps really need local ssd. DO effectively forces you to pay for local ssd for all of your servers.

<comment><USER38> Well, they beat Linode..

With DO, at least there is an additional option available for users.

With GCE and AWS, the outbound bandwidth is expensive. 1TB = $90 (AWS, GCE) vs 1TB included (Linode, DO).

<comment><USER39> This. I'm always a little confused by how people even evaluate the big three clouds, with prices per request and all bandwidth "not included". It's so strange coming from cheap dedicated servers with typically 10tb bandwidth/month "included". I mean, what do people _do_ in the cloud that requires 100gb of storage, and insignificant transfer?

</comment></comment></comment></comment><comment><USER40> Thank god! Highly Available Block Storage. From Digital Ocean. Great! Now I can finally store all the 300mb/s streaming in on my server. Oh wait. I cannot, because DO cancelled the service again. Bummer.

<comment><USER41> Cancelled which service?

</comment></comment><comment><USER42> Sweet. This makes digitalocean much more appealing as a potential substrate for a kubernetes cluster.

<comment><USER43> I just migrated my Kubernetes/Rancher stack from NYC2 to NYC1 in order to use block storage. Eager to see whether this plays well with GlusterFS.

</comment></comment><comment><USER20> I can't wait for this to roll out to more regions.

This is EXACTLY the thing I need for some stuff I'm working on!

</comment><comment><USER44> Going to perform some very basic DD i/o benchmarks using: [https://haydenjames.io/web-host-doesnt-want-read- benchmark-v...](https://haydenjames.io/web-host-doesnt-want-read-benchmark- vps/)

Will post results.

</comment><comment><USER45> What's the backend? Ceph?

<comment><USER46> Ceph is not block storage -- ceph is _object storage_

<comment><USER47> It's actually both, and a file system.

<comment><USER46> Oh I guess I thought ceph was all about _eventual_ consistency. I didn't know it was strongly consistent like what I expect from block storage.

</comment></comment></comment></comment><comment><USER48> How is it taking huge cloud providers so long to catch up with things we do self hosted every day? It obviously has to be well engineered, yet it's relatively simple. Woefully poor performance too.

</comment><comment><USER49> Some early benchmarks about the new block storage, [https://simos.info/blog/trying-out-lxd-containers-on- ubuntu-...](https://simos.info/blog/trying-out-lxd-containers-on-ubuntu-on- digitalocean-with-block-storage/)

I did not get good speeds and I am wondering why that may be...

<comment><USER34> > The immediate benefits are that the latency is much lower with the new block > storage

I think you might have misread the units: locally attached 50105 us (50 ms) vs block 546 ms.

The throughput numbers are on par with AWS and GCP block storage. This seems reasonable aside from the high latency.

</comment></comment><comment><USER50> A bit pricey for the long term but great if you just need to add some disk space to your vm and don't need the other improvements more expensive vms give you.

I use DO mostly to compile stuff on Linux when i don't have access to a physical server, and storage size is always a problem.

<comment><USER46> The purpose of the block storage in this instance isn't about giving your vm/droplet more space. It's separation. That way any data that's on that device can be attached to another vm/droplet. It probably would be more cost effective just to upgrade the vm/droplet if space were a concern. It's at least how we've marketed the same feature for the past year or two via Codero's portal. Not to say I don't like how DO has entered the space: keeping it simple.

<comment><USER51> Exactly! It's about having an easy upgrade path, generally reducing the amount of work for common operations, and for more flexibility engineering your cloud architecture. It's nice that it happens to be SSD, but highest performance or lowest cost per GB isn't necessarily the only cost saving factor.

This a main reason I still use Amazon AWS: I can create an instance and if it doesn't perform, upgrade it until it does. Then when I'm finished, kill the instance and save the volume. Next time I need it, just create the instance for the job, perhaps at spot pricing, then kill it again.

<comment><USER50> What you describe was already possible on DO using snapshots (that can be stored for free), and that's what I usually do too.

</comment></comment></comment></comment><comment><USER52> Edit3: Remountable/movable flexible storage for DO instances is what this gives you and it's kind of pricey. The comparison to B2 is not valid. Leaving the original mess for posterity.

\---

TWENTY times (edit) the price as B2 from Backblaze ($.10 vs $.005 per GB per month). It is one of the more expensive ones. But that gets you two things:

* (moot See edit 2) SSD! (significant iops improvement)

* (moot See edit 2) No transactional costs! (not sure if just between digital ocean instances, but they say none)

Improved performance and no transaction costs MAY (edit) be worth it for some applications.

Edit1: made it an order of magnitude cheaper in my head after looking at backblaze. It is no where close to the same price. Thank you for the catch, scq!

Edit2: and I'm just all kinds of off on this. Block not object storage. Essentially storage you can mount and move between digitalocean instances. That makes no transaction costs moot. You still have to get data out of the instance.

Thank you all for quickly catching how backwards this post was! I need coffee.

<comment><USER53> B2 is blob storage, like Google Cloud Storage or Amazon S3. What they have launched is SSD block storage, like Google persistent disks or Amazon EBS.

(The TechCrunch article also got this confused.)

<comment><USER54> > Amazon EBS.

Not more like amazon EFS?

<comment><USER55> No. EFS is NFS in the cloud which is NAS.

</comment></comment></comment></comment><comment><USER25> I don't suppose DO or anyone else is already working on an Ansible addon for this?

</comment><comment><USER56> I benchmarked the new block storage and it is not that faster than the virtio disk.

<comment><USER52> Um. Can you provide context? What virtio setup? How much faster? How are you accessing each? This is the reason benchmarks are often worse than useless.

</comment></comment><comment><USER13> $.10/GB block storage is too expensive.

[https://www.online.net/en/dedicated-server/rpn- san](https://www.online.net/en/dedicated-server/rpn-san)

There are several places you can get ~5 TB for +/-10% of the 1TB price at DO.

DO is offering a SAN at Object Store prices. :/

<comment><USER57> So is that service. You linked to the non SSD version. Their SSD version is $0.12/GB</comment></comment>


Outsmarting the smart meter - <AUTHOR> http://honeynet.org/node/1179 <comment><USER3> I'm saddened by the naïveté (or perhaps it's simple incompetence) of the smart meter manufacturers. Why isn't all communication encrypted? Why doesn't each meter have a unique public/private key pair associated with it? That way even if a key was extracted from a meter, it wouldn't be possible to use that key to access any other meter. It's not like there's a shortage of 1024-bit primes. Each meter in the world can have a unique 2048-bit key.

Maybe this isn't important for initial deployment. Maybe each meter is simply read-only. But, eventually, I think utilities want to be able to signal to the meter and associated devices in the house when to turn on and turn off. E.g. temporarily shut off air conditioning during a demand peak. I don't want some random hacker in another continent communicating with my meter, either for malice or for lulz.

Anyway, that's just my simple view on how things should be done. Obviously the real world isn't nearly as paranoid.

<comment><USER4> as far as i can tell the ability to manage financial risk of non-payment is the business driver for smart meters despite the rhetoric of 'smart'. electricity companies hate the fact that you consume energy and then get billed in arrears since obviously some people can't/won't pay. smart meters let you move instantly to a pre-pay plan where the risk moves from the provider to the user... it's also a more graceful method of cutting people off - you can just transition to a pre-pay method with a fixed credit limit so that it's clear what is going to happen when you get to zero - this is useful if there are follow-on liabilities ie, people depend on power if they have medical needs etc

<comment><USER5> From my experience (I work in this industry) the biggest driver is definitely billing, but not quite what you picked out.

What providers really want is to move to tiered and time-of-use pricing across the entire grid. Most utilities already do this with the various commercial tariffs, but that only accounts for about 40% of the total grid. All of us, in the near future, will be paying rates that vary by how much we use AND (more critically) WHEN we use it.

It's not really something to fear, for a lot of people this is going to mean lower utility bills in general. For the folks running swimming pools and the like, things are probably going to get a bit more expensive.

This is a great place for startups, btw. There are a lot of interesting opportunities around helping people make sense of the changing landscape surrounding utility billing.

</comment></comment></comment><comment><USER6> If I were designing secure SCADA protocols, the first thing I'd think of is the ways in which we communicate with submarines. Are any of those protocols public?

<comment><USER7> Submarines communicate using extremely low frequency radio transmissions to penetrate water. They have a bandwidth of 2-3 characters per minute, so the messages are almost exclusively orders to surface and switch to standard satellite based communications. Oh, and the transmitter requires a very unique piece of land with low ground potential that only exists in a dozen or so places worldwide.

Completely impractical for SCADA systems. :)

<comment><USER8> I'm really interested in learning more about underwater ULF - can you point to any keywords, papers, or milspec prefixes to read up on?

I've explored all the ISM bands - I just want to play with something even slower/longer range whilst sailing (in international waters, of course :)).

<comment><USER9> There a little here: [http://en.wikipedia.org/wiki/Communication_with_submarines](http://en.wikipedia.org/wiki/Communication_with_submarines)

and interesting detail here: [http://en.wikipedia.org/wiki/Project_Sanguine](http://en.wikipedia.org/wiki/Project_Sanguine)

<comment><USER8> Thanks guys.

I was familiar with Project Sanguine, but had hoped there were less ambitious public projects I had overlooked (perhaps closer to VLF which operates <30m seawater depths).

After reading about the Navy E-6B aircraft, which trails a 5km antenna behind it to communicate with subs, I had presumed modern 'submarine -> other underwater radios' were akin to large commercial fishing trawling nets - <USER2> the really long antennas were packed into hilbert curves and epoxied to the hulls or something.

Anyhow, it seems 'acoustic modems' using 'CSMA' [1] are the norm for commercial underwater ROVs (such as James Gosling's 'wavegliders' [2]).

[1] [http://www.mit.edu/~millitsa/resources/pdfs/royal.pdf](http://www.mit.edu/~millitsa/resources/pdfs/royal.pdf)

[2] [http://liquidr.com/prodserv/wg/gateway.html](http://liquidr.com/prodserv/wg/gateway.html)

</comment></comment></comment></comment></comment><comment><USER10> As a consumer, you can usually still avoid the installation of a smart meter. Ask yourself if it's practical that a washing machine starts in the middle of the night and wakes you up you or your neighbours and the laundry will have wrinkles in the morning? You can buy old refurbished analog/digital meters for cheap and measure what you want and it will consume no power nor will it send your confidential data (privacy). And you can combine it with a Arduino if you want the data on your computer. Many smart meters also send the data over a GPRS modem every 15min (have an inbuilt SIM card), that you cannot snoop so easy.

The whole smart meter movement is not about technical advancement, it's about they want that you pay higher prices for consuming less and they don't have to build a better power grid infrastructure in the next few years. It reminds me of the telcos 15 years ago when phone calls and internet over modem was cheaper in the night when most people sleep, and very expensive during working hours.

<comment><USER11> > You can buy old refurbished analog/digital meters for cheap and measure what > you want

But can you use them? New Zealand power retailers install their own meters - you can't supply your own. You can certainly install additional metering of your own but you can't get rid of the smart meter.

Are things different where you live?

Also, I disagree with your final paragraph. Our local distributors have been spending lots of money installing new lines and upgrading the network, and the retailers are changing the meters because it means we get an accurate bill every month (rather than an estimate for two out of every three months) and they don't need a meter reader to visit every property every three months. It's a significant improvement in service and it simplifies their operations at the same time.

<comment><USER10> > But can you use them? New Zealand power retailers install their own meters - > you can't supply your own.

You can use you your own meters, but the last one facing the power grid is installed by the infrastructure provider. So you can mearsure with old refurbished meters your washing machine, swimming pool, etc. Ask your power infrastructure provider, they may sell you old analog and/or digital meters. And depending on the local law, you have the right to "say no" to the installation of a smart meter.

> Our local distributors have been spending lots of money installing new lines > and upgrading the network

That's great. At my location the power infrastructure is the same since around 1960 (old wooden power poles on the country side, etc.) and the power line goes also from roof to roof, and if you want underground cable you have to pay it yourself. Almost every thunderstorm we have a short power-outage, because of the wooden poles. They only repair, and don't upgrade the infrastructure at all.

> the retailers are changing the meters because it means we get an accurate > bill every month.

Where I live, central Europe, you have to read the traditional (analog or digital) meter yourself every month and send the data via email/postcard and someone controls your meter infrequently every few months/years at your location. If the device would be called digital meter (like the industrial ones) and send the data only once a week or month, it would be fine. On the other side the new smart meters send the data _every 15 minutes_ over GPRS cell-phone technology.

<comment><USER12> Don't know about NZ, but in Australia the networks own the meters, and the law is clear: you connect to the grid, you must give them access to their network gear. And they send over their own meter readers.</comment></comment></comment></comment>


Discussing Blackness on Reddit? Photograph Your Forearm First - <AUTHOR> https://www.nytimes.com/2019/10/08/us/reddit-race-black-people-twitter.html <comment><USER2> I wonder if I could pass this test, being technically 'high-yella'.

[https://en.wikipedia.org/wiki/High_yellow](https://en.wikipedia.org/wiki/High_yellow)

<comment><AUTHOR> I understand and respect their need for a private group but their method of verification certainly isn't fool-proof.

<comment><USER2> I was honestly just surprised to find out /r/blackpeopletwitter still existed; maybe it's a Mandela moment for me but I could have sworn it was either banned or at least quarantined some time ago. At least for my part I went out of my way to make sure no posts from the sub ever appeared in my browser, so maybe that's what it was.

I'm Afro-Cuban and to be honest the whole thing feels kind of minstrel-y to me.

<comment><USER3> /r/coontown was banned.

<comment><USER2> That's the one I was thinking of. Thanks for the correction.</comment></comment></comment></comment></comment>


Is It Controllable? - <AUTHOR> https://medium.com/@ruizhidong/is-it-controllable-6b0eab4285ed <comment><USER2> Short, sweet, simple (maybe overly simple?) philosophy.

<comment><AUTHOR> Thank you for the feedback :)

As Einstein once said, "Everything should be made as simple as possible, but not simpler"

Hopefully, I didn't make it simpler</comment></comment>


Taking Traffic Control Lessons - From Ants - <AUTHOR> http://blog.wired.com/wiredscience/2009/02/anttraffic.html <comment><USER2> This reminds me of an experiment I once read about. It was a comparison between 'angry' and 'calm' driving, during the angry days the driver was supposed to avoid letting people in front of them and on the calm days they were supposed to slow down and let people turn into their lane from side roads. This wasn't about fuel efficiency, it was about how much time it would cost you and quite counter-intuitively the 'calm' drivers who let multiple vehicles go in front of them actually ended up getting to work faster.

They also mention the ants collecting behind an ant that's carrying something. It's probably for a different reason (IE not losing the food) but it reminds me of the fleet-style of driving where they want to get vehicles to drive behind each other, sharing the same air pocket, which IIRC Mythbusters showed that tailing a big rig could reduce drag by up to 80%, however that was equivalent to being like 5 feet behind the big rig and they couldn't drive 'calm' enough at that distance and reduced efficiency, but at around 10 feet they got like a 60% drag reduction. Fleet driving (by computer) would allow vehicles to drive at 5 feet behind a big rig and get that drag reduction. However, I don't know how you'd pay for fleet-style driving because vehicles like a big rig would be improving efficiency for dozens of vehicles, which would assume nationalized transportation control so big rig drivers don't get ripped off on gas.

<comment><USER3> > However, I don't know how you'd pay for fleet-style driving because vehicles > like a big rig would be improving efficiency for dozens of vehicles, which > would assume nationalized transportation control so big rig drivers don't > get ripped off on gas.

You're assuming that the "drafters" cost the big rig gas (actually diesel), presumably by increasing air-resistance. They don't. Drafting reduces air- resistence for the lead vehicle too.

You can see this in car races. Two (or more) cars running close together, front-to-back, are faster than any of the cars are when running by themselves.

I don't know how much fuel drafting would save big-rigs (there's a somewhat an odd relationship between fuel used and work done in diesels) but at the very least, it doesn't cost them fuel.

<comment><USER2> I know it doesn't _cost_ the big rig, but it doesn't _save_ them much either (only a nominal amount from reduced tail drag as they don't form a complete air bubble), so why should they allow someone to basically metal-hump their vehicle for marginal to no benefit? For example: If it saves you $0.50 for every 5 minutes you tail a big rig, should the drafter be paying $0.25 to the truck driver?

I'm also talking about the electronic fleet, where it's controlled by computers in each vehicle. However there's also the mechanical fleet, where two (or more) vehicles actually latch on to each other, this would be great for electric cars and would also allow them to power-share. For example, 5 vehicles join together but only 2 keep their engines running but the power comes from all 5 vehicles' batteries equally.

<comment><USER3> > For example, 5 vehicles join together but only 2 keep their engines running > but the power comes from all 5 vehicles' batteries equally.

Why is power-sharing interesting for electrical vehicles?

Yes, you want to avoid free-riders, but IIRC the efficiency of electrical motors isn't that dependent on load, at least for partial loads.

</comment></comment></comment></comment><comment><USER4> more fascinating reading on this subject:

<http://www.itu.dk/~mikkelbu/projects/ars.html>

and this classic:

[http://www.cs.nott.ac.uk/~gxk/courses/g5baim/papers/ants-002...](http://www.cs.nott.ac.uk/~gxk/courses/g5baim/papers/ants-002.pdf) (pdf)

</comment><comment><USER5> Hofstadter has an article about traffic and the benefit of defection. A Universally Optimal (UO) path isn't necessarily Individually Optimal (IO) if everyone else is obeying the laws of UO. So if I know that everyone is going to obey UO laws, there's an increased incentive to defect to an IO route and save substantial time off my travel. However, if everyone follows the IO route, travel is slower for me than the UO route is for me.

The underlying theme of cooperation v. defection reappears through time, from the Ring of Gyges to the Prisoner's Dilemma to experiments with splitting money _. I'd be all for a computerized traffic system if everyone had to follow it.

_ Experiments where n units are divided between two people where one person can split the money, and the other person can approve/disapprove of the split. If the second person approves, both walk away with the money, if they disapprove, nobody gets anything. I'm sure someone has links to experimental results, but if the divide is too large ($9 / $1), even though it's a gain for the second person, they reject the offer.</comment>


Moving to Ghana to 'escape US racism' - <AUTHOR> https://www.bbc.com/news/world-africa-49394354 <comment><USER2> > In 2018, he successfully led a campaign to force the University of Ghana to > remove a statue of India's independence leader Mahatma Gandhi.

Standing at the empty plinth, he gave the Black Power salute, and called for the recognition of African heroes rather than a man who had once referred to black South Africans by a highly offensive racist slur - and had said that Indians were "infinitely superior" to black people.

<comment><USER3> Those statements were made as part of a legal petition when he was a young lawyer in his early 20s, freshly arrived in South Africa, and one who'd had little contact with the Africans.

He later came to publicly support African rights and gave many more speeches which asked Indians to draw inspiration from Africans.

Even Mandela forgave him:

> Mandela was well aware of the racist statements made by Gandhi when he was > young. He wrote in an article in 1995, "Gandhi must be forgiven those > prejudices and judged in the context of the time and circumstances. We are > looking here at the young Gandhi, still to become Mahatma, when he was > without any human prejudice save that in favour of truth and justice."

Source: [https://thewire.in/history/gandhi-and- africans](https://thewire.in/history/gandhi-and-africans)

</comment></comment><comment><USER4> As, famously, did WEB DuBois, who went so far as to renounce his Us citizenship.

</comment><comment><AUTHOR> Sad to see my submission flagged! Posted it because I came across this story on a reputable site (BBC) and I was wondering what people abroad, especially US, might think of it. I've lived my whole life in Europe so I don't have much context about the issue.

</comment><comment><USER5> The structural racism that my family faces because of the fact that I am married to a non-citizen SE Asian is a big reason I don't relocate my family back to the US. And this is a bipartisan issue. The issues discussed in the article there or that I have experienced have very little to do with one party alone.

The US built its industrial base on the backs first of slaves then of the freed slaves, poor immigrants (particularly from China) and dispossessed Latinos whose land claims were not recognized post-annexation. Until we understand that racism in the US is a tool of classism and we address the underlying economic injustices, there will be no social justice. But there is no political party willing to take that on and no real hope for the foreseeable future.

<comment><USER6> All these things never have happened in SE Asia correct? I mean it's not like the largest country in that region didn't have a caste system with a huge population of untouchables with no mobility.

<comment><USER7> "Everyone else is doing it" is a _terrible_ argument. Individuals are not averages of the nations or regions they come from — an Asian in the USA is not one third Indian, one third Chinese, and one third Other; an Indian is not 16% Dalit.

Even though I don't know of any history of SE Asian slaves in the USA, I can easily believe that a specific individual Indian with dark skin and Muslim religion might find America unwelcoming both on religious and racial grounds — if that person was rich enough to even try to move to America, I'm going to guess they're probably not one of the untouchables. Other people's pain doesn't invalidate theirs; it only becomes hypocrisy if they turn a blind eye to it after experiencing it, and I have no evidence of that here.

<comment><USER8> But it's not an argument - it's giving context. The USA is not uniquely guilty of all the injustices in the world, and refusing to go along with this narrative should not be framed as an attack.

<comment><USER5> The US is uniquely guilty of a particular kind of racism used to build a particular kind of economy. Racism in Europe is different than it is in the US. Racism in SE Asia is different than it is in the US. In the US, racism is solely about economic exploitation of working classes by a small group of mostly white (though with some token individuals from other groups) executive/investor class and it has always been this.

Europe's industrialization was built on the back of religious divisions rather than racial ones for example, which makes the problems somewhat different.

<comment><USER8> > Europe's industrialization was built on the back of religious divisions > rather than racial ones for example

Is this something you just casually throw out, or do you have some data showing that... countries with greater religious divisions industrialized faster?

In any case, "built on the back of" seems overstating things. Find some sin in the past, and claim that it is the source of all the success of a country, for which it must feel eternally guilty. That there are similar, successful countries without this sin, or unsuccessful ones that share it, are details best ignored. E.g. slavery was widespread in Africa and the Ottoman Empire, with wildly different outcomes than in the US, and imperialism _certainly_ wasn't unique to Europe or the US.

<comment><USER9> Jesus, your obsession with data... Just read history...

<comment><USER7> The history lessons that I grew up with told me that my country was great, and that everything which was bad in the world was the fault of some other nation. Asking for data is one way to get past that.</comment></comment></comment></comment></comment></comment></comment></comment>


Photos Of The Incredible Bombshelter Fortress of The Wikileaks Archives - <AUTHOR> http://www.businessinsider.com/wikileaks-sweden-server-pictures-2010-12#a-woman-stands-alone-9 <comment><USER2> Same story from days ago, but with no comments:

<http://news.ycombinator.com/item?id=1951171>

Same story from one day ago, with nearly 50 comments:

<http://news.ycombinator.com/item?id=1960295></comment>


Street-Fighting Mathematics is published under CC [pdf] - <AUTHOR> http://mitpress.mit.edu/books/full_pdfs/Street-Fighting_Mathematics.pdf People seemed pretty interested in this book being published. I recently discovered that it's under a Creative Commons license, so enjoy!<p>Sanjoy also has previous course material floating around (eg. Order of Magnitude Physics) and if anybody has links to those or comments I'm sure people would appreciate that. <comment><USER2> I think the book is interesting, but I'm slightly disappointed it really wasn't about street-fighting.

I think a lot about mathematics education is a domain issue. Problems are too easily wrapped up in tedious domains that don't engage the imagination. For one of my nationally-assessed Maths projects at school, the teacher had us analyzing football (soccer) scores, looking at standard deviations and the like. The class was far more engaged with the math than I'd ever seen them.

All boys dream of being a bad-ass, so I'm sure mathematics in the domain of street-fighting would also work! ("A perp is able to accelerate his fist at 10 m/s, and his fist has a weight of about 0.5kg. A broken jaw requires 4N of force. How much force will he put in your face? Will he break your jaw?")

</comment><comment><USER3> Can anyone comment on what would be the 'prereq' for this? Are the video lectures available anywhere (he seems to have taught a class by same name before)? And, approximately how much time would it take for someone going over it as self-study? I know the answer to the last question probably depends, but hopefully a ball mark estimate can be given

<comment><USER4> It looks like an understanding of Calculus and basic Physics is all that is required to read this book. Ballkpark estimate (how ironic, having to do a ballpark estimation before having read a book on how to do ballpark estimates): between 10 and 30 hours.

<comment><USER5> When do you people in the US start to learn about integrals?

Is it expected that a first-year undergraduate student already has the knowledge to understand the book?

I don't have the required "prerequisites" as the GP puts it to understand the book, but I should :( and it might shame me into studying a bit :) (then again, I probably won't)

<comment><USER6> (Single) integrals are freshman calculus, which is nominally taught in the first year of university. In reality, my experience is that many students (and, I would guess most science/engineering/math majors) take AP calculus during their last year of high school (17-18 years old), and proceed to differential equations, multivariable calculus, or whatever is next in their major's sequence during the first year of university.

<comment><USER5> Thanks for the answers. For all the flak US education receives, the option of taking AP classes sounds good (and it gives college credits, which is even better!).

From your post I could find <http://en.wikipedia.org/wiki/Advanced_Placement_Calculus> which shows that AP Calculus does indeed include integrals.

<comment><USER7> _Thanks for the answers. For all the flak US education receives, the option of taking AP classes sounds good (and it gives college credits, which is even better!)._

This is utterly derailing the main topic of conversation, but...

I know that a lot of mathematicians dislike AP Calculus. The original thinking was that Calculus couldn't be taught to highschoolers, so you waited until University to take it. Aspiring mathematicians would take a rigorous, proof- based Caclulus course, which would prepare them to tackle harder subjects in the future. Everyone else (engineers, chemists, physicists, etc.) would take a more general/applied course. Now, all but a handful of universities offer such courses, under the assumption that anyone who wants to be a mathematician has _surely_ taken AP Caclulus. So the idea of proof-based Calc. for future mathematicians has been lost in transition, and the end result is that you have kids hitting Multivariate Calculus and Differential Equations who haven't seen a proof in their lives.

The current AP Calculus courses are 99% computation. I was challenged while working through Spviak's book with no teacher guidance in highschool, but I scored a perfect 5 on the AP test with little effort.

<comment><USER8> In short, the lack of a proof-based class renders students Calc-clueless. (groan)

This happened to me. I went straight from high school calc into college differential equations because the AP score allowed me. It took a Rudin-based analysis course, much later, for me to appreciate proofs of convergence or epsilon-delta arguments, because my H.S. calc did not have them, and the college diff-eq assumed you knew them already. The shock was painful.

Eventually though, you learn what you need to know.

<comment><USER7> _It took a Rudin-based analysis course, much later, for me to appreciate proofs of convergence or epsilon-delta arguments, because my H.S. calc did not have them, and the college diff-eq assumed you knew them already. The shock was painful._

I have yet to take a course that uses the so-called "terse little blue book from hell". (:

 _Eventually though, you learn what you need to know._

Indeed, although I wonder about people becoming discouraged about being mathematicians simply because they've been misled for so long about what's on the "other side" of college math.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> A highly recommended read.

Not to spoil anyone's fun but gems like this abound:

A valid economic argument cannot reach a conclusion that depends on the astronomical phenomenon chosen to measure time.( _Discussing GDP and Multinationals in Nigeria_ ).

</comment><comment><USER10> Looks great! Now can someone get me more hours in the day?

<comment><USER4> Unless you are a very unique case, you'll should be able squeeze a book this size into your daily life. Let's assume that given the content and the font- size, an average HN reader would require an hour to read 10 pages (that's underestimating most people's reading speed of course). Then the book would require, on average, about 12 hours to read. If you dedicate 2 hours a day to it, you'll have finished it before the week is over. If you take the time to solve all the problems presented, it may take you a few more weeks, but it's not a major project like reading and doing all the exercises from SICP.

<comment><USER10> This issue isn't so much the time to read the book as the time to read all the books I want to read. I already have two texts underway ("Concepts of Nonparametric Theory" and "Prediction Learning and Games").

</comment></comment></comment><comment><USER11> I'm very excited :) The book is a combination of developments from the author's phd thesis and a likewise named course that has been run the past few years at MIT's iap term, and is chock full of great tricks and habits to do sophisticated calculations mentally (or at least with greater simplicity)!

<comment><AUTHOR> Do you have links to those? I know of the MIT ones, but I've also seen links to his Order of Magnitude Physics materials (at Caltech) floating around before.

</comment></comment><comment><USER12> Anyone have a epub version for the iphone/android ?

</comment><comment><USER13> Since its a small pdf it looks great on my kindle. I believe I read an earlier draft of this and I was impressed.

</comment><comment><USER14> I would much rather have this book in dead tree form. It seems like it would be great to sit on the couch with.

</comment><comment><USER15> Why link to the PDF? Why not link to the book's page?

[http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&...](http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=12156)

<comment><AUTHOR> I thought it might be a bit hard to notice the pdf link - I'd visited several times but only noticed the CC license because of an Amazon review.</comment></comment>


Ask News.YC: Advertising with Revenue Sharing or Free? - <AUTHOR>

My product is a service for other sites to re-use. I'm trying to decide whether I should make it advertising supported with revenue sharing (50/50 split with the webmaster) or make it totally free with no advertising. I can afford to offer it free. My primary goal right now is just to drive adoption. It seems intuitive (to me) that free would further that goal more than including advertising, but I've also realized that offering a new revenue stream for webmasters might be just as (or more) effective.
I'm afraid that letting individual webmasters choose would result in a muddled "brand image", with the people who want a new revenue stream (falsely) believing it's not good for that and the people who want it free believing (falsely) it's not available advertising-free.
I find myself flip-flopping trying to decide which direction to go. Any thoughts or advice? <comment><USER2> The goal of any startup is to find out early and often how customers will react to your vision. If you have a vision of charging for a service, and you give it away for free; you may find people leaving to a competitor.

If ads are your vision, then establishing trust with your consumer base is critical. They need to know that their information is not being used inappropriately.

You will also demonstrate value sooner to investors if you have an established revenue stream.

One option we have considered is establishing 2 brands, just for this purpose.

One can be ad based, and the other could be free or subscription based.

</comment><comment><USER3> > My product is a service for other sites to re-use

If it adds value to the website for the webmaster then maybe you should not split anything from ads. The value would be in the new feature.

</comment><comment><USER4> One thing I've tried to do with new sites, at least, is put small, unobtrusive ads, so that people aren't bothered much, but do get used to the idea of there being ads. Maybe they're so common these days, though, that it's not really necessary. Anyway, the idea is to manage expectations, whatever that may mean for your product.</comment>


Why Not the Funnel Chart? - <AUTHOR> http://practicetrumpstheory.com/2013/09/why-not-the-funnel-chart/

<comment><USER2> Link is bad. Correct link:

[http://practicetrumpstheory.com/2013/08/why-not-the- funnel-c...](http://practicetrumpstheory.com/2013/08/why-not-the-funnel- chart/)</comment>


Samsung Kicks Off Mass Production of 8 TB NF1 SSDs with PCIe 4 Interface - <AUTHOR> https://www.anandtech.com/show/13003/samsung-kicks-off-mass-production-of-8-tb-nf1-ssds-with-pcie-4-interface <comment><USER3> Samsung also demonstrated 16TB NF1 SSDs. 36x 16TB in 1U so ~24PB in a 42U rack...

[https://www.anandtech.com/show/12567/hands-on-samsung- nf1-16...](https://www.anandtech.com/show/12567/hands-on-samsung-nf1-16-tb- ssds)

[http://www.samsung.com/us/labs/pdfs/collateral/Samsung- PM983...](http://www.samsung.com/us/labs/pdfs/collateral/Samsung- PM983-NF1-Product-Brief-final.pdf)

</comment><comment><USER4> > NF1 SSDs enabled an undisclosed maker of servers to install 72 of such > drives in a 2U rack for a 576 TB capacity

That's 12PB in a 42U rack! I wonder what sort of storage densities Google, Amazon, and Dropbox are achieving.

<comment><USER5> Note that 2U box needs something like 250gbit internal bus just for disk IO, assuming you wanted the full throughput of every drive.. there is a huge trade off in terms of processing capability and networking bandwidth putting so much disk in a single box, the rest of the hardware can't cope with it.

These things look useful only if a tiny fraction of the stored data is hot, I imagine lower density that provides more flexibility in terms of processing room on the box, and networking around the box is probably a better option in a lot of cases.

Been pondering these devices for the past 10 minutes or so, can't think of a use for them except elaborate storage servers.

<comment><USER6> I think AMD Epyc is a perfect match for this... 128 PCIe lanes per socket.

On a dual socket Epyc system with 36 of these, each with a 4x PCIe connection 'only' takes up 144 lanes, leaving plenty of lanes free for even 8x Dual-QSFP+ 40GbE NICs.

Of course, I'm not sure how you'd physically pack all of that into even a 2U form factor.

Usefulness... data warehouses. If the price was right, we'd probably move to that to speed up various ingestion & reporting functionality. At 288TB (assuming RAID10) capacity, it'd be a perfect fit.

<comment><USER7> Epyc is 128 lanes _period_. The dual-socket configuration steals half the lanes for the interconnect.

It's also PCIe 3 for now.

<comment><USER6> Ah, right. My mistake.

Still, I think it might be in a better position than Xeon for raw storage IO.

</comment></comment></comment></comment></comment><comment><USER8> Death-of-the-disk stories usually have to deal with price. But, if you drive the density north far enough, at the price per GB level I think this may no longer be an issue.

Simplistic price (bought price) vs power budget, speed, retention time/replacement time, all gets very complicated.

I haven't had a (company) laptop in three rounds with spinning disk. I replaced my @home with an SSD in the last pre-tax-claim spend cycle. I still buy commodity USB drives to be the disks for rPI small devices, but I am beginning to wonder how long that will sustain?

The floppy->semi-floppy->USBdrive story had a longish arc, but at a lower level of penetration. Once the hardshell floppy appeared, the larger units died out pretty quickly, and once USB bootable became ubiquitious, boot floppies ceased to inform my life pretty quickly, and now with RAC and iDRAC cards, I barely touch boot media either (admittedly, the RAC card has an SD and I write "images" to it in ISO format but I keep wondering how long boot media will depend on emulating a spinning CD/DVD drive.

TL;DR This feels like right size, right time. If the spec for the backplane is good, I'd like to see this baked and shipped in other vendors. Goodbye spinning metal?

<comment><USER9> Not for backup and long term storage. Spinning rust is far more dependable than flash when sitting on a shelf, and much less sensitive to temperature.

I know where I'm archiving my data.

<comment><USER10> So you use tapes?

Seriously tapes are way more dependable than disks. Less moving parts and less chance internal eletral mechanical break down. And check all the other boxes you mention.

<comment><USER9> Updated with more detail and maybe fewer typos:

I certainly considered tape and used it in the past, but there are two huge upsides to hard disks:

1\. Obviously access and transfer time is much better.

2\. I can access hard disks from 30+ years ago. My oldest drives use SCSI and IDE, but even SATA has been around for ages. My tapes however can only be read of the particular branch of tape drives and there are an infinitude of standards/models around so I'd have to store the tape drive along with it. I'm not sure I'd be able to restore my father old backups as the tape drive was some weird PC thing.

A few years ago I finally transferred my tapes to my FreeNAS box and breathed a sigh of relief as I don't know when I'd be able to run my DAT changer again.

So, no thanks, I'll stay with hard drives (mind you, drives _will_ die as well so you still need redundancy).

PS: I want to emphasize that I was talking about shelf life. For data sitting in a good NAS, like ZFS based FreeNAS, I have little worries as there's redundancy and weekly scrubbing (and I do have a backup of that as well). I do worry a great deal about any data that might sit on a USBKey or an SSD in the closet.

</comment></comment></comment></comment><comment><USER11> Like to hear everyone's guesstimate on when/if the $/TB of SSD will cross over the HDD.

<comment><USER12> Never, because if it did cross over suddenly flash demand would surge by 10x and then there'd be a massive shortage, pushing the price back up. [https://blog.dshr.org/2018/03/flash-vs-disk- again.html](https://blog.dshr.org/2018/03/flash-vs-disk-again.html)

<comment><USER13> No. If it costs less to produce, you'd just increase production until one passed the other. The demand surge might delay the inevitable for a year or two, but fabs would catch up and then the HDD would retire.

dshr misses the mark a lot of the time.

<comment><USER14> >No. If it costs less to produce,

The point is it doesn't, and not anytime soon.

<comment><USER15> It may never cost lower to produce in the environment in which both are massively available as manufacturing. But it may happen that manufacturers concentrate on flash storage (could be better profit margins, could be bigger demand from corporations or IAAS operators, could be many other things) and thus, in a cascading way make prices of disks higher for manufacturing, due to limited supply of materials, machines, trained workers for manufacturing of disks.

The only way I see the current status quo to be static by definition is if there is indeed a (above-mentioned) shortage of some raw material that would prevent the switch. Can someone comment on what exactly that shortage would be? Edit: I read the linked post (didn't see this before, sorry). TL;DR - it's just lack of enough manufacturing and issues with building more fabs.

<comment><USER14> Fabs are expensive. The article already points out NAND is reaching its limits. What those fantasies about NAND stack that could do 1024 or 2048 layers may never appears. At least in the next 5 years. Basically we have used up all of the tricks. ( For now )

Assuming a 128 Layer yield is good enough and solvable within the next 3 - <USER2> years. Micron is only just started shipping 96 layers and may take another 2 - 3 years for it to be mainstream. QLC only brings 33% capacity improvement over TLC at the expense of much lower write cycles and slower latency. Node scaling are also much more expensive, we don't have the node scaling with density increase while per transistor is half anymore.

Build more Fabs? Well China is pouring in $100B to brute force this problem. Nothing has come out of it just yet. And if it wasn't China, who has the incentive to build expensive Fabs, with little to no expertise in memory, patents, for a possible profits margin where its industry has a habit of cycles or long losses? I.e High Risk

TSMC ex-CEO Morris mentioned it multiple times, their company will not produce DRAM or NAND.

While the three DRAM and NAND manufacturer are well aware of what China is trying to do, and are milking the market for as long as they could.

No where in the NAND 5 years roadmap points to it being technically feasible, and economically feasible that is could be cheaper then HDD in the 5 years AFTER the current 5 years roadmap. I.e Nothing is showing it could happen in the next 10 years. While the HDD camp, Western Digital has the roadmap and tech, that they can reach 40 to 80TB per HDD in the next 8 years.

So I have no idea what I am getting downvoted. Not to mention one of the poster is correct about how cheaper NAND and higher demand will means prices go up higher.

That is like those people who keeps talking about OLED replacing LCD, nothing in the roadmap, of all possibilities show OLED will even be the same price as LCD within the next 5 years. The most optimistic forecast has it being double the price of LCD within 5 years. That is including the printable OLED that Sharp is investing in.

<comment><USER15> Thanks. This is very informative.

</comment></comment></comment></comment></comment></comment></comment><comment><USER16> Notably the specs don't support OPAL encryption like many of Samsung's other products. That's a bit disappointing.

</comment><comment><USER17> How are people using NVMe drives in servers? Is soft raid the only option?

<comment><USER12> NVMe RAID now exists: [https://www.broadcom.com/products/storage/raid- controllers/m...](https://www.broadcom.com/products/storage/raid- controllers/megaraid-9460-16i) But I suspect most people are using soft RAID or no RAID.

</comment></comment><comment><USER18> Is this suitable for NVM usage?

</comment><comment><USER19> Top option in the new Mac Pro?

<comment><USER20> This SSD will probably be out of date when Apple decides to care enough about the Mac to ship the Mac Pro. Maybe 2023?</comment></comment>


React Rainbow Components. Build your web application in a snap - <AUTHOR> https://react-rainbow.web.app <comment><AUTHOR> Hi guys, I'm one of the creators of React Rainbow. We're very excited to share a little project our team's been working on.

\- More than 60 components built on top of React. \- Proper documentation based on Interactive Examples. \- Our components are test infected. \- We designed each component with i18n in mind. \- Accessibility is part of our definition of done. \- Our npm package provides webdriver.io page object.

We are excited that you are interested in contributing to this open-source project! We'd love to hear what you think!. Give it a hack and let us know what you think.

<comment><USER2> Very cool great looking collection!

</comment></comment><comment><USER3> very nice collection of components!!!</comment>


Shoplifters? Studies Say Keep an Eye on Workers - <AUTHOR> http://www.nytimes.com/2009/12/30/business/30theft.html?_r=3&hp

<comment><USER2> So, we now have the credit infrastructure widely disbursed. In fact, it is so in place that we now give out credit cards (well debits... but functionally the same thing) instead of checks for gifts, rebates, refunds etc.

The presence of different types of accounts: credit, checking, savings, etc. backing these cards further indicates the expensive bits of infrastructure are wired togeher.

A while ago people developed cryptographically secure e-cash systems.

Why oh why can't we just combine them together now? Sophisticated fraud protection is already very good, and can still apply, meaning we don't need perfect crypto. Good e-cash systems for in-store credit would allow the tracking of who frauded someone. The general concept would allow for secure online transactions. Fraud would be easier to track down. Conversions to real cash, as well as natural pooling places such as banks would allow for mixing the units, such that the privacy folks don't have much to worry about, not much more than currently anyway.

What am I missing for such to occur?</comment>


Ask HN - What Virtual PBX Do I Use - <AUTHOR>

I'm looking to ditch Vonage VOIP in our small office (less than 10 employees).  I'd like to migrate to a VOIP/Virtual PBX system.  Grasshopper looks nice but we need to be able to send/rcv faxes from a "real" fax machine as well as have "real" phones ring (as opposed to cell phones, etc).  Any recommendations? <comment><USER2> I asked a similar question sometime back and there were good responses: <http://news.ycombinator.com/item?id=865297>

I ended up rolling FreeSwitch on a dedicated server on-site. I found it to be _very_ extensible and reliable and I would almost always recommend it over Asterisk. Also deployed a couple Linksys SPA-3102 analog-to-IP gateway devices for existing lines - this is your best bet with faxes which transmit poorly over the public internet (unless you deal with T.38 and all that). I also use Flowroute (<http://www.flowroute.com/>) for all outgoing calls and for an extra incoming phone line (what's known as a DID). Their rates are reasonable and support is good, no problems so far. All extensions are using Polycom Soundpoint phones which sound great, have the best speakerphones I've heard, and are extremely customizable to boot. Although their XML config format was created by a madman :) Email me if you want more info about our setup.

Alternatively, check out Phonebooth, which I believe just launched the other day. They allow you to use Polycom phones too: <http://www.phonebooth.com/> and <http://news.ycombinator.com/item?id=1194615>

<comment><USER3> Flowroute now supports inbound t.38, not sure about outbound support. Makes faxes work over ip!

</comment></comment><comment><USER4> Disclaimer: I work for a hosted PBX provider, but not in US, so won't recommend anyone anyways.

If you value your mental health and don't want to hire an onsite, on-call VoIP specialist do _not_ try to setup your own PBX. Sure - sometimes it works and you can do a lot of crazy stuff with a software PBX; in reality you want it to be up 24/7 and ensuring the quality of pstn termination, local network and configuration of such setup is at least one additional full-time job for the first couple of months. In reality if you have a problem with some "prepared" solution like freepbx / trixbox / whatever and have no support contract, you're left on your own with a system you don't understand and with no telephony - not a good option. Unless you need some very special functionality, just pay for the service. Do not be fooled by "simplicity" of solutions like trixbox - if something doesn't work, you still have to learn the whole stack to know how to fix it.

Solutions like grasshopper are cool and that's the kind of service I'd recommend. What you should look for is: how do they handle faxes (only T.38 is reliable), can you setup a divert number (like a mobile) for situations when your network is down, how much will it cost to get additional internal and external numbers. Bonus point - do they support wideband codecs? It might seem stupid, but the quality difference is amazing (only for internal calls).

If you're an MS-based shop, you might want to look at hosted OCS offers. Phones supporting it are a bit more expensive, but the ability to integrate with your emails, calendar, voicemail, etc. over the phone is really cool if you travel a lot. If you have a high volume of calls, you might also look for solutions that provide reception with a virtual switchboard apps - they're a bit more comfortable for a lot of call transfers than looking up numbers.

</comment><comment><USER5> I've been curious what the real benefit is of an operation that needs an onsite (versus multisite) phone system for fewer than, eh... the size of a single 16-24 station line card to use VoIP in the first place? If you've got under 10 people, you might be able to get away with a used 3x8 Norstar which has the approximate maintenance overhead of a desk lamp. And in some areas, you could probably be even better off than getting a key system with good old Centrex.

Again, this is largely curiosity. But when you start getting into maintenance and UI and latency and encoding support and QoS from your ISP and whatever else you have to care about to make simple dialtone work, I wonder if maybe the problem is getting nerded to death for no good reason?

<comment><USER2> You're right, the complexity of these VOIP-based systems is much higher. But sometimes it's worth it for both cost and flexibility. I think each business needs to evaluate what's right for them.

For example, in the setup I described in my other comment, the business wanted to move away from a Lucent/Avaya Partner system they had been using for over a decade. They had 3 lines and about 15 extensions and were somewhat happy with it. The system for many years had, as you say, the "maintenance overhead of a desk lamp" but it was starting to drop voicemails randomly, the phones sucked, adding more lines meant buying one of those extension modules, etc. Plus the building was being renovated and running just CAT6 everywhere instead of separate ethernet and phone lines was very appealing.

Buying an inexpensive Dell rack server, Polycom phones, and a couple Linksys analog gateways was much cheaper than the initial investment they spent a decade ago. Since they kept the analog lines, which are useful for fax machines and security systems, there was no need to really worry about QoS initially since internal network bandwidth was plentiful. This alone fixed the problems of the aging system and they got voicemails and faxes delivered via email, shared directories browsable directly from phones, proper call parking, and a ton of other small features.

The cost benefit came when we added an external VOIP provider later. Now they could have unlimited simultaneous inbound and outbound calls for a single DID/caller ID and just pay a flat-rate per minute. At their usage, this came out much cheaper (4x) than the deals negotiated with Verizon locally. And the concept of individual lines that could be busy just melted away. The system is still configured to fallback on the couple analog lines if the internet goes out, but the rate plans have been reduced to the bare minimum so that incoming calls stay free.

EDIT: Almost forgot, another key benefit of a software switch is the easy integration it allows with your backend apps. For our internal rails-based order processing app, I implemented dial-on-click (ala Google Voice) and call history logs for all our vendors and customers (see librevox and mod_xml_cdr for FreeSwitch if you want to do something similar) in a day.

But viraptor is right, ensuring all the fallbacks work correctly and are tested takes time, and it might not be the wisest way to spend time if you don't understand what you're getting into.

</comment></comment><comment><USER6> I would recommend asterisk (<http://www.asterisk.org/>). Beware: it may be a bit overkill, and configuration is a bit in depth but it will have all the features want now, and could ever need (check out AGI scripting in your research).

You may also want to look into trixbox, which is an asterisk server with a bunch of addons, including full web based configuration already configured.

I do not have much experience with faxing but I'm pretty sure if you want to fax from a real machine over VOIP you may run into problems, but I believe there are options for email to fax.

As for real phones ringing you'll need ATAs, which can be picked up pretty cheap, or you can go with SIP phones (a bit more pricey but more functions) so the whole idea may be a bit more than you're looking to get into.

</comment><comment><USER7> I currently have a small office that depends on faxing. I'd ditch fax if I could, but my clients are funny in their ways.

Anyway, I'm using Asterisk 1.6.2 with FreePBX 2.0 and Broadvox is my SIP provider. My system is set up to receive faxes on the Asterisk PBX and email them to a mailing list of employees (this is configured through FreePBX). I also have a Grandstream GXW-4004 to provide FXS ports for the fax machine and analog phones that aren't important enough to replace with expensive VOIP handsets.

Note: I never got the analog fax machine to send _and_ receive. Depending on how I set the fax detect on the Grandstream, I could do either-or, but not both. So my PBX receives faxes and my machine sends them. This is likely a PEBKAC matter, but I can't really be bothered to put forth the time to work this final gremlin out of my system.

My advice to anyone doing faxing over VoIP is the following:

1) If possible, don't.

2) Make sure your SIP provider supports VOIP faxing. Some don't with good reason: it's a bitch to support.

3) Make sure your SIP provider support G.711 with T.38.

4) Make sure your ISP does not strip QoS headers (In this case DiffServ EF)from IP packets. Your ISP will ignore them, obvious, but you definitely don't want them stripped or overwritten.

5) Make sure your VOIP packets have a DiffServ EF value in their headers.

6) Make sure any router(s) and switch(es) are configured properly to give top priority to your VOIP traffic. Failure to do so will likely result in jitter on your calls and you'll be unable to successfully send/receive faxes.

7a) If possible, don't run your _primary_ PBX in a virtualized enivornment. Competition for system resources may create unwanted latency on your Asterisk VM resulting in jitter and failed faxes.

7b)With that said, I would definitely keep a cold or hot "spare" PBX running in a VM to spin up in the event of an outage.

Good luck!

</comment><comment><USER8> Unless you don't value your time much, I wouldn't bother dealing with any software setup yourself. (I've hacked a bit on the FreeSWITCH project, and I still would just buy a hosted solution from someone for my own PBX rather than deal with it myself.)

There's plenty of hosted PBX systems out there, but I've heard a lot of great things about RingCentral. (And no, I don't work for them. If anything, they're somewhat of a competitor.)

</comment><comment><USER9> I'm not in the US so I can't recommend any particular provider. <http://www.voip-info.org/> has a lot of information about both providers and equipment.

\- For hosting the PBX yourself without learning Asterisk or FreeSWITCH, I can recommend Askozia PBX, which is based on m0n0wall: <http://www.askozia.com/>

I've also used FreePBX-based systems like TrixBox CE, which are more full- featured and more customizable, but might take more effort to upgrade.

\- For reliable faxing, make sure your VoIP provider supports the T.38 protocol and get an ATA (analogue adapter) that supports it. You can then set up T.38 passthrough in your PBX.

\- For equipment, I'd recommend going with one of the "known" brands. I've used Linksys SPA9xx and Snom phones, but read forums to see what people recommend for the PBX you choose. Keep in mind that a lot of the buttons and features on the phone might not work "out of the box", depending on the PBX you choose. I'd stay away from no-name brands unless you have tried them yourself or have very good recommendations.

</comment><comment><USER10> This is a hard space to get real reviews that are not either PR puff pieces or providers stuffing in shill reviews. Are you working with phones that support a real SIP stack or will this be taking existing analog/POTS phones and attaching adapters? Are you willing/interested in dealing with the setup and maintenance costs of running your own PBX via Asterix? Two that I have had relatively good experiences with are Nextiva and Vocalocity and I have heard reasonably good things about Packet8 (those subtle qualifiers you see there are because there is always _something_ you can complain about for any particular VOIP provider -- no one has reached the level of being "great" yet.) Nextiva has decent virtual fax services and can support a real fax machine with an adapter, but my adapter just shipped yesterday so I can't yet say whether or not it works as advertised or how well it integrates into the rest of their vFax system.

</comment><comment><USER11> We have been using phonebooth.net for our 4 geographically distributed work- at-home offices (Pittsburgh, Dallas, Austin and Denver), and it's worked pretty well so far. They worked (and are working) hard with us to get out the little glitches, and we got to deal with their actual developers, since it was (still is?) a beta test situation.

They don't handle faxes, though.

Pretty good deal, though: $20/month/line, 5 line minimum. Cheaper than POTS, and much more flexible.

(I just dragged my Polycom IP phone out to our farm and plugged it in for the first time, and I'm up and running with my phone line with no obvious issues.)

</comment><comment><USER12> We just set up virtualpbx.com. The web ui is kind of clunky, and the automated text-to-voice for the message is terrible, but it had the options we needed: setting up out toll-free number to be routed to sales and support and send out emails if the call went to voicemail. We didn't care about setting up actual office phones at this point - everyone just uses their cells. We mostly wanted a way to avoid people having to give out their personal numbers or a single office number as a point of contact. This gives us the flexibility we need as we hire more people, reassign people, or have people leave.

</comment><comment><USER13> I'm in a similar boat right now. I haven't pulled the trigger on anything but I'm taking a good look at Fonality (<http://pbxtra.fonality.com/products/standard>).

Anyone else hear anything about it?

Edit: Now looking at FreePBX.org / Asterisk as well - thanks for the suggestions

<comment><USER4> If you want anything more than just call forwarding, have a look at solutions not based on Asterisk. FreeSwitch and Yate are quite good. In my experience Ast. is like Windows - people complain about it, but continue to use it because most other people use it... and it's not that good compared to others. If you do want only forwarding, have a look at sip proxies instead (like Opensips) - that's more reliable and can save you one hop on the media path.

</comment></comment><comment><USER14> Hi This is John Smith I suggest you to signup and go through the features.

<http://www.pbxplus.com> \- offers speech recognition, faxing, extensions, voicemail transcription, trees - visual drag and drop designer. Free US number with 50 mins. Can't get better.

</comment><comment><USER15> Get a scanner, Efax (<http://www.efax.com/>) and VoIP provider (we use <http://www.gradwell.com/> but are in the UK) and you're all set.

<comment><USER16> Better than efax is RingCentral. You can get an 800 number with multiple voice mail boxes plus the number doubles as a fax number. It can also be used to receive or make calls, but that costs extra and I haven't used it yet.

<comment><USER17> I'm going to add a cautionary note about RingCentral. I hesitate to make my first post a negative one, but they've inspired me to feel strongly enough to break the lurker barrier. My experience with RingCentral has been almost unilaterally bad, and I'm now actively looking to switch away as quickly as possible.

We have had persistent bugs and inconsistencies. By itself, this wouldn't be a huge issue, but their level one support is untrained, powerless, and seemingly reluctant to help. In all my support calls to them, they have yet to put me on the phone with someone who understands my problem and has the power to fix it.

<comment><USER16> That's interesting to hear. What kinds of issues were you having? We primarily use it to receive voicemails (then call back using Google voice or Skype) and for sending/receiving faxes.

</comment></comment></comment></comment><comment><USER18> Wow, reading through all these comments, it seems like pretty much a no- brainer what the Google Voice team is up to. :) I didn't realize this space was such a disaster.

</comment><comment><USER19> We use fuguphone.com but its Canada only. Its basically just hosted asterisk. I'd look for a local one that can transfer you numbers.

</comment><comment><USER20> I've been using PhonePower for a few years now and they are very impressive with very good customer service and great features.

</comment><comment><AUTHOR> thanks for all the great responses... phonebooth.com seems pretty close to perfect to what we need... next question - any recommendations on IP Phones?

<comment><USER4> Snom / Linksys / Polycom / Yealink (which model depends on what you need from your phone). Cheaper products tend to crash, have bad audio quality, etc. Definitely try to stay away from Grandstreams...</comment></comment>


Arcade Fire Film/Browser hack - <AUTHOR> http://www.thewildernessdowntown.com

<comment><USER2> There were quite a lot of comments about this yesterday when announcement on Google blog was posted (instead of the direct link):

<http://news.ycombinator.com/item?id=1646430></comment>


How Clang Compiles a Function - <AUTHOR> https://blog.regehr.org/archives/1605 <comment><USER2> How my brain optimizes sentences:

> we're not going to look at any non-trivial C++ features

Becomes: we're only going to look at trivial C++ features.

Which is then recognized as a no-op and replaced with the null sentence ;-)

Nice article though.

<comment><USER3> Careful with that brain. It's where the nasal demons come from.

<comment><USER4> Nasal demons come from little plastic bags.

<comment><USER3> I think I may have lost some people - [http://www.catb.org/jargon/html/N/nasal- demons.html](http://www.catb.org/jargon/html/N/nasal-demons.html)

</comment></comment></comment></comment><comment><USER5> Clang noob here. Can someone explain in non tech terms what's happening in this article?

<comment><USER6> You possibly need some background concepts.

Compilers work by taking a (computer-)linguistic analysis of the source code text, a _syntax tree_ , converting it into an _intermediate representation_ (IR) which is then progressively _lowered_ to the raw machine code form by a series of transformations.

A connected graph of _basic blocks_ (BBs) is generally how such IRs work. Basic blocks have a formal structure involving them being connected to other basic blocks only at their tops and bottoms, and not being multiply-connected internally. A perhaps simplistic way to visualize them is as the polygons on a flowchart.

So the article, which assumes that you know this background, is showing what kinds of basic blocks in Clang's intermediate representation result, before any lowering and optimizations, from the syntax tree. It's skipping over the part where a syntax tree is constructed from the source code shown.

In many compilers, you don't get to see the intermediate representation. In compilers like OpenWatcom C/C++ and GCC you can always go and look at the source code to see what the IR data structures are. But the IR is in memory (or internal temporary files) and not accessible outwith the compiler programs themselves. Clang's design involves a publicly specified intermediate representation that has documented serialization formats for writing to file and then accessing with other, potentially third party, tools.

* [https://llvm.org/](https://llvm.org/)

<comment><USER7> > A connected graph of basic blocks (BBs) is generally how such IRs work

I was wondering. When is this graph actually built? is it before creating the IR or after? The way I see it, you can generate IR without worrying about control flow graph. Then you can build the CFG from the IR in order to perform common optimization or register allocation. Is that correct?

<comment><USER8> In LLVM, a function is linked list of BBs, each of which is a linked list of instructions. The final instruction in each basic block is a special kind of instruction that includes a list of all possible successor BBs.

In effect, the IR is exactly a control-flow graph represented in adjacency list form, so it's not possible to construct the IR without constructing the control flow graph. You could theoretically write the IR in textual form, but that's definitely not the common case, and you generally need to construct the CFG anyways to properly emit the IR (particularly since adding the phi nodes for SSA requires knowing the predecessors of every basic block).</comment></comment></comment></comment>


Pirate Bay co-founder charged for hacker attack - <AUTHOR> http://www.thelocal.se/47376/20130416/#.UW6DQLWSKSp

<comment><AUTHOR> That seem to some very serious charges and also a bit far fetched. I mean: running the Pirate Bay _and_ doing very complicated hacks against high profile targets is a lot of work...</comment>