New AMD EPYC-based Compute Engine family, now in beta - <AUTHOR> https://cloud.google.com/blog/products/compute/announcing-the-n2d-vm-family-based-on-amd <comment><USER3> Disclosure: I work on Google Cloud.

This has come up a few times, so I wanted to reiterate that these are the Zen2/Rome parts not the first generation "Naples" parts. We didn't bother launching Naples for GCE, because (as you can see) Rome is a huge step up.

<comment><USER4> Are you using a custom CPU from AMD? I spun up a N2D instance and it's showing up an "Epyc 7B12" and I can't find any details about this CPU anywhere.

</comment></comment><comment><USER5> Since people from Google Cloud are likely here, one thing I'd like to ask/talk about: are we getting too many options for compute? One of the great things about Google Cloud was that it was very easy to order. None of this "t2.large" where you'd have to look up how much memory and CPU that it has and potentially how many credits you're going to get per hour and such. I think Google Cloud is still easier, but it's getting harder to know what is the right direction.

For example, the N2D instances are basically the price of the N1 instances or even cheaper with committed-use discounts. Given that they provide 39% more performance, should the N1 instances be considered obsolete once the N2D exits beta? I know that there could be workloads that would be better on Intel than AMD, but it seems like there would be little reason to get an N1 instance once the N2D exits beta.

Likewise, the N2D has the basically same sustained-use price as the E2 instances (which only have the performance of N1 instances). What's the point of E2 instances if they're the same price? Shouldn't I be getting a discount given that Google can more efficiently use the resources?

It's great to see the improvements at Google Cloud. I'm glad to see lower- cost, high-performance options available. However, I guess I'm left wondering who is choosing what. I look at the pricing and think, "who would choose an N1 or N2 given the N2D?" Sure, there are people with specific requirements, but it seems like the N2D should be the default in my mind.

This might sound a bit like complaining, but I do love how I can just lookup memory and CPU pricing easily. Rather than having to remember name-mappings, I just choose from one of the families (N1, N2, E2, N2D) and can look at the memory and CPU pricing. It makes it really simple to understand what you're paying. It's just that as more families get added and Google varies how it applies sustained-use and committed-use discounts between the families, it becomes more difficult to choose between them.

For example, if I'm going for a 1-year commitment, should I go with an E2 at $10.03/vCPU or an N2D at $12.65/vCPU. The N2D should provide more performance than the 26% price increase, yes? Why can't I get an EPYC based E-series to really drive down costs?

Again, I want to reiterate that Google Cloud's simpler pricing is great, but complications have crept in. E2 machines don't get sustained-use discounts which means they're really only valuable if you're doing a yearly commitment or non-sustained-use. The only time N1 machines are cheaper is in sustained- use - they're the same price as Intel N2 machines if you're doing a yearly commitment or non-sustained-use. Without more guidance on performance differences between the N2D and N2, why should I ever use N2? I guess this is a bit of rambling to say, "keep an eye on pricing complexity - I don't like spending a lot of time thinking about optimizing costs".

<comment><USER3> Disclosure: I work on Google Cloud (and really care about this).

The challenge here is balancing diverse customer workloads against the processor vendors. Historically, at Google, we just bought a single server variant (basically) because almost all code is expected to care primarily about scale-out environments. That made the GCE decision simple: offer the same hardware we build for Google, at great prices.

The problem is that many customers have workloads and applications that they can't just change. No amount of rational discounting or incentives makes a 2 GHz processor compete with a 4 GHz processor (so now, for GCE, we buy some speedy cores and call that Compute Optimized). Even more strongly, no amount of "you're doing it wrong" actually is the right answer for "I have a database on-prem that needs several sockets and several TB of memory" (so, Memory Optimized).

There's an important reason though that we refer to N1, N2, N2D, and E2 as "General purpose": we think they're a good balanced configuration, and they'll continue to be the right default choice (and we default to these in the console). E2 is more like what we do internally at Google, by abstracting away processor choice, and so on. As a nit to your statement above, E2 _does_ flip between Intel and AMD.

You should choose the right thing for your workloads, primarily subject to the Regions you need them in. We'll keep trying to push for simplicity in our API and offering, but customers really do have a wide range of needs, which imposes at least some minimum amount of complexity. For too long (probably) we attempted to refuse, because of complexity, both for us and customers. Feel free to ignore it though!

<comment><USER6> I mean, this mentality often _is_ wrong. Scaling out actually isn't the right solution for everyone. It works for Google given that primarily web services are offered. It does not work for workloads that heavily rely on the CPU (think financial workloads, ML, HPC/scientific workloads) or have realtime requirements. In fact, for many ETL workloads vertical scaling proves _far_ more efficient.

It's long been the "google way" to try and abstract out compute but it's led to an industry full of people trying to follow in their way and overcomplicating what can be solved on one or two machines.

<comment><USER7> Except, almost without exception, eventually the one or two machines will fall over. Ideally you can engineer your way around this ahead of time - but not always. Fundamentally relying on a few specific things (or people) will always be an existential risk to a big firm. Absolutely agree re: start small - <USER2> the problem with "scale out" is a lack of good tooling - not a fundamental philosophical one.

<comment><USER8> Plenty of services can deal with X hours of downtime when a single machine fails for values of X that are longer than it takes to restore to a new machine from backups.

<comment><USER9> I do not agree, and it is not my experience.Mind you that I've always worked in small/mid-sized businesses (50-300 employees) and basically every service has someone needing it for their daily work. Sure, they may live without it for some times, but you will make their lives more miserable.

And anyway if you already have all in place to completely rebuild every SPOF machine from scratch in a few hours, go the extra mile and make it an active/passive cluster, even manually switched, and make the downtime a minutes thing.

<comment><USER8> A small amount of work over a long period of time (i.e. setting up a redundant system) may be worse than losing a large amount of work in a short period of time.

Single machines just don't fail that often. I managed a database server for an internal tool and the machine failed once in about 10 years. It was commodity hardware, so I just restored the backups to a spare workstation and it was back up in less than 2 hours. 15 people used this service and they could get _some_ work done, without it, so there was less than 30 person-hours of productivity lost. If I spent 30 hours getting failover &c. working for this system over a 10 year period, it would have been more hours lost for the company than the failure caused.

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> Now its getting really interesting: In the end you have to compare pricing for a vCore (which is a thread on a cpu) with per-thread performance on AMD vs. Intel. Does anyone know a benchmark like this? Epyc Processors are most often tested on heavy parallelizable tasks and not strictly single thread tasks.

<comment><USER11> I did some rudimentary testing with AMD vs Intel on AWS recently and found that AMD lacked enough in single threaded perf that it meant they weren't worth the savings for our workloads (Rails apps).

<comment><USER12> AMD-based AWS instances are running on first-generation Epyc processors.

Compared to the second-generation Epyc processors that Google is using, the first generation has lower clock speeds, can execute fewer instructions per clock (particularly in terms of floating-point operations), has substantially less cache, and has a more complicated memory topology that can negatively impact the performance of workloads that aren't NUMA-aware.

In short, your experience with AMD in AWS isn't relevant to Google's offerings.

</comment></comment></comment><comment><USER3> Disclosure: I work on Google Cloud.

cvallejo is the PM, so ask her anything!

<comment><USER13> Does the new n2d machine type support nested virtualization?

(Asking because Azure supports nested virtualization but only on some machine types. AWS doesn't support nested virtualization at all. Google Cloud seems to support nested virtualization on other machine types.)

Also, why 224 rather than 256?

<comment><USER3> We do not yet support AMD's nested implementation (we do on Intel). _But_ cvallejo is _also_ the PM for Nested :).

As for 224, we've always reserved threads on each host for I/O and so on. Figure 2 from the Snap paper [1] is probably the best public reference. We also don't make it clear (on purpose) what size the underlying host processors are, though you can clearly guesstimate pretty easily.

[1] [https://research.google/pubs/pub48630/](https://research.google/pubs/pub48630/)

<comment><USER13> > We do not yet support AMD's nested implementation (we do on Intel).

Any particular reason for that limitation, or just "not implemented yet"? (Not asking for product roadmaps, just wondering if there's a specific technical issue that makes it more difficult to support.)

> As for 224, we've always reserved threads on each host for I/O and so on. > Figure 2 from the Snap paper [1] is probably the best public reference.

That's helpful, thank you.

<comment><USER3> Not implemented yet. In the stack rank of "stuff needed to update our hypervisor for AMD again" it wasn't at the top :).

Note the again as well: GCE _originally_ had it such that N in N1 meant iNtel, and A1 was for AMD (as Joe said publicly here: [https://twitter.com/jbeda/status/1159891645531213824](https://twitter.com/jbeda/status/1159891645531213824)). By the time I joined though, we didn't see the point of the A1 parts, since the Sandybridge's smoked them.

</comment></comment></comment></comment></comment><comment><USER14> Does AWS have a comparable offering? I haven't seen anything on EPYC - congrats to GCP for moving quickly. I'm mostly AWS based currently.

<comment><USER15> See [https://aws.amazon.com/about-aws/whats-new/2019/04/amazon- ec...](https://aws.amazon.com/about-aws/whats-new/2019/04/amazon- ec2-t3a-instances-are-now-generally-available/)

<comment><USER16> Your link is for the 1st gen EPYC cpus, while this announcement is for the 2nd gen.

</comment></comment></comment><comment><USER17> Its really great to see more AMD options for cloud instances. Now I'm just waiting for more ARM architecture options. Not having to cross compile code (a1/m6g AWS instance types) has been very useful in my day to day job.

</comment><comment><USER18> Well, they may be hypothetically "available" in us-central1, but they don't show up for me. All I see is "Cascade Lake powered" N2.

<comment><USER3> Disclosure: I work on Google Cloud.

Sorry for the confusion. We should be more clear and explicitly state "now rolling out". Over the course of our multi-day rollout, different regions and the console will start showing these.

</comment></comment><comment><USER19> What are the implications? Higher perf and/or lower price?

<comment><USER20> Looks to be about $5/Month cheaper, based on this page:

[https://cloud.google.com/compute/all- pricing#n2_machine_type...](https://cloud.google.com/compute/all- pricing#n2_machine_types)

Jeez, I didn't realize how expensive cloud compute was. I always wondered why my school still has a datacenter. Having your own servers still makes sense for a lot of orgs.

<comment><USER21> It's a question of how dynamic your usage is and how much the better security and management features save you. If you have very consistent workloads with little idle hardware and modest admin needs you can beat cloud environments even with reservations but usually when I see the numbers it means major costs like staffing or power / HVAC aren't being factored in.

</comment></comment></comment><comment><USER22> Geekbench Multi Core Benchmarks:

1\. n2-standard-16 (Intel Cascade Lake): [https://browser.geekbench.com/v5/cpu/1257619](https://browser.geekbench.com/v5/cpu/1257619)

2\. n2d-standard-16 (AMD EPYC): [https://browser.geekbench.com/v5/cpu/1257340](https://browser.geekbench.com/v5/cpu/1257340)

3\. n1-standard-16 (Intel Skylake): [https://browser.geekbench.com/v5/cpu/1257420](https://browser.geekbench.com/v5/cpu/1257420)

Intel Cascade Lake is with a very small lead on 1st place. But AMD is with three-year commitment the cheapest option. Great to have a choice. Thx @cvallejo

</comment><comment><USER23> Can those instances be used in GKE node pools?

<comment><USER3> Disclosure: I work on Google Cloud.

Should be, once the rollouts complete. So once you can see N2D types in the Console for your project, I _think_ it'll just flow naturally to GKE.

<comment><AUTHOR> Disclosure: I also work on Google Cloud

That's correct! Once the rollout completes, you will be able to use N2D instances for GKE!

<comment><USER24> And if it doesn't, it was always possible to break GKE state into "offering" node types that were not supported. Source: me, using preemptible instances for a year before they got into GKE ;-)

Modifying instance group templates is your friend.

<comment><USER25> Yeah, this can be done for nested virtualization instances (images) too.

</comment></comment></comment></comment></comment><comment><USER26> What's the intent / timeline for N2D in other regions, e.g. us-west1?

<comment><AUTHOR> disclosure: I work at google!

We will be expanding the regional footprint of N2D. US-west1 should come online in the first half of this year.

<comment><USER27> Not directly related, but do you know when we can expect to see N2 instances in us-east1-b? Currently they're in 2/3 of the zones, just one annoying zone short of being able to use it in my GKE cluster.

</comment></comment></comment><comment><USER28> AMD EPYC has SHA hardware accelerations but in my test "openssl speed -evp sha1" is slower than "openssl speed sha1". Any idea why?

</comment><comment><USER29> Any idea when these will be available on the genomics pipeline API (now "Cloud Life Sciences" API)?

</comment><comment><USER30> What's the topology of these machines? Dual socket 64c chips with some reserved (or disabled)?

<comment><USER31> They don't say that but that's the only way to provide 224 threads.

<comment><USER32> Their topology tells far richer tales than the press releases when you dig deeper on the numbers.

224 threads = 112 HT cores = 2 x 56 core CPUs. This is 8 cores short of the 64 core flagship. 8 cores == 1 CCX.

It seems exceedingly unlikely that AMD would produce a Rome CPU with 7 out of 8 CCX in perfect health, but have the 8th CCX completely missing (functionally). It seems more likely that the 8th CCX is there with all 8 cores, and that it is reserved for some other type of service. One possibility could be that there are higher guarantees of side channel protection at the CCX boundary, and google intends to use this for secure internal functions or sell to clients who have side channel sensitivity. Another may be that they simply want the hypervisor to have a very fat budget of 8 cores per socket to work with. Considering the amount of potential IO going on, this might be required in some cases.

<comment><USER33> See also boulos' comment:

"As for 224, we've always reserved threads on each host for I/O and so on. Figure 2 from the Snap paper [1] is probably the best public reference. We also don't make it clear (on purpose) what size the underlying host processors are, though you can clearly guesstimate pretty easily."

Ie. if there's even one thread reserved on the host, your speculation comes to naught. Sorry.

</comment></comment></comment></comment><comment><USER34> AMD's stock is wild. It was around $2 just a few years ago and has been on a non-stop trend up to almost $60 today.

<comment><USER35> They really are doing something disruptive. I can't quite remember if this is correct (it has been a while since I last studied business), but in business there is a "blue ocean strategy". The basic premise is, if you can provide a product for half the price, with the twice the value, you will destroy the incumbent.

What AMD is doing is really insane in my opinion. I'm not sure if they are pricing their processors low on purpose and/or if they have found a way to manufacture cheaper and/or Intel was screwing consumers with their pricing since they were so dominate.

No matter what, AMD is able to provide something that is measurably better and significantly cheaper than the incumbent, and if the blue ocean strategy holds, they should become the new incumbent in the near future.

<comment><USER36> Smaller chips have better yield. As AMD's current chips are composed from several smaller ones (I believe two or three), each composite has better yield than one bigger of same real estate size.

So yes, they figured out how to produce cheaper solutions.

<comment><USER37> > As AMD's current chips are composed from several smaller ones (I believe two > or three)

For EPYC, AMD is using nine chips: [https://images.anandtech.com/doci/13561/amd_rome-678_678x452...](https://images.anandtech.com/doci/13561/amd_rome-678_678x452.png)

That's 1x I/O chip (kind of like a router), and 8x chips, each of which has 8x cores on it. Total for 64-cores / 128-threads across 8-compute chips, talking together through a central 1x I/O and Memory chip.

The I/O chip is the biggest for reasons: 1. Its made on a cheaper process. 2. It has worse performance than the compute chips. 3. Its required to be big because driving external I/O requires more power.

So the I/O chip can be made on a cheap / inefficient 14nm process, while the CPUs can be made on a more expensive 7nm process (maximizing clock rates, power-efficiency). The big I/O ports are going to eat up a lot of power regardless of 7nm or 14nm process, so might as well save money here.

</comment></comment></comment></comment><comment> 867-5309 no pricing mentioned

I was surprised to discover the other day that one of my VPSs had been upgraded from 1 old Xeon 26XX core to 2 EPYC cores. other stats unmetered 10Gb/s up/down, low latency A'dam location, 2GB RAM, SSD.. it even outperformed my i7-8700T in a single-core openSSL benchmark. most importantly it costs €3/mo

I really can't see google competing with that

<comment><USER38> I see 10gps listed at USD 569/month?

[https://www.scaleway.com/en/virtual-instances/general- purpos...](https://www.scaleway.com/en/virtual-instances/general-purpose/)

<comment><USER39> Euros is not the same as US Dollar

569 Euro equals 613.73 United States Dollar

<comment><USER38> Right. Either way it's a little over 3 USD/month..

<comment> 867-5309 €3/mo, and for a different plan</comment></comment></comment></comment></comment>


Let your theme song be your password - <AUTHOR> http://www.newscientist.com/blog/technology/2008/08/let-your-theme-song-be-your-password.html

<comment><USER2> Actually, this reminded me of a decent password "generator" and rememberer scheme I read about on /. many years ago. Take the first letters of the first line of a favorite song and use that as your password. CIRPOTB was the example used there, from the Pearl Jam song "Jeremy". Add a checksum digit if you like (a number that corresponds the the length of the alpha part (cirpotb7). In most cases this is as secure as anything else and easy to remember.

Another method that I have personally used is to take the FCC ID of a device I carry (pager, cellphone, etc) and use that. After a few iterations you remember it, it's very random, but you also have it written down in plaintext with you if you forget.

<comment><AUTHOR> Those are decent because they generate random passwords. I don't really understand this scheme, because if you have to keep track of a file to use as your password you might as well just use a public key crypto system instead</comment></comment>


Cheap MacBook chargers create big sparks - <AUTHOR> http://www.righto.com/2016/09/why-you-shouldnt-use-cheap-macbook.html <comment><USER2> This is all fine and good, but I used my FLIR gun to register my official Apple charger hitting 139ºC because of a cord that had frayed due to poor strain relief: [https://twitter.com/mmastrac/status/741066994598846465](https://twitter.com/mmastrac/status/741066994598846465)

There are no good options here, not even official ones.

<comment><USER3> Are you suggesting that the official chargers fail at an unacceptable rate? Because that has not been my experience.

<comment><USER2> It's been entirely my experience. I've got four of them for a house full of MacBooks - three are dangerously frayed at one end or the other and required me to use electrical tape+spring to repair them.

Somewhat amusingly, I bought a cheap knockoff MacBook charger off Amazon and it's been super-reliable other than running about 10ºC hotter than a standard Apple one.

<comment><USER4> In my experience, the frayed cords come from pulling or sitting directly on the cord. Don't pull or sit on the cord. Use the connector directly to manipulate the connection.

<comment><USER5> The insulation on Apple cables is notably fragile in comparison to other cables. It would be OK for a permanent installation, but these are portable machines. Moving around just a bit on a regular basis is enough to cause failure after a few months.

I've gone through multiple power bricks and multiple lightning cables. The experience has made me even more hostile in the face of Apple's long history of proprietary cabling.

<comment><USER3> >Moving around just a bit on a regular basis is enough to cause failure after a few months.

This is shocking. The only power adapter that every started to have fraying was one that went into a tight backup situation that I biked with every day for three years, plugged in and out of several times a day. There was a ton of wear on that cable, every few months I'd clean it off because of the abuse it took. And it still lasted three years.

Also, which proprietary cables are you talking about? Charging cables are the only proprietary connector on a Mac and this is an industry-wide phenomenon, not an Apple phenomenon. It's an odd juxtaposition of an extremely unusual cord experience along with an unusual opinion on proprietary cords.

</comment></comment></comment></comment></comment></comment><comment><USER6> Up next,"Build this small welding unit using only things you find in your backpack!"

I think Ken did a great job but missed an interesting point, if you have one of these cheap chargers and leave it plugged in with the end laying about on your desk waiting for you to bring your Macbook and plug it in, then anything on your desk can short it. Pen clips, paper clips, metalized trinkets from the last trade show you visited. If you aren't the clean spartan desk type (and I'm not) you may find burn marks on your desk where things have hit the end of the connector, or worse.

<comment><USER7> I thought I mentioned that in the article: "Maybe you think these safety issues don't matter because you don't poke your charger with a paperclip. But if you have any metal objects on your desk, a random contact could yield a surprisingly large spark."

<comment><USER6> Apparently I was blinded by the spark video!

Interestingly I went back and looked at it again, and my eyes see the big pictures of the complexity, the warning is under that those pictures but not part of the conclusion. So my reading went right past it. I'd suggest that you move that paragraph under the Conclusion header (I think it can be justifiably considered part of the conclusion, do you?) then it pulls the attention right to it.

<comment><USER7> Ok, I moved it to the conclusion. I try to have a good pacing between the text and images, but I guess that paragraph was too short to be wedged between the images.

</comment></comment></comment></comment><comment><USER8> I once bought a fake charger and you could feel a tinkly sensation around the mousepad while it was charging. Obviously I threw it out quickly.

<comment><USER9> That also happens with the official chargers if you use the two-prong plug.

<comment><USER8> Never had that happen with an official charger, and have only used the european 2-prong plug and the US 2 prong flat.

<comment><USER9> It's a well-known phenomenon and was the subject of an HN submission a few months ago: [https://news.ycombinator.com/item?id=11838349](https://news.ycombinator.com/item?id=11838349)

</comment></comment></comment></comment><comment><USER10> The standard barrel connector protects against short-circuiting with a simple plastic ring. The magsafe connector requires a fucking microcontroller.

<comment><USER11> The standard connectors have a horrible tendency to break through mechanical wear after a few years. On the laptop side, not the charger side.

They also take the laptop with them then pulled by a kid or fell over on.

If there's a problem with the MagSafe, it's that the cable itself breaks after 12-18 months due to lack of stress relief, requiring an expensive replacement and inspiring people to buy $15 euro fire starters.

Complain about Apple being too cheap to hire some proper engineers to build the cable, not the brilliantly designed connector.

<comment><USER10> I've repaired dozens of laptops and the one thing the broken power connectors had in common was being soldered directly to the motherboard instead of using a separate assembly.

</comment></comment></comment><comment><USER12> I bought my off-brand MBA charger from Prime Cables (like Monoprice, but in Canada) and wonder how well they're made. I messaged them on social media linking the last article along these lines but they never responded.

</comment><comment><USER13> So in essence, if you want to verify the authenticity of your macbook charger, you should put a paperclip in the connector.

</comment><comment> josh-wrale I can tell you from personal experience. When you have a baby or toddler who is curious about wires and quick about it, those safety features are very handy!

<comment><USER14> Even as an adult, I like Apple chargers because I know they were engineered well enough to not shock me. I have no problem leaving the connector on the couch, accidentally sticking my finger on the contact points, etc.

It is a nice peace of mind when my brain is not on edge that it might be plugged in when I see it laying around. I could not say the same thing for my cheaper USB phone charger.

</comment></comment><comment><USER15> The real problem I have here is that even if you want to buy genuine apple equipment its very difficult aside from buying from apple.com or an apple store.

Specifically, buying from Amazon is basically a crapshoot with Apple gear of getting genuine stuff or cheap off-brand knockoffs and there really isn't any easy way to figure out what you're getting before you order.

</comment><comment><USER16> This article illustrates a great point, but it makes me wonder why there isn't a middle ground between an overpriced apple charger, and the cheap (and unsafe) knockoffs. Eg, why can't I purchase a MFi certified charger at 1/2 or 2/3 the price of an official charger that I know will be safe?

<comment><USER17> The catch-22 is that Apple shoehorned substantial complexity into the charger, to make it and associated cables/connectors very lightweight, while still supporting batteries with very high power density. Compare the article's photos of the PCB, etc, of the Apple product and the cheap knock-off, both of which have to squeeze into the same plastic housing. That complexity costs money. Trimming out complexity generally means trimming out safety features (which already are possibly insufficient, considering some of the other responses on this thread).

<comment><USER16> I agree; complexity costs money. But I guess I'm wondering if there is still a lot of profit margin built into the cost of the apple charger that could be shaved by a decent third-party such as Monoprice.

Take lightning cables: I can buy the Apple lightning cable for $25, or I can buy a perfectly good Monoprice, MFi-certified lightning cable for $7, or I can buy a no name, non-certified cable (that will likely have issues) for a buck or two on ebay.

<comment><USER17> I would expect any 3rd party manufacturer seeking to replicate the safety margins of Apple's own power adapters may ultimately find it difficult to produce a competitive alternative, once you factor in costs of NDAs with Apple and such. (Apple likely works to ensure it stays this way.) Cables are passive, USB chargers are generally low wattage, while the Macbook adapters have active regulation, plus higher power ratings.

</comment></comment></comment></comment><comment><USER17> The power density of modern Li-Ion batteries is surprising. These chargers have to move substantial current over cables and connectors that just become ever lighter.

</comment><comment><USER18> That's a bit sensational. There are tons of legitimate, high quality power supplies that will create "big sparks" when you short them out. You can stick a scissor into the next power outlet and chances are you will come away with much more than sparks.

The only difference here is that Apple does a handshake prior to switching the power supply outputs on. The cheapie saved on the extra logic. Of course this is only necessary in the first place because the connector isn't very safe with the exposed pins.</comment>


Complaints Against YouTube Doubled After "Demonetization" - <AUTHOR> http://www.hissingkitty.com/complaints-against-youtube <comment><USER2> This has to be an opportunity for any competing service willing to carry exactly the demonetized content.

<comment><AUTHOR> I just checked and demonetizedtube.com is available on GoDaddy. Go for it :)

<comment><USER2> Haha, do you happen to have a social video storage and delivery system in your couch cushions I could borrow?</comment></comment></comment>


5 Reasons This Is the Best Time to Become a Writer - <AUTHOR> https://medium.com/@akinlaby/5-reasons-this-is-the-best-time-to-become-a-writer-f0dbcd766e49#.x8fps8tja <comment><USER2> This is the most meta post I've ever seen in a while. Basically this guy is writing about why "this is the best time to become a writer" and uses an example of another guy who became successful because he wrote about "how writing everyday changed his life". Dude, you are doing it wrong if you are writing for the sake of writing. There are too many of these people nowadays, especially with the rise of Medium. There used to be a time when people used to write because they had something to express to the world, not because "it's the best time to become a writer". These new "writers" lack genuine quality because the motivation is wrong from the beginning. Nowadays these people write for the sake of writing, because "writers are in high demand". Don't dishonor people who actually write things to make a difference.</comment>


Announcing Heroku Enterprise for Java - <AUTHOR> http://blog.heroku.com/archives/2012/9/19/announcing_heroku_enterprise_for_java/ <comment><USER2> Sadly I think the OpenJDK will scare off most "Enterprise" developers. I see this as a good option for testing/development but I have never seen an Enterprise use OpenJDK in production. Most are using Oracle's JDK or some other special purpose JDK. And if you can't make it look exactly like production most shops are not going to touch it.

<comment><USER3> Any reason why? OpenJDK is the reference implementation of Java 7, so if you are targeting Java 7 why not use it?

<comment><USER2> There are probably any number of reasons, valid or not. The easiest to cite is "Who do I call if we have a problem with OpenJDK?"

Outside of OpenJDK there are other issues that make this a hard push for Enterprise customers. I haven't seen a large installation that uses memcached. Most go with something that is tailored for Java like Terracotta's bigmemory. Again it is "Who do we call if it breaks?"

Heroku is in an interesting place here I think. They are certainly on the right trail but this is VMWare's back yard now. Being a polyglot platform is probably a disadvantage for Heroku in this arena. If you look at VMWare they have a complete system tailored to work with Java. The hardest hurdle I see is that VMWare's solution can go from developer size deployments to production all on the hardware owned by the Enterprise. Heroku can't do that and I assume will never do that. Not "owning" the entire system will be an uphill battle for a lot of Enterprise customers.

</comment></comment></comment><comment><USER4> All it appears to be is WAR deployment and memcached session support. If your deploying a WAR switching to embedded tomcat is trivial as is tying memcached/redis to tomcat's session manager. I also don't see how they support adding libraries to tomcat if they control the container, maybe a maven plugin?

If they were going after the 'enterprise' market I would have expected EAR deploys to JBoss or Glassfish.

</comment><comment><USER5> They are also supporting Java 8, pretty nifty. Exciting news for the Java community.

<comment><USER6> No. They are OpenJDK. The only use of OpenJDK is to see java programs crash in unexpected ways.

<comment><USER7> This matches my experience. Eclipse Indigo and Glassfish both love to crash for me on OpenJDK (amd64 ubuntu). When I switch to Oracle's JDK binaries it all works fine.

</comment></comment></comment><comment><USER8> I looked, but couldn't find any information on max Java heap size, etc.

</comment><comment><USER9> Glad to see the Heroku updates. It is definitely becoming one of my favorite tools.

Next step, I hope they roll our websockets support.

</comment><comment> m-i Still can't find out how much will it cost me. Those pricing "tools" say nothing concrete and measurable to me.

</comment><comment><USER10> So, like regular Heroku, but more expensive and enterprisey?

<comment><USER11> Consider it the "invisible hand" of Salesforce.com at work. They have a single-minded focus on Java technology, and they will go after the easy money at the Java shops (they know how to sell these things).</comment></comment>


Ask HN: What are your day to day pain points while coding? - <AUTHOR>

At my job we collectively have several small but annoying, tedious, confusing or slow operations. What are the things that bug you?
For us:
* Needing to enter our username and password every time we push/pull to github
* Having numerous processes running at once: our app's server, a couple instances of Jasmine, a couple instances of Coffeescript guard, selenium, etc
* Keeping track of which of our many builds are failing/passing
* Debugging and generally dealing with IE
I know there are good solutions to all the above, but for various reasons they don't work for us (I'm leaving the details on that out on purpose). My point isn't to look for answers to these annoyances, but rather get an idea of what annoyances devs see often. <comment><USER2> My problems:

* Payment. Indirectly related to coding. Credit cards aren't common in my country and debit cards often are not visa or mastercard compatible. Fortunately prepaid credit cards are available at nearly every bank. With PayPal and a normal credit card tied to my account I just don't feel save carrying my wallet and laptop around.

* Spread of knowledge. I need to know so much to be able to choose the right tool in every situation. Right now I'm working on a blog/portfolio. I like to use a static site generator. Despite there beeing thousands of comparisons on the web it is hard to assess whether I'm able to build what's in my head. The one that seems capable enough is written in a language I don't speak. I probably have to write my own solution or write a patch in a language I can at least partly understand due to my knowledge of other languages.

With the advent of JavaScript it has become even harder. Backbone? Ember? Angular? Use existing knowledge to write my backend or switch there to JavaScript too? Meteor? Express? Which build system, which module system, async helpers or not? Promises? How to tie everything together?

You could spend more time figuring that out than you spend time writing code...

* Anonymity and my lack of creativity. You need an username for everything. I have yet to find a bookmarking service that just requires my mail adress and a password. And what's with all the twitter and facebook authentication crap sites starting to pop up that don't even allow you to sign it without it...

</comment><comment><USER3> * IDEs. I can't find one I like. I prefer VIM for typical editing, but Rails projects are so large, in terms of filesystem navigation, that I like a normal tabbed IDE. Aptana with VIM keybindings is okay, but heavy and has some pain points.

* Heroku db:push. Have to use ruby 1.9.2, and it's a bitch to get it working on Windows machines. I eventually did, but it was a big problem for a couple of days.

* Interruptions. My office space is public. I can't always concentrate with headphones on, though.

</comment><comment><USER4> Our biggest difficulties are:

* The difficulty of eliminating interruptions and isolation

* Inability to work as fast as we would like

* Choosing the right tool for each job while reducing technical debt

* Finding ways to delight internal ande external customers while delighting ourselves, and finding slack to implement them.

</comment><comment><USER2> I'll better split this into two posts. My solutions:

First problem: Use ssh-agent to store the key.

Second problem: I have several scripts that start tmux sessions with every necessary program running. There are tools like tmuxinator to make the creation of presets easier. I run my own solution though as I need the flexibility.

Third problem: Something like Travis CI?

Fourth problem: Taking a look at all the shims, frameworks and helpers helped me get a general idea of what works and what doesn't on certain IE versions. I wouldn't worry much about the inbetweeners (IE7, IE9). Windows XP is the biggest reason why people use IE8 and if you're targeting that market it would be best to invest in a workflow that minimizes the differences and get as much knowledge about the lacking features of IE8 as possible.

</comment><comment><USER5> Public key authentication for Github has not worked for you? Can't imagine how that is possible.

<comment><AUTHOR> Well this submissions is failing spectacularly :) I'm not looking for solutions, but other problems people have.

We have to enter our credentials into github each time for two reasons: we decided to follow the TOS of github and not create work specific accounts for everyone (instead you are supposed to take advantage of github's organizations feature). And almost all of our work takes place on pairing machines. Leaving our accounts open to github across random machines throughout the org is not something we want to do.

<comment><USER6> checkout `ssh -A`

</comment></comment></comment><comment><USER7> For your first point make sure your git remote is using the correct format.

.git/config should look like:

        [remote "origin"]
            fetch = +refs/heads/*:refs/remotes/origin/*
            url = email@example.com:user/repo.git

</comment><comment><USER8> > * Keeping track of which of our many builds are failing/passing

Might I suggest circleci? We use it to build every branch for every developer's fork. It's pretty easy.

</comment><comment><USER9> Why does that first problem exist? I could not deal with that.

</comment><comment><USER10> Interruptions.</comment>


Automattic caught A/B testing pricing for Vaultpress - <AUTHOR> http://brianbreslin.com/automattic-caught-ab-testing-pricing-for-vaultpress/

<comment><USER2> This isn't "new" news, Matt revealed the A/B testing right here on HN quite awhile back when I pointed out the price seemed high:

<http://news.ycombinator.com/item?id=1230679>

(over three months ago)

<comment><AUTHOR> ah. never saw that. thanks. kinda hard to keep track of everything said on HN in the comments.

</comment></comment><comment><USER3> __* Page attempts to load malware __*

This site is linking to some shady shit. Some embedded page tries to load a Java app from: extraditelbds.info

      Too many arguments supplied: {http:, -J-jar, -J\\extraditelbds.info\smb\new.avi, http://extraditelbds.info/e9t/l.php?i=2, none

Edit: Symantec claims this is:

      HTTP Phoenix Toolkit Executable Download

<comment><AUTHOR> any idea where to find that and how to remove it?

<comment><USER3> Search for every place where you're pulling in external content via javascript, iframe, etc. Something that you're pulling content from has been hacked or had malicious content injected.

</comment></comment></comment><comment><USER4> Chrome threw a google malware warning for this site

<comment><AUTHOR> yeah i had a page hacked, but thought I had cleared it up. not seeing the malware warning on safari or chrome on my end anymore.

I think its due to mediatemple hosting this and their servers being compromised recently. [http://www.google.com/safebrowsing/diagnostic?site=brianbres...](http://www.google.com/safebrowsing/diagnostic?site=brianbreslin.com)

<comment><USER5> I didn't get a malware warning in chrome either. Did you have to do anything to have it cleared from the malware databases, or did you just fix it and wait?

<comment><AUTHOR> I fixed it. then went into webmaster tools and scanned the site, said no problems found. I remember having to request a scan once before when another site had been hacked.

</comment></comment></comment></comment><comment><USER6> I got the chance to hear Matt talk about this in person at my college. He said what was really interesting was that what people fill in for what they would pay in "a perfect world" is very strongly correlated to the price they're offered.

</comment><comment><USER7> I'm not sure that's A/B testing really... they're asking a question and the question is 'how much would you pay?'. And the way they've phrased it is to include an example of what it _could_ be worth, and then you can ignore the field (agree with that pricing) or enter your own suggestion.

It's A/B testing pricing when they actually offer different prices to person A than person B, right now they're just asking a question and including in the question an expectation setter. It's market research and not something dodgy.

</comment><comment><USER8> I'm not sure I see a problem with this. Companies experiment with different pricing all the time.

For example, you ever received a postcard from the cable or phone company with a special offer? Do you think that all the customers get the same card? No...they segment their customer groups to see what types offers work best.

</comment><comment><USER9> Amazon tried this a long time ago, and people were upset.

See [http://www.allbusiness.com/marketing- advertising/4188108-1.h...](http://www.allbusiness.com/marketing- advertising/4188108-1.html) for confirmation.

<comment><USER10> What Amazon was doing was a little different. They were trying to figure out how much people would pay based on available information that Amazon had. They were actually figuring out how much a specific individual would pay for something. Cool, but a little creepy.

This is just simple price testing and is done all the time with minimal fanfare.

<comment><USER9> _What Amazon was doing was a little different. They were trying to figure out how much people would pay based on available information that Amazon had._

Evidence, please.

My memory of the event, backed up by Amazon's statements in the link that I provided, were that Amazon was randomly varying prices and no demographic information was used to do so.

Now there were lots of people who thought that they might be doing something more nefarious. But to the best of my knowledge, no evidence exists. Furthermore my understanding of A/B testing and how Amazon runs A/B tests makes Amazon's version of the story seem extremely reasonable.

<comment><USER10> Just going off memory. Even if they were just varying prices randomly, the resulting data set could (and I would guess would) be used to price dynamically based on personal information. The difference is the possession of personal data, which Amazon already had by the ton.

<comment><USER9> In that case I believe your memory is incorrect.

Furthermore my strong expectation is that they were using A/B tests to collect data on the optimal price point to show everyone. I don't dispute that eventually they likely would have gotten around to correlating geography and price to be able to dynamically show the right price point, but there is absolutely no reason to doubt their assertion that they hadn't done so, or to believe that they had any near term plans to do so. (For one thing A/B tests to try to figure out that level of detail are very difficult to do.)

</comment></comment></comment></comment></comment><comment><USER11> FWIW, it was $30 when I went there a week ago. It looks like they're coming down on the price somewhat. I thought $30/month to backup a blog was crazy at the time.

</comment><comment><USER12> Price elasticity testing is about the simplest thing you can do to drive substantial gains in profit - wrote up an article about this a while back here:

[http://conversionvoodoo.com/blog/2010/06/one-simple- secret-f...](http://conversionvoodoo.com/blog/2010/06/one-simple-secret-from- an-1890-economist-nearly-doubles-website-profits/)</comment>


Driving the San Joaquin Valley - <AUTHOR> http://harpers.org/blog/2015/04/driving-the-san-joaquin-valley/

<comment><USER2> I grew up in the San Joaquin Valley and it's one of the most impoverished areas I've seen in the United States. It's ranked the least educated, highest drug consuming population in American for a few years running. There aren't a lot of opportunities for tech or business outside of agriculture.

There are a few people changing that in Fresno though. If you're interested in what it looks like to bootstrap tech in an area like this you should checkout bitwise [0]. They have been doing some good stuff there and have started to turn Fresno in a hub for tech for the SJV.

[0][http://bitwiseindustries.com/](http://bitwiseindustries.com/)

</comment><comment><USER3> The San Joaquin valley has more in common with other natural resource extraction areas such as the coal fields of Kentucky and West Virginia than it does with the Santa Clara Valley, just a few mountain ranges over.

I've always wondered why more back-office type of work doesn't move from the Bay Area over to Stockton, Fresno, or Merced - places that aren't that far away by car outside of rush hour.

<comment><USER4> > I've always wondered why more back-office type of work doesn't move from the > Bay Area over to Stockton, Fresno, or Merced - places that aren't that far > away by car outside of rush hour.

I used to wonder about that when I worked in Silicon Valley in the late '80s. It was the typical '80s startup where we often worked long hours. Generally, no one had time for much during the week other than work, a little sleep, and maybe an occasional movie. It was the weekend generally when people did their non-work fun stuff.

I grew up in the San Joaquin valley near Merced from 8 until I left for college. My Mom was still there when I was in Silicon Valley, and I'd visit her on major holidays. While visiting, I'd look at the real estate section in the local paper and compare to Silicon Valley.

What I saw was that the cost of reasonably nice Silicon Valley housing you could afford a nice, large house in the Merced area, on a few acres of land, with some luxuries like a pool and hot tub, and still have enough left over to afford to drive to San Francisco every other weekend and spend that weekend in a nice hotel, and eat at nice restaurants. It's only a couple hours drive between Merced and San Francisco.

Oh, and that Merced area house would not be in an HOA, and would be in an area with few zoning restrictions, so on those weekends you are not heading to San Francisco you have great flexibility to pursue your hobbies and interests. Ham radio operator? Put up as big an antenna as you like, and as many antennas as you like. Amateur astronomer? You've got reasonably dark skies. Have a lot of noisy tools in your home machine shop? No one cares--your neighbors are too far away to hear you. Into gardening? You've got lots of room for that. Like animals? You've got room for many pets, and you don't have to limit yourself to small ones that can spend most of their time in an apartment.

That analysis was for the late '80s. Merced is certainly quite different now. Back then, it was an ordinary place...it was not on the top of various "worst" lists. I have no idea if putting a company there now makes more sense or less sense.

</comment></comment><comment><USER5> I grew up in the San Joaquin Valley. I usually called it "the asshole of California", but I suppose "the armpit of California" is an equally apt moniker.</comment>


Steve Jobs & Farhad Manjoo are wrong: Dropbox can do what Apple can never do - <AUTHOR> http://garry.posterous.com/steve-jobs-and-farhad-manjoo-are-wrong-dropbo

<comment><USER2> This thing that works really well doesn't.</comment>


Mysterious new brain cell found in people - <AUTHOR> http://www.sciencemag.org/news/2018/08/mysterious-new-brain-cell-found-people <comment><USER2> Previously (~18 days ago)

[https://news.ycombinator.com/item?id=17861224](https://news.ycombinator.com/item?id=17861224)

</comment><comment><USER3> Pretty cool. Another example of how single cell transcriptomic techniques allow for the identification of cell type signatures in a way that was impossible or very difficult previously.

</comment><comment><USER4> I wonder what would happen if you could somehow grow mice with these brain cells

(Planet of the mice!)

<comment><USER5> Aren't mice already projections into our dimension of hyper-intelligent pan- dimensional beings?

</comment></comment><comment><USER6> i'm curious if further understanding may yield additional techniques used in ML/DNN or even match some of the things we have already been doing

</comment><comment><USER7> The show heroes is coming true!

</comment><comment><USER8> We finally found the raid controller. wooosh</comment>


The Case for Reparations: A slow convert to the cause - <AUTHOR> https://www.nytimes.com/2019/03/07/opinion/case-for-reparations.html <comment><USER2> Why should _righteousness_ have to wait for policy?

Surely there are plenty of old black ladies who lived under segregation--right there in DC! David can treat them to lunch at his favorite gourmet sandwich shop.</comment>


Tech Conferences Done Right - <AUTHOR> http://blog.quarterspiral.com/post/31926614896/5-steps-to-tech-conferences-done-right

<comment><USER2> Curious about one thing the author wrote. He said that the young programmer he met was one of a rare breed who liked doing game UI development. Is it viewed as not very sexy, or just plain difficult and unrewarding compared to hard- but-fun stuff like AI, or graphics?

<comment><AUTHOR> UI is the perfect storm of annoying mess for the type of coder that lives solving difficult problems. It's tedious work, takes a lot of effort and iteration but isn't truly challenging, documentation is usually spotty, etc. it tends to feel like a chore if you're not the type who cares deeply about user presentation. Plus, almost every social and mobile game needs lots of it. The supply of people who enjoy the work does not match the demand for the amount of UI work on a typical game project.</comment></comment>


Startup Quote: Kevin Hale, co-founder, Wufoo - <AUTHOR> http://startupquote.com/post/2699274777

<comment><AUTHOR> The hardest thing about decision making is not taking things personally.

\- Kevin Hale (@dreamface)

<http://startupquote.com/post/2699274777></comment>


The hum that helps to fight crime - <AUTHOR> http://www.bbc.co.uk/news/science-environment-20629671 <comment><USER2> So if one were to set up a continuous recording of mains electricity, and then provide a UI that lets you grab a slice of the hum (a time range), and sell that, you could make some rather evil money. Or, if you were the police, have your "real" recordings verified very easily.

<comment><USER3> But you'd have to get rid of the _existing_ hum, which is, as the article states, a challenge even for recording professionals.

<comment><USER4> Just thinking about it, couldn't you get an isolated recording of the hum, and then mix the inverse of it over the appropriate audio sections? That should leave you with a humless track.

\- step 1: Set up a recorder in a relatively isolated environment, ensuring that the hum is being recorded.

\- step 2: Record a 30 minute conversation with the target, ensuring that you have enough to splice together something incriminating.

\- step 3: invert the hum recorded from step 1 and mix it into the track from step 2. This produces 30 minutes of humless audio with the target.

\- step 4: edit the 30 minute recording to produce incriminating audio clip of about 30 seconds.

\- step 5: overlay legitimate audio from 30 seconds of real hum onto faked recording.

It seems like it could work.

\- step 6: Get caught for something much simpler that you overlooked, and ruin your life.

Please don't actually use this maliciously. I suppose it could be a decent defense in court if you could prove that it is possible to fake the hum.

<comment><USER5> If you're doing that, then you're better off recording your audio with balanced XLR or such like and not have any hum to begin with.

<comment><USER4> I'm not sure that an XLR feed will eliminate the hum. Ideally you could record the victim somewhere isolated from the hum.

<comment><USER5> It wouldn't entirely eliminate the hum, but a balanced feed (note that XLR can be unbalanced as well) will reduce the EM interference and cross talk that adds to the background hum.

Another contributing factor is when power supplies to the equipment are different or "dirty". Often this can be resolved with something as simple as a multi-adapter with the earth pin disconnected (note, this shouldn't be attempted unless your hardware is already insulated. but then if you're trying to commit a crime, then an electric shock is likely preferable to life sentences in jail).

</comment></comment></comment></comment></comment></comment><comment><USER6> "This buzz is an annoyance for sound engineers trying to make the highest quality recordings."

It hit me a while back when I was writing the software for a test stand for a hearing aid as a summer intern. Mysterious 60/120/180 Hz frequencies appeared on our analysis, soon to be discovered as the motors for the building's ventilation unit. It was barely noticeable to anyone, but it was quite obvious to our test equipment, even in an insulated box.

<comment><USER7> It may be that the reason was not the sound of the motors, but the effect of the motors on your electricity supply for the audio gear. Isolating the electricity (running stuff from batteries) might give more benefit than the audio insulation of that box.

</comment></comment><comment><USER4> OT, but I've seen this come up before and I wonder if HN could explain/justify this grammatical curiosity:

> _"A gang were accused of selling weapons..."_

"A gang" implies a singular entity (gang), but "were" is a pluralized use of was, as if "gang" was plural (as in, 'several gang _s_ '). (I lack the vocabulary to properly articulate myself, since grammar is not my strong suit. I am probably not describing terms completely accurately). I've noticed this more and more in regard to singular forms of entities (typically compromised of many singular parts, such as corporations). For example: "Apple were..." or "Google have..." or "Microsoft are..." I notice that this seems to be more of a British English phenomenon.

My question is this: Why are people using what I will call pluralized modifiers on what I would consider singular nouns? What I would consider the "more correct" forms of the above examples are: "The designers at Apple were..." or "Google's board of directors have..." or "Employees of Microsoft are..."

Is this just a cultural clash between American and British grammatical conventions, or is there an elusive (to me) practical reason why one version is "better" than the other?

I apologize for thread jacking. Hopefully the more relevant comments will rise above this one.

<comment><USER8> It's a quirk of British English. Collective nouns are often treated as if they were plural. [http://en.wikipedia.org/wiki/Comparison_of_American_and_Brit...](http://en.wikipedia.org/wiki/Comparison_of_American_and_British_English#Formal_and_notional_agreement)

<comment><USER9> As an American English speaker I find it delightful to read or hear occasionally. For me it's a nice, occasional reminder that groups are made up of individuals.

<comment><USER10> As a Brit who has lived in the US for a couple of years on and off it gets very confusing.

I can never remember which spelling is correct for the current situation for some terms: centre/center, tire/tyre, singular or plural collective nouns, etc. I know there are two variants, I just can't remember which one is the US one and which is the UK one; stuff like colour/color is easy, as is when to use sneakers/trainers (only made that mistake once) but some words play tricks on me.

<comment><USER11> Meh. Just be British and don't worry about it! ;-)

</comment></comment></comment></comment></comment><comment><USER12> Last time this was posted on HN, this question went unanswered: wouldn't variations in the AC line frequency be dominated by variations in recording speed? Even digital recordings have to be perfectly clock synced or they drift out of sync, and not necessarily monotonically. I've made digital recordings just minutes long that sounded fine to the ear alone, yet even after lining up the beginning and end, the middle was noticably mismatched.

<comment><USER13> It seems to me (and I don't have any formal qualifications to answer this questions) that computer clock drift is on the order of mega- or gigahertz, while this is on the order of 50hz - 5-10 orders of magnitude slower.

<comment><USER12> The speed of the clock that is drifting is unrelated to the rate or the amount of the drift. An audio sample rate of 48kHz may be driven by a 12MHz clock, and that 12MHz clock may exhibit thermally induced drift between, let's say, 11.99MHz and 12.01MHz. That will result in a sample rate drift between 47960Hz and 48040Hz. A perfect 50Hz tone recorded in those conditions will vary between 49.9583Hz and 50.0417Hz.

In the case I gave before, I was trying to synchronize a 48kHz USB audio interface recording with a 48kHz/30fps DV tape recording. If I lined up the beginning of the recordings, the ends were off by ~500ms (IIRC), which for a 2min clip means a 0.4% deviation. However, if I adjusted the speed of one of the clips to align the ends, the _middle_ would be off by 500ms, suggesting a fluctuating deviation as high as 0.8% (if my middle-of-the-night mental estimation is correct).

According to [0], the UK grid is allowed to vary between 49.5Hz and 50.5Hz, or ±1%. Watching the meters at [1], [2], and [3], it looks like deviations of 0.2% are common. Depending on the frequency of the mains and recording rate deviations, it seems mains deviation could be swamped by the 0.4% variation I observed in real-world recording scenarios. Thus, I am skeptical of the forensic utility of mains frequency analysis, and would need to see evidence that forensic analysts are compensating for recording rate deviation, or arguments why it's irrelevant, before I would change my mind.

[0] [http://www.nationalgrid.com/uk/Electricity/Balancing/service...](http://www.nationalgrid.com/uk/Electricity/Balancing/services/frequencyresponse/)

[1] [http://www.nationalgrid.com/uk/Electricity/Data/Realtime/Fre...](http://www.nationalgrid.com/uk/Electricity/Data/Realtime/Frequency/Freq60.htm)

[2] <http://www.dynamicdemand.co.uk/grid.htm>

[3] <http://www.mainsfrequency.com/>

<comment><USER14> I think you are missing how much the spectrum of the drift matters. Just knowing the range over which the clock varies (say 5%) isn't enough. If the drift is slow - which thermally induced drift usually is, because it's driven by daily heating/cooling cycles - then it's possible to correct for it in the recording. Analysis techniques which are based on frequency-domain variations would tend to reject this type of slow drift automatically, but the details depend on the technique used. If, on the other hand, the noise on the clock is fast jitter rather than slow drift, things become much more difficult. On typical consumer recording devices, the noise floor due to jitter is way below the noise floor due to the limited SNR of the microphones and amplifiers used. The jitter noise is non-linear, which makes things harder, but doesn't tend to be a limiting factor.

It's been most of a decade since I worked on this stuff (in the context of radar and sonar), so that knowledge may be out of date. Still, I'd be surprised if clock (LO) jitter is the limiting factor in this type of analysis.

</comment></comment></comment></comment><comment><USER15> It sounds cool, but I wonder how reliable this method is. What's the false positive and false negative rate? Errors here would have real consequences -- for example, according to the article, it was used as a crucial piece of evidence in putting several men behind bars for decades.

</comment><comment><USER16> Great. It's a modern day equivalent of the lie detector for recordings.

<comment><USER17> > Great. It's a modern day equivalent of the lie detector for recordings.

Unless it's a lot more reliable than the polygraph, it shouldn't be admissible in court, either.

(The polygraph, sometimes called a 'lie detector', does not create evidence admissible in court. It has too many false positives and false negatives.)

<http://antipolygraph.org/>

[http://www.usnews.com/news/blogs/washington- whispers/2012/09...](http://www.usnews.com/news/blogs/washington- whispers/2012/09/25/nsa-whistleblower-reveals-how-to-beat-a-polygraph-test)

<http://www.skepdic.com/polygrap.html>

<comment><USER16> I agree with you.

</comment></comment></comment><comment><USER18> So why is an article about the Metropolitan Police forensic lab in south London (UK) illustrated with a stock photo of what appears to be a US power socket? (It might be a Euro socket, but it's certainly not a UK one.) Do they not have electricity sockets anywhere at the BBC that they could have taken a picture of?

<comment><USER19> Those are identical to the outlets I saw in Germany, so I guess they're European. Definitely not American though.

<comment><USER20> Yep, they're German.

</comment></comment></comment><comment><USER21> The mains frequency is an interesting topic. While it is true that it floats still its daily average is purposefully kept at quartz-like stability so that clocks can use it for synchronization. It also needs precise phase synchronization across the whole network (otherwise generators would blow up.)

<comment><USER7> I'm not so sure if the daily average is kept so stable actually (or maybe it depends on the country).

I worked at a power grid a long time ago, and I recall that in days where the grid had undersupply issues due to climate and high demand, the frequency was kept stable at 49.5-49.6 Hz for weeks or so.

The clock-sync is a nice wishlist feature; but the frequency affects power consumption which often may be more important or financially valuable.

<comment><USER22> In some of the power grids in the US, the clock-sync thing was something that the operations were required to maintain over a 24 hour period. One technique used to monitor this was a simple electric clock. Electric clocks have synchronous motors, meaning that changes in frequency of the supply would change the time. The target was to be sure that at, say, 6pm, that the clock would show exactly 6pm, even though it would vary a bit during the day.

</comment></comment></comment><comment><USER23> Curious, is this "mains frequency" the same ringing I often hear when in alleged silence? And, no, I'm not talking about my hearing because I've had conversations with others who witnessed it :P - more like the ring old tube TV's used to make.

<comment><USER24> No, the frequencies emitted by CRTs are sonic (this is EM) and are much higher in frequency.

You can't hear mains hum without a speaker (or transformer acting as a speaker).

<comment><USER7> I believe that you should be able to hear overtones (2* or 4* mains freq) coming from lightbulbs and power supplies (computers, chargers, etc) in an average room - it could be tested by running a decent mic + solidstate recorder on batteries and checking how loud it is.

</comment></comment></comment><comment><USER25> Application idea: locating criminals, hostages using video / audio recordings

<comment><USER26> It can be defeated by jamming the relevant frequency bands, i.e. by overlaying other mains recordings with random fluctuations. No deed to remove the original.

I'm not sure a notch filter would be enough because, even though the mains is supposed to be a perfect sine wave, some harmonics occur (sometimes with a much higher frequency). Perhaps by filtering all harmonics as well? Sound quality isn't exactly a must in these situations.

It requires a technically astute criminal, though, and most aren't.

<comment><USER25> _It requires a technically astute criminal, though, and most aren't._

Exactly. Many don't even use gloves. I was wondering on how precisely you could locate the hum and if you could deliberately create a hum signature for a region.

</comment></comment></comment><comment><USER27> I'm a little disturbed that they are using this as evidence, yet one of the "forensic scientists" is quoted as saying

      Normally this frequency, known as the mains frequency,
      is about 50Hz," explains Dr Alan Cooper

That is incorrect. Normally it is _exactly_ 50Hz. That's why the time displayed on battery-powered clocks drift over time, but clocks plugged into the wall stay correct.

<comment><USER28> It is not very exact. The total phase error can be hundreds of cycles.

 _In the synchronous grid of Continental Europe, the deviation between network phase time and UTC (based on International Atomic Time) is calculated at 08:00 each day in a control center in Switzerland. The target frequency is then adjusted by up to ±0.01 Hz (±0.02%) from 50 Hz as needed, to ensure a long- term frequency average of exactly 50 Hz × 60 sec × 60 min × 24 hours = 4,320,000 cycles per day.[21] In North America, whenever the error exceeds 10 seconds for the east, 3 seconds for Texas, or 2 seconds for the west, a correction of ±0.02 Hz (0.033%) is applied. Time error corrections start and end either on the hour or on the half hour._

\-- <https://en.wikipedia.org/wiki/Utility_frequency>

</comment></comment><comment><USER29> Any thoughts on how "unhackable" this system is?

<comment><USER30> The general premise is that there is an arbitrary, recoverable signal which has been convolved with the data signal. The generating function for this signal is a matter of record if you have access to the utility logs, but difficult to obtain otherwise without being physically at the time and place where the effect occurred.

I strongly suspect that it can be compromised under both of the following conditions:

1\. You take recording A, then you take recording B at a time and place which you want to assert that recording A took place at. You recover the thumbprint from recording B, suppress the thumbprint in recording A, then apply the thumbprint from B to A.

This is not a trivial process, but you only really need a plausibly consistent result. A reasonably basic understanding of signal processing theory, a copy of MATLAB, and many pots of coffee should do the job. Then, you could automate most of it for the next guy.

2\. You take many recordings at a series of locations of interest, while taking data about the power grid from nearby locations and from distribution nodes. You then attempt to predict the signal at a location from the characteristics of the surrounding area.

This is almost certainly possible, as generalization from distribution logs to the local effect is what makes their fingerprinting technique possible in the first place. It is not a trivial undertaking, and it's questionable how well it would be generalizable. But at the same time, it's largely a question of if you want the data badly enough to do the legwork, and whether you have a reasonably functional understanding of machine learning.

<comment><USER31> Regarding point 2, the article gave me the impression that they used historical measurements of the the mains frequency as part of the analysis. Since Britain is on a single grid, the local conditions shouldn't affect the recorded signal. I'm sure it is possible to find patterns (frequency drops slightly in the morning as the public utility tries to keep up with increasing demand), but you wouldn't know, for example that the utility overestimated demand on the particular morning that the audio was alleged to have been recorded (and thus that the frequency was actually higher).

As for point 1, there are a number of plausible angles to approach this, but I think that the forensic adversary has a huge advantage: Synchronous detection. The approximate time of the recording is known, as is the historical record of mains frequency. That allows for the possibility of huge processing gain, which might allow for recovery even after the amplitude of the mains hum is filtered to below the quanta of the audio system. I almost think you might have to Fourier Transform the whole audio record and zero out any component at f_mains +/- delta (and harmonics). That, of course, would look pretty suspicious to a forensic analyst.

Even that might not be enough if mains hum has a determinable effect on the data compression algorithm used to store the audio data.

Edit: The point I was trying to make in the first paragraph is that if you wanted to forge a recording, you'd need to have the grid frequency data. Having it for anywhere would be good enough and not having it for anywhere simply wouldn't.

</comment></comment></comment><comment><USER32> > If millions of people suddenly switch on their kettle after watching their > favourite soap, the demand for electricity may outstrip the supply, and the > generators will pump out more electricity, and the frequency will go up.

<http://en.wikipedia.org/wiki/TV_pickup>

</comment><comment><USER33> Obviously, if a building is off the grid then there wouldn't be the same background hum, but what if the building has its own energy generation system (e.g., solar) and feeds back to the grid?

</comment><comment><USER34> I wonder if the same technique could work for the microwave range and the cosmic microwave background. That is, is there a discernible "hum" in microwave signals from the CMB?</comment>


Did you know that WWW is deprecated? - <AUTHOR> http://no-www.org

<comment><USER2> Although I agree that "www" isn't really needed, this is one of those conventions that should have been corrected on day 1 or not at all. To change it now only invites people to encounter more errors than necessary.

There's no way 100% of web sites will change to this. So maybe half the time I'll get an error if I get in the habit of writing web page links to use "xyz.com" instead of "www.xyz.com", or typing them as such in my browser.

Conversely, no business in its right mind would accept only "mybusiness.com", scaring customers away by returning an error for "www.mybusiness.com". Of _course_ they'll accept both, so why not use the one that's sure to work?

</comment><comment><USER3> I didn't know it was deprecated, but I know people still use it. Especially in casual conversation between non-geeks. I hear it all the time. "Oh, I got the CD to get on to the internet at Double-U Double-U Double-U Dot AOL Dot com"

That last part was a joke, but people still use it.</comment>


Location matters for your startup (with city comparisons) - <AUTHOR> http://scott-allison.net/2011/10/21/location-matters-for-your-startup/

<comment><USER2> Why density? Does density _really_ matter? Isn't the absolute number more interesting because whatever be the chance of meeting an important (to your success) person, you have a bigger pool to pick from.

Case of city A: Say, there are 50,000 residents of which 500 are of interest. There's an event hosted and 25% of those "interesting" people show up. You, who's attending this event, have a pool of 125 people out of which, say 5% can help you AND you'll bump into. So chances are you'll meet _6 people_. Density: 0.01.

Case of city B: 5,000 residents, 100 interesting, 25 show up, chances are you'll meet only _1 person_. Density: 0.02 (double).

<comment><AUTHOR> I think the truth is that the absolute number, along with density are both important. That's why Boulder - apparently 70% more entrepreneurial than Silicon Valley - cannot really claim to be better than SV, which has an entrepreneurial population 15 times larger.

<comment><USER3> I was really surprised to see Boulder on that list... had no idea that there were apparently many start ups there, even if it is just by percent.

<comment><USER4> Boulder's startup scene has skyrocketed in the past several years. At monthly startup meetups, hundreds of people usually show up to check out what everyone else is working on.

It's not quite a Silicon Valley, and its score is probably exaggerated by the difficulty in defining a population area that's relevant to the startup scene. However, most startups in the scene are in Boulder proper, and most founders seem to live in the city as well. So the density is there, and it's quite easy to run into other startup people on the sidewalk or in coffee shops.

</comment></comment></comment></comment><comment> brown9-2 I don't think takes away from the main point of the article, but there is a bit of a logical fallacy in the opening:

 _18 months ago I relocated from my home town of Glasgow, to London, just 400 miles away. An important reason for the move was because I had just started working on my new startup, Teamly, and I know that location matters, even when running an internet business. Don't kid yourself otherwise, your chance of success is seriously improved when you're in a startup hub.

18 months later and moving to London has proved to be a smart move, for all the expected reasons, as well as the unexpected recognition by the UK Government of London's startups with the launch a year ago of "TechCity"._

I don't think you can really say that moving to London to operate your business there versus operating in Glasgow was a "smart move", as you have no way of knowing what level of success your startup would have had in Glasgow (since you never experienced this alternate reality).

Sure you made a lot of great connections as a result of your move, but it is impossible to know what connections you never made because you weren't in Glasgow during those 18 months. There is no way of knowing whether you missed out on something (a chance meeting/encounter/coffee, etc) which, while it may have been less probable, would have been of more value to the success of your startup.

<comment><AUTHOR> Thanks for pointing that out, but putting it another way, the things I expected I would benefit from by being in London did occur, along with the pleasant and unexpected bonus of the government's tech city initiative.

</comment></comment><comment><USER5> Location matters. Even if you're boot strapping. I'm boot strapping Hackers & Founders.

In the past three months, I've met a CEO of a 300 person gaming company, M&A execs from Fortune 500's, Engineer[0] from Google Finance, the head of the Singularity Institute, artists that make 2 story flaming sculptures, an architect that works on computational architecture, a man who built a battery that charges in 10 minutes that is currently in use by Special Forces in Afghanistan, a former Yahoo VP, a VP from AT&T, the CTO of America and the man who wrote the Jobs bill... a person on the US Council of Foreign Relations, a man who helped the Arab spring flower in Egypt...

That's not to brag. I've simply been extremely deliberate about networking over the past 5 years.

In Silicon Valley, even after being here for 6 months. You probably don't realize it, but you're only 2 or 3 degrees of separation from everyone else who's really, really impressive in town.

To that density, add a culture that places an extremely high value on helping others out, and you have something amazing.

<comment><USER6> Are there any practical (mathematical?) techniques to measure "openness" of a startup community's network?

You may be 2-3 degrees from John Doerr, but the question is, could you actually get a meeting with him?

Just my intuition, having lived in the Valley (8 years) and Austin (10 years) is that the Bay Area gets a B- for openness, compared to Austin which gets an A-.

<comment><USER5> _You may be 2-3 degrees from John Doerr, but the question is, could you actually get a meeting with him?_

That depends.

Adam Rifkin, the founder of the 106 Miles meetup and pretty kick ass nice guy and serial entrepreneur, was funded 12 years ago by Kliener Perkins. So, if you got to Adam's meetup, and bump into him, you're now 1 degree of separation away from John Doerr.

So, if you take Adam out to coffee, and talk to him about getting an introduction to John, Adam can certainly provide an into to John, but he's far more likely to talk to you, get a feel for what you're doing, and then provide introductions to 10 other investors and amazing people that could probably provide as much or more value than John Doer.

Good question.

There's also a very strong culture in Silicon Valley of "quality introductions".

The perceived value of a person's personal network in Silicon Valley is directly proportional to the quality of introductions that person can make to others. (Read: matching person to investor)

</comment></comment></comment><comment><USER7> I'm sure that this is true, but it's awful and broken and we ought to find ways to fix it. There's way too much talent and energy spread all over the world, and frankly the group-think that melts young, brilliant brains into caring way too much about mobile apps for taking pictures of your food is sad, and boring.

<comment> All-ln Quite true,but one would argue how many of those minds are actually brilliant.I guess that's what happens,when far too many people are "doing they're own thing" and "solving needs/wants"

</comment></comment><comment><USER8> A more important factor compared with density is, in my view, whether startups are the main game in town. This determines what kind of talent you can attract.

In London you are competing with cash flow oriented industries that will set expectations for the compensation mix, so you are in practice left with somewhat marginal people and not the best. This crowding out effect, compounded by the relatively small exits produced in the UK and an inferior work ethic compared to the US, explains why both founders and employees are less good on average than in some other startup hubs.

For the UK startup scene to really succeed we basically need to destroy the banks and agencies.

<comment><USER9> For the UK startup scene to really succeed ..... how about if people located their startups in places where people can actually afford housing costs ? such as - Sheffield, Leeds, Birmingham, Nottingham, Newcastle etc .

<comment><AUTHOR> I completely disagree with this, simply because there is already a fantastic and large hub in London. There's hundreds of startups here already and proximity to VCs, in one of the world's best cities. It makes a lot more sense to build on something that's already strong, and make it stronger.

I think one of the smartest things this Government has done is to recognise that and ignore calls from people outside London to create something elsewhere. But yes, London is frigging expensive. That said, you can live close to, but outside London and it's no more expensive than the other places you mention.

<comment><USER9> I acknowledge London is in many ways a fantastic place to live and grow a business. Its great for the UK if we can have a growing startup hub based in London. However, while many startups will flourish in London, so many others would be completely dead in the water if they tried to grow while facing London costs. Living just outside London is surely more costly than living in a comparable area of a city further north, when you take into account a rail season ticket? Also, commuting times get silly. There's loads of talent scattered across the UK. Whats the point of trying to get that talent to move somewhere where everyone else already wants to live, and quality of life in many ways is poor as result? Silicon Valley is different because they actually pay enough for people to live there. Maybe that's the answer - startups need to start paying £70k to developers in Shoreditch when the same person would have been happier being paid £20k to live in Derby? Thanks for the response though- its interesting to debate these things. :)

</comment></comment></comment></comment><comment><USER10> If you calculate backwards from these figures to the absolute size of the community, you find that NYC's startup scene is actually _twice_ as large as San Francisco's. As someone from the NYC area, that doesn't seem plausible.

<comment><AUTHOR> You're right, it's the other way around.

</comment></comment><comment><USER11> Unfortunately it's almost impossible to move to these hot cities from many countries. Visa is really a problem. So we need to work hard to get our own local startup hub. I'm from Armenia.

<comment><USER12> Agreed. Not just in other countries, but many rural areas of the US. There are some incredibly talented people out there that may not be able to move for one reason or another. How do we bring these startup hub advantages to everyone?

<comment><USER13> This is how:

[http://steveblank.com/2011/06/01/why-board-meetings- suck-%E2...](http://steveblank.com/2011/06/01/why-board-meetings- suck-%E2%80%93-part-1-of-2/)

[http://steveblank.com/2011/06/02/reinventing-the-board- meeti...](http://steveblank.com/2011/06/02/reinventing-the-board- meeting-%E2%80%93-part-2-of-2-virtual-valley-ventures/)

<http://www.leanlaunchlab.com/>

<comment><USER11> Last one looks similar to Lean Startup Canvas - <http://leancanvas.com>. Though not sure how is it helpful in this case.

</comment></comment></comment></comment><comment><USER6> Seems like a better approach would be to identify cities with the most startup "clusters"? Identify a cluster, then look at the population(s) of that census tract or zip or whatever.

Using city-level population just makes small cities look great (Boulder) and big cities look terrible (NYC), when I'm pretty sure there are neighborhoods in NYC that blow away Boulder.

<comment><AUTHOR> Agreed, a hybrid of absolute numbers of startup people and density is probably the most appropriate.

</comment></comment><comment><USER14> Don't relocate, bootstrap :)

<comment><USER15> I completely agree. Your customers will be your best source of funding.

<comment><AUTHOR> It's not all about funding... it's about the less tangible benefits of being around other entrepreneurs.

</comment></comment></comment><comment><USER16> In Los Angeles this really means Santa Monica and maybe El Segundo. Basically closer to the ocean.

</comment><comment><USER17> Good points, well made - I'll be over to SF in Jan!

<comment><AUTHOR> Look forward to seeing you there!

</comment></comment><comment><USER18> I'm relocating to the Bay area in December.

<comment><AUTHOR> As you're in Seattle, it would be interesting to know why you're making the move.

</comment></comment><comment><USER19> The density/city comparison thing is really hard to measure, but I'd argue it doesn't make a whole lot of sense:

Since people are showing off re SV, I'll do the same for Chicago- where I've in the past couple of weeks met: CTO for Barack Obama (ran into at falafel place), Founder of Groupon (stood next to at Weezer concert sponsored by NEA, talked to him for a bit), Dave McClure (He's an investor in us, we had dinner. He comes to Chicago often and spends a lot of time with us because we were his only investment here until last month), VP from Motorola Mobility (architecture tour), State Senator from Ohio (bar), A Macarthur Fellow Architect (female, met at bar), Vice Chairman for Goldman Sachs Investment Banking (happy hour), etc etc etc.

Furthermore, just by chance (ie we didn't know this going in and it is NOT the startup hub area within Chicago), within 3 blocks of FeeFighters office are: 37signals, SproutSocial, Threadless, Crowdspring, and several other startups. I don't see how it helps us significantly to bump into them (though we have helped our neighbors when they've had merchant account issues). Many people think it is a wasteland anywhere not the valley but I have seen that not to be the case here, in New York, and even in Pittsburgh.

So what is the difference between here and there? The largest differentiator I've noticed between the valley and elsewhere is funding. We raised most of our money in Chicago from VC's we really like, but could have raised easier on the west coast. A lot of top VC's prefer to have you close and told us flat out they would fund us if we moved there. We had term sheets dependent on us being in Austin, SF, and LA. I don't think it has had an effect one way or another on our business thus far.

Startups need money, and often flock to where it is. Things are changing all over the place though (Rich people in Chicago all of a sudden want to fund startups, will be interesting to see if that continues post-Groupon IPO) It also depends on your type of business. If you sell mainly to startups, you should probably be in the valley.

BTW in case you missed the discussion re: PG's original article, it is here: <http://news.ycombinator.com/item?id=3077165>

<comment><USER20> A local (Chicago) angel/entrepreneur & friend of mine told me that it is a lot harder to get follow through from VCs & angels in Chicago. I wonder if your experience comports with that view.

I personally have pessimistic view on the Chicago area in the intermediate term (5-10.)

Regards, TDL

</comment></comment><comment><USER21> Just remember quality of life with these things.

I know this will be an unpopular opinion round these parts, but SF is not somewhere I'd ever chose to live. The bad points vastly outweigh the good points for me.

<comment><AUTHOR> I guess everyone is different, I love silicon valley for many things, the quality of life being one of them (and the weather). Maybe it's just me but I'm from a cold, wet and windy country called Scotland :)

<comment><USER21> Actually weather is one reason I wouldn't live in silicon valley - it doesn't really have any. It's either flatline boiling hot, or if you go north it's windy and foggy all the time. It's like being stuck in a never ending summer.

I live in England, where we get glorious boiling summers, wet and windy autumns, snowy winters. I know that's not for everyone, but the sheer variety is something I love.

To see the scenery and wildlife change by season is something that really makes life interesting.</comment></comment></comment>


SpiderKeeper: Admin UI for scrapy/open source scrapinghub - <AUTHOR> https://github.com/DormyMo/SpiderKeeper <comment><USER2> SpiderKeeper least received an update 2 years ago. Scrapyd last received an update 9 months ago. Scrapy (the actual spider) is being actively maintained.

<comment><USER3> Loads of not maintained open source projects make it to the front page lately..

</comment></comment><comment><USER4> This project seems like a more updated fork although it's also not maintained

[https://github.com/fliot/ScrapyKeeper](https://github.com/fliot/ScrapyKeeper)

</comment><comment><USER5> Are these spiders able to fetch content that loads in segments after scroll events?

<comment><USER6> Scrapy doesn't run javascript so it can't scrape client rendered content at all. There are projects that combine scrapy with a headless browser or javascript engine.

</comment></comment><comment><USER7> How does this compare with a general solution like Apache Airflow?

<comment><USER8> The following is a simplification but should suffice.

Airflow is for hooking together and managing a bunch of data sources, data- processing nodes, and data sinks.

This project seems like a GUI for Scrapy, which is a web crawling framework, i.e. one kind of data source.

</comment></comment><comment><AUTHOR> Anyone here using this in production (even for a side project)?

What are some of the challenges you've faced as opposed to just using ScrapingHub's Scrapy Cloud service?

</comment><comment><USER9> All such admin tools use scrapyd which is not maintained and not good enough to be used in production.

<comment><USER10> What do you recommend using instead?

<comment><USER11> One of the companies I've worked for developed in-house solution (with some domain-specific stuff).

I've had to write some kind of admin tool for one of my clients.

And then I've used to start another project with scrapy and I feel like I need to develop another one tool for scrapyd management :)

</comment></comment></comment><comment><USER12> I think it can faster than the normal then a basic function. ti increase the case memory.

</comment><comment><USER13> Nice. I'm stealing this concept for my scraping engine.</comment>


Subreddit connections - <AUTHOR> http://redditstuff.github.io/sna/ <comment><USER2> It's unfortunate you removed all the primary subreddits (or degree > 75), it makes a lot of the networks confusing (the gaming network, for example) because they seem to only show niche topics.

<comment><USER3> Agree on that point. Also, I believe you have an extremely useful data set here, but the visualization of it could be improved. My goal when looking at the graph is to discover other subreddits that may be of interest, but this is very difficult to do in the current form.

Perhaps a way of inputting and highlighting subreddits of interest (as a short term solution)?

</comment></comment><comment><USER4> This is really well done! I would recommend putting the defaults back in and playing around with and finding a good value for "remove connections that have less than" using a number based on a percentage of the posts in sub instead of an arbitrary 8 connections.

For example, remove connections from /r/programming to subs that are cross referenced from /r/programming less than 5% of the time.

Again, very nice!

</comment><comment><USER5> Not surprising that the drama bomb meta subreddits are the biggest connectors (TrueRedditDrama being the largest graphically).

I personally think the overall quality/SNR of the entire site would go up if meta subs (a sub who's sole purpose is to link to other subs) were banned outright. They don't seem to do anything but stir up strife and abuse.

<comment><USER6> A counter-example: r/DepthHub is a meta sub which is both civilized and edifying.

<comment><USER5> Thanks for that. I've been looking for a counter-example to the rule of "meta subs are crap" for a while.

<comment><USER7> This link I came across has a nice sampling of particularly insightful (mostly meta) subs: [http://www.reddit.com/r/depthhub+academicphilosophy+aestheti...](http://www.reddit.com/r/depthhub+academicphilosophy+aesthetics+theagora+culturalstudies+econpapers+foodforthought+hardscience+indepthsports+indepthstories+laymanjournals+ludology+neurophilosophy+philosophyofscience+politicspdfs+resilientcommunities+scholar+stateoftheunion+truefilm+truereddit+worldevents+interview+designthought)

<comment><USER8> Thanks, didn't know about the plus trick that allows to read posts from multiple subreddits in one feed.

</comment></comment></comment></comment></comment><comment><USER9> With others, I recently played around with a custom built dataset, using unsupervised machine learning to cluster the subreddits, based on content.

You can play with it here:

<http://demo.xplr.com/xplr/umbreddit/>

The system can be queried:

<http://demo.xplr.com/xplr/umbreddit/?search=cyclists>

<http://demo.xplr.com/xplr/umbreddit/?search=species>

For the curious, more technical information is available here: [https://xplr.com/xplr-umbrella-dataviz-on-top-of- unsupervise...](https://xplr.com/xplr-umbrella-dataviz-on-top-of-unsupervised- machine-learning/) and here: <https://xplr.com/a-subreddit-recommender-with- xplr/>

This analysis is slightly different than the study of cross-posed links. Here the content of posted URLs (not yet comments)is analyzed and subreddits are put into clusters within a search engine.

This allows the easy building of a subreddit recommender for Reddit (Chrome only for now): [https://chrome.google.com/webstore/detail/preddit/epicmjpmnm...](https://chrome.google.com/webstore/detail/preddit/epicmjpmnmjgbmahjcigppkenngbdjbd)

</comment><comment><USER10> This is really neat, but the magnify effect makes it absolutely infuriating to hover over one of the points to see its label.

<comment><USER11> I really like the magnification effect. One of the things I don't like about these weighted graphs is that mousing to tiny targets can be a huge PITA. The magnification rectifies this.

<comment><USER12> I think a flat magnification would solve this. (eg real magnifying glass). The "curvy" magnification is simply annoying.

</comment></comment></comment><comment><USER13> Very neat! I wonder if there would be a decent way of including more of some of the default subs without adding too much noise.

</comment><comment><USER14> This is really useful to me, I'm always wondering about specific subreddits that may interest me. Now I can actually find them. I think there may be other ways to present this information to help people discover subreddits of interest.

<comment><USER9> Check this out maybe, [http://www.reddit.com/r/MachineLearning/comments/1dzoim/a_su...](http://www.reddit.com/r/MachineLearning/comments/1dzoim/a_subreddit_recommender_with_machine_learning/)

</comment></comment><comment><USER15> Similar work:

<http://www.cs.utexas.edu/~elie/networks.html> (on HN: <https://news.ycombinator.com/item?id=4914267>)

[http://ajverster.github.io/blog/2013/04/01/redditinteraction...](http://ajverster.github.io/blog/2013/04/01/redditinteractionmap/)

</comment><comment><USER16> Can anyone find the cluster of technical/programming subreddits? Closest ones I see are the Startup hub, and java_help. I know they're in there somewhere though.

</comment><comment><USER17> Nice work! This got me to venture into some sad subreddits though... Had no idea that /r/niggers was even a thing (wouldn't it even be illegal?).

<comment><USER18> Why would something that's distasteful be automatically illegal?

<comment><USER19> It's not just distasteful. There is legitimate hate speech there, which some countries deem illegal.

<comment><USER20> Legitimate hate-speech.

"No sir, ours is 100% legitimate!"

<comment><USER19> As opposed to "sarcastic" hate speech, like a joke you might hear on Tosh.0. These people really do believe the things they say, and they are intent on spreading their vitriol as far as they can.

I'm not sure what point you were trying to make, though. Unless you really didn't understand what I meant?

</comment></comment></comment></comment></comment><comment><USER21> Wow, I didn't realize a gephi file can be so easily exported to an interactive web object via sigma.js -- great, simple implementation!

</comment><comment><USER22> Off topic but I don't like the "best viewed in Google Chrome". Your code should be standards compliant and it will work everywhere.

<comment><USER23> Why? This looks like a side project, a bit of fun. Author should be free to knock it up how they want.

It should also be noted standards compliant !== work everywhere.

</comment></comment><comment><USER24> For some reason this runs terribly on firefox, blank in ie, chrome works great however (which you kindly point out at the end!)

</comment><comment><USER25> "...we are all Touched by His Noodly Appendages"

</comment><comment><USER26> This reminds me vaguely of Randall Munroe's Map Of Online Communities:

<http://xkcd.com/802/></comment>