Show HN: Mux Video, a simple API to powerful video streaming - <AUTHOR> https://mux.com <comment><AUTHOR> Hi there - Jon from Mux here. We've been working hard on this product for the last 6 months and are excited to open it up.

Try it out and we'd love feedback. Free to start.

As background, here are a few blog posts about what we're doing and why.

Video is too hard - [https://mux.com/blog/video-is-too- hard/](https://mux.com/blog/video-is-too-hard/) An API to Video: Why Abstraction Matters - [https://mux.com/blog/an-api-to-video-why-abstraction- matters...](https://mux.com/blog/an-api-to-video-why-abstraction-matters/)

<comment><USER3> Yes yes a thousand times yes.

</comment></comment><comment><USER4> I'm building a web app that needs video streaming but ultimately serves to provide high-quality image extraction from the videos. I've procrastinated solving the video delivery part and so your service intrigues me. While we have your team on the line, I have some questions :):

\- Is this a crazy use of your service?

\- If I request the full-size thumbnail will it be a lossless image from the original video?

\- Your front-page says "With Mux Video request a single thumbnail with a simple request or an entire storyboard to use in your player to scrub preview images." but the docs don't mention anything about storyboards. Is it not done yet? Or is the implication here that I can just make several separate requests to the thumbnail API to build a storyboard scrubber?

\- One of your pages says "Note that there is a default limit of 30 thumbnails per asset. If you need more, contact Mux." but the thumbnail API doc doesn't mention this limit. Is that actually a limit?

\- The thumbnail API doc says the timecode param is a float. Does this mean that if I calculate the timecode based on the framerate I can get high- precision, frame-by-frame stepping in my video player? Any chance you'll offer a version of the thumbnail API that takes the frame instead to make frame- stepping easier to build?

\- Can I send you a hard-drive full of videos to ingest into your system?

\- Is a signed playback_policy the only way to secure the video from public viewing, and if so, when will this be implemented (doc currently says "signed: (coming soon) ...")

\- Your pricing seems extremely reasonable but I'll be looking to cut corners at least initially. If I require fewer output formats (e.g. I don't need to support mobile devices, only desktop browsers), can I get a cheaper storage rate? I'll ultimately have high storage needs (starting with 31,860 minutes of video and then eventually getting closer to 100-200K) but low streaming demand.

<comment><USER5> Ok going to try and answer these in order:

> Is this a crazy use of your service?

Nope, not a crazy use of the service at all! To be blunt, it might not be the most profitable for us, but we tried to structure our pricing that we could support edge cases.

> If I request the full-size thumbnail will it be a lossless image from the > original video?

If you give us an h.264 input that is 1080p or less and request a png thumbnail without specifying width and height, the thumbnail will be lossless (no _additional_ loss over the png compression, to be precise for the pedants out there).

> Storyboards

Storyboards are working and in the final stages of development. We should have that documentation updated in the next few days.

> 30 thumbnail limit

That's a soft limit. We can do more than that, but we should talk if you're going to be blowing through that limit consistently.

> The thumbnail API doc says the timecode param is a float.

Yes, if you have constant framerate video. It's technically a 64bit floating point, so it's highly accurate. Due to the complexities around variable frame rates, we don't support frame-stepping in our API.

> Can I send you a hard-drive full of videos to ingest into your system?

One of our engineers volunteered to take care of it, so sure, we can make that work as a one-off. Probably not going to put that on the marketing site anytime soon and deny this conversation ever happened in a year :)

> signed playback_policy

That will be available very soon, followed shortly by other types of restrictions (geo, domain-based, etc). Anything specific you'd like to see there?

> pricing

We're happy to talk! We actually only store the mezzanine long-term and then encode into the different bitrates at request, so that's not the only consideration for storage. Feel free to shoot me an email (matt at our domain).

</comment></comment><comment><USER6> This looks great! But my question is... what's the story behind the three- letter domain name? If you bought it, was that for 6-7 figures, and what made you decide to invest that much money in the brand?

<comment><AUTHOR> Great question. Yes, it was quite expensive. We were able to negotiate a deal where we buy the domain over a long period of time, with a relatively low cost for the first few years, a much higher cost for the last few years, and the option to cancel. If we grow/sell/IPO, no big deal. If we die, we can stop paying. We just have to do it within 5 years or so. :)

<comment><USER7> That's certainly a very interesting way to structure a domain deal. If you don't mind me asking, was this through a specific marketplace or via private sale? A domain marketplace with this model could be very beneficial to bootstrapping startups that could use the branding of an expensive domain, but don't necessarily have all of the capital yet.

<comment><USER8> Hi, lauren here from Mux. It was a private deal negotiated through a broker and escrow agent. The price only gets higher if your company is successful (maybe why eshares just changed their name to Carta), so better to lock in a deal early (with the bulk of payments down the road).

</comment></comment></comment></comment><comment><USER9> My company is very interested in moving into this space (building a video community and charging for paid videos not hosting & processing like Mux).

We might be very interested in Mux, however, here is my question and please forgive my ignorance: isn't this pretty much what you can do with Vimeo via their api?

<comment><AUTHOR> Hey clintavo - your use-case is exactly what we built Mux Video for.

One way to think about it: Mux is more like a EC2/S3-level service, while Vimeo is more like a Squarespace-level service.

We think Vimeo is a great platform for publishing video if you want a platform that goes all the way from CMS to streaming to player to sharing. But if you want to build video into your own application, you might want something a bit lower-level, closer to a web service, that lets you bring your own CMS, player, workflow, etc. Our primitive is just `video` and you can do whatever you want with it.

<comment><USER9> OK, but I looked at Vimeo's api docs and they looked pretty feature complete for someone who wanted to build their own site and never use the vimeo CMS. It's possible (it looks like) to add video all through the api, and simply host the videos on Vimeo infrastructure. They even have the ability to lock videos so that they can ONLY be streamed on certain domains (so you can paywall videos etc). Am I mistaken about that? If not, is there an advantage to Mux?

<comment><USER10> (Mux founder) The Vimeo API is an API to the full Vimeo platform, which is an incredible platform all together, but requires using their player, at least according this comment from a Vimeo employee on stack overflow.

> Video files are not available for basic or Plus users, and are not available > for users beyond the one making the request. For those videos you should use > the embed codes we provide in the API request (under the key > response.embed.html, or through oEmbed. > [https://stackoverflow.com/questions/31384129/is-it- > possible-...](https://stackoverflow.com/questions/31384129/is-it-possible- > to-get-streaming-url-with-vimeo-api)

We want people to build new Vimeos on top of Mux, and new interesting uses of video.

With Vimeo you would start to run into limitations when you want to build custom experiences around the video, new player features, tracking specific analytics, advertising. Really any customizing of the player outside of what Vimeo feels is important for the Vimeo platform. The Vimeo player is also very recognizable. If you're hoping to brand the player as your own, that would be difficult. We have one customer switching off of YouTube because they currently serve premium (paid for) content from a private youtube playlist, and their customers assume that because it's the YouTube player, it must be free online somewhere. I'm not sure if you'd get the same issue on Vimeo but I thought it was an interesting detail.

<comment><USER9> OK, that makes sense. Is it possible on your platform to make videos private? IE, I add a video, make it private and then add something that says "this video can only be played from subdomain.mydomain.com" (which I could then have behind a paywall of course).

I ask because we're working on a solution for our users and we have 10,000 of them.

<comment><AUTHOR> This is something we're working on. If you look at our API, we have a concept called "playback_policies" that will encompass this. [https://docs.mux.com/v1/reference#create-an- asset](https://docs.mux.com/v1/reference#create-an-asset) At first, it's just "public" and "private", but we'll soon have "signed" (e.g. short-term expiring URLs), and will then build in other security options, like georestriction and possibly domain-based restriction. We'll probably have to add full-on DRM support as well at some point.

Would signed URLs work for your use-case?

<comment><USER9> It's likely a no go for us without domain based restriction. I'd rather do that then deal with signed urls, we wouldn't want them to expire. We deal with artists who have painting instruction videos, they're going to want to have those behind a paywall on their own domains (and then we'll offer them in our marketplace on our domain too).

<comment><USER10> Thanks for the feedback. It's good to know this is a blocking feature for some use cases.

<comment><USER9> Just signed up and looked at the api docs. Even the "signed" option for privacy says coming soon. I guess, at the moment, there is no way to secure a video on mux?

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> Seems like an amazing service and very much needed by everybody that is not YouTube, because indeed video is too hard.

Also very cheap.

Video Stored: $0.007 per minute per month Video Streamed: $0.0013 per minute streamed

<comment><AUTHOR> Thanks flatjaf! Try it out and let us know if you have any feedback.

</comment></comment><comment><USER12> I think the streaming business is really interesting, I assume the reason Twitch (Amazon) is so successful is that they own their own infrastructure. Youtube is a competitor for the same reason.

Using Akamai/other CDNs is just crazy expensive, wonder what smashcast (formerly azubu and hitbox), Mixer (microsoft), AfreecaTV, and I assume there are some large chinese streaming sites use

Is this being used right now by Vimeo and Livestream?

I kind of get the use case, I wonder what the cost is like with traditional CDNs.

<comment><USER13> >Is this being used right now by Vimeo and Livestream?

Vimeo (and Livestream) have their own distribution and do not use Mux.

<comment><USER10> (Mux founder) That's correct. Vimeo and Livestream are customers of Mux's video performance monitoring, the product we built first and what's informing decisions in our video API.

</comment></comment></comment><comment><USER14> This looks like a great alternative to Wistia for online course sites/platforms like Teachable. I believe many developer built and focused subscription video sites like gorails.com for example use Wistia because it makes video easier, just like mux, even if they have no use for the marketing focused gui tools.

Wistia is much better than other OVPs for startup projects but they charge 100/month if you want more than 3 videos which meant paying for while my project was still in development. Amazon like pricing would have been a major relief.

The pricing is far better as well. For HD video at 3GB per hour thats .026/GB compared to .085/GB for low volume on cloudfront, although hard to compare without knowing how much streaming will be full HD for a given video service.

</comment><comment><USER15> Great to see you guys doing so well. As a BCOV'er, we really miss you guys!

</comment><comment><USER16> Can it mix livefeeds ? The first page really does not say much and I wonder if it's because all it does is streaming a feed, like many other players do in this space.

<comment><USER17> > The first page really does not say much

Indeed. It tries very hard to get me to sign up (four "Create Account" buttons; three red, one blue) but I still have no idea what I'd be signing up for.

It's a CDN for video files? Someone help me out here.

<comment><USER5> (Mux founder) To be completely honest, I'd be pretty bummed if all we'd built was a CDN (don't mean that as a diss on CDNs, we just think other folks have solved that problem better than us).

Right now, a developer has to know quite a bit about video/delivery to get your own stack up and running. At the highest level, we're a clean API around ingesting and serving videos to your customer. What's actually happening is we use our analytics data to power smarter decision making around what formats and renditions we deliver to your viewers.

We do just-in-time encoding, which means we can quickly change what we deliver to those same viewers as their needs change over time without needing to do massive library re-encodes. On the edge of all of that, we also make real- time, data-driven decisions around which CDNs we use around the world.

But if you don't really care about what's going on under the hood, what you'll get in practice is adaptive bitrate playback within seconds of creating a new asset in our system, and cool features around grabbing thumbnails and stuff.

Hope that helps!

</comment></comment></comment><comment><USER18> Great looking service, excited to check it out further. The company I work for builds lots of self service video portals for clients and we use Azure Media Services for ingestion, transcoding, storage and streaming.

Care to elaborate on how this related to Azure Media Services?

Thanks!

<comment><USER5> Thank you! Sounds like a great fit, ping me if you have any questions down the road (my email is just matt at our domain).

Azure Media Services (and other, similar products such as Amazon's media offerings, etc) is a suite of products that ultimately take you from ingesting a video to playing it back, but you're going to do a lot of gluing together parts to get there. While gluing those parts together you're making a lot of decisions around formats, encoding, etc, and let's be honest...those probably won't get revisited soon enough. Our system makes those decisions on the fly based on actual performance data for your viewers.

When you ingest an asset into Mux, we give you a playback ID that's playable in seconds (which, at least imo, is very cool on its own), and you can just use that in your players and go back to making amazing video portals. You can also use that same ID to do things like request smart thumbnails immediately (had to plug that because I spent _way_ too much time on that part of the marketing website).

We do some really cool things behind the scenes around how we encode the assets just-in-time, CDN switching, etc, but ultimately the goal is that you can just wire our API into your CMS and think less about encoding minutia.

Hope that helps!

<comment><USER18> Yes very much thanks for the breakdown and look forward to an email from me in the future!! best!

</comment></comment></comment><comment><USER19> I'm totally clueless about video streaming (other than being end user), but isn't video streaming a solved problem with RTP/RTSP ages ago? With products like Icecast, Helix, Darwin Streaming Server, etc?

<comment><USER20> Matt Ward - engineer at Mux here. The simplest answer here is that most browsers and native platforms don't easily support RTP/RTSP these days. It isn't technically impossible to stream RTSP to say an iPhone, but it I'd say it is non-trivial compared to simply using AVPlayer with HLS. Our goal with this product is to make video easier for developers to understand and use so that they can focus on the other parts of their apps. We are running those rather complex streaming servers and we even provide some guides for how to get started streaming video to common platforms - [https://docs.mux.com/docs/playback](https://docs.mux.com/docs/playback)

If you have questions about video streaming on a specific platform feel free to reach out! (my full name at mux)

</comment></comment><comment><USER21> interesting, also looking at the team behind, the product must be good,

</comment><comment><USER22> This may be a dumb question, but would you describe the product as an alternative to something like Wistia? How does it compare to it?

<comment><AUTHOR> Not a dumb question. We're lower-level - the biggest value Wistia brings is helping marketers be more effective with video. Mux Video is like the plumbing; we get the video from ingest to device/player efficiently with high quality, but we don't actually offer a video player, engagement tracking, content management, metadata, CRM integrations, etc.

We think of ourselves as more of an alternative to S3 + EC2 (or Elastic Transcoder) + CloudFront + weeks of engineering time.

Use Mux Video if you're a developer who wants to build software around video streaming. Use Wistia if you're a business who wants to use video to communicate.

</comment></comment><comment><USER23> Does mux support temporary videos? e.g. a user-uploaded video that is only available to watch for a few hours?

<comment><AUTHOR> Yes, but with a sightly different pricing structure. Send us an email - I'd be interested in hearing more about your use case.

</comment></comment><comment><USER24> Nice! For companies streaming internal only videos, how do you satisfy their infosec requirements?

<comment><USER5> Thanks! One of my co-founders mentioned this in another thread, but you can set up specific playback_policies when you create an asset ([https://docs.mux.com/v1/reference#create-an- asset](https://docs.mux.com/v1/reference#create-an-asset)). For internal-only videos with strict infosec requirements, they'd probably want to use signed URLs, but we're also working on things like geo and domain-based restrictions.

</comment></comment><comment><USER25> Do you have support for 360 videos? (Streaming from a Ricoh Theta S for instance)

<comment><AUTHOR> Not at this point, but it's something our team has some experience with, so don't be surprised if we add it in the future.

If you're doing 360 streaming right now, shoot me an email (jon at mux) - I'd be curious what you use today and/or how you want to use it.

</comment></comment><comment><USER26> Having had experience in streaming to many thousands of users myself I think what you're doing here is certainly interesting - having taken a look at your service it seems likely you're using nginx and possibly RTMP module? At least that's what I've used for HLS streaming in the past (which works really well). It's a shame you only support HLS - which famously chrome doesn't support on desktop, so this doesn't really solve the whole problem with multiple codecs and maximising cross-browser support, which relies on MSE at the moment to get HLS to work.

I also noticed in the m3u8 file there are multiple versions of your file depending on quality - it'd be nice to expose this via the player so I can select the quality I want to view it at - if I am bandwidth limited 480p may just be sufficient! Also with 4k and even 8k becoming ever popular, how do you plan on dealing with this?

I'm also intrigued about video storage - will you store videos forever? How will you handle storing multiple qualities? Is there server redundancy? I noticed your video is streaming via mux.com for the m3u8 and fastly CDN for the TS fragments themselves - In theory there's nothing stopping anyone using this as a rather nice transcoding service - POST a video, and download the m3u8 file to gather the .TS fragments and join them together for a nicely transcoded video, you should also allow people to download them and host it themselves, it's not like you can stop that anyway.

Will be intriguing to see how this develops, it's a nice idea, but you're soon going to figure out _why_ video is hard - making this was for sure the "easier" part, keeping it growing/sustainable is where you are going to struggle once you are dealing with multiple PB of data storage and streaming, it's the same reason Youtube isn't profitable.

Relying on a CDN like fastly is of course possibly cost efficient, but I worry if they decide to kick you off what your plan is, there's absolutely no way in hell you could _ever_ serve the video fragments yourself so whilst you're solved (possibly) one problem, you haven't really invented anything revolutionary.

I am also interested how you're going to stop people hotlinking the videos themselves - this is something I tried to prevent _really_ _really_ hard - <USER2> with m3u8 it's simply not possible (well, it is kind of if you use a query string with an expiration token, but that has its own problems).

As I said, I could just link the CDN m3u8, for example, downloading:

[https://stream.mux.com/SdLAIpK2YUpIGOgVosm8Hyv6F7t01RadXYZWF...](https://stream.mux.com/SdLAIpK2YUpIGOgVosm8Hyv6F7t01RadXYZWFuAd8YEh8m2gLrF8XZ2UzSKPWvy00iWOl00OlqxPSE/rendition.m3u8)

tells me all the .ts files fastly is using - why bother with YOU and not just fastly themselves since THEY are the ones hosting your video.

<comment><USER27> > _Will be intriguing to see how this develops, it 's a nice idea, but you're > soon going to figure out _why_ video is hard_

You might want to take a look at the Team page before saying silly things like this.

<comment><USER26> Thanks for adding nothing to the discussion - the team page tells me nothing about the challenges they're going to face, and how they're going to overcome them. Even Google, struggles, since they've been running Youtube at a loss since 2005. Streaming video is ridiculously expensive, and they are relying on CDNs, who can give them the boot at _any_ time, and I doubt they have fallbacks beyond those that have partnered with them. It'd be no different to akamai hosting the video (which I believe Apple uses IIRC, or some other big names).

<comment><AUTHOR> I'm curious about your comments on CDNs. Have you had a CDN boot you first- hand (or seen it happen second-hand)? If so, what were the circumstances? Video delivery is a huge part of every CDN's business, and almost everyone doing video streaming relies on third-party CDNs. (Except for the biggest operations like Twitch and Google that have their own peering relationships and caching infrastructure.)

<comment><USER26> The only one was Cloudflare on another project (file hosting) - and being fair, we did do 20tb of transfer in a single month on the free tier, which they weren't so pleased about :P. that really crippled the website and we struggled to run the service after that point (oddly enough, going from their CDN and network to our own which could barely squeeze past 1gbps wasn't ideal).

It's more the longer term potential - businesses change, or see valuable partners and may decide to react accordingly, I guess having many CDNs is the best approach and swapping them out on the fly.

I do have other concerns, mainly about the pricing and how you plan to deal with things like hotlinking, taking a .m3u8 is super easy to abuse. You charge $780 for 416 days worth of streaming or in easier terms $1/12 hours of streaming so I wonder whether you will let people impose limits or what happens once those limits are reached.

I could easily take someone elses m3u8 and let them run up the charges on your service whilst I get the wonderful video hosting for myself.

I especially liked this part:

"Please do not upload video, download the playable assets, and then delete video."

You might as well offer that as a service since there's not really much you can do to prevent people using you as a transcode service directly (well, you can, but do you want to?)

<comment><USER10> Good questions. To address hotlinking we'll be encouraging people to use signed URLs. They're a little more complicated to work with but can protect from situations like you're describing. We also have a mechanism called Playback IDs, which allows you to kill a URL to a video (and create a new one) without deleting the video itself. Account limits are less ideal, but may need to be a reality. Mux is a utility service similar to AWS and Google Cloud, where you'll also have this issue, and we'll work with users to find the best approach.

Re: transcoding, this is something we've wrestled with a bit. If you only use us for transcoding then you lose half the value of the system we've built: the dynamic renditions based on real usage, real time CDN optimization, automatic new codec and device support at no extra cost. So we've tried to create the simplest pricing for people who want all of that, not just transcoding. At the same time, we do want to support real video publishing use cases that start to look more like just transcoding (videos with a short life span and few views). Hopefully we can find more real cases like that to help us think through that more.

</comment></comment></comment></comment></comment></comment><comment><USER28> What video codecs do you support as inputs?

<comment><USER5> The goal is to support just about anything you can throw at us, but the reality is that's going to take months of debugging whatever insanity the internet can throw at us. Derek from the Vimeo/VideoLAN team gave a really great talk on just how awful the world of user uploads can be at Demuxed 2017: [https://www.youtube.com/watch?v=cRSO3RtUOOk&list=PLkyaYNWEKc...](https://www.youtube.com/watch?v=cRSO3RtUOOk&list=PLkyaYNWEKcOfntbMd6KtHhF7qpL9hj6of&index=6)

That being said, we should handle most modern codecs you can throw our way. PDFs are going to fail (you'd be shocked at how often I've seen this). If you can, use h264 video and AAC audio as that's what our system is optimized for and will you give you the best performance.

<comment><USER29> Which profiles / h264 settings? I know that YouTube prefers "streamable" files, which I've never really understood.

<comment><USER30> I'm Nick, a video engineer at Mux!

We can support almost any valid h264 file :D

In an ideal world, you'd send 1080p high profile h264 encoded with a bitrate above 8Mbps (or CRF 18 for x264 nerds), with a max keyframe interval of around 5 seconds. Closed GOPs also makes processing more efficient.

</comment></comment></comment></comment><comment><USER31> FFmpegaaS?

<comment><USER10> That was Zencoder (YC W10). We did that already. :)</comment></comment>


HCrystalBall – a unified interface to time-series forecasting in Python - <AUTHOR> https://medium.com/@michalchromcak/hcrystalball-a-unified-interface-to-time-series-forecasting-6fb57384ad57 <comment><AUTHOR> GitHub repository [https://github.com/heidelbergcement/hcrystalball](https://github.com/heidelbergcement/hcrystalball)

Docs [https://hcrystalball.readthedocs.io/](https://hcrystalball.readthedocs.io/)

Examples with the prebuilt environment on binder [https://mybinder.org/v2/gh/heidelbergcement/hcrystalball/mas...](https://mybinder.org/v2/gh/heidelbergcement/hcrystalball/master?filepath=docs/examples/)

</comment><comment><AUTHOR> HeidelbergCement's first open-source package is now ready for the wider usage. Read the full blog post to learn more!</comment>


Cyral launches data breach detection - <AUTHOR> https://cyral.com/news/cloud-based-data-layer-security-startup-cyral-announces-general-availability-of-enterprise-platform/ <comment><AUTHOR> I'm excited to announce that we're launching Cyral today, with a security sidecar that detects data breaches in progress.</comment>


This App helps you report an accident to Family & Friends of an accident victim - <AUTHOR> https://accidentinformer.com/ <comment><USER2> An accident is an unforeseen, unplanned and uncontrolled event. In such cases, the Emergency response should be accessible and fast.

Typical response time in Metros when the police station is nearby is 30Min. To add, the family get to the person way too late.

Accident Informer App helps you report an accident to Family and Friends of an accident victim. Download the app and share with people you care.

Download the app here:

Google Playstore_: [https://goo.gl/8o1dnY](https://goo.gl/8o1dnY)

Apple App Store: [https://goo.gl/cqdBwb](https://goo.gl/cqdBwb)

website: [https://accidentinformer.com/](https://accidentinformer.com/)

#beresponsible #savelives</comment>


The End of the Financial World as We Know It - <AUTHOR> http://www.nytimes.com/2009/01/04/opinion/04lewiseinhorn.html?_r=1 <comment><USER2> _These oligopolies [the rating agencies], which are actually sanctioned by the S.E.C., didn't merely do their jobs badly. They didn't simply miss a few calls here and there. In pursuit of their own short-term earnings, they did exactly the opposite of what they were meant to do: rather than expose financial risk they systematically disguised it._

We've heard a lot in recent months about how the current crisis proves that "unfettered capitalism" doesn't work and that libertarian political philosophy is (intellectually) bankrupt. Libertarians, we are told, are running scared.

The quote above is one small example of why this claim is rubbish. The banking and financial sectors have been heavily regulated for at least 70 years; this has created the kind of cartel virtually impossible to form in a free market. And what happened to the SEC is so common it has a name: _regulatory capture_ , the capture of the regulatory agency by the special interests of the industry being regulated.

There's plenty of blame to go around: the global financial crisis has many causes. _Laissez-faire_ capitalism isn't one of them.

<comment><USER3> Really, really, really _real_ free markets never seem to exist, though. So the whole "if only" game doesn't seem like a worthwhile one to me - if there's a failure, it's never, ever, about the market actually failing, it's that it just wasn't free _enough_?

Sorry, back to the drawing board then, and design a system that works in the real world. It's probably not too far off from what we have now, and should be reasonably free, no doubt about that, but it should be designed to muddle along with the muddly creatures that comprise its actors, rather than some sort of idealized system that will never come to fruition. Those apes tend to have both collectivist and individualistic tendencies, and ignoring one in favor of the other is a bad idea.

<comment><USER4> I think the discussion is getting lost here. We recognize that the current system is a partially-free, but heavily-regulated structure. Now the question is, what were the weaknesses in the system? What were the points of failure?

Leftists are saying that the problem was that the system was too free. Libertarians are saying that the main problem lies with the corruption and inefficiency of regulation. Neither point is trivial, neither point can be dismissed as ideological rhetoric.

There are those libertarians who will always blame the government for every problem until we live under Rothbardian anarchy. You may think that position is idiotic, and you are welcome to do so. However, the leftists who blame every problem on "the unfettered free market" sound equally idiotic to my ears in a world with regulatory agencies with multi-billion dollar budgets and hundreds of thousands of pages of rules controlling trillions of dollars of capital every day.

So, don't dismiss all claims of misregulation even if some of the critics are unhinged. After all, if something is wrong there, we would expect it to have large effects. My old firm spent at least a seven-figure number each year to comply with regulatory rules, and it wasn't very large.

<comment><USER3> Oh, I agree completely that some on the left make plenty of bad statements of their own and are altogether too ready to blame markets for various things. I'm sort of a centrist and believe that many things need to be worked out by trial and error. This means that people basically need to be free to _try_ , but that, in some cases, _errors_ also need to be dealt with, and sometimes, the government is the most efficient way to deal with them. Often, local governments are preferable to farther away/bigger ones, but some issues are also dealt with best on a larger scale.

<comment><USER4> I think a major problem is that governments are large, monopolistic, and tend to copy each other. How do you know what really works if you don't have different models to look at?

We see some competition between governments in areas like the European Union that have free trade and free movement. This has resulted in tax rates being lowered and business being liberalized across the EU. However, the European Union cooperates on some things, such as financial regulation, so we won't see any competition between different models there.

I think the US is much too large in terms of people, GDP, and land area to be efficiently governed by one entity. The law is crusted over with special interest regulation and really needs a rethinking. However, I doubt that is likely to change any time soon.

And I agree with you that trial and error is a good way of proceeding. The problem is that the visibility and the severity of an error are not always correlated. It is easy to fix visible problems but not get closer to an efficient model.

<comment><USER3> > tend to copy each other

Sure. We've found a model that works pretty well, and a whole lot better than various recent experiments, so it's a fairly safe bet. Who wouldn't want to copy that?

> US is much too large in terms

Difficult to say. It's not run worse than plenty of smaller countries, and has one huge advantage: it's an ironclad free trade area.

</comment></comment></comment></comment></comment></comment><comment><USER5> Article summarized by this quote from it:

"... nothing has been done to change that, or any of the other bad incentives that led us here in the first place."

The headline is a little misleading.

</comment><comment><USER6> <i>Richard Fuld, the former chief executive of Lehman Brothers, E. Stanley O'Neal, the former chief executive of Merrill Lynch, and Charles O. Prince III, Citigroup's chief executive, may have paid themselves humongous sums of money at the end of each year, as a result of the bond market bonanza. But if any one of them had set himself up as a whistleblower had stood up and said "this business is irresponsible and we are not going to participate in it," he would probably have been fired. Not immediately, perhaps. But a few quarters of earnings that lagged behind those of every other Wall Street firm would invite outrage from subordinates, who would flee for other, less responsible firms, and from shareholders, who would call for his resignation. Eventually he'd be replaced by someone willing to make money from the credit bubble.</i>

And yet that's exactly what upper management at Goldman and JPMorgan did. Goldman never bought massive quantities of ABS CDOs for portfolio. Chase never originated a single option ARM. Not all of Wall Street was so cravenly focused on short-term profits.

<comment><USER7> Is this a joke?

Uhh ... you do realize Blythe basically INVENTED the CDS right?

Maybe I am misunderstanding what you are trying to say, but ALL of the big investment firms were in on this. In fact, the system would not have failed if any one of the investment banks had not played their role.

If you are a quant we can get into arguing the fatal flaw of uncleared counterparties. But from my perspective when everyone is a counterparty, and EVERYONE was, there is nowhere to run when things go bad.

Could you clarify what you mean for me please?

Incidentally, the quote is correct, the heads of everyone from Bear to Goldman would have been summarily dismissed if they had failed to play their roles. All because of the 'innovation' of one, as it turns out, not so bright quant at JPMorgan. Astounding isn't it?

<comment><USER6> I wasn't talking about CDS, and I am aware that Blythe Masters is credited with inventing the CDS while at JPM. I think it's still too early to tell if CDS are good or bad on the whole. Ultimately, they're just insurance contracts, which have been around for hundreds of years. But that's another discussion.

I'm talking specifically about subprime. JPM and Goldman explicitly decided at the very highest levels of management to remain uninvolved in certain areas of the subprime mortgage market. Chase did originate subprime loans, but it did not securitize them and for a long time, did not make a market in subprime bonds. Chase also stayed away from certain areas of alt-A lending, in particular option ARMs, which have become one of the most toxic forms of mortgage debt. Also, JPM didn't purchase billions upon billions of subprime bonds and subprime-backed CDOs, like Merrill, UBS, Citi, and other banks did. That's why JPM has taken "only" $6bn in writedowns over the past 18 months, vs. over $30bn each for Citi, Merrill, and UBS. And yet Jamie Dimon remained CEO of JPM through the mortgage lending boom, despite staying away from its most lucrative areas.

You might find the following article interesting: [http://money.cnn.com/2008/08/29/news/companies/tully_jpmorga...](http://money.cnn.com/2008/08/29/news/companies/tully_jpmorgan.fortune/index.htm)

</comment></comment></comment><comment><USER8> What an incredible year and a great article.

It's hard to imagine a scenario where 12 months ago I would have imagined an editorial in the New York Times with the title, "The End of the Financial World As We Know It."

<comment><USER9> It's called "yellow journalism." This crap sells right now, and the major news outlets are all over it. Remember all of the "dirty bomb" fearmongering a few years ago?

<comment><USER10> you need this kind of articles to change the system, make sure senators or entrepreneurs see the problem and act upon it.

<comment><USER9> Hyperbole and invective are the domains of politicians who use them to great effect. No need for it in a balanced press (wishful thinking).

</comment></comment></comment></comment><comment><USER11> It seems clear to me the SEC has failed as an institution. I tend to agree with the author that the revolving door policy whereby ex-regulators become bankers for the banks they were previously regulating is no longer acceptable.

</comment><comment><USER12> No matter what the government (or anyone) does, the same process will repeat itself over and over again.

<http://en.wikipedia.org/wiki/Tulip_mania>

<comment><USER13> Actually, I think a slightly different process will appear each time. Different enough that people will not recognize it for what it is except in hindsight. Since regulation is most effective as a reaction to a problem, perhaps we should accept that there is very little we can do to stop these sorts of things and concentrate on preparing ourselves to weather the storms when they appear.

<comment><USER12> The best indicator? _"This time it's different."_

-1980's Leverage Buyout Craze (RJR Nabisco) coupled with junk bonds

-Dotcom Bust

-Day trading

-Too much leverage (Long Term Capital Management and repeated today)

-Mortgage back securities

-People making 50k per year buying 500k homes

</comment></comment></comment><comment><AUTHOR> continued at: How to Repair a Broken Financial World [http://www.nytimes.com/2009/01/04/opinion/04lewiseinhornb.ht...](http://www.nytimes.com/2009/01/04/opinion/04lewiseinhornb.html)

</comment><comment><USER14> So, apparently (according to the article), the bond raters are paid by the bond issuers. How in the world is that not a conflict of interest? There must be some "pro" for the financial world being this way?

<comment><USER15> I completely agree. How can anyone look at this fact and not see it as totally undermining the intent of bond raters?

</comment></comment><comment><USER16> Phase 4, "Winter": [http://en.wikipedia.org/wiki/Decline_of_the_West#Phases_of_r...](http://en.wikipedia.org/wiki/Decline_of_the_West#Phases_of_rise_and_decline)

</comment><comment><USER17> article summary: the world is full of sheeple.

</comment><comment><USER18> It's the end of the world as we know it.

God bless REM

</comment><comment><USER19> Flagged, not hacker news.

<comment><USER20> I thought this paragraph in the guidelines was clear enough, but I've added another sentence to make it more explicit:

"Please don't submit comments complaining that a submission is inappropriate for the site. If you think something is spam or egregiously offtopic, you can flag it by going to its page and clicking on the "flag" link. (Not all users will see this; there is a karma threshold.) If you flag something, please don't also comment that you did."

<http://ycombinator.com/newsguidelines.html>

<comment><USER19> Funny. Most of the time when I make this comment I get 5+ upvotes just for it, because I generally make it quite accurately on articles that are judged not desirable by others too (and the article is usually killed soon after).

I'll refrain from making it in the future if it hurts your HN experience, so this will be my last comment on the topic.

I still maintain that this article is irrelevant to this site, btw. This is reddit/r/business material - i.e. grandiose, inflammatory politico-financial opinion pieces that will result in no significant change of anything whatsoever, but get lots of upvotes and a plethora of comments that look like they make a lot of sense (until you read the response to each comment and find that someone else is making just as much sense with the exactly opposite view).

Imho, this is a fantastic waste of this site - and I shall continue to flag this kind material whenever I find it.

Edit: I'll just add that on occasion even this kind of post that you appear to dislike does generate a useful outcome, e.g.: <http://news.ycombinator.com/item?id=417744></comment></comment></comment>


VDSO, 32-bit time, and seccomp - <AUTHOR> https://lwn.net/SubscriberLink/795128/6b8ce4fe54123148/ <comment><USER2> This class of problem for seccomp has been known for ages. This makes it quasi-unusable IMO (or reserved for full-contained binaries, maybe in some JIT scenario but probably not a lot else), and even quasi in self-contradiction with the kernel non-regression rules (because by design seccomp can not achieve that in the general case -- at least when you consider the bigger picture that is a classic userspace design and ecosystem, like in a mainstream standard distro). To detail more: under this "standard" model the applications do not often do syscalls themselves, but use intermediate libraries (ex: glibc) that provide (quasi)-Posix and reserve some of the syscalls or even provide syscall like functions that actually use others -- or just start to use new syscalls in new versions to provide higher levels of abstraction. But they do not provide another abstraction to obtain a seccomp-like services suitable for this model. So seccomp is basically unusable in this context, which is one of the most important.

Now the kernel itself made the mistake, proving that the whole idea was not practical; it breaks too easily, and it breaks even when used in most restricted ways or when the whole (non-kernel provided) userspace has been designed for it (which was not a practical condition at all, to begin with).

IMO seccomp should be phased out entirely and eventually replaced by something else. Trying to "fix" it will yield nowhere: it is broken by design, since forever.

<comment><USER3> The other issues: Parent cannot seccomp itself without being highly coupled to the child (filters persist across exec), Different syscall numbers, socketcall, etc, etc make it almost mandatory to use something to build the filter.

But there isn't really anything that Linux can replace it with, pledge works because Openbsd controls both libc and the kernel.

<comment><USER4> > Parent cannot seccomp itself without being highly coupled to the child

You can workaround this sometimes though. Unless you actually care about fork with current memory copy, (i.e. you care about spawning new processes only) you can fork a "spawner" process early which is only a thin proxy for pipe->exec commands. You apply seccomp after spawner is ready and you're all good.

</comment></comment></comment><comment><USER5> seccomp is such a sad story. In theory seccomp can do a lot more than OpenBSD's pledge. In practice, the OpenBSD devs added pledge support with comparatively little effort to 100s of programs, while seccomp is a constant headache for the few programs which use it.

<comment><USER6> Afaik it's not even possible to implement a pledge-like wrapper around seccomp because seccomp is inherited while pledge leaves the responsibility of securing itself to each process.

<comment> X-Istence > Allows a process to call execve(2). Coupled with the proc promise, this > allows a process to fork and execute another program. If execpromises has > been previously set the new program begins with those promises, unless > setuid/setgid bits are set in which case execution is blocked with EACCES. > Otherwise the new program starts running without pledge active, and > hopefully makes a new pledge soon.

[https://man.openbsd.org/pledge](https://man.openbsd.org/pledge)

Pledge can be inherited by child processes too.

</comment></comment></comment><comment><USER7> You could make a quiz based on "what syscalls does this familiar libc API end up calling". selinux and seccomp are dead ends.

<comment><USER8> Yet somehow we run SELinux successfully on every Android device. I've never understood the antipathy towards SELinux (which by the way, doesn't operate at syscall granularity).

<comment><USER9> One reason to dislike SELinux is the arbitrary assumptions baked in by default that barely anyone knows how to or cares to change, which makes carefully designed software look broken when the framework itself is broken. One example I've faced is UNIX pipes, which for all purposes were obsoleted by UNIX domain sockets in the mid 80s as part of the original intent of the BSD socket API, but to SELinux they are profoundly different things. You can pass pipes across a user->root boundary but not a socket.

To the end user, they only see your code broken by SELinux, and assume you haven't done your job. On the other hand, SELinux is codifying rules about UNIX that never existed and amount to emotional heuristics about the risk of Internet domain sockets being inherited around the system by the wrong process. The effect isn't to prevent Internet domain sockets being inherited by privileged processes, but breaking all sockets. That's garbage design, hidden behind marketing suggesting because the NSA contributed some code that the problem couldn't possibly be SELinux

<comment><USER8> But unix domain sockets are not pipes, not least in the important way that you can reliably determine the identity of the other end. That can in some cases be an infoleak, and therefore needs independent access control.

<comment><USER9> If SCM_CRED passing were a real problem in _any_ scenario, SELinux should target that instead, not an entire subsystem on which half the system is built

<comment><USER8> The point is you need the ability to express both "these are the same thing" and "these are different". You can do both with SELinux. What's your alternative?

</comment></comment></comment></comment></comment></comment><comment><USER10> MySQL also still has Y-2038 issues, even on 64 bit machines.</comment>


Ask HN Apple no longer supporting widgets? - <AUTHOR>

Tried to download multiple widgets featured on the apple site but all download links were broken: http://www.apple.com/downloads/dashboard/status/istatpro.html <comment><USER2> If you would like an operating system that doesn't arbitrarily and irrevocably yank support for applications, I suggest Debian or another GNU/Linux OS.</comment>


Show HN: TerminusDB 2.0 - time traveling FOSS database that stores data like Git - <AUTHOR> https://github.com/terminusdb/terminusdb-server <comment><AUTHOR> TerminusDB 2.0 will allow for the whole suite of revision control features: branch, merge, squash, rollback, blame, and time-travel. Our goal is to enable the types of CI/CD used extensively in software engineering to be used with data. We believe DevOps and other data-intensive teams are poorly served with current tools.

This is an Alpha release, so there will be glitches and some features that are not switched on yet.

We are also releasing a new documentation site that is a big step forward in documenting TerminusDB features: [https://terminusdb.com/docs/](https://terminusdb.com/docs/)

We had a Hacker News comment that gave the background to the project: [https://news.ycombinator.com/item?id=22867767](https://news.ycombinator.com/item?id=22867767)

Remain very much open to contributions and are very appreciative of the support of the Hacker News Community.</comment>


Fluid Tests Hint at Concrete Quantum Reality - <AUTHOR> http://www.simonsfoundation.org/quanta/20140624-fluid-tests-hint-at-concrete-quantum-reality/ <comment><USER4> I've liked:

Clearing up Mysteries - The Original Goal [http://bayes.wustl.edu/etj/articles/cmystery.pdf](http://bayes.wustl.edu/etj/articles/cmystery.pdf)

" ...we must keep in mind that Einstein's thinking is always on the ontological level; the purpose of the EPR argument was to show that the QM state vector cannot be a representation of the "real physical situation" of a system. Bohr had never claimed that it was, although his strange way of expressing himself often led others to think that he was claiming this.

From his reply to EPR, we find that Bohr's position was like this:

"You may decide, of your own free will, which experiment to do. If you do experiment E1 you will get result R1. If you do E2 you will get R2. Since it is fundamentally impossible to do both on the same system, and the present theory correctly predicts the results of either, how can you say that the theory is incomplete? What more can one ask of a theory?"

While it is easy to understand and agree with this on the epistemological level, the answer that I and many others would give is that we expect a physical theory to do more than merely predict experimental results in the manner of an empirical equation; we want to come down to Einstein's ontological level and understand what is happening when an atom emits light, when a spin enters a Stern-Gerlach magnet, etc. The Copenhagen theory, having no answer to any question of the form: "What is really happening when - - - ?", forbids us to ask such questions and tries to persuade us that it is philosophically naive to want to know what is happening. But I do want to know, and I do not think this is naive; and so for me QM is not a physical theory at all, only an empty mathematical shell in which a future theory may, perhaps, be built."

<comment><USER5> You are misrepresenting physicists. There have been many many different attempts to talk about quantum theory on an ontological level. Case in point, the widely debated interpretations: copenhagen, many worlds, bohemian etc.

However, one reason that physicists don't spend too much time thinking about what quantum theory really means (besides laziness and bad science) is because we know that quantum theory is not the ultimate theory of reality. Why try to force ontology on a theory that can not even answer all empirical questions without ad hoc additions (standard model of particle physics) or force mating with another theory (general relativity)? We are all waiting for a better theory of physics. Of course rewriting the quantum theory in different forms, like this attempt, might help in getting there.

<comment><USER6> I agree and disagree with what you're saying. I agree it is laziness and bad science that physicists stopped caring about the philosophical underpinnings of quantum mechanics but I disagree that it was because quantum mechanics not being the ultimate theory.

The reason most physicists don't think about the philosophical underpinnings has more to do with the way quantum mechanics is taught. Many of the founders were interested in the philosophical questions but after WWII you can see a sharp decrease in that aspect and more of the hardcore calculation and the philosophy kind of disappeared. Here [1] is a great talk that discusses this very point. Unfortunate for quantum mechanics because understanding the philosophical underpinnings is critical to understanding what quantum mechanics says and doesn't say. But, thanks to things like quantum computing and better experiments, interpretations is becoming more mainstream since there are experiments being planned (at least when I was heavily involved in the area in 2010) that would be able to test some of the claims made by different interpretations.

>We are all waiting for a better theory of physics. Of course rewriting the quantum theory in different forms, like this attempt, might help in getting there.

To me this is a contradiction. If we are all waiting for a better theory, then why are we re-writing the theory? A critical part of all the interpretations I've encountered [2] is making different ontological claims about quantum mechanics. Take Bohmian, which posits a guiding wave. Sure we can make some of the mathematics nicer, but we can't go very far without explaining what this 'guiding wave' is, which itself is rather ad hoc. Plus, how do we rectify it with Bell's theorem on hidden variables? To me, we can't really begin re- writing quantum mechanics without inevitably running into ontological statements.

But I still agree with your premise. I believe that by pushing the limits of quantum mechanics both scientifically and philosophically we will get a better grasp of what things to look for / where to look, for a deeper theory. Case in point, wavefunction collapse. Some might consider it a huge problem of quantum mechanics, but if you take a different ontological view, wavefunction collapse isn't that interesting or important. So maybe wavefunction collapse isn't as important and trying to develop a theory that doesn't have it is the wrong idea.

[1] [http://pirsa.org/08090000/](http://pirsa.org/08090000/) [2] [http://pirsa.org/C10002](http://pirsa.org/C10002)

<comment><USER7> My understanding is that the pilot wave theory is a non-local hidden variable theory, which doesn't contradict Bell's theorem at all. Of course, this presents ftl information transfer, presumed to be impossible, which is part of the reason such theories have been unpopular for so long.

<comment><USER8> Non-locality does not imply the possibility of FTL information transfer - <USER3> theory can be non-local but make exploiting the non-locality impossible.

<comment><USER9> This is a vitally important point. Many people still naively believe entanglement allows communication. Simultaneous randomness doesn't permit a signal.

</comment></comment></comment></comment></comment></comment><comment><USER10> FAQ about Bohmian Mechanics (it's very accessible): [http://www.mathematik.uni- muenchen.de/~bohmmech/BohmHome/fil...](http://www.mathematik.uni- muenchen.de/~bohmmech/BohmHome/files/Frequently_Asked_Questions_about_Bohmian_Mechanics.pdf)

<comment><USER7> Its certainly accessible, but I'm not sure how much I trust the authors to understand what they're talking about after point 3.

> But very often the conclusion that is drawn from this observation is that > our _knowledge_ about the path followed by the electron causes the > interference pattern to disappear.

Although we couch the idea in terms of "knowledge", the usual meaning of that word is not what anyone thinks is actually the trigger. What exactly is meant by "knowledge" is not well-agreed upon, but no one thinks it is something particular to humans (or sentient beings, I suppose) that causes the wavefunction to collapse. It has more to do with whether the information exists - if the particle interacts with something in such a way that its position suddenly becomes deducible, then it collapses. Like I said, the specifics of this are a known hole in the Copenhagen theory - but this "knowledge" tangent is a clear strawman.

<comment><USER11> That FAQ is generated by the students of a very trustworthy group. Students typically setup strawman when explaining things. Look more to the source. Also, they are trying to argue against a theory which no one wants to fully specify. So they have to come up with something definite. And as soon as they do, the other side says "Strawman!" And then proceeds, as you have just done, to refuse to give a definite theory.

As a student of that group as well, I can't say that I am all that pleased by that kind of argument.

The bottom line is that collapse is untenable. There is no good place to put it and yet it is needed by the standard theory.

<comment><USER7> There is a range of possibilities of what might cause a wavefunction collapse, and the constructed strawman is nowhere near that. It would be like if we claimed the speed of light was 3.2e8 +/\- 5e7 m/s and someone decided our whole theory was bunk because the speed of light is obviously faster than 100 m/s. Well yeah, it is, but there's no contradiction here.

"if the particle interacts with something in such a way that its position suddenly becomes deducible, then it collapses" would be a theory in the range of possibilities. It would be much more convincing to argue against something like that.

</comment></comment></comment></comment><comment><USER12> Yes! But the best theory is not Copenhagen or Pilot Wave.

Eliezer Yudkowski has a brilliant treatise on the Many Worlds interpretation here: [http://lesswrong.com/lw/r5/the_quantum_physics_sequence/](http://lesswrong.com/lw/r5/the_quantum_physics_sequence/) that really should be required reading for anyone that wants to talk intelligently on the subject.

Edit: seriously, don't even bother reading the article. It (like most science journalism) is garbage. Take the time to work through Eliezer's sequence.

<comment><USER13> So please explain in a few sentences what is that that Yudkowski writes, did he make any new contribution to the standard model or made something else or do you agree that the standard model is the most researched and most usable model up to now, and he just made a lot of posts where he just writes a lot of text?

I see a lot of links in the article you gave, but I don't understand what we're supposed to discover in Yudkowski's writings after trying to follow most of them. There's a lot of free text, not much physics. The standard model is a lot of smart formulas supported by the decades of expensive elaborate measurements (and vice versa), however his texts look more like writings of some philosophy student who knows a little of the math than like a physicist's material. I'd also really welcome opinions of professional physicists.

Edit: Wikipedia entry about him seems to fit my impression: [http://en.wikipedia.org/wiki/Eliezer_Yudkowsky](http://en.wikipedia.org/wiki/Eliezer_Yudkowsky) "Yudkowsky (...) is an American blogger, writer, and advocate for Friendly artificial intelligence (...) Largely self-educated."

<comment><USER14> The Standard Model does not concern itself with the various interpretations of quantum mechanics. It can, as of now, neither verify or falsify any of the interpretations which, consequently, are not scientific theories or even hypotheses, but firmly on the side of philosophy of physics. The interpretations are attempts to answer _why_ Nature works as it does, _within_ the framework of the Standard Model which only appears to answer the question _how_.

Yudkowsky certainly hasn't _invented_ the many worlds interpretation, which was originally formulated by the physicist Hugh Everett in 1957. Even though originally scorned, in the more recent times it has gained popularity among physicists. The series of blog posts by Yudkowsky are (in my opinion, at least) a persuasive argument in its favor against the competing interpretations, and are very much recommended reading for anyone who would like to better understand the issue.

<comment><USER13> > the interpretations which, consequently, are not scientific theories or even > hypotheses, but firmly on the side of philosophy of physics.

Thanks, that's exactly what I wanted to know.

I'm completely satisfied with the "shut up and calculate" approach. For me, until somebody shows that he/she can calculate (that is, predict) more than what physicists achieve, they are the ones the closest to "the truth" and not the "interpreter."

<comment><USER12> Bring it back to the scientific method. The way to differentiate between competing hypotheses is by devising experiments that falsify some of them, and then running the experiments and either falsifying or failing to falsify them.

The issue here is subtle, and it's that the most popular interpretation, Copenhagen, isn't a complete theory because it doesn't tell you algorithmically when collapse occurs. For any possible algorithmic way to handle collapse, there's a corresponding experiment that could (at least in theory) differentiate between Copenhagen and Many Worlds. But the Copenhagen is inordinately slippery in that collapse is defined to occur ex post facto in whatever way is needed to make the experimental results match the theoretical results.

It's perhaps not so surprising that this shortcoming was overlooked in the beginning because Copenhagen was hypothesized before we really had a clear handle on the study of algorithms. But the fact that Copenhagen is still as popular as it is means that Yudkowski needs to spend a lot of time on philosophy of science, because that's what's holding back most people from seeing the problems with Copenhagen, and why at first glance it looks like philosophy.

<comment><USER13> I think I can agree with your first paragraph, even if I'd phrase it somewhat differently, but I don't see the connection between it and the next two.

<comment><USER12> I certainly meant those three paragraphs to be connected; perhaps I'm just poor at explaining. As I mentioned, the shortcomings of the Copenhagen Interpretation are subtle, and I originally linked to Yudkowski's treatment because I believe he does much better job of explaining it than I ever could.

<comment><USER13> Allow me to not agree with your classification of the top article as "garbage." I've got from it more than from the text you linked to and I'm even less satisfied with your explanations whereas Natalie Wolchover did a rather good article. Please tell me if you have any background in physics, or what is your background?

The major thing missing from the article for me is that she didn't mention that the co-author of the most recent paper on the topic is much more known as the security researcher than as somebody who does anything related to quantum physics:

[http://arxiv.org/abs/1401.4356](http://arxiv.org/abs/1401.4356)

[http://en.wikipedia.org/wiki/Ross_J._Anderson](http://en.wikipedia.org/wiki/Ross_J._Anderson)

[http://www.cl.cam.ac.uk/~rja14/](http://www.cl.cam.ac.uk/~rja14/)

He's author of (to me) very useful book "Security Engineering":

[http://www.cl.cam.ac.uk/~rja14/book.html](http://www.cl.cam.ac.uk/~rja14/book.html)

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER15> Another anti-Copenhagen argument by Carver Mead: [http://www.cns.caltech.edu/people/faculty/mead/Nature_Of_Lig...](http://www.cns.caltech.edu/people/faculty/mead/Nature_Of_Light_What_Are_Photons.pdf)

</comment><comment><USER16> Liquid dynamics sounds like an interesting hypothesis, but I agree with the critics that entanglement presents a tough analogy. If Pilot Wave theory suggests that the particle-wave duality is actually two distinct actors - <USER3> wave and a particle riding the wave - then 1) each particle has an independently generated wave? and 2) it's difficult to see how that wave could be exactly preserved for both particles over the long distances of entanglement experiments. That's like envisioning a ripple on the far side of a large lake exactly mimicking a ripple here. Granted, the entanglement result of simultaneous collapse of probabilities is also tough to rationally understand. But Pilot Waves maintaining their effect, regardless of various asynchronous interactions encountered between the distances of entanglement experiments, seems naive. Then again, I'm not a particle physicist.

<comment><USER17> In interpretation of dual slit experiment people do the main mistake of mixing up the 2 waves - one is wave like quantization of position and another is de Broglie wave-equivalent of the particle. The mix up is caused by photon where both have the same wavelength. They are different in the case of all other particles. Hitachi experiment shows its the best - it clearly shows interference of the positional waves of different particles, not the de Brogile wave (ie. not the particles themselves). It thus also lends very credible support to statistical ensemble interpretation (which, as far as i know, Einstein favored)

People frequently bring up single-particle self-interference as explanation for Hitachi too. Well, in case of photon the self-interference pattern looks the same as positional interference because of the same wavelength. In case of electron and especially neutron the self-interference pattern would be much more dense than observed one because of the much higher de Brogile frequency than the frequency of the positional quantization.

The wave-like quantization of position is the Pivot Wave. The really important thing here is to understand that Pivot Wave isn't a real wave/object. It is just a description of possible positions of the particle at specific times. Like a trajectory of a bullet isn't a real thing, it is just description of possible positions of the bullet at specific times. It is just in QM the trajectories are wave-like quantized and probabilistically spread - that the non-smooth at small scales structure of our Universe shows its "ugly" face :)

Again, taking the quantized wave-like trajectory description of a particle for a real thing, the particle itself, is the main misinterpretation that has been going for years, especially in Copenghagen interpretation.

<comment><USER18> >>The really important thing here is to understand that Pilot Wave isn't a real wave/object. It is just a description of possible positions of the particle at specific times.

Is it possible that pilot waves are ripples in space-time? We should not make strong assertions especially when a theory is young. You make it sound like nothing more than the a regular QM wave function.

<comment><USER8> Ripples in space-time are gravitational waves. It could be a real wave but the danger is that one introduces a preferred references frame. Not that it would be the end of physics but it would surely be a big surprise.

</comment></comment></comment></comment><comment><USER19> I was under the impression that Bohmian mechanics were known to be equivalent if and only if hidden variables were strictly non-local (in the case of the macrophysical observations, the "universe" can for a reasonable higher-order approximation be the limits of the chamber being observed).

IIRC, There is also an interesting 'alternative' relativity which has non- local effects and a universal frame of reference formulated by a physicist named Frank Tangherlini (I'll be interviewing him this month). It also has weird properties like anisotropy of the vacuum speed of light!!

Might be interesting if Bohmian and Tangherlini mechanics provided a better mathematical rapprochement of quantum mechanics with relativity than Copenhagen/Lorentz/Einstein

<comment><USER20> _I was under the impression that Bohmian mechanics were known to be equivalent if and only if hidden variables were strictly non-local_

Correct. The same is true of Bell's theorem; it shows that no local hidden variable theory can reproduce the predictions of standard quantum mechanics. It's true that the "local" part often gets left out in pop science treatments of Bell's theorem; but Bell himself was quite clear about it, and about the fact that Bohm's pilot wave theory is nonlocal. The article completely fails to mention this, which IMO is a huge omission.

</comment></comment><comment><USER21> This sums up my thoughts exactly. The pilot-wave theory "...seems to me so natural and simple, to resolve the wave-particle dilemma in such a clear and ordinary way, that it is a great mystery to me that it was so generally ignored."

I don't run in experimental-physicist circles, granted, but I've definitely encountered countless cases of a clear, obvious, correct solution being brought up and summarily ignored for what proves to be a poor solution. The probabilistic theories have always made for good Science Fiction, but that should hardly matter.

<comment><USER22> This seems to be common problem. Watzlawick describes an experiment where two people (A and B) are given the task to classify tissue samples as "healthy" or "sick".

There is an initial training session where a light tells them whether their answer was correct, with one little caveat: only A gets real feedback. B's light just duplicates A's, so the answers that B receives are essentially random (I think they get different picture as well).

Since the task is not very difficult (on purpose), the As learn the task in 80% of the cases. The Bs have a much more difficult task, they are required to try and find order in a random world. They form _very_ complex theories to account for this.

However, that's not the experiment quite yet. The real experiment is that As and Bs are then put together to discuss their results. What happens then is stunning: instead of rejecting the B's theories as unnecessarily complex, the As are usually so impressed with the subtle complexity and detailed brilliance of the B's theories, that they change their mind and accept the B theories!

When asked who will improve in the next round, all the Bs and most of the As pick the Bs. And they are right, because the As will have accepted at least some of the Bs ideas and thus perform more poorly.

Reference: [http://omg.pytalhost.net/dls/ebk_wwidw.pdf](http://omg.pytalhost.net/dls/ebk_wwidw.pdf) (German)

</comment></comment><comment><USER23> By far the most fantastic and unbelievable part of this article is the calm thoughtfulness and collaborative criticism in its comments. Thank you, OP, for sharing such a rare gem.

</comment><comment><USER24> I'm impressed by the probability distribution graph in the video. I would never expect the random motion to form a wave!

<comment><USER25> that's the point, it's chaotic, not random.

</comment></comment><comment><USER26> Interesting. Maybe this will discourage chuckleheads and ne'er-do-wells from citing "quantum physics" as the justifixation for their particular variety of woo?

Probably not, but one can hope!

<comment><USER27> If you can't bullshit your way through criticism with vague-sounding phrases like "As the particles move along, they feel the wave field generated by them in the past and all other particles in the past", you're not trying hard enough. :v

This magical bracelet is powered by the wave field shaping the contours of the superfluid of space time! Carrying you through life on positive pilot waves!

<comment><USER28> I can imagine supporters of homeopathy would have a great time drawing unfounded conclusions from the bit in that article that talks about fluids exhibiting a "path memory".

<comment><USER29> Would be even worse if it was proven that molecule actually have this kind of memory.

</comment></comment></comment></comment><comment><USER30> If you try to create a classical situation that obeys similar mathematics to a quantum situation, you're bound to get "quantum-like behaviour" isn't it? Wouldn't this be some kind of an analog computer for simulating the two-slit experiment in the classical realm? If so, what would this offer to the interpretation of quantum mechanics that cannot be gleaned from the mathematics itself?

<comment><USER31> The significance is that this is a model of a special, alternate formulation of quantum mechanics called Pilot-wave Theory.

<comment><USER30> pwt is an interpretation. All interpretations of QM share the same mathematics .. or effectively the same maths. Different "interpretations" cannot produce different predictions in experiments. So using an experiment designed to emulate pwt (which afaik violates special relativity) can only say one thing possibly - that if we find it convenient to think in this way in some limited cases of quantum mechanics, we may do so. In other words, it may at best serve as a heuristic to teach children, that they can later grow out of.

<comment><USER31> > All interpretations of QM share the same mathematics .. or effectively the > same maths.

No they do not.

> Different "interpretations" cannot produce different predictions in > experiments.

Yes they do.

</comment></comment></comment></comment><comment><USER32> I was unable to figure out one key thing about this article. Are they claiming that if the droplet is observed while going through one of the slits that the interference pattern will vanish? If not can we really claim a strong analogy with QM?

<comment><USER33> Yes.

"And just as measuring the trajectories of particles seems to "collapse" their simultaneous realities, disturbing the pilot wave in the bouncing-droplet experiment destroys the interference pattern."

</comment></comment><comment><USER34> Excellent. Hope this gets support and more attention. I'm getting tired of explaining how hand-waving about probabilistic magic of subatomic behavior is successful guessing of reality without explaining it.

</comment><comment><USER35> Any ideas on how quantum computing would be explained with this model?

If quantum behavior is really classical like I think they're claiming, wouldn't that mean quantum computers wouldn't provide any benefit?

<comment><USER36> I'm not sure how quantum computing would be explained, but if it is classical then yes, quantum computers really wouldn't provide any benefit. This brings to mind headlines recently on how so-called quantum computers could not outperform optimized classical computers on algorithms specifically designed to cater to a quantum computer's strengths. Of course, then you would have to argue whether there actually are any quantum computers in existence today, or the quantum computers that are claimed to exist are actually classical computers exploiting some quantum effects.

<comment><USER37> The D-Wave devices that you are referencing are actually quantum annealing machines, not universal quantum computers. A quantum annealing machine is to a universal quantum computer what a mechanical computer is to a digital processor.

</comment></comment></comment><comment><USER38> Are there any experiments where the pilot wave theory and the Copenhagen interpretation predict different results?

<comment><USER39> It depends a bit how you define things. The Copenhagen interpretation has a collapsing wave function doing so in a way where we can't observe it collapse and the pilot wave theory has pilot waves all over the place that have no effect other than guiding the particle in question then effectively disappearing when not needed. If either the collapses or pilot waves were real things I would kind of expect them to be observable in some way, but in both interpretations they are not observable, which leads me to suspect they are not real things and will go the way of the "luminiferous aether".

<comment><USER11> Funny that you say that since the particles are pretty much all we do observe. You never "see" the wave function, but you do see where stuff is.

To be charitable to Copenhagen, we would say that we do see the results of collapse. The main problem with the collapse is that is simply not well- specified by the theory.

</comment></comment></comment><comment><USER40> I would not be surprised if it turns out that the Copenhagen interpretation is a modern phlogiston theory.

</comment><comment><USER41> We changed the url from [http://www.wired.com/2014/06/the-new-quantum- reality](http://www.wired.com/2014/06/the-new-quantum-reality) to be the original source.

<comment><USER33> Thanks, because that is a really well written article so I added that magazine to my RSS.</comment></comment>


IPv6 domain readiness tester - <AUTHOR> http://ip6.nl/ <comment><USER2> [http://ipv6-test.com/validate.php](http://ipv6-test.com/validate.php) is a better one. It also tests if the host can be reached. The one you submitted only tests DNS.

</comment><comment><USER3> Not that I have any choice with regards to my ISP in my area (NYC - I'm stuck with TWC), but what ISPs in the US actually provide IPv6 support for even a majority of their residential customers?

Time Warner Cable certainly doesn't, but I was surprised to find that even Verizon doesn't provide it for many (most?) of their FiOS customers.

When pressed about IPv6 support, ISPs, typically respond that very few services require it, so it's not a priority for them. But this is a catch-22 - the only reason so few services require it is that so few people have _access_ to it!

And the people who suffer the most aren't their customers - it's people in other countries who are forced to live behind NAT because of IP address exhaustion.

For example, the _entire_ country of Qatar shares a single IP address: [http://en.wikipedia.org/wiki/User:126.96.36.199/header](http://en.wikipedia.org/wiki/User:22.214.171.124/header)

<comment><USER4> "Comcast's IPv6 deployment continues to expand, over 25% of our customers are actively provisioned with native dual stack broadband! The following areas of the Comcast broadband footprint are now fully IPv6 enabled - Colorado, New Mexico, Minnesota, Kansas, Missouri, Maryland, Ohio, Pennsylvania, West Virginia, and Houston."

[http://www.comcast6.net/](http://www.comcast6.net/)

<comment><USER5> I live in a suburb of Houston and I do not have IPV6 via comcast...

</comment></comment></comment><comment><USER6> [http://ip6.nl/#!mi5.gov.uk](http://ip6.nl/#!mi5.gov.uk) 0 stars [http://ip6.nl/#!fbi.gov](http://ip6.nl/#!fbi.gov) 3 stars - yay hope [http://ip6.nl/#!cia.gov](http://ip6.nl/#!cia.gov) 3 stars again

My ISP, nope, BBC again no. Would appear the UK in no rush to move into IPv6, least publicly. Bet some people wondering what IRC is like over IPv6 and others wondering why Snowden never warned us about the lack of intelegence support of IPv6. If he did, then things would improve, but until people are forced into a corner then nothing will change.

I just think it is sad how IPv6 could of been and still can be pushed via mobile phone usage and in that case it is an area which makes 4G voice calls and easier from what I have heard (least networking level). Certainly the need is there in the mobile market, just support elsewere and it is not even a chicken and egg situation more a case of slowly dripping into excistance.

That all said it is such an investment in skills and effort for a demand that can just use an alternative. That and the NAT been abused so well that the biggest area (mobile networking) has been fine without it. Just makes it hard to justify, even if current hardware and software support it. One does not just go IPv6 overnight without some business tears.

</comment><comment><USER7> Cloudflare offers transparent IPv6 support for the sites using it. It's just not enabled by default.

<comment><USER8> And news.ycombinator.com runs on Cloudflare, with IPv6 disabled. I weep for the future.

<comment><USER9> letting the admins know that the option is available is probably a good way of changing this. ([http://ycombinator.com/contact.html](http://ycombinator.com/contact.html))

</comment></comment></comment><comment><USER10> Okay so the domain my site is run off gets 0 of 5 stars. I know my VPS provider has ipv6 support, so how do I close the gap?

<comment><USER11> Looks like: IPv6 addresses for NS (nameserver) and MX (mail exchange) records, and root and www subdomain. Looks to check DNS responds to queries over IPv6, too. Easy mode to get at least four stars: enable IPv6 with a free Cloudflare account.

</comment></comment><comment><USER12> Hmm, 2 out of 5 stars only because the nameservers (which are commonly only queried by recursive nameservers such as your ISP's) don't have AAAA records? I tried disabling IPv6 some time ago, and my site was one of the few still working next to Google services and a select other few. The recursive DNS servers did all the v4 work for me.

Sure, 4/5 stars because I don't have one of the items (v6 nameservers), but 2/5? The second point is inherent from the first, and www.lucb1e.com having no AAAA record is no problem because I don't use it (and it's resolved now, they cache results).

</comment><comment><USER13> Not even Amazon has ipv6 dns entries. So it cant be very important.

<comment><USER14> I can now say Amazon supports IPv6 internally throughout most of the infrastructure.

</comment></comment><comment><USER15> Please, github, enable ipv6. Please.

<comment><USER16> What's so special about github?

<comment><USER15> Its the only service that stops me being able to run test machines ipv6 only. Everything else I need supports ipv6 already. And hey they have loads of people, surely one of them is interested...

<comment><USER7> Just install NAT64 on the network so that they can reach ipv4 if needed

<comment><USER15> As far as I can see I would still need to run ipv4 NAT in order to get ipv4 addresses for NAT64, so it doesnt really seem worth it.

<comment><USER7> One per network is fine though, you can run thousands of hosts behind one ipv4 in nat64

<comment><USER15> OK will try it at some point...

</comment></comment></comment></comment></comment></comment></comment><comment><USER17> It was a disappointed to find my host, Digital Ocean, does not support IPv6: [https://www.digitalocean.com/community/questions/is- ipv6-ava...](https://www.digitalocean.com/community/questions/is- ipv6-available)

After I made an effort to get configure it within Nginx I was hoping things would work.

<comment><USER18> Hurricane Electric tunnel broker: [https://tunnelbroker.net/](https://tunnelbroker.net/)

</comment></comment><comment><USER19> They only check the DNS server and DNS entries, not actual connectivity. No attempt is made to verify whether your mail and web servers are actually reachable over IPv6.

</comment><comment><USER20> So, facebook's IPv7 is 2a03:2880:2110:df07:face:b00c:0:1</comment>


Last post of the year from 24ways.org - <AUTHOR> http://24ways.org/2013/run-ragged/

<comment><AUTHOR> Just wanted to highlight what a fantastic resource this is and thanks the effort of everyone involve. Every year I learn something useful from it.</comment>


SSL with Rails - <AUTHOR> http://collectiveidea.com/blog/archives/2010/11/29/ssl-with-rails/ <comment><USER2> Thanks, yoinking this for Appointment Reminder.

In the spirit of creating wealth through exchange:

<https://gist.github.com/721114>

A frequent problem I have is using CSS files which include <http://> linked background images, which will pass through every test with flying colors and then trigger prominent mixed-content security warnings in a browser I very seldom use. Slap this code in your initializers directory and you never have to worry about it again.

Just write your stylesheet_link_tag as normal, and then include :cache => request.ssl? ? "foo-secure" : "foo-standard"

Bonus: the included code also minimizes CSS and JS files in production, saving you page load time.

<comment><USER3> The minimizing is good, but doesn't using a path like //example.com/my- image.jpg instead of <http://example.com/my-image.jpg> fix the issue with http/https?

<comment><USER2> That works if you can enforce that 100% of the time, against all CSS loaded by your site. The first time you forget it in e.g. a lightbox script you add to get used on a landing page whose implementation you didn't even glance at, your registration page will break. This has cost me thousands of dollars before, so I enforce assumptions like "there are no <http://> urls in my CSS files" in code.

<comment><USER4> How about part of your test suite that just greps for <http://> in public/ __/*.css and fails CI if it finds anything?

</comment></comment></comment></comment><comment><USER5> Wow, that tip about using protocol relative URLs is going to clean up so much code for me.

For those who missed it, apparently this works across browser to avoid mixed content warnings:

<script src="//ajax.googleapis.com/ajax/libs/prototype/1.6.1/prototype.js" type="text/javascript"></script>

<comment><USER6> Ach! Accidental downvote.

Thanks for the detail.

</comment></comment><comment><USER7> I usually just use whatever is the most robust, stable open source choice for Rails app. Sphinx for search, etc. and Apache for the web server. Doing SSL is well documented and pretty much everything is possible and documented and supported by webmin too. I have a virtual host sending all non-ssl requests to ssl.

It doesn't have to be Ruby, especially if I won't be programming for it anyway.

</comment><comment><USER8> If you want a super simple way to require SSL in Rails 3, you can use route constraints: [http://kyleslattery.com/entries/requiring-ssl-using-route- co...](http://kyleslattery.com/entries/requiring-ssl-using-route-constraints- in-rails-3)

</comment><comment><USER9> This part was new to me: <http://en.wikipedia.org/wiki/Strict_Transport_Security>

This is an additional (draft status) header that tells the user agent to always (or up to X seconds in the future) use HTTPS for that site and it seems, to disallow any connections using bad certificates.

Currently only Chrome and latest dev version of Firefox seem to support it. Chrome apparently has a pre-loaded list of always-https domains, like www.paypal.com -- who are also the people behind this draft proposal.

</comment><comment><USER10> The one problem nobody talks about is that browsers won't cache content served over ssl, which means your website will be a lot slower to your users. That will directly costs you money.

<comment><USER11> Firefox does cache content - I just ran Firebug to confirm. I can only assume from how fast my site works in other browsers that the same applies to them. What browsers are you referring to?

</comment></comment><comment><USER12> Interesting. I didn't know it was this easy with rack.

Am I right in thinking that if third party embeds/widgets dont answer at an https url, I am stuck with mixed content warnings?

<comment><USER13> You could proxy the request through a back-end server that does support HTTPS.

<comment><USER2> A word of caution: it is very easy for naive attempts to do this to end up creating an open proxy. You don't want that.

<comment><USER14> You _profoundly_ don't want that.

<comment><USER2> Thank you, your comment reminded me of why this is Severity: Critical. Here's where Thomas and I live in different worlds: I hear "open proxy" and I think "a malicious user can use your server to attack other services on the Internet or download kiddie porn.". Thomas hears "open proxy" and starts to see visions of a host possibly _within the firewall_ which can be used to make arbitrary HTTP requests against servers _within the firewall_ , which let's you target the weak underbellies of e.g. intranet applications which have no security because all users are presumed to be employees and where compromise can do interesting things like e.g. read out all your source code through Trac, initiate wire transfers through the accounts payable application, etc.

Yeah, you _profoundly_ do not want open proxies.

<comment><USER14> Sev: biblical. You're coughing up the lights-out management consoles on servers and storage appliances with them. I'm sure there's all sorts of funny business to be perpetrated through them with admin and account management applications. But I wouldn't know, because you're usually 5 minutes from command execution on the server once you find one.</comment></comment></comment></comment></comment></comment>