Why lawmakers just voted against their own bill to reform the NSA - <AUTHOR> http://www.washingtonpost.com/blogs/the-switch/wp/2014/05/22/why-76-lawmakers-just-voted-against-their-own-bill-to-reform-the-nsa/

<comment><USER2> So the House Committee on Rules committee butchered this bill to the point where it actively went against the legislative intent. In case anyone is curious, here's the current members of the House Committee on Rules:

\- Pete Sessions, Texas, Chairman (R)

\- Virginia Foxx, North Carolina Vice Chair (R)

\- Rob Bishop, Utah (R)

\- Tom Cole, Oklahoma (R)

\- Rob Woodall, Georgia (R)

\- Rich Nugent, Florida (R)

\- Daniel Webster, Florida (R)

\- Ileana Ros-Lehtinen, Florida (R)

\- Michael C. Burgess, Texas (R)

\- Louise Slaughter, New York, Ranking Member (D)

\- James P. McGovern, Massachusetts (D)

\- Alcee Hastings, Florida (D)

\- Jared Polis, Colorado (D)

</comment><comment><USER3> It would be nice if Congress had a concept of a "code freeze".

This quote as also odd:

> _"While far from perfect, this bill is an unambiguous statement of > congressional intent to rein in the out-of-control NSA, " said Laura W. > Murphy, director of the American Civil Liberties Union's Washington > legislative office._

A statement of congressional intent sounds like asking someone nicely.

<comment><USER4> In a parliamentary system the analogy of a code freeze would be a "vote" on the substantive motion.

At the moment the bill is in committee and is being amended - I trust that who suggests and votes for amendments is a matter of public record? if not then the USA has a lot more to worry about that the NSA.

<comment><USER5> Except in the US after Congress and the Senate pass the bill a conference committee gets to make the two versions match so I believe the "vote" on the substantive motion couldn't occur until it's in conference.

Also the votes for amendments may only be recorded as voice vote (not a roll call I believe) which means you wouldn't know who voted for what. Who offered the amendment should be public record.

</comment></comment></comment><comment><USER6> Why do news stories like this never link to the actual text of the bill? Isn't it a little worrying when the participants in a democracy consider their governance "over their head" or simply boring? It seems symptomatic of the broader problem the bill is meant to curb: that we've let our rulers get away from us and left them dabbling in their own hidden sandbox.

</comment><comment><USER7> Is there any way to tell who is responsible for the watered down sections that allow for bulk data collection?

<comment><USER8> After a period of major reform at the end of the 19th century, and especially since the procedural struggles over civil rights legislation in the 1960s, power in the House has been increasingly concentrated in the hands of the Speaker. So it is a good bet that whoever was responsible had Boehner's acquiescence if not outright support.

The Speaker in turn answers to the caucus which selects him, and Republicans in particular by and large hold to the Hastert rule which says that nothing should be brought to the floor unless it has support of a majority of the majority.

<comment><USER9> So what is meant to be an approximation of the will of the people, becomes under the Hastert Rule, an approximation of the will of the people so long as the people's will meets the approval of a select group of ~25% of the people's representatives.

<comment><USER10> The greatest flaw of representative democracy is that the people's will must meet the approval of some or all of their elected representatives. Due to the many and varied issues that come across the legislative bodies, you will inevitably have issues where the majority of the people are at odds with the majority of representatives. Every candidate in an election is a compromise; the voter agrees with them on some issues, but not others. The bigger Congressional districts get, the worse the problem becomes.

</comment></comment></comment></comment><comment><USER11> Typical legislative sleight-of-hand - welcome to the sausage factory. Further developments will probably reveal that the new law enshrines the status quo.

<comment><USER12> Here you go -

[http://www.wral.com/house-poised-to-pass-curbs-on-nsa- survei...](http://www.wral.com/house-poised-to-pass-curbs-on-nsa- surveillance/13666020/)

"NSA officials were pleased with the bill for another reason: The new arrangement will give them access to mobile calling records they did not have under the old program."

</comment></comment><comment><USER13> We begin therefore where they are determined not to end, with the question whether any form of democratic self-government, anywhere, is consistent with the kind of massive, pervasive, surveillance into which the Unites States government has led not only us but the world.

This should not actually be a complicated inquiry.

[http://snowdenandthefuture.info/events.html](http://snowdenandthefuture.info/events.html)

<comment><USER14> Would you kindly stop spamming this link into every tangentially related article?</comment></comment>


Revisiting iChat Hacking - <AUTHOR> http://the.taoofmac.com/space/blog/2012/01/07/2350 <comment><USER2> i'm curious why people go through all of this work to hack ichat instead of just using adium.

<comment><AUTHOR> Probably because Adium tries to do too much, whereas iChat comes with the OS and does the job.</comment></comment>


Building a mini Trello using Firebase in 2 hours! - <AUTHOR> http://blog.stackhive.com/post/114386894969/login-sign-up-and-a-real-time-app-using-firebase

<comment><USER2> this isn't Trello. Its just a demo to create sticky notes. Change the title. Movable Sticky Notes in 2 hours.

<comment><AUTHOR> You can actually extend the functionality quite easily to create a Trello like task management tool.

I changed it to "mini" Trello to make it more clear...

<comment><USER3> What's "Trello" about Trello is more than just movable notes, as the GP mentioned.

It's customer support, it's the silly but engaging "power-ups" and all the additional work put in after making movable notes.

A more accurate title: How to make the movable notes feature of Trello (not as catchy, though).

<comment><AUTHOR> I completely agree. I use Trello myself.. I got excited with Firebase and how easy it makes creating a real-time task list management system. That was what the post was supposed to highlight..

I just got the heading wrong I guess... It was really exciting to venture into real-time apps and I could see the demo being extended into a Trello like application.</comment></comment></comment></comment>


The End of Car Culture - <AUTHOR> https://www.nytimes.com/2013/06/30/sunday-review/the-end-of-car-culture.html?smid=tw-share&_r=4& <comment><USER2> One thing missed by most of these post-auto stories: transit used to be _sooo boring_ , and no longer is.

I'm old enough to remember riding the bus before mp3 players and smart phones. Your only options were to read or to bring a CD player. Walkmans (Walkmen?) were a pain: batteries got expensive, and carrying more than 1 or 2 CDs was a pain, so you ended up listen to a few tracks over and over. Newspapers and books were good, but not as stimulating as the internet or email or Words With Friends. I still remember waiting for the train with no music, nothing to read, and nobody to talk to; it was excruciating.

Nowadays, I prefer transit to driving, even when it's a bit longer, because I can just zone out on my iPhone the whole way.

<comment><USER3> I wish I could say this about transit in Beijing. The subways are so crowded you barely get to sit down, perhaps you would have enough room to fiddle with your smartphone if only there was signal for it in the tunnel (damn China Unicom), and standing for 45 minutes kind of sucks.

Taxi!

<comment><USER4> That's an economic issue, the roads in beijing are most likely croweded too and to boot most of those people are using public transport because they can't afford cars.

(I am using genaralisations about China to make these statements)

<comment><USER3> True. The subway is actually very nice, and in Beijing, very cheap (just 2 RMB a trip, no distance limit!). Those who can afford cars aren't using transit, and many of us in the middle are using taxis anyways because they are still not that bad (50 RMB = ~$8 taxi vs. 2 RMB subway...).

I would prefer nice public transit though, or better yet, the ability to bike without getting killed by a car.

<comment><USER5> I think it's a shame that private automobiles were introduced in China. There are too many people in large Chinese cities for them to all want to ride in a car. Imagine how much more livable these cities would be if everyone was either riding a bike or the subway.

<comment><USER6> I don't know about Beijing but in Shenzhen, lots of the newer apartments are located in "no car" gardens that can span over a few blocks. There are lots of trees, swimming pools, tennis courts, etc. Quite livable places actually.

<comment><USER3> All nicer apartment blocks are like that, actually; they usually have a garage underneath but even this is too expensive for most people (whose work unit will give them a Mercedes even if they can't afford a parking space!), so they park on the street...fun stuff.

<comment><USER6> I've actually always wondered about that. I often see loads of Mercedes/BMW/Lexus/Porsche cars parked around old buildings where the rent is less than 150$/month. If what you say is true (companies giving Mercedes to their employees), why is this? Or do some Chinese prefer owning an expensive car rather then living in a nice place?

<comment><USER3> I said "work unit" and I'm talking about government employees :) The government gives them the car and they use their connections to "qualify" for subsidized housing (the kind they are building for the poor!). So the result is a lot of Audi's parked around public housing.

Whatever works.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> There is an interesting intersection between this piece, the first comment by BadassFractal, and Philip Greenspun's recent post about Denmark's bike infrastructure: [http://blogs.law.harvard.edu/philg/2013/06/17/danish- happine...](http://blogs.law.harvard.edu/philg/2013/06/17/danish-happiness- bicycle-infrastructure/) :

 _Because a bicycle can be used safely on every segment of almost any trip it becomes much more practical to dispense with car ownership and its $9100 annual bleed (according to AAA). The 24-hour metro, excellent intracity buses, and frequent trains other parts of Denmark complement the utility of bicycles in situations involving miserable weather and /or long distances. . . .

The way that we have things set up in the U.S. contributes to our higher per- capita GDP, but it does not make us better off. For example, if you think that a trip by bicycle will be unsafe due to a lack of bike lanes and take a car, you will burn gasoline, maintenance, and depreciation on that car, thereby boosting the measured GDP compared to if you had used your muscle power (not figured into GDP). If you then sign up for a gym membership and personal training to get rid of the fat that you've accumulated during all of these car-instead-of-bike trips, that boosts GDP though at the end of the process you will be no thinner or fitter than if you had biked. If you get hit by a car the GDP can easily be boosted by $25,000. You will buy a new bicycle. The car will need bodywork. You may be concerned about your health and get an MRI done at one of the world's most expensive MRI clinics. Your MRI will be read by one of the world's highest paid radiologists. What the radiologist says will be read to you, for a GDP-boosting fee, by one of the world's highest paid neurologists._

</comment><comment><USER8> For urban dwellers blessed with proper public transportation and a good Walk Score, a car is a huge headache and drain on finances. They simply all need to go and be replaced with extremely efficient, clean and safe transit systems. Nobody enjoys sitting in traffic for hours a day commuting. Nobody enjoys wasting time on maintenance. Nobody enjoys the process of acquiring the vehicle.

At the very least, car sharing is a step in the right direction. Eventually as the Google car becomes reality, I hope that the # of cars on the roads is reduced to a bare minimum and we don't need to drive / pay people to drive any longer, and free up a lot of human talent to do something more productive.

<comment><USER9> > Nobody enjoys sitting in traffic for hours a day commuting. Nobody enjoys > wasting time on maintenance. Nobody enjoys the process of acquiring the > vehicle.

Maybe not, but some of us find it vastly preferable to being crammed in with others on public transport. So don't tar everyone with your particularly preferences thanks.

<comment><USER10> I guess one has to balance the time spent in traffic and being crammed with others in public transport. Here in the bay area, one can get stuck on 101 for 2 hours when it would take only 40 minutes via Caltrain. I'm sure quite a few people would rather save 1.5 hours every day being crammed in with others.

<comment><USER3> Caltrain also has seats most of the time. The worst is that you might have to sit next to someone...cry me a river.

<comment><USER11> One of the more depressing things is that the conversation around transit is so frequently: "But... [horrified whisper] _other people_!"

</comment></comment></comment></comment></comment><comment><USER12> I don't know, man.

I live in Colorado, in a super bike-friendly city, and I'm buying a car. Can't take public transportation to a campground. Can't take public transportation to a mountain bike trail. Public transportation for skiing is a huge pain in the ass.

<comment><USER13> But will you ride it daily to work? I think an argument isn't that nobody needs cars but more people will be relying on them less and the total # of hours spent driving them will be reduced.

<comment> qu4z-2 Indeed. As you carry on in that direction, hiring a car when necessary may become much more attractive compared to the upkeep on a car you only use occasionally.

<comment><USER12> I belong to a non-profit carshare, and that shit adds up fast, especially if you want a car for a whole day.

</comment></comment></comment></comment><comment><USER14> Right now I'm stuck in the suburbs and the city I live in just doesn't have good public transportation and probably won't ever. There is one scenario that I'm looking forward to that could make my life easier.

What if Google or another company perfects self-driving cars? You could have "cars" that are basically motorized shopping carts. You go online and register your car with a particular store. Your car could drive from store to store having the workers load everything for you. If a lot of people use this it could be efficient since these self-driving vehicles drive optimally and won't get into accidents and will cooperate with each other to get to where they're going.

This way I can spend more time doing what I want and less time stuck in traffic and more time doing what I want. Basically it's automating a set of tasks you used to do with a car. I hope that day comes very soon.

<comment><USER15> Wouldn't this usage fall in the "better mouse-trap" model?

Instead of having _your_ car driving around, it would be much better to have businesses' trucks and vans going to customers. This way, companies get a better distribution process, parking lots become a thing of the past, you get to reduce your car usage... and perhaps this even makes it viable to have car- sharing companies working in the suburbs.

</comment></comment><comment><USER16> My dads tire and wheel business is screwed. 40+ years in the business, and it has dwindled down to practically nothing.

<comment><USER17> Uhh, there are still plenty of cars last time I checked. Help your dad adapt to modern age and he'll do just fine.

</comment></comment><comment><USER18> As a car guy, maybe I am biased, but:

1) Where are the data/charts behind this statement: "Part of the explanation certainly lies in the recession, because cash-strapped Americans could not afford new cars, and the unemployed weren't going to work anyway. But by many measures the decrease in driving preceded the downturn and appears to be persisting now that recovery is under way. " If the whole premise of the article is that this is more than just something related to the recession, shouldn't we be able to see that data for ourselves, and the justification should be more than a passing statement. The wording of "by many measures" also makes it sound like the author is cherry picking metrics to make his point. I'd also like to see the car usage compared to unemployment rates for the same age group. Overall, I just don't believe the validity of the claim without seeing more data.

2) As the author says "Whether members of the millennial generation will start buying more cars once they have kids to take to soccer practice and school plays remains an open question." I'm not sure it is an open question. When you're single and can live near where you work, it's reasonable to get on without a car. Then life changes, and you need a car.

3) If there is an actual drop-off in car usage, then it could be because we likely are at an inflection point of some significant changes in the industry. The two big ones being electric cars with Tesla leading the way, and the second one being self-driving cars with Google leading the way. We might just be seeing a temporary drop as people reject the "old" way, and as these two new technologies take-off and become more mainstream, so again will car usage .

</comment><comment><USER19> There's still a few billion people with no transport who will be buying vehicles soon.

I really freaking hope we've sorted out some better engine than internal combustion.

But even batteries might be tricky - how much rare earth metal do you need for a billion electric vehicles?

The problem with Beijing population all driving cars isn't so much that they're driving cars - it's weird of me to demand they still ride bicycles - but that they mostly drive badly made badly maintained cars.

Also, this is the kind of thing where international travel helps you see other perspectives. The first 'drive through mailbox' I saw was in the US. At Gilroy Outlets I would see someone leave one store, get in their car, drive 20 yards to be outside the next store, get out, go shopping, get back in their car, drive 50 yards to the next store they wanted to visit, and so on. I have no idea if that's common in the US, but I saw it in a few different malls in CA. It kind of freaked me out. (I understand if these people have a disability that makes walking painful or whatnot. That wasn't obvious from looking at them. Maybe they just hate the smell of garlic?)

</comment><comment><USER20> For me, this is where lyft is so exciting. It used to be, if you could depend on walking/biking/public transit for 90% of your usage, you either still had to get a car for the other 10% or suffer a 10% downgrade in lifestyle.

Services like lyft perfectly patch over that hole. The cost of taking lyft indiscriminately is still less than owning and maintaining a car and the time I spend waiting for a lyft is counterbalanced by the time I would have spent parking.

</comment><comment><USER21> > When adjusted for population growth, the number of miles driven in the > United States peaked in 2005 and dropped steadily thereafter

The reason for this is more directly "The Internet" than they suggest. Nearly everything that used to require you to drive somewhere is now done online. You order products off Amazon today.

</comment><comment><USER22> I know it has a great deal to do with this city, but since I moved to San Francisco, I've found myself using my car so rarely that every time I pull back into my garage, I need to disconnect one of my battery terminals, or I'll have a dead battery the next time I need to drive somewhere.

</comment><comment><USER23> The end of car culture can't come soon enough.

<comment><USER24> this article is wrongheaded in many, many ways. The crux of the problem is that cars are designed and produced ahead of what is actually wanted and desired without concern whether there is actually an impedence mismatch with what consumers want in a car. For example, modular design in cars no longer exists. You can't find a car anywhere that can be transformed with add-ons between a commuter car to a weekender car to a car for hauling things to a car for being sporty. The same is true with manual transmission, you can't find it on the market ANYWHERE. For tinkerers, you can't get information about your car's computer chips or any kind of API that could serve a buyer's use, not someone elses. Not only that, you pay a lot of money for a car, and then find out that it's easily hackable, by anyone except the rightful owner. People love cars, and definitely prefer them to public transformation, which is unreliable at best, but there just aren't any cars on the market that match what consumers want.

<comment><USER25> What? When was the last time a car could be converted from a sports car to a capable tow vehicle with an _add-on_? You can bolt a tow hitch on just about any car, but tow vehicle/sports car/commuter/weekender runs _so much deeper_ than that- at least if you want it to be better than "poor" in all of the above categories.

 _The same is true with manual transmission, you can 't find it on the market ANYWHERE._

Patently false. Manual transmissions are not as common as automatics, and certain vehicles no longer offer a manual transmission, but they can still be found in many different vehicles.

<comment><USER24> "certain vehicles no longer offer a manual transmission"

Exactly right.

<comment><USER24> if the make and model of the car of your interest is not available at all in manual transmission, then yes, absolutely yes, it's simply not available anywhere.

Henry Ford famously said "Any customer can have a car painted any color that he wants so long as it is black" and that's the problem today. An entire market of buyers is being ignored.

If you accept that only grey color cars are being made and wonder why people aren't buying grey cars, nothing can help you.

<comment><USER24> Manual transmission is but an example. The idea is that ownership rates declined before the downturn unrelated to the economy. What is declining, is the value proposition that the current set of new cars offers compared to older cars and, that many would-be car buyers just aren't finding what they want in a car. Imagine the cell phone market before the iPhone, that is the state of the car market today.

<comment><USER25> _What is declining, is the value proposition that the current set of new cars offers compared to older cars_

What is happening (IMO) is that "old" cars (in relative terms) are lasting longer and longer. The car I learned to drive in had a 5-digit odometer. The car I drive today can reasonably be expected to reach 250k with regular maintenance.

 _many would-be car buyers just aren 't finding what they want in a car_

Ok, what _do_ they want? 'Cause I assure you, "it doesn't have a manual transmission" is not the complaint of the majority.</comment></comment></comment></comment></comment></comment></comment>


Clamping Down on High-Speed Stock Trades: US, Canada, European regulators - <AUTHOR> http://www.nytimes.com/2011/10/09/business/clamping-down-on-rapid-trades-in-stock-market.html?hp

<comment><USER2> It's supposedly regulators driving this crackdown, but almost all the quotes in the article come from _competitors_ to high-speed trading firms claiming — surprise — that high-speed trading firms need to be regulated.

<comment><USER3> Not that they don't have a point, of course.

</comment></comment><comment><USER4> Stocks should only be allowed to trade at the speed that value is created or lost. The stock market is a human institution, (supposedly) designed to measure value of human businesses.

Trading in microseconds is ridiculous.

Trades should be in human time - essentially, minutes and hours, not seconds and microseconds. Business value does not change that quickly. HFT is representative of a financial industry set on creating ever more abstract tools and products that are more akin to gambling than representing the value of the actual business. The stock market needs to return to it's roots. There's been enough chaos already.

<comment><USER5> First off, business value can change quickly.

Sudden events happen. You can think of dramatic events such as terrorist attacks or death of an important executive; but there's other information, such as the public announcement of a new product (e.g. 'Apple are making a new type of device!', perhaps rapidly followed by 'Google have just said they'll support it!') that also result in fast changes.

But, most importantly, the market price provides information that influences further decisions. Maybe I'm not an expert on oil prices, but I believe if it goes below $70 a barrel, its a signal that the world economy is slowing down, and I want to dump my Google stock, because I'm only willing to accept a certain level of risk in my personal investments.

The fact that I can see the price of the oil provides me with a signal (maybe noisy), derived from aggregated intelligence of other investors. This is a useful function.

Its completely legitimate to want to sell my Google stock in response to other market signals, and as quickly as possible, in response to the new information they provide.

The fundamental value of holding the Google stock, to me, has changed, fast, because of other information that's become available.

So fundamental business value can change fast, for a variety of reasons, and it can add value to be able to respond to quick changes quickly.

<comment><USER4> _Perception_ of business value can change quickly. Actual business value itself does not tend to change so fast. Dumping your Google stock if oil drops below $70 per barrel is solely based on your own perception and your own trading rules. The value of Google's products remain the same. If a competitor releases a new device, that is based in human time.

Does it make a difference if you sell at 12.01pm after a competitor's product announcement rather than 1pm? Right now, yes. Do I think it should? No.

The reason prices swing so quickly is because trades happen so quickly. If trading was slowed down, prices would not swing as quickly. The current situation reinforces a vicious cycle where the fastest trader has an advantage in the market.

Public share prices is a measure of _how much the company is worth to people that buy shares_ not of how much value the company creates by selling their products. PepsiCo should be valued on how well they sell soft drink, not millisecond-by-millisecond analysis of unrelated issues.

<comment><USER6> There is no such thing as 'actual business value'. The very definition of value is based on perception; something of value is something people want. The closest concept I can think of to what you mean is "book value", which comes with its own set of drawbacks.

</comment></comment></comment></comment><comment><USER7> it isn't even a crude/primitive AI, it is just algorithms that are let loose for a limited periods of time in a limited activity space - and humans are already feeling threatened(xenophobic) as this algorithmic trading feels completely unnatural. (The reason of fairness is complete BS because professional traders have always had informational and speed advantage, it is just that it was on the human scale - seconds vs. minutes, and now it crossed into non-human microseconds scale)

Well, welcome to the 21st century. The algorithmic trading will look like a child play when in the "drone arm race", the sides would be forced to delegate final shoot decision to the drone algorithms for exactly the same reason - human speed wouldn't be enough as the millisecond advantage makes the critical difference between wining and loosing. That would be a reason to feel slightly xenophobic i guess, though we'll possibly get used to it :)

Wrt. unfair advantage vs. investors not utilizing this advanced tool of algorithmic trading - whole human history since the time of apes is just a continuous sequence of "slow" humans being ruthlessly overtaken by other humans taking [unfair like any advantage] advantage through better tools and weapons. We're direct beneficiaries of such an unfair advantage and, as we aren't the ultimate goal, just an intermediate point in this history, we'll be a victims of it in due course of that history.

<comment><USER8> Here's your +1 to get you out of gray area, but WRT "Wrt. unfair advantage" you should really read up on HFT in detail. _All of HFT_ is based on the premise of getting an access to the order flow a split second sooner than everyone else. To get such access HFT firms pay very hefty fee to Nasdaq and other exchanges. That's where the unfair advantage is - not in the use of computers, but in an exclusive access to the information that not even all banks can afford.

<comment><USER9> everyone can purchase fast access. If a brokerage or a bank wanted to do it, they certainly could. It's like saying that Kinko's has an unfair advantage at making copies because it bought expensive copiers and regular folks have not.

<comment><USER10> I'd say it's more like Kinko's discovering you want to make some copies...and quickly erecting a store right outside your house so you stop there instead of Staples.

<comment><USER9> and what's wrong with that? Or rather, would you ban that too?

<comment><USER10> Yes, because that key word... _discovering_...implies that our rhetorical Kinko's has somehow gleaned advance knowledge that you're about to leave home to purchase some copies.

I don't think HFT in general is ruining things. I think the ability of HFT firms to quickly enter orders and quickly widthdraw those same orders _millions of times a minute_ enables them to gain some insight into the trading market that the market was never designed for. And I believe _that_ is a perversion of the original open-call trading that the stock market was built upon. All buyers should be able to see the ask at the exact same time.

</comment></comment></comment></comment></comment></comment><comment><USER11> What's the harm in HFT? It certainly increases liquidity in the market, making it easier for other investors to enter or exit a position on short notice. That's a good thing.

Let's not fall pray to generalized anti-finance populism without good reason.

<comment><USER12> My suspicion (which I can't prove--this is just a gut feeling) is that there is an economically correct price for any given thing, and that markets will move toward that price. This economically correct price only changes when factors in the "real world" change, or we have reason to believe factors in the real world are going to change (e.g., bad weather might reduce crop yield, or forecasts of bad weather make is expect a reduction in yield).

Actors in the markets get information not only from real world sources (e.g., news reports), but also infer information from the activity of other actors in the market, so there is a bit of a feedback look in the market.

My suspicion is that as you increase the frequency of trades, starting from very infrequent (e.g., people trading in person directly with the person they are buying or selling from) and then going to faster and faster methods (trades by mail, then telegraph, and so on), there are two effects. First, the market is able to more quickly converge to near the "correct" price for the item, and there is some increase in fluctuation around the correct price due to strengthening of the feedback loops.

I suspect that there is some optimum trading speed, which depends on the rate that the real world events (things like weather changes, fashion, wars, and so on) happen, and when you push trading speed past that you stop gaining on convergence speed, but continue increasing instability due to the feedback loops.

The result is a market that is less and less tied to the real world. In effect, the market ends up _making_ _up_ information from the trading activity itself. The trading essentially ends up being based on noise rather than signal. It's hard to see how that can be good.

<comment><USER9> trading activity is information because it is real people putting real money at risk and expressing their views of what the assets are worth. As in any competitive market, HFT traders cannot make money if they only trade with themselves - that is a mean zero proposition. But by trading with people who price the assets incorrectly at a point in time, they help move asset prices to their "correct " levels

</comment></comment></comment><comment><USER13> I keep saying that one of the biggest problems with HFT is the false illussion of at will liquidity.

This is what happened at the "flash crash", many traders providing liquidity, which included most of HFT scene getting out of market, because they (and their algorithms) decided it was not worth the risk.

Old market makers did not operate like this, they were providing quotes even in a very volatile markets. Of course, many lost their shirts doing so, but were obliged by the exchanges to provide this service.

If none of the liquidity providers have an obligation, then we have no liquidity in volatile times, when we need it the most.

<comment><USER9> yes but if there is an obligation, you could end up with less total liquidity because if you oblige traders to trade when it is a losing proposition to do so, you will get less traders and less liquidity all the time, including volatile times. That's why exchanges actually pay traders to provide liquidity and charge liquidity takers. Imagine if you ordered convenience stores to sell twinkies to people even if they have no money. How would that work in practice ?

</comment></comment><comment><USER14> HFT = [https://secure.wikimedia.org/wikipedia/en/wiki/Algorithmic_t...](https://secure.wikimedia.org/wikipedia/en/wiki/Algorithmic_trading) \+ <https://secure.wikimedia.org/wikipedia/en/wiki/Front_running>

_Front running is illegal._

</comment><comment><USER15> All I read was "Oh noes, the computer, we don't understand it".

I would suggest that the people involved take the time to understand and learn the new branch of their art, instead of calling it "unholy", as if was some evil device by the antagonist of $religion.

</comment><comment><USER16> Gosh. You mean allowing billion-dollar-poker by professional gamblers who have a six-orders-of-magnitude speed advantage over regular investors, and are also indemnified against any and all losses, might be dangerous? I'm shocked. Truly shocked.

No fucking duh. If ANYONE with any power were serious about fixing society or the world economy, this shit would be banned. Yesterday.</comment>


Full characterization of polarization states of light via direct measurement - <AUTHOR>

http://www.nature.com/nphoton/journal/vaop/ncurrent/full/nphoton.2013.24.html <comment><AUTHOR> The full paper is behind a paywall. An article about it from one of the research institutions involved: <http://www.rochester.edu/news/show.php?id=5692></comment>


Containers vs. Zones vs. Jails vs. VMs - <AUTHOR> https://blog.jessfraz.com/post/containers-zones-jails-vms/ <comment><USER2> Jails are actually very similar to Linux namespaces / unshare. Much more similar than most people in this thread think.

There's one difference though:

In namespaces, you start with no isolation, from zero, and you add whatever you want — mount, PID, network, hostname, user, IPC namespaces.

In jails, you start with a reasonable secure baseline — processes, users, POSIX IPC and mounts are always isolated. But! You can isolate the filesystem root — or not (by specifying /). You can keep the host networking or restrict IP addresses or create a virtual interface. You can isolate SysV IPC (yay postgres!) — or keep the host IPC namespace, or ban IPC outright. See? The interesting parts are still flexible! Okay, not as flexible as "sharing PIDs with one jail and IPC with another", but still.

So unlike namespaces, where user isolation is done with weird UID mapping ("uid 1 in the container is uid 1000001 outside") and PID isolation I don't even know how, jails are at their core just one more column in the process table. PID, UID, and now JID (Jail ID). (The host is JID 0.) No need for weird mappings, the system just takes JID into account when answering system calls.

By the way, you definitely can run X11 apps in a jail :) Even with hardware accelerated graphics (just allow /dev/dri in your devfs ruleset).

P.S. one area where Linux did something years before FreeBSD is resource accounting and limits (cgroups). FreeBSD's answer is simple and pleasant to use though: [https://www.freebsd.org/cgi/man.cgi?rctl](https://www.freebsd.org/cgi/man.cgi?rctl)

</comment><comment><USER3> While I'm not sure I agree entirely with the "Complexity == Bugs" section, the main point, that containers are first-class citizens but a (useful) combination of _independent_ mechanisms is spot-on. This has real repercussions: most people I've spoken do _don 't know these things exist_. They know containers do, they have a very vague idea what containers are, but they have no fundamental understanding of the underlying concepts. (And who can blame them? Really, it was marketed that way.)

For example, pid_namespaces, and subreapers are an awesome feature¹, and are extremely handy if you have a daemon that needs to keep track of a set of child jobs that may or may not be well behaved. pid_namespaces ensure that if something bad happens to the parent, the children are terminated; they don't ignorantly continue executing after being reparented to init. Subreapers (if a parent dies, reparent the children to _this_ process, not init) solve the problem of grandchildren getting orphaned to init if the parent dies. Both excellent features for managing subtrees of processes, which is why they're useful for containers. Just not _only_ containers.

But developers aren't going to take advantage of syscalls they have no idea that they exist, of course.

¹although I wish someone could tell me why pid_namespaces are root-only: what's the security risk of allowing unprivileged users to create pid_namespaces?

<comment><USER4> I think the biggest problem is most namespace functionality _is_ root-only.

If I could create pid namespaces for my user-space apps, then every program I write forever would, as it's first step, launch into a pid namespace.

<comment><USER5> You can do that by creating an unprivileged user namespace. To be fair, this does break some things, but this is the key feature that makes rootless containers (in runC) possible.

<comment><USER6> Is it safe to turn that on in current kernels?

<comment><USER5> Well, I can't guarantee there are no kernel bugs in user namespaces, but the work that Eric and others have done to make user namespaces more secure does make me personally confident about running machines that have CONFIG_USERNS=y.

If you have SELinux, AppArmor, SMACK, Yama, or even good seccomp filters set up then I would classify it as "relatively" secure (most of the security issues in user namespaces have revolved around POSIX ACLs providing access where it doesn't make sense -- supplementing those ACLs with something like SELinux will eliminate entire classes of bugs).

Ultimately though, security is relative. Is a kernel that has CONFIG_USERNS=n more secure than one that doesn't? Yes (because it has less code running) but that doesn't mean that CONFIG_USERNS=y is insecure (it depends on what your paranoia level is dialed to).

</comment></comment></comment></comment></comment><comment><USER7> Ignorance admission time: I still have no idea what problem containers are supposed to solve. I understand VMs. I understand chroot. I understand SELinux. Hell, I even understand monads a little bit. But I have no idea what containers do or why I should care. And I've tried.

<comment><USER7> These comments are helpful. Thanks. Sounds like for a given piece of hardware you might be able to fit 2 or 3 VMs on it, or a lot more containers. But without the security barriers of VMs.

That being the case, why not just use the OS? And processes and shared libraries?

<comment><USER8> > That being the case, why not just use the OS? And processes and shared > libraries?

That's essentially what a Linux container is: a process (that can fork) with its own shlib. If you have lots of processes that don't need to be isolated from each other and can share the same shlib, then no, you don't need this mechanism.

<comment><USER7> Okay, so it's a nice self-contained packaging mechanism that obviates dependency hell. Sounds a bit like a giant lexical closure that wraps a whole process. And from which escape is somewhat difficult. Makes sense.

</comment></comment></comment></comment><comment><USER9> As a lowly user¹: linux containers are more like gaffer tape around namespaces and cgroups than something like lego. You want real memory usage in your cgroup? let's mount some fuse filesystem: [https://github.com/lxc/lxcfs](https://github.com/lxc/lxcfs) \- [https://www.cvedetails.com/vulnerability- list/vendor_id-4781...](https://www.cvedetails.com/vulnerability- list/vendor_id-4781/product_id-32696/Canonical-Lxcfs.html)

We have to gaffer tape with AppArmor and SELinux to fix all the holes the kernel doesn't care about: [https://github.com/lxc/lxc/blob/master/config/apparmor/conta...](https://github.com/lxc/lxc/blob/master/config/apparmor/container- rules)

Solaris Zones are more designed and an evolution from FreeBSD Jails. Okay, the military likely paid for that: [https://blogs.oracle.com/darren/entry/overview_of_solaris_ke...](https://blogs.oracle.com/darren/entry/overview_of_solaris_kernel_zones)

Maybe it's Deathstar vs. Lego. But I assume you can survive a lot longer in a Deathstar in vacuum than in your Lego spaceship hardened by gaffa tape.

1: I have uttermost respect for anyone working on this stuff. No offense, but as a user sometimes a lack of design and implementation of bigger concepts (not as in more code, but better design, more secure) in the Linux world is sad. It's probably the only way to move forward but you could read on @grsecurity Twitter years ago that this idea is going to be a fun ride full of security bugs. There might be a better way?

<comment><USER10> I really wish this post went into more detail. It feels too high level to be useful.

I ran into the memory issue recently. In DC/OS when you use the marathon scheduler, if you go above the allocated memory limit, the scheduler kills your task and restarts it.

The trouble is, if you ran top inside your container, and you're running on a DC/OS node with 32GB of memory, top reports all 32GB of memory. So interpreters that do garbage collection (like Java) will just continue to eat memory if you don't specify the right limits/parameters. The OS will eve let it allocate past the container limit, but just kill the container afterwards.

Now the container limit is available under the /proc/cgroups somewhere, but now interpreters need to check to see if they're running in a container and adjust everything accordingly.

Of course you could always tell your scheduler not to hard kill something when it goes over a memory limit, which is why we never ran into that when we were running things on CoreOS since we didn't configure hard limits per container.

<comment><USER11> > top reports all 32GB of memory.

Same applies to CPU and number of cores. A common pitfall for example for ElasticSearch, which bases its default threadpools on the number of visible CPUs. The isolation layer is indeed very thing and leaky in all places.

<comment><USER10> But with CPU cores, it's not a hard limit. At least in Marathon, you can ask for 0.5 CPUs and marathon will use that to decide where your container gets scheduled (so if it has 16 cores, all scheduled containers should add up to 16; which is why you should use fractions of a core to start and then monitor to see real CPU usage) , but the container gets all of the CPUs.

I don't know if that's specific to our Marathon/DCOS setup or if that's a limitation of the underlying Docker daemon. Not sure how K8s or Nomad handle CPU/memory limits either.

<comment><USER12> Nomad's exec & java drivers handle CPU and memory through cgroups (via libcontainer). In particular CPU uses CPU shares[0] so all cores/CPUs are treated as a single pool of shares. In the future we plan on adding CPU pinning and might add cores as a unit of CPU resources.

The idea of constraining CPU resources by core is especially important on ARM servers which don't always report their clockspeed. Packet.net for example has 96 core ARM boxes, so users would probably prefer to set how many cores they need instead of MHz.

[0] [https://access.redhat.com/documentation/en- US/Red_Hat_Enterp...](https://access.redhat.com/documentation/en- US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-cpu.html)

</comment></comment></comment></comment></comment><comment><USER13> It feels like Ms Frazelle's essay ends abruptly. I was looking forward to the other use cases of non-Linux containers.

I think most people are considering these OS-level virtualization systems for the same or or very similar use cases: familiar, scalable, _performant_ and maintainable general purpose computing. _Linux_ containers win because Linux won. Linux didn't have to be designed for OS virt. People have been patient as long as they've continued to see progress -- and be able to rely on hardware virt. Containers are a great example of where even with all of the diverse stakeholders of Linux, the community continues to be adaptive and create a better and better system at a consistent pace in and around the kernel.

That my $job - 2, Joyent, re-booted Lx-branded zones to make Linux applications run on illumos (descendent of OpenSolaris) is more than a "can't beat them join them strategy" as it allows their Triton (OSS) users full access, not only to Linux API and toolchains, but to the Docker APIs and image ecosystem and has been an environment for their own continued participation in micro services evolution.

Although Joyent adds an additional flavor, it targets the same scalable, _performant_ and maintainable cloud/IaaS/PaaS-ish use case. In hindsight, it's crazy that I worked at three companies in a row in this space, Piston Cloud, Joyent, Apcera, and each time I didn't think I'd be competing against my former company, but each time the business models as a result of the ecosystems shifted. Thankfully with $job I'm now a consumer of all of the awesome innovations in this space.

<comment><USER14> I think an interesting bit here is that e.g. Solaris first had Zones (i.e. "Containers"), while virtualisation was added later (sun4v), while the story is exactly the other way around for Linux.

</comment></comment><comment><USER15> Its probably a good time to stop using containers to mean LXC considering the new OCI runc specs containers on Solaris using Zones and Windows using Hyper-V:

[https://github.com/opencontainers/runtime- spec/blob/master/s...](https://github.com/opencontainers/runtime- spec/blob/master/spec.md)

<comment><USER16> I don't think anyone in the container dev community thinks that containers means LXC only. Even back in 2013, docker's front end api was designed to support other runtimes such as VMs and chroot. Perhaps this is a marketing story gone awry?

<comment><USER17> Afaik even the docker noob tutorial already points out that it is not just LXC only.

<comment><USER5> Docker hasn't support LXC as a backend for at least a year.

<comment><USER17> You mean they replaced it? May also what I've read. Unsure.

<comment><USER5> They've been using libcontainer and then moved to runC (which is effectively a standard-compliant wrapper around libcontainer) since before 1.0. LXC was only used in the early history of Docker, and it was pretty bad to be quite honest (it's better now but there's no chance Docker will switch back).

</comment></comment></comment></comment></comment></comment><comment><USER18> I think it's important to realize that the reduced isolation of containers can also have pretty significant upsides.

For example monitoring the host and all running containers and all future containers only means running one extra (privileged) container on each host. I don't need to modify the host itself, or any of the other containers, and no matter who builds the containers my monitoring will always work the same.

The same goes for logging. Mainly there is an agreed-upon standard that containers should just log to stdout/stderr, which makes it very flexible to process the logs however you want on the host. But also if your application uses a log file somewhere inside the container, I can start another container (often called "sidecar") with my tools that can have access to that file and pipe it into my logging infrastructure.

If I want multiple containers can share the same network namespace. So I listen on "localhost:8080" in one container, and connect to "localhost:8080" in another, and that just works without any overhead. I can share socket files just the same.

I can run one (privileged) container on each host that starts more containers and bootstraps f.e. a whole kubernetes cluster with many more components.

You can save yourself much "infrastructure" stuff with containers, because the host provides them or they are done conceptually different. For example ntp, ssh, cron, syslog, monitoring, configuration management, security updates, dhcp/dns, network access to internal or external services like package repositories.

My main point is that by embracing what containers are and using that to your advantage, you gain much more than by just viewing them as lightweight virtualisation with lower overhead and a nicer image distribution.

Edit: I want to add that not all of that is necessarily exclusive to containers or mandatory. For example throwing away the whole VM and booting a new one for rolling updates is done a lot, but with containers it became a very integral and universally accepted standard workflow and way of thinking, and you will get looked at funny if you DON'T do it that way.

</comment><comment><USER19> The meme image ("Can't have 0days or bugs... if I don't write any code") is incorrect.

You can't have bugs if you don't _have_ any code, but not _writing_ code just means that your bugs are guaranteed to be someone else's bugs. Now, this may be a good thing -- other people's code has probably been reviewed more closely than yours, for one thing -- but using other people's code doesn't make you invulnerable, and other people's code often doesn't necessarily match your precise requirements.

If you have a choice between writing 10 lines of code or reusing 100,000 lines of someone else's code, unless you're a truly _awful_ coder you'll end up with fewer bugs if you take the "10 lines of code" option.

<comment><USER20> > If you have a choice between writing 10 lines of code or reusing 100,000 > lines of someone else's code, unless you're a truly awful coder you'll end > up with fewer bugs if you take the "10 lines of code" option.

I disagree, this is only true if you understand why the other code has 100k lines [Although this example is a bit extreme].

A good example that could send a junior developer astray is date handling. Or most likely date mishandling if they are coding it themselves.

<comment><USER19> Sure. I'm talking about the case where the 100k lines of code provides a large set of features you're not intending to use.

</comment></comment></comment><comment><USER21> Nobody mentioned unikernels yet? It's a bit unrelated to the containers discussion in this thread, but I thought I'd mention it anyway. They let you create an operating system image, which only includes the code you need. Nothing more, nothing less. This improves security, because the attack surface is reduced.

It makes a lot of sense too me when I think about how cloud computing works. Most of the time an operating system container, zone, jail, VM... is booted just to run a select number of processes. There is absolutely no need for a general purpose system. I think unikernels could really shine in this area.

MirageOS is a project that lets you create unikernels. It's written in OCaml, so it's interesting in more than one way. MirageOS images mostly run on Xen, by the way.

[1] [https://en.wikipedia.org/wiki/Unikernel](https://en.wikipedia.org/wiki/Unikernel)

[2] [https://mirage.io/](https://mirage.io/)

<comment><USER22> So first step is containers as a service. And then this container can run as a unikernel as an implementation detail. Did I understand it correctly?

<comment><USER23> I asked a similar question yesterday. [0] The problem is that containers share the kernel of the hostOS, so you cannot host a unikernel without some kind of hardware virtualization, since the unikernel is obviously a different kernel from that of the host OS. However you can run qemu inside docker if you want to inherit its sandboxing and namespace configurations. The problem comes when you have to isolate resources, like network devices, at both the namespace level on the host OS, and the virtualization level inside qemu.

Intel's Clear Container project tries to solve this problem, but it's still limited by some virtualization overhead because qemu requires a tap device, which then connects to eth0 in a netns, which is one half of a veth pair with the host. So you end up creating 3 or 4 virtual Ethernet links just to route packets down to the guest.

[0] [https://news.ycombinator.com/item?id=13976125](https://news.ycombinator.com/item?id=13976125)

<comment><USER24> Yeah, it seems to me that there's a sort of duality between a server OS + containers vs. a hypervisor OS + unikernels. Their both attempting to minimize the overhead of process isolation and deployment flexibility.

</comment></comment></comment></comment><comment><USER25> It's a sad reflection of a technical community when 3 years later many do not seem to still clearly understand the bare basics of how containers work. HN has been complicit in massively hyping containers without a corresponding understanding of how containers work outside the context of docker.

How many container users understand namespaces and how easy it is to launch a process in its own namespace, both as root and non root users? Or know overlay file systems and how they work. Or linux basics like bind mounts, and networking.

The docker team leveraged LXC to grow from its tooling to container images but didn't shy from rubbishing it and misleading users on what it is. LXC was presented as 'some low level kernel layer' when it has always been a front end manager for containers like Docker, the only difference is LXC launches a process manager in the container and Docker doesn't. Just clearly articulating this in the beginning would have led to a much better understanding of containers and Docker itself among users and the wider community.

How many docker users know the authors of aufs and overlayfs? The hype is so intense around the front end tools that few know or care to know the underlying tools. This has led to a complete lack of understanding of how things work and an unhealthy ecosystem as critical back end tools do not get funding and recognition, with the focus solely on front ends as they 'wrap' projects, make things more complex and build walls to justify their value. Launch 5000 nodes and 500000 containers. How many users need this?

And this complexity has a huge cost and technical debt, when you are scaling as many stories here itself report and when you are trying to figure out the ecosystem so much so that its now at risk of putting people off containers.

A stateless PAAS has never been the general use case, its a single use case pushed as a generic solution because that's Docker's origin as a PAAS provider. The whole problem with scaling for the vast majority is managing state. Running stateless containers or instances does not even begin to solve that in any remote way. Yes, it sounds good to launch 5000 stateless instances but how is it useful? Without state scaling has never been a problem. A few bash scripts which is what Dockerfiles are will do it. But now because of hype around Docker and Kubernetes users must deal with needless complexity around basic process management, networking, storage and re-architect their stack to make it stateless, without any tools to manage state. Congratulations on becoming a PAAS provider.

</comment><comment><USER26> A couple of observations from someone not-so-familiar with containers:

If the consensus is that containers for the most part are just a way to ship and manage packages along with their dependencies to ease library and host OS dependencies, I'm missing a discussion about container runtimes themselves being a dependency. For example, Docker has a quarterly release cadence I believe. So when your goal was to become independent of OS and library versions, you're now dependent on Docker versions, aren't you? If your goal as IT manager is to reduce long-term maintainance cost and have the result of an internally developed project run on Docker without having to do a deep dive into the project long after the project has been completed, then you may find yourself still not being able to run older Docker images because the host OS/kernel and Docker has evolved since the project was completed. If that's the case, the dependency isolation that Docker provides might prove insufficient for this use case.

Another point: if your goal is to leverage the Docker ecosystem to ultimately save ops costs, managing Docker image landscapes with eg. kubernetes (or to a lesser degree Mesos) might prove extremely costly after all since these setups can turn out to be extremely complex, and absolutely require expert knowledge in container tech across your ops staff, and are also evolving quickly at the same time.

Another problem and weak point of Docker might be identity management for internally used apps; eg. containers don't isolate Unix/Linux user/group IDs and permissions, but take away resolution mechanisms like (in the simplest case) /etc/password and /etc/group or PAM/LDAP. Hence you routinely need complex replacements for it, adding to the previous point.

</comment><comment><USER27> As a sysadmin I just want to point out to this mostly dev crowd, that my current favorite method of operations is to have multiple compartmentalized VM's which then may or may not hold containers or jails.

Why do I do it this way? Because having a full stack VM for each use-case on a good server is realistically not that much more resource hungry than a container, but the benefits are noticeable.

Lots of the core reason stems from security concerns. For example, there are quite a few Microsoft Small Business Server styled linux attempts at hitting the business space, but instead of playing to the strengths of modern hardware, they all mostly throw every service on the same OS just like SBS does... which is a major weakness. So instead of an AD server that also does dns and dhcp and the list goes on, each thing in my environments get it's own seperate VM (eg, SAMBA4 by itself, bind by itself, isc-kea by itself, and so on)

Another reason for this is log parsing related. It's much easier to know that when the bind VM OSSEC logs go full alert, I know exactly what to fix. On multi container systems, a single failure or comprimise can end up affecting many containerizations and convoluting the problem/solution process.

Of course, the main weakness to such a system is any attempt to break out of the VM space illicitly could comprimise many systems, but that's why you harden the VM's and have good logging in the first place, but also do it to the host system, along with using distributed seperation of hosts and good backups.

Just some real world usage from a sysadmin I wanted to convey. I still will do a container or a VM with many containers for the devs if needed, but when it comes time to deploy to prod, I tend to use a full stack VM. I'm also open to talk about weaknesses in this system, as I'd be curious to hear what devs think.

To be fair, I still haven't fully caught up with the whole devops movement either, so perhaps I'm behind.

Also, a big shoutout to proxmox for a virtual environment system, FOSS and production quality since 4.0. I have also run BSD systems with jails in a similar way. The key pont of the article is that zones/jails/vms are top level isolations and containers are not (but that doesn't make containers bad!)

<comment><USER28> Have you considered a solution like graylog for centralized logging with both the VMs and Containers? Thats what my company tends to do with clients and we frequently choose containers and multiple VMs together running something like docker stack. As long as you do proper security practices on the container itself, such as not running the application as root, I dont see any downside to running things in containers over VMs as long as you have proper logging. Combined with something like ansible or another config management you can automate this whole process and it works really well for us.

<comment><USER27> Yeah I use graylog and elk stack and ansible already, there is still an isolation level with full VM's you don't get with containers. (namespace sharing issues, kernel-level system call attack issues, etc). Of course VM's have the same issues with the host OS, but there is a single layer more for an attacker to penetrate, and as we know, security is all about layers.

On that note, a FOSS ansible tower alternative popped up on my radar recently that looks interesting.

[https://github.com/ansible-semaphore/semaphore](https://github.com/ansible- semaphore/semaphore)

<comment><USER28> Thanks for sharing! Currently only using jenkins for our ansible deployments but this looks like a great WIP for an eventual replacement.

</comment></comment></comment></comment><comment><USER29> A little bit off topic but I've been following Jess for a while and I think that developers like her are great. In my country is hard to see a happy developer and she seems to enjoy everything she does. That's why I follow her, because of her great work and great personality. I'm happy to see one of her blog posts here in HN

</comment><comment><USER30> In this post the author links to one of her previous posts[0], where she wrote:

> As a proof of concept of unprivileged containers without cgroups I made > binctr. Which spawned a mailing list thread for implementing this in > runc/libcontainer. Aleksa Sarai has started on a few patches and this might > actually be a reality pretty soon!

Does anybody know if this made it into runc/libcontainer? I'm not an expert on these technologies but would love to read through docs if it has been implemented.

[0] [https://blog.jessfraz.com/post/getting-towards-real- sandbox-...](https://blog.jessfraz.com/post/getting-towards-real-sandbox- containers/)

</comment><comment><USER31> I had to read the post twice before I really got what she was saying. I think the distinction I would make is that while there are many more use cases that you can apply to Containers that may not apply to Jails, Zones, or VMs the most common use case of "run an app inside a pre-built environment" applies to all of them. Since I believe most users (or potential users) of Containers are only looking at that use case, it's harder to see the differences between the different technologies.

My only hope is that anyone in a position of making a decision on which technology to use can at least explain at a high level the difference between a Container and a VM.

</comment><comment><USER32> I'm not an OS person, so forgive me if this is a stupid question: Lots of people are excited about Intel SGX and similar things. Are there any interesting ways people are thinking about combining, like, Docker containers with SGX enclaves and such? One could imagine (e.g.) using remote attestation to verify an entire container image.

<comment><USER33> Yeah, it's called SCONE: [https://www.usenix.org/conference/osdi16/technical- sessions/...](https://www.usenix.org/conference/osdi16/technical- sessions/presentation/arnautov) It's pretty kludgey due to SGX limitations like not supporting fork().

</comment></comment><comment><USER34> It doesn't matter how many distinction you make on these things (first-class, last-class, second-class, poor-class, etc...). These kind of discussions are always relative.

All is good as long as your decision is conscious of the compromises taken by each approach and what they entail (what other security mechanisms do you have at your disposal ? how could they enhance your app ? will your solution depend on external tools like ansible/puppet/etc ? do you actually need "containers" or jails or [insert your favorite trendy tech here] ?).

Running a *BSD or a Linux is a way bigger design decision than what kind of isolation mechanisms you have as many of the underlying parts are becoming different.

</comment><comment><USER35> As a novice, this was a great informative read. More posts like this on the Internet, please!

</comment><comment><USER36> I'm trying to understand something. At my last work we had a big problem with "works for me". We started using Vagrant and all those problems disappeared. Then Docker became popular and all of a sudden people wanted to use that instead.

But is Docker really suitable for this? While each Vagrant instance is exactly the same Docker runs on the host system. It feels like it will be prone to all sorts of dissimilarities.

<comment><USER37> It depends on what you want to keep constant between hosts; for many (dare I say most?) projects, docker will provide a sufficiently identical environment that it "just works", because the filesystem as seen by the same image on two different computers will be exactly the same. This is sufficient for many, many projects, and is the primary source of "works for me" problems, in my experience.

However, if your application requires things like the CPU architecture set is the same, or the amount of memory available to the process is the same, etc... then no, docker will not give you this level of "sameness". I have had bugs that manifested on one docker host and not another because one host had certain x86 extensions available (AVX, or something similar) and the other didn't, and this caused a certain codepath to be followed. However this is probably extremely rare for non-performance-targeted code, as most projects will compile for a conservative subset of x86_64 instructions so that they can run everywhere.

Anecdotally, I find learning the "Docker way" has transformed my development paradigm in a much more fundamental way than Vagrant ever did, mostly because docker containers are almost instantaneous to start, which enables their use for a whole range of applications that I would never have even considered for something like Vagrant. Because I can launch a docker container in ~300ms, I can use containers like native applications. I do everything from building software to compiling latex documents using docker just so that I don't have to worry about installing toolchains or configuring things just right; I get an image that does what I want, and then I invoke the docker containers as if I were invoking the actual tools themselves.

Docker is just a tool, but it's a darned powerful one, and it's pretty fun to see how it continues to evolve right now. I highly suggest you check it out.

</comment></comment><comment><USER38> SmartOS run containers in zones get the best of both worlds

<comment><USER39> I always run my containers in a jail which I run in a zone which is running in a VM just in case.

<comment><USER38> I appreciate the humor but having an ability to run containers in a zone gives you very good isolation without overhead of a VM.

</comment></comment></comment><comment><USER40> "container is not a real thing". But what could we say about real things within software field?

</comment><comment><USER41> Also see [http://dtrace.org/blogs/brendan/2013/01/11/virtualization- pe...](http://dtrace.org/blogs/brendan/2013/01/11/virtualization-performance- zones-kvm-xen/)

</comment><comment><USER42> We wrote a paper comparing containers and VMs for the middleware conference: [http://people.cs.umass.edu/~prateeks/papers/a1-sharma.pdf](http://people.cs.umass.edu/~prateeks/papers/a1-sharma.pdf)

</comment><comment><USER43> I don't understand why anyone would say to give up containers and just use Zones or VMs? Containers are solving a very real problem. The problem is that containers weren't as well marketed before Docker (or as user friendly).

<comment><USER44> Many people's use case for containers is more or less identical to that of Zones/VMs -- they aren't sharing anything across containers.

<comment><USER45> The majority of use cases for containers seem to be as more of a deployment and/or scheduling solution and less of an overlap with the use cases for virtualization.

</comment></comment></comment><comment><USER46> I have been learning about containers fairly recently. What are these security vulnerabilities that the post talks about? I haven't come across any docs that mention security yet.

<comment><USER47> It's mainly an attack surface issue. A process running in a Linux container is just a process running in a regular OS with some extra bells and whistles for resource constraints and isolation. So when it comes to making a kernel call, it's a call directly to the same kernel all of the other containers are calling.

This means the entire kernel/userspace API is the attack surface for a malicious container. Compare that to a VM where the attack surface is the API the hypervisor exposes to a virtual machine.

It's not that the former is necessarily smaller, it's just that the modus operandi in systems administration has always been that if a person executes malicious code as a user on the OS, you better wipe the system because kernel vulnerabilities aren't treated with the same severity of hypervisor vulnerabilities.

This is the reason your containers are actually executed in a dedicated VM if you use something like GCE.

</comment></comment><comment><USER48> Is it possible to design a process that isn't a self-contained OS instance depending on a lot of horseshit overlay controls to perform a task in a well designed way that still allows privilege separation from the host OS in a manageable way?

Of course not. It's the basis for all this other shite.

</comment><comment><USER49> I think jess is on the money here. The complexity in linux containers vs zones show up in two ways:

1) the linux kernel container primitives are implemented in ways that are more complicated. for example in zones pid separation is implemented by just checking the zone_id and if the zone_id is different then processes can't access each other. this also means that in zones pids are unique and you can't have two processes from two different zones with the same pid [with the exception: i believe they may have hacked something in to handle pid1 on linux].

similarly, in zones there is no user mapping if you are root inside the zone you are also root outside of the zone. the files you create inside a zone are uid: 0 and also uid: 0 outside the zone.

if you look at how device permission is handled in linux we have cgroups that controls what devices can be accessed and created. while in solaris zones they use the existing Role Based Access Control and device visibility. so inside a zone you can either have permission to create all devices (very bad for security) or create no devices. In zones access to devices is mediated by whatever devices the administrator has created in your zone.

in zones there is no mount namespace instead there is something that is very similar to chroot. it is just a vnode in your proc struct where you are restricted from going above. zones have mostly been implemented by just adding a zone_id to the process struct and leveraging features in solaris that already existed [i guess the big exception would be the network virtualization in solaris] while in linux there are all these complicated namespace things.

this complexity means there are probably going to be more bugs in the linux kernel implementation. however, because you don't have as much fine grain control this can also create security bugs in your zone deployment. for example i found an issue in joyent's version of docker where you could trick the global zone into creating device files in your zone and these could be used to compromise the system. under a default lxc container this would not be possible because cgroups would prevent you from accessing the device even if you could trick someone else into creating it. you also have to be careful in zones with child zones getting access to files inside the parent zone. if you ever leak a filesystem fd or hard link into the child zone from the parent zone then all bets are off because the child is able to write into the parent zone as root. (i believe this situation was covered in a zone paper where they describe the risk of a non-privileged user in the global zone collaborating with a root user in a child zone to escalate privileges on the system)

2) because all the pieces are separate in linux then something has to put it together and make sure all the pieces are put together correctly. like i wouldn't trust sysadmins to do this on their own and luckily there are projects like lxc/lxd/docker etc that assemble these pieces in a secure way.

<comment><USER2> Yep, exactly. Since zones were inspired by jails, FreeBSD works in pretty much the same way. For device permissions though, we have rules support directly in devfs. [https://github.com/freebsd/freebsd/blob/master/etc/defaults/...](https://github.com/freebsd/freebsd/blob/master/etc/defaults/devfs.rules) — there's a default ruleset for jails that makes sense (allows log, null, zero, crypto, random, stdin/out/err, fd and tty stuff).

By the way, I couldn't find this anywhere on the internet — is there a simple way to just run something in a zone on illumos, without the installer stuff? Like on FreeBSD you can just do this:

jail -c path=/my/chroot/path command=/bin/sh

and you have a jailed shell. What's the Solaris/illumos equivalent of this?

</comment></comment><comment><USER50> I found this article to be humorous and informative.

</comment><comment><USER51> In earlier versions of ProxMox the openvz vms were called containers and the KVM vms were called vms. So it is pretty confusing overall.

For myself I would point out that Zones, Jails, OpenVZ and LXC , even KVM, all pretend that they are fully separate from the host node OS.

While Docker et al do not pretend this; in fact if you are running Apache on your host system and try to run a Dockerized web server on port 80 the Docker container might refuse to start. The other methods mentioned, can't even determine what they are running under.

<comment><USER52> > pretend that they are fully separate from the host node OS

No! You can see all processes of the OpenVZ and jails from the host system

>in fact if you are running Apache on your host system and try to run a Dockerized web server on port 80 the Docker container might refuse to start

depends on network options.

<comment><USER51> Sorry for not being clear ... the OpenVZ, LXC, Zones (not sure about Jails) all have their own startup options, and if they don't start "init" as the first process inside, then they have a method that fakes it. You can install random new packages of software and run them, from inside the "VM" \- from what I know, you can't do this from within a container.

FWIW using KVM all you can see from the host node (unless you use debugging tools) is the PID and related process info of the KVM instance (one process per VM you have started).

<comment><USER53> You can easily run top inside Xen VM via xl console. Or via SSH. Manage it with a stack like salt or ansible.

This is a super weak argument.

<comment><USER51> Hm, that isn't what I am trying to say at all. In drawing a distinction, some of the virtualization tech acts like a full blown OS in and of itself, while others do not. x

</comment></comment></comment></comment></comment><comment><USER54> "PERIOD."

This is almost always the harbinger of lies, deception, propaganda, or lack of nuance.

</comment><comment><USER55> before i even heard of containers i found out about jails and wondered if you could serve each users jail using an nginx config file in their home directory.

</comment><comment><USER55> "Lego" it's "Lego"

</comment><comment><USER56> SHOUTING IN TEXT DOESN'T HELP

<comment><USER57> emphasis added

</comment></comment><comment><USER52> Some point are just wrong. Containers and jails have many design similarities which were dismissed by author. Notably PIDs, both containers and jails are nearly identical with regard, you can kind of have one leg here another there, although that harder to achieve with FreeBSD jails; both implementations do not hide PIDs from the host systems. Networking - jails can run on top of non- virtualazed IP/net dev, containers can run in such modes as well. Link is someones rant without tech details.

<comment><USER58> Do you have a good overview of how to use pieces of FreeBSD jails without using the whole thing? I've interacted (under duress) with FreeBSD jails in production, but I definitely found it a lot easier to learn about Linux containers / namespaces / cgroups.

<comment><USER2> sudo jail -c path=/ command=/bin/sh

to get a shell in the least isolated jail possible. It's that simple. Read the "Jail Parameters" section in `man jail` to see what you can add to this, e.g.:

sudo jail -c path=/ ip4.addr=192.168.1.123 command=/bin/sh

to isolate the IP address…

</comment></comment></comment><comment><USER59> Where does Docker fit into all of this? Asking for a friend.

</comment><comment><USER60> > Legos

"LEGO is always an adjective. So LEGO bricks, LEGO elements, LEGO sets, etc. Never, ever "legos."" [1]

The other one that gets me is "math". I know it's not really plural, but "mathematics" has an a on the end, so it's "maths!"! Or do Americans say "stat" for "statistics" as well?

[https://twitter.com/lego_group/status/842115345280294912](https://twitter.com/lego_group/status/842115345280294912)

<comment><USER61> If you have two red lego bricks and two yellow lego bricks, do you not have two reds and two yellows? You definitely have A couple of bricks of each colour.

</comment></comment><comment><USER62> I think that the important feature of containers is that they are running from image containing the application. Zones, jails, and VMs are other isolation mechanisms that could be used to run containers. Running an application image unpacked into VM would be container.

One place where the difference in namespace is visible is with Kubernetes pods. Containers running in pod share network and volumes namespaces.

<comment><USER13> Your definitions are not the common ones. Dropping the Linux from the full name Linux Containers doesn't make it a different fundamental. Containers, Zones and Jails are all OS layer virtualisation :[https://en.m.wikipedia.org/wiki/Operating-system- level_virtu...](https://en.m.wikipedia.org/wiki/Operating-system- level_virtualization) . The style of application packaging that came with Docker compliments OS virt but the prior art demonstrates that it is its own innovation. "Docker will do to apt what apt did to tar" @bcantril

<comment><USER62> There are Windows Server containers. Docker runs on FreeBSD using jails and Solaris using zones. Hyper runs containers on hypervisors and RancherVM on KVM.

Container now has two meanings: the isolated application, and the container- based virtualization on Linux. The latter is the original usage (LXC is older than Docker).

But the important innovation of Docker is combining containers with image distribution. The isolated application is more general then virtual machine so the concept is extended to other virtualization techniques.

</comment></comment></comment><comment><USER63> Great rant about something way over my head by someone who knows way more than me about it!

If I can transfer to a domain I understand better (front-end dev): It sounds like VMs, Jails, and Zones are like Ember.js: it comes with everything built in and is simple if you stay within the design.

Containers are more like React: it gives you the pieces to build it yourself, and building it all yourself can lead to complexity, bugs, and performance issues.

Disclosure: I have no idea what I'm talking about

<comment><USER17> Thanks for sharing what people see here that usually don't work in the backend.</comment></comment>


Ask YC:  I would appreciate feedback on my "Friend News Game" Facebook App (a twist on reddit) - <AUTHOR> http://www.facebook.com/apps/application.php?id=8405926627

<comment><AUTHOR> It's sort of like reddit except that you only get points from the ratings your friends give to links that you send them.

So one would expect better personalized recommendations. Moreover, there is an incentive to get your friends to add the app, thus making it potentially viral.

URLs accumulate points from ratings as well and there are global (i.e., non- personalized) hot/top/new rankings.

</comment><comment><USER2> Good concept. I find myself still sending things directly to other people that I find on digg/reddit/delicious. This has potential.

</comment><comment><USER3> nice idea</comment>


Scroll Back: The Theory and Practice of Cameras in Side-Scrollers - <AUTHOR> https://docs.google.com/document/d/1iNSQIyNpVGHeak6isbP6AHdHD50gs8MNXF1GCf08efg/pub <comment><USER2> This is so much more interesting than it should be. The GIFs must have been a ton of work to put together. Fantastic.

<comment><USER3>
        > This is so much more interesting than it should be.

Is this a sibling expression to "I laughed harder at this than I should have"? It's okay to find things funny and interesting, even superlatively so.

Like zombo.com says, the only limit is yourself.

<comment> rkuykendall-com They probably mean "This is so much more interesting than somebody would expect based on a description."

If someone asked me "Do you want to watch a 52 minute talk on how they move the window on side-scrollers?" I would say no. That doesn't sound like it would even be worth talking about.

But it is, and I'm really psyched to watch this video.

</comment></comment></comment><comment><USER4> If you prefer, here's the 52 min video of the talk: [http://www.gdcvault.com/play/1022243/Scroll-Back-The- Theory-...](http://www.gdcvault.com/play/1022243/Scroll-Back-The-Theory-and)

</comment><comment><USER5> Great article. (Unfortunately I had to finish it in w3m because the gifs were so huge.)

I remember doing lerp and projected-focus before I knew what they were called. It definitely has more depth than it seems, and as I was reading I was surprised at how much thought went into some of these decisions, especially Super Mario World where I had never noticed it. I suppose it's like UX design -- you only notice it if it's bad.

<comment><USER6> How does w3m compare to lynx?

<comment><USER5> I use it mainly because it defaults to http; I never took the time to actually compare the two.

<comment><USER6> What's good in defaulting to http?

<comment><USER5> I suppose it wasn't as bad as I thought. If you type

        lynx news.ycombinator.com

it tries to "intelligently" choose the news:// protocol, which I have never used and probably never will. With w3m, it just goes to http no matter what the subdomain is.

</comment></comment></comment></comment></comment><comment><USER7> I wish every game developer read this. Some really popular games have bad camera movement. For example, Limbo has poor vertical camera and you have to make "leap of faith" often. Trine has problem with scene transition with multiple players so you often play off-screen. And Vlambeer games have too much shaking ;)

<comment><USER8> Sreenshake can be turned off, but they have a problem with their offscreen action. That is, enemies that are 2 screens away from you can shoot you mercilessly which is even worse than a leap of faith because it's purely random and can't be prevented with skill or knowledge.

<comment><USER7> Yes. I've had such moments in Nuclear Throne very often.

</comment></comment></comment><comment><USER9> It would be neat to have an interactive demo where you could explore the space of possible cameras.

</comment><comment><USER10> Platform snapping... That's a really good idea that I've never noticed. I'm going to experiment with this.

</comment><comment><USER11> Awesome work!

Unfortunately, some game engines just give you the Camera and basic stuff, and you must implement scrolling yourself while wondering, "Am I doing this right?" (mostly I use LibGDX). Would be great to see this codified as a library. I think it's not game engine dependent.

</comment><comment><USER12> Great essay. Content is interesting, and the composition style is innovative.

</comment><comment><USER13> Has this been seen by [http://www.reddit.com/r/gamedev/](http://www.reddit.com/r/gamedev/) ? If not, it definitely should be :)

<comment><USER14> Strangely, it didn't do that well on /r/gamedev - I get the feeling that this talk, coupled with the article ([https://docs.google.com/document/d/1iNSQIyNpVGHeak6isbP6AHdH...](https://docs.google.com/document/d/1iNSQIyNpVGHeak6isbP6AHdHD50gs8MNXF1GCf08efg/pub?embedded=true)) this will become the "definitive" article on 2D camera boxes. Great work!

</comment></comment><comment><USER15> I took a game programming class back in college and ended up coming with a scheme very similar to the lerp smoothed box shown there. Interesting to see all the history of this.

</comment><comment><USER16> The vision and neurology section at the start was all messed up, but the review of approaches used in different games was great! Oh nostalgia :)

</comment><comment><USER17> Abuse (1996) had some kind of (inverted) target focus I think. Or is that a snappy lerp-smooth?

</comment><comment><USER18> Thank you so much for this!

</comment><comment><USER19> awesome work!

</comment><comment><USER20> This is so phenomenal. Just being aware of the different ways camera systems are used is incredible when playing and making games.</comment>


Nunes moved farm to Iowa to exploit migrant labor - <AUTHOR> https://www.esquire.com/news-politics/a23471864/devin-nunes-family-farm-iowa-california/ <comment><USER2> This post has had its title substantively changed from the original. Is there a way to change it to the original title "Devin Nunes's family farm is hiding a politically explosive secret"?

<comment><AUTHOR> Isn't the title I used for submission more objective and less sensational? I don't understand your objection to it.</comment></comment>