It Takes More Than Mass Protests to Drive Change - <AUTHOR> https://www.gsb.stanford.edu/insights/it-takes-more-mass-protests-drive-change <comment><USER2> >"With the rise of the internet," Soule says, "modern movements can mobilize constituents through their websites and social media. If your end goal is to get 500,000 people to turn up on the Mall in Washington, D.C., Twitter is great at that. Facebook is great at that. But if your goal is to actually make lasting change in the system, you have to work within the system — to essentially get a seat at the table."

>"Occupy Wall Street," she continues, "mobilized a lot of people and got a key point across about income inequality in the U.S. But did it result in real change? Unfortunately, no. Had there been a nice set of organizations able to get access to the political and economic system that the movement criticized, the changes may have been more enduring."

The idea that OWS could have worked within the system to get the changes they wanted seems misleading to me.

<comment><USER3> There's no way inside the system because it's the system OWS wanted to change (And Extinction Rebellion wants nowadays).

There's no nonviolent way outside the system because the police are violent if you actually inconvenience the system (which marches don't do but occupations do).

The problem basically is the fetishization of nonviolence.

Revolution can change the system.

<comment><USER4> Extinction Rebellion is strictly non-violent and they seem to be very successful afaict. Or do we different definitions of "nonviolence"?

Also, what do you think about "The success of nonviolent civil resistance" [https://www.youtube.com/watch?v=YJSehRlU34w](https://www.youtube.com/watch?v=YJSehRlU34w) ?

<comment><USER5> We certainly seem to have different definitions of violence, yes.

If 'violence' has been re-defined to the extent that mis-gendering someone is an 'act of violence' ([https://everydayfeminism.com/2017/01/misgendering- trans-peop...](https://everydayfeminism.com/2017/01/misgendering-trans-people- is-violence/)), then you could certainly argue that ER have acted violently.

 _EDIT_ I see I'm being downvoted. Are we not allowed to have a conversation about re-defining words and being intellectually consistent with the re- defined meaning?

<comment><USER6> If you want to make that argument, you have to spell it out and engage seriously with what people mean. At the moment it's not even clear whether you're in favour of or against mis-gendering trans people, let alone what argument you're making about violence.

<comment><USER5> OK, I'll try to be more coherent.

It seems these days that the word 'violence' has been re-defined such that it no longer requires there to be any abusive physical contact between the perpetrator and the victim. As such, merely using words that may cause an individual some emotional discomfort is considered an act of violence.

I don't agree with that definition, and for me abusive physical contact is the defining characteristic of violence. But if that's the general consensus now then that's what I'll use. We can't have a debate if we're all speaking a different language.

Given that 'violence' no longer requires any physical contact, one could make the argument that some of the activities that ER have engaged in could be considered violent. The many ER road-blocks have very real consequences for all the people who are unable to go about their daily lives without being disrupted. When ER blocks a bus full of people in traffic, it's distinctly possible that those people were on their way to job interviews, doctors appointments, meetings with long-lost relatives, charity fundrasiers or any number of other things. This kind of disruption might easily cause significant emotional distress, I would argue at least on par with the emotional distresss suffered by an individual being mis-gendered. So why should one be considered violence and the other not?

That's not even getting to the topic of Ambulances being caught up in ER road- blocks, which has been documented numerous times. ER even address that issue explicitly in their FAQ ([https://rebellion.earth/the- truth/faqs/](https://rebellion.earth/the-truth/faqs/)). They attempt to hand- wave this problem away by saying that ambulances can be re-directed away from the road-blocks. It hardly needs to be said that time is often a critical factor when an ambulance is transporting a patient to hospital. The additional time spent avoiding ER activists could easily be the difference between life and death. Is this not a form of violence?

<comment><USER7> > Is this not a form of violence?

By your own definition, the answer is clearly no.

<comment><USER5> Do you need to re-read my post? The entire point is that there are now multiple definitions of 'violence'. We mustn't pick and choose definitions to fit a particular narrative.

One definition of violence appears to be used when describing ER as 'non- violent'. But another, very different definition of violence appears to be used when declaring that mis-gendering someone is an act of violence.

<comment><USER7> Sorry for the very late reply. You're right, I must have skipped the rest of your comment.

My answer to this is to insist on the 'proper' definition - the one you yourself prefer - and to push back whenever the term is misused (I find this is generally done as a smearing tactic).

The alternative is to yield ground and to start saying 'physical violence' whenever one means 'violence'.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> A related point made by Zeynep Tufecki is that mass protests in the past used to be show of force. If you could get half million people to travel to Washington and participate in a mass protest, that meant these people cared enough to take time off from work for a few days, spend money on travel and participate in a policitically and socially risky activity. It also meant you had several tens of thousands of organizers in diverse parts of country who managed the logistics of getting so many people to come together for the protest. The implication was clear -- give us what we want, if you don't, we definitely have the votes to take you down.

In the age of Twitter and Facebook, protests are a lot more convenient and a lot less difficult to organize. It is not clear that half a million people protesting in NYC or DC means much in terms of actual electoral consequences. Whether they can articulate it or not, politicians definitely know this and react accordingly.

<comment><USER6> I think after OWS, and before that the various "G20" etc anti-globalization protests, it became very clear that the police could effectively contain and suppress mass protest and that it would not be allowed to change anything.

And with better gerrymandering and voter targeting, the votes of protestors could be made irrelevant as well.

(There is a history that needs to be written of how "anti-globalization" moved from a left position to a right one, along the way changing from focusing on goods to focusing on immigrants)

<comment><USER8> I think we disagree on this. I interpret the failure of OWS et al. as the inability of the protestors to take their ideas mainstream. While gerrymandering, voter targeting and propaganda via mass media do dilute the protesters' votes, change is clearly possible. We saw this with the Arab Spring where the protesters were up against a much more authoritarian and anti-democratic state.

If I had to guess, I'd say the problem facing the American left is complacency. A lot of people still think things are going reasonably well and simply do not empathize with the concerns of OWS or the sunrise movement or the BLM movement. It's hard to move the needle politically if a significant minority simply do not believe that there's a crisis worth reacting to.

</comment></comment></comment><comment><USER9> > If your goal is to get 500,000 people to turn up on the Mall in Washington, > D.C., Twitter is great at that. But if your goal is to make lasting change, > you have to work within the system.

That's partially true, but it is still more false than true. Stand this statement on its head:

> If your goal is to tweak the current state of affairs with minor reforms, > you should probably seek a seat at the table. But if your goal is to make > fundamental change, you have to work outside the system.

That is to say, the author assumes that power relations in society are static, and therefore one can either shout at established power from outside or join it - never deconstruct, topple or replace it.

<comment><USER10> Yeah the whole thing has a "Let them eat cake" sort of vibe.

Oh, just get a seat at the table WHY DIDN'T WE THINK OF THAT

<comment><USER11> A lot of people didn't think or that though. So many people would prefer to yell from the sidelines evey now and then about how politicians are all the same and don't reflect the will of the people. How many of them have actually joined a political party p ran for office and even _tried_ to get a seat at the table.

<comment><USER12> "One of the penalties for refusing to participate in politics is that you end up being governed by your inferiors." — Plato

<comment><USER13> Didn't at all sound like Plato so I checked. I guess it's this (my italics) :

"Well, then," said I, "that is why the good are not willing to rule either for the sake of money or of honor. They do not wish to collect pay openly for their service of rule and be styled hirelings nor to take it by stealth from their office and be called thieves, nor yet for the sake of honor, for they are not covetous of honor. So there must be imposed some compulsion and penalty to constrain them to rule if they are to consent to hold office. That is perhaps why to seek office oneself and not await compulsion is thought disgraceful. _But the chief penalty is to be governed by someone worse if a man will not himself hold office and rule._ It is from fear of this, as it appears to me, that the better sort hold office when they do, and then they go to it not in the expectation of enjoyment nor as to a good thing, but as to a necessary evil and because they are unable to turn it over to better men than themselves or to their like. - Republic, I 347.

[http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%...](http://www.perseus.tufts.edu/hopper/text?doc=Perseus%3Atext%3A1999.01.0168%3Abook%3D1%3Asection%3D347b)

<comment><USER12> Given that Plato didn't originally write in English, it seems fair to say that's a "more or less within the margin of error" quote of his?

</comment></comment></comment></comment></comment></comment><comment><USER14> Protesting is about bring awareness and education to the public over topics the media was not airing. For example, the media was not talking about income inequality before OWS protest started happening. OWS shifted national discourse for a lot of people. That means it was a very successful protest.

So yes, real change has to happen somewhere else. Newly educated people could drive that change, new alliances could be formed, or next steps like boycotts could be organized. Those things could have started at a protest, which is another reason why protesting is valuable.

<comment><USER15> It's funny how one of the aims of a protest is about getting their story in the media because the media is not paying attention to it. It basically means that a protest has to play by the medias's games. There's always critical commentary after a protest about the amount of coverage given to it. Coverage becomes the aim rather than the actual aim. The media for their part do cover them - they have to play their side - but they won't be on the protest's side!

Focusing on the media, marketing, communications and influence also means fake news, propaganda, Russian troll armies etc are also important. It's the medium and the message and not the aims of the movement that's most important. It over plays the media's hand and down plays how effective a protest movement could be.

</comment></comment><comment><USER16> "Seek a seat at the table."

A very undervalued tool for making lasting change. It would have been great if someone from the Occupy Wall Street movement had run for mayor of New York.

<comment><USER10> Unless your goal is to flip the table over...

</comment></comment><comment><USER4> > But if your goal is to actually make lasting change in the system, you have > to work within the system — to essentially get a seat at the table."

Did civil rights movement work within the system?

Did Suffragettes work within the system?

<comment><USER12> As compared to a revolution or doing nothing, yes the civil rights movement worked within the system.

If instead of gathering people and motivating them with eloquent and carefully constructed arguments, Martin Luther King had led violent protests, he'd have likely changed nothing. He (and others) worked within the system of laws and public behavioral mores so as to not forfeit the right to be heard.

Rosa Parks sitting in the front of bus was exactly the type of action that was needed, not burning the bus. I see that as 80+% working within the system.

The civil rights leaders earned a seat at the table. They got publicity during the Apollo moon launches. They had meetings with important leaders who held the original seats at the table. They did it by being peaceful, strategic, unrelenting, and incredibly savvy.

I can't imagine how frustrating it must have been to live as a disfavored racial minority prior to the civil rights era. To have leaders exercise the strategy and patience they did is almost unthinkably impressive to me.

<comment><USER17> Much that my naive self wants believe that nonviolence works and violence doesn't. Often its the potential for violence that the powerful even starts a token conversation with the ignored and the powerless. Its then they start preaching that non-violent means needs to be adopted and there needs to be a negotiation.

<comment><USER12> Yes, I think the peaceful civil rights leaders we think of today were benefited by the _threat /spectre_ of violence combined with overt actions and communication on their part to declare the violence not part of their plan.

<comment><USER4> I believe (hope) you guys are mistaken. For example, what kind of "potential for violence" from a starving student was King's College afraid of when they agreed to divest fossil fuels? [0]

[0] [https://londonstudent.coop/kings-college-agree-fossil- fuel-d...](https://londonstudent.coop/kings-college-agree-fossil-fuel- divestment-following-student-hunger-strike/)

<comment><USER12> When I said "peaceful civil rights leaders we think of today", I meant "ones we readily recall today based on their actions in the 1950s and 60s", not that the threat of violence is a _required_ element in the system.

I can see how that could very easily be read as "peaceful civil rights leaders of today", so the error is on me as the author, but that was not my intended usage.

</comment></comment></comment></comment></comment></comment><comment><USER18> From what I remember of Occupy Wall Street and similar topic, the headline will find a receptive audience here at HN.

BUT I won't tire of pointing out that protests sometimes have _dramatic_ effects. So much so that I'm still not sure about the exact mechanisms.

It seems so obvious that a government with all the guns should not be scared by unarmed protesters, right?

Yet there is no shortage of examples showing the opposite. In Ukraine, it was some low five-figure number of protesters on a single square in the capital that brought down the government. Across the middle east, "Arab Spring" rebellions deposed of quite a few entrenched autocrats. Hong Kong, right now, is entirely peaceful and mostly students, and has China closer to a nervous breakedown than Trump ever will.

Even Occupy Wall Street, favorite target of derisively dismissing "Hippies" and the left, has had significant impact: How often do you now see arguments framed as "The 1%" vs "The 99%". If that doesn't count as impact, your expectations are probably off.

I got to experience this first-hand when the G20 meeting happened in Hamburg, Germany: Every single interview with one of the politicians attending that I saw mentioned Climate Change. If that wasn't the result of the protests happening outside, I don't know what did it.

The Arab Spring especially has obviously not resulted in the sort of change people had hoped for. But to suggest it did not have an impact is simply wrong.</comment>


Instant Bitcoin Purchases at Coinbase - <AUTHOR> http://blog.coinbase.com/post/55203204550/instant-bitcoin-purchases-at-coinbase <comment><USER2> Someone credible is finally trying to compensate for the abysmal failure that is Bitinstant (see [http://bitinstantscam.com](http://bitinstantscam.com) and basically any Bitcoin forum). That is fantastic.

However, let's not get carried away. This is not instant for first-time users. You still have to go through verification and wait at least 4 business days for your first purchase to go through before reaching the qualifications for instant purchases. Coinbase really needs to adopt convenient, instant deposit methods such as Western Union and Moneygram in order to reach critical mass.

<comment><USER3> Bitinstant isn't any more a scam than Mt. Gox is.

They're just both services that are run by folks who weren't expecting to hit the scale they are now, have no experience building financial systems, and are waaaay in over their heads.

<comment><USER2> Bitinstant recently laid off all of their support staff and has gone quiet for nearly two weeks on support related matters. Their CEO, Charlie Shrem, used to constantly tweet several times per day. He hasn't tweeted in 9 days now and seems to have vanished ([https://twitter.com/charlieshrem](https://twitter.com/charlieshrem)). Over the holiday weekend, their $8/mo SquareSpace account used for hosting their blog was suspended for billing issues (though they have since scraped up the $8).

Bitinstant is taking money from new orders and allocating it to old orders (aka Ponzi Scheme). I suppose that means that as long as new orders continue to arrive, customers will be made whole and eventually they could earn enough in fees to cover up whatever they did that caused the mess they are in. That's if someone doesn't complain to the FBI first and get them shut down.

[https://bitcointalk.org/index.php?topic=128314.8180](https://bitcointalk.org/index.php?topic=128314.8180) (that is one of the most recent pages of a thread containing ~411 pages of complaints). The last few pages are especially disturbing.

<comment><USER4> Their bitcoin service to top up phones disappeared pretty quickly too and never came back, also their debit card service never happened. They used 'VouchX' which was run by a shady guy who disappeared after LR went pop so maybe that's why they can't afford staff.

</comment></comment></comment></comment><comment><USER5> I have an issue with Coinbase. I have an account that I opened a few months ago but didn't use. I'm trying to sign into it now, and it brings me to a screen asking for a code from "Authy" (whatever that is) or Google Authenticator.

Ok, so I grab GA again. GA wants me to setup an account using a key provided by Coinbase. Coinbase has never provided such a key to me, as I have never setup GA to work with them before.

Fine, click "Code not working" and they'll text the code to you. Why they don't do this to begin with, I have no idea. Anyway, its now been 15 minutes and no text has come through.

Exactly what is a user supposed to do when their entire login system is broken and offers no information on what to do?

Please, please, PLEASE test your systems BEFORE you go public and make big announcements.

<comment><USER6> Sorry to hear that happened.

Please send us a note on [http://support.coinbase.com/](http://support.coinbase.com/)

SMS is unfortunately unreliable in some instances. You should be able to receive a call with the verification code also as a backup.

</comment></comment><comment><USER7> To be honest, it's kind of creepy. I just identified myself, and they somehow knew what month and year I purchased my car, a list of names I may or may not have been 'associated' with, a list of counties I may or may not have lived in, and stuff like that. I'm not particularly comfortable with a company knowing that much about me, but it's already complete so... horse, meet barn door?

I wish they would at least tell me what sources they're getting this information from.

<comment><USER2> That's actually pretty simple. Equifax is the biggest provider of these types Q&A's. I'm not even sure that the company itself is exposed to the actual answers - I believe the credit bureau just responds with a pass/fail. Admittedly, it would be less creepy and more welcoming if they sent you to the Equifax website to do the verification. That way you would know that no one else actually has or will receive your personal info.

<comment><USER7> They did have a blurb that said this was all "public information", but if I pop my name and car into $search_engine (for example) nothing relevant comes up.

<comment><USER8> Search engines haven't indexed everything yet. There are lots of public records that aren't indexed by google, unless they make the news or get the attention of some blogger. This includes civil judgements, tax liens and bankruptcy to name a few.

</comment></comment></comment></comment><comment><USER9> Different strokes for different folks, but I'll never understand why someone would want to buy bitcoins with their personal info attached to the coins. It seems to defeat the purpose of a pseudonymous, decentralized currency.

<comment><USER10> What, did you expect bitcoin to suddenly create an anonymous economy out of thin air? Unless you can create value, and then charge bitcoin anonymously for that value you have to convert previous holdings into bitcoin. You can do so anonymously via cash P2P. But if you would like to convert funds from the current system, you have to 1) use the current system 2) play by their rules.

This really doesn't even hurt the anonymity of bitcoin anyway, simply buy your bitcoins, run them through a tumbler, and voila, out comes anonymity.

<comment><USER9> I've never used Bitinstant, Coinbase or any of the other services that would attach my info to btc. I've worked to establish trading trust with other Bitcoin users and have traded physical metals and electronic gear for btc.

It takes a little more effort but all good things do.

</comment></comment></comment><comment><USER11> I've wondered about starting a service to sell bitcoins via credit card but the catch is he service holds your coin for you for 6 months or whatever the period for chargebacks is.

Is that a viable idea?

<comment><USER12> Buy BTC from tocomment Inc with credit card. If in 5 1/2 months the price of Bitcoin is lower, issue a chargeback and take no loss. If the price is higher, do not issue a chargeback and take the profit.

Sounds like a bad deal for tocomment Inc.

<comment><USER13> This could be "fixed" by holding USD for 6 months and then converting it to BTC, but it's still fairly pointless.

</comment></comment></comment><comment><USER14> How can I see what level my account is at?

What are the limits for Level 1?

<comment><USER14> Ah: [https://coinbase.com/verifications](https://coinbase.com/verifications)

</comment></comment><comment><USER15> I filled out my verification details and it said failed to verify, you have 1 day to wait. Details were correct, so not sure what happened.

<comment><USER16> I was told "Insufficient Data for Questions". I guess my public record is not juicy enough.

</comment></comment><comment><USER17> Way to go Coinbase!

It's awesome to see some innovation coming to Bitcoin. This model (instant purchasing) is similar to Paypal's Instant payment, and even accounting for the risk/reward, I think it's an absolutely great method to help push Bitcoin adoption and position Coinbase for a goto for merchants and more importantly customers.

</comment><comment><USER18> Instant my ass. It took 2 weeks for the ACH transaction waiting period, then they said they couldn't do it anyway because they were being my conscience for me & decided I was being to risky. Said to try again in 2 weeks. Not to mention the initial account setup time I've waited.

Needless to say I'm looking for a new btc vendor.

</comment><comment><USER19> If I give you cash, can I buy bitcoin without the hassle of fully verified identification?

<comment> Brian-Puccio Yes.

[https://localbitcoins.com/](https://localbitcoins.com/)

</comment></comment><comment><USER20> Love it! You guys keep getting awesome-er. If I may make one humble request...would love to see a lower transaction limit (<0.01BTC) in the API. My personal project (and I) would be forever grateful.

</comment><comment> atom-morgan This is huge for Bitcoin. Keep up the good work guys!

</comment><comment><USER21> Think about it -- this is news because Bitcoin is a less approachable currency than....stamps, or baseball cards.

<comment><USER22> Those are simple because they are physical artifacts, but let us consider the process of using those for trade.

A thirteen year old wants to send $4 to his friend. He wants to put money in a pool to buy a 4-pack of Terraria on a Steam sale for $10. He will receive one of the copies for his $4. His friend lives in another state.

He buys an envelope for 20 cents. He buys a stamp for 46 cents. He tries to look up his friend address on Facebook. It's not there. He sends his friend a text message, asking his address.

He addresses the envelope with his friend's address and his own return address. He licks and places the stamp in the top-right corner. He folds the dollar bills in a piece of scrap paper from the printer. He seals the envelope and drops it in a mailbox.

Several days later, his friend receives the $4. The friend deposits this with his own $6 to get a Visa prepaid debit card. Friend uses the prepaid card to buy Terraria on Steam, a gaming service which requires an account and email address.

Now, consider a transaction of $50 for a baseball card. Sending cash in the mail requires two stamps and two envelopes.

Who sends the letter first? What happens if one letter doesn't arrive?

Even stamps and baseball cards can present challenges for real-world transactions where physical co-presence between two parties is not possible.

<comment><USER21> Agreed.

Now consider Bitcoin:

Somebody wants to pay somebody else for something using Bitcoin, but can't because You Can't Buy Bitcoins.

</comment></comment></comment><comment><USER23> Tried to sign up twice. Didn't receive the required email either time. :(

<comment><USER6> We're a little backed up on sign up emails at the moment! Should be arriving shortly (scaling up workers). Apologies for the delay.

</comment></comment><comment><USER24> Still cant purchase bitcoin in the UK

</comment><comment><USER25> Love it! Nice work guys.

</comment><comment><USER26> Lol, I bought bitcoin with coinbase when this HN article was first posted 3 hours ago, and the price has gone up almost 5% since then. Is bitcoin magic or something?

<comment><USER27> There's a big rally going on. It found support around 68 a week ago and it's been steadily climbing since.

</comment></comment><comment><USER28> Isn't the point of bitcoin to stay anonymous? It's nice to be able to purchase it instantly, but it seems to be going against the purpose of bitcoin.

<comment><USER13> For some people the point of Bitcoin is to get rich via speculation.

<comment><USER29> While your comment is very much true, there is some apparent snarky feel to it that seems unnecessary.

In my case at least, there is no speculation element attached to bitcoin. In fact, its volatility is very detrimental.

I am dividing my life and work projects between the US, Europe and Brazil. I'd love to be able to just charge people in BTC. I wouldn't lose 6-7% every time I needed to convert money from USD to BRL, or from BRL to EUR. I can't say how much easier it would be for me to report taxes...

Widespread adoption of BTC could be a good thing in itself. There are other groups of people interested in bitcoin besides the "fiat must die", the silk roads and the speculators.</comment></comment></comment>


Derive Yourself a Kalman Filter - <AUTHOR> https://ngr.yt/blog/2019-04-10-kalman.html <comment><USER2> "It's really quite simple. Here, let me show you all these formulas..."

This reminds me of last week. I bought a house and want to get into woodworking so I looked up intro videos on YouTube. "It's easy. Just follow me over to this table saw and router and planer and all these other tools you don't own."

I know I'm not being fair. But a recurring frustration I have is when experts claim it's easy or simple or for beginners and then talk right over you. They don't mean to, but it can be insulting and demoralizing. "well if it's for beginners and I don't know what these glyphs mean, the problem must be me."

So back to the topic. Am I wrong or could you begin with, "a kalman filter is a way to get a guesstimate of a value from different sources where the trust in each source can vary."

<comment><USER3> You could, but you should follow it up with "where one of those sources is your (incomplete, noisy) information about what the value _used_ to be and how it changes over time".

If you just have (say) three different ways of estimating the position of an aeroplane, then you don't need a Kalman filter. The place where the Kalman filter adds value is where you measure its position _repeatedly_ and make use of the fact that it's known to be moving approximately in a straight line.

<comment><USER4> How do EKF and ensemble Kalman Filters fit into this?

<comment><USER3> The Kalman filter assumes that all errors are gaussian, that all updates are linear (i.e., new state = linear function of old state), and that observations are linear (i.e., what you measure is a noisy version of a linear function of the state).

The _extended Kalman filter_ allows for state updates and observations to be nonlinear, by the straightforward expedient of replacing them with linear approximations near to the current estimate.

The _ensemble Kalman filter_ also allows for nonlinear updates and observations; instead of keeping track of the expectation and covariance of the state (i.e., everything you need to define a Gaussian model, things that behave nicely under linear transformations but _not_ under nonlinear ones) it keeps track of an "ensemble" of sample values, applies the update and observation functions to those, and then estimates expectations and covariances from this ensemble. It's a sort of Monte Carlo Kalman filter.

A couple of other things worth knowing about:

Intermediate between the extended KF and the ensemble KF is the "unscented Kalman filter". Like the ensemble KF it estimates things using a number of samples; but instead of propagating those samples through repeated steps, it picks the sample points at each step on the basis of the estimated expectation and covariance, and uses them only to compute new expectations and covariances. More expensive than the EKF but copes better with substantial nonlinearities.

Extrapolating beyond the ensemble KF is the "particle filter", which uses the same "track an ensemble of samples" approach but gives up the assumption that all the errors are Gaussian. You need a larger ensemble to get good results, I think, but it can cope with a wider range of scenarios. (I find the name "particle filter" annoyingly distracting; the "particles" are the samples, which I guess you're supposed to think of as a cloud of points in possible- configuration-of-the-system space, and of course it's a "filter" in the same way as the Kalman filter is.)

</comment></comment></comment></comment><comment><USER5> If you enjoyed this, you might also like this article from my blog:

[http://heinrichhartmann.com/blog/2014/12/11/Generative- Model...](http://heinrichhartmann.com/blog/2014/12/11/Generative-Models-for- Time-Series.html)

This is a study of generating time series/stochastic processes. The estimators for the parameters lead straight up to Kalman filters. The state space models are taken from the Kalman setup. This was the first time I understood how Kalman filters come about. Its really a three step process:

1\. Stationary processes --> Classical parameter estimation.

2\. Discrete state space --> Markov models

3\. Continues state space --> Kalman filters.

</comment><comment><USER6> Make sure to read this bit to make the formal explanation easier to understand.

[https://aguaviva.github.io/KalmanFilter/KalmanFilter.html](https://aguaviva.github.io/KalmanFilter/KalmanFilter.html)

</comment><comment><USER7> This one is great to me, less formula and more drawings :)

[https://www.bzarg.com/p/how-a-kalman-filter-works-in- picture...](https://www.bzarg.com/p/how-a-kalman-filter-works-in-pictures/)

</comment><comment><USER8> A few months ago I try to use Kalman Filter as one of the filters to stabilize the GPS from mobile phones due to various factors: loss of signal, jumpy stationary coordinate, fake GPS, etc. I found that Kalman Filter was not giving me satisfactory result, adjusting its parameters many times without further improvement. Finally, I just need to write my own simple filter, without fancy math filters (kalman/etc) as I can differentiate the speed of car/motorbike/pedestrian, distance of jumpy signal (less than 20 meters in four directions), accuracy of gps signal in the city/suburb, etc.

</comment><comment><USER9> I think there is no such thing as a conditional random variable. There are conditional probability distributions, but I've never heard of A|B itself being a random variable and I think it doesn't fit the definition. A random variable (I'm not a mathematician) is a function that takes an elementary event and returns the value of the variable.

You may say this is pedantry, but I think it's important to keep track of what is what, especially when being a beginner. You can afford to be sloppy once you're more advanced.

<comment><USER10> [http://www.maths.qmul.ac.uk/~pettit/MTH5122/notes15.pdf](http://www.maths.qmul.ac.uk/~pettit/MTH5122/notes15.pdf)

<comment><USER9> Hm, I googled it now, the issue is discussed in detail here: [https://math.stackexchange.com/questions/612468/how-to- forma...](https://math.stackexchange.com/questions/612468/how-to-formalize- conditional-random-variables)

<comment><AUTHOR> Thanks for your comments! I don't see how what is discussed here conflicts with the notation I introduced into the post, do you still believe there is a soundness issue in what I have written?

<comment><USER9> I'm just saying that the notation of say A * X|Y * B seemed unfamiliar to me. I only know conditional notation within a P(...). Or an expectation, etc. Apparently your way of writing is used by others as well, but it may be good to know that it is not fully rigorous.

Again, there are different people preferring different presentations. I as a student was often frustrated by abused notations and was often confused by such things when trying to understand something in detail. For a more cursory and "practical" understanding it could be good enough.

<comment><USER10> > it may be good to know that it is not fully rigorous

What is the problem with A|B=b being a random variable? (Apart from you unfamiliarity with the concept, I mean.)

Edit: I don't say there are no problems, I ask what do you think the problem is? There is no problem in the discrete case. In the continuous setting things are indeed more complicated (but if the limiting process is well defined there are no issues).

Note that the same lack of rigour that you find in conditional ramdom variables affects conditional probabilities. If you can accept the latter there is no reason to reject the former.

<comment><USER9> A random variable is different concept from a distribution. For me personally it is helpful to keep them separate, but I can see that others may not care about the complete conceptual picture.

In the PDF file linked above I can see conditional probabilities, conditional distributions and conditional expectation etc, which are all valid and rigorous. I can see that the author thinks it's a good idea to merge these into a single concept of conditional random variable for didactic reasons, but that's not a rigorous concept.

Practically, if you have two random variables then you can take their joint distribution. What would be the joint distribution of (A|B) and (C|D)? For actual random variables it's simple: you can take intersections in event space, but a "conditional random variable" does not correspond to any subset of the event space.

Very simply speaking (this is my working model, not the exact precise math definition which involves a lot of measure theory): in probability theory we have an event space containing atomic events that cover all possible outcomes for _the whole experiment /observation_. A random variable is a function that maps from each such potential (atomic) event to a number. That's right. The random variable is a function but not the mass function, which maps from a number to a probability.

Conditional probability P(A|B) is an expression defined to mean P(A,B)/P(B). That's a clear definition. I am yet to see the actual definition of a conditional random variable.

Again, disclaimer 1: I can see the practicality of disregarding formality. Still I argue this is best done only when you do know better but it would be tedious to be technically correct all the time. But as a beginner I find it more useful to keep track of the correct concepts. For example not distinguishing random variables and distributions can be very confusing when considering more advanced things, like mutual information and KL-divergence. The former operates on random variables, the latter on distributions. I remember this was a difficult realization for me because the material we used didn't emphasize the difference enough, probably in the name of practicality.

Disclaimer 2: my point is a minor one overall.

<comment><USER11> I think it will help if you think in terms of conditioning on (for example, a coarser sigma algebra). You would get another random variable that is measurable on the sigma algebra you conditioned on. If that is coarser so would be the new function you obtained by conditioning.

<comment><USER9> Let's talk about a fair dice roll to make it concrete, and let the rolled number be X and let the event that we rolled an even number be E. P(X=6|E) = 1/3\. P(X|E) is a distribution where 1,3,5 has 0 probability mass and 2,4,6 have 1/3 each.

If we consider X|E as a random variable, what is its value if we roll an odd number? Undefined? What does that mean? Random variables always have some value.

Sure you can build a new event space (sigma algebra) but then you can't use random variables over the original one.

Let's consider two independent rolls, X and Y. You can't compute the joint distribution P(Y, (X|E)), it just doesn't make sense as the two "variables" are defined over different spaces. Note that this is not the same as P(X,Y | E). The latter is simple a conditional _probability_ , without any concept "conditional _random variables_ ".

Again, this is totally obvious to people who have experience with probabilities, but could be confusing to students. Such cases are where students who try to understand the details may be left more confused than students who just want to get the main idea.

<comment><USER11> Sure you can. The TLDR would be "piecewise constant projection"

I think picking up a standard graduate probability book will clear this up better than any long comment trail. There are no problems defining a coarser sigma algebra using an original one and then defining a function measurable on the new sigma algebra. Note this continues to be an r.v. in the original space as meaurability is preserved. A consistent definition the values of the conditioned r.v. would be the piecewise constant approximation of the original r.v. over the indivisible elements of the coarser sigma algebra.

Let me try another route.

You seem to be accepting of a conditional expectation. Now what is a conditional expectation if not a function. Now all we need is that function be measurable with respect to the new sigma algebra, thats ensured byconstruction. Hope it helped some

<comment><USER9> > I think picking up a standard graduate probability book will clear this up > better than any long comment trail.

Can you recommend one? I just picked up _Probability and Measure_ by Billingsley and it does not mention "conditional random variable" a single time in over 600 pages. It does have a lot of "conditional probability", "conditional distribution", "conditional expectation" etc.

> You seem to be accepting of a conditional expectation.

Conditional expectation is defined in terms of conditional _probabilities_ , and those are in turn explicitly defined as P(A|B)=P(A,B)/P(B), so there's nothing not to accept.

<comment><USER11> Billingsley is pretty darn good. It might have left the connection as a dotted line given that the notion is no different from conditional expectation. The only connection you have to make is conditional expectation is a function and a random variable. You must have seen expectation taken of a conditional expectation. That should should convince you that condititional expectation is indeed a random variable. Since that r.v. was obtained by conditioning its not a stretvh to call it a conditioned r.v.

Any book that explains conditioning over a sigma algebra should suffice. You could try Loeve, Dudely or Neveu but dont remember if its mentioned explicitly.

BTW conditional expectation is really more fundamental than conditional probability. Its the former that yields the latter in measure theoretic probability. If you want to drink from the source that would be Kolmogorov.

Finally if you are reading Billingsley you are adequately qualified to call yourself a mathematician.

<comment><USER9> It's getting a little tedious. Please show me a concrete citation of a serious textbook (not a tutorial/handout by a grad student or a paper by a random researcher) that puts the three words "conditional random variable" next to each other (consistently, not simply as a one-off potential mistake). Google doesn't show serious sources for it.

While I agree with isolated points of your comment I think it doesn't add up to a useful/coherent concept of conditional random variable.

<comment><USER11> Thats a little too much to ask, perhaps if they were grep'able I could have obliged, unfortunately I dont have a photographic memory.

More concretely its just another name for conditional expectation. I am assuming you are aware that conditional expectation is a random variable obtained via conditioning (equivalently as a piecewise approximation in L_2). If you arent familiar with that view point that would be the place to start. Kolmogorov, Neveu, Dudely, Billingsley will all cover that view point.

<comment><USER9> > I am assuming you are aware that conditional expectation is a random > variable

That's not what we're considering here, but things of the form X|Y=y for a concrete y. Even as E[X|Y=y], that's not a function, y is specified. Do you agree we shouldn't call X|Y=y a conditional random variable?

<comment><USER11> Oh absolutely for a specific y its not function (or a random variable) one usually thinks of Y as a variable and not a constant.

<comment><USER10> The expectation E[X|Y=y] is a fixed value. (Edit: it's the expectation of the random variable "X|Y=y", while E[X|Y] is a random variable because it's a function of the random variable Y: for each element in the sample space there is a corresponding value of "y" and in turn there is a value of the expectation E[X|Y=y].)

X|Y=y (as used in the blog post being discussed) is a random variable: it's a function from a subset of the original sample space (corresponding to the elements for which the value of the random variable Y is y) to real values (or whatever the image of the X random variable is).

<comment><USER11> Yes you are right. I had messed up in the comment above. It continues to be a function on the restriction Y=y

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> In my very ignorant world view Kalman filters make some prediction from some input. As do various ML techniques. As do various statistical models and techniques. What are some good sources that show the connection between all these techniques and help me pick the right one for specific use cases?

For example, I want to predict a time series let's say the number of visitors of a site. I know some characteristics of the series (periodic, seasonal), but how should I go about it?

<comment><USER9> > In my very ignorant world view Kalman filters make some prediction from some > input.

The word "filter" indicates that it turns one sequence (usually a time series of measurements) into another sequence (an estimate for underlying states). You can think of it as denoising the sequence of measurement by using knowledge about how the underlying system behaves. For example, if our GPS measurement says we suddenly jumped 100 meters compared to a second ago, we can weigh this against our prior knowledge that a car (the underlying system) is not likely to make such a sudden position change.

The Kalman filter weights the incoming measurement against what our model would predict. Both the prediction and the measurement are probabilistic and they are weighted according to the uncertainties of them. The more certain source of information is weighted higher.

> For example, I want to predict a time series let's say the number of > visitors of a site. I know some characteristics of the series (periodic, > seasonal), but how should I go about it?

That's not what the Kalman filter is for as you are not trying to denoise some sequence of noisy measurements.</comment></comment>


Show HN: A curated list of awesome web apps - <AUTHOR> https://github.com/vasanthv/awesome-web-apps <comment><USER2> It's not a curated list of awesome anything if you don't annotate each entry and tell us what it does and why it is awesome at doing it.

IMO, of course.

<comment><AUTHOR> Makesense, renamed it to [https://github.com/vasanthv/web-app- hunt](https://github.com/vasanthv/web-app-hunt)

</comment></comment><comment><USER3> I like lists like this.

Server not found for Typen...

<comment><AUTHOR> I am marking those dead services which were once alive. Will change it to dead service.</comment></comment>


A history of branch prediction - <AUTHOR> https://danluu.com/branch-prediction/ <comment><USER3> The use of previous branch history and branch address as a "context" for prediction reminds me of the very similar technique used for prediction in arithmetic compression as used in e.g. JBIG2, JPEG2000, etc. --- the goal being that, if an event X happens several times in context C, then whenever context C occurs, the probability of X is more likely.

Also, since modern CPUs internally have many functional units to which operations can be dispatched, I wonder if, in the case that the "confidence" of a branch prediction is not high, "splitting" the execution stream and executing both branches in parallel until the result is known (or one of the branches encounters another branch...), would yield much benefit over predicting and then having to re-execute the other path if the prediction is wrong. I.e. does it take longer to flush the pipeline and restart on the other path at full rate, or to run both paths in parallel at effectively 1/2 the rate until the prediction is known?

<comment><USER4> You could do it, but 'work' produces heat.

From that point of view a branch predictor /saves/ you from spending the heat of the cases you /don't/ need to have processed.

The performance per watt of such a design would probably leave it on the back of a napkin as an educated guess of how costly that would be.

<comment><USER5> Pie-in-the-sky idea here, but only irreversible computations produce heat. Maybe in the distant future we can make chips that do many parallel computations of all branches reversibly, and only make the results irreversible once the correct branch is known?

<comment><USER6> In theory, irreversible computations have to produce heat, while reversible do not. In practice this is mostly irrelevant, because the heat involved is so minuscule that there will always be other significantly larger sources of inefficiency involved. Also it is somewhat questionable whether one could actually construct physical realization of useful logic primitive that is truly reversible.

</comment></comment></comment></comment><comment><USER7> Very informative. I missed the part about 1500000 BC though – a time when our ancestors lived in the branches of trees?

Another beginner-friendly explanation of the effects of branch prediction is this Stack Overflow post which compares a processor to a train: [https://stackoverflow.com/questions/11227809/why-is-it- faste...](https://stackoverflow.com/questions/11227809/why-is-it-faster-to- process-a-sorted-array-than-an-unsorted-array)

<comment><USER8> Maybe it is just an arbitrarily long time in the past, since there were no computers then there is no history related to branch prediction so it's a joke that makes it sound like a longer spanning history than it actually is.

</comment></comment><comment><USER9> One surprising thing that I discovered recently is that after Haswell, Intel processors got much much better at predicting "interpreter loops", which are basically a while true loop with a very large seemingly unpredictable switch statement. It lead to a dramatic improvement in micro benchmarks and made some traditional optimizations involving computed goto and " indirect threading" obsolete.

Does anyone know how it achieved this?

<comment><USER10> It doesn't look like a series of comparisons from the CPU point of view. Normally switch statements are compiled like a series of "if" statements, but the interpreter loop style switch gets compiled into a table of jump targets that is indexed by bytecode. Same kind of indirect branch prediction features that were previously designed to help C++ "virtual" functions help here - <USER2> branch target buffer, etc.

The VM interpreter loop is mostly a main bottleneck in languages that have rather low-level VM instructions and data types. In high level VMs the dispatch on operand type is the main bottleneck. This too benefits from indirect branch prediction.

<comment><USER11> Longer switches are usually compiled into binary search trees.

<comment><USER10> It depends on whether the switched-on values are sparse. Contiguous ranges of bytecodes are more efficiently compiled to straight jump tables (and enable indirect branch prediction mechanisms in CPUs to work).

For another boost to leveraging indirect branch prediction, threaded code is still a little better since each VM instruction has a unique jump call site: [http://eli.thegreenplace.net/2012/07/12/computed-goto-for- ef...](http://eli.thegreenplace.net/2012/07/12/computed-goto-for-efficient- dispatch-tables)

<comment><USER9> That blog post is from 2012. What I observed is that with the more recent processors threaded code doesn't make much of a difference.

</comment></comment></comment></comment></comment><comment><USER12> > PA 8000 (1996): actually implemented as a 3-bit shift register with majority > vote

This actually seems interestingly different from the two-bit saturating counter. Like, it's not just a different way of implementing it; you can't realize the saturating counter as a "quotient" of the shift/vote scheme.

<comment><USER13> It's the same just with redundant representations (the 2-bit repr is the count of ones in the 3-bit repr)

      '00' -> '000'
      '01' -> '001', '010', '100'
      '10' -> '011', '101', '110'
      '11' -> '111'

this kind of transformation is truly bread & butter in hardware; we regularly numbers between binary counts, mask/number-of-set-bits and one-hot representations for optimisation purposes.

<comment><USER12> That's the obvious attempt at an equivalence one would come up with on being told that they're the same, but, as I stated, it doesn't work. As an example: In the 2-bit saturating counter, if you start at 00, if you see a 1 and then a 0 you're back to where you started. Whereas in the bit-shift register, if you start at 000 and see a 1 and then a 0, you're now at 010, which would correspond to 01 rather than 00.

</comment></comment></comment><comment><USER14> I seem to be missing something when the two bit scheme is introduced it's said that it's the same as the one bit scheme except for storing two bits (seems logical), but then the index in the lookup table seems to involve both the branch index (already the case in the one bit scheme) and the branch history (as far as I can see never introduced).

<comment><USER12> Looks like he just used the wrong picture -- used the picture from "two-level adaptive, global" instead.

</comment></comment><comment><USER15> Is there any system out there that supports branch 'annotations', of a sort, so that the programmer or the compiler can just _tell_ the CPU what the branch behavior is going to be?

Like -- it seems kinda silly for the CPU to do so much work to figure out if a loop is going to be repeated frequently, when the code could just explicitly say "fyi, this branch is going to be taken 99 times out of 100".

Or, if there's a loop that is always taken 3 times and then passed once, that could be expressed explicitly, with a "predict this branch if i%4 != 0" annotation.

<comment><USER16> Yes and no. I say _yes_ because C and C++ both have likely() and unlikely() functions (well, technically It's part of a compiler extension rather than the language), which you wrap around the condition inside your if() statement like so:

      for(i=10000; i<100000; i++) {
        if(unlikely(isPrime(i))) {
          // do something special
        }
        else {
          // do something boring
        }
      }

I say _no_ because most modern compilers simply ignore the functions. Compilers have become so sophisticated over the years that developers trying to help them along or optimize often make things worse, whether that's by getting in the compiler's way or just writing code that's harder to read and debug.

I say _no_ (or at least _probably not_ ) to your second questions as well. No language I know if implements anything as sophisticated as a specification for regular intervals of branch switching. Many modern compilers have sophisticated branch prediction routines which can detect simple regular intervals like you describe.

To develop such a specification would optimize the branch prediction by a tiny margin which would be absolutely _dwarfed_ by the overhead of learning the syntax for the specification, not messing it up, debugging it if you do mess it up, communicating the decision to other team members, and all of the other real-world stuff that gets in the way.

Computers are faster than ever and branch prediction algorithms are smarter than ever. Yes, you could help it along in theory but the portion of applications which _really_ require you to do so is dwindling all the time.

</comment></comment><comment><USER17> I really have problems reading this website. You don't have to make a website bloated to make it readable: [http://bettermotherfuckingwebsite.com](http://bettermotherfuckingwebsite.com)

<comment><USER18> Have a look at the Chromium/Chrome's extension Just Read

before / after [https://i.imgur.com/Ihy5wQh.png](https://i.imgur.com/Ihy5wQh.png)

<comment><USER19> I've tried a couple of these types of extensions, Just Read is the best one I've found so far.

</comment></comment></comment><comment><USER20> Ryzen has rolled out a Neural-Net based branch predictor, would be curious to see its accuracy compared to the listed approaches.

<comment><USER21> It's even mentioned in the article:

> Some modern CPUs have completely different branch predictors; AMD Zen (2017) > and AMD Bulldozer (2011) chips appear to use perceptron based branch > predictors. Perceptrons are single-layer neural nets[0].

[0]: [https://www.cs.utexas.edu/~lin/papers/hpca01.pdf](https://www.cs.utexas.edu/~lin/papers/hpca01.pdf)

</comment></comment><comment><USER22> Top quality article. Now we need one with specifics of how to write code that's aware of this. For instance when do use what compiler hints. Anyone have links or books?

<comment><USER23> The compiler is (probably) smarter than you. Generally speaking, it will automatically decide which branches are most likely and arrange them accordingly (e.g. for things like for loops and while loops especially, where the biggest gains are). You'd likely gain more performance out of algorithmic changes, and then a number of other processor optimizations (like vectorization and pre-fetch hints) first. Also, CPU manufactures ignore the classic hints, and don't really have information on how to tune branch predictors. So while GCC/Clang does have a special intrinsic (`__builtin_expect`) for it in c/c++ - most other languages are too high level for it to matter - it probably won't do much and is an insanely early optimization to consider making.

<comment><USER24> Whether or not it's an early optimization depends on when you add the __builtin_expect, no? Perhaps you already have identified a very hot branch in your code that the compiler fails to treat correctly even though you provided it with profile data.

<comment><USER23> Except that anything newer than 1995 probably ignores the hint provided by __builtin_expect, and you'd probably have more luck changing the algorithm or vectorizing the code.

</comment></comment></comment></comment><comment><USER25> Is this correct? "Without branch prediction, we then expect the "average" instruction to take branch_pct * 1 + non_branch_pct * 20 = 0.8 * 1 + 0.2 * 20 = 0.8 + 4 = 4.8 cycles"

other than branch_pct and non_branch_pct being reversed, this seems to be assuming that 100% of branches are guessed incorrectly. Shouldn't something like 50% be used, to assume a random guess? ie 0.8 * 1 + 0.2 * (0.5 * 20 + 0.8 * 1)=2.96

<comment><USER26> It's correct if you take "without branch prediction" to include any pipelining of instructions after a branch.

The very first branch prediction algorithm ("predict taken") is to simply enable pipelining by assuming the generally more likely branch.

>...this seems to be assuming that 100% of branches are guessed incorrectly...

Rather, it's assuming that 100% of branches are not guessed at all.

<comment><USER25> Ah I see, I assumed it was a penalty for only mis-guessed, not the number of cycles required to evaluate the statement. Thanks!

</comment></comment></comment><comment><USER27> I love your posts dan. High quality writing, no fluff and bullshit every time :)

</comment><comment> unkown-unknowns Figures 12 and 14 are the same but I think the figure used is only supposed to be like that for figure 14, not for figure 12.

The "two-bit" scheme that fig 12 is for does not have branch history, whereas "two-level adaptive, global" which has fig 14 fits the bill.

</comment><comment><USER28> Beautiful article. The kind that makes you want to dig deeper in the whole field.

</comment><comment><USER11> TAGE and perceptron combined are the SOTA right now, right?

<comment><USER29> Yes. TAGE still does better than perceptron, but combining the two is likely to give you the best performance currently. Of course, all this is dependent on area allocated to the predictor and the workloads being run.</comment></comment>


Reinforcement Learning – The different challenges, methods and frontiers - <AUTHOR> http://louiskirsch.com/maps/reinforcement-learning <comment><USER2> I applaud this kind of effort. It is super useful to have context for the explosion of work in RL.

One thing that is missing however is that reinforcement learning is a small fraction of what's know about learning through reinforcement. Decades of work in behavioral psychology and neuroscience lead to a very different understanding of learning where agents are not maximizing reward but vigorously pursuing homeostasis.

Additionally the challenges map desperately needs a 'catastrophic forgetting' box. Perhaps this only applies to DRL but it is so fundamental IMO that to leave it out is to miss the foundation for the house.</comment>


Leaderless Debian - <AUTHOR> https://lwn.net/SubscriberLink/782786/1b0334c3a2a9d8b1/ <comment><USER2> Controversial opinion warning, and I would love to hear some counterpoints outside of my thought bubble.

I've never understood the appeal of Debian or Ubuntu, or I guess I don't understand why people would invest so much time in them compared to Fedora and Redhat. In actual buisness dealings Redhat has always led the way in enterprise (whole data centers), kernel development (a bunch of the maintainers work for them and they lead efforts like cgroups v2), and other highly technical pieces of the ecosystem, including acquiring all of the start developers (such as the systemd folks). Investing in Redhat is an investment into a deeper understanding of LInux through their excellent documentation and well paying positions through their famously good certifications.

There is of course a lot of debian and ubunutu out there in small deployments, usually individuals, small buisnesses, and education, but they are ultimately less lucrative and more oriented towards beginners/hobbyists. At least Ubuntu used to be. I'm not really sure who the target audience of debian is.

I've also never understood the appeal of .deb or apt, or what debians end goal is. They never seem to be doing anything that interesting except grinding away at repackaging everything. Canonical is just flailing to find some sort of niche and historically (Ubuntu phone, Mir, Open Stack, Upstart, probably snap) they are incredibly bad at finding relevance. I think having multiple packaging ecosystems is the stupidest idea in the world, and don't see the value add except giving people the illusion of more choices.

<comment><USER3> The thing is, for a long time, deb/apt were way superior to what rpm/yum offered. Heck, Debian had apt way earlier than Red Hat had yum.

If anything, Red Hat should have adopted apt.

In terms of sheer number of dumb competing initiatives, Ubuntu > Red Hat > Debian.

<comment><USER2> So I've heard this before, and I really want to know what are the things that make apt better? The only thing I can see is that yum was pretty slow (dnf seems to be marginally better). Otherwise, the rpm package format seems pretty on par with deb and apt seems like a super crusty interface with multiple commands and stuff.

<comment><USER3> Debian had a much more rigorous packaging policy than Red Hat/Fedora Core early on (the same policy that slows down development and annoys developers today, as you can see in the various recent HN threads) and way more packages in the main repos.

Using RH easily led you into an "rpm hell" of packages from various third- party repos that conflicted: [https://en.wikipedia.org/wiki/Dependency_hell#Platform- speci...](https://en.wikipedia.org/wiki/Dependency_hell#Platform-specific)

Now they're both comparable. Yum/dnf are ok, the rpm repos are big. And apt actually has "apt", the tool, that consolidates those separate tools you mention. Before it had aptitude (since at least 2006 or so, I'd say) but it wasn't used much, but now apt is installed by default and you can use it. No more "apt-get", "apt-cache", etc.

So today, nothing makes one of them substantially better, IMO.

</comment></comment></comment></comment><comment><USER4> > Since Debian developers are famously an agreeable and non-argumentative > bunch, there should be no problem with that aspect of things.

Is this sarcasm? I don't read LWN enough to know if they are dryly joking and I don't read Debian listserv's to know if it's true or not.

<comment><USER5> Before, when Debian Wheezy was the current stable version, I would have read this and assumed no sarcasm and agreed wholeheartedly. But these days, I'm with you, I just can't tell.

Given the recent article by Michael Stapleburg[1] that made it to the front page of HN the other day and now this it seems like things are a bit rocky at Debian these days.

A shame too because even though I don't use it that much any more I still consider it one of, if not the, most important Linux distributions available. Hopefully this is just a glitch that will get sorted sooner than later!

[1] [https://michael.stapelberg.ch/posts/2019-03-10-debian- windin...](https://michael.stapelberg.ch/posts/2019-03-10-debian-winding- down/)

<comment><USER6> I actually think things are a lot _less_ rocky these days - nothing in Michael Stapelberg's post is new, it's just a difference of scale and perhaps the developer population getting a bit older and more committed to their day jobs. There are a lot of things helping with scale: Alioth (running a fork of SourceForge's code) was replaced with Salsa (running GitLab), which effectively collapsed the options for version control from CVS, SVN, Git, Darcs, etc. to just Git. Debhelper 9 standardized a ton of workflows. Team maintenance is a lot more common than individual maintenance, and uploading packages owned by unresponsive maintainers has gotten a lot more socially acceptable. But a lot of the old problems have scaled more quickly than solutions: if you're doing an archive-wide change, there are more packages you need to touch, more people you need to contact, etc., so each old problem happens more.

(I also don't think any of the problems in that post are about developers being disagreeable or argumentative - just about Debian having a lot of technical debt and no easy way to pay it down.)

<comment><USER5> Fair enough and based on what I read yesterday your comments about problems scaling faster than solutions makes a lot of sense. You seem to be more on top of it then me.

As someone who no longer uses it but has a lot of history using Debian I hope you are correct and that this latest issue of not finding a leader according to the standard schedule is just something they route around. Based on the tone of the end of today's article it sounds like that's the case.

</comment></comment></comment></comment><comment><USER7> I know how loved Debian has been for the community and all that has been built on it but as a user I kind of always felt that Debian was always doing their own thing, at their own pace and their own requirements. Maybe this is a sign that that way of running a Linux community is over? We really have a fairly robust ecosystem and development has become much more refined? I might be missing the inner politics of the community that has caused this?

<comment><USER8> Maybe this is an effect linux ecosystem fragmentation of recent years has on communities.

<comment><USER7> I would say the opposite. Things have been consolidating in companies and in technologies. It used to be that you had 10 options for everything. Now things are getting more and more standardized. I like to think we are much better today in Linux than when we were 10 years ago. You use to pick out what applications were important and see what version was supported by what distro. That isn't a thing anymore.

</comment></comment></comment><comment><USER9> I would hope an Amazon, Google, Digital Ocean, Rackspace, or other major vps player could get someone involved for an oss project critical to their business.

I guess as a user I frequently take this for granted.

<comment><USER10> Amazon has Amazon Linux which is based on CentOS, and Google has invested in CoreOS which also based on CentOS just for K8s. I don't know if other vendors like DO would invest in any specific distro tbh.

<comment><USER11> Additional data point: The Linux that runs Google corp Linux machines is a debian derivative:

[https://en.wikipedia.org/wiki/GLinux](https://en.wikipedia.org/wiki/GLinux)

</comment></comment></comment><comment><USER12> > The good news is that this possibility, too, has been foreseen in the > constitution. In the absence of a project leader, the chair of the technical > committee and the project secretary are empowered to make decisions — as > long as they are able to agree on what those decisions should be. Since > Debian developers are famously an agreeable and non-argumentative bunch, > there should be no problem with that aspect of things.

Even if nobody puts their name this week or the next or the week after, and even with the sarcastic remark in the last sentence, it looks like this is not such a big deal after all.

> In other words, the project will manage to muddle along for a while without > a leader, though various aspects of business could slow down and become more > awkward if the current candidate drought persists.

Why should it muddle along or slow down? All the responsibilities listed (and linked to in a mail in the mailing list) seem to be things that others could take up, either individually or as smaller groups. It's something that could be worked out over a period of time.

> One might well wonder, though, why there seems to be nobody who wants to > take the helm of this project for a year. _The fact that it is an unpaid > position requiring a lot of time and travel might have something to do with > it._

The last sentence is probably the issue. Couldn't sponsorships help? The last thing we need is Debian to become corporatized.

Donate to Debian at [https://www.debian.org/donations](https://www.debian.org/donations)

<comment><USER13> > Couldn't sponsorships help?

As implied and cited by the article itself, money and Debian are uneasy bedfellows.

> The last thing we need is Debian to become corporatized.

I highly doubt this will be a likely outcome here.

</comment></comment><comment><USER14> I am switching to Devuan which is Debian without the service manager known as systemd (not trying to flame about systemd but it is no longer my preference) .

So far it has worked great for me on my test laptop and I will be rolling out some VM's shortly.

<comment><USER15> I run KVM VMs on devuan and could not be happier. (flame backspaced out, you're welcome...)

</comment></comment><comment><AUTHOR> I've just browsed [https://en.m.wikipedia.org/wiki/List_of_Debian_project_leade...](https://en.m.wikipedia.org/wiki/List_of_Debian_project_leaders)

There are quite a few names I've never heard of, and some that I've heard of, but not as DPL.

The last six (after Hocevar) are total blanks for me.

Have my interests changed so much, or did the recent DPLs keep a much lower profile?

<comment><USER16> Chris Lamb is a remarkably active and outgoing DPL. But yes, I've been active in Debian for well over a decade and I've seen both a lot of people leave and new people that I don't know or get to know as I used to. All 5 DPL candidates ring a bell for me, though.

</comment></comment><comment><USER17> What whould be the case today anno 2019 to use Debian over CentOS, Ubuntu or one of the other distro's?

<comment><USER18> Arguably rpm/yum is still a kludge compared to dpkg/apt.

<comment><USER19> It's not, although I would admit that it's painful to learn. I spent my fair share of time learning it, and can now understand what's wrong with my configs in a split second when it doesn't work. Overall it's faster and easier to search with than apt, from my (very small, I admit) experience with apt. The dependancy system works really well, even when messing around with upgrades/downgrades/uninstalls too.

And I admit that a few years ago I would rank apt as much better than yum. Matter of preference / habit I guess.

</comment></comment></comment><comment><USER20> I wonder how this will affect the Raspbian as default Raspberry Pi distro...

<comment><USER21> I wish it gets obsolete as it's an abomination.

<comment><USER22> How is it an abomination?

It seems to have served its purpose well so far. It is relatively easy to use and hasn't gotten in the way of anything I've wanted to use it for yet.

<comment><USER23> Am not the parent, but RPi is the only ARMv6 target of any significance; supporting an outdated architecture has non-trivial costs, which is why Ubuntu and other distros dropped the support.

</comment></comment></comment></comment><comment><USER24> This could be a great opportunity for someone looking to step into a leadership role.

<comment><USER25> It is not: [https://danielpocock.com/what-does-democracy-mean-in-free- so...](https://danielpocock.com/what-does-democracy-mean-in-free-software- communities)

The current Debian "developers" have reached uniform agreed that:

1) Neither of them will participate in election

2) People outside of "developer" circle (including major contributors, who do most of actual work on Debian) can't participate in election

The whole "crisis" is artificially created. Not like that matters, — most of decision making is already done by corporate employees, not some fictitious "community leader".

<comment><USER24> Reading through those articles it indeed seems like a political nightmare. When you have both volunteers _and_ paid members, politics will be inevitable, as those getting paid will do everything in their power to keep it, while those not getting paid will do everything they can to gain it. Just like in a startup, money can kill relations. Money is really the root of all evil. In any organization there will be politics, so as a leader you have to learn how to deal with it. You do not necessary need any power, as you do not lead by giving orders, you lead by showing. I think solving the inequity problem would be a top priority, maybe dedicate 50% of your own FAANG salary and give it to people that do the same job as you do, but doesn't get paid. It's probably a really bad idea, but it's just an example, the purpose is that others will follow.

</comment></comment></comment><comment><USER26> Has the number of candidates been decreasing steadily, or has this happened out of the blue? If so, what has happened? Are there more responsibilities to this position, making it harder to do now than it was years ago? Is the interest in Debian dwindling?

<comment><USER27> > Is the interest on Debian dwindling?

On the contrary, the project is growing in terms of contributors, packages and internal projects.

[https://reproducible-builds.org/](https://reproducible-builds.org/) mostly came out from Debian (see [https://wiki.debian.org/ReproducibleBuilds](https://wiki.debian.org/ReproducibleBuilds) )

Despite the name "leader", the role of DPL is close to an ambassador with no power to tell other people what to. The DPL appoints a handful of roles, then hand-on work is delegated and done by other people.

The drama on blogs and mailing lists is in no way representative of how most DDs feel about the project.</comment></comment>


Turning a bunch of numeric attributes into a single score - <AUTHOR> http://stackoverflow.com/questions/27711982/turning-a-bunch-of-numeric-attributes-into-a-single-score/27745140

<comment><AUTHOR> HiScore author here. HiScore allows domain experts to easily create and maintain scores. It is currently being used by a major environmental non- profit and by IES, a startup that assesses the safety and sustainability of fracking wells.

The algorithm itself is a mashup of a paper I wrote for AAAI: [http://www.cs.cmu.edu/~aothman/splines.pdf](http://www.cs.cmu.edu/~aothman/splines.pdf) and this paper: [http://link.springer.com/article/10.1007/s10543-005-0028-x](http://link.springer.com/article/10.1007/s10543-005-0028-x)

</comment><comment><USER2> Asker of the StackOverflow question here! (And payer of the 500-point bounty to the author of HiScore, which I think is brilliant!) I'm excited about this for the potential applications for personal (or startup) goal tracking, like turning a bunch of constituent numbers into a single metric for how well you're doing.</comment>


Russians Hack FBI Comms System - <AUTHOR> https://www.schneier.com/blog/archives/2019/09/russians_hack_f.html <comment><USER2> The first few comments on this article seem odd. Has Schneier got his very own personal IRA bots?</comment>


Mark Shuttleworth orders Kubuntu leader to step down - <AUTHOR> http://www.itworld.com/article/2926838/linux/mark-shuttleworth-ubuntu-community-council-ask-kubuntu-developer-to-step-down-as-leader.html

<comment><USER3> Am I missing something?

So what happened was that

1) a founder of Kubuntu truthfully stated that money from Ubuntu-oriented donations didn't go to Kubuntu

2) It was discovered that a developer of Kubuntu was listed as the recipient of some of this funding, but didn't actually receive any of this money due to an alleged clerical error

3) The clerical error has not been corrected, and a founder of Kubuntu has been asked to step down as a representative of the Kubuntu community/council by the CEO of Ubuntu's parent company

I don't understand how anything that anyone did on the Kubuntu side warranted this reaction. Not only that, it's not clear to me what level of authority Canonical's executives have over the Kubuntu community in the first place.

Is this the full story?

Why do CC/ShuttleworthCo care this much?

<comment><USER4> Not exactly, the problem was that Canonical collected some money during 2012 to 2013 that was supposedly earmarked for Ubuntu flavors. However, there is no accounting for that money at the moment, though something is being worked on.

There's also the much heated debate about Canonical's IP licensing policy which is quite vague and broad IMHO. It also goes against the FOSS licenses such as the GPL and LGPL. Jonathan bought this up with the Ubuntu CC and has been pursuing this for 3 years, without much luck so far. He raised the issue again recently and has been reprimanded for supposedly using language that violates the CoC, however there has so far been no evidence backing this claim.

Due to the above the CC and Mark Shuttleworth have asked Jonathan to leave leadership positions within Ubuntu.

Disclosure: I'm on the Kubuntu Council, so I might have a biased opinion. These are purely personal views.

</comment></comment><comment><USER5> May I suggest a separate project, a la LinuxMint.

Perhaps LinuxKawakawa or another k-herb. [https://en.wikipedia.org/wiki/List_of_culinary_herbs_and_spi...](https://en.wikipedia.org/wiki/List_of_culinary_herbs_and_spices#K)

<comment><USER6> Or they could merge into (or absorb) the KDE flavor of Linux Mint (which - <USER2> - is more-or-less redundant given Kubuntu's continued existence), taking over the development of the Mint+KDE and LMDE+KDE projects (thus leaving the main Mint folks to focus on Mint/LMDE with Cinnamon and MATE (and Xfce, but I personally think that Xubuntu should follow Kubuntu's footsteps and absorb Mint/LMDE+Xfce).

To me, this particular Canonikerfluffle is just yet another straw on a camel's already-broken back.

<comment><USER5> Even better, both times.</comment></comment></comment>


Russia offers $110,000 to crack Tor anonymous network - <AUTHOR> http://www.bbc.com/news/technology-28526021 <comment><USER2> How on Earth is $110,000 a reasonable value for this level of vulnerability? (And with a multi-thousand dollar up front cost to boot.) I'm not remotely an expert on security, much less nefarious black-market hacks, but I would think that a whole lot of nations around the world would be willing to pay millions (each!) for this capability. Heck, if not for this, I would have included Russia on that list.

<comment><USER3> The "hackathon" spirit has caught on at the state level. Get a bunch of college kids to slave away for free and give the best one a paltry sum for their efforts and retain all rights to the work.

<comment><USER4> How is that the "hackathon" spirit?

<comment><USER5> It's the spirit ever since the term "hackaton" caught the attention of the employers and the events are organized by companies for their profit instead of hobbysts/enthusiasts just for fun.

<comment><USER6> For ex: Hackathon's shouldn't primarily be about cash prizes. It defeats the purpose and ideology behind hacking culture.

<comment><USER7> Isn't that the parent comment's point? That it's such a relatively small amount of money that the only people who are into hacking stuff anyway would attempt it.

<comment><USER6> There shouldn't be any money at all in hackathons, it should be voluntary and tinkering/hacking on technololgy for fun or societal benefit. This is for a project with a well-defined goal defined by a large government that may be used against dissidents by a authoritarian state.

That doesn't quite fit into hacker culture at all.

If they aren't going to hire a well-paid hackers full-time like the NSA does to do the same thing, then at least make the prize significant similar to X-prize.

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> As Schneier says:

"The reward of $114,000 seems pretty cheap for this capability. And we now get to debate whether 1) Russia cannot currently deaonymize Tor users, or 2) Russia can, and this is a ruse to make us think they can't."

[https://www.schneier.com/blog/archives/2014/07/russia_paying...](https://www.schneier.com/blog/archives/2014/07/russia_paying_f.html)

<comment><USER9> That's because they're asking if someone's done it and wants some cash. If you had, monetizing the knowledge isn't exactly easy. No intelligence agency will bother paying for it.

<comment><USER10> Why wouldn't they? This is exactly the stuff I'd want to pay for if I was an intelligence agency; you want the hackers to come back with more exploits, after all.

<comment><USER11> No intelligence company would pay for it _if it were publicized_ they had paid for it. It's more useful to keep quiet and let people think Tor is still anonymous.

<comment><USER12> Could the person who sold it to them publicize it? Assuming they would be outside of Russia or unbound from any sort of gag order (not likely)?

</comment></comment></comment></comment></comment><comment><USER13> Heh. This is low-ball. I was once offered $150,000 in a discussion with a broker when I inquired about a hypothetical Tor 0day. After the broker's fees, I would have still walked away with $120,000 if I had one. (Then taxes, of course.)

If anyone wants to do this, I recommend shopping around first ;P

</comment><comment> kar-kub Last paragraphs are very interesting:

"In its 2013 financial statements, the Tor Project - a group of developers that maintain tools used to access Tor - confirmed that the US Department of Defense remained one its biggest backers. The DoD sent $830,000 (£489,000) to the group through SRI International, which describes itself as an independent non-profit research centre, last year. Other parts of the US government contributed a further $1m. Those amounts are roughly the same as in 2012."

I'm not familiar at all how those founding works, could someone, from US, explain how and why US government is giving money to TOR?

<comment><USER14> I've seen two explanations for why the US government gives money to the Tor Project. One reason is to support dissidents in countries like China. Another is that US agents use Tor, but that the network requires a degree of popularity in order for agents to "hide" in it.

<comment><USER15> Another option is an effort to identify cryptographically-capable individuals around the works as targets for potential contact, work, on behalf of the U.S. and its allies.

TOR as a recruitment tool?

</comment></comment></comment><comment><USER16> I don't think that you need to crack anything.

$100,000 is for a research paper.

Translation of the auction lot title: "Research the possibility of getting technical information about Tor users (their hardware)".

Here's the talk that claims the possibility to deanonymize Tor users for less than $3,000 [http://web.archive.org/web/20140705114447/http://blackhat.co...](http://web.archive.org/web/20140705114447/http://blackhat.com/us-14/briefings.html#you- dont-have-to-be-the-nsa-to-break-tor-deanonymizing-users-on-a-budget)

There are published papers on the topic e.g., [http://www.syverson.org/tor- vulnerabilities-iccs.pdf](http://www.syverson.org/tor-vulnerabilities- iccs.pdf)

 _Lesson one is that Tor guards against traffic analysis not traffic confirmation. If there is reason to suspect that a client is talking to a destination over Tor, it is trivial to confirm this by watching them both._ </quote>

</comment><comment> a-nikolaev Bear in mind that in Russia, many "offers" like this are not ment to be real competitions for accomplishing something (to crack Tor, to build a bridge, etc), but simply a way to appropriate state's money.

<comment><USER17> Right, someone's CS drop out cousin is getting this.

</comment></comment><comment><USER15> Are we saying that Russian intelligence services cannot hack TOR and need some random hacker to do it?

Imagine if the CIA offered $1M to crack TOR. They would be the laughingstock of the intelligence community.

I think there is something else going on. I would not touch this. It looks like bait.

</comment><comment><USER18> I wonder if this would backfire and make Tor even stronger, as now even more attention is given to Tor's robustness.

<comment><USER19> maybe that is their goal. Perhaps they want to (or already are) use(ing) tor to hide their own activities from the NSA. They want to make sure what they are doing is truly as secure as it claims to be and if not motivate devs to make it so. After all, it seems it would be cheaper to offer this small reward than to have to pay full time employees to help keep their activities secure from prying eyes.

</comment></comment><comment><USER20> I don't know why anyone would do this for $110,000. Especially after the entry fee, probably wouldn't make much money after the hardware costs, though if you're good enough to take on TOR, you probably also have a botnet. Also, why the hell would anyone give it to the Russians? Of all people, they're definitely who need exclusive access to a TOR hack. Especially if you consider that some of those people who are using it in Russia could be regular people who are trying to not be persecuted for their sexual orientation. Bad idea overall.

</comment><comment><USER21> All the sources refer the same government requisition for a "performing the scientific research, code "TORUS/Fleet". The details should be available for people who chose to participate and foreign nationals are specifically banned from participation.

With the Russian word for torus being "тор" which could be transliterated as "tor" I see why people might get excited. But I'd like to see something more concrete than word play to support the news articles' theory.

</comment><comment><USER22> "Applicants must pay 195,000 roubles to enter the competition"

This seems rather bizarre... ~$5500 cannot simply be a symbolic sum to deter idiots.

<comment><USER23> IMO Russians intend to make their venture profitable.

<comment><USER24> It wasn't clear to me from the article that there was any sort of time limit. Presumably a researcher could simply enter the contest once he was sure he had an exploit? The math doesn't really work out for profitability.

<comment><USER25> The deadline is August 13th, and the winner (if any) will be announced on the 20th. (According to an Helsinki Times article, but the English translation[1] doesn't have the dates)

[1]: [http://www.helsinkitimes.fi/finland/finland- news/domestic/11...](http://www.helsinkitimes.fi/finland/finland- news/domestic/11341-reward-offered-by-russia-to-crack-tor-likely-to-improve- the-anonymity-network-finnish-expert-views.html)

<comment><USER24> Then the point stands. If a Russian researcher has an exploit on the 13th and thinks the prize is worth it, she'll enter. Why would _anyone_ else enter? With that in mind, the proposed profit model seems unlikely.

</comment></comment></comment></comment></comment><comment><USER26> I'd be really surprised if Russia were actually not able to do this already, they're known for having a very strong national community of security experts and overall excellent mathematicians.

<comment><USER27> They _had_ this community.

Nowadays most of them emigrated and those who don't, they mostly will not work for KGB spooks.

</comment></comment><comment><USER28> What's to stop someone from selling one of these exploits to multiple nations and companies?

What is the normal process for selling these exploits? They'd want to see the exploit first, I'm guessing in person, then they transfer over the money, then you give the code and details?

What if someone wanted to remain anonymous during the transaction? What would be the best method of doing that? You couldn't really send a friend because it might be easy to trace back to you, and it would be hard to trust a stranger.

<comment><USER29> They don't want to 'buy' the exploit outright, just fund the R&D. Thank you sensationalist media.

<comment><USER28> I don't understand, it seems like researchers have to pay to enter and then are only given the funding if successful, that's not exactly funding R&D, more of a contest. It seems really strange that Russia would be offering this kind of bounty in effort to improve the program's security, don't they know how many activists and dissidents use it. Is the sole reason to aid their own spies?

They should just provide funds on a site such as [https://hackerone.com/](https://hackerone.com/)

I really wish the US government would offer bounties for their sites and systems. Right now if people try to exploit a US government system, even if they have the intention to properly disclose the vulnerability they face prosecution.

<comment><USER29> It is definitely R&D to find a vulnerability in TOR or lack thereof, it's just that BBC as usual is arbitrarily choosing what to report and what to stay silent about.

Look no further than the tender page: [http://zakupki.gov.ru/epz/order/notice/zkk44/view/common- inf...](http://zakupki.gov.ru/epz/order/notice/zkk44/view/common- info.html?regNumber=0373100088714000008)

Here they explicitly state that it's a tender for 'Выполнение научно- исследовательской работы, шифр «ТОР (Флот)»' (Research and Development works, code "TOR (Navy)")

Then it's a closed tender (stated in the same document), meaning that they come up with a list of organisations they invite to participate in this tender. No organization they did not invite can participate.

So you see this is nothing like a bounty.

>it seems like researchers have to pay to enter

I wager they are required by law to demand some sum of money, maybe this sum is determined as a function of a tender value; I don't believe there is some additional meaning to asking people to pay 5500 usd to participate in a closed tender.

</comment></comment></comment></comment><comment><USER30> I already posted publicly online how to find the identity of a Tor user.

To reiterate:

1\. Get the Tor user in question to visit a website controlled by you ( or at least a site where you can cause JS to run; such as an advertisement )

2\. Know which ISP the user is on, and be allowed to install a high speed device watching all traffic for a sequence of specific sized packets.

3\. Use the JS to send a specifically crafted sequence of sized packets with specific time periods in between them. After sending this preamble, send sized packets to send the 'pseudo identity' of the user ( whatever pseudonym you wish to attach back to their real IP )

4\. Use your monitored ISP device to detect the preamble, then log IP and the data.

Note this method could be done en-masse and would only require high speed FPGA devices at each ISP "trunk". Inject JS code correlating users back for any system which you wish to identify the users.

Done. Whichever Russian demonstrates this and wins the $100k; throw me a bone please. :)

<comment><USER31> I assume a Tor user with js turned off would be immune to this?

<comment><USER32> I wonder if something similar could be done with detecting specifically sized images?

<comment><USER33> You're all making this much too complicated. Who needs the client to make multiple requests when you control the server? Client does "GET /" and the server starts sending a large index.html using irregular sized packets at specific intervals.

But suppose we broke it, now we have to fix it, right? Start padding everything to power-of-two size boundaries with a minimum of 16. Or if that would make Tor traffic too identifiable, then instead add random()%packetsize padding to each packet. Either would reduce the number of detectable packet sizes below a 1500 byte MTU to 8 at the cost of less than doubling the bandwidth consumption.

</comment></comment></comment></comment><comment><USER34> They are also charging an entry fee for this "contest" in addition to the prize being ridiculously small. The good news is that it's quite unlikely this will be successful regardless of the prize.

</comment><comment><USER35> I'd assume you wouldn't have to look to hard to find someone willing to pay $110k+ to identify specific individual Tor users, let alone find general exploits in Tor.

</comment><comment><USER36> Previous thread about it:

[https://news.ycombinator.com/item?id=8079195](https://news.ycombinator.com/item?id=8079195)

<comment><USER37> I submitted this one as well but was not very successful: [https://news.ycombinator.com/item?id=8083790](https://news.ycombinator.com/item?id=8083790)

</comment></comment><comment><USER38> Must be some small agency there, not really representing "Russia"

And anyway why cant people then just use Freenet or some such network?

<comment><USER39> Freenet is a resource hog, and can be rather slow. It's also not particularly interesting for people who just want to browse and access the "normal" web anonymously. There are few services. There's little interesting content.. Plus it's not clear Freenet can really provide that much as far as anonymity goes. The consensus recently has been that opennet is quite vulnerable, and the only way to be really safe out there is with a global darknet where everyone only connects to trusted peers. Achieving this is not so easy, and there are potential complications.

</comment></comment><comment><USER40> Great news. $110,000 from Russia to find and (although they did not intend it) patch Tor bugs. Thanks, Putin!

</comment><comment><USER41> ugh, capitalism. someone somewhere will actually do work towards this goal with that much money in mind as a worthwhile payoff. ending tor anonymity should have at least 2 more 0s on the end of the figure.

</comment><comment><USER42> I bet NSA is willing to pay much more.</comment>


The Glider: A Universal Hacker Emblem - <AUTHOR> http://www.catb.org/~esr/hacker-emblem/

<comment><USER2> I knew what Conway's game of life was and had implemented cellular automata before, but when I first saw this symbol I did not recognize it as a gilder even though I knew the concept of one.

Actually, I thought it was an incorrectly drawn portion of a Go board.</comment>


Ask HN:  Why use hosting company instead of hosting your own website? - <AUTHOR>

I recently set up a LAMP server at home and used DynDNS to point a registered domain at it.  I found the experience so straightforward and satisfying that I'm currently wondering why people pay hosting companies (GoDaddy or whoever) to perform this service?  What is the killer issue, bandwidth?  Security?  Support?<p>My sense is that for small-to-medium websites self-hosting is probably superior for lots of reasons.  Am I missing something? <comment><USER2> I'd actually say you've got it backwards. The bigger you are, the more sense it makes to host internally. The smaller you are, the more it pays to pay someone else.

I don't know what your hourly rate is, but if I had to spend even one hour looking at ANYTHING to do with my own server, I would've been better off paying for it to be hosted somewhere else.

The only way it would be at all sensible to do this is if you had no traffic, weren't selling anything and hosting the app was as much a hobby as the app itself.

</comment><comment><USER3> In addition to the things already mentioned here: Hosting companies have economy of scale. Because they are running hundreds or thousands of machines:

\- They can afford to pay people to stand by on call 24/7. Do you want to get paged in the middle of the night? Incidentally, did you set up Nagios and something like Pingdom and a pager service on this box in your closet, or are you just happy to go down for hours at a time?

\- They see all the common problems all the time. When you experience common problems -- e.g. database crashes and needs to be restored from backups, bad PHP configuration is eating too much memory on the box, DDOS attack needs to be parried with iptables -- you are going to spend hours reading instructions and applying them. Going to all that effort for the sake of one small box is kind of a waste. What you want is to amortize your skills over many, many boxes. That's what professional sysadmins do.

The first time you have to spend a couple of hours reading some FAQs in order to keep your box running you will burn up 100% of the money you saved by not paying for a hosting service.

Of course, if you're trying to learn what hosting is about trying to run your own box is educational. Though I'd still do that learning on a VPS or EC2 rather than struggle with a home network and a random home box.

</comment><comment><USER4> Your home internet connection will not be as reliable or as fast as multiple redundant backbone connections. Check with your ISP, because hosting might be against the terms of service, even if the upload bandwidth seems sufficient for a small site. You can find hosting for as low as a few dollars a month these days, and for that price you get someone else to maintain the hardware, take care of software upgrades, etc.

</comment><comment><USER5> Most ISPs don't give you much upload bandwidth.

</comment><comment><USER6> 3 reasons: uptime (your internet connection), uptime (your electricity supply) and uptime (your hardware).

</comment><comment><USER7> The root is this: is your job maintaining LAMP servers, or is your job running your business?

I know how to maintain a server, reasonably well. But I don't want to actually do that. I'd rather just 'git push heroku' and be done with it.</comment>


Show HN: Secrypto API - <AUTHOR> https://secrypto.herokuapp.com <comment><USER2> This is just a remote key/value store with a key that's so long that it looks like a hash?</comment>