Identity and the web: what Moot overlooked - <AUTHOR> http://michaelsiedlecki.com/post/11883308882/identity-and-the-web-lessons-from-social-evolution-and

<comment><USER2> This is really quite interesting, but only from a social sciences perspective. Here's where social sciences and real socialization differ: the plural of anecdote is data in the real world. Here's a great example: "He asserts that being restricted to a real name (G+ no longer does this) rather than anonymity or a pseudonym is incorrect, because it restricts what we do and share. Is he wrong? Yes, absolutely."

How many gay teens are posting on LGBT support and community sites using pseudonyms because they don't want to publicly out themselves? How many people are posting about breaking DRM anonymously or under pseudonyms to get around draconian laws? These directly fly in the face of the assertion that requiring your discussions and actions to be tied to your real name doesn't impact what you say or do. There's value in real-life identities, but that does not in _any_ way diminish the value of anonymity and pseudonymity.

<comment><AUTHOR> For that case, I agree with you. Where I refute Moot is where he calls the model of reality-reflected identity on Facebook incorrect, which comes off as essentialist. Imagine if Facebook at the start tells you to choose any name of your liking. On a social network, how effective would the content that you'd rather post under a pseudonym be? The idea is that without Facebook, the share-ability of such content would be diminished. So my argument, admittedly unclear in the post, is that pseudonymity and anonymity actually derive their audience from the effects of rigidly reflected identities--but if you throw out the strictness that Facebook imposes entirely, on Facebook in particular, your audience size dries up considerably. People feel comfortable SHARING from a point of forced identity, but their will to EXPRESS themselves is most manifest in anonymity and pseudonymity. Expression without an audience is almost meaningless. It's a foundational issue.

<comment><USER3> Reddit seems to be doing pretty well... and they grow every day. "Sure" I hear you say, "they are a pittance compared to facebook", but really, what out there isn't. And Reddit, unlike Facebook for most of it's existence, has had serious competition. (I don't consider MySpace to be FB competition, since really the migration to FB began pretty much as soon as FB opened up to full public consumption).

Just a data point in the other direction.

<comment><AUTHOR> I mention Reddit and in the post. I remark that it has to do with autobiographical memory, which is a very postmodern force. But it would have a hard time gaining a userbase if something like Facebook and Myspace had never existed. The latter bring the userbase to the Internet in the first place, where Reddit is a secondary outcome that could not have possibly gained its userbase without the effects of a social network (aka people posting reddit links on Facebook).

<comment><USER4> The web existed prior to Facebook. You seem to be arguing that Facebook's very existence is the reason that other sites on the web can even exist, which seems to fly in the face of the history of the web.

<comment><AUTHOR> I am indeed arguing that Facebook's very existence and the effect it had, insofar as it greased the wheel of Internet use, is a huge part of why Reddit can exist at the scale it does. Reddit is a better-organized web forum, which are not new, and maybe by its own design merit does it succeed. This, I do not argue with. But I do not think it could exist at its current or anywhere-near- comparable scale and level of success without the social network effect.

<comment><USER5> Unfortunately your theory is silly and doesn't line up with the demographic studies that have been done.

No doubt you could find some interesting examples or some specific subgroups (my guess: the farmville playing middle aged mother on facebook with little technical aptitude, which is not an insignificant demographic) that have been pulled onto the net by facebook.

The numbers just don't show the kind of massive effect that you seem to be promoting here. It's hard to overestimate facebook's effect, it's a giant. But by saying that it's pulling up the entire rest of the internet you actually managed to do it.

What on earth is this theory based on?

</comment></comment></comment></comment></comment></comment></comment><comment><USER6> Jesus man, I'm very interested in this topic yet I still had a terrible time working through that brain dump. Superfluous acronyms, overly verbose, excerpts from (egads!) your undergrad papers! What the hell, man. Are you trying to appear smart for your professor or are you trying to persuade nerds with ADD? It can't be both.

<comment><AUTHOR> Sorry, naner. When I try to be precise about things and not essentialize, it does require some amount of padding with jargon (pertaining to specificity) which would otherwise be taken as bullshit-like if not read by the fielded eye. What I try to do is stay away from generalizing and making indefensible claims. Next time, I'll stick with graphs, which is how I formulate my ideas and relate different social forces in the first place. To be dry: I wanted to be as accurate as possible, but it clouded all meaning.

</comment></comment><comment><USER7> Gosh that was a longwinded technical article.

All this talk about real identity on the web is a cover for the only real issue. Corporations can track, market to, and monetize what they learn about you more effectively with real id policies for internet use. Governments can track, monitor and control you more effectively with real id policies for internet use. The end.

</comment><comment><USER8> I had trouble following the train of thought in this article; it may be too "stream of consciousness" for this particular audience. While this may be a problem between my keyboard and chair, I do not generally find HN articles hard to follow.

</comment><comment><AUTHOR> After reading and responding to the comments, I want to add an overall, more clear message:

The argument is that without rigid identity on the web, anonymity and pen names could not really take off and receive audience. Many individuals' first encounter with the Internet and any web communities is with Facebook, and this is because these people would not feel comfortable starting out on a place like Reddit for 4chan. For this reason, rigid identity is a necessary foundation for the usefulness of pen names and anonymity. Moot's argument that pen names are the way to go is flawed insofar as it undermines the "necessary evil" of rigid identity, which is a natural, socially evolved stepping stone for anonymity's value.

Without people going online and possessing monuments to themselves (facebook profiles) in a way they feel comfortable and secure in, anything posted online (including content under pen names) is in aggregate less valuable. Rigid identity is a necessary characteristic of social media content which can result in inexperienced web users taking the plunge to explore the rest of the less-rigid web.

<comment><USER9> Anonymous communities existed before rigid identity on the web. Those anonymous communities clearly provided some kind of value to their members and, thusly, must be considered useful.

How, then, can you argue that rigid identity is a necessary foundation for the usefulness of pen names and anonymity?

Further, I find your claim that men didn't have an identity prior to the advent of leisure time, frankly, extraordinary and, as such, the claim requires extraordinary evidence. Can you provide such evidence? Hell, even the argument that nomads don't engage in leisure time seems dubious. _Wild animals_ have leisure time.

<comment><AUTHOR> Something on the web has more value when it is viewed by more individuals than otherwise, correct? Early anonymous communities did provide some kind of value to their members, but they comparatively provide more value today when more individuals are present.

The problem when talking about social science is that everything is relational/relative. Rigid identity is necessary as a foundation for the MASSIVE (modern-day scale) usefulness of pen names and anonymity. It is not necessary for the smaller audience, but for the larger one which is online and present which we take for granted, it is necessary. Wild animals are often always looking for food. The claim of no-identity-before-statehood is, as stated, very, very relative, and I even wrote that this contrasts with identity as we know it today. The point is that identity did not matter when nobody had anything to do with themselves through it. Aristocrats, as written, are the exception due to their privilege of free time and surplus. The identity of a man who does one thing out of survival necessity is orders of magnitude less complex than the identity of a man who does several things and prides himself on all of them, and continuously has the option of re-defining himself.

Moreover, I have to admit that the assumption of no identity before the Industrial Revolution is something I studied in the entirety of two social geography courses dealing with how context defines the modern imaginary (as everything is only imagined with us).

</comment></comment></comment><comment><USER10> Both Moot and the author are correct.

It is well accepted that anonymity brings out the worst in people (the author has said so himself). It is also understood that context guides behaviours. For example, one might be more disinhibited on a tropical holiday. Another example is footballers behaving badly on tours. Another one is the unbelievable atrocities soldiers commit away from their home "context".

Moot's thesis is that people become interesting when you provide them a different context, while the author is saying that it is far healthier to have one well integrated identity.

<comment><AUTHOR> To connect the dots here: I think both Moot and I employ a sort of essentialist rhetoric to get attention, the same way you would write something enormously more controversial under a pen name. A more strict interpretation of Moot's argument is that Facebook should be filled with pen names, as people need to be creative instead of simply informative, and that rigidity kills creativity. A strict interpretation of my argument is just as you said, but also that the well integrated identity comes before anonymity before anonymity may become useful / audienced. Moot's simple championing of anonymity misses the point that you need an audience first, which is gathered by putting people in a position where they can willingly be exposed to its content. It's the same as why farmville isn't 100% evil: it brings people to the web who otherwise would never be there, and acts as a positive externality on all other web sub-industries.

</comment></comment><comment><USER11> > [Autobiographical memory] might be boosted by anonymity and pseudonyms, as a > way to pull down barriers to risky behavior, but anonymity and pseudonyms do > much less for aggregate [utility per person] than postmodern, perceivably > rigid reflections of reality on Facebook do.

There's a lot here about self-gratification through performative identity, but nothing about the value society derives from the unflinching evaluation of unpopular ideas. I find this to be nearly absent in "this will go on your permanent record" oriented venues, which in turn is why I find those all but useless. It's reception and counters to my arguments that I value; what I write would be anonymous (and on slashdot it was) if a dissociated pseudonym weren't required.

</comment><comment><USER12> Why is he capitalizing moot's name? Is it just to be rude?

<comment><AUTHOR> I did not mention the name Christopher Poole anywhere. Capitalizing on his pen name was within his argument's wishes.

<comment><USER13> Capitalizing, not capitalizing on. As in spelling with a capital M, rather than a lower case m.

</comment></comment></comment><comment><USER14> _Key points:

Identity refers to how we reference ourselves, primarily in relation to relationships, experiences, and possessions._

What is your basis for this assertion?

<comment><AUTHOR> Identity comes from the concept of identification/classification. These are notions which evolved with modernism in Western society, or the ability to measure things exactly for the sake of predicting and knowing. There is no exact meaning for identity due to its elusive nature, as its meaning belongs to the fluid individual. My post presumes the definition, and yes, that makes it just an assertion.

<comment><USER14> Well, your entire paper hinges on a definition of identity that is completely internal, yet your measurements are external.

</comment></comment></comment><comment><USER15> I am amazed that someone would write so much so thoroughly yet completely miss the point that some people don't give a damn about UPP or identity. Want to know what my reputation is? Too bad, it doesn't exist. Excepting the handful of close, personal friends, everyone who communicates with me is forced to judge me based purely on the merits of what I am saying, and not any sort of accumulated respect.

That's exactly the way I like it, because it's honest. If I say something stupid, I want people to say it's stupid. If I say something intelligent, I want them to say it's intelligent. The concept that I would be dishonest or hold my tongue instead of saying exactly what I thought for fear of what the listener would think of me is little better than lying to their face. The world would be a healthier place if everyone was called out on their stupidity, every time, by everyone there.

Anonymity enables honesty. Rigid identities are for people who are afraid of what they might hear if there was nothing protecting them from the truth of other peoples' thoughts.

Do you think this comment is the most ignorant, idiotic thing you've read today? Then tell me so, without being influenced by who I am, or dissuaded by anything I might do or think in the future. I welcome your honest opinion, for I know it is based purely on what I have said.

<comment><USER16> You're ignoring something important: Talk is cheap, and listening is expensive. Judging only the merit of ideas is _nice_ , but it is only applicable within a small community. Once you have more people than you can easily listen to (even on a cursory level), it becomes a search problem. I don't have time to seek and consider opinions from all 7 billion people on the planet whenever I want an opinion on anything — particularly the kind of large topics where I'd generally want someone else's opinion — so I will focus my attention on people I know to be helpful. If I can't identify you as somebody who might have something to say, most of the time you'll get mentally lumped in with the teeming masses who don't even have an informed opinion on the matter.

<comment><USER17> _Talk is cheap, and listening is expensive. Judging only the merit of ideas is nice, but it is only applicable within a small community._

Even in a small community attention can be expensive. I've worked with a number of developer groups. Within each I would eventually reach a point where people knew enough about me that I could ask what might initially seem a stupid question but often lead to some interesting discussion because I was trying to a) make sure I understood this or that assumption and b) see if some assumptions were just plain wrong.

And the group knew I wasn't just dense or playing games; I might be wrong about this or that, but I was sincere and likely had at least _some_ insight to offer.

I'm sure there are various ways to get these same kinds of group discussion going but reputation saves a whole lot of time when trying to explore a counterintuitive idea.

Even in small groups people have lots to do, so having an "in" based on reputation is important.

BTW, I do share the OP's point about bluntness, and I've been fortunate to work with people who, after considering something I've suggested, would give a direct, matter-of-fact opinion.

But this may be another case where personal knowledge helps; most people respected each other well enough to understand that there's a difference between dumping on an idea and dumping on a person. The nice thing was that no one thought you stupid because you said or did stupid things (if your overall reputation was for general non-stupidity) and it made communication easier.</comment></comment></comment>


Why I'm Ditching CoffeeScript - <AUTHOR> http://toshokelectric.com/blog/2013/04/04/why-im-ditching-coffeescript/

<comment><USER2> I'd be tempted to fork if it were me.

IE6? They insist on supporting IE _6_???

The jQuery guys finally figured out what a _really amazingly horribly bad idea_ it was spend support and dev effort on old versions of IE. The Coffeescript guys will figure it out sooner or later.

</comment><comment><USER3> tl:dr; pull request gets rejected, author enters a drama queen mode taking it personally and over emotionally.</comment>


Remote Jobs from Companies Hiring Remotely in Feb 2020 - <AUTHOR> https://docs.google.com/spreadsheets/d/1R8PdujcmQV5xGbiPmrLiC5KQfi6ehePujrpQ2BnVFWE/edit#gid=1369056372 <comment><USER2> Nice list, what tools are you using to scrape these jobs and then format the sheet? I've used Octoparse to pull from Indeed and it worked well, but I wouldn't be able to format the spreadsheet like this automatically.</comment>


Onyx: fault tolerant data processing for Clojure - <AUTHOR> https://github.com/MichaelDrogalis/onyx/blob/0.5.x/README.md <comment><USER2> This looks very interesting. I'm doing some log file processing in Apache Spark in Clojure. Spark is written in Scala, but has a Java API, which is wrapped by Flambo. It looks and feels entirely Clojure.

The semantics look very similar indeed. Does anyone have a comparison between Onyx and Spark?

<comment><USER3> I've used Onyx, but I haven't used Spark, so take this with a grain of salt.

A few key differences:

Onyx aggressively uses data structures to define the structure of computation, defining the data flow (Onyx workflow) and parameterization (Onyx catalog) of the the computation via clojure maps and vectors. In comparison Flambo and Spark define the structure of computation via functions over collections. One way in which Onyx's approach is powerful is that it becomes trivial to manipulate workflows or catalogs before submitting jobs at runtime, allowing you to add additional tasks, task options, etc.

Onyx also implements batching over streaming operations, whereas Spark appears to be the opposite. There are likely to be trade-offs between these approaches.

Spark is also a lot faster, though this isn't necessarily intrinsic to the approaches.

<comment><USER4> I'm interested to know if you've used Storm at all and how it compares to Onyx. I'm currently considering both for a project.

<comment><USER5> Hello, Michael Drogalis - the author here.

I'm also not a Spark user, but I have used Storm:

\- Storm is significantly more mature and performant the moment.

\- Storm has a better cross-language story in terms of bolt functions.

\- Pretty much everything in Onyx is much more open ended. This applies to deployment, program structure, and workflow creation - and is mostly an artifact of how aggressively Onyx uses data structures.

\- Onyx has a far better reach across languages in terms of its information model.

\- Onyx will be adopting a tweaked version of Storm's message model next release to get on the same level of performance and reliability. We're dropping the HornetQ dependency.

\- Onyx is born out of years of frustration of direct usage of Storm and Hadoop.

<comment><USER6> As someone who has been using Storm, this looks very interesting. What I particularly like are the clean, well thought-out ideas. Also, easily reconfigurable (at runtime) topologies are something we'd be interested in. I will definitely take a very close look at Onyx.

Performance is important: in our case, decreasing it significantly below Storm's level would not be acceptable.

Also, I watched the Strange Loop presentation and the tree model looks limiting to me: I have topologies where I need to merge information from two streams (but perhaps I haven't understood the Onyx model yet).

<comment><USER5> Performance - wait until the 0.6.0 release. We'll be caught up with Storm by then.

The tree model is being removed in 0.6.0 in favor of a vector of vectors (DAG), which allows multiple inputs. See [https://github.com/MichaelDrogalis/onyx/blob/0.5.x/doc/user-...](https://github.com/MichaelDrogalis/onyx/blob/0.5.x/doc/user- guide/concepts.md#workflow) The tree model wasn't one of my better ideas.

Edit: to be clear, you can do stream joins right now in 0.5.3 with the DAG model.

</comment></comment></comment></comment></comment></comment><comment><USER5> Hi folks! I'm Michael Drogalis - the primary author. I'm happy to answer any questions.

<comment><USER7> What were the main pain points that motivated you to develop Onyx? What capabilities do you want to add or have already added that Storm doesn't provide?

<comment><USER5> See: [https://github.com/MichaelDrogalis/onyx/blob/0.5.x/doc/user-...](https://github.com/MichaelDrogalis/onyx/blob/0.5.x/doc/user- guide/what-does-it-offer.md)

These are all the things I wrote down that I wanted before I wrote the first line of code.

</comment></comment></comment><comment><USER8> Checkout the original video introducing Onyx: [http://youtu.be/vG47Gui3hYE](http://youtu.be/vG47Gui3hYE)

<comment><USER9> Live open sourcing !

</comment></comment><comment><USER10> If this interests you, then you should also check out the post where Michael Drogalis first introduced this:

[http://michaeldrogalis.tumblr.com/post/98143185776/onyx- dist...](http://michaeldrogalis.tumblr.com/post/98143185776/onyx-distributed- data-processing-for-clojure)

</comment><comment> dj-wonk Re: Onyx's architecture. I would wonder about performance when keeping a shared log in ZooKeeper. Why not use something like Kafka -- it is designed for high-volume, immutable logging. ZK works best for less-frequently changing configuration, such as node connection information or snapshotting. I could be wrong. I'd like to hear your thoughts and experience.

<comment><USER5> \- Picking up Kafka means introducing another dependency.

\- Onyx's log doesn't grow particularly large because it's only used for coordination, not for messaging.

\- Because the log isn't huge, and can be GC'ed, consumers don't experience high volumes of messages.

\- ZooKeeper offers sequential node creation - making it a really good fit for what the log needs to do.

</comment></comment><comment><USER11> Looks superficially simmilar to [https://github.com/aphyr/tesser](https://github.com/aphyr/tesser) anyone know both and can give a comparison?

From a brief examination tesser looks a lot simpler (probably because of encoding most of the folding using various monoids). Does onyx have a similar abstraction model that I missed?

<comment><USER12> Onyx is distributed and Tesser just uses all the available cores of a particular machine AFAIK.

Both libraries are awesome.

<comment><USER11> Tesser also allows you to distribute it using hadoop i think. I haven't used it, I only happened to hear about it why @aphyr gave a talk at the clojure exchange in London.</comment></comment></comment>


Raspberry PI car OBD2 computer - <AUTHOR> https://github.com/stevelacy/movel <comment><USER2> Looks very cool! I coincidentally spent most of the weekend reading about OBD- II and CANs. I'd like to get a RasPi/Arduino with a temperature sensor connected and send CAN messages to regulate interior temperature for example.

From what I gather, CAN messages are not broadcasted through the OBD port for sniffing, but once we know the proprietary CAN commands (VW in my case), we can send them through? Ie. For finding what car button sends what message, does one have to splice some cables to tap into the CAN bus? Dr. Charlie Miller & Chris Valasek wrote a nice paper on the subject back in 2013, and they hacked together their own cable: [http://illmatics.com/car_hacking.pdf](http://illmatics.com/car_hacking.pdf)

Being able to control the car's to implement my own features from reverse camera to remote start sounds very appealing.

<comment><AUTHOR> I have not yet played with CAN messages, please do let me know how your testing goes. From what I understand yes, the messages would go through the BUS to the car's CPU.

<comment><USER2> Thanks!</comment></comment></comment>


The MOOC Revolution That Wasn't - <AUTHOR> http://techcrunch.com/2014/09/11/the-mooc-revolution-that-wasnt/

<comment><USER2> I think the MOOC still has potential, but it needs to follow the same evolution as the rest of the internet. The trend over the last decade has been to put the power of creation/production into the hands of everyday people, not the few elite who control the means of production. Notable examples would be YouTube, which allowed people to do from their bedrooms what only television and movie studios did a generation ago, airbnb, which lets anyone be a hotel, Uber, which lets anyone be a taxi, WordPress, which let anyone publish online, Amazon or Ebay, which let anyone sell online, Wikipedia, which lets anyone edit the encyclopedia.

I think the true power of the MOOC is not letting Stanford teach the world, but letting the world teach itself.</comment>


AWS Application Load Balancer - <AUTHOR> https://aws.amazon.com/blogs/aws/new-aws-application-load-balancer/ <comment><USER2> We plan to do a blog post about this at some point, but we had the pleasure of seeing exactly how elastic the elb is when we switched Cronitor from linode to aws in February 2015. Requisite backstory: Our api traffic comes from jobs, daemons, etc, which tend to create huge hot spots at tops of each minute, quarter hour, hour and midnight of popular tz offsets like UTC, us eastern, etc. There is an emergent behavior to stacking these up and we hit peak traffic many many times our resting baseline. At the time, our median ping traffic was around 8 requests per second, with peaks around 25x that.

What's unfortunate is that in the first day after setting up the elb we didn't have problems, but soon after we started getting reports of intermittent downtime. On our end our metrics looked clean. The elb queue never backed up seriously according to cloud watch. But when we started running our own healthchecks against the elb we saw what our customers had been reporting: in the crush of traffic at the top of the hour connections to the elb were rejected despite the metrics never indicating a problem.

Once we saw the problem ourselves it seemed easy to understand. Amazon is provisioning that load balancer _elastically_ and our traffic was more power law than normal distribution. We didn't have high enough baseline traffic to earn enough resources to service peak load. So, cautionary tale of dont just trust the instruments in the tin when it comes to cloud iaas -- you need your own. It's understandable that we ran into a product limitation, but unfortunate that we were not given enough visibility to see the obvious problem without our own testing rig.

<comment><USER3> I was coming here to ask whether pre-warming is still an issue with the ALB service. Maybe jeffbarr can comment on whether that's changed?

GCE's load balancer does not use independent VM instances for each load balancer, instead balancing at the network level. So you can instantly scale from 0 to 1M req/s with no issues at all.

<comment><USER4> You can request pre-warming for additional ELB capacity, when you know far enough in advance that you will have a spike. AWS customer service responds by asking 10 clarifying questions via email. The thing is, we can't look under the hood to see currently provisioned and utilized ELB capacity, so we just have to trust that AWS engineers will properly allocate resources according to the answers to those questions. IMO, it's a rather cumbersome process that would be better implemented as a form.

<comment><USER5> its cumbersome, and not remotely useful in this specific situation from OP anyway.

<comment><USER6> It's not clear why it's not useful? The OP said _hot spots at tops of each minute, quarter hour, hour and midnight of popular tz offsets like UTC, us eastern, etc._ , so wouldn't he just tell Amazon "Hey, we get xx requests/second at our peak, so we'd like the ELB ready scaled to handle that load"?

</comment></comment></comment></comment></comment><comment><USER7> Can we agree on the terminology for Application Load Balancer and Elastic Load Balancer?

* ALB: Application Load Balancer

* ELB: Elastic Load Balancer

I have seen Application Elastic Load Balancer/AELB, Classic Load Balancer/CLB, Elastic Load Balancer (Classic)/ELBC, Elastic Load Balancer (Application)/ELBA.

In any event, I think it is great that AWS is bringing WebSockets and HTTP/2 to the forefront of web technology.

<comment><USER8> In my opinion, it is HTTP and WS LB rather than Application LB as it supports just two protocols. In contrast, if you look at F5 load balancer, it can look at LDAP packets or Diameter packets and do a L7 load bakancing. So ALB seems misleading terminology to me as HTTP and WS != All L7 applications

<comment><USER9> Judging by how Amazon has handled other things, I wouldn't stick too hard to that conclusion. They tend to focus first on something generally useful and easy then come back to fill in more difficult/less popular options over time.

</comment></comment></comment><comment><USER10> The real question: does this provide a faster elasticity component than ELBs?

At a previous employer, we punted on ever using ELBs at the edge because our traffic was just too unpredictable.

Combining together all of the internet rumors, I've been led to believe that ELBs were/are custom software running on simple EC2 instances in an ASG or something, hence being relatively slow to respond to traffic spikes.

Given that ALBs are metered, it seems like this suggests shared infrastructure (binpacking peoples ALBs onto beefy machines) which makes me wonder if that is how it actually works now, because it would seem the region/AZ-level elasticity of ALBs could actually help the elasticity of a single ALB.

If you don't have to spin up a brand new machine, but simply configure another to start helping out, or spin up a container on another which launches faster than an EC2 instance... that'd be clutch.

Deep thoughts?

</comment><comment><USER11> AWS still doesn't support IPv6. Good to see them talking about HTTP/2.

Waiting for AWS to embrace IPv6.

<comment><USER12> That's my post, and my misunderstanding. I'll clean it up now, thanks!

<comment><USER11> BTW, is there any place to log feature requests?

<comment><USER13> Become a big enough customer

<comment><USER14> I run an AWS partner who deals with many large ($millions a year each but not tens of millions each) aws customers.

It seems like the number of customers asking for a feature is more important than the size of those customers in my experience.

</comment></comment></comment></comment></comment><comment><USER15> So this is pretty much the same as Google HTTP load balancing [https://cloud.google.com/compute/docs/load- balancing/http/](https://cloud.google.com/compute/docs/load-balancing/http/) \+ websocket & http2?

<comment><USER16> Google's load balancer does do HTTP/2\. It doesnt have native websocket protocol support (you have to use the network LB for that traffic) but it does provide cross-region balancing.

<comment><USER17> Websockets over SSL is in beta: [https://cloud.google.com/compute/docs/load- balancing/tcp-ssl...](https://cloud.google.com/compute/docs/load- balancing/tcp-ssl/)

<comment><USER16> Yes, saw that, but it requires setting up a whole new LB just for websockets. It would be nice if it was just a natively supported protocol on the existing HTTPS LBs, is there a reason why that cant be done?

</comment></comment></comment></comment><comment><USER18> +1 for CloudFormation support on launch day. +1 for support for ECS services with dynamic ports (finally!) -1 for no CloudFormation support for ECS

(To configure an ECS service to use an ALB, you need to set a Target Group ARN in the ECS service, which is not exposed by CloudFormation)

<comment><USER19> Looks like you can set the Target Group ARN in the LoadBalancers property group now [http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuid...](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws- properties-ecs-service-loadbalancers.html)

</comment></comment><comment><USER20> Exciting! Disappointing that you can't route based on hostname yet, though. I've got 5 ELBs set up to route to different microservices for one app, and because we couldn't do path-based routing before, that's all segmented by hostname. As soon as ALB supports hostname routing, I can collapse those all into a single LB.

<comment><USER21> The "Hostname" is in the HTTP Header, "Host." The article states that you can route based on Layer 7 headers, so this shouldn't be an issue.

<comment><USER20> Yeah, but they only allow you to route via path matching right now. The ALB has access to the headers, but configuring it based on arbitrary headers isn't exposed to end users yet.

I excitedly set up an ALB as soon as I read the post, because I've needed it, only to find that support for what I want isn't available to me yet!

</comment></comment></comment><comment><USER22> > 25 connections/second with a 2 KB certificate, 3,000 active connections, and > 2.22 Mbps of data transfer or

>5 connections/second with a 4 KB certificate, 3,000 active connective, and 2.22 Mbps of data transfer.

"2KB certificate" and "4KB certificate"? Is this supposed to read "2048 bit RSA" and "4096 bit RSA"?

<comment><USER23> Most likely, yes.

</comment></comment><comment><USER24> This looks pretty sweet. The next big thing for api versioning would be header instead of url based routing, looking forward to 'give you access to other routing methods'.

<comment><USER25> Already has it! "An Application Load Balancer has access to HTTP headers and allows you to route requests to different backend services accordingly."

Edit: On a second read, it's less clear if header based routing is actually available yet...

<comment><USER20> It only allows you to route via path matching right now.

"each Application Load Balancer allows you to define up to 10 URL-based rules to route requests to target groups. Over time, we plan to give you access to other routing methods."

The ALB clearly has technical access to the headers, but use of them isn't exposed to users yet.

</comment></comment></comment><comment><AUTHOR> They finally added support for websockets! Really looking forward to giving this a try with Phoenix.

<comment><USER26> I just did a little dance at my desk.

<comment><USER27> Same here, can't wait to start using WS and HTTP/2 without extra work involved.

</comment></comment></comment><comment><USER28> These new features are cool... but they still pale in comparison to something like HAProxy.

I guess the tradeoff is that with ELB/ALB, like most PaaS, you don't have to "manage" your load balancer hosts. And it's probably cheaper than running an HAProxy cluster on EC2.

But for the power you get with HAProxy, is it worth it?

Does anyone have experience running HAProxy on EC2 at large scale?

<comment><USER29> ELB _is_ HAproxy. :-) Sure, you get a lot of flexibility configuring HAproxy yourself, but you also have to run it yourself. 90% of the time it's easier to just use ELB (plus it has some direct integration with other services, like IAM-stored server certs/keys, ASG, etc).

I have swapped out ELB for HAproxy and/or nginx on a couple of occasions. If you know your load and feature requirements intimately, you might be able to do a better job. But it's work.

</comment></comment><comment><USER30> I'm curious if this will Convox to route to multiple services with just a single ALB instead of the historical default of 1 ELB per service. Would be a real cost savings for a micro-services architecture.

<comment><USER3> It should allow Convox to do that, yes. Although it doesn't appear (from the screenshots) that you can route by Host header, so you'd have to put all your services on the same hostname with different path prefixes to make it work.

<comment><USER31> We've got some changes coming along these lines. Looks like we might also be able to use CloudFront for hostname routing.

</comment></comment></comment><comment><USER32> This is very good. Recently my workflow has been ELB -> NGINX -> Cluster.

Nginx was a cluster of machines that did routing based on rules into the ec2 machines. Now that the AELB has some of those capabilities it's time to evaluate it.

</comment><comment><USER33> Any love for Elastic Beanstalk with these? They seem well matched. Though EB always feels a bit of a red-headed stepchild in the AWS portfolio.

<comment><USER34> I love elastic beanstalk (minus it's mediocre docs). Agreed here, does elastic beanstalk support setup of new-style elbs (without me doing extensive customization)? When will it if not?

<comment><USER12> Elastic Beanstalk supports ALB.

<comment><USER12> Small correction to my previous answer, Elastic Beanstalk will support ALB in the very near future.

<comment><USER35> Great. I'm looking forward to the day when I can retire my hacky workarounds to facilitate websockets on Elastic Beanstalk.

<comment><USER36> Out of curiosity, what are these workarounds ?

<comment><USER35> TCP load balancing with a modified nginx config that forwards the websocket upgrade header

</comment></comment></comment></comment></comment></comment></comment><comment><USER37> It seems that routing is done in the following way /API/* goes to applications and expects :8080/api/ rather than the root. Would be nice to have the option to direct traffic to just :8080.

<comment><USER38> What client webserver can't handle this though?

</comment></comment><comment><USER39> It looks like the big missing piece is auto-scaling groups as target groups...

<comment><USER40> It looks like it supports ECS services as target groups, and ECS services can do auto-scaling.

<comment><USER41> Not much help if your services aren't containerised though

</comment></comment></comment><comment><USER42> This sentence sums up one of my main reasons for appreciating AWS:

The hourly rate for the use of an Application Load Balancer is 10% lower than the cost of a Classic Load Balancer.

They frequently introduce new features while cutting costs.

<comment><USER43> no, they replaced bandwidth costs with new pricing component $0.008 per LCU- hour[1]. If you have 1M idle websocket connections you will pay 100 times more for ALB vs ELB (i.e. $2K/mo vs $18/mo).

Good thing ELB is still here, so you can choose between them depending on your workload.

[1] LCU - Load Balancer Capacity Units

[https://aws.amazon.com/elasticloadbalancing/applicationloadb...](https://aws.amazon.com/elasticloadbalancing/applicationloadbalancer/pricing/)

<comment><USER42> I don't believe classic ELB supports websockets[1] making this a tenuous comparison. There might be workarounds that I'm not aware of (our production network isn't currently on AWS so I'm a couple of years behind in my day-to- day experience with them).

That said, I don't dispute that there might be use cases where classic ELB is a better option. And I'm glad it's still available (as opposed to ALB replacing classic).

[1][https://aws.amazon.com/elasticloadbalancing/classicloadbalan...](https://aws.amazon.com/elasticloadbalancing/classicloadbalancer/faqs/)

<comment><USER43> it supports any TCP-based protocol, like websockets or MQTT, similar to Network LB on GCE.

</comment></comment></comment></comment><comment> shawn-butler Anybody know whether the new ALB handles a client TLS (SSL) when operating in http mode?

I was trying secure an API Gateway backend using a client certificate but found ELB doesn't currently support client side certificates when operating in http mode.

There was this complicated Lambda proxy workaround solution but I gave up halfway through...

[https://aws.amazon.com/blogs/compute/using-api-gateway- with-...](https://aws.amazon.com/blogs/compute/using-api-gateway-with-vpc- endpoints-via-aws-lambda/)

</comment><comment><USER44> As a heavy ECS user, all I can say is thank you, finally!

<comment><USER45> What are you using for configuration / orchestration of both the container servers and the tasks? CF? Terraform?

<comment><USER46> Take a look at Convox as a fast way to bootstrap managing instances, ECS, and load balancers via CloudFormation.

We have ALB working already.

[https://github.com/convox/rack/pull/1045](https://github.com/convox/rack/pull/1045)

Disclaimer: I work on Convox.

</comment></comment></comment><comment><USER36> I'm the process of containerizing an app that includes a Websockets service, and given ECS / ELB limitations we'd just decided to go for Kubernetes as the orchestration layer.

This ALB announcement + the nicer ECS integration could tip the balance though.

Any thoughts on how likely it is that Kubernetes can/will take advantage of ALBs (as Ingress objects I suppose) soon ?

</comment><comment><USER47> Do ALBs support more than a single SSL certificate?

<comment><USER48> Like via SNI? No mention of it and the screenshots make it seem unlikely.

<comment><USER49> The Amazon Certificate Manager uses SNI, and you can request certificates with multiple hosts and even wildcard domains. I would imagine if you upload your own multi-domain certificate that it would work in the same way, but I have never tested that.

<comment><USER50> I think you mean SAN instead of SNI. SNI is like host headers for TLS connections, while SAN on certs allow you have to very valid for multiple names.

</comment></comment></comment></comment><comment><USER51> This is definitely nicer than having to create subdomains for microservices and mapping each subdomain url to its own Elastic Loaad Balancer + Elastic Beanstalk instance. But I have already gone down this path so I am unlikely to use AWS Application Load balancer. I wish I had this option a year ago.

</comment><comment><USER52> Nice haproxy / nginx alternative. It's got http2 support though which puts it ahead of haproxy.

<comment><USER53> I wouldn't consider this a full haproxy/nginx replacement just yet. It doesn't support host based routing, so you would need a different ALB for each host in order to get the same thing you would get with haproxy/nginx.

</comment></comment><comment><USER54> Any idea if sticky TCP sessions will be supported on ELB/ALB any time soon?

</comment><comment><USER55> I wonder if they fixed the routing algorithm for TCP connections. It's round- robin on ELB, which is performs terribly for long lasting connections.

</comment><comment><USER47> So what would be a use case for using ELBs now? Seems like ALBs do everything ELBs do, but with websocket and HTTP/2 support.

<comment><USER56> If your security requirements are such that AWS can't terminate TLS would be one reason.

<comment><USER57> Also if you're load balancing any other application protocol other than HTTP/HTTPS. (e.g. SMTP load balancer)

</comment></comment></comment><comment><USER58> Disappointing. I was hoping they were launching a service discovery stack to compliment ECS.

</comment><comment><USER59> Does anyone know if it (finally) supports sticky websocket sessions?

<comment><USER60> Yes they do.

For a single connection the websocket will always go to the same back end regardless of sticky sessions being enabled.

If stickyness is enabled, and the same client creates a new websocket, it will go to the same back end as previous connections.

<comment><USER61> This isn't working for me, I'm not getting an upgrade header back from the server, it just stalls. Hm.

Did you have to do anything special to get this to work?

</comment></comment></comment><comment><USER62> Virtual Host Load Balancer would be great.

</comment><comment><USER63> Any idea if it's HIPPA compliant?</comment>


Google Calendar is not available - <AUTHOR> https://www.google.com/calendar

<comment><USER2> What do you mean by not available? It's working for me</comment>


Windows Command-Line: The Evolution of the Windows Command-Line - <AUTHOR> https://blogs.msdn.microsoft.com/commandline/2018/06/27/windows-command-line-the-evolution-of-the-windows-command-line/ <comment><USER2> "We ignored the console for 28 years because at first we thought everyone should use a mouse for everything. Then we wanted everyone to use PowerShell, despite being designed by morons and universally hated. And then we just sort of ignored the problem ( it's not like you were using Vista or Windows 8 anyways), hoping it would go away. Now we're doing the minimum we can to support the Linux subsystem because we want to be hip, but we're terrified of adding in security issues. Which would be very possible because of how wonky Windows security is and how badly the Console works... Typing 'dir' goes to the kernel level to check permissioms, but is written in 30 year old C code. We can barely get the thing to compile. But hey - ANSI colors! Huh?? Pretry great, right?"

</comment><comment><USER3> Please, improve Powershell. The interface is lacking mainly because:

\- it's not scrolling fast enough

\- it's easily possible to send processes to the background (and job management)

I'd really like to use Powershell on windows but git bash works better.

</comment><comment><USER4> waste of lifetime... get a proper os.</comment>


11 meter wide asteroid passing close to Earth today (11am EST) - <AUTHOR> http://www.geek.com/articles/news/11-meter-wide-asteroid-passing-close-to-earth-today-20120127/ <comment><USER2> How hard would it be to trap this in our orbit for future mining? Or bring it in for a soft landing?

Any idea how much it weighs? Would there be enough material to make it worthwhile?</comment>


11 Top Tips for a Successful Technical Presentation - <AUTHOR> http://www.hanselman.com/blog/11TopTipsForASuccessfulTechnicalPresentation.aspx

<comment><USER2> For the non-MSFT programmers here, Scott Hanselman is well known in the MSFT community as one of the top authors and speakers for ASP.NET and .NET in general. The man is awesome.

</comment><comment><USER3> I've always liked Mark Jason Dominus' Conference Presentation Judo talk: <http://perl.plover.com/yak/presentation/>

<comment><USER4> that was pretty weak, actually.

</comment></comment><comment><USER5> One that I've used successfully is to consciously transform my "um"s and "ah"s into moments of silence.

Instead of looking fumble-y, I look thoughtful.</comment>


JPMorgan Chase and Citibank Have $2.96T Exposure to Credit Default Swaps - <AUTHOR> https://wallstreetonparade.com/2020/03/jpmorgan-chase-and-citibank-have-2-96-trillion-in-exposure-to-credit-default-swaps/ <comment><USER2> Just what we need, potentially another banking collapse. The timing couldn't be worse.

Are there any good forecasts of this looming recession? Will this be worse than the Great Depression? How long will it take things to snap back?

<comment><USER3> My thought is that the banks won't be failing here. They are in much better shape than '08, have tighter policies, and perform stress tests.

<comment><USER4> That's a common refrain but just how true is it considering the shadow banking system? People said similar dismissive things in 2008, too.

</comment></comment></comment><comment><USER5> Is this source trustworthy? Edit: to clarify, CBC says that the Canadian government is warning about a lot of Russian "disinformation" going around, so it's nice when news sources are familiar or vetted

<comment><USER6> If it's true, we will certainly hear about it in a more reputable newspapers soon enough. I personally wouldn't consider that site trustworthy.</comment></comment>


Booking.com and Credit Card Fraud - <AUTHOR> http://vladfr.github.io/blog/2014/11/25/booking-dot-com-and-credit-card-fraud/

<comment><USER2> Booking.com uses an agency model with hotels and doesn't handle any payment from the user. They basically get a commission once your stay is completed. What this means is that they pass on your information to the hotel so that they handle billing/charging, etc.

This is in contrast to most OTAs in the US that are on a merchant model where they charge your credit card and have a pre-agreed upon net rate they pay for rooms to the hotel. This means the hotel has no idea what your credit card is or how you paid, just that a paid reservation with their partner was made.

This isn't very different than calling up a hotel and providing your credit card number and CVV / Expiration date on the phone to make a reservation. Only difference is that they sent it electronically.

<comment><AUTHOR> One more thing: they are sharing my personal data without my consent - in this regard, it is very different than me providing my card data over the phone.

<comment><USER2> They do have to share your personal data to the fulfilling agent just like you have to share your personal data to book a flight. The hotel has to have your information to complete the reservation and I'm sure it's buried somewhere in the terms and conditions.

<comment><AUTHOR> Hotels will still ask you for your credit card, most (if not all) payments on Booking.com reservations are done upon arrival, and not online. So, honestly, Booking asking for your card details is completely useless.

Hotels might as well shoot you an email or call you to confirm - in fact, I think that would make for a better experience.

</comment></comment></comment></comment><comment><USER3> Hi Guys, recently i suffered a credit card fraud traveling to Innsbruck in Austria or to a little village in Germany close to Fussen. I was almost sure that was thanks to Booking.

<comment><AUTHOR> There would be no way of telling - someone might've just found your cc data some other way - but yeah, knowing that it's out there is scary. I'm still checking my balance every day.</comment></comment>


Codeless Mobile Analytics - <AUTHOR> https://mixpanel.com/codeless-mobile-analytics/ <comment><USER2> So, you basically copied heapanalytics.com

<comment><USER3> Well, I get what bsammers said because is what heap have been offering since day one and what made it special. You are even using gradients with that orange that reminded me about them.

I'm happy you took the non-coding approach, it will be much easier for the analytics or marketing team that won't have to wait for the engineers to push changes to the tracking code.

I didn't find it on the website so I'll ask here: Do you do like heap and track all user interactions and then allow to track whatever you need on the fly, event past events?

Hope you take the same approach on the web version. It's the only pain point I see on Mixpanel.

Thx, and keep up the great work, I use MixPanel a lot :)

<comment><USER4> No, Mixpanel does _not_ auto-capture user interactions. Heap does. (Disclaimer: I build Heap.)

The Heap approach (i.e. auto-capturing events) lets you run retroactive/on- the-fly analytics. Without it, this feature is closer to a mobile equivalent of Google Tag Manager's click listener tag ([https://support.google.com/tagmanager/answer/3415369?hl=en#C...](https://support.google.com/tagmanager/answer/3415369?hl=en#ClickListener)) or KISSMetrics' click tracker ([http://support.kissmetrics.com/tutorial/event- library-tutori...](http://support.kissmetrics.com/tutorial/event-library- tutorial/events-clicks-tutorial.html)).

Happy that Mixpanel is making their event-tracking easier, though. It's a major pain-point, especially for mobile apps.

</comment></comment></comment><comment><USER5> I regard mobile and web analytics as unethical when they are used to correlate my activities between two different apps or websites, however I am cool with webmasters analyzing their own server logs, as well as for mobile developers to track my use of their app during beta testing but not during production.

Other than those I use technical measures to defeat them and work tirelessly to enable others to do so as well.</comment>


The End: NZBMatrix closes - <AUTHOR> http://nzbmatrix.com/the_end.html <comment><USER2> This is a Usenet binary sharing site whose front page, in a Usenet "video review", was a table of movies, TV shows, and video games. In other words, it wasn't even possible for the owners to look at their own site and not know that it was being used almost exclusively as a tool for piracy.

Extremely common DMCA misconception: it's not enough to be "takedown compliant". That's not how the law works. You also can't ever operate with specific knowledge of infringing content; you are, in effect, required to "take down" any pirated content _you_ find.

<comment><USER3> Has it actually been established that merely linking to copyrighted content is illegal in the United States? I'm not trying to troll, just genuinely curious.

My technical understanding of this could be incorrect, but it seems to me that there is a meaningful difference between a BitTorrent tracker actively coordinating copyright infringing downloads vs. a website like NZBMatrix hosting nothing but static links to another location on the Internet.

<comment><USER2> The NET act criminalizes any willful "copyright infringement" (the term used in the act) so long as it's done for financial gain. Contributory or vicarious liability for copyright infringement for linking is well established.

At any rate, services like these rely on the DMCA Safe Harbor provision. Regardless of whether the potential liability is civil or criminal, you can't have actual knowledge of infringing activity on your site _and_ the ability to remove that activity from your site and claim safe harbor.

<comment><USER4> What about Google? They make a copy of almost every image on the internet with the full knowledge that they are nearly all covered under an absolute copyright license of "All Rights Reserved", including the right to store and rehost a copy of that content. How would Google's financial outlook change if they decide to follow the letter of the law and never use pieces of copyrighted works "for financial gain" without explicit permission from each rightsholder first?

It seems to me that the only difference between Google and NZBMatrix et al is that the media companies like one more than the other.

<comment><USER5> > _It seems to me that the only difference between Google and NZBMatrix et al > is that the media companies_ NEED _one more than the other._

</comment></comment></comment></comment></comment><comment><USER6> Gotta say that I find it very questionable how many people dump on nzbmatrix or say that it was not necessary anyway.

Sure, if you only watch high quality stuff like Skyfall and listen to Lady Gaga, a site like nzbmatrix is probably useless to you. But for those who appreciate things that aren't that easy to come by, it is essential to have some sort of an index to even HEAR about them. My estimate is that at least half of the movies and games I got through nzbmatrix, I had not heard about anywhere else. And as every usenet user knows, it's basically just not possible to "browse" the usenet itself.

Then there are software releases where 95% are fake or bug-ridden. Where it saves a LOT of trial & error if people help each other out by posting only those that are for real.

And I found no other nzb site as useful when it comes to finding all new releases. And I have looked at them all because I did not appreciate nzbmatrix's own censorship.

<comment><USER7> You don't need a usenet indexing site to learn about good indie movies or music. There are dozens of websites that review and discuss these things without releasing them for free against the authors wishes....

</comment></comment><comment><USER8> _Coupled with this is problems with payment providers, we have been through pretty much everyone out there, in the end they all pull out._

Here lies the real problem

<comment><USER9> These guys could benefit from BitCoin, if it were ubiquitous. Too bad...

<comment><USER4> They did utilize bitcoin. I purchased my VIP account with them via bitcoin.

<comment><USER9> My point was more that Bitcoin isn't widespread enough to compete with traditional payment processors yet, which would have saved these guys.

<comment><USER10> I would guess that also Bitcoin is nowhere stable enough to be taken seriously. <http://en.wikipedia.org/wiki/File:Bitcoin_exchange.png>

</comment></comment></comment></comment></comment><comment><USER4> Would be nice if they would a) publish their code and b) upload their nzb database on Pirate Bay. Would make it much easier for someone to pick up the torch.

<comment><USER11> NZBs die pretty quickly, so a back catalog wouldn't be that useful. They even mention on the website that content is getting taken down faster than NZBs can be updated.

<comment><USER4> I've used NZBs that were 3+ years old and they've mostly worked fine. Occasionally blocks are missing, which of course is more likely the older an NZB gets, but _most_ of the catalog is still together.

They're right that many providers are getting hit with notices, which causes them to remove a few chunks of each rar from their server making it impossible to complete the download and too pervasive for repair by typically-sized parity file, but this can be circumvented by using a lesser-known news server as a backup; you still use Astraweb or Giganews for 99% of the transfer, but your fallback picks up the pieces that have been DMCA'd out of AW/GN/another major carrier.

I would be happy to see a more intelligent splitting system than rars, as one missing block in a couple of rars will often make it really difficult to extract the content you DO have and use a more robust delivery system, like a torrent, to download the blocks you're missing. This is going to become increasingly important.

News servers are interesting because they really are a non-optimal method for this kind of transfer that has incidentally become a hot bed for filesharers, probably just because of the plausible deniability ("yes officer, of course we only intend our server to be used by those discussing photography..."). I too wonder if it's not time for a new protocol, something more direct than torrents (webseeds kinda works here, but not quite what I'm looking for) but less hacked-up than NZBs and ASCII-encoded binaries in split files.

It's time to stop skirting around the issue and try to put together a serious underground analog to direct-downloaded binaries, something hearty and immune, or as immune as possible, to interference and foul play like this.

</comment></comment></comment><comment><USER12> This is a major blow to usenet. But more importantly this could be the start fo something very very bad for usenet. It, and it's users of who I am sure there are plenty on HN, have so far been able to fly under the anti-piracy radar of the big media companies. This clearly demonstrates that they know exactly what is going on, and may be on the war path to shut down more indexes and search engines.

<comment><USER2> Nauseating. Piracy killed the real Usenet, and now pirates are upset that enforcement is killing the goofy little playground they built inside its corpse.

When I was 18, I ran a competitive (on the Freenix leaderboard) full-feed Usenet server for the ISP I worked at. Every ISP in the world could have provided full-feed Usenet access, but for the assholes who loaded the system down with ASCII-encoded binaries. Even while it was possible for an independent provider to offer Usenet to customers, it was still a total nightmare because of the ludicrous storage requirements for binaries, which ensured that only an ever-dwindling number of providers would take the time to offer it at all.

It's startling to me that there's a Usenet at all anymore, since it's essentially been reduced to a collaborative system for sharing pirated binaries.

<comment><USER13> I like the imagery.

So I am chuckling at the idea that some young guns who want to be "free" to do what ever they want in their network of computers to come up with a scheme where they are will use telephones to call up one computer to the next, and addressing will be free form a series of "hops" where you tell the computer what sequence of machines will have to be called in order to get your message from you to your destination. You could use a character like ! to separate the hops making an address 'bobs-machine!piratebay!alices-machine!alice' delivering through three hops to Alice.

<comment><USER14> Honeydanber + Node.js. Ugh, can't wait.

</comment></comment></comment></comment><comment><USER9> Can anyone explain what this was, for us who didn't know it?

<comment><USER15> Think of it like Pirate Bay, but for newsgroups. Newsgroups are massive with some providers archiving binary groups for up to 4 years (and growing). Sites like NZBMatrix index the content and provide nzb files which tell your client what to download so that you don't have to constantly download the headers for a bunch of sites and search through them. Sites like NZBMatrix were also somewhat curated to help keep passworded files out of the results.

<comment><USER16> And combined with a private usenet server, it was non-P2P and relatively secure file downloading at exceedingly high speeds.

<comment><USER17> The usenet servers live on, and all the files (minus a small percentage that are the subject of takedown requests) are still there.

NZBMatrix was just one of dozens of indexing sites.

</comment></comment></comment></comment><comment><USER18> This is awkward... When newzbin shut down, I was without a NZB provider for a week. Then 2 days ago I paid for my membership at NZBMatrix. And now it's gone?

<comment><USER19> Similar position.. but I was waiting a week or two before committing to actual money :) Glad I did. Going a bit old school at the mo, downloading headers from a.b.mm for example. Get some retention up myself, might take a while!

</comment></comment><comment><USER20> The second Newzbin also closed this week: <http://www.newzbin2.es/>

Apparently their last payment gateway wouldn't deal with them any more.

</comment><comment><USER21> Despite being 99% illicit content, could somebody explain why this is not a case for Safe Harbour? Is this principle enshrined in any law at all, or is it just some neutrality idyll that we assume others share?

*Thought it was part of the DMCA

<comment><USER22> The "safe harbor" includes more requirements than responding to take-down requests. In particular, the service provider must "not have actual knowledge that the material or an activity using the material on the system or network is infringing", and "in the absence of such actual knowledge, is not aware of facts or circumstances from which infringing activity is apparent".

In other words, though you don't have an active duty to police your site, if you do become aware, and don't take action, you're not protected. Similarly, if you set up something with the sole purpose of distributing copyrighted material, and try to hide behind a "gosh, we never knew _that_ kind of thing was going on until you told us", you're probably not going to get a court to believe you.

<comment><USER21> I was commenting on this sentence, in particular:

>Once this notice is completed we are left with an impossible task of policing our indexing bots. Even then it won't stop there, there will be follow-up notices etc.

This seems to me like they're being told to actively police automatically- indexed content after this notice is processed?

I'm not sure how provable that 'knowledge of infringement' clause is.

<comment><USER22> "knowledge of infringement" basically boils down to "you know damn well what you're running here". It's not that every site operator has to actively police every single submission for possible infringement; it's that if even a ten- second glance at your service reveals obvious massive presence of infringing content, you don't get to pull the "well, gosh, we never knew" and claim safe harbor (this is the "is not aware of facts or circumstances from which infringing activity is apparent" clause).

</comment></comment></comment></comment><comment><USER23> I guess in this era of Bitcoins and Tor hidden services, it's inevitable that a site using credit card funding and a public IP address for piracy is being shut down. Low-hanging fruit.

</comment><comment><USER24> Binsearch [0] usually does the trick for me. Admittedly I'm not a hardcore user.

[0]: <http://binsearch.info>

</comment><comment><USER17> Never understood the appeal of NZBMatrix. Just use a search engine like Binsearch, NZBIndex.nl, etc.

In any case, with more and more takedown requests being directed at usenet providers, I think usenet's days as a safe haven for piracy are numbered.

<comment><USER4> Binsearch is much harder to navigate.

<comment><USER25> Coupled with <http://abteevee.allfilled.com> and <http://abmoovee.allfilled.com> it's easy.

<comment><USER17> If you're looking for scene releases, just use a pre database like preDB.com. Find a release on preDB, copypasta the release name to binsearch, create the NZB, done.

Generally speaking it's very easy to find material just using binsearch. If you want a 1080p copy of a particular film, just search for the film name along with '1080p' and you're almost guaranteed to get relevant results on the first try.

</comment></comment></comment></comment><comment><USER26> I had noticed almost everything had been failing over the past few weeks with NZBMatrix...

Back to torrents / DC++

<comment><USER21> I haven't experienced any missing binaries or bad NZBs recently, neither from NZBMatrix nor Astraweb. Can anyone corroborate this?

<comment><USER4> I was getting a good amount of missing blocks on Astraweb only; I bought a backup news account from blocknews and haven't had any missing block issues since.

It helps if people put big pars out.

<comment><USER21> I'll not jump ship until my downloads are frequently disrupted at the block level, across multiple providers. Pretty difficult to achieve, given the architecture.

Indices come and go... Much of the Usenet crowd know how to set up an indexer if need be. Perhaps a hidden service is warranted?

</comment></comment></comment></comment><comment><USER27> Question for those with Usenet know how. When downloading raw headers, I have observed a convention where the title is a random string of letters and numbers, and within the post are for example 50 rars and 10 pars. How are consumers 'decoding' the title so they know what they are getting? Does the uploader provide a translation? Or can the 'real title' be recovered programmatically? Many thanks.

<comment><USER28> Can you give an example?

<comment><USER27> Here's one from a.b.boneless...

"2TU4K75C6Z" (45 rars, 9 pars)

And here's one from a.b.multimedia...

"DTTBiAp23.101.by.NYCrules" (14 rars, 11 pars)

no .nfo included and no way to know what's inside. Does anybody know how people are decoding these titles into meaningful information?

<comment><USER27> Here's one from a.b.cheerleaders...

"291112_08_29" (60 rars)

There are lots of these and they are interesting because the first part suggests a date, 29/11/12, and the second part suggests a sequential number. The size is almost 6 gig.

<comment><USER28> Aaaaand I'm glad I didn't look that one up on the work PC. I'm assuming you can get the filename info by looking at either the RAR headers or the first archive.

</comment></comment></comment></comment></comment><comment><USER29> running your own custom tailored usenet indexer seems pretty easy nowadays thanks to <http://www.newznab.com/> \- if you have a need for one in the first place, that is.

</comment><comment><USER30> I've never used torrents and I've always used newsgroups. Does that make me old? Anyway, I feel naked now, with newzbin down and now this I feel I'm going to have to end my fun or try torrents which I'm worried about the legal issues. Are torrents "safe" now a days, what's the best approach (Seedbox?)

<comment><USER4> Torrents aren't really safe or unsafe, it's mostly about the quality of the trackers you use. There are some OK cultivated communities out there, but the worry over ratio and constant leech on one's upload stream is a real bummer when you can just purchasing blocks from a news server and not have to worry about it. A seedbox is way more expensive than a news account and usually comes with quotas which one must mind.

All in all, nzbs are a much more pleasant experience if you can afford the account (and pretty much any employed person can) and have a good indexing site (harder to come by, and now I don't have one either :( ).

<comment><USER31> Wouldn't connecting via a VPN sufficiently hide your tracks from the secret squirrels?

<comment><USER11> No. You should watch this: <http://www.youtube.com/watch?v=9XaYdCdwiWU>

</comment></comment></comment></comment><comment><USER32> Open source the code. Id gladly run my own private nzbmatrix

<comment><USER33> Feel free.

<http://newznab.readthedocs.org/en/latest/>

<comment><USER32> You've derailed my Sunday. Thanks. Is this the nzbmatrix core?

<comment><USER33> No, it's a GPLv3 clone of an nzbmatrix-esque site. Somewhat popular sites like nzb.su use it. It has a Sickbeard compatible API too.

</comment></comment></comment></comment><comment><USER34> This is a sad day.

<comment><USER32> I really feel some type of way about this. This site (and netflix and hulu) allowed me to live without tv for almost four years

</comment></comment><comment><USER35> Move the site to TOR and let digital copyright enforcement cops try to figure that one out. All they can do it push it completely underground and once the site is on TOR they have zero recourse or means to track down the operator.

<comment><USER36> Do not do this. The Tor network cannot handle that kind of traffic.

<comment><USER4> The site is just text transmission, like torrents. Your connection to the news server would not have to be piped over Tor, just your access to the web interface where you download text files telling your news client what to download.

<comment><USER36> Right, sorry. I misinterpreted the parent.

</comment></comment></comment></comment><comment><USER37> Such disappointing news compounds last weeks troubles. When nzbs'r'us shutdown over similar payment problems. Payment is a huge achilles heel for these services. Blocking payments was used to hurt WikiLeaks, and Dutch usenet providers have recently lost the ability to use paypal for payments too. A popular bitcoin-esque service can't happen soon enough.

[http://usenetreviewz.com/dutch-usenet-providers-banned-by- pa...](http://usenetreviewz.com/dutch-usenet-providers-banned-by-paypal/)

<comment><USER37> I like that Europe scolded visa for stopping payments to wikileaks, but that is unlikely to ever happen for anything piracy related.

[http://falkvinge.net/2012/11/20/europarliament-scolds- visa-m...](http://falkvinge.net/2012/11/20/europarliament-scolds-visa- mastercard-paypal-for-killing-wikileaks-donations-initiates-regulation/)

</comment></comment><comment><USER38> alternative: <http://www.nzbsearch.net/>

<comment><USER39> That's not really an alternative..

<comment><USER38> why not? admittedly, i didn't use nzbmatrix, but i do use nzbsearch and the ui is simple, easy to use - search is _ok_ , but it seems to get the job done.

<comment><USER40> Sounds like NZBMatrix had an API for auto-download clients to consume. If NZBSearch doesn't have that it's not really a replacement.

</comment></comment></comment></comment><comment><USER41> The masses need to pick a new site to invest their energies in, rather than scatter at random to lots of different sites! Any particular one out there that's better than the rest?

<comment><USER42> As someone who hasn't used usenet for downloading content, why is it that it seems that a few central indexing services are the way people access this medium? Wouldn't it be possible - and preferable - to distribute the index through p2p? That way, there would be no single point of failure.

<comment><USER43> The index would be insanely huge. A lot of these indexing sites are used for their API. Dedicated applications like SickBeard automatically compile lists of wanted files (movies, TV rips and music) and use them to search against the provider. The provider returns a file (NZB) that contains references to hundreds of encoded files hosted on newsgroups.

Here's a breakdown of one NZB, for a single 3.88GB file.

Inside are 74 RAR files, and 15 additional PAR files containing parity for recovering corrupted data.

Each of these 89 files is split into 37 parts of 1.4MB for ~3200 pieces total.

These pieces are then encoded with yENC (similar to base64) and then uploaded to the newsgroup.

Every single one of these yENC pieces is referenced in the NZB file, making for a total of 400KB. Just because of this, the NZB indexs are enormous; hundreds of gigabytes, probably terabytes in some cases.

<comment><USER44> There's no need for everybody to serve the entire index. Set up a DHT and let interested parties exchange the NZBs. All problems in computer science can be solved by another level of indirection.

<comment><USER45> it's a reputational and data quality issue as well. completely decentralized content distribution doesn't usually work unless there is some kind of curation, there are just too many motivations for disruption. it's not an impossible problem, but long ago ceased being mostly about the technical occupation of shipping bits between nodes.

<comment><USER40> You could always have sites that only serve up indexes of NZB files (e.g. "NZB file => SHA-256"). Then you could use these to determine which NZB files to trust. I would assume that distributing a "NZB => Checksum" hash would be a lot smaller/easier than running an entire site on top of a terabyte of NZB files.

</comment></comment></comment></comment></comment></comment><comment><USER46> Is USENET anything other than file sharing (mostly piracy) now?

<comment><USER47> I know that comp.lang.lisp is still active.. I assume there are plenty of other groups still active as well.

</comment></comment><comment><USER48> I'm currently using www.nzbzombie.com and its been great , has an API that works with CouchPotato & Sickbeard and phpBB forums.

</comment><comment><USER49> you can bet your bottom this fact''Federation Against Copyright Theft Limited''will be into all nzb providers now by attacking the index list making sites like nzb matrix take down massive amounts of copyrighted movies and other copyrighted items, changes in the usenet indexing this looks bad all round for us downloading they cant get the server providers so they will go after the next best thing the index system that list content.

<comment><USER50> And why aren't they able to get the server providers?

<comment><USER51> It was probably cheaper to force or induce the closure of index sites that are done more as a love of community than of profits, not make nearly as much as the usenet providers themselves. These index sites, I'm sure, understand the legal costs and risks of continued operation if those options were presented to them. With the majority of them down and out of the way the providers themselves will see a reduction in revenues and may not put up a fight when they fall into legal pressure to close up shop.

</comment></comment></comment><comment><USER52> A bunch of fucking assholes. They took my money the day before. Wish I could get my hands on them....

</comment><comment><USER53> a lot of chat about "we never use them and get by" ,,how about some feedback for alternatives!!!!! i have only ever used newzbin ,,then yesterday paid matix ,and today not sure where to go?

</comment><comment><USER54> Welcome to the NZBMatrix.

I don't care who you are, if you see an agent, you do what the rest of us do. You run. You run your ass off.

</comment><comment><USER55> MacGuffin!! Noooooooooooooooooooooo!!!!!!1

<comment><USER56> Yup. Mr. "a:10 v:10"

<comment><USER32> Those were the most helpful comments. You'd see a four on either metric and know not to waste time.

</comment></comment></comment><comment><USER57> There is NO such thing as priracy, there is only sharing of property between people. When I buy something I own it, no matter what those fools say, and when I own it, I copy it if I like!

<comment><USER51> And thus we explore a brief glimpse of the tragedy of the commons.</comment></comment>