Twitter's garbage problem - <AUTHOR> http://jmillerinc.com/2010/05/16/twitters-garbage-problem <comment><USER2> I can't believe nobody has mentioned naive Bayesian text classification yet. It sounds like it could work wonders for Twitter. I'm much more likely to be interested in tweets with words like "hylomorphism" than tweets with words like "omglol", and a text classification algorithm could learn that if you trained it up some. It doesn't have to be perfect; it just has to improve the signal-to-noise ratio significantly.

<comment><USER3> I've looked into this a bit, albeit more in a spam-filtering context; tweets have very little text for naive Bayes to latch onto. 140 characters would be 20-30 words, tops. That is so few words that it is hard to move the prior very much, unless there are blockbuster words that almost _always_ indicate a bad tweet; as the article suggested, "breakfast", "beer", etc.

<comment><USER4> Filter by account, then, instead of by post? With a sample body of, say, the ten or twenty of an account's most recent posts.

<comment><USER3> The article's whole premise was that tweet quality does not correlate well within an account; e.g., some marvelous twitter streams include breakfast tweets.

</comment></comment></comment></comment><comment><USER5> I've always thought of this problem in reverse for both Twitter and Facebook. I have certain followers/friends that are interested in my thoughts about programming and business and others who would care more about where I'm going this afternoon. It would be nice if there were different publishing channels I could publish to different friends.

<comment><USER6> You can do that on Facebook: when posting a status update, the drop-down box under the lock icon has a "Customize" option, which will pop up a dialog where you can select specific people or pre-created lists of friends for it to be visible to (the lists function essentially as channels, like "colleagues" or "family"). You can also set one of those channels as the default. I know a few people who do that fairly regularly to separate friends vs. colleagues vs. family updates. But it's a bit of a hassle and sort of buried.

For Twitter, I know a few people who have two Twitter accounts, a "work" and a "personal" one. Also somewhat of a hassle, though maybe actually less of one.

<comment><USER7> That's close, but not exactly the same thing as selecting people to publish to. If I set a post to go to the "Tech People" friend list on Facebook, none of my other friends can see it, nor can the general public. I'm okay with everyone seeing it; I just don't want to pollute people's news feeds with things I know they won't be interested in.

</comment></comment></comment><comment><USER8> In all honesty, I always got the impression that the point of twitter was to tune into other people's noise, not to harvest useful information.

<comment><AUTHOR> That's how I view Facebook. Twitter on the other hand is rich with useful information.

</comment></comment><comment><USER9> Sometimes I really miss Jaiku. There you could add multiple streams to your feed, but when following, could also decide to follow certain streams, or not. So I could follow someone's post, but not their flickr pics or delicious bookmarks. Comments on a post where separate from posts as well. Twitter makes this all one big mess. Let's hope annotations will be used to add this filtering (although I haven't seen post client, which is an annotation as well, used for this).

<comment><USER10> That's about what I envisioned while reading the article. All you'd need is different sections on Twitter, and to choose which to subscribe to when following someone. All the clients would need to know is that you're subscribed to X's stream 0 but not their stream 1.

</comment></comment><comment><USER11> I've never met the author of the article, but I assume he's the type of person who is bothered when his Reader unread tally switches over to the plus mark.

I mean, honestly, tweets are 140 characters or less. The average tweet takes a handful of seconds to read. Is my time so important that I can't spend a few minutes of my day learning what my friends thought was important to share with me? Must I tailor their interests down to only those that I deem relevant? Am I so bad at skimming content or choosing which content is worthy of in-depth inspection that I must have a computer do the editing for me?

Obviously, the answers to those questions are highly subjective and use- dependent. Personally, the only editing I need is the unfollow button. If I respect a person enough to want to hear what he has to say, I gladly take the risk that sometimes his output won't be immediately relevant. (As an aside, a year ago I met one of my favorite journalists. I asked how his dog was, since he had been tweeting about his new puppy. It was a nice ice breaker. I didn't follow him for dog-training updates.)

I should make a disclaimer: I'm not a heavy Twitter user. The ratio of feeds:twitters followed for me is something like 10:1.

<comment><USER12> I respect your sympathy for and willingness to read what those you follow deem interesting. But I think the author of the blog post is speaking of a problem that emerges at a larger scale when you follow more & more people. Your stream becomes bigger while the amount of time you have stays the same so your options are 1) unfollow people, 2) don't read everything, or 3) put in more time. Personally, I don't like 1 or 2 because I feel like I'll miss out on timely, relevant information and 3 isn't an option for most of us. Thus, the need for a solution like the one the blog post describes. Shameless plug: here's how I'm trying to fix the problem - <http://slipstre.am/>

<comment><USER11> I agree with all of that, except I've found 2 to be the most useful, especially with regards to how I manage Reader. My time spent in feeds has been a lot less hurried since I learned to stop worrying and love the Mark as Read button.

I just feel like I personally get information in so many different ways, that if something is big enough, I'll see it in many places and am pretty much bound to read one of them. I can count on seeing any big tech story here on five different feeds, Google News (go to time-killer on my phone in the rare occasion my feeds are empty), several twitter accounts, and probably here on HN (more like 15 feeds if it has Apple in the title.)

I went to your site but you lost me with the lack of content--maybe at this stage that's important to weed out the laziest of the "testers." Have you thought of doing just a quick before/after screenshot to show default twitter vs. your application, or is it too early for that?

Edited to add: er, my fault. My brain skips over flash almost automatically. I didn't watch your video.

</comment></comment></comment><comment><USER13> The author misses a trick (or I missed a mention of it). Filtering out by the _client used_ by the third party helps a lot. You can immediately filter out tweets coming automatically or semi-automatically from systems like Gowalla, Foursquare, blip.fm, Sharefeed, last.fm, auto news posting services, or even just through the Twitter API. Anyone else who posts crap in a manual, deliberate way should just be unfollowed.

</comment><comment><USER14> This isn't _Twitter's_ garbage problem. It's the garbage problem of the people this guy follows. Seems to me that building out a complicated system for channeling different tweets would hardly be worth the resultant complexity to Twitter and their users.

Is it so much to ask to employ a little restraint in publishing, and on the other hand, a little taste in following?

<comment><AUTHOR> I disagree. Some of the tweets I subjectively label as garbage might be legit. Hey, maybe some guy's mom is on twitter and wants to know what he had for lunch. It's not a people problem, it's a drawback of the platform that all of this different subject matter has to be broadcast in the same stream.

<comment><USER15> It's a people problem, but the problem is you. If you don't care about what he's having for lunch, then I'm not so sure what's so hard about ignoring his tweet. It's only going to be on your screen for a split second as you're scrolling through other stuff. Presumably anyone you follow would post more things that are worthwhile than not; otherwise, you should just find new people to follow.

One of my friends tweets quite a bit about Magic: The Gathering. I don't care about that. Somehow I get along just fine by not paying much attention to the posts I don't care about. I don't quite understand your dilemma.

<comment><AUTHOR> I value my time and believe in improving things. What more can I say?

<comment><USER15> What I intended to convey was that your mind already possesses a filter for this type of thing that you could never hope to reproduce in software. I just don't understand what's preventing you from unconsciously applying it, since plenty of other people seem to be doing perfectly fine at doing that.

</comment></comment></comment></comment></comment><comment><USER16> I would post much more "garbage" if there was a decent filtering system. I only post substantive tweets as I'm worried I'll annoy people with useless stuff. Why would someone care about where I'm eating or what movie I'm watching if they don't even know me. So I keep it focused on purely technical topics.

</comment><comment><USER17> I honestly don't see this as a problem. I don't read every Tweet that comes through my stream its kind of random access information, I look at twitter every now and then and if something interesting strikes me I look into it.

While filtering would be a good feature, I think saying its a killer feature is going a bit far. I don't think the reasoning that you could follow twice as many people would make much sense from a roi perspective. If your interested in roi, you wouldn't even worry about following people, you would create custom searches about topics you are interested in and deal with those. You would certainly get more info for your time spent on twitter that way and much more focused. You can do this with basic tools like Tweetdeck search columns or even or saved searches in the standard twitter web interface.

<comment><AUTHOR> Custom searches don't capture all of the good information that's out there because I may not specify the right keywords. It's ANTI-search that I'm personally looking for.

<comment><USER17> You are not going to catch all the good information that is out there with your follow list. So searching fills in the gaps(if your interested in a subject area or something). Twitter lists help too. (ie/ gather up all the people who are known to talk about x subject and put them in a list)

</comment></comment></comment><comment><USER18> I have the same problem with Hacker News. I would love to be able to filter it based on my preferences.

<comment><USER19> If you caught the thread on whether _search_ should be available on HN, and pg's response, I feel pretty safe saying that ain't going to happen, ever.

<comment><USER18> Thanks. I'm okay with that. Hacker News seems more like a labor of love than a monetizable product, so I don't expect pg to focus on it.

</comment></comment></comment><comment><USER20> Anyone else find the "noise" posts less banal than the "signal"?

<comment><USER21> My thoughts exactly. "quote from a blog post that everyone read three weeks ago" <http://shrt.rl/omgaweomse> #imanentrepeneuroneoneone!!

I read Twitter because I like to see what "meaningless" activities my friends are up to, not because I want a rehash of HN.

</comment></comment><comment><USER9> BTW, twitstat mobile, <http://m.twitstat.com/> has some of this filtering in place: don't show 4sq, etc, if you want.

</comment><comment><USER22> I totally agree and was just contemplating the same topic. Spooky.

What gets me is even some of Twitter's own employees don't know how to use the service in a 100% useful manner. And you would be surprised who....Tweets like: "He said that?" or "Can't wait to see it" without references are GARBAGE. I also don't care that you are currently eating or thinking, in general, without learning SOMETHING in the process. They don't teach that at the company? In this age of videos, location-based tagging and pics, we have the power to do so much more than just talk.

Finally, I think this is an author controlled issue. We have the power to dictate what we write/ tweet and should respect our audience for listening/ following us. Unless we don't understand the tool, then it's a training problem.

I, @super74, try to direct message any personal or conversational tweets to only those who know what I'm talking about. My public messages tend to lean towards disseminating information, sharing creative ideas and offering my opinion on public topics.

Although I'm not perfect and may err from time to time, I have occasionally looked back and have been somewhat pleased with the bulk of my tweets.

</comment><comment><USER12> Excellent post! I came to the same conclusions in February and am working on fixing the Twitter garbage problem (what I see more broadly as information overload) with Slipstream: <http://slipstre.am/>

Would love your thoughts here or over email: firstname.lastname@example.com

</comment><comment><USER23> >A first approach is to simply filter out tweets by keyword. I think of this as anti-search: specify a keyword, and never see any updates containing that word.

>Keyword filtering alone could probably solve 1/3 of the Twitter garbage problem

This sounds like an old problem, reiterated to me, with working solutions: [kill files](<http://en.wikipedia.org/wiki/Kill_file>) in news reader programs, especially [score files](<http://en.wikipedia.org/wiki/Score_file>).

</comment><comment><USER24> If you are interested in particular subject matter, as opposed to genuinely interested in the people you follow, just run a search or advanced search. Both auto-update and provide filtering functionality.

</comment><comment><USER25> I can't tell which of the examples are supposed to be the valuable ones...

<comment><USER26> Oh, come on:

 _"A startup is a human institution designed to deliver a new product or service under conditions of extreme uncertainty."_

 _"I just ousted Owen S. as the mayor of Samovar Tea Lounge on @foursquare! "_

Clearly, the _second_ tweet can possibly be informative to _some_ people.

</comment></comment><comment><USER27> Would be nice too if I only saw an RT once, not each time someone else on my list RT's it.

Heck, why not update the RT'd by info to include several people - to make it clear. But please, only show me the thing once!

Only a slightly different note, would be nice if twitter would store the "read" status - so that my different clients know what I've read and what I haven't. If it was core twitter functionality then different clients could all sync together nicely.</comment>


Technololgy Companies. We Hate You. - <AUTHOR> http://www.whattofix.com/blog/archives/2009/03/technology-comp.php

<comment><USER2> What!?

Alternative: Go to the Apple store and buy a MacBook. Turn it on. You're done.

<comment><USER3> Wait, doesn't the macbook cost the same amount of money for what he just went through?

macbook $ = Articles $.

Hmm. How much is your time worth....

<comment><USER4> Regardless, even if the MacBook cost more, at least you don't have the do the following:

\- Install Norton, or any sort of anti-malware anti-spyware crap. A stock machine out of the box will work and be reasonably secure for the home user.

\- No need to image the HDD before giving the machine to the user... Because unless your grandma suddenly learned how to sudo, the odds of them crapping up their partition is really low.

\- No need to order instructional material on Amazon. Apple's website provides a large number of introductory videos that show a user how to use MacOS X.

My mother still hasn't quite learned how to use a computer. My hope is to get her a Mac in the next year and maybe finally she can check her own email :)

<comment><USER5> _My mother still hasn't quite learned how to use a computer. My hope is to get her a Mac in the next year and maybe finally she can check her own email :)_

When I used Windows I never used Outlook. I didn't actually think other people ever used it. Even though it was built-in, I just assumed that Windows pre- installed software was roundly worthless.

OS X does the opposite: they really place emphasis on their built-in programs. As a result, I tried every program they gave me, and found myself liking almost all of them. I don't think I could switch back from Mail to Gmail. Perhaps I'd have liked Outlook, too, but Microsoft didn't give me that same incentive to switch.

<comment><USER6> > When I used Windows I never used Outlook. I didn't actually think other > people ever used it. Even though it was built-in, I just assumed that > Windows pre-installed software was roundly worthless.

You're confusing Outlook with Outlook Express. Outlook does not come with Windows, only with the Office suite.

edit: although fwiw, outlook express is worse than worthless.

<comment><USER5> Aha, gotcha. So I guess I wasn't missing out on much? (In any event, I think now if I had a Windows machine I'd get Thunderbird, which never struck me as a useful idea last time around.)

</comment></comment></comment></comment></comment></comment><comment><USER7> I found the last bullet really interesting. At first I thought "wait, UPS isn't a tech company" but when you think about it, any time you buy something on the internet, you're giving money to UPS (or a competitor).

<comment><USER4> Actually, interesting anecdote... my former roommate once had to ship a load of stuff (100+ lbs) from the US to Canada via DHL. DHL quoted him about $1000 for the whole thing.

Then we went through our employer, who had a deal with DHL... $70 and change.

<comment><USER7> A friend of mine used to ship things to his office for much the same reason, it was cheaper to ship a large load to a business address (something to do with rates for freight). Once he ordered ten huge CRT monitors, and then sold nine of them to his friends. Ended up both getting a new monitor and making a profit.

</comment></comment></comment><comment><USER8> I noticed he didn't bother to cancel is wacko internet account.

And he wonders why things are so bad...

<comment><AUTHOR> "he" is stuck with wacko internet account because he lives in a place where you have 2 options: dial-up, or wacko.

<comment><USER8> Heh, didn't notice you were also the submitter. Sorry. I feel for you, but seriously, no other options? Wow.

Could I ask then why you bought Norton? I've installed avast on my mother's machine and haven't heard from her since, and it was free.

Also, as much as I hate the phrase "buy a mac", I'm convinced that all non- tech users should be using one of these machines.

<comment><AUTHOR> Macs are cool, and if the old folks in the park where mom lived all had Macs, I'd be buying her one. I'm hoping that she'll get a lot of support from her peers - especially since I'm 1200 miles away.

Norton was probably the worst choice on my part, and it came from habit. I've used Norton for a long time, and it has a good reputation. But you're right: there are much better options for security software.

<comment><USER9> Can you return it?

I gave my mom a Macbook 5 years ago and she's never had to ask anyone for support.

<comment><AUTHOR> Probably not. I spent all day "configuring" it -- resizing the icons and text size, doing all that install stuff, imaging the drive, etc. Plus it's shipped.

How old was your mom when you got it for her? Did she have previous computer experience?

<comment><USER9> She was 55 and uses a PC at work.

<comment><AUTHOR> I think that's a little different. My mom is 75 and the last keyboard she used was a selectric typewriter sometime around 1990.

So we're starting completely from zero.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> simple solution: go out of your way to use and promote smaller/better companies wherever possible.

i go out of my way to avoid giving money to best buy, norton, microsoft, etc..

these companies won't get better if you complain about them but continue to give them money. feed their competitors and you might see change.</comment>


Should *Mozilla* Fork Firefox? - <AUTHOR> http://www.computerworlduk.com/community/blogs/index.cfm?blogid=14&entryid=2967

<comment><USER2> Of the people I know who have switched from Firefox to Chrome, the reason is always speed. Chrome is undeniably faster than Firefox. Unfortunately for the author of this article piece, the problem isn't putting a new frontend on Gecko, the problem is that Gecko isn't fast enough.

I haven't been involved with the project in years other than following the blogs of key developers, but I've always been under the impression that the platform guys had the latitude to do whatever they wanted but making big changes for unknown gains is undesirable. This is very different from the Netscape controlled bureaucracy that led to the m/b fork.

<comment><USER3> When people actually test these things I think they generally show that Chrome isn't undeniably faster than Firefox.

Chrome wins on Javascript execution and Firefox currently has major startup problems because of a large codebase and poor UI that continually interrupts you and asks for restarts.

But Firefox wins, against stereotype, for memory usage and page load speed (more so with ad blockers installed).

They've clearly lost the faith of many vocal geeks, but I'm really not sure how much of that is based on fact rather than some kind of herd instinct.

<comment><USER2> > But Firefox wins, against stereotype, for memory usage and page load speed > (more so with ad blockers installed).

I'll agree on memory, and that's one of the reasons I continue to use Firefox but memory is not speed. It's my impression that Chrome wins on JS execution and DOM manipulation/reflow, which is really what I are about as a developer and (generally) as a user. When you're on a low end machine, these make a big difference. When you're on a machine like most developers have, it really doesn't except in technology demos.

The Jaegermonkey and Layers work should help out, but I'm not sure that Firefox will ever catch Chrome. It's a difference in development philosophy. Webkit reverts ANY performance degradation while Mozilla reverts all performance regressions except for new features. I prefer Mozilla's philosophy since the places where this matters is generally a performance vs edge case correctness tradeoff.

</comment></comment></comment><comment><USER4> This could be seen as a success of chrome, given Google's stated reasons for releasing the code:

 _Primarily it's because one of the fundamental goals of the Chromium project is to help drive the web forward. Open source projects like Firefox and WebKit have led the way in defining the next generation of web technologies and standards, and we felt the best way we could help was to follow suit, and be as open as we could._

A bit of competition has prevented the market from stagnating again.</comment>


Random Image Experiment Reveals the Building Blocks of Human Imagination - <AUTHOR> http://www.technologyreview.com/view/532231/random-image-experiment-reveals-the-building-blocks-of-human-imagination/

<comment><USER2> Wow, this one really rocked me back on my heels. Its an interesting way to extract the 'feature codes' that people use in recognition. Going from there into a set of properties that you can feed back into an analysis system would make for a fun machine learning exercise.

</comment><comment><USER3> Reminds me of "Self-Organizing Maps" [1]

1\. [https://en.wikipedia.org/wiki/Self- organizing_map](https://en.wikipedia.org/wiki/Self-organizing_map)</comment>


Your degree determines your salary: Choose wisely - <AUTHOR> http://www.independent.co.uk/news/education/education-news/education-degrees-that-almost-guarantee-jobs-1295023.html

<comment><USER2> Not just this, but your GCSEs and A-Levels too.

I was actually turned down from a graduate job because my GCSEs were not all straight A, despite my A-Levels being so.

Although this might seem a bit ridiculous from a graduate point of view, the market has become so saturated with graduates. So much so that these companies can be VERY selective, for no other reason than to get the number of people's CVs they have to look at down to a manageable amount.

Additionally, just because you did a History degree, doesn't mean you have to do something USING that degree. University is an academic and educational pursuit, it isn't an apprenticeship.</comment>


Apache CouchDB is Alive and Kicking - <AUTHOR> http://email@example.com/msg18043.html

<comment><USER2> maybe you should have tagged it more like "Official response to Canonical dropping CouchDB" ;-)</comment>


(2003) Burning Buried Sunshine: Human consumption of ancient solar energy - <AUTHOR> http://globalecology.stanford.edu/DGE/Dukes/Dukes_ClimChange1.pdf

<comment><AUTHOR> A good summary at The Economist: <http://www.economist.com/node/2155375?Story_ID=2155375>

TL;DR: A single year's fossil fuel consumption represents 422 years of ancient plant growth, only 10% of which is converted to petroleum or coal. Replacing fossil fuel use from direct use of modern biomass would require 22% of current plant growth, on top of the 15% already used by humans.

This excludes converting biomass to other forms (biodiesel, ethanol, etc.), which would reduce end fuel energy by ~90% (or increase land area requirements tenfold).

The lesson: while liquid fuels are hugely convenient and valuable, they'll be very difficult to provide from biological origin. Humans will likely use much less liquid fuel, particularly in transportation. Air travel especially will be severely affected.</comment>


Adblock Plus has already defeated Facebook's new ad blocking restrictions - <AUTHOR> http://www.theverge.com/2016/8/11/12439990/facebook-unblockable-ads-defeated-by-adblock-plus <comment><USER2> Here is the Adblock Plus blog post on the work around ...

[https://adblockplus.org/blog/fb-reblock-ad-blocking- communit...](https://adblockplus.org/blog/fb-reblock-ad-blocking-community- finds-workaround-to-facebook)

Here is the new filter they are using ...

facebook.com##DIV[id^="substream_"] ._5jmm[data-dedupekey][data-cursor][data- xt][data-xt-vimpr="1"][data-ftr="1"][data-fte="1"]

<comment><USER3> I use facebook.com##div[data-xt-vimpr] to remove sponsored ads.

</comment></comment><comment><USER4> the internet arms race isnt going anywhere soon. but I think brave could be a game changer.</comment>


Kickstarter Freezes Anonabox Privacy Router Project for Misleading Funders - <AUTHOR> http://www.wired.com/2014/10/kickstarter-suspends-anonabox <comment><USER2> It's an interesting world we live in where Reddit and Hacker News users are the ones doing investigation into the truth of claims journalists publish.

With real-time feedback and real-time coverage, it's almost inevitable that our news sources will become increasingly less investigative and more reactive. The first to report wins the traffic, which creates pressure to get stories out the door. Unless you take The Atlantic or New Yorker approach, it's a race, which leaves users users to do the fact checking. We can be cynical about the state of online journalism, and maybe it's partly warranted, but this is definitely where we're headed.

<comment><USER3> Many of the "investigations" on Reddit..actually result in Internet mobs destroying the reputation of a company or individual..even if they've done nothing wrong (besides offend the anti-capitalist hive mind).

It just happened to have worked out in this case, but this is the minority from what I've seen in the past couple of years.

<comment><USER4> >(besides offend the anti-capitalist hive mind).

People love showering people they agree like with money, this case proves it. People also don't like to feel they are being taken advantage of and online you can find that out with a few clicks and 10 minutes of googling.

If someone gives $5 to a campaign that promises awesome wonderfulness but then finds out the awesome wonderfulness is being made in a Thailand sweatshop by 10 yearolds eating arsenic for breakfast they will pull the pledge and ruin the companies reputation.

Which is all to the good. Just look at Apple, they had their hipsters locked in when it was just the ridiculous 50% margins they were putting on their products. When we found out people were being worked to death and we had to pay those ridiculous margins that's when people became anti-Apple on mass.

Now if by anti-capitalist you mean anti-Dickensian exploration capitalist you're damned right. And I hope every half decent human being would be.

</comment></comment></comment><comment><USER5> Hooray. This project was misleading and naive in many ways, and it definitely seemed to violate Kickstarter's rules against resale. But that's not why I'm happy this project got killed.

The much bigger problem was the open SSL account and sheer misguidedness of running all network traffic through Tor. This project risked causing real harm to people thinking that this would safeguard their privacy.

<comment><USER6> It would pretty much flood the TOR exit nodes, would it not?

It seems like TOR isn't exactly a commercializable network.

<comment><USER7> It could certainly be commercialized if the company making the product funded exit node servers to offset the increased usage from their customers (X gigabytes worth of exit node traffic capacity for each customer or something).

<comment><USER8> On this note, is there a reputable group that create exit nodes that can be funded?

<comment><USER7> Some Googling reveals [https://www.torservers.net/](https://www.torservers.net/) which seems to be an European (German) charity that runs exit nodes and relays you can donate to and they seem to be funding a couple Gbit worth of exit node traffic ( [https://www.torservers.net/exits.html](https://www.torservers.net/exits.html) ) plus some relays.

I'm sure there are other exit node providers that one could donate to also. And of course, there's always the option of setting up your own exit node on a VPS -- find a Tor-friendly ISP and installation is basically an 'apt-get install tor' away (default configuration works fine). I run a few relays myself on my servers and it uses very little system resources and bandwidth is cheap.

</comment></comment></comment></comment></comment><comment><USER9> If nothing else, and it seems like this specific project could be salvagable at something closer to Germar's original $7,500 level, Anonabox has proven that there is large, consumer-level demand for online privacy.

A better (more secure) consumer solution may be developed by an established company as a result. More desirably, though sadly less likely, governments may take notice of what the people they serve are demanding.

</comment><comment><USER10> Anyone know where to get the Chinese board they were using? Seems pretty actually useful.

<comment><USER11> Better advice is to use the grugq's PORTAL: [https://github.com/grugq/portal](https://github.com/grugq/portal)

<comment><USER10> How is his a hardware platform?

</comment></comment></comment><comment><USER12> Wonder how many entrepreneurs are looking at this and seeing the awesome market validation of the idea. I know there's some debate around whether this is technically possible or not, but hopefully some smart hacker somewhere can figure those issues out and make a success with this.

<comment><USER13> Grugq has been working on this since -- 2012. He's had working code; it was just a problem of getting COTS hardware which would support it. Making custom hardware is a bad idea for this for a variety of reasons.

Grugq, Marc Rogers, and I have shown a (much more functional, actually secure) version of this, called PORTAL, at a couple conferences this summer, and will have a retail version on Amazon by December -- none of this presale BS.

Free downloads of the firmware for some of the most commonly available routers out there, and sold at cost ($20-25).

<comment><USER14> Sounds promising. Please create a website or other citable URL that others can use to promote it in advance of December.

<comment><USER15> See the following:

[https://github.com/grugq/PORTALofPi](https://github.com/grugq/PORTALofPi)

[https://www.portalmasq.com/portal- defcon.pdf](https://www.portalmasq.com/portal-defcon.pdf)

</comment></comment></comment></comment><comment><USER16> When I read this, "As the controversy around Anonabox grew, Germar told WIRED earlier in the week that he had never intended the project to be aimed at normal, non-expert users, so much as developers who would contribute feedback and continue to improve the router". What I hear is, "I never thought people would investigate me and call me out for lying".

<comment><USER17> I've said this in other threads on this but I think "lying" is an extreme exaggeration for what happened here. I've read the whole listing and I see nothing claiming that he designed the whole thing from scratch (and he would be an idiot if he had tried to, as all of the parts are commonly available).

Basically he offered a router that was pre-configured to use TOR. There are lots of people not comfortable with/capable of flashing their routers to use tools like OpenWRT or DD-WRT...and for good reason, as a novice could pretty easily brick their router. It's obvious by the response that a plug-and-play solution does offer value to some people, whether he painstakingly designed the hardware or not.

<comment><USER18> There was this photo with the caption that only generation 1 was stock hardware, heavily implying the rest was custom designed: [https://s3.amazonaws.com/ksr/assets/002/617/995/b560c4715481...](https://s3.amazonaws.com/ksr/assets/002/617/995/b560c47154817abfd7e570c4e78e7b17_large.jpg?1411333699)

Right below it is a statement "By our fourth round of prototypes we had created a model with 64mb memory and a 580mhz CPU" which uses the word "created" not "found" or "utilized".

Then there was this AMA on reddit: [http://www.reddit.com/r/anonabox/comments/2ja22g/hi_im_augus...](http://www.reddit.com/r/anonabox/comments/2ja22g/hi_im_august_germar_a_developer_for_the_anonabox)

In it he says things like:

1\. "I am claiming my device is not the same as the wt3020 off the shelf hardware yes." Self explanatory. He then states that "maybe" the only difference is the amount of storage, which is odd considering he just denied it's the same thing.

2\. "Yes that is my picture and yes, I am claiming that the anonabox is not an off the shelf WT3020" Ditto.

3\. "Yes honestly that does look like the same circuit board. I can't help but wonder if the factory that we sourced is going to try to sell them too." Here he even infers that their design was stolen and being resold behind their back. Again, this heavily infers they created/designed the board, and clearly denies knowledge of any other existing boards like his, meaning they didn't simply take an existing board and ask for more storage.

THEN he starts to walk it back, admitting to Wired they didn't design the board as they had first claimed as noted here: [http://www.wired.com/2014/10/tiny-box-can-anonymize- everythi...](http://www.wired.com/2014/10/tiny-box-can-anonymize-everything- online/)

Which has the line "This piece has been corrected from an earlier version that included his claims that both the board and case were custom-built for the project." Note the explicit statement that they designed everything.

Wired further stated "he clarified that the router was created from a stock board." Clarified from the original statement that it was all custom designed.

Wired ends with "Germar also says now that the case was supplied by Gainstrong and was not custom-designed by the Anonabox developers, a partial reversal of how he initially described it to WIRED."

So, yes, he DID lie. He explicitly claimed it was all custom hardware multiple times, even when DIRECTLY confronted with evidence to the contrary. Only after several days of this did he begin to walk back his comments.

</comment></comment></comment><comment><USER19> I'm curious about the initial 'success' of the fund raise before the unfortunate issues that occurred. Any thoughts on the reason they exceeded their goals so quickly? Does this support indicate there is a substantial demographic of privacy advocates who are interested in personal solutions to privacy? Or, are we looking at a more widespread support of any project (given they don't misrepresent their claims) which advocates a more generalized freedom of information?

Personally, I'd order the thing because I don't trust my router's software and want something quick and easy to deploy.

<comment><USER20> Well it represented that there are a number of people willing to pay $50 to have a "set it and forget it"[1] tool to enhance their privacy. One might think of that as a validation for the product space. No doubt you could actually build these things now and sell them (although you would want to do a better job of marketing them)

[1] Yes, I'm fully aware that this is not a security conscious point of view but a lot of people just want someone else to solve the problem in an 'ok' way for them apparently.

<comment><USER21> What about this: [https://www.kickstarter.com/projects/1052775620/wemagin- smar...](https://www.kickstarter.com/projects/1052775620/wemagin-smart-usb- drive)

</comment></comment></comment><comment><USER22> "I'm working on something to announce (hopefully) today. Will try to get a PORTAL to everyone that wants/needs one. :)" \- thegrugq

[https://twitter.com/thegrugq/status/523299858581430272](https://twitter.com/thegrugq/status/523299858581430272)

</comment><comment><USER23> My 2 cents : sure the initial description of the project was misleading BUT after all the noise the people who were still funding the project KNEW what they were getting into.

I think they should have the right to risk their money.The project sounded FISHY they knew it,well their problem now. Noone's going to cry if the project turns south. Noone's ever care anyway.

<comment><USER24> I'm not sure Kickstarter is really motivated to wade into the caveat emptor waters any further than they have.</comment></comment>


Wycheproof – Tests that check cryptographic software for known weaknesses - <AUTHOR> https://security.googleblog.com/2016/12/project-wycheproof.html <comment><USER2> For anyone who isn't a crypto nerd:

Daniel Bleichenbacher is one of the most famous living cryptanalysts. He's famous for a number of attacks on public-key cryptosystems like RSA and ECSDA, and published one of the very first error oracle attacks. Many attacks on SSL/TLS trace, intellectually, back to him. Crypto nerds tend to be surprised to learn that he's been working at Google for years now.

Thai Duong is part of the teams that discovered the TLS BEAST attack, the CRIME attack, and the POODLE attack, in addition to a bunch of other crypto attacks on software (for instance, he and his partner-in-crime Juliano Rizzo discovered the CBC padding oracle attack on .NET, and the SHA length extension attack on the Flickr API).

The most important thing about this work is the crypto knowledge encoded into the unit tests. Relative to the effort it takes to come up with these test cases, it's comparatively easy to port them to other languages.

It's worth reading through the source code on this project just for the comments. For instance: there was apparently a bignum primality flaw in Gnu Crypto that broke SRP --- which I learned only from a comment in the bignum test cases.

<comment><USER3> This is fascinating (to a non-crypto nerd such as myself). It would be nice to collect some code that will fail all the unit tests to study _why_ I think as well as reading the commentary. Also if theres a way to use this to audit legacy Java apps that would be fun as well.

<comment><USER2> Some of these are in the cryptopals challenges, at [http://cryptopals.com/](http://cryptopals.com/).

I agree, the negative test cases would be good to see.

</comment></comment></comment><comment><USER4> They did an amazing job at commenting the code. Example: [https://github.com/google/wycheproof/blob/master/java/com/go...](https://github.com/google/wycheproof/blob/master/java/com/google/security/wycheproof/testcases/EcdsaTest.java)

<comment><USER5> And when I comment code like this, I'm being needlessly verbose such that my time could be better spend doing something else.

I wish more people saw comments (and by extension, commit messages) in this same light. Fan-freakin'-tastic.

<comment><USER6> TBF, that large comment precedes 150 lines of arbitrary hex strings without even a variable name to tell them apart. I'm normally someone who prefers the code to speak to itself, rather than have a large prose comment - but if I came across that in a code review without an accompanying fifteen line comment I'd probably flag it as needing some explanation.

<comment><USER7> You have one comment for them. I think this is enough, unless you have no idea about what they're doing.

// Signatures with special case values for r and s (such as 0 and 1). // Such values often uncover implementation errors.

</comment></comment></comment></comment><comment><USER8> Ah good old Wycheproof. They used have a race called the King of the Mount were racers would struggle up the "mountain" carrying a 63kg bag of wheat. They had to shut it down because after the race everybody would get blotto and trash the town, that and insurance costs I believe.

<comment><USER9> And the train line running up the middle of the main street.

</comment></comment><comment><USER10> This is a fascinating project. The hardest part of writing crypto software will always be avoiding the little mistakes, and every time we can bottle the understanding needed to detect some of them, we free cognitive labor for finding others. This is a necessary part of the future of crypto development. I tip my hat to the Wycheproof team!

</comment><comment><USER2> Incidentally: since there are similar concerns for most of the AE cipher constructions, not just EAX and GCM, I wonder if the EAX test suite here is an indication that Google uses a lot of EAX.

I liked the diss on GCM in the GCM test suite.

</comment><comment><USER11> It would be awesome if they port this to c/c++.

On a related note: how prevalent is Java among the security folk?

Most of the crypto algorithms I've read are done in C. As a person that wants to try rolling my own crypto, I just don't see myself writing it in Java.

<comment><USER4> Not an expert, but I'm pretty sure that the first rule of crypto is that you should not roll your own, at least not in a place of any actual importance.

</comment></comment><comment><USER12> So this would be something one could throw the arq backups at? I find very little about whether those are actually secure.

Generally this sounds like a great effort to harden important software.

</comment><comment><USER13> Request for crypto review: We've been working on an explainer series for how to build end-to-end encrypted P2P social networks. It was very hard for me to figure out /conceptually/ what pieces needed to go where and fit with others, so we made this series ( [http://gun.js.org/explainers/data/security.html](http://gun.js.org/explainers/data/security.html) ) on how to use Public/Private key pairs, PBKDF2, Proof of Work, AES, and Digital Signatures. Anybody up for reviewing it and telling me if this is the right combo/structure?

<comment><USER14> Don't post things like this in a largely unrelated thread. Put up an Ask HN instead.</comment></comment>


Online recipe ratings are broken - <AUTHOR> http://blog.recipepuppy.com/online-recipe-ratings-are-broken/

<comment><USER2> This is not only a problem with recipe sites, but with most user input. [http://youtube-global.blogspot.com/2009/09/five-stars- domina...](http://youtube-global.blogspot.com/2009/09/five-stars-dominate- ratings.html)

YouTube noticed a similar issue. Getting users to rate on a star rating just isn't effective. You may have more luck with a simple 'yes/no' option. Would you make this again? And offer the opportunity for comments.

I find the comments in the reviews on epicurious to be very influential in how I follow the recipe, and if I cook it at all.

<comment><USER3> Agreed, and thanks for the link.

I think this applies to almost all online rating systems, in particular ones where it's just pick a number out of five or ten. Most will be either very high (perfect, or perfect with a slight problem), or the lowest (completely shit). Barely anybody will mark something as 2/5 because it was utterly terrible, but this one feature was good. They'll remember the bad parts. No, no data at the moment and it's just a hunch, but one I hope to be able to measure soon.

I'm hoping to apply this to webhosting, rather than simply rate out of ten and leave a 2 sentence review, it'll be much more indepth, with multiple options (support, uptime, features etc). I'm betting that the more effort that you go through to leave a thorough rating will be balanced by the fact you won't have to rate that often (at most maybe once a year once you find a good host).

Oh well, still in the idea stage but I think people should reconsider just a simple rating out of five for anything non-trivial (like rating videos). Anything important (major purchase decision) should be more detailed and have more information for the prospective user.

<comment><USER2> I think you have another challenge when dealing with something like rating hosting. The problem, I believe, comes in where you will likely have many people leaving reviews after a bad experience, rather than a good one. For any negative experience, a person will tell 10 people, for a positive experience, they will share with 2 people.

Videos are a bit different, because people review as they are watching. However with something like hosting, you need to give considerable thought to how you get people to report positive or average experiences equally as often as negative ones.

That's just my initial reaction.

<comment><USER3> Yeah, being filled with negatives would be annoying although I'm hoping to come up with something, probably along the lines of discounts/coupons regardless of whether they left a positive or negative one (as long as it's an informative one).

It's definitely something to think about and it's a pretty difficult balance I'll have to achieve but I'm confident I can figure something out.

</comment></comment></comment></comment><comment><USER4> "So how does the user actually find the best recipe for a particular dish?"

Users may have better luck finding quality recipes by going to curated sites Cooks Illustrated rather than mob-rule sites like RecipeZaar. As with social sites (Digg, Reddit, HN), the chemistry of the community and the influence of the admins can have a powerful effect on the quality of the content. It also depends on what you're looking for. If all you want is cheesy enchiladas, recipes from the popular sites will work just fine.

There are many sites (not to mention books and periodicals) which have thought and care put into their recipes, often with supplementary material which helps you understand what you're doing and become a better cook, rather than just an ingredient-assembling automaton.

Another aspect to consider is that many people like to read (or watch) things about food but don't actually cook much. That's why the Food Network has become so popular (ever notice how all their commercials are for ready-made products?). A pet peeve of mine is to see 5-star reviews with a comment like, "Looks delicious! Will try making it sometime." Pretty annoying, but some people just like to watch. Maybe that's why recipes with pictures get higher ratings.

</comment><comment><USER5> Yes, online recipe ratings are broken, which is why I don't pay attention to them. Of course, I'm a fairly good cook, so I know how to look at a recipe and tell whether it makes sense or not. Often, I'll end up with 60% of one recipe, 15% of another, 10% of a third, and 15% my own experience to pull together a recipe for a dish I want to try to make.

If you're looking for something on how to make a dish (like I did recently for baguettes) and find several recipes to choose from, I find a few things make it easy to choose from:

1\. Real, individual ingredients are better than composite ingredients (that is, flour, baking soda, oil, etc. rather than Bisquick™). 2\. Simpler recipes are usually better than complex recipes, especially if you've never made the dish before. There are exceptions, of course. (I'm _certain_ that the King Arthur baguette recipe that I found would be better than the one that I tried, but it requires making a sourdough starter some fourteen hours in advance. I will try it some day.) 3\. Well-written descriptions suggest better recipes. This is a bit like code comments, though—there is such a thing as too much description. 4\. Recipes that depend on exact timing of multiple pots and pans are probably not a good idea to experiment with if you're not already good at making similar dishes.

That said, I'd pay attention to recipe ratings if there were only two yes/no questions: I made this; I'd make it again. Report the resulting value as a simple ratio of (repeat)/(made once). Seriously, I don't make recipes more than once if I wasn't enthusiastic about the results or weren't convinced that I could make it better by changing the ingredients or cooking instructions better. I don't know anyone who does. Star ratings are a waste of time.

</comment><comment><USER6> I'd suggest that a bell curve in this case isn't a particularly good goal. Most people who cook already know what collection of flavours they're likely to enjoy and will make more dishes that combine these flavours in interesting ways.

A good example is dishes containing coriander, a large number of people hate it with a passion while others love it. People who hate it won't make the dish and won't review it. Those who love coriander will make it and most likely enjoy it.

Applying statistical analysis with the goal of getting a bell curve isn't the correct direction to reason in.

<comment><AUTHOR> I guess I just want a recipe rating system that is consistent and has some sort of meaning. I want to know of the people that like coriander and made a dish with coriander in it, what recipes did they find the tastiest (the ones with extra cumin or not). This is not possible when coriander loving people rate the coriander containing recipe 5/5 9 times out of 10.

<comment><USER6> There was a post a while back suggesting having a few of your previous ratings display beside the rating scale.

It requires more of a slider approach though, which is something you have to spend more time teaching users about.

<comment><AUTHOR> I just found the thread. Interesting <http://news.ycombinator.com/item?id=910976>

</comment></comment></comment></comment><comment><USER7> I don't think recipes themselves can really be rated; we probably have to rate the person who created the recipe or the person evaluating the recipe. Let me explain:

The number of people that come back and rate a recipe is probably small. And how many people who had a middling experience with the recipe would make the effort to come back and rate it? Or even a good experience, for that matter? A bad one? You bet they're going to rate it.

So right there, you have all kinds of bias problems with the data. And just to make it more fun, how many of those people rating the recipes prepared it properly? How many substituted ingredients that just completely ruined the dish? You see this all the time on these sites - "This recipe is horrible. I vomited for 3 hours straight. BTW I didn't have any red onions so I used pickled cocktail cherries instead."

Somehow you have to find a more indirect way of evaluating a recipe's worth. Maybe it could be a measure of how many times a person comes back to look at it (on the theory that each hit is presumably them coming back to look up that recipe again)? Or perhaps how many people chose to make it.

<comment><USER8> I think your are on to something. Report how many people saved a recipe (and unsaved it?), how many printed it and how many views it received.

Additionally, I think the mentioned websites ignore an important fact about recipes: they are not static. Most recipes have variations in technique or ingredients. Many cooks, as you stated, substitute ingredients that subtly, or not so subtly, change a recipe. If a mechanism exists to capture those variations (wiki style, maybe?), those people might have an incentive to come back and share their experience rather than their impressions.

</comment></comment><comment><USER9> Recipes are interesting in that most reviewers actually alter the recipe, making star reviews misleading.

So someone might post a bad recipe and have other members alter it and rate it highly after their own modifications. You can only figure this out by reading the comments.

So some kind of yes / no on whether you have modified the recipe is pretty essential.

<comment><USER10> You can observe this same multi-factor effect in restaurant or product reviews.

When a restaurant serves one customer unsatisfactorily or a product is delivered in poor condition, the rating suffers substantially. This rating no longer represents the quality of the thing itself but also the individual's experience.

The same is true with a cook's experience preparing a recipe. Depending on the reviewer's bias, a recipe rating might describe circumstance and culinary skill better than the quality of the instructions.

As other folks here have mentioned, there's also the issue of average reviewers being not-so-great at representing their true opinions with linear rating systems, but that's a whole other topic.

</comment></comment><comment><USER11> Slight tangent: anyone built a recipe site that works by presenting a common item (chicken, flour, milk, etc) with "Got this? Yep or Nup" and so on to build a list of search terms?

CookThing seems similar I guess, but not quite the same.

<comment><USER12> <http://www.supercook.com/> is a front-end to allrecipes.com

<comment><USER11> Not sure if it wasn't working when I tried it, but I entered 'capsicum' and it showed no recipes. What I envisaged is a site that would randomly pick a common ingredient (rice, potato, coconut milk) and confirm that the visitor had it at hand. Then go from there, rather than starting by having them name something they had.

</comment></comment></comment><comment><USER13> I do not use the sites listed in the article, but I have found that Epicurious (<http://www.epicurious.com/>) has plenty of bad recipes that are rated as such. I have had good success following the ratings and checking the comments on the recipes before trying them.

Of course, with any crowd sourced system it is good to take the time to really read through who you are getting your trusted advice from. I still remember reading a review of a Korean restaurant on yelp that had two or three one star reviews because "they did not have sushi".

<comment><AUTHOR> I just checked and I unfortunately do not have the epicurious rating data. I did just check their website and they have a guide for rating 1-Okay 2-Good 3-Delicious 4-Exceptional.

Their rating system is not broken to the degree of these other websites.

</comment></comment><comment><USER14> There is a factor at work in this recipe ratings systems that is a common bane to all rating systems which present a user with a range of choices. One commenter (pedalpete) noted already that people gravitate high or low. But the larger issue is the number of people that will gravitate toward the current average rating.

This is a natural desire to conform, and it isn't always conscious. If a user perceives that most people have rated this item X, he will naturally want to rate it X, and will even re-evaluate his judgment based upon the ratings of others.

Of course, for an online-rating system with instant feedback, this gives the earliest ratings the highest weight, immediately skewing all the results which follow.

But what effect does the average rating have on a person who disagrees? They will naturally tend toward the opposite extreme. They want to make their disagreement obvious. If the average rating is high, the negative rater will gravitate toward the lowest ratings, when in fact he might have gone 2 or 3 stars had he had no such prior feedback.

This is a rather intractable problem, since about the only way to prevent this sort of feedback would be to hide the rating until the person has actually tried the item in question and rated it themselves. But that, of course, makes a rating system useless.

</comment><comment><USER15> Another thing you can't do with recipe ratings is prove the the rater actually made the item in question. At least on shopping sites, you can say "jrockway bought this item on 5/19/2009", so you have some idea whether or not the person has any clue.

</comment><comment><USER16> Taking a page from the OKCupid marketing department? Bravo! Smart way to attract people to your service.

<comment><USER17> This is a good thing. The public usually does not have access to such detailed data, so I'm glad that other companies are copying what OKCupid pioneered.</comment></comment>


Identifying a SaaS Market That Machine Learning Can Address - <AUTHOR> http://tomtunguz.com/ml-cost-reduction/ <comment><USER2> I'd say that by far the greatest potential in process optimization still lies with improvements in areas that don't require machine learning, such as:

\- fully or partially paper-based processes

\- digital systems that can't easily exchange data with one another

It always makes me want to cry when I sees systems where due to the lack of a proper interface between software components the API between these components essentially can be described as "Print it out on paper and hand it to someone else to type it into another software."

You don't need machine learning for that. You need APIs and first and foremost you need people to talk to each other and agree upon joint requirements for different components and the system as a whole.

<comment><USER3> chances are that if they haven't automated it, it's not revenue generating or important enough to invest capital to fix it

<comment><USER2> Could be but then machine learning doesn't help either.

There's still a lot to fix and automate though with a significant ROI.

Did you know for example that the paperwork required for sending a shipping container from Shanghai to Rotterdam is more expensive than the actual transport of the container itself?

Freight forwarder Maersk considers this to be a painful enough problem to try and solve it with Blockchain technology now (one of the few cases where using a Blockchain actually makes sense because not all participants involved in the process necessarily trust each other).

</comment></comment></comment><comment><USER3> > Consequently, if you looking to build a machine learning based SaaS company, > find a really expensive internal process and automate it.

so basically the "how to" is "find a problem and improve it"

this falls short of the expectation I had going into the article...hoping this was some sort of framework to identify markets

I feel like this is clickbait

<comment><USER4> Most of Tunguz's posts are like this. His analysis of SaaS startups are very meta and a few sound like a armchair quarterback. His style is not for everyone and if you are an entrepreur, his advice is rarely practical and specific.

</comment></comment><comment><USER5> TL;DR "find a really expensive internal process and automate it."

<comment><USER6> Do you need AI for that ?

<comment><USER7> No but you can charge more if you call it AI.

<comment><USER8> The snazzy misnomer of the day is "robotics", short for the more accurate "robotic process automation" or "RPA":

[https://en.m.wikipedia.org/wiki/Robotic_process_automation](https://en.m.wikipedia.org/wiki/Robotic_process_automation)

// Note: If you're interested in getting paid to drive automation efficiencies at massive scale with any of RPA, AI, ML, whatever, I'm hiring.

<comment><USER9> We'd been hearing about this "robotization". At some point our team leader talked in depth with some prospects and brought back the good news -- hey, this is what we were trying to sell all along.

So now we're robot makers apparently.

</comment></comment></comment></comment></comment><comment><USER10> > To date, most software imbued with machine learning reduces costs rather > than increase revenues.

Source?

ML includes trading algorithms, recommendation systems, self driving cars, search, translation, asset pricing, etc., all of which are revenue generating.

<comment><USER11> The point is that all of those processes were already being done, just in a less efficient manner.

<comment><USER10> That can be said of any product, not just ML.

</comment></comment></comment><comment><USER12> This article is nonsense. Grammerly or Amazon Echo are examples of how AI can do _new_ things in a frictionless way. Reducing friction is a huge use case for machine learning. Lowering costs in this splintered economy of haves and have nots is certainly valuable but hardly the most interesting or valuable opportunity for ML.

<comment><USER13> I think people, including the author, vastly underestimate what will happen when things get cheaper. I think smart speakers are still a case of transcription/understanding being cheap enough that we can do it for trivial things like turning on the lights.

But I think ML is even more powerful, in situations where we learn from real world data rather than some human/programmatic data labeling, we can exceed the abilities of people. Google's recent ML for health stuff comes to mind where it's much better at humans at certain tasks.

</comment></comment><comment><USER14> With Machine Learning, obviously!

Meta Machine Learning.</comment>


A Go Dependency Manager, Johnny Deps - <AUTHOR> https://vividcortex.com/blog/2013/07/25/johnny-deps-a-go-dependency-manager/

<comment><USER2> First off: cool!

A few thoughts:

* This only works with Git. It doesn't look it'd be much work to make it work for all 4.

* This only pins deps you've declared; that set may or may not contain transitive deps. If it's not complete, the transitive deps will be frozen, forever, until you blow them out of your GOPATH.

* Issue #3 looks fun.

<comment><USER3> Thanks! it was pretty fun to get it working, I'm glad you like it.

* Yeah, we use git internally, but we should definitely add support for other vcs when we can squeeze the time for it.

* True, I purposely kept this as simple as possible and forcing people to keep a complete Godeps file didn't sound like a terrible tradeoff (I actually like that it makes it that explicit).

* Issue #3 will have me scratching my head for quite some time, but I'll find a way!

Thanks so much for the feedback!

</comment></comment><comment><USER4> On first glance this looks similar to a pubspec.yaml file in Dart. Maybe you could borrow ideas from there.

It seems like for a business you'd also want to download your dependencies and check them in, in case the original repo isn't available.</comment>


Tabhunter - Find and Switch between Firefox Tabs Easily - <AUTHOR> http://blogs.activestate.com/ericp/

<comment><USER2> Tabhunter is like Quicksilver for tabs.

The current version of Tabhunter uses the same keyboard shortcut as OS X's "Special Characters" character palette. An easy workaround is to create a dummy entry in the Keyboard & Mouse preferences (in the "Keyboard Shortcuts" section) that doesn't do anything but uses the same Command-Option-T keyboard shortcut.</comment>


Maker Pro: The Supply Chain - <AUTHOR> http://www.bunniestudios.com/blog/?p=4266

<comment><USER2> We took "Soylent" out of the title, even though it's a legit reference to the movie, because the word "Soylent" has a different local binding here, and its presence in the title would likely trigger an off-topic discussion.</comment>


Bastion hosts - <AUTHOR> https://www.pandastrike.com/posts/20141113-bastion-hosts <comment><USER2> Jump boxes are a good idea. A better idea, if you're in AWS, is to use a VPC and just VPN into your environment; the VPN terminator in the VPC serves the same purpose as a jump box.

If OpenVPN scares you, you can still do layer-4 filtering to minimize the impact of losing the OpenVPN server (ie, to get it to "no worse than if we hadn't used VPCs in the first place).

"Single-packet authentication" is, I think, silly, and I've never recommended it to anyone. It's hard to think of a case where hand-rolled authentication in front of SSH has saved anyone with a proper SSH configuration (keys-only-no- passwords). You can skip that step.

<comment><USER3> > A better idea, if you're in AWS, is to use a VPC and just VPN into your > environment;

Is it though? Is having your (and your co-workers) desktops/laptops/and possibly more directly able to access production servers at a packet level a good idea?

I would posit that it is not. A bastion host can be used to lock down more than firewalls, and reduces the attack surface living on your network considerably.

> VPN terminator in the VPC serves the same purpose as a jump box.

It certainly serves the same network separation purposes a jump box does, but is not able to dictate the access environment nearly as much. If only your jump box has access to servers (and production servers properly filtered), every bit of remote access must go through that single ingress/egress point. And that point can have security policy built-in as well for multi-person teams. For example: I can enforce certain key types being utilized on remote hosts, logging of ssh sessions, etc.

I also agree the "single packet authentication" is silly though. Most bastion hosts these days will be behind VPN already. Just another riff on port knocking really.

<comment><USER4> Bastion hosts are generally a bad idea. There is not a single implementation of a bastion host that doesn't have significant security holes. This is doubly true if you want to do proper group based access control.

Most notable of these issues is the nature of either needing SSH agent forwarding enabled or storing a copy of the private keys on the bastion host (please don't do that). There are ofcourse ways of dealing with this but they aren't worth it.

It's fundamentally much better to just harden every system and implement proper federated authentication and authorization with LDAP. It also makes it much nicer to implement things like MFA and device level authentication.

Remember, it's very rarely that an attacker is going to be able to successfully walk in or break down the front door. Your security holes are very unlikely to be in your administrative channels like SSH/RDP but rather crappy applications or services you may be running on your systems.

Those crappy bits of software + insider threat are the 2 main things you generally need to deal with. Ensuring systems are sufficiently isolated from one another helps with both of these and bastion hosts just add another layer of shared stuff between workloads that should stay entirely separate.

<comment><USER2> To a one, every medium- and large- sized application deployment I've tested that failed to set up VPC (or something like it) has fallen to some stupid oversight on some stupid backend, support, or dev machine. Don't rely on individual host hardening for security; you'll fail.

<comment><USER4> I didn't say forgo the VPN. That would be unnecessarily increasing your attack service. In fact I specifically mentioned doing whatever you can to increase isolation between workloads, this includes ensuring access between dev, support and production machines is either not possible or is via careful controlled and audited means.

I only said bastion hosts are bad and generally decrease overall security because it's almost impossible to craft one that doesn't create more security holes than it closes.

</comment></comment></comment></comment></comment><comment><USER5> Bastion hosts are the wrong approach, as are VPNs. They might have had some merit in the days of handcrafted servers, where "hardening" was something you could do to a single machine. But nowadays, especially on AWS, there's no point in a configuration that isn't automatically reproducible - in which case, why not harden all your machines?

Then you avoid having a single point of failure. If and when a single SSH key is compromised, there are a limited number of hosts that key had access to - and unless that's the log server, you have secured logs so you can figure out what happened. Different keys can have different capabilities, rather than a one-size-fits-all policy on the bastion. It makes it harder for an attacker to know where to get the boss's keys. And you can expand your system geographically without the performance compromises of a VPN; your servers already make encrypted, secured calls to each other (over SSH, SSL or a very small number of vetted protocols), expose only those ports that are actively used. Your attacker could steal your entire system documentation and it wouldn't make any difference, because the system is truly secured, not relying on obscurity.

<comment><USER6> You're describing an overly optimistic scenario. "Why not just harden all your machines?" Because it's more complicated than that. Yes, you can automate basic hardening (configuration of access controls, limited use of SSH keys), but how do you automate hardening of RPC between one server and another, or hardening of those 3 internal administration web apps which may or may not have some local file disclosure and crypto flaws?

Defense-in-depth is still the only sensible way to handle security. You do what you can on the host side, but bastion hosts, jump hosts, VPNs, etc. provide an additional layer to make lateral movement much more difficult for an attacker.

</comment></comment><comment><USER7> Direct quote from the page at the other end of this link :

"You appear to be using a browser that may not be able to display our site correctly. Generally, this is beause it's an older browser that doesn't support a lot of the newer, awesome features in open Web standards.

Rather than provide an untested and possibly subpar experience of our site, may we suggest downloading one of these fine browsers? (We think you'll find your overall experience of the Web will improve if you do.)"

This is followed with links for Firefox and Chrome. I am running Firefox 24.4 from CentOS repos. This is unskippable, and there's no simple way to bypass it on the page. On a browser from March of this year.

Urgh.

<comment><USER8> TBH, > 6 months old is hopelessly outdated. Welcome to the future, grampa.

<comment><USER9> The point of those LTS browsers is so that an employer doesn't have to watch for UI changes in their browser every 6 weeks, and then retrain all their staff when the browser moves things around or makes things look different.

</comment></comment></comment><comment><USER10> In the Enterprise world, jump boxes are more typically just a hack implemented by IT workers who don't want to go about properly setting up a firewall and 802.1x/vlan/vpn/ipsec/etc to allow strict control to a protected network resource. (Or because they don't have the hardware capability to do it, or their network admin decided against it).

The only two proper excuses for ever using a bastion host are "I don't have the money to do it right" and "I don't have the time to do it right."

</comment><comment><USER11> I'm a fan of the VPN terminator as jump point approach.

OpenVPN with tls-auth switched on is a pretty robust front door. If you combine that with enforced key only auth for ssh, ensure all internal machines run their own firewall and logically split different bits of your internal network off in to their own subnets you've got a fairly secure foundation to build upon. Another easy win is switching internal services over to TLS/SSL only (where it makes sense/is possible to do so).

</comment><comment><USER12> You appear to be using a browser that may not be able to display our site correctly. Generally, this is beause it's an older browser that doesn't support a lot of the newer, awesome features in open Web standards.

Rather than provide an untested and possibly subpar experience of our site, may we suggest downloading one of these fine browsers? (We think you'll find your overall experience of the Web will improve if you do.)

</comment><comment><USER13> Jumphosts have their uses, but in this case, I really don't see it. As others have pointed out, better yet is VPN only access.

As for the port knocking silliness, well, I don't see any reason for it.

[http://bsdly.blogspot.com/2012/04/why-not-use-port- knocking....](http://bsdly.blogspot.com/2012/04/why-not-use-port- knocking.html)

<comment><USER14> I've been doing this sysadmin thing for a long, long time, and have built some large, high profile platforms,[1][2] and I can tell you that few things have made me happier than port knocking.

Yes, it is a password with a tiny number of "bits". But nobody suggested using _only_ port knocking, so that's not significant.

Yes, if my attacker is sharing the exact same NAT'd IP as me then they have a window to attack ... my fully hardened server, which was _no worse_ than what was presented to them before.

Finally (and this is what makes me happiest) my sshd is not exposed to whatever new exploit might be discovered tomorrow that doesn't require user credentials. It's not exposed to being DOS'd by a buffer overflow vuln. It's not filling my logs with brute forcers all day and night.

I love port knocking.

[1] JohnCompanies (first VPS provider)

[2] rsync.net

<comment><USER10> Mr. Sysadmin, I have four questions:

1\. When was the last time your sshd was hit with a 0day?

2\. Why aren't you running grsec to prevent buffer overflows (amongst other things) ?

3\. Wouldn't it be easier to DOS your host with excess traffic (like all modern DDOS attacks), rather than find a remote-code-execution buffer overflow 0day and use it solely to starve resources on a bastion host?

4\. Why don't you use a VPN? If you used a udp-based VPN nobody would even know the port was open (connectionless) and you could even require a preshared key, which is the more secure equivalent of port knocking.

<comment><USER14> The answer to #1 is "never". That's no reason not to be aware of the threat and to guard against it.

What is the state of grsec on FreeBSD ? I'm not sure...

Answer to #3 is yes.

I don't use a full-blown VPN in a lot of cases because it's a lot of complexity that isn't needed. knockd is extremely simple (look at the source) and simple to use.

\---

This is the way all discussions about port knocking go:

"I like port knocking"

"But port knocking is not the christ child that will deliver us from all of the evils of the world!"

No, it's not. It simply adds a small amount of incremental security at almost zero cost. All else being equal, my sshd was visible before, and now it's not.

<comment><USER10> Why do you encrypt your traffic?

If the port knocking feature exists to "hide" your network service, you're saying the security of your service is based on the idea that nobody can see your traffic.

But if someone _could_ see your traffic, and therefore your connection, and thus the port your service runs on, they could also clearly see the contents of your traffic. So port knocking would be useless, and everyone could see your traffic, and probably be able to do anything they wanted with it.

But since you use port knocking, you clearly believe nobody can see your traffic. So it then follows that you don't need encryption.

So clearly you should be using telnet.

<comment><USER15> _So clearly you should be using telnet._

This is just an absurd statement. You have conflated to completely separate issues.

 _you 're saying the security of your service is based on the idea that nobody can see your traffic._

No, it's because no one can find the service, not see the traffic.

<comment><USER10> Okay, let me see if this explains it better:

1\. Anyone who can see your traffic can see where the service is.

2\. We assume anyone can see your traffic because you're using an encrypted connection. (The conversation should have stopped here, but i'll continue anyway...)

3\. The only thing port knocking does is obscure your service from port scanning.

4\. The equivalent method to preventing someone finding your service via port scanning is changing the port number.

5\. Changing the port number prevents all non-direct-attacks from finding your service. (Nobody scans every TCP+UDP port of every IP on the internet for one service, because 563.6 trillion network packet send+recvs takes a long time, which doesn't even factor multiple tries or error retransmissions).

6\. A direct attack by a committed attacker will eventually find your network service.

7\. Port knocking does not add to security. It _partially_ obscures information, but it does not prevent access, and it operates on faulty assumptions about network security; the information is still there, you just have to look for it differently. It is less reliable than even IP whitelisting.

8\. In the real world (and not the "gee I think this sounds more secure" theoretical world) every single other network service you have will be exploited before OpenSSH. So trying to hide OpenSSH while not hiding your other services becomes farcical.

<comment><USER15> _1\. Anyone who can see your traffic can see where the service is._

Dude, right there you lost me. 99.99999999% of the internet can't see your traffic. The NSA can. Your ISP can. Random threats on the 'net at large __cannot see your traffic __.

<comment><USER10> You are radically misinformed. As a hacker, I have about a dozen ways to see your traffic. We don't encrypt credit card transactions because we're afraid the NSA or your ISP's network admins might pick up your card and buy some sneakers on Amazon.

Ways I can see your traffic:

      * fucking with bgp
      * fucking with routers in general
      * fucking with switch port tables
      * fucking with proxies
      * spoofing
      * sniffing wireless networks
      * sniffing wired networks
      * tapping a cage backbone
      * rooting an isp
      * rooting a dslam
      * rooting pops
      * rooting firewalls
      * rooting endpoints on management ports
      * abusing hosted networks

The single easiest way to view someone's traffic is to use the completely fucked-up retarded configuration of an intermediary router. Do you know how many routers sit between you and your destination? Do you know how many of those have 10-year-old configurations enabled via telnet on publically- routable interfaces with default snmp community strings and 6-character passwords?

Don't assume that just because you aren't aware of a particular attack that it doesn't exist or doesn't apply to you. Hackers are only limited by their creativity.

Please don't trust the network.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER16> This article goes from mediocre to bad when it starts advocating security by obscurity.

<comment><USER17> Security by obscurity is perfectly fine when it's part of a layered approach. It will help keep attackers from learning the details of your architecture and will also stop skiddies in their tracks.

<comment><USER18> This. Obscurity has too much bad press. It's not to be depended on, but it can be a useful addition sometimes.

</comment></comment></comment><comment><AUTHOR> Bastion host(s) are a useful and important component of a system management infrastructure. A bastion-host, in this context, is actually more properly, but more obscurely, called a jump server. In this post I will simply use the term bastion host. It is the most commonly used term for the system's function: a server, which has undergone security hardening steps, that is the operational and administrative control point for systems and hosts in a datacenter (or AWS Region).</comment>


Lawdingo (YC W13) Raises $690K More To Help You Talk To Lawyers Instantly - <AUTHOR> http://techcrunch.com/2013/10/30/lawdingo-fundingo/

<comment><USER2> There are many services for many kinds... Not intention of putting you down but 'competition'. Some come later and just do it smoother better. Overall, lawyer experiences are hard and expensive and there is huge room for improvement so I wish these guys the best and I hope they can make some significant change.

<comment><USER3> Very early in the article the author contrasted this service with LegalZoom, but the first thing that came to mind was something like Pearl.com. It's good when there's a service like this that's focusing on only one thing so I'm interested in seeing where this goes and how it fits into this space.

<comment><USER4> Hi, I'm the founder. Another article came out today, in which the author talked about the comparison to Pearl: [http://vator.tv/news/2013-10-30-lawdingo-raises-690k-to- help...](http://vator.tv/news/2013-10-30-lawdingo-raises-690k-to-help-you- find-a-lawyer)

In general I think the narrow vertical approach makes more sense that the general purpose expert advice network. Legal is supposedly Pearl's largest vertical in every country around the world.

<comment><USER3> Thanks, this is a good article. I too like having a site that only focuses on legal. Legal is big enough and there isn't much of an overlap like you have with verticals like computer tech, electronics tech, car repair. Best of luck.

</comment></comment></comment></comment><comment><USER5> I am not sure how different is this from a service like www.rocketlawyer.com which I tried using once and had a crappy experience. They tried to sign me up for a recurring subscription and claimed to have unlimited access to lawyers. But the lawyer I spoke to said that the unlimited access is limited to basic consultation only. Anything custom will cost you the usual lawyer money.

<comment><USER6> Lawdingo does not have subscriptions for people looking for lawyers- you just pay the lawyer for time spent

[https://www.lawdingo.com/about/clients/phone- consultations](https://www.lawdingo.com/about/clients/phone-consultations)

<comment><USER5> Thanks.</comment></comment></comment>