From a Frat Social Network to a GPU Compiler for Hadoop - <AUTHOR> https://medium.com/game-changing-ideas/c3c516146599 <comment> daniel-levin From the product website:

>> Wish that O(n^3) algorithm could execute in real-time? Now it's possible! We encourage you to push the limits of our platform and disregard what was previously intractable.

One of the most important things to note about asymptotic analysis is that the speed of your computer is immaterial. An O(n^3) algorithm _will_ experience cubic slowdown, irrespective of how it's implemented. A blazing fast, parallelised solution just reduces the constant multiple in front of the n^3, but ultimately, n^3 will outgrow that constant. I'm sure you know this, but it's disingenuous to say that you can disregard an algorithm's complexity because of your data / results. A particular algorithm with particular space complexity (also important) was run on a particular input and was much faster - granted, this is impressive - but not definitive enough to justify labels like '1000x faster' and 'disregard what was previously intractable'.

Also, O(n^3) is considered polynomial, which is absolutely not intractable. The whole point of intractable problems is that they are too complex to solve (given current best known methods). This might be very useful for applications that are currently too slow to run in real-time, but absolutely not a solution to problems of much higher order (read: non-polynomial) complexity

<comment><USER3> Right, we're not changing the complexity itself, N-ary parallelization in the ideal case modifies the runtime s.t. C' = C_0 / N.

<comment> daniel-levin By Amdahl's law, that idealisation assumes that your program is totally parallelisable. Very few real world applications actually are. Out of interest, was yours? (Or at least very close to it?)

<comment><USER3> Yes, our algorithm performed an O(N^2) loop over the graph vertices so B for us was very close to 1.

<comment> daniel-levin OK, but I think you mean B was close to zero then?

<comment><USER3> Fair enough.

</comment></comment></comment></comment></comment></comment><comment><USER4> Hadoop & MapReduce are usually an iffy choice for compute-bound workloads and definitely a bad choice for memory-bandwidth bound problems.

Solving problems caused by misuse of familiar tools (SQL, Hadoop, Excel, etc) seems to be a good meta-strategy for finding good business niches.

<comment><AUTHOR> We've proven that we can also speed up IO-bound and disk-bound workloads.

Memory-bound jobs are not an issue either. In fact, 12GB Nvidia and AMD GPUs were announced this week.

<comment><USER4> I don't doubt that you have, although I would be curious about the mechanism. Spindles are spindles, whether they're feeding to a GPU or a CPU.

I would be careful about phrasing it as "memory bound is not an issue". There's a distinction between memory bandwidth (which GPUs excel at as long as the data is resident), memory latency (which GPUs are pretty bad at), and memory capacity. 12gb is slim in the grand scheme of things; my laptop has 16gb and you can get machines with >1TB of memory. Outside of the GPU's memory limits, you're relying on streaming data to/from the device, and potentially bound on PCIe bandwidth, or partitioning the problem across multiple GPUs/machines.

<comment><AUTHOR> Correct.

PCIe is the least of our concerns at 16GB/s. Some of our tests indicate that our GPU analyzes data at a rate of ~5GB/s. If we can feed it the data in that amount of time, we're in business. We're also parallelizing disk reads, which is one bottleneck at this point.

</comment></comment></comment></comment><comment><USER5> GPU acceleration sounds all great.. until anyone actually generalizes the framework, implements applications, and measures even the most trivial workload. There should be some niche applications you may get 2-10x benefit, but 1000x is either measurement error, apple-to-banana comparison, or a new marketing gimmick. Unless you have a very fitting application, it is very easy for GPU to have less performance than CPU. I would be very curious what application actually could have 1000x performance enhancement. Even memory bandwidth, the main benefit of GPU, does not differ by that amount.

<comment><AUTHOR> We are not making up these numbers. Even for an IO-bound workload, we are seeing 350x speedup.

The very fitting application is MapReduce, which is embarrassingly parallel.

<comment><USER5> This is a very bold statement. IO-bound means that the bottleneck is at IO. GPU has little benefit in IO.

Also, mapreduce is not embarrassingly parallel. Out of mapreduce, map is the only embarrassingly parallel phase. It has input parsing, shuffle, reduction phases that are not embarrassingly parallel.

<comment><USER3> I should clarify -- initially I/O bound, but with pipelining, we are able to "impedance match" the flow of data coming off the backing store with the GPU's computational progress. I suppose the better word is "I/O heavy," rather than "I/O bound."

We're certainly not the first to describe mapreduce as "embarrassingly parallel," but I can defend myself nonetheless:

A shuffle executes in parallel where each unit re-indexes into another unit. Reduction is parallelized over each key, and even finer granularity can be achieved by considering a tree-based approach:

[http://developer.download.nvidia.com/compute/cuda/1.1-Beta/x...](http://developer.download.nvidia.com/compute/cuda/1.1-Beta/x86_website/projects/reduction/doc/reduction.pdf)

In fact, I used this tree reduction approach to merge non-disjoint clusters of friend groups in my social network clustering algorithm.

Parallelizing the input parsing is problem-specific, but even that is possible in many cases.

</comment></comment></comment></comment><comment><USER6> Why bring Hadoop into it? MR is a paradigm for parallel programming, and having it compiled into OpenCL for you is hugely convenient. But the best Hadoop use cases always leverage massive storage capacity: people who do ML on Hadoop are doing it because the data is already in Hadoop, not because it seemed like a nifty thing to do. Do you actually leverage HDFS and the Hadoop scheduling infrastructure to run ParallelX jobs? Or can you safely jettison them and just sell this as: "Write your ML/graph analysis/whatever code in Java, have it execute on a GPU"

Also, what are the limits like for input data sizes? I've done a little OpenCL, but I've never gone past the GPU RAM size.

<comment><AUTHOR> "Do you actually leverage HDFS and the Hadoop scheduling infrastructure to run ParallelX jobs?" \-- Yes.

We could also provide what you're suggesting outside of Hadoop, no problem!

"What are the limits like for input data sizes?" \--- There are no limits. You can store your data in AWS, and we will crunch it for you. That said, initially, for IO-bound and disk-bound jobs, ParallelX might not be ideal. This is a problem we are solving as we scale.

Thanks for the feedback! We appreciate it!

<comment><USER7> Are you porting string processing routines to OpenCL? The grid model of GPU computing seems to be a bit of a mismatch with the jagged-edge model of a bunch of variably-sized records in 2 1TB files that I want to join on in my typical hadoop use-case.

I guess I've only delved into GPU stuff for dense matrix math, though, which is a pretty bad fit with Hadoop. Maybe you guys can come up with some other use-cases for them.

<comment><USER3> Yes, we are porting string processing routines to OpenCL. We are going to have many of the Java libraries accessible within the compiled GPU code, including file system I/O.

</comment></comment></comment></comment><comment><USER8> " My co-founder, Chuck Moyes, discovered a clustering algorithm that identified friend groups in a given social graph. However, the algorithm took an hour to run parallelized across 6 computers. Then Chuck had the clever idea of coding the algorithm on a single GPU. The run time? 0.2 seconds! WHOA."

I do a lot of work with graph clustering algorithms and have done a little with GPUs and I have to say this is, at best, rather unexpected.

<comment><USER9> I genuinely don't understand what "'at best' 'rather' 'unexpected'" means.

<comment><AUTHOR> I'm not sure either! Either way, implementing a clustering algorithm on a GPU was thinking out of the box for us :)

<comment><USER8> What it means is I'm hesitant to believe an 18000x speedup.

<comment><USER10> This was my reaction as well. I think there should be a whitepaper explaining the comparison as many GPU/FPGA application acceleration companies tend to do. I would say a typical GPU speedup would be in the 10-20x range with 100x being possible for highly "regular" data parallelism. I have no doubt the GPU wallclock time is correct so I am guessing the CPU implementation is just very poor.

<comment><AUTHOR> We used parallel Python to implement the first version of the clustering algorithm. Although the CPUs weren't particularly beefy, the implementation was by no means poor. It's worth mentioning that the laptop GPU wasn't powerful either.

<comment><USER8> In that case, how much of your 18000x speedup is due to the GPU and how much of it is due to re-implementing the algorithm in a compiled language? Python->C/C++/etc is a 100x-1000x speedup right off the bat.

That's not a fair CPU/GPU comparison at all.

<comment><AUTHOR> You're right, it isn't a perfectly fair comparison. However, even if we had coded it in C++, the run time difference would still have been significant.

We're just sharing our story at this point. Next, we'll be sharing our whitepaper and data. We rather start gathering feedback now, as opposed to after writing 100k LoC for the compiler.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> Love the serendipity moral; loved surviving the vicissitudes. The story reads well.

There's a lacuna between _Codentical_ and _Parallel X_ : 1. people willing to pay for it; 2. a spark went off; 3. "validate _before_ you build"

You say you did have paying users, so why did you stop? What was the spark? I'm guessing that there weren't enough paying users - but that's not stated. This makes an inexplicable gap as you race down the homestretch of the story. It's not a huge problem, but since the rest of the story is so great, it's a shame to mar it.

Also, the story would be absolutely compelling if you mentioned the specific CPU, GPU and task that gave the incredible speedups. Your reader wonders, "Why aren't they mentioned?"

<comment><AUTHOR> We had paying users for GraphMuse, not Codentical. GraphMuse was shut down by Facebook, whereas we quit Codentical.

As I stated, we had a good number of people "willing to pay for it," but they weren't paying customers yet.

We will include this information on our website soon! :)

</comment></comment><comment><USER12> Not to be negative, but how soon before these founders quit ParallelX for their next great idea?

I would be extremely hesitant investing the engineering resources to use a product where the founders themselves don't have any conviction in their previous companies. I'd hate to be the victim of another flight of fancy, where the response is "Sorry, but we're pivoting."

<comment><AUTHOR> ParallelX has been an idea in the making for 2.5 years. Your skepticism is valid, and only as we continue developing, iterating, and working with customers will we convince you that we're serious.

This is the first time we'll be working full time, in the same city, if that matters for anything.

</comment></comment><comment><USER13> Hey Tony - nice to see that you've bounced back after GraphMuse! I was the PM for Bingo and GraphMuse really did help with the #'s, and Facebook really did shut the whole thing down with no notice.

The lesson with FB is more nuanced though - if you want a startup then dependance is bad, but you can have a very successful lifestyle-type business on them and do well though.

<comment><AUTHOR> Hey Idoh! I'm happy you saw our post, and thank you for the kind words.

You're definitely right about the lesson with FB. I'm just bitter about it :)

</comment></comment><comment><USER14> 1000x? I won't believe it until you can provide a simple, open source example illustrating your technology, or at least a technical write-up on your testing methodology and associated raw data from the benchmarks.

Or maybe all that time building apps in college you forgot to attend your statistics class?

<comment><USER10> There should definitely be some kind of whitepaper accompanying this marketing claim.

<comment><AUTHOR> We are working on that paper right now. If you're interested, feel free to provide us your email on the site.

</comment></comment></comment><comment><USER15> I've known Tony now for 6 years. He's smart and extremely resilient - <USER2> great qualities of an entrepreneur.

I know little about GPU compilers though. What are the key factors to make this successful?

<comment><AUTHOR> I can't thank you enough!

There are a number of factors that will make this product successful:

1\. The compiler needs to work well. 2\. We need to teach average developers that they can build new applications/algorithms that were previously unfeasible. 3\. We need to reduce Hadoop costs for large corporations.

Simple enough, right? :) Also worth noting that ParallelX will be an Heroku add-on as well (with a freemium plan!)

</comment></comment><comment><USER16> Very nice read! How does this work compare to work on java 8 to bring GPU compute to the jvm by AMD, ARM and Oracle? Do you see this as a competition or do you feel you are in a separate niche?

<comment><USER3> Yes, we are in our own separate niche. Rather than optimizing the computation of individual loops and the like, we're attacking the problem from a "Big Data" perspective where we seek to optimize processing large data sets. We're talking about special filesystems a la FB's Haystack to optimize disk sector layout coupled with RAID schemes to bring data to the GPU as efficiently as possible to overcome I/O bottlenecks. We are initially interop'ing with Hadoop, and we are going to extend this to support Pig and Hive queries (along with database support). I really do think this is an "apples and oranges" comparison, and it's great that they are taking the C++ AMP route of parallelizing loops (perhaps with some parallel primitives mixed in), but at the same time, there's a lot more to the problem we are trying to solve than just parallel compute.

</comment></comment><comment><USER17> Great work!

How would this overlap with the upcoming support for GPU computing target for Java 9, being developed by Oracle and AMD under the HSA foundation?

Assuming it really gets integrated, that is.

<comment><USER3> From what I understand, the GPU computing target is a loop parallelizer a la Microsoft's C++ AMP.

</comment></comment><comment><USER18> "Extraordinary claims require extraordinary evidence".

</comment><comment><USER19> "It was originally designed for traffic lights and vending machines, whereas it ended up fueling the computer revolution" ... I thought silicon chips/cpus were designed for military purpose initially, such as radars ?

<comment><AUTHOR> Not from what I've read, but I could be wrong:

 _" The microprocessors that made the microcomputer possible had originally been developed to run traffic lights and vending machines. They had never been intended to power computers. The first entrepreneurs to attempt this were laughed at; the computers they had created looked hardly worthy of the name-- they were so small and could do so little. But they caught on with just enough people for whom they saved time, and slowly, the idea took off."_

[http://www.fastcompany.com/3002810/aha-moments-made-paul- gra...](http://www.fastcompany.com/3002810/aha-moments-made-paul-grahams-y- combinator-possible)

</comment></comment><comment><USER20> Great read - I've known Tony for some time, and seen him through a few of his ventures, and this strategy seems like an incredibly savvy way of solving a real problem. Congrats, Tony!

<comment><AUTHOR> Who is this? Either way, thank you very much :D

</comment></comment><comment><USER21> One can already run Java->OpenCL workloads on Hadoop using [https://code.google.com/p/aparapi/](https://code.google.com/p/aparapi/) you don't need buy any magic sauce. It compiles kernels written in java down to OpenCL which can run on SSE or GPU backends.

</comment><comment><USER22> _Then we received this email from Facebook's Platform Policy Team [Notice of Violation] I used the Wayback Machine to see when the clause was added… 3 days prior_

Always reserve the right to change your API TOS!

<comment><AUTHOR> While I realize that an TOS is always subject to change, I still think it's a low blow to pull something like that. They intended it to be a one-way conversation as well. We had no way to respond.

Facebook is notorious for doing this too.

<comment><USER23> I love how the email from FB began with "hi," as if it were a note from your dear grammy checking in with you.

The deadline was 5PM PST - how many days (or hours) did you have to comply?

<comment><AUTHOR> We had less than 3 days to comply if I recall. Considering we had multiple customers in a handful of countries, it wasn't much time.

</comment></comment></comment></comment><comment><USER24> Where is the paper for this novel clustering algorithm?

<comment><AUTHOR> Here you go - [http://www.eecs.berkeley.edu/~isabelle/papers/cci.pdf](http://www.eecs.berkeley.edu/~isabelle/papers/cci.pdf)

</comment></comment><comment><USER25> Cool story, and good luck with your company!

</comment><comment><USER26> Great read, and ParallelX looks great too.

</comment><comment><USER27> i like this piece. great journey. thanks for sharing

<comment><AUTHOR> You're very welcome. I appreciate the kind words.

</comment></comment><comment><USER28> Enjoyed this :)

</comment><comment><USER29> Great read!</comment>


Crypto shocker: four of every 1,000 public keys provide no security - <AUTHOR> http://arstechnica.com/business/news/2012/02/crypto-shocker-four-of-every-1000-public-keys-provide-no-security.ars <comment><USER2> (<http://news.ycombinator.com/item?id=3591429>)

(<http://news.ycombinator.com/item?id=3591386>)

</comment><comment><USER3> Please note that there is another article here which provides some more information and another choice of words: <http://news.ycombinator.com/item?id=3593653>

</comment><comment><USER4> Wow, what a terrible headline. Even by the usual linkbait standards, this is egregious.

<comment><USER5> It's not that bad.

" _Crypto shocker_ ": well, yes, it is shocking and it is about crypto.

" _four of every 1,000_ ": 27,000 out of 7.1 million from the researchers' current collection. (Most of the paper is based on an older version of their collection, for which the figure is more like 2.7 out of every 1,000.)

" _public keys_ ": this is the least accurate bit; it applies only to RSA keys, not to all public keys. That's pretty bad -- though, according to the paper, somewhat over half of the public keys in their sample are in fact RSA rather than ElGamal or DSA or anything else.

" _provide no security_ ": the paper says "... that offer no security". Perhaps "no security" is slightly overstated -- the keys are vulnerable only to attackers who can be bothered to do much the same work as these researchers did -- but they make it clear that doing that work is far from being rocket science, and that there are "people who know how to do the computation quickly".

What about this justifies saying that "even by the usual linkbait standards, this is egregious"?

</comment></comment><comment><USER6> <http://news.ycombinator.com/item?id=3593709> is more coverage here on HN</comment>


Ask HN: What's your opinion of the dating/relationship nature depicted in "Her"? - <AUTHOR>

What would be the pros and cons of such a scenario? Would it be better than the current dating scene? Alternatively, what would be the ideal dating scene if it were up to you?<p>For a trailer and more information about "Her": http://www.imdb.com/title/tt1798709/ <comment><USER2> Fucking weird.</comment>


AirBnB hit 10 million nights booked - <AUTHOR> http://m.techcrunch.com/2012/06/19/airbnb-10-million-bookings-global/

<comment><USER2> The link provided is to the mobile site which doesn't include the pretty infographic, here's the full desktop version:

[http://techcrunch.com/2012/06/19/airbnb-10-million- bookings-...](http://techcrunch.com/2012/06/19/airbnb-10-million-bookings- global/)

Very impressive numbers:

        10 million guest nights have been booked worldwide
        through the site since it was founded in August 2008

But more impressive:

        at this time last year, Airbnb had yet to reach 2
        million nights booked

<comment><USER3> That's one helluva hockey stick.</comment></comment>


Xkcd: PGP - <AUTHOR> http://xkcd.com/1181/

<comment><USER2> Is this aimed at people like us who could poke a hole in this advice, or like the password sentence complexity comic it is for non-technical people?

<comment><USER3> It's aimed at technical people but sarcastically written as if talking to non- technical people who can't be bothered with the complexity of verifying signatures.

</comment></comment><comment><USER4> Great, now spammers are going to start including fake PGP blocks :)</comment>


Ask HN: Who are some of the most inspiring people/users you've come across on HN - <AUTHOR> I felt the need to pose this question after discovering <i>@rdlecler1</i> and doing a bit of background reading about said user's journey through life. It's been an inspiring discovery to say the least and I would like to know if there are any more such people on HN. <comment><USER2> New account because I don't wish to identify my affiliation publicly.

A user who posted before the New Year regarding his financial situation is someone I quickly found myself aspiring to. I didn't have alot of work for him but he was more than helpful for my project. After I learned a bit more about his background, family situation and journey to getting where he is now, I felt compelled to try to help him more.

During one of our many conversations I casually mentioned how my funded startup is suffering from an influx of chargebacks and fraud purchases. I regret not mentioning it sooner because this it was a problem we have been struggling to solve and in no time at all he solved it. He helped us and showed us some of the "carder tricks and tips."

Unfortunately because we (startup) are bound by certain financial regulations we couldn't hire him full time because of mistakes he made as a kid. I know he is still looking for work. I urge anyone who has any kind of work to contact him, I know he could desperately use it still.

Post in question is [https://news.ycombinator.com/item?id=10813746](https://news.ycombinator.com/item?id=10813746) and here is his follow up for anyone who is curious. [https://news.ycombinator.com/item?id=10921576](https://news.ycombinator.com/item?id=10921576)

</comment><comment><USER3> cperciva. Originally "found" him when a user by the name of "sanj" responded to cperciva, pretty much flaming him and asking him if he won the Putnam, an annual math competition held in the US and CA for undergrads.

cperciva was making a few "bold" statements and sanj replied "Did you win the Putnam?" to which cperciva replies, "Yes, I did." and later goes on to gloat a little bit saying, he considered his earlier score of 53 his most impressive considering he was 14 at the time.

A user called dbosson proceeds to calls him out on his gloating and talks about helping others. He replies with this: [https://news.ycombinator.com/item?id=35120](https://news.ycombinator.com/item?id=35120)

The thread: [https://news.ycombinator.com/item?id=35015](https://news.ycombinator.com/item?id=35015)

It was one of the few times I've seen News.YC/HN delve into the usual internet forum banter and not being disappointed with it.

You might know of cperciva also as the creator of Tarsnap. He's very open about not being money-driven and the service continues to keep to its roots as a product only made for technologists. He's not concerned with developing the product for Enterprise, nor marketing. He's also someone who hasn't bent to external pressure to further develop his product for anyone else. Everyone else is business-minded, the likes of patio11 included, who are practically screaming for him to take advantage of his brand and following to build up a company.. But he just doesn't want to.

To me, he's someone who's truly gifted, not humble about it but doesn't gloat in such a way that you feel like he's going overboard. His claims are valid, he's matter-of-factly saying statements that are all true. Akin to McGregor, a UFC fighter who always seems to predict how he'll put the other fighter down in the interviews: [https://www.youtube.com/watch?v=jJeziuaShow](https://www.youtube.com/watch?v=jJeziuaShow)

Sure there are an insane amount of people here I can admire and I follow incessantly because they're such valuable sources of knowledge. But cperciva especially I can respect, because he just doesn't seem to worry about whatever anyone else thinks or expects from him.

<comment><USER4> _goes on to gloat a little bit saying, he considered his earlier score of 53 his most impressive considering he was 14 at the time_

Just to set the record straight -- that wasn't gloating. That was me being annoyed with how people focus on the "Putnam fellow" bit to the exclusion of all else.

</comment></comment><comment><USER5> I've found many of the users the other comments mentioned quite inspiring in their own way, each towards some aspect of life. A little embarrassed to say this, but _sivers_ is the user I would nominate because he's a person who inspires how to _experience_ life. Checkout his blog at [http://sivers.org](http://sivers.org).

<comment><USER6> Derek Sivers is super inspiring. He has a book called Anything You Want about his days running CD Baby, it's my favorite business book (and also my favorite audiobook). I think I've bought at least 10 copies now, I've given several copies to clients & friends.

He gave a talk at WDS 2015 that includes some of the highlights from his book & blog: [https://player.vimeo.com/video/134968996](https://player.vimeo.com/video/134968996)

</comment></comment><comment><USER7> patio11 - bar none.

This guy is a successful businessman, and provides so much value with so much of what he writes. For free. Go read his stuff, I dare you.

</comment><comment><USER8> I found hugely inspiring/interesting things written by pg, spolsky, pmarca, zedshaw, AndrewWarner, dhh and others.

None of them comment here much these days. This is natural as HN has become a much, much larger and more mainstream site.

Interestingly, it's not like the shifts away from Slashdot and Reddit, where there's a new forum where people have moved to (except their own personal communities). If there's a place that has sucked up most of the conversation, it would be Twitter.

<comment><USER9> > If there's a place that has sucked up most of the conversation, it would be > Twitter.

The conversations on Twitter are unfortunately not of the same fidelity as the ones on a traditional forum where you can actually compose full thoughts.

Perhaps it's better to say that Twitter has sucked up most of the attention of those same people.

Subtle distinction but huge difference imo because I can't go follow those same people on Twitter and expect the same type of conversation as they would have made on HN.

<comment><USER10> _Perhaps it 's better to say that Twitter has sucked up most of the attention of those same people._

I suspect the cause and effect relationship runs the other way: Most likely, people are going to Twitter because HN does not have the same quality of atmosphere it had when I joined.

</comment></comment></comment><comment><USER11> patio11, tptacek, tzs, cperciva, WalterBright, ChuckMcM, Animats, rayiner, RogerL, jerf, jacquesm, nostrademons, mark_l_watson, edw519, lmm

</comment><comment><USER12> dangrossman, creator of improvely.

</comment><comment><USER13> Brudgers</comment>


Show HN: HTML5 Database using data-attributes (very raw – proof of concept). - <AUTHOR> http://tgolson.com/html5-data-base/

<comment><AUTHOR> A few follow up notes:

GitHub repo here - [https://github.com/TGOlson/html5-data- base](https://github.com/TGOlson/html5-data-base)

Save link doesn't work. Right now it just logs all the 'db' data in the console. I'm thinking I'll need to implement a back-end to be able to save the updated pages, but totally open to suggestions.

</comment><comment><USER2> Kinda cool.

But I'm not sure I understand the reasoning for this though.

Essentially, you've created a way to parse and query objects using an object structure to hold them? so json.

Then I assume you are storing this in localStorage or something similar?

If you are going to go down the back end route though you'd surely just opt to be using mongoDB no?

<comment><AUTHOR> All good points - I'm not sure about the practicality of it, just intrigued by the ability to store and retrieve data from HTML elements.

I can't imagine at the moment any project where it would be used, but who knows, maybe someone would, or maybe not. Mainly just seeing what it possible to create and then seeing if there are any ideas to help take it away.</comment></comment>


Pedagogy of the Oppressor - <AUTHOR> http://city-journal.org/2009/19_2_freirian-pedagogy.html <comment><USER2> Interesting article, however I question how widespread this book actually is, and the bias of the author of the article. Has anyone here ever come across this book? Are there any teachers here?

In all the years of my schooling, I never thought my teachers may have been influenced by such a book. Not many of my teachers were ever interested in talking about the social injustices or encouraging us to fight for any right, but quite the contrary actually. Once in a while I would come across an inspiring teacher who recognized the limitless potential of a young mind and attempt to pursue that ideal, but in the end, the overwhelming feel was always simply to spit out another well-rounded, good and contributing citizen of the society we live in. How dehumanizing is that?

</comment><comment><USER3> Tell us how you really feel: "...His declaration in Pedagogy of the Oppressed that there was 'no such thing as a neutral education' became a mantra for leftist professors, who could use it to justify proselytizing for America- hating causes in the college classroom..."

I know hackernews doesn't obsess about being "fair and unbiased" as much some sites, but I'm unwilling to consider an argument couched in such ridiculous terms.</comment>


Unsupervised learning for comprehensive forecasting of Alzheimer's Disease - <AUTHOR> https://venturebeat.com/2018/07/12/researchers-use-ai-to-predict-alzheimers-disease-progression/ <comment><USER2> Dr Dale Bredesen reverses Alzheimer's for at least 5 years and in his view there are 3 different causes of Alzheimer and hence patients need to get one of 3 treatments.

There is nothing mysterious about the treatment of Bredesen: he optimises the functioning of the whole body with optimization of 36 parameters. Varying from sleep, exercise, Magnesium, Zinc, to stress management.

<comment><AUTHOR> This result concerns the simulation of disease progression among untreated individuals. I'm not sure why it is relevant to mention therapeutic results...</comment></comment>


An LSD Trip "Off-Switch" May Be Coming Soon - <AUTHOR> https://www.freethink.com/articles/lsd-trip <comment><USER2> So no info on what it actually is. Just some company filing a patent. And the fact that they call it "a neutralizer technology" suggests that it's probably just a bunch of BS combined with previously known drugs to make it patentable.

I'm pretty sure there's at least some published research on using at least one antipsychotic for that and mountains of "folk research" on using benzodiazepines.

<comment><USER3> I'm curious what would be the market for this? wouldn't that be an extremely niche scenario within the emergency-response medicine where people inexperienced with LSD who suffer a bad trip ask to be taken off it? afaik LSD isn't a drug that one is likely to OD from either.

<comment><USER4> Psilocybin (from mushrooms) and MDMA are both fast tracked for FDA approval as adjuncts to therapy. Psychedelics are poised to become the most common psychiatric treatment in the world, with ongoing trials showing huge effects for treating PTSD, addiction, and treatment-resistant depression. Despite a lot of therapeutic promise, LSD has not received as much research attention, primarily because of its duration of effect. Having therapists present for up to 24 hours (compared to 6 for psilocybin) is prohibitively expensive. A true off switch would facilitate its study and use in treatment.

<comment><USER5> Results from studies that used an antipsychotic across the board would not be generalizeable to traditional experiences. At that point, why use L in the first place if older, faster acting substances are available. Mescaline and psilocybine have been used in spiritual and therapeutic settings for hundreds, if not thousands, of years. Where is the interest in L coming from, if not ease of slipping it past customs and other law enforcement?

<comment><USER6> I think if one class of drugs has the effect you want (in this case "psychedelics") you'd want to study other drugs in that classification as well.

Since people are studying psilocybine for depression, PTSD, etc, why not others like Mescaline, LSD, or DMT? Why leave something off the table? Maybe LSD is great for a specific type of PTSD and psilocybine for depression? You got to study it to figure out. And who knows, maybe these drugs don't actually help at all. Either way, you have to study them to figure anything out.

> Mescaline and psilocybine have been used in spiritual and therapeutic > settings for hundreds, if not thousands, of years.

Yes and no. Those are the active ingredients, but these have not been distilled. From my extremely naive understanding, other chemicals here could affect people differently. So to create a safer and more therapeutic drug. Maybe you even need some of those other chemicals. But you can't find out without testing.

One thing is for sure, in a clinical setting you want very controlled dosages. You're not going to get that by ingesting plants/fungi.

</comment></comment></comment></comment></comment><comment><USER7> We already have one. Give someone having a bad trip a Xanax and they're good.

<comment><USER8> How about alcohol? I've only had one third of a trip and towards the end whiskey took some of the edge off.

<comment><USER9> I've never been able to even contemplate eating while tripping and the idea of drinking alcohol while tripping turns my stomach, I find it much easier to just take some alprazolam or diazepam if things start getting out of hand. That's just me personally though, I've tripped with a guy who went and bought and ate a whole chicken while tripping...

<comment><USER10> In my experience, processed food of all sorts appears utterly weird and unappealing in this state.

On the other hand, fresh fruit is usually marvellous.

<comment><USER11> Set and setting. Trip with some poor people who haven't seen a fresh fruit since grade school and look out instead for the magical "fruit" drinks that taste of their literal colors.

<comment><USER12> On one trip I had a memorable and absolutely glorious time stuffing my entire mouth with Gushers[0] and taking deliciously fruity flavored bong hits.

[0] [https://en.wikipedia.org/wiki/Fruit_Gushers](https://en.wikipedia.org/wiki/Fruit_Gushers)

</comment></comment></comment></comment></comment></comment><comment><USER13> To be honest, I'm more interested in an _easy to acquire_ "On-Switch" than anything else.

<comment><USER14> Meditation. There are certain kinds of meditation that light up the brain in a near identical way to magic mushrooms, though imho the experience is better. An advanced practitioner only takes about 20 minutes of meditation to get into that kind of state. They can stay in that state as long as they want, and they can strengthen it or weaken it even turning it off whenever they want.

<comment><USER15> Can you share with us some of the mentioned meditations?

<comment><USER14> Many different kinds of meditation can end up with that result. In some traditions it starts with the cultivation of samatha and incorporates piti.

Zazen works pretty well for me. ymmv.

</comment></comment></comment></comment><comment><USER16> Any competitive 5-HT2A antagonist will block psychedelic effects from classical psychedelic drugs, and D2 antagonists will block the potential for stimulant-esque psychosis that drugs like LSD can trigger through dopamine agonism.

There are probably hundreds of drugs that fall into these classes, and many of them have significant clinical data to derive their safety profiles from.

If someone comes into a hospital with a bad reaction to psychedelic drugs, it's not uncommon for them to be given something like risperidone, both a 5-HT2A and D2 antagonist, and an anxiolytic to calm them down.

</comment><comment><USER17> >LSD has been classified in the vast majority of countries as a Schedule 1 illegal drug for more than 50 years, ever since the adoption of the United Nations' Convention on Psychotropic Substances.

>That scheduling means regulators believe the drug has "no currently accepted medical use" — and according to scientists from Imperial College London's Centre for Psychedelic Research, it's had a hugely negative impact on LSD research.

The US fucked up a lot of research forcing other countries to sign that convention.

</comment><comment><USER18> What was wrong with Niacin? 1500mg of Niacin stops it in less than 20 minutes for me.

<comment><USER19> It seems odd (in a funny manner) that you'd need 1500mg of something to counter the micrograms of the thing you previously took.

<comment><USER20> I think this is pretty normal in biology. Like how someone might have to drink 2L of water to feelbetter after a few shots of alcohol

</comment></comment></comment><comment><USER21> Kind of defeats the purpose. If you can bail you don't have to figure it out.

<comment><USER22> Sometimes trips can really get out of your control and can get pretty scary... might make sense to allow people to bail from that.

</comment></comment><comment><USER23> People regularly use Trazodone [1] as a trip killer. I personally haven't and I don't like using them, but it does its job pretty well from what I've heard.

[1] [https://en.wikipedia.org/wiki/Trazodone](https://en.wikipedia.org/wiki/Trazodone)

</comment><comment><USER24> Now im curious how effective a placebo is at ending a bad trip

</comment><comment><USER25> Wow. So basically they re-invented Xanax? :P

Joking aside, No way there is any chance that you would stick this in a human (huge potential for harm, no real clinical use case).

LSD is an insanely potent 5HT agonist, it seems highly unlikely any drug that is not "serious" would cause any significant reduction in LSD effects. Benzo's like Xanax can make the trip go in a direction that is less negative... but you are still tripping balls.

</comment><comment><USER26> Are there typically lingering physiological effects on the brain after taking LSD? Would this counteract those, or merely the psychological aspects?

</comment><comment><USER27> On a related note, lavender oil and theanine have recently been (preliminarily) shown to increase the likelihood of a good trip: [https://www.reddit.com/r/shrooms/comments/gwh87x/scientific_...](https://www.reddit.com/r/shrooms/comments/gwh87x/scientific_revelations_about_what_causes_good_and/)

which references this article: [https://www.nature.com/articles/s41386-020-0718-8](https://www.nature.com/articles/s41386-020-0718-8)

</comment><comment><USER28> They aren't being clear on what they are doing exactly but Why not just use Ketanserin. It's what they have been using on mice for a while now .

</comment><comment><USER29> note: i'm not a doctor or anything related.

Wikipedia says "5-HT2A antagonists block the psychedelic activity of LSD"[0], so wouldn't 5-ht2a antagonists like mirtazapine work well enough here?

No idea as to how quickly the trip would be tempered, though

[0]: [https://en.m.wikipedia.org/wiki/Lysergic_acid_diethylamide](https://en.m.wikipedia.org/wiki/Lysergic_acid_diethylamide)

<comment><USER30> Yes, or at least it blocks effects if you take it pre-exposure. Risperidone, an atypical antispychotic, is also known for this. Some idiotic big pharma startup will try and milk this, rather than examining cheap existing medications. Plenty of medications block 5-HT2A, I think the important requirement here would be rapid absorption and high receptor occupancy.

<comment><USER12> Anecdotally, I've used risperidone specifically to abort LSD trips on two occasions and found it quite successful both times. Now when I introduce people to LSD I give them a Risperdal to keep in their pocket "just in case", though to my knowledge none of them have had to use it.

<comment><USER31> Simply knowing you have a backup strategy if you panic can go a long way to preventing panic attacks. Psychiatrists sometimes do something similar where they write a script for a benzo, just one or two for your bag.

Knowing you can control panic if it occurs goes such a ways in preventing panic at all.

<comment><USER32> Anecdotally, one of the best treatments for social anxiety I've found is to always know where the exits are. At a concert, identify the exit signs. In a conversation, have someone who can take over for you.

Knowing there's a way out of the situation can completely nullify the scariness of that situation.

</comment></comment></comment></comment></comment><comment><USER33> The old urban legend is that Vitamin D brings you down and Vitamin C, up. Milk if tripping too hard, orange juice to trip harder.

<comment><USER34> That's a folksy legend indeed, where is that from? The 1960s?

<comment><USER33> I'm not sure how far back it goes, but it was popular in the 90's.

</comment></comment></comment><comment><USER35> Xanax?

<comment><USER36> I came here to see this. We didn't have Xanax in Sweden growing up but we did carry Valium or Xanor. Xanor has the same active ingredient as Xanax. Someone claimed Valium would help too but I doubt it.

I only ever heard of this being used once to stop a bad trip, and I wasn't present.

<comment><USER37> All benzos are pretty much the same, why would xanax work but not valium?

</comment></comment></comment><comment><USER38> "On April 1, MindMed announced...".

Haven't their marketing droids heard of April Fool's Day? Or is it just a UK thing...

</comment><comment><USER39> Not loading. Internet hug of death? Anyone describe what they are talking about?

<comment><USER40> Cached: [https://webcache.googleusercontent.com/search?q=cache:fL9NNm...](https://webcache.googleusercontent.com/search?q=cache:fL9NNmjWngkJ:https://www.freethink.com/articles/lsd- trip)

</comment></comment><comment><USER41> I'll stick to RCD trip off switches :-)

<comment><USER42> Whats that?

<comment><USER41> Residual Current Device - a circuit breaker that opens a circuit when too much current is not returned via neutral, implying leakage to earth. They "trip" too and with the similarity to LSD, a TLA ending in D! I couldn't resist the joke.

</comment></comment></comment><comment><USER43> That's terrible.

We're going to have fewer 'unicorns' ;-)</comment>


N.S.A. Calls Violations of Privacy 'Minuscule' - <AUTHOR> http://www.nytimes.com/2013/08/17/us/nsa-calls-violations-of-privacy-minuscule.html?hp

<comment><USER2> Officer, I only murdered one person - that's minuscule compared to the amount in a year! Surely you can let me go? Comparatively, I've basically done nothing wrong.

Excellent defence..

</comment><comment><USER3> Well, compared to the size of all of Utah, that place for storing your documents and communications is "miniscule."

The striking thing is that they say things like this while, simultaneously, Wyden and Udall are warning everyone it's the tip of the iceberg, and there are vast quantities of documents yet to be released by Greenwald. What do they expect to happen?</comment>


CERT Secure Coding - Consider using Gotos - <AUTHOR> https://www.securecoding.cert.org/confluence/display/seccode/MEM12-C.+Consider+using+a+Goto-Chain+when+leaving+a+function+on+error+when+using+and+releasing+resources

<comment><USER2> I can see how this works but it still looks unmaintainable.

I avoid _all_ goto, continue, early-return and similar statements though successive nesting. This solves problems:

\- Code quickly shows when it has become too complex and needs to be split up.

\- Code is naturally arranged in a way that makes it easier to split into functions.

\- Code is naturally arranged to clean up a resource only if it has been allocated (no extra labels) and as soon as it is no longer needed.</comment>


Air Force's new tool: 'We can see everything'   - <AUTHOR> http://www.washingtonpost.com/wp-dyn/content/article/2011/01/01/AR2011010102645.html

<comment><USER2> And so can Patrick Stewart: <http://www.youtube.com/watch?v=Fg_cwI1Xj4M>

</comment><comment><USER3> Information without action is merely trivia. Let's hope that information is put to good use, and used for good.

<comment><USER4> This is the US military we're talking about - they're in the business of blowing things up on behalf of US economic interests.

<http://en.wikipedia.org/wiki/War_Is_a_Racket>

</comment></comment><comment><USER5> when will the US stop? anyway the data collected is immense; probably machine learning can help, probably something like adaptive image-capable classifiers?</comment>


Show HN: A spectral ocean wave modeling framework - <AUTHOR> https://github.com/wavebitscientific/wavy <comment><AUTHOR> wavy is a spectral ocean wave modeling framework. It is a Fortran library you can use to simulate ocean surface waves.

Why wavy? Existing ocean wave models (e.g. Wavewatch III, SWAN, WAM) are built as opaque black-boxes, and they are difficult to extend or incorporate as part of another project. wavy redesigns the approach to wave modeling as a transparent and reusable library.

This project will be most interesting to ocean wave modelers. It will also be interesting to some Fortran programmers. Finally, it may be interesting to anybody that is curious what a modern Fortran project looks like. In early development, and open to contributors.</comment>


Marshmallow-jsonschema – portable descriptions for marshmallow schemas [python] - <AUTHOR> https://github.com/fuhrysteve/marshmallow-jsonschema <comment><AUTHOR> This is just a prototype, but it is an attempt to solve a big problem I've had: describing a schema to a remote client system (browser / mobile / etc) in a portable way.

Sure, I could just use a traditional form library to render HTML (such as wtforms, etc), but what if my client doesn't understand HTML? Or even if it is a browser - what if my template system resides on the client side (in javascript), and I don't want to make presumptions about how my javascript chooses to render the form on the server side.

JSON Schema ([http://json-schema.org/](http://json-schema.org/)) provides a way to describe schemas in a language / system agnostic way.</comment>


Sony beat the Apple TV without Google TV - <AUTHOR> http://www.ajaimk.com/2010/09/04/sony-beat-the-apple-tv-without-google-tv/

<comment><USER2> There's a ton of connected TV's which do this. Samsung/LG/Sony - they all are playing in this space.

The big downside is that right now most of the TV chipsets (outside of the latest models) are severely underpowered, and no one wants to upgrade a TV, when they can upgrade a box for about 10-20% the price.

If you want a TV with all the cool interactive bells and whistles wait for the TV's which can run full flash 10.1 - not so much for the flash, but so you know you have something that will be able to handle much of what's being thrown at it.

<comment><USER3> I don't see a compelling reason for a smart display when there is so much competition for cheap boxes that drive displays.

</comment></comment><comment><USER4> If you need any of

* MKV support

* 1080p support

* Subtitles (.sub/.idx/.srt)

* Remote control

XBMC on a Gen 1 Apple TV with a Crystal HD card is pretty simple to get going, and has decent performance.

I've tried:

* Gen 1 XBox running XBMC (no 1080p on a Celeron)

* XBox 360 with WMC (flakey codec support, requires separate Windows PC, no subtitle support, remote control via Media Control is bad). Microsoft fluffed this one so bad. All the codec packs for it require you hacking your standalone host system to bits to get playback working.

* DLNA on my Sony Bravia to PlayOn. PlayOn sucks, to put it kindly. Still requires a seperate standalone PC.

In 2008, XBMC streaming over SMB directly from a NAS was the answer.

It still is, if you don't want to muck around for hours fiddling trying to get stuff going by following the cargo cultish forums that exist for video playback (blind leading the blind).

<comment><USER5> I've got XMBC on an Acer nettop which as well as being an actual full computer, cost the same as an Apple TV with the same size hard drive. It also has a powerful video card that means you don't need to add a video accelerator card, just add a remote (either a cheap USB-IR model, or use your bluetooth playstation remote, or your Android or iPhone, iPad or netbook).

Unlike Apple, who expect you to have their stuff out on show, it came with a mount that attaches to the back of your TV which hides it's cheap'n'cheerful plastic exterior while minimizing cables.

I believe xmbc also does DNLA too if you wanted to plug it into one TV via HDMI and stream to another.

</comment></comment><comment><USER6> My Samsung TV has most (if not all) of that, and already has Hulu Plus.

And Hulu Plus was a huge disappointment. For those that don't know, ONLY 'plus' content can be streamed to a device directly. You can't watch the vast majority of Hulu's catalog directly on your TV/PS3/whatever. It cost me $10 to learn that.

<comment><USER7> Also, doesn't Hulu Plus have ads? I will pay $10 for a bunch of ads when hell freezes over.

(This is why I don't read magazines.)

<comment><USER8> Yes. I could stomach the ads during the TV programs but the commercial breaks during Hulu-streamed movies really sucked. There's definitely something to be said for the commercial-free programming available via Netflix streaming.

The Hulu Plus content was also disappointing (compared to what's offered on the free Hulu site).

</comment></comment></comment><comment><USER9> Apple has missed a trick it should have learned from the iPhone in the value of allowing controlled third party apps on to a platform. Apple TV + downloadable/purchasable iOS apps aimed solely for TV use would be awesome. Or maybe they're hoping to pull the same idea off using the iPad->Airplay->Apple TV integration coming later in the year?

<comment><USER10> You hit the nail on the head – if I was a betting man, I'd put as much money as I had on a 'surprise feature' of 4.2 being that you can use the Apple TV as a screen for outputting from an iOS device.

Why do I think this? <http://twitter.com/lllucas> \- Apple employee, apparently worked on Airplay (note the "happy airplay day!" tweet) Joe Hewitt (of Facebook) tweets "If only iOS apps could use AirPlay to stream their own stuff to AppleTV." and then lllucas replies "what makes you think they can't? :-)"

Why might I be wrong? It doesn't necessarily make sense that an app could output arbitrary video data (namely, what we're all thinking of, 3D games) to an Apple TV unless there was something seriously cool going on behind the scenes, like using a H.264 encoder in hardware to compress the stream.

If that's happening, then I would think certain devices (such as the iPod touch 2G/3G, iPhone 3G) would be incompatible because there's no reason for them to have an H.264 encoder on their chipset since they don't have video cameras. This would be an odd thing for Apple/app devs to try to support, because now you have to worry about OS revisions + device capabilities, and Apple's been fairly good about coming down on the side of keeping things progressing in software even if the hardware is too slow for it in reality (see the iOS 4.0 on iPhone 3G debacle)

I think it's certain given those tweets that you'll be able to at least stream arbitrary H.264 content from your app to the Apple TV, but to be encoding whatever is on the screen in real time and also be putting load on the graphics processor...I'm not sure that would work.

<comment><USER9> I think they already announced that, actually (?): <http://www.apple.com/ipad/software-update/>

<comment><USER10> It's definitely in Apple apps, and I imagine that 3rd party apps will get it for "free" using the standard media player classes. The issue is if apps could send arbitrary video data, because then your $99 Apple TV turns into a game console if you have an iPod touch.

<comment><USER9> Good point - that would be like the VGA adapter. It'd be like having a DS where the top screen could be 30 inches or more.. :-)

</comment></comment></comment></comment></comment><comment><USER11> The funny thing about buying a TV roughly every two decades (currently a twelve year old Sony Trinitron is doing its duty) is that I just don't know that there are TVs which can do that.

It seems that TVs have turned into twins of the bloated laptops Sony likes to sell. I'm not sure whether I like that. Is that software at least user friendly?

<comment><USER12> The position of the TV manufacturers reminds me a bit of the ISPs. The ISPs don't want to be just a dumb-pipe so they add on "features" that few want. And the TV manufacturers are the same way. They no longer want to be just a dumb monitor, so they're embedding these features that would be better implemented in an external box that's replaceable.

As far as user-friendly, my experience is that the interfaces look flashy on the surface, but once you dig deeper, you'll find poor integration, bugs and rarely a new release. All the better reason to rely on an upgradeable, external box for these types of features.

</comment></comment><comment><USER13> It is obvious that Apple's direction is streaming. Streaming from the cloud is easily done. Streaming from users content is via AirPlay. Apple probably looked at DLNA. It looks unlikely that AirPlay will support 'non-Apple' format such as divx, mkv, flac. And I hope Apple will change its mind. It is not a Flash vs non-Flash issue.

Will Apple TV turns into an app & games platform? I see alot of pluses. Something not in the interactive TV as we know today. I think this is coming, based on what Google is doing. Google probably has some inside info that Apple is developing such, thus Google is developing its own Google TV to compete. But I think Apple's implementation will be something different. Maybe: using your iPad/iPhone as controller, select the game you want to display on your TV and push it via AirPlay, the game will then load and play on Apple TV. Then you use your iPad/iPhone as controller to play the game.

</comment><comment><USER14> This sounds like he is the kind of guy who in less than a year will be writing about how he wished he owned one and will be first in line for the next revision later blogging about how awesome it is.

</comment><comment><USER15> My Sony TV is just a TV, and that is the way I like it, no apps, no internet; just channel buttons. Currently I am watching "Starwars II: Attack of the clones" in high def and commenting on this post at the same time.

Two screens are a far better experience than trying to both large video and interactive apps in one device. This is one of the reasons that interactive TV has never taken off. It just isn't feasible for general consumption.

<comment><AUTHOR> I'm personally not a big fan of the interactive TV idea myself. What I am all for is a TV that connects to different sources of media. I have music from Pandora and movies on netflix.

</comment></comment><comment><USER16> "AirPlay (which is kind of cool but inherently flawed in the fact that when I switch it to the TV from say my iPad, I'd like to use the iPad for some other task.)"

By the time this comes to the iPad, it'll be iOS 4.2 which probably means they've designed this as a background task, which would allow you to do other stuff on your iPad.

</comment><comment><USER17> There's a company called PlayOn. They use DLNA to stream things like Netflix, Hulu, and your files to DLNA capable devices. DLNA is actually built into 8,000+ devices such as XBOX,PS3, and some of these snazzy new TVs. PlayON is installed on one computer and then streams the rest. Is anyone here well versed in DLNA?

<comment><USER12> I've used a lot of DLNA devices and servers over the last few years. Most are "meh". They work, but most seem to have a user-interface reminiscent of Windows 3.1 especially when dealing with video. They typically have no photo or video metadata knowledge so you're limited to browsing through directories of files and selecting what your want by filename. Have your non-techie significant other try that. No movie-poster art, IMDB synopsis, etc. When you have a library of hundreds and hundreds of ripped DVDs, good luck.

And format support is spotty as well. There's often a mismatch between what the server will serve up and what the device will play back.

I have a Sony TV with most of what was shown here and I rarely, rarely use any of the Internet features. There's DLNA support built-in, but of the 800 or so videos I have being served up on my network, it refuses to even recognize any more than a dozen or so. Everything else is either ignored or causes an error. And the user-interface is completely divorced and separate from my set-top box. if I want to use the Internet features, I have to find the TV remote vs. the cable-box remote and then deal with a different user interface. The lack of transparent integration is a deal-breaker for many people. Switching back and forth between Internet video sources and "normal" sources has to be as seamless as changing channels. Google and Echostar are working together, so they may pull it off, at least for Echostar set-top boxes. Otherwise, without tight-integration, these efforts are going to be a flop.

</comment></comment><comment><USER18> Does it seem to anyone else that he has missed a few other service integrations that Apple mentioned?

Now, I'll give you that the Sony TV probably is capable of more, but I thought the Apple device was going to do YouTube and Flickr at least.

</comment><comment><USER19> Apple TV is an experiment. They're trying to find a market fit, so to speak. If you're comparing anything Apple is doing to an existing product, you're looking at it the wrong way.

<comment><USER20> Agreed, and at the $99 dollar price point, you can tell they're trying to make it work. Unfortunately, I don't know anybody technical or non-technical that would enjoy it.

<comment><USER21> the market is already there, every iPad, iPhone, and iTouch owner will get one. God its cheaper than an iPod nano.

</comment></comment></comment><comment><USER21> These are my thoughts on this topic: <http://news.ycombinator.com/item?id=1660396>

</comment><comment><USER22> but the guy has an iPhone 4 instead of an android phone...</comment>


Ask HN: Is there a flat fee, web based ad serving service? - <AUTHOR>

I'm looking for a flat fee (upfront or montly) ad system that doesn't try to take a percentage of transactions.
Yes, could use phpadsnew (aka openx or whatever they're calling it these days).
But I'd rather have just a simple javascript based program that gives basic 'banner ad' serving ability.  (Or even text.)
Thank you!! <comment><USER2> Use a content delivery network and track the clicks yourself ?

Instant nearly unlimited capacity and no percentage taken, other than what it costs you to serve the ads.

<comment><AUTHOR> Thanks, that would work. But I'd love more functionality, the ability for people to control their own ads that kind of thing.

In this day of age, I don't think I should have to build it.

<comment><USER2> ok, so how about this then ?

<http://oasis.sourceforge.net/></comment></comment></comment>


Better understand the bias of your news diet - <AUTHOR> https://aaronzlewis.com/blog/2018/11/25/bias-barometer/ <comment><USER2> I'd be interested in a tool like this but I basically only look at content like that (i.e. news) on my phone. Has anyone tried the 3rd party apps that claim to be able to add Chrome extensions to Android?</comment>


Ask HN: Startups in Brazil? - <AUTHOR>

I really would like to know about startups in Brazil, and if they are hiring. I am a polyglot myself (9 languages) and a lisper (but not uniquely so, and I learn quickly) and I would love to work at a Brazilian startup. And I am sure there are more brazilians on Hacker News that would like to know about each other. <comment> doc-film mixergy.com</comment>


Ask HN: How to improve code quality at a new job? - <AUTHOR> I recently was hired to a new position as a senior developer (probably in the upper 1/3 of seniority) and found that their code quality is kind of abysmal. The code base is pretty sizable and has a bunch of issues: one giant project with all the code in it, very few unit tests, 15 levels of interfaces for single objects, stuff like that. I want to bring it up to my boss and try to push the team toward better practices but I'm not sure what the right way to do this is. I'm not sure whether it's from a lack of knowledge on the part of the team or an old cold base that they just keep iterating on. It's not really feasible to suggest they stop doing new work in order to fix all these problems. Does anybody have experience in a situation like this? What's the right way to push overall code quality in the right direction? <comment><USER2> We've had this problem with an old codebase. Some things that happened at my company. Note that this was driven by my boss, who is excellent, and not myself.

* Started collecting metrics on code quality. Sonar was setup to do analysis every day. We could see lines of code + test coverage. Its harder to argue against facts. * Introduced better guidance on what is required for a change (detailed tickets, unit tests) and started enforcing code reviews. This can help reduce the "damage" being caused. * Started educating management/business sponsors on why code quality is important. Explaining that poor code can/will make business changes more difficult, they were happy for us to dedicate more time on improving. Used the metrics as evidence. * Hired people who shared his vision. Soon after he joined, he hired others who were dedicated to writing high quality code and keen to refactor the codebase. * Identified and prioritised the biggest issues. Estimating developer time cost vs business/development value. These required sign off from business sponsors/management.

We still have problems trying to get everyone onboard. Often it can be easy to get everyone to verbally agree to address an issue, but walking the walk is never as easy as talking the talk!

Its been 2/3 years and looking forward there's still a whole mountain to tackle, it seems like we've barely scratched the surface. Looking back, you can see how bad it was and it has actually improved significantly.

Good luck!

</comment><comment><USER3> Take a look at this old article by Joe: " _Getting Things Done When You 're Only a Grunt_" [https://www.joelonsoftware.com/2001/12/25/getting-things- don...](https://www.joelonsoftware.com/2001/12/25/getting-things-done-when- youre-only-a-grunt/)</comment>


Why New Zealand is a lifestyle superpower - <AUTHOR> http://news.bbc.co.uk/1/hi/programmes/from_our_own_correspondent/8683377.stm

<comment><USER2> They do make good wines, though I've mostly had the whites (spouse's preference).</comment>


Behind TaxiForSure's sellout - <AUTHOR> http://www.livemint.com/Companies/t7TozTlZCAmvtSxog3OQ7L/Behind-TaxiForSures-sellout.html

<comment><USER2> "It would have required one more guy in the world to say, 'Yes, we will fight SoftBank'. It never happened, but that's okay."

Got me there. i like their spirit.</comment>


Announcing DatHTTPD - <AUTHOR> https://pfrazee.github.io/blog/announcing-dathttpd <comment><USER2> This is a part of the Beaker p2p browser project [1]. This server lets you host websites over the Dat protocol with DNS shortnames, as well as HTTPS as a fallback for (let's say) "legacy" browsers.

Happy to answer questions.

1\. [https://beakerbrowser.com/](https://beakerbrowser.com/)

EDIT: direct link to the repo [https://github.com/beakerbrowser/dathttpd](https://github.com/beakerbrowser/dathttpd). We also have Prometheus/Grafana integration which is pretty handy; it's currently the easiest way to watch the health of a swarm.

<comment><USER3> Is there any other way to convert from short-ish names to dat addresses, currently or planned?

(If short names always depend on some content hosted at [https://same- domain.tld](https://same-domain.tld), what's the point?)

<comment><USER2> Yes - In Beaker we have plans for a userland namespace, which will be under the "app://" scheme. And we may someday move from DNS-over-HTTPS to something like DNSSEC, but that's not planned yet.

<comment><USER4> I'd love to see a kind of reverse-DNS-style tagging system, so that when my browser comes across a dat:// link it can automatically show me that this content is commonly tagged as "book, Fahrenheit 451" or such.

<comment><USER2> That's not far off from what we're thinking.

</comment></comment></comment></comment></comment><comment><USER5> This is really cool. Nice to see the Dat protocol[0] getting more uses and implementations.

[0] [https://datproject.org/](https://datproject.org/)

<comment><USER6> I think that's the wrong URL. I believe you meant [https://www.datprotocol.com/](https://www.datprotocol.com/)

<comment><USER2> Both are right. Dat project is the nonprofit that runs the protocol development

<comment><USER5> pfraze, what are you impressions of Upspin?: [https://news.ycombinator.com/item?id=13700492](https://news.ycombinator.com/item?id=13700492)

<comment><USER2> Very sensible design decisions and it's hard not to get excited about Rob Pike's involvement. I think Dat confers significant advantage by being hostless, but with Google's name behind it, you never know where Upspin could go. We have plans for secure key-distribution via Dat that's similar to Key Transparency, but it'll take us a couple months to a year to implement it. I feel bad for the Keybase team right now, that's for sure.

<comment><USER5> _> but with Google's name behind it_

Yeah, given the developers, I wonder how seriously to take the _" not an official Google product"_ statement.

 _> I feel bad for the Keybase team right now_

I wonder what they are going to do to respond to these developments...

</comment></comment></comment></comment></comment></comment><comment><USER7> Has anyone tested Dat and also knows ZeroNet and IPFS? How do they compare? Dat seems very similar to ZeroNet.

<comment><USER8> I can chime in with a different perspective here - code quality.

I helped out ZeroNet for a while. I even ran a bridge for public access without ZeroNet for a while.

The main task I was undertaking was bringing in testing.

There were large problems.

The main developer refused to use pip for dependencies, a misunderstanding that it requires admin privileges.

So instead, the dependencies were half a list that the user must install, and half lived inside the git repo.

I suggested git submodules but it was shot down as inconvenient for manual testing.

I countered that the dependencies were increasingly out of date, including some critical security issues - and was told to go find another project.

On the other hand, IPFS has been incredibly professional, really on top with critical security bugs.

Beaker has been a rather pleasant experience when patching any holes that appear.

<comment><USER9> "I countered that the dependencies were increasingly out of date, including some critical security issues - and was told to go find another project."

Classy... Note to self to never consider that project.

</comment></comment></comment><comment><USER10> The "Configure the variables in this post" is an incredible touch that makes reading the examples so much more instantly relatable and recognizable. I will definitely be stealing this pattern for my own purposes!

</comment><comment><USER11> This is perhaps slightly off-topic, but a concern I've had with systems like IPFS and Dat is that they seem to harm privacy, because objects are not encrypted, and consequently also would among other things not be censorship- resistant, and be more likely to cause legal issues for an owner of a node which ends up holding illegal content involuntarily. Has there been any work in this regard?

<comment><USER12> I've discussed this at some point in the IPFS IRC channel.

Their line of reasoning was that you could just as easily host encrypted blobs on IPFS and be done with it. That somewhat appeals to me, encryption of content seems orthogonal to hosting it. It feels like the Unix dogma 'do one thing and do it well'.

<comment><USER11> You could do it yourself, but in 2017, I'm not sure optional DIY privacy and censorship-resistance is acceptable. It ought to be done transparently and automatically by the implementation.

Encryption isn't slow, it wouldn't hurt performance or anything like that. Plus, if everything's encrypted, it doesn't attract attention to the people who appear to have something to hide.

The model I have in mind would be what mega.co.nz does: store opaque ciphertexts, and distribute links containing the keys to said ciphertexts. So instead of a link to someproto://hash/ you'd link to someproto:://hash/decryptionkey/. And your IPS/Dat/whatever client would handle decryption for you.

</comment></comment></comment><comment><USER13> would be really interesting to see a social network built on top of this.

<comment><USER2> stay tuned

</comment></comment><comment><USER14> How does Dat compare to Noms?

[https://github.com/attic-labs/noms](https://github.com/attic-labs/noms)</comment>


Show HN: Jwalk, a Streaming JSON Parser for Unix - <AUTHOR> https://jwalk.sh <comment><USER2> A JSON parser in Bash, impressive. I've just finished an HTTP server in Bash and had to rely on _jq_ for parsing JSON requests, but maybe would love a pure POSIX solution instead. Do you happen to have any benchmarks?

<comment><AUTHOR> Performance depends on what toolset you're using and how large your data set is.

For small JSON documents, like a _package.json_ file, jwalk is around ten times slower on my machine: 0.03 seconds, vs 0.004 seconds for jq.

For larger documents, like the 1.5 MB GitHub API response tree.json [1] in the jwalk corpus, jwalk on mawk takes 0.37 seconds vs jq's 0.11 seconds, but jwalk uses just 1.18 MB of memory, vs jq's 5.81 MB.

Other interpreted parsers don't fare as well: JSON.awk [2], which only runs on gawk, takes 0.8 seconds and 27 MB to parse tree.json; JSON.sh [3] takes 201 (!) seconds and 43 (!) MB, and JSONPath.sh [4] takes 7.4 seconds and uses 1.7 MB.

[1] [https://raw.githubusercontent.com/shellbound/jwalk/master/te...](https://raw.githubusercontent.com/shellbound/jwalk/master/test/corpus/tree.json)

[2] [https://github.com/step-/JSON.awk](https://github.com/step-/JSON.awk)

[3] [https://github.com/dominictarr/JSON.sh](https://github.com/dominictarr/JSON.sh)

[4] [https://github.com/mclarkson/JSONPath.sh](https://github.com/mclarkson/JSONPath.sh)</comment></comment>