Principles of a Decentralized Web - <AUTHOR> http://nicola.io/decentralized-principles/2015/ <comment> aaron-lebo Am I missing a fundamental part of the article? There is almost no content here.

This is less of a complaint about why this was upvoted as it is a question of why.

<comment><USER2> I think it's a response to this [https://ipfs.io/ipfs/QmNhFJjGcMPqpuYfxL62VVB9528NXqDNMFXiqN5...](https://ipfs.io/ipfs/QmNhFJjGcMPqpuYfxL62VVB9528NXqDNMFXiqN5bgFYiZ1/its- time-for-the-permanent-web.html)

</comment></comment><comment><USER3> The TCP/IP equivalent layer: [https://ethereum.org](https://ethereum.org)

The WWW/browser equivalent layer: [https://www.youtube.com/watch?v=IgNjs_WaFSc](https://www.youtube.com/watch?v=IgNjs_WaFSc)

Done. *at least as an awesome starting point. The easier it is for developers to build on the platform and the more network benefits/incentives that arise, the more high utility Dapps will be deployed, the more likely Chrome/Firefox/IE will integrate seamless DNS/pointers to the Ethereum platform. The web will become decentralized by default and the average user won't even notice.

And I mean "platform" in the sense here of a distributed blockchain or lower- stack layer, not some corporate-owned data silo built on proprietary API endpoints and protocols.

<comment><USER4> I'm excited by Ethereum. It definitely is interesting to think about distributed computation like that. However, doing it on a blockchain, along with data storage might prove to be prohibitively expensive for various types of apps. UBS is looking at Ethereum to do a bond market. I imagine that's an appropriate use of the technology.

But with apps I personally want to make, I'm more interested in MaidSafe Network. It's basically a distributed data storage network, fast enough that it's good for communication/message passing in apps.

MaidSafe got me interested in making apps for it that are pure client-side JS, with the intention of them being served from the distributed storage. While I dislike running JS in the browser, I will support it when it means there's no server. For now, my intention is to write apps using the Elm language.

Other interesting work continues to be done in Tahoe-LAFS, I2P, and Freenet. There's also IPFS. I'm looking forward to the future of this tech!

</comment></comment><comment><USER5> Does the web here mean a network of hyperlinked documents, does it mean HTTP, or does it mean "browser stuff"?

Personally, I think we should retire the idea of The Decentralized Web. It doesn't help clear thinking. BitTorrent is one of the most successful decentralization stories out there right now, but it's 0% part of "The Decentralized Web".

<comment><USER6> I think most people on HN think of the web as ports 80/443, and bittorrent doesn't operate on those ports. The internet is the superset, the web is the subset.

<comment><USER5> I totally agree -- and not just on HN -- everywhere. Now let's update that title:

" The Principles of a Decentralized Port 80/443 Ecosystem "

Personally, I don't think it survives the update well:(

</comment></comment></comment><comment><USER7> I think all three of these principles are important but miss an obvious one that should come first. A decentralized web should have decentralization as its central principle.

If we're talking about our ideal pie-in-the-sky protocol (where people have total control over their data) then let's dream big. Web hosting should be completely decentralized. DDOS attacks should become infeasible because there are no dedicated servers to target and web services should already scale directly in proportion to usage. Entities should receive incentives for participating in the protocol, but have no feasible way of negatively affecting others no matter how much relative computing power they control in the network.

<comment><USER8> This. Decentralization needs to be the foremost feature of a decentralized system. No single points of failure. Scaling up and down with demand. Resilience and choice for everybody.

Money has been decentralized.

Social networking hasn't, yet.

Security hasn't, not really. Have you installed the latest SquirrelMail patches yet? You probably use GMail so you don't have to worry about it.

<comment><USER9> You should look into diaspora, an open-source decentralized social network. IMO diaspora has formidable foundation and interesting content. Highly recommend. However, as Eben Moglin said, the next facebook should never happen.

<comment><USER8> The next facebook should be a decentralized, open source platform. Like Wordpress was the "next LiveJournal".

</comment></comment></comment></comment><comment><USER10> I don't think control of data, user or otherwise, is an inherent part of decentralisation. If fact it may be the opposite.

So as a principle for a Decentralised Web its a contradiction. Maybe it works as a part of a more broad set of principles (eg. a Healthy Web).

<comment><USER11> Can you expound? Why is user control if data the opposite? To me this seems in keeping with the basic principles of decentralization of power, that I should have autonomy over my own domain.

<comment><USER10> In a truely decentralized system it would be hard to control the flow of data. It's simply not inherent in decentralisation.

Just because data isn't centralized doesn't __mean __users will have the control.

</comment></comment></comment><comment><USER12> An important feature of the decentralized web is that one Harvard/MIT person doesn't get to design it.

<comment><USER13> I'm not sure what your objection is. Why shouldn't one Harvard/MIT person design it? I mean, somebody has to.

<comment><USER12> Web centralization is a structural and economic problem that is not going to be solved by the tech elite. We don't need new protocols; the ones we have worked fine back when the web was decentralized.

<comment><USER14> Our idea with Portal ([https://portalplatform.net/](https://portalplatform.net/)) is to make it possible for regular people to run the exact same kind of Linux servers that many of us on HN run.

We're using stock Ubuntu cloud images w/CloudInit so everything that runs on Portal will run anywhere Linux VMs run. Once there are some standards for importing/exporting data we're going to make that a one-click operation. Users can already download all of their data and completely wipe all remote copies.

The original promise of the internet was that we would all control our own domain names. Instead we ended up @gmail.com facebook.com, and twitter.com. We can fix this for less than $10/user/mo now.

</comment></comment></comment></comment><comment><USER15> Somewhat relevant, the Open Mustard Seed project;

      The Open Mustard Seed (OMS) project is an open-source framework
       for developing and deploying web apps in a secure,
      user-centric personal cloud. The framework provides a
      stack of core technologies that work together to provide
      a high level of security and ease of use when sharing and
      collecting personal and environmental data, controlling
      web-enabled devices, and engaging with others to
      aggregate information and view the results of applied
      computation via protected services.

[https://docs.openmustardseed.org/](https://docs.openmustardseed.org/)

[https://github.com/IDCubed/oms-docs](https://github.com/IDCubed/oms-docs)

[http://p2pfoundation.net/Open_Mustard_Seed](http://p2pfoundation.net/Open_Mustard_Seed)

</comment><comment><USER14> HTTP, SMTP, and XMPP are already functionally decentralized. We still need decentralized protocols for web search, tweeting, social networking, VoIP, and a bunch of other things.

In a world where every person runs their own cloud server, all we really need to do is install the same apps as each other. Apps can just invent their own (open) protocols and other developers can make apps that interoperate. Standards can emerge naturally over time.</comment>


The lifestyle business bullshit (2009) - <AUTHOR> http://37signals.com/svn/posts/1686-the-lifestyle-business-bullshit <comment><USER2> Some people don't realize the implications of hours worked on effective wages. If you work more for the same income you are giving yourself a pay cut, and worse a free time cut. If you work less then you are giving yourself a raise, and increasing your free time as well, a double whammy.

Most people have trouble thinking like this though. They lose themselves in their work, filling up their time with trivial and menial work (or even, as is typical, spending a lot of time in the office not actually working very much) instead of trying to maximize their effectiveness. They think that somehow they shouldn't work too little because of some sense of a work ethic based on economic systems that are now outmoded. And too they think that earning a high income while working only a little is too much like being actually "rich". Nobody likes being rich these days, they like the money well enough, but they don't like the idea, they want to justify their wealth with long hours at the office.

<comment><USER3> There are still many menial jobs that cannot be automated (not cost effectively). Encouraging people with menial jobs to seek higher effective wages increases everyone's cost of living.

Until robots can do all the menial work, and build themselves too, a lot of people are going to have shit jobs. I don't see any way around that.

Philosophically, is it better to have more people each doing X hours/yr of menial work, rather than fewer people doing X+n hours/yr of menial work?

In the spirit of the recent thread about EVE, it seems to me that the impulse which game makers depend on to keep people playing games, particularly MMOs, is similar to the impulse that keeps people doing menial jobs. Both involve long hours spent doing uninteresting work, and rare random rewards. People are conditioned to feel good about getting through tedious real work as long as they get occasional random high-value rewards (salary or praise or both).

<comment><USER4> I don't think it isn't that we can't build robots that can do menial work (mcdonalds casheers, farm laborers, road workers) it is that the investment to get the return is not worth the cultural wall to replacing dead end depressing work for people who would otherwise have no jobs due to limited education or drive for excellence.

</comment></comment></comment><comment><USER5> I think it's mostly investors that make fun of lifestyle businesses. Other entrepreneurs, not so much.

<comment><USER6> Very true in my experience. And as recently as 4-5 months ago, I was listening.

Really not far from "brainwashing" as DHH called it. The 37Signals message is a very, very welcome and refreshing signal amongst the noise of investments, valuations, and exits.

</comment></comment><comment><USER7> Amen. There's more to life than business.

If you want to devote your life to creating and running a world changing business, do it.

If you want to devote your life to your family and kids, and use a "lifestyle business" to fund it all, do it.

Just remember one is not better than the other. To each his /her own.

<comment><USER8> I don't think it is that clear, businesses that people to devote their lives to also get called lifestyle businesses. People that aren't taking big rounds of funding, only reinvesting profits. People that don't intend to IPO or be acquired, rather are building a sustainable business.

Whether that is the official definition or not, that seems to be the distinction I have seen around here.

<comment><USER7> Agreed.

My point is simply that either way is fine. There is no right answer here.

</comment></comment></comment><comment><USER9> > It's been a long time since there was a direct correlation with the number > of hours you work and the success you enjoy. It's an antiquated notion from > the days of manual labour that has no bearing on the world today.

It has never been true. Success has always been primarily correlated with the circumstances of your birth. This is not an antiquated notion so much as a fabrication invented to trick the less fortunate into putting up with it.

In that sense, it is as relevant today as it has ever been.

<comment><USER10> I have no data on this, but my spidey-sense says you're both wrong. If I had to put money on it, I'd say:

\- success is correlated with hours worked (though it's by far not the only variable)

\- education is more strongly correlated with success than birth is

\- education is strongly correlated with birth

What do you reckon?

</comment></comment><comment><USER11> I have read on here several times that many entrepreneurs want to squeeze their entire working lives into a few short years, banking on the big payout that will set them free.

37signals, on the other hand, was in business for a long time before Basecamp was even released to the public. It has been a slow road, relatively speaking.

I respect what 37signals has done. It is the way I would want to build my own business. However, have they really worked fewer hours to reach some arbitrarily high income threshold? I would suggest probably not. It has just been spread out over a longer period of time.

<comment><USER12> But perhaps one could argue that this resulted in those hours being more productive because they were more spread out. So assuming what you're saying is true, they could have gotten more productivity out of the same number of hours as another business that condensed its work more.

</comment></comment><comment><USER13> First of all, "lifestyle business" is a terrible name. I'd like to know who coined that term?

To my main point - With risk of getting downvoted. Why not think bigger? Building business is hard. Be it local publishing house or web based product serving millions of customers all across the globe. It requires same amount of (to quote Churchill) "blood, toil, tears and sweat". Then why not to swing for fences? In smaller markets chances of survival might be higher but survival doesn't equate to success.

<comment><USER14> Seriously? There's loads of reasons why not, and while I agree that it requires similar amounts of blood/toil/tears/sweat I'd say there's a bunch of differences.

[Background: 12+ years as an early employee in fence-swinging tech startups, also started and ran/run a modestly successful wedding photography business with my wife]

Our wedding photography business is _definitely_ a "lifestyle business" by the usual HN definition. What a silly term and a silly set of ideas. For one thing, it's profitable. That makes is _more_ profitable than the _vast_ majority of startups. It gets old spending other people's money and I have to say that one of the things I found most informative and invigorating about the photo business was that it really re-reminded me the value of profit and the fundamental idea of being in business in the first place.

[It's also craaazy educational to be _any_ kind of founder. Honestly, standing up the photography business might be the single most valuable thing I've done in my _tech_ career]

Beyond that, our business (really my wife's business at this point.. I'm so busy with another startup that she does everything nowadays) makes a couple of dozen couples incredibly happy every year. I can't tell you how satisfying we find this. My wife shoots their weddings, becomes their friend, shoots photos of their children's first years.. It's a really incredible gift. The assumption that scaling up/out to serve a larger market would be such an obviously better idea is short-sighted and frankly kind of sad.

Don't get me wrong, I'm really glad that there are loads of people swinging for the fences and tackling huge problems that have huge markets. So many awesome things come out of that! But don't fall into the trap that ultimately leads to thinking that it's the only way to go.

<comment><USER7> That's awesome.

And the best part is that you already realize how awesome your business is and why it makes more impact on the world than any of the hundreds of random social-photo-buying-contact-managing startups that come out of the Valley every 6 hours.

</comment></comment></comment><comment><USER15> Lifestyle businesses are awesome if it is _your_ business (ownership as in equity, not 'responsibility for all the bugs'). Be careful if you are joining someone else's lifestyle business and make sure you know what you are getting into. Depending on the person, there can be nothing worse than only being employed to maintain the founder's lifestyle with no plans for growth, company or career-wise.

</comment><comment><USER16> I've said it before, and I'll say it again.

The only people who use "lifestyle business" as a way to downplay, minimize or trivialize a business are often people who have the "lifestyle" and are saying it to people who don't have the "lifestyle".

The best lifestyle to me? Being free of time and obligation to bills to truly chase any crazy idea you want and any funding that may or may not come it's way.

There's a lot to learn from building any business seen as a lifestyle business, namely, you pick up the end to end business skills you didn't have to go big.

The alternative? Get funded and pushed out when you don't have the business chops to grow.

Go hard or go home? Get real, I want to create great things forever, sustainably, not try, fail and go back to finding a job. The part of being successful that makes you a well rounded person is non-neogotiable. Maybe folks here are still in their 20's with no real other responsibilities or obligations or priorities. This is my second internet boom and I can say as much as I've enjoy every second, I enjoy living for a living just as much.

</comment><comment><USER17> You would think that this article is obvious to the point of not really being necessary to even write. But, I think hanging out on tech forums like HN you read about billion dollar buyouts, massive IPOs and basically I think it gives the impression that is the only way you can consider yourself successful is if you are a part of one of those deals.

I kinda equate it to wishing you were a rock star. It's a lot of show, but in most cases it doesn't result in any particular happiness. And it doesn't even result in a lot of financial success except in those extreme cases - and only for the early employees. There is a perception of excitement but even the day- to-day working long hours is sometimes not that much fun either.

On the other hand there's a lot of businesses out there that don't get much attention. Unfortunately it's not all that exciting to read about on HN with a headline like "I make a decent salary at a no-name company and have a nice, comfortable life"

</comment><comment><USER18> I work from home and I eat breakfast, lunch and dinner with my kids. I've built a successful startup and am launching another without compromising anything. It IS possible and our breed of entrepreneur is more common however underrepresented in the startup community.

<comment><USER19> "I work from home and I eat breakfast, lunch and dinner with my kids."

That's crazy impressive. I tried working from home and was only ever productive if I worked till early in the morning. Daytime productivity was terrible with 2 little kids detracting me... perhaps my house is just too small.

</comment></comment><comment><USER20> Yes, you can work very little and be very successful if you play your cards right. And there is nothing wrong with that, certainly. But hard problems are hard. There are a lot of easy problems I could work on, and probably get funding and then flip the hypothetical company for a quick profit, but I am more interested in the risky, hard problems that most people wisely would not invest in. If you look at the names that history remembers, they all worked their asses off to achieve something great.

</comment><comment><USER21> We're building an online business while traveling. We're 3 months in Mexico this time. Tomorrow we'll go for a month long trip by car around Mexico and we'll spend those long hours on the road talking about the next feature, enjoying the scenery or stopping for some tasty tacos somewhere. The million dollars may come one day but I don't take any chances: I spend my time capital wisely as you can't buy time. Once it's gone, it's gone. A

</comment><comment><USER22> I think the main point is, 37 signals runs a bit like a lifestyle business - agile, low stress, no overtime; because that's what they think is the most profitable way to run a software business. Even if they _wanted_ to do overtime, or were in a more cut-throat niche (i.e. taking on Facebook) they would run their shop the same way..

They aren't doing things the way they do because they think it's glamorous, or because it's easier.

</comment><comment><USER23> This is pretty trite nothingness, even for 37signals. I am not sure why it's been upvoted either as it's not even recent either.

</comment><comment><USER24> (2009)

</comment><comment><USER25> All work and no play makes Sally a dull boy. Or whatever. But seriously: don't over-work yourself. I mean if you really want to, go for it, but the quality of your work will decline with you. Meet some new people, go hiking, go outside, eat better, live better, work better, and your work will look better with you.

</comment><comment><USER26> Lifestyle business is just a business that is not scaled. That can be either because of its nature or by choice.</comment>


MailCore 2, a new email library for Mac and iOS - <AUTHOR> http://www.libmailcore.com

<comment><USER2> In the past couple months I have been building a mail app on iOS based on MailCore v1. I can't wait to adopt version 2! Congrats Guys!

</comment><comment><USER3> Congrats!</comment>


Async Iterators: These Promises Are Killing My Performance - <AUTHOR> https://medium.com/netscape/4767df03d85b <comment><USER2> I agree that the decision that Promise continuations run always in a new eventloop iteration will have a serious impact in performance. Nevertheless I think it is a great design, because it provides you with lots of guarantees. I have worked a lot with C# Promises (Tasks), where the implemenation allows synchronous completions, and that gives way more room for errors. E.g. whenever you call TaskCompletionSource.SetResult(xyz) you might directly jump into the completion handler which executes arbitrary code (and may even call recursively back into your code). If you don't know that, don't know when it happens (understanding all the SynchronizationContext stuff) and don't take precautions against it (like checking for state modification after promise resolval) you might run into hard to find errors. Similar things can happen on the using side. In JS-land a continuation will always run in the next eventloop iteration, which guarantees that code after Promise.then() will still get executed before it. In other designs the continuation might run directly in .then sometimes, which is also another thing to account for. In my opinion the JS promise spec provides the most sensible and least error prone behavior. In my opinion that's great, because for the average programmer correctness is more important than performance. The smaller amount of coded that really needs to have a very high performance can still be hand-optimized, e.g. by falling back to synchronous callbacks where it matters, or for the usecases of the link reading bigger parts of the file at once and then performing more of the parsing synchronously.

<comment><USER3> I appreciate the fact that you've taken the time to offer a comment which I personally think is interesting & valuable, but with all due respect, could you type in paragraphs please… it makes it a lot easier to read :)

</comment></comment><comment><USER4> Not sure which spec the article is referring to but it's not the one implemented by V8 natively.

> The problem is that Promises have to be resolved in the next turn of the > event loop. This is part of the spec.

That statement is false unless I'm missing something major here. Promises aren't resolved in the next turn of the event loop. They _should_ be resolved as part of the microtask queue which can happen at the end of the tick (recursively).

Example (tested on node 8):

        Promise.resolve()
          .then(() => console.log('1'))
          .then(() => console.log('2'))
          .then(() => console.log('still the same tick!'));
        setTimeout(() => console.log('timeout happening in the next tick'), 0);

Result:

        1
        2
        still the same tick!
        timeout happening in the next tick

The same will happen with async functions:

        (async () => {
          console.log(await Promise.resolve('1'));
          console.log(await Promise.resolve('2'));
          console.log(await Promise.resolve('still the same tick!'));
        })();
        setTimeout(() => console.log('timeout happening in the next tick'), 0);

<comment><USER5> You're correct. The spec only specifies that "onFulfilled or onRejected must not be called until the execution context stack contains only platform code." This doesn't require that the implementation use the same macro-task scheduling mechanism that setTimeout might use.

</comment></comment><comment><USER6> "Unfortunately, generators and iterators are synchronous. Unless we're willing to read our entire file into memory, they can't help us."

This doesn't seem to follow to me. Synchronous reading means you have to read a given chunk in before you can process it, which means that if you're not reading out of cache that you're going to stall on the spinning disk, but synchronous code doesn't require you to read the whole file at once. And you can easily write something that will process it into nice lines for you. The "standard library" may not do that because there isn't much standard library for Javascript, but it's not that hard.

The proof is, that's what the Python snippet is doing. It's built into Python in this case, and I would expect it does some magic internal stuff to accelerate this common case (in particular doing some scanning at the C level to see how long of a Python string to allocate once, instead of using Python- level string manipulation), but the API ought to be one you can do in Node, even synchronously, with generators.

<comment><USER7> >Synchronous reading means you have to read a given chunk in before you can process it, which means that if you're not reading out of cache that you're going to stall on the spinning disk, but synchronous code doesn't require you to read the whole file at once.

It's implicit that stalling the node.js event loop is not allowed, so a synchronous "line" generator or iterator would block the event loop when it has exhausted the data read so far. The node.js event loop cannnot suspend the current task and process other tasks while waiting for more data to be read.

Edit: The point of the author's post is that async iterators generate a Promise (and thus a yield to the event loop) for every element, even when the element is already available synchronously. Even if the generator has read three lines in the latest call to read(), it must still yield three promises of one line each (each involving one yield back to the event loop) before all three lines are processed.

Edit 2: That said, it's not hard to fix this, and the author actually comes close to the solution but doesn't realize it. They say the result of the async iterator in the spec should contain an array of values, but it's just as easy to write the "line" iterator to yield an array of lines as its values without changing the spec.

<comment><USER8> I think the problem with yielding an array of lines is that all the code that uses the result of the function would have to be aware of this grouping of results all the way up the stack.

What he was proposing was a solution that _looked_ like you were only returning one value per iteration (so none of the code doing the processing had to be aware there are arrays of values) but underneath the covers was actually being processed in batches on each run through the event loop.

It's actually a very elegant solution. The only code that would have to be aware that it's actually dealing with batches of values is the code producing those batches right at the top of the stack of calls.

<comment><USER7> That's correct, and I understand the author's solution. My point was that changing the spec is not a hard requirement, since it can be accomplished in user code.

<comment><USER8> Then I suppose we don't need async iterators at all since this can all already be done with callbacks.

</comment></comment></comment></comment></comment><comment><USER9> If you're using Node.js as a stateless web server (the principle use case motivating its design), when are you reading files off the local disk other than in the initialization phase? What's the problem with applying the 'right tool for the job' mindset and defaulting to other platforms for more generalized computing tasks?

<comment><USER10> > If you're using Node.js as a stateless web server (the principle use case > motivating its design), when are you reading files off the local disk other > than in the initialization phase? What's the problem with applying the > 'right tool for the job' mindset and defaulting to other platforms for more > generalized computing tasks?

what's the point using nodejs if you can't do a simple task such as reading a file? Use something else to begin with. NodeJS wasn't designed to be "stateless", no more than Javascript was. If you want "stateless" use clojure or Haskell.

</comment></comment><comment><USER11> Not a scientific test:

      var fs = require('fs'),
       csv = require('csv-parser');
      var start = new Date().getTime();
      fs.createReadStream('./Batting.csv')
       .pipe(csv())
       .on('data', function() {})
       .on('end', function() {
         console.log(new Date().getTime() — start);
       });

This takes about 400 ms on my machine to parse a 6.1 MB CSV (15.25 MB/s). I didn't do anything useful with the data, but I did verify it exercises the parsing function.

Clearly all the work/time is spent in the actual parsing and handling corner- cases in CSV trying to be correct over high performance/throughput.

</comment><comment><USER12> > (note: I'm using TypeScript to convert the async iterators and generators > into something that node.js can run. See my code here.)

This is your problem, TypeScript's Promises polyfill is really really slow. Switch TypeScript to emit ESNext and run it on Node 8 and you'll see a much much different situation.

<comment><USER7> TS does not provide any Promises polyfill.

</comment></comment><comment><USER13> I've written event-driven, non-blocking style C for a couple of different projects on POSIX platforms. When a file descriptor becomes readable, you read from it until read(2) returns EAGAIN[1] or until a condition is fulfilled, such as the read buffer containing a delimiter. People new to event-driven programming may just invoke read(2) once for each readable event, check for a condition and then yield to the event loop. This sounds like the same mistake, only with more turtles stacked on top of it.

[1] This applies to sockets and pipes (incl. stdin), not regular file I/O which usually ends up being off-loaded to a thread/process pool (if needed)

<comment><USER14> I don't think it's quite the same thing, but that description reminds me of Ted Unangst's "accidentally nonblocking" post.

[https://www.tedunangst.com/flak/post/accidentally- nonblockin...](https://www.tedunangst.com/flak/post/accidentally-nonblocking)

</comment></comment><comment><USER15> re: "Promises have to be resolved in the next turn of the event loop. This is part of the spec."

This seems imprecise? There are multiple queues and the details seem to be implementation-specific [1]. The Promises/A+ specification doesn't seem to specify a particular implementation [2].

The reason this might matter is that if you use promises to do something CPU- intensive without doing I/O, it might be possible to starve the event queue, similar to what happens if you're just writing a loop.

On the other hand, the example in the article is waiting on reads, so they'll be delivered through the event queue. Can reading larger chunks at a time and buffering be used to speed that up?

[1] [https://github.com/nodejs/node/issues/2736#issuecomment-1386...](https://github.com/nodejs/node/issues/2736#issuecomment-138607657)

[2] "This can be implemented with either a 'macro-task' mechanism such as setTimeout or setImmediate, or with a 'micro-task' mechanism such as MutationObserver or process.nextTick. Since the promise implementation is considered platform code, it may itself contain a task-scheduling queue or 'trampoline' in which the handlers are called."

<comment><USER16> I think the main concerns is that the following code happens in the expected order.

console.log("First");

promise.then(_ => console.log("Third"));

console.log("Second");

This means that even if `promise` is fulfilled already the callback must be delayed _at least_ until the next turn of the event loop.

<comment><USER15> Yes, it does need to be delayed until the stack is unwound, but it doesn't need to be delayed until after any events already on the main event queue. (I'm not sure what "the next turn of the event loop" means in the presence of multiple queues.)

</comment></comment></comment><comment><USER17> The number of excuses for node's poor performance here is shocking. Talk about high-level architectural concepts and quibble about the likelihood of this implementation being necessary if you like, but if you can't provide a decent implementation for this very basic use case, then node has a hole in it.

<comment><USER18> Node is not optimized for this use case at all. Try parsing 64 files at once then compare them again.

</comment></comment><comment><USER19> The biggest problem I had with forced promise asynchronicity was with browser promise polyfills, which use setTimeout to force success and error handlers to be called asynchronously.

This is fine, unless you're:

1\. Modifying the DOM and hoping to avoid jankiness 2\. Trying to work with popup windows, in a browser which deprioritizes setTimeout in all but the focused window.

I ended up implementing [https://github.com/krakenjs/zalgo- promise](https://github.com/krakenjs/zalgo-promise) to get around this. It intentionally "releases Zalgo" by allowing promises to be resolved/then'd synchronously, but my belief is this doesn't have to cause bugs if it's used consistently.

<comment><USER7> You're using poor polyfills if they use setTimeout. Good polyfills [1] use MutationObserver which uses the (a?) microtask queue.

Invoking a Promise's then() callback synchronously is a violation of the spec. IIRC there are some tests to prevent that in the test suite.

[1]: [https://github.com/petkaantonov/bluebird/blob/2c9f7a44/src/s...](https://github.com/petkaantonov/bluebird/blob/2c9f7a44/src/schedule.js#L31)

</comment></comment><comment><USER20> Interesting and surprising. Node 8 supports async/await as well as iterator functions. So the target for the typescript transpiler can be set to 'es7'. Would be interesting if this has an inpact on the performance.

<comment><USER21> Poking around with it, it looks like that gives a 20% or so improvement in node v8.4.0, from 1239ms to 1022ms (vs 56ms for the sync version).

</comment></comment><comment><USER22> Would be nice to say what version of Node.js.

<comment><USER23> It doesn't matter which version of node. The fact that promises must be resolved in the next tick is baked into the spec, no upgrade of node will ever improve that situation.

<comment><USER22> Right but there were a lot of specific numbers so the context for specific benchmarks is versions.

Anyway it sounds like we should try to fix the spec.

</comment></comment></comment><comment><USER24> My problem with promises is that you can't "uninvoke" them. I.e., if all listeners unsubscribe, the original job they were waiting for could be killed, but the design of Promises doesn't allow that. Instead, the job will continue to be executed (without any listeners), and thus wasting resources.

<comment><USER25> I won't argue that sometimes you might want to cancel a promise that you no longer need (ex: user cancels ajax operation) but just because there are no listeners does not always mean you don't want it to complete (ex: Fire and forget, not always a great idea sure but a thing nonetheless).

<comment><USER24> True, but you can implement fire and forget by adding a dummy listener. The functionality I'm suggesting is more general.

</comment></comment></comment><comment><USER26> For efficient text processing the only language that I'd use is Perl5.

<comment><USER27> Perl is great, but not that much (if any) faster to run than the equivalent Python, these days.

If you use perl regularly, then the $_ syntax stuff is super fast to type and do all kinds of cool stuff, but I find as a irregular scripter that I forget after 6 months or so. Python's monotonous syntax means stuff is generally still understandable even by non programmers.

</comment></comment><comment><USER28> In the documentation, there is a specific example for reading a file line by line, and it is very concise[1]. I'm too lazy to test how fast it is though.

[1]: [https://nodejs.org/api/readline.html#readline_example_read_f...](https://nodejs.org/api/readline.html#readline_example_read_file_stream_line_by_line)</comment>


Show HN: Emojifs – Create Animated GIFs from Emojis - <AUTHOR> https://www.emojifs.com/ <comment><AUTHOR> Hi All!

I'm calling this one of my ugliest MVPs I've ever launched, sorry. I really launched this as a way for me to easily take a script coded by Pieter Levels and shared on Twitter ([https://twitter.com/levelsio/status/1060282680338636801](https://twitter.com/levelsio/status/1060282680338636801)) to create GIFs from Emojis and make an easy to use form for myself.

</comment><comment><USER2> My keyboard has no emojis. Some default ones would be nice to see what this does.

</comment><comment><USER3> I dig it, people are already telling little stories with a string of emojis. It would save a step if you had reusables ones.

<comment><AUTHOR> On my "gotta make that happen" list.

trying to think of a way to do something like

[https://www.emojifs.com/🆒](https://www.emojifs.com/🆒) where people can place the emojis in the URL and it will generate the GIF ... just not sure how supported emojis in URLs are.

</comment></comment><comment><USER4> 🆒

🆕

🆓</comment>


Where can I find flash developers? - <AUTHOR>

My company is looking to find a php / flash developer that has experience creating a custom flash players, hopefully with experience with pdf2swf. Where should we post the gig? <comment><USER2> In other blogs these people would read, using services like the YC funded snaptalent which does just that :)

</comment><comment><USER3> on jobs here?

<comment><AUTHOR> I total forgot / missed that new feature, thanks :)</comment></comment>


Creating Cities From Scratch - barry-cotter http://www.fastcompany.com/node/1514547/print

<comment> barry-cotter Takeaway: 300k pop. city to go from drawing board (2001) to completion (2015), planned from when it was a reclaimable mudflat 40 miles from Seoul. It's intended to be an "aerotropolis" using Incheon airport to plug it into Asia's trade routes.

They're taking advantage of the fact that it's new build to make it dense, energy efficient, and very, very heavily wired and networked as well, and the article mentions Cisco an irritating amount as the lead partner on utilities and services.

What I found most interesting was the idea that they're using the experience they've gained so far to do the same thing elsewhere, faster. The idea of building a 300k city in under a decade on 1,500 acres (6 km²) blows my mind, and if this can be done (and it can) it makes Charter Cities suddenly more believable.</comment>


The category design pattern - <AUTHOR> http://www.haskellforall.com/2012/08/the-category-design-pattern.html

<comment><USER2> Leaving aside the category stuff for a moment, I think the first couple of paragraphs of this article make an excellent point.

> Functional programming is all the rage these days, but in this post I want > to emphasize that functional programming is a subset of a more important > overarching programming paradigm: compositional programming.

Indeed. So there is another article just begging to be written: "Introduction to Compositional Programming". I've seen any number of articles that say something like, "I especially like these because they are composable." But I've seen very little on figuring out how to solve problems by using composable components, or so that the solution itself is composable.

Another thought: we really should be giving more thought to composition when designing programming-language syntax and features. Haskell excels at writing composable components; it also has nice syntax for composing them. Both are much less natural in "C". Various replacements for "C" are being proposed; is thought being given to composability in them? Similarly, the popularity of Python's generators is largely because they allow for the easy design of composable components. Etc.

<comment><USER3> The point of this article is that you _don't_ want to leave the category stuff aside. A category is essentially a well-defined and systematic way to reason about--and program with--composition. An introduction to programming with categories would, by its very nature, have to be an introduction to programming compositionally.

Moreover, I think a introduction using terms and ideas from category theory would be more useful, more thorough and more enlightening than an introduction to "compositional programming" without any math. It provides a nice structure and enables you to consider composition more abstractly, which lets you use compositional programming in more places.

<comment><USER4> _> Moreover, I think a introduction using terms and ideas from category theory would be more useful, more thorough and more enlightening than an introduction to "compositional programming" without any math._

Yeah, this article left me wanting to know more about category theory. But that in itself is valuable. I googled and found a couple introductory papers [1,2], some slides introducing category theory for software engineers [3] (with a NSFW diagram on page 8), and an introduction to category theory as it applies to Haskell [4].

[1]: <http://wwwhome.cs.utwente.nl/~fokkinga/mmf92b.html>

[2]: (PDF) [http://www.staff.science.uu.nl/~ooste110/syllabi/catsmoeder....](http://www.staff.science.uu.nl/~ooste110/syllabi/catsmoeder.pdf)

[3]: (PDF) <http://www.cs.toronto.edu/~sme/presentations/cat101.pdf>

[4]: <http://en.wikibooks.org/wiki/Haskell/Category_theory>

</comment></comment></comment><comment><USER5> On an aside if you haven't checked out Gabriel's ``pipes`` library, its definitely worth a read. It's some of the most elegant Haskell I've had the pleasure of reading.

[http://hackage.haskell.org/packages/archive/pipes/2.2.0/doc/...](http://hackage.haskell.org/packages/archive/pipes/2.2.0/doc/html/Control- Pipe.html)

</comment><comment><USER6> This is a nice explanation about how category theory relates to programming, but it doesn't explain how mathematical rigor helps us in this area. Making a rough analogy with function composition and unix pipes seems sufficient to get the point across to most programmers. I suppose you could actually write generic code that works with more than one category but that seems like unnecessary and unhelpful abstraction, compared to just writing the code twice in more concrete terms.

<comment><USER3> There are several advantages to writing code at that level of abstraction.

It's a great way to encourage code reuse, even in fairly disparate domains. Haskell has some of the most versatile libraries of any language I've used by using mathematical abstractions like this. As a concrete example, there are functions like foldM which is like a normal fold except it can handle errors, state, nondeterminism or even parsing!

The abstractions are well-understood, which makes it easier to reason about. You get a bunch of theorems and conclusions for free just by formulating your program in mathematical terms. You can then easily take advantage of these to simplify, verify or optimize your code. For example, if you know that your data type forms a valid functor, you can always rewrite fmap a . fmap b as fmap (a . b), turning two passes into one! If you wrote a map-like function separately for each different type, taking advantage of this pattern would be more difficult.

You are also restricted in what you can do--any code that is generic across all categories can _only_ use composition. This leaves you less room to make mistakes. If you write a function just for strings, you can accidentally add characters or always return an empty string or any number of string-specific behaviors. If you write a function against some abstraction like a functor, you cannot make any of those mistakes--there is simply no way to formulate them for all functors.

These abstractions are just a reification of a common pattern across a bunch of domains. Making the pattern explicit makes it easier to see and use and makes your code simpler.

<comment><USER6> This sounds promising, but it's far too abstract to be convincing. Can you give a example of a useful program where you need to do any of these things?

<comment><USER5> Sure, look at most Haskell libraries. A lot of them expose some of their APIs as monads, arrows, functors, and kleisli categories.

Some good examples: hakyll, Pandoc, pipes, conduits, warp, scotty, or anything by Edward Kmett ( <https://github.com/ekmett/> )

<comment><USER6> Thanks, but that's far too much to take in at once. How about just one simple example?

I did skim through them, so to be specific: hakyll, Pandoc, warp, and scotty are what I'd call practical programs, though they approximate more popular programs written in other languages, so there's the question of why someone who wasn't already a Haskell programmer should choose them. Pipes, conduits, and most of the stuff by Kmett appear to be libraries implementing other abstractions that themselves would have be justified by use in a practical program.

<comment><USER5> > Pipes, conduits, and most of the stuff by Kmett appear to be libraries > implementing other abstractions that themselves would have be justified by > use in a practical program.

I'm not sure I understand what you mean by "practical programming".

This may just be a cultural difference but one of the big ideas of Haskell is using a library whose "core calculus" is provably correct and then combining them in ways that preserve that correctness. So I would argue that the justification of the libraries is not in the practical ( if I understand your usage of the word?) usage but in the thoroughness and compositional semantics that they provide. That's where the category theory inspired patterns ( zippers, monads, arrows, etc ) become important.

<comment><USER6> Okay, let's define "practical programming" as creating programs that will be of use to someone who doesn't know the programming language that the program was written in. So in this case, this means writing Haskell programs that will be used by people who don't know Haskell.

This isn't the only reason to write a program of course. You can do it just for fun or to explore the mathematics. But I'm mostly interested in abstractions that might be useful for solving problems outside mathematics.

<comment><USER5> None of the libraries or patterns I mentioned above are designed for the purpose of exploring category theory. Though some people do do that, but they tend to live in their own world.

A lot of Haskellers just happen to exploit bits of CT it because its a great framework for writing and reasoning about code. Some people do use Haskell as a way to explore category theory but for example Tekmo isn't writing pipes as a means to explore the theory of free monads, they've been well studied since the 80s. I believe he just happens to find that they are an excellent way of chaining together bits of control flow.

At that heart of the problem he's tackling is very practical problem that's tied to data flow and resource management. For example, I've written a library against pipes which handles resource management of ZeroMQ sockets for a large distributed system and find that my code under pipes is much more manageable.

<comment><USER6> Okay, sure, resource management is what I'd call a practical problem, and pipelines are familiar from Unix (and Go). It looks like this pipe library has a nice practical example: enforcing the category laws means that it handles terminating pipelines consistently:

[http://hackage.haskell.org/packages/archive/pipes/1.0.1/doc/...](http://hackage.haskell.org/packages/archive/pipes/1.0.1/doc/html/Control- Pipe.html)

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> related

[http://golem.ph.utexas.edu/category/2012/06/the_gamification...](http://golem.ph.utexas.edu/category/2012/06/the_gamification_of_higher_cat.html)

[http://golem.ph.utexas.edu/category/2012/08/categorical_unif...](http://golem.ph.utexas.edu/category/2012/08/categorical_unification.html)

<http://math.ucr.edu/home/baez/rosetta.pdf></comment>


Science: your body odor reveals when you are sick - <AUTHOR> http://pjmaurel.tumblr.com/post/138042687935/science-your-body-odor-reveals-when-you-are-sick <comment><USER2> I try to make a practice out of simply refusing to read any single study that says something exciting.

If it's true, people will replicate the study, and we can read the meta-study on their replications and be appropriately amazed later.

But chances are, publication bias has caused something statistically insignificant to be claimed as true, and people will lap it up because that's the sorry state of science journalism.

The fact that in this study the sweat appeared to be of the same amount in both cases is another sign that the experiment was likely just flawed.</comment>


No IE onboard Windows 7 in Europe - <AUTHOR> http://news.bbc.co.uk/2/hi/technology/8096701.stm

<comment><USER2> Yeah as someone living in Europe, I'm not really happy about that. Internet Explorer is an excellent tool to download Chrome or Firefox. It's like a bootstrapping browser. Very useful.

<comment><USER3> I'm curious if Europe will enforce the same policies on Apple - eg, ensuring that safari does not come pre-installed on snow leopard.

If not, blatant double standards given Apple's market share is increasing.

EDIT - another thought, since access to internet via phones is becoming more widespread, will the EU start imposing the same policies on mobile phone manufacturers...

EDIT #2 - Personally I think if the EU wants to force competition in the browser market, they should ensure that users get a choice of browsers with the pre-installed OS, otherwise with the browser not being included is potentially stumping the growth of internet availability to people who otherwise wouldn't have it.

Just to be clear, I'm not anti-Apple, I just dislike silly rulings that in themselves are anti-competitive, given they only apply to one company.

<comment><USER4> Anti monopoly rules are usually unfair sounding. Why shouldn't a larger company use its position and money to dominate a new market? The thing is, monopolies can be so damaging to society that treating them in a manner that seems instinctively fair is often bad for us. Like it or not, they _are_ different, and they must be treated differently. It's important to remember that corporations are not people, so if they're treated unfairly the only reason it's a big deal is if society as a whole loses out as a result.

That said, I think the browser market is competitive enough nowadays that MS including a browser really is not that big a deal - OSes include browsers in the same way that cars come with steering wheels. I think this is more about punishing MS for past transgressions than anything else.

</comment></comment></comment><comment><USER5> <http://news.ycombinator.com/item?id=654414>

The European Comission: "If the Commission were to find that Microsoft had committed an abuse, the Commission has suggested that consumers should be offered a choice of browser, not that Windows should be supplied without a browser at all."

That's quite a different situation than what MS imply with their response.

<comment><USER6> The article states that Microsoft will make it easy to find and download browsers, which is essentially the same thing as having a choice of browsers.

The EC is probably referring to Windows XP N, which didn't include a media player at all, and sort of left users in the dark as to how to play media files using it. I'm sure MS got some criticism from that from the EU and will not repeat the mistake.

</comment></comment><comment><USER7> _... by bundling its browser software in with its operating system._

I still don't understand why this is wrong. It's their os, let them do whatever they want with it. There are other operating systems around. They don't need to use Windows.

<comment><USER8> If IE is bundled with the OS, their de facto monopoly in the OS market reduces innovation, competition and customer choice in the _browser_ market.

<comment><USER9> Does the bundling of Command Prompt reduce innovation and competition in the command line client market? Does Windows Explorer reduce innovation and competition in the file browser market?

<comment><USER8> That could be argued. It's a shades of grey thing. Some things stand out as bundled more than others - especially those that are obviously not part of an OS, for which a market exists etc.

<comment><USER9> Perhaps in the 90s a Web browser would have not obviously been part of an OS, but nowadays it's one of the simplest, yet most used, applications going and it makes sense to bundle one with any desktop-based OS.

Come to think of it, I can't think of any current desktop-based OS that _doesn't_ come with a Web browser installed by default..

The whole anti-competition lawsuit over this stuff was based on early 2000s assumptions. It's time to forget all about it now and let MS get on with it.

</comment></comment></comment></comment></comment><comment><USER10> How would people install Firefox/Chrome/Opera/Safari whatever then?

It's a shame that they choose to provide no browsers instead of providing a few

<comment><USER9> I'm usually anti-Microsoft, but it's ridiculous that Internet Explorer was a problem to the EU to start with.

Why not unbundle Explorer? Why not unbundle the control panel? Isn't it unfair that Notepad is included? What about Calculator.. surely that's hurting Mathematica? (Before you think I'm going over the top here, IE compared to Firefox is like Calculator vs Mathematica IMHO..) These EU commissions are run by dim-witted politicos who fail to understand any of the realities of technology or business.

<comment><USER11> It's not ridiculous at all. Microsoft were using their monopoly in desktop operating systems to gain browser market share. This doesn't benefit anyone apart from Microsoft.

<comment><USER12> I don't think people would have minded as much if IE weren't the worst browser ever created. Add to that the fact that Microsoft abandoned IE for years, leaving everyone high and dry.

<comment><USER9> Actually, IE being a hideous browser should make it better as it makes it easier for competition to succeed and gives people more reason to investigate and accept alternatives.

Consider if all manufactured cars came with a really crappy "default" radio with horrible sound. The majority of drivers would choose to upgrade the radio to a better one, thus making the marketplace for car radios a healthier, more competitive one. If the cars came with amazing radios built in, hardly anyone would switch.

</comment></comment></comment></comment></comment><comment><USER13> Why does it matter which browser has bigger market share as far as antitrust laws go? Nearly all browsers are free and have been for donkeys years, so does it really matter -business wise- if one company has the majority?

<comment><USER13> Does this mean that AOL will finally have an excuse for sending out all those annoying CD's again?

</comment></comment><comment><USER14> Netscape's ftp site was always handy, can't seem to find any ftp sites for any browsers now.

<comment><USER15> How do you get an ftp client without a browser?

<comment><USER16> Windows Explorer has an FTP client built-in.

<comment><USER9> Uh-oh, I smell another lawsuit. Think of the damage that's doing to all those FTP client vendors!</comment></comment></comment></comment>


Searching and Replacing Jobs' Flash Statement - <AUTHOR> http://hooptyrides.blogspot.com/2010/04/searching-and-replacing-jobs-flash.html# <comment><USER2> You would have had me if you didn't do this:

> _Replace Flash with closed, as a catchall for Apple's myriad of closed > technology_

That turns an otherwise well-put argument into an unfair comparison. Swap "Adobe Flash" with "Apple iPhone" and your point works, with less bias.

<comment><USER3> Plus, it fixes a few of the odd tautologies in there.</comment></comment>


Ask HN: Has Hacker News been hacked/cracked? - <AUTHOR>

It seems like you can change the about field under PG's account http://news.ycombinator.org/user?id=pg using this appjet app http://notabank.appjet.net/ <comment><USER2> Yes. I made an unbelievably stupid mistake in the code that generates forms with labelled fields. It was basically functional programming taken a little too far: I generated the same form whether the fields were editable or not, and then later if there were no editable fields I just omitted the submit button. So anyone looking at the source of one of these pages could find a fnid that would work to modify the object displayed on it. (There's still a fnid, but it no longer does anything.)

<comment><USER2> Actually on closer examination the bug was subtler than that. I did check whether the incoming request was allowed to modify each field it was trying to modify. But the code that decided whether a field was modifiable was using a four variable list to destructure on a five value description of the field. This caused the variable determining whether a field was modifiable to be bound instead to the value saying whether or not it should be displayed. Which meant every field that was displayed was modifiable.

<comment><USER3> That's actually pretty interesting to me. I didn't notice that when I was looking at the Hacker News code. Perhaps Arc shouldn't allow a list to be destructured except with a list of the same length? Allowing differing lengths seems likely to cause subtle bugs like this.

<comment><USER2> I generally tried to err on the side of flexibility, and it seemed more flexible not to require the lengths to match. I could imagine cases where you might want to use patterns both longer and shorter than the lists they were matched against. On the other hand, I'm not sure I've ever ended up taking advantage of this.

<comment><USER3> Maybe you could use something along the lines of &rest to optionally support a pattern shorter than the matched list?

<comment><USER2> You could do that with a dot now. The trouble is, you end up creating a junk variable:

        arc> (let (x . y) '(a b c) y)
        (b c)

<comment><USER3> Making users create junk variables seems to me like better behavior than the current implicit behavior. It's your decision, though, of course.

Anyway, is it safe to assume that you're not going to try to get me arrested or anything?

<comment><USER2> I feel like unnecessary variables are extra bad; in cognitive terms they feel like they add more than one token to the length of a program.

Do you mean your username is actually descriptive? Since you are obviously a Lisp hacker, I'd be happy to have a truce. Can you send me an email?

<comment><USER3> Thank you. I've sent you an email.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER4> Whoever did this, please post a Tell HN or otherwise an article on how you did it. I'm sure others are curious (and would make a good starting point for patching Arc)

From the source, it looks like there was a vulnerability in which the _fnid_ (I'm guessing a string that authenticates a user to edit an item?) was searched for on PG's profile page (using the regex /<input type=hidden name="fnid" value="([^"]+)">/. Then a POST request was made on the standard profile saving resource news.ycombinator.com/x, with the _fnid_ which authenticated the user's permission to edit the page, along with the about text, as parameters.

 _Edit: PG says the fnid just points to a closure on the system. See above. Which means... all you needed was a randomly generated fnid, and that's all that you needed to edit anyone's page. Apparently?_

Clever, or just poor authentication design. But that's only one half of the exploit. How were the points done? I'm going to rule out millions of accounts created.

<comment><USER3> The points were editable through similar means. Pages like <http://news.ycombinator.com/edit?id=519433> contained a form that allowed all of a story's details -- including title, url, and score -- to be edited. Just like with profiles, the story editing forms didn't have fields for non-admins, but just like with profiles, it was possible to submit a request with the fnid from the form regardless of admin status.

</comment></comment><comment><USER5> Yes, it was hacked. Here's the original post from a lesser hack earlier today with pg's response containing the "not a bank" quote:

<http://news.ycombinator.org/item?id=518752>

<comment><USER6> Okay, trust doesn't work as well as correct security measures. That's sad, but true.

</comment></comment><comment><USER7> <http://source.notabank.appjet.net/>

<comment><USER8> Just in case the original app gets taken down, I copied the source to pastebin:

<http://pastebin.com/f1a67398f>

</comment></comment><comment><USER9> There are a lot of bogus links showing up on the front page with obviously falsified vote counts (numbers starting with the digits "1337"). I'd say, yeah, it's been seriously compromised.

<comment><USER10> Re: 1337 <http://en.wikipedia.org/wiki/Leet>

<comment><USER11> Yes, that's the joke.

</comment></comment></comment><comment><USER12> I caught it here:

<http://skitch.com/pelle/beksu/hacker-news-hacked>

</comment><comment><USER13> Yes, it has been. Looks like the site had some major vulnerabilities.

I emailed PG, if he didn't know already, and slowly some of the things are being fixed back. PG's account is still vulnerable as of this posting. EDIT: No it's not.

</comment><comment><USER14> Certainly seems like it.

</comment><comment><USER15> Dodged a bullet there I'd say. At least your "hacker" seems to be a reasonable guy. Seems like he could have done a _lot_ more damage. I hope your backups are up to date and verified recoverable; a more malicious intruder might not be so kind.

While I appreciate and admire the sentiment that this is a "community of trust", security still must be taken seriously. There are plenty of guys out there with the ability to pull such tricks; they may not care about trust, and the website is accessible to anyone, good or bad.

</comment><comment><USER16> Of course it was hacked... By the creator of it :). That's why other hackers find it interesting.

For the question whether it's cracked... I dunno but nothing is perfect.

<comment><USER16> Looks like the hackers don't understand that Hacker News is a nice hack altogether.

<comment><USER17> Nice and insucre one.

Anyways, the fact it is hacked drove some nice traffic to HN from reddit :)</comment></comment></comment>


Multi Screen Hell in Android - <AUTHOR> http://www.cavdar.net/2013/11/05/multi-screen-hell-in-android/

<comment><USER2> One small quibble: ONLY declare which screen sizes your app supports if it really does not support some screen sizes. You should usually consider that a bug. You need not declare non-support of an obscure size or configuration if not having size-specific resources is the only issue. Scaled images are not ideal, but it's better for the app to work and look a little fuzzy than to try to specify exactly where it does not look perfect.</comment>


Chess - A Quandary for the Game in a High-Tech Era - <AUTHOR> http://www.nytimes.com/2013/01/13/crosswords/chess/chess-borislav-ivanovs-performance-is-scrutinized.html

<comment><USER2> It's fascinating to read the report by the professor mentioned in the article: [http://www.cse.buffalo.edu/~regan/chess/fidelity/ACPcover- an...](http://www.cse.buffalo.edu/~regan/chess/fidelity/ACPcover-and- report.pdf)

The relevant conclusion: "My model projects that for a 2300 player to achieve the high computer correspondence shown in the nine tested games, the odds against are almost a million-to-one. The control data and bases for comparison, which are wholly factual, show several respects in which the performance is exceptional even for a 2700-player, and virtually unprecedented for an untitled player."

A NYT article from last year on his methodology: [http://www.nytimes.com/2012/03/20/science/a-computer- program...](http://www.nytimes.com/2012/03/20/science/a-computer-program-to- detect-possible-cheating-in-chess.html?ref=science)

and more details on his site: <http://www.cse.buffalo.edu/~regan/chess/fidelity/>

<comment><USER3> It's quite unlikely, but... 1 in a million doesn't sound so amazing when you recall how many thousands and maybe millions of games get played over the last decade, and how people are going to pay the most attention to anyone who wins a proverbial chess lottery.

</comment></comment><comment><USER4> Clearly it's cheating to use computers in pure human chess tournaments or to use humans in pure computer chess tournaments. However, I'd find the idea of a hybrid tournament (human+computer vs human+computer) to be fascinating.

Imagine the challenge of excelling in searching the space, evaluating positions, selecting strategy, etc when it'd be legal to creatively combine human and computer capabilities in the game.

<comment><USER5> This has been the norm since computers could play board games. I read Blondie24[0] about a year back about 2 guys who build a complicated neural network for a checkers program which teaches itself to play by playing against itself and is eventually entered into 'Human' tournaments. It's a really fast read even if you have no experience in AI.

There is also Deep Blue[1], which is famous for once defeating Garry Kasparov in a high profile match.

[0] [http://www.amazon.com/Blondie24-Playing-Kaufmann- Artificial-...](http://www.amazon.com/Blondie24-Playing-Kaufmann-Artificial- Intelligence/dp/1558607838) [1] <http://en.wikipedia.org/wiki/Deep_Blue_%28chess_computer%29>

<comment><USER6> Deep Blue was one computer against one human.

The comment you're replying to is suggesting teams with two members: Computer A and Human A VS Computer B and Human B.

Thanks for the link to the Blondie24 book!

</comment></comment></comment><comment><USER7> This is an excellent analysis of Ivanov's suspect games by FIDE master Valeri Lilov. His conclusion–there is little doubt Ivanov was cheating. He also details several low cost plausible methods Ivanov could have used.

[http://www.youtube.com/watch?feature=player_embedded&v=J...](http://www.youtube.com/watch?feature=player_embedded&v=Jr0J8SPENjM)

<comment><USER8> Lilov's analysis is good but he is a little quick to jump to conclusions. I wasn't convinced until I saw Regan's data.

</comment></comment><comment><USER9> I'm a bit confused. It is strongly implied that Ivanov cheated by using computer assistance. But I don't understand how he would have done this. He is playing at the same chessboard as another player, correct? How could he have received assistance unnoticed?

<comment><USER10> How I did it: Moves are transmitted by squeezing the right toes (1 to 8 squeezes for rank, long pause, another 1-8 for file). Computer comes up with countermove, which is transmitted by vibration pulses in the same encoding scheme.

<comment><USER11> "How I did it"... that is begging for a story. Got one?

<comment><USER10> Not really; I built a thing, I tried fooling some people with it, but the trouble with a system like that is it makes you extremely inflexible. If you make a mistake and get out of sync, you're screwed. And it's really obvious that _something_ is up, because it takes you suspiciously long to make a move even when it's blindingly obvious what to do - because you still have to do the round trip in order to stay in sync.

</comment></comment></comment></comment><comment><USER12> "One of Ivanov's losses was in a long game in a closed position (the kind where computers perform poorly)..."

The top computer engines have been strong in closed positions for many years now. If this person did cheat, he most likely was selectively using the engine in this game, or not at all.

There's a reason we don't see computer vs. human matches these days. The computers are too good. You can beat today's elite grandmasters with a free engine running on your laptop.

</comment><comment><USER13> If it is that much of a concern, perhaps we should focus more on games that remain difficult for computers. Go, Arimaa, and Texas Hold'em (but perhaps not for much longer) all come to mind.

<comment><USER14> Texas Hold'em, contrarily to chess or Go, is a game of imperfect information.

So psychology comes into play and that is basically impossible to "solve" unless we have real AI (not anytime soon and the implication would be way broader than "beating Hold'em").

Fixed-limit Hold'em is easier to play because the decision tree is trivial. It's still an imperfect information game but the best bots can already beat some of the best professional players.

But No-Limit Hold'em is another thing entirely: the decision tree is so huge that it's impossible to bruteforce. Impossible as in: unless a major scientific discovery we'll never ever be able to bruteforce it (there's not enough entropy in the know universe to do so).

So we're stuck and waiting for amazing discoveries in the AI field (and psychology field).

<comment><USER6> I'd be interested to see the results of two bots playing against humans.

Bot One just plays the odds.

Bot Two just plays the odd, but has a small chance of bluffing - it has a poor hand but plays as if it has a great hand.

I'd be interested to see the results after several thousand hands. And if people could tell the bots apart from people.

This is, obviously, nothing like AI. But sometimes trivial tricks are powerful. Look at the early discussions about ELIZA for an example.

I don't play poker, but my friend tells me that it's fun to play in tournaments because it's not just odds. You get people who have terrible poker faces, who are easy to read. That same friend also plays (and makes a little bit of money) from online poker. I have no idea how the psychology works there.

<comment><USER15> So here is poker theory in a very, very brief nutshell.

One way to play is to (attempt to) play game theoretically, or more commonly (and less rigerously defined) "balanced". Your entire actions are defined in such that your opponent, even if they knew your strategy, could not exploit your decisions. Bets have the "perfect mix" (defined by pot size, but also other factors) of value hands to bluffs, so that your opponent is indifferent to calling down/rebluffing. This is what you called "not playing the odds" but, contrary to popular belief, involves bluffing (lots of it!), lots of aggression, and in general may look kinda crazy.

The second way is exploitative poker. This is the more common "not playing the odds" type poker, where you play in a way that maximally extracts value, because you think your opponent is not bluffing enough/betting too big with his range, folding too much, etc. The "too much" is basically "deviating from the correct GTO play". However, to exploit this, you yourself now need to deviate from optimal play, thus opening yourself to exploitation.

There is still a debate between which is "better." Obviously exploitive will, at least in theory, lead to higher winrates, but runs the risk of yourself being exploited. If your "read" is wrong, you could be losing. By playing GTO, you assure you never lose (except your share of the rake), and money naturally flows to you through opponents just basically exploiting themselves.

</comment></comment></comment></comment><comment><USER16> You can replay Ivanov's games (halfway down the page):

<http://www.chessbase.com/newsdetail.asp?newsid=8751>

</comment><comment><USER17> As a chess outsider, this sentence gave me pause until I noticed the capitalization:

"Though he was Black, Ivanov achieved a slight edge after 11 moves"

</comment><comment><USER14> It's sad that an article targeted at people playing a smart game forces you to use a Java applet to replay moves : (

Hint: smart people disable Java applets nowadays ; )

<comment><USER18> Its easier to embed dozens of java applets that are freely available then to write a custom solution just for one small moderately interesting article.

<comment><USER19> True, but nytimes has proven they have some HTML5 chops ([http://www.nytimes.com/projects/2012/snow- fall/#/?part=tunne...](http://www.nytimes.com/projects/2012/snow- fall/#/?part=tunnel-creek)), and JavaScript-based chess viewers are readily available (<http://code.google.com/p/pgn4web/>). Considering the recent Java scare, I think it's reasonable to be mildly surprised by them using applets. I'm starting to be surprised even when I see Flash these days, much less applets.</comment></comment></comment>


The Open Mathematics of Crystallization [pdf] - <AUTHOR> http://www.ams.org/publications/journals/notices/201706/rnoti-p551.pdf <comment><USER2> Unrelated to the content of the article, but I took two courses with the author. He had a style of teaching that often broadly related the material to seemingly unrelated topics in ways that made lectures fun and easy to remember. I think that style comes across in his writing here.

</comment><comment><AUTHOR> From a mathoverflow thread "Open problems in mathematical physics":

[https://mathoverflow.net/questions/275011/open-problems- in-m...](https://mathoverflow.net/questions/275011/open-problems-in- mathematical-physics)</comment>


Ask News.YC:  Where are good books/sites for semantic analysis, economics, and genetics? - <AUTHOR>

<comment><USER2> <http://www.biologynews.net/> is good for biology news, surprisingly enough, but there's not a whole lot there about genetics</comment>


Bitcoin mining is NP-hard - <AUTHOR> https://freedom-to-tinker.com/blog/jbonneau/bitcoin-mining-is-np-hard/ <comment><USER2> Misleading title.

The article talks about how choosing the optimal set of transactions to include in a block is NP-Hard, and the typical implementation uses a simple greedy approximation.

But I _thought_ I was going to read about someone doing a security proof showing that a fast way to generate blocks and extend the chain would give us a polynomial algorithm for 3-SAT. Which would have been awesome.

<comment><USER3> Finding the hash for a block is NP-complete, I believe [edited to add: I should note that this is more speculative than my wording may have implied; the rest holds]. We don't have, algorithmically, "a fast way to generate blocks" \- we do it through brute force, and size the problem such that collectively someone can solve with reasonable frequency.

<comment><USER4> Actually, no. Proving that finding the has is NP-complete would require a proof that any boolean formula (the SAT problem) can be translated into a Bitcoin block such that a nonce yielding a small hash (which is what Bitcoin asks for) would somehow encode a solution to the SAT problem.

Nothing like this has been proved (neither directly nor indirectly), and it seems very unlikely to happen in the foreseeable future.

In fact, _none_ of the commonly used cryptographic primitives have NP-hardness proofs.

<comment><USER3> "X is NP-Complete" is a different statement than "X is proved to be NP- Complete". I said "I believe" the former. Bitcoin mining is clearly in NP - you can check blocks in polynomial time. The only real question is whether it is also NP-Hard. Contrary to your implication, there is nothing about the shape of the problem that implies other NP problems couldn't be reduced to it - indeed, we can express any NP problem as a pile of bits that, fed to an appropriate algorithm, generates a single 1 bit; that is the nature of a decision problem.

I fully recognize that the NP-Hard-ness of SHA256 is speculative, and I run a not-insignificant risk of being wrong. The strongest case for my correctness, so far, is that no one has found a polynomial attack against any of the SHA functions - this is _far_ from conclusive, of course.

</comment></comment></comment></comment><comment><USER5> Optimal blocks aren't really a priority for miners because the fact that you win the race is worth, ballpark, 250X (the coinbase transaction's segniorage award of 25 BTC) what the transaction fees for the transactions included in the block is. Optimizing for a few percent extra transaction fees doesn't meaningfully change the economics, and if it slows you appreciably, could potentially cost you that segniorage if someone else beats you to the punch.

Accoringly most miners appear to just use really cheap heuristics.

<comment><USER6> Aren't block awards supposed to reach 0 after a while, so that tx fees become the only incentive for mining? In those cases optimizing may be well worth the effort?

<comment><USER7> Hopefully by that point the block size will be bigger. If the block size is bigger, than the problem is just "include every transaction". But yes, the bitcoin rewards will continue halving until sometime in 2110-2140, where they hit zero. At some point, transaction fees will be more important than the block reward. Therefore, to be efficient miners would need to solve the knapsack problem if they wanted to create a maximum value block. However, we are lucky here, the knapsack problems is one of the few NP-complete problems that isn't a total mess to deal with, and can be solved in O(nW) time, where W is the greatest common divisor for the weights and n is the length of the input.

</comment></comment></comment><comment><USER8> "but this formula has other benefits as it enforces minimum transaction fee amounts"

Why is this good/important?

<comment><USER9> It would reduce friction in the bitcoin economy. Consider, in a more conventional economy, a significant tax levied by the government. This tax either reduces the profits for the seller or increases the cost for the buyer or both.

1\. The seller can optionally soak the cost of the transaction fee. But this means less profit for them while the buyer sees no changes (until later when the seller finds this is unsustainable as fees increase).

2\. The seller can raise prices by the fee amount (how sales tax works in the US, effectively). The buyer is then paying more and more over time as fee amounts increase.

3\. Some combination, seller determines the amount they can afford to lose while keeping prices low enough to not lose too many customers.

In the end, all 3 options will create a drag on the economy. Either by reduced buying power or reduced profits resulting in reduced future growth.

We also see this with credit/debit card transaction processing. Sellers, per their agreements with CC companies, aren't supposed to have different prices between CC and cash transactions. This means they lose part of their profits on CC transactions, with some companies (AMEX) costing them more than others.

<comment><USER10> Those kind of credit card agreements were outlawed under an Obama consumer- protection law. You can now offer a discount for people who pay in cash, though I believe the agreements can still require it be advertised as a discount for cash, not as a fee for credit card usage, even if they are effectively the same.

</comment></comment></comment><comment><USER11> Unfortunately, the problems he reduces from are not so hard in practice. Knapsack has what's called an FPTAS, which means you can get as good an approximation as you want, and the runtime only scales polynomially with your desired accuracy. And independent set has nice approximation algorithms for bounded degree graphs and other restricted graph classes (though there is no general constant-factor approximation unless P = NP).

So it's funny that the author quotes knapsack as the possible true difficulty...</comment>