Dat: A P2P hypermedia protocol with public-key-addressed file archives - <AUTHOR> https://www.datprotocol.com/ <comment><USER2> Kind of stagnant, but I liked the IDEA of what was being done with the Beaker Browser using the DAT protocol to run a self-hosted P2P web. I just wish someone would take the project to completion/usability. It would really disrupt things. [https://beakerbrowser.com](https://beakerbrowser.com)

<comment><USER3> We decided to take a year to go heads down and rework a lot of stuff, so we did get publicly stagnant but we're fulltime on the next version and should release a public beta soon. I tweet a lot about our progress if you can sort through my sillier tweets (@pfrazee).

Past year has had a lot of improvements

\- Protocol moved to a hole-punching DHT for peer discovery (hyperswarm).

\- Protocol now scales # of writes and # of files much better. We were able to put a Wikipedia export, which is millions of files in 2 flat dirs, into a single "drive" and get good read performance. This performance bump came from an indexing structure that's built into every log entry (hypertrie).

\- Protocol now supports "mounting" which is a way to basically symlink drives onto each other. Good composition tool, esp useful for mounting deps in a /vendor directory.

\- Browser now has a builtin editor that splits the screen for live editing. Feels similar to hackmd.

\- Browser added a bash-like terminal for working with the protocol's filespace. It's glued to the current page so you can drive around the web using `cd`.

\- Browser added a basic identity system. Every user has a profile site that's created automatically and maintains an address book of other users.

\- We built out application tooling a fair amount. It's fairly easy to build multi-user apps now, where previously it was a bit of rocket surgery.

Some of the year was spent prototyping ideas and throwing them away as well. A bit inefficient, but helped us learn.

<comment><USER4> I've been reading about dat and doing some basic projects for about a year in my spare time, stagnating a bit lately as am finishing my degree. As I remember there was some talk about Python and Rust implementations at the time, is that still under development as well? I am not as enthusiastic with Js but really like the protocol and the idea behind it.... Thanks

<comment><USER3> Yeah I do know there's a Rust impl that people are working on

</comment></comment></comment></comment><comment><USER5> "How Dat Works" linked from the page is still a great visualization of all the bits that make up the protocol:

\- [https://datprotocol.github.io/how-dat- works/](https://datprotocol.github.io/how-dat-works/)

\- [https://news.ycombinator.com/item?id=20363813](https://news.ycombinator.com/item?id=20363813)

<comment><USER6> Also [https://news.ycombinator.com/item?id=20811936](https://news.ycombinator.com/item?id=20811936), a related project from 2019.

</comment></comment><comment><USER7> Conceptually, this sounds very similar to Freenet; a DAT URL seems analogous to a Freenet SSK.

What I don't see is anything analogous to a USK -- there's no obvious way for an author to distribute an update to content they have published. It's also unclear how much (if any!) privacy this protocol provides to content publishers or requestors -- the use of discovery keys only provides protection for users requesting content which an eavesdropper has no knowledge of.

<comment><USER8> There is no equivalent to an USK because dat doesn't give direct access to old revisions. What you get is always the latest.

The old content is still there though, and you can access it, just not in an "easy" manner: [https://docs.dat.foundation/docs/faq#does-dat-store- version-...](https://docs.dat.foundation/docs/faq#does-dat-store-version- history).

The dat url references a given private key, and that's about it in terms of privacy. Transfer is done between two endpoints on the normal internet, so neither peer is "hidden".

It's a shame but the dat content is spread in many places and it's hard to get access to all the documentations. The most impressive and interesting part of the project right now is probably Beaker, you should have a look: [https://beakerbrowser.com/](https://beakerbrowser.com/)

<comment><USER7> > There is no equivalent to an USK because dat doesn't give direct access to > old revisions. What you get is always the latest.

How do you guarantee that you're getting the _latest_ content, though? If a peer has a cached copy of an older revision, wouldn't you end up with that instead, since there's no way to distinguish between them?

> The dat url references a given private key, and that's about it in terms of > privacy. Transfer is done between two endpoints on the normal internet, so > neither peer is "hidden".

What concerns me here is that, from what I'm reading, it seems like any client on the local network could eavesdrop on mDNS requests to determine what content other clients are viewing. Worse, a client could announce itself with the discovery key for a well-known piece of content to be notified when any other client, anywhere, requests that content.

 _This is a worse privacy model than unencrypted HTTP._ Are you aware of any plans to mitigate this?

<comment><USER8> > How do you guarantee that you're getting the latest content, though?

Only the original creator can update the content. You can never know you're at the latest version until you've connected to them and they've told you "this is the last I have"

> What concerns me here is that, from what I'm reading, it seems like any > client on the local network could eavesdrop on mDNS requests to determine > what content other clients are viewing. Worse, a client could announce > itself with the discovery key for a well-known piece of content to be > notified when any other client, anywhere, requests that content.

Disclaimer: I'm not an expert on the project, only following it because it's cool.

As far as I know, the only obfuscation is that keys are hashed so that you can't infer what content is being exchanged just by listening to the network. However when you want to watch a specific key, you can get on the swarm and see who's there.

Note that dat doesn't attempt to solve the same problems as Freenet does. They have different goals, and as such can't be compared on something that one explicitly focuses.

<comment><USER3> You have the details right. I think it's fair to say that the protocol is very leaky with its metadata right now. In a local network, it would be wise to only exchange announcements with trusted devices. In the global network, it would be wise to introduce some kind of proxy (distributed or not).

</comment></comment></comment></comment></comment><comment><USER9> I have failed time and again to grasp how exactly Dat works. IPFS is easy for me to grok; your content is hash-addressable, thus immutable. Can someone explain in a sentence or two how Dat handles content?

<comment><USER8> Bittorrent is immutable, Dat is the mutable version of Bittorrent.

You create an archive, that is identified with a cryptographic pubkey. You add files in it, dat stores some metadata in it. You give the archive's id to a friend, who starts retrieving the metadata, and can then see there are files; he can download files as he pleases. Syncing can also be realtime, so he gets the new content as soon as you put it.

Only you, the holder of the cryptographic privkey, can add content to the archive. Crypto is used to sign all content, so there's no doubt it was legitimately written by you. Since the id doesn't move, it is possible for multiple peers to inter-connect and exchange data as needed in a swarm fashion, even if you're offline

<comment><USER9> That's a great explanation, thank you. How does this handle versioning? If I add one file to a vast dataset, can people download just that file if they already have the previous version? IPFS hashes at the chunk level, so even if you append something to a large file, you can download just the new chunk and be up to date.

<comment><USER8> Yes, in fact dat was initially created for this use case: data analysts want to exchange their data, and that data evolves in time so it needs to be transported "efficiently", as in, you don't need to redownload a full .zip just for a single file change. The only metadata you'll receive will concern the new file and the old content is still valid. You can even seek inside any file if you don't want to download the whole file.

Changes _inside_ a file, though, are not handled. Today if a file is modified dat will consider any bytes of the old one to be garbage and will not reuse it.

dat is a sexy frontend on top of hyperdrive([https://github.com/mafintosh/hyperdrive](https://github.com/mafintosh/hyperdrive)), I personally think it's easier to see what dat can do by looking at what hyperdrive does

<comment><USER9> Thanks, that's very informative, and the intro in the Hyperdrive README clarifies the goals very well. I have a much better idea now, thanks again.

</comment></comment></comment></comment></comment><comment><USER10> When I last looked at dat some time ago I seem to recall that there was no way to revoke a dat or set a TTL-type field.

 _Can dat forget?_

E.g. If I created a dat, that was it and the data in the dat was then potentially out there forever and ever and ever in the distributed network. There was no way to tell a client "this dat is only good for 90 seconds/minutes/hours/days/years/decades/etc - after that please drop/delete".

I know there is this desire in certain circles for Blockchain-stylee "everything is stored for ever and we can cryptographically prove every single byte all the way back to the dinosaurs!" sentiment, but I am not sure that really jives with a distributed data/Web publishing protocol (at least in my mind) - I want to be able to reliably "delete" something. If I know that _anything_ I ever publish will be irrevocably around forever, it has a chilling effect on what I choose to do and publish with dat (plus maybe legal challenges? IANAL but e.g. GDPR? Right to be forgotten?)

E.g would we actually gain anything if hypothetically Google supported dat and so then every single Google search result page ever generated was stored forever in a dat? Would future users benefit from storing decades of archived versions of Google search result pages for "Facebook" (because people search for "Facebook" then click the first result instead of just typing "facebook.com") or "weather" which then need to be endlessly duplicated around the network for the rest of eternity? Seems unlikely to be of any benefit to me - surely better to mark some of it as ephemeral and let the data naturally timeout and die?

Does anyone know if that has changed with dat and data can now die? Or have I just misunderstood?

<comment><USER11> Data in a dat, just like data in a torrent, only lives as long as there are peers interested in seeding that particular dat; it's not a blockchain-style "everybody has to replicate everything".

You can't unilaterally withdraw content somebody else is also seeding; the best you can do to my knowledge is publish a new version that replaces all the dat's content with a "please delete your history of this archive" message, but as with any distributed system you have no real way of knowing if that was respected.

</comment></comment><comment><USER12> Here are all the projects that are using components from the dat team over the years: [https://dat.foundation/explore/projects/](https://dat.foundation/explore/projects/)

and [https://cobox.cloud/](https://cobox.cloud/)

</comment><comment><USER13> Wouldn't a protocol like this be a killer feature in the next package manager to solve problems npm had? Then you've just got a discoverability issue without the whole infrastructure headache on top to maintain.

</comment><comment><USER14> DAT is not new, I've been building stuff on top of it. Good to see more people are awared of it.

</comment><comment><USER15> Nice. Aral Balkan of small-tech.org is having plans with Dat for Tincan and Site.js:

[https://small-tech.org/research-and-development/](https://small- tech.org/research-and-development/)

</comment><comment><USER16> i swung and missed :-( [https://github.com/datproject/dat/issues/1008](https://github.com/datproject/dat/issues/1008)

</comment><comment><USER17> Can Dat really be described as a "new" protocol 3 years in?

<comment><USER18> Yes, as it's still under heavy development and has not yet seen heavy usage.

This mentality that things either flip instantly to mass adoption or are "old" needs to die. All the easy stuff that can be done in 6-12 months has already been done. Anything worth doing today is going to take a minimum of 1-2 years of R&D unless it's nothing more than a packaging and polish/branding of something already in existence.

<comment><USER17> > This mentality that things either flip instantly to mass adoption or are > "old" needs to die.

That's honestly not where I'm coming from, though. I think Dat is a cool project and I've been aware of it from, likely, close to the beginning. The scale between "new" and "old" is not binary, nor is it necessarily related to "mass adoption". There are people who still treat React as though it's "the new hotness", despite the fact that it's been out for 6 years and is widely adopted.

I'll grant that different people will have different thresholds for what they see as "new". I just personally think Dat has gone beyond "new" and is in a phase of maturation. It's even got a browser with deep support!</comment></comment></comment>


Back doors, black boxes and #IPAct technical capability regulations - <AUTHOR> http://www.cyberleagle.com/2017/05/back-doors-black-boxes-and-ipact.html <comment><AUTHOR> Some interesting passages:-

> The new IPAct communications data requirements, however, include a paragraph > under which a technical capability notice could require a provider to > install a government-provided 'black box': "10\. To install and maintain any > apparatus provided to the operator by or on behalf of the Secretary of State > for the purpose of enabling the operator to obtain or disclose > communications data, including by providing and maintaining any apparatus, > systems or other facilities or services necessary to install and maintain > any apparatus so provided".

> A technical capability notice would also be able to require the operator to > "consider" the obligations and requirements imposed by any technical > capability notice when designing or developing new telecommunications > services or telecommunications systems.

> The most controversial aspect of technical capability notices throughout the > passage of the Bill was whether the obligation to remove encryption could be > used to prevent use of end to end encryption. The proposed IP Act > regulations say (for interception):"8\. To provide and maintain the > capability to disclose, where practicable, the content of communications or > secondary data in an intelligible form and to remove electronic protection > applied by or on behalf of the telecommunications operator to the > communications or data, or to permit the person to whom the warrant is > addressed to remove such electronic protection".

> For a technical capability notice the central concept is technical > feasibility. Clearly it is not technically feasible for an operator who > provides its users with true end-to-end encryption facilities to remove the > encryption, since it has no decryption key.</comment>


The complexity of modern pop music - <AUTHOR> https://qz.com/1035049/the-mozart-like-complexity-of-carly-rae-jepsens-biggest-hits/ <comment><USER2> There was an excellent program aired on PBS in the USA featuring a technical analysis and historical context of the songs the Beatles recorded during the "Sergeant Pepper" sessions:

[http://www.pbs.org/program/sgt-peppers-musical- revolution/](http://www.pbs.org/program/sgt-peppers-musical-revolution/)</comment>


Noam Chomsky – "On Anarchism" [video] - <AUTHOR> http://booktv.org/Watch/15167/quotOn+Anarchismquot.aspx <comment><USER2> The book in discussion:

"On Anarchism provides the reasoning behind Noam Chomsky's fearless lifelong questioning of the legitimacy of entrenched power. In these essays, Chomsky redeems one of the most maligned ideologies, anarchism, and places it at the foundation of his political thinking. Chomsky's anarchism is distinctly optimistic and egalitarian. Moreover, it is a living, evolving tradition that is situated in a historical lineage; Chomsky's anarchism emphasizes the power of collective, rather than individualist, action.

The collection includes a revealing new introduction by journalist Nathan Schneider, who documented the Occupy movement for Harper's and The Nation, and who places Chomsky's ideas in the contemporary political moment. On Anarchism will be essential reading for a new generation of activists who are at the forefront of a resurgence of interest in anarchism—and for anyone who struggles with what can be done to create a more just world."

[http://www.amazon.com/Anarchism-Noam-Chomsky- ebook/dp/B00E25...](http://www.amazon.com/Anarchism-Noam-Chomsky- ebook/dp/B00E25LZZI/ref=sr_1_1?ie=UTF8&qid=1387168819&sr=8-1&keywords=on+anarchism#)

<comment><USER3> (Confession: I'm on a lunch break, and have not sat through the hour plus documentary - so pardon me if it already addresses this point)

In democracy (imperfect as it may be) we get to elect people who choose laws and thus authorize violence only in mandated situations.

Without rules - including law - is all violence permitted?

Edit: from the book review: "Chomsky's anarchism emphasizes the power of collective, rather than individualist, action."

So people come together as collectives, which represent them? Would these collectives presumably agree on some standards? And enforce them? Isn't that democracy?

<comment><USER4> A common misconception about anarchism is that it means an absence of rules and laws. Instead, the various flavors of anarchism propose models for distributed, bottom-up authority and decision-making.

Statism : 90s Microsoft :: Anarchism : Web Standards

<comment><USER5> > Statism : 90s Microsoft :: Anarchism : Web Standards

Maybe I'm nitpicking, but Web Standards are mandated by the W3C, which I don't see as an anarchistic organization, but more as an extended oligarchy.

To participate, an organization (from what I found, individuals cannot join) needs at least 1950.00 EUR per year.[1]

[1] [http://www.w3.org/Consortium/fees](http://www.w3.org/Consortium/fees)

I'm not criticizing W3C per se, just the fact that it's being compared to an anarchist way of doing things.

<comment><USER4> Hm, I didn't know that. Nonetheless, I feel the analogy stands somewhat; decisions are at least partially made via participatory consensus rather than from one entity by fiat. There's also the WHATWG, which is much more democratic, and which has been very influential in the ongoing standards process.

Perhaps a more accurate comparison would be ICANN vs. NameCoin; however in that case, the anarchist solution is far less proven. ;)

</comment></comment></comment></comment></comment><comment><USER6> Youtube link: [http://www.youtube.com/watch?v=oB9rp_SAp2U](http://www.youtube.com/watch?v=oB9rp_SAp2U)

<comment><USER7> Thanks :)

</comment></comment><comment><USER8> Meet Noam Chomsky - Academic Gatekeeper: [http://www.youtube.com/watch?v=EEDf7OkRCxk&list=PLN6xa7kD9dZ...](http://www.youtube.com/watch?v=EEDf7OkRCxk&list=PLN6xa7kD9dZ_6_GzGjZGlKKElb6ceOq0p&feature=player_detailpage)

<comment><USER9> So someone who calls for overthrowing the US government and replacing Capitalism with Anarcho-Syndicalism is now a gatekeeper?

<comment><USER8> Feel free to argue against the points presented in the video.

Capitalism is such a messed-up term these days. To me it means free, voluntary exchanges and agreements/contracts between people, but most people have been propagandized to view it as "evil capitalist pigs exploiting and oppressing the poor".

But capitalism cannot be replaced, it's what people do when they're allowed to, after all. It's about people pursuing their personal gain through commerce and investments etc, and you know self-interest is not going anywhere, regardless of what societal system happens to be in place.

<comment><USER4> The countervailing argument to "voluntary exchange" is that many trades are highly unequal, and in extreme cases, difficult to distinguish from choices made under duress (starvation vs. sweatshop, expensive medication vs. death, etc)

On top of that, trades that involve large quantities of capital must still rely upon law and violence to guard ownership, and therefore at least somewhat subject to public jurisdiction. The idea that land, or mineral rights, or intellectual property belongs exclusively to either (a) an individual, (b) a corporation, (c) a state, or (d) everybody, are all legal fictions and voluntary illusions. Other than perhaps the clothes on one's back, property is not a natural right (in the sense of natural law). We made it up. For better or worse, we should take responsibility for the relationship between private capital and the social contract, even if it is for the purpose of maintaining private ownership.

I can accept the argument that interfering with the quasi-evolutionary capitalist engine does more harm than good, and that capitalism is the worst system, except for all the others. But given the history of unearned legacy wealth, systemic ethnic and national oppression, and the state's role in maintaining disproportionate advantages for those with capital, calling the trades that occur between haves and have-nots "voluntary" seems a little disingenuous.

<comment><USER8> >> The countervailing argument to "voluntary exchange" is that many trades are highly unequal

If an exchange is genuinely voluntary, it leaves both parties better off by definition. You value whatever you're giving up lower than what you're getting in exchange, because otherwise you just wouldn't go through the trade.

>> On top of that, trades that involve large quantities of capital must still rely upon law and violence to guard ownership, and therefore at least somewhat subject to public jurisdiction

Everyone naturally has the right to defend his property, through whatever means necessary. Not only that, but we all intuitively agree on what property is. An apple in your hand is automatically assumed to be your property, and so is the apartment you live in, or the factory everyone has always known you to own and run. If you see a car on the street, you don't assume it's no one's property even if there's no one in it.

>> For better or worse, we should take responsibility for the relationship between private capital and the social contract, even if it is for the purpose of maintaining private ownership.

I can't tell what that's supposed to mean. But again, your property is your property, and _you_ have the right to defend it regardless of whether a State exists.

>> capitalism is the worst system, except for all the others

Capitalism is not a "system". It's not something that's _imposed_ on anyone, but instead, it's just all the things people do in their pursuit of personal gain - exchanges, commerce, investments, and contracts etc. If you see problems with "Capitalism", they're actually problems with _Crony_ -Capitalism, and caused by the State. Those evil corporations buy political power, and then use it to prevent competition, to rape nature, pass their losses on to _us_ , and so on. All the nasty stuff.

<comment><USER10> We should also be careful to distinguish an exchange in a positive-currency, vs one in a debt-currency, which by definition is a trade of indebtedness. Can you really have a "voluntary" exchange of indebtedness?

No, no, no. We do not agree intuitively on what property is. What about a factory that you built with your own hands that becomes a utility for a community? At some point, your private property and private creations become integral to a community, and that's where things become muddy.

I want to add, lastly, that if we have a true competition of non-debt currency, we may have part of a solution. I have some thoughts written on my blog post, but it's not well thought out and perhaps somebody can build upon it.

[http://jaekwon.wordpress.com/2013/12/02/the-supernova- theory...](http://jaekwon.wordpress.com/2013/12/02/the-supernova-theory-of- deflationary-currency/)

<comment><USER4> I agree with your core thesis about the role of alt-coins in a deflationary spiral. There's one thing I haven't drunk the kool-aid on, though, and I'd like you to tell me if you see a hole in my logic: I don't see how a deflationary spiral is a problem at all.

It's true that if a currency appreciates in value, buyers are disincentivized to spend. However, sellers also have the same knowledge, that the currency will gain value over time. Wouldn't that incentivize them to lower their prices in order to acquire the currency-as-investment, until the two meet in the middle? (Many of the merchants who accept BitCoin offer a BTC discount for this very reason.)

One practical way that we see this happen is in technology, where goods rapidly decrease in value. While a dollar might lose 5% of its value over the course of a year, that's nothing compared to buying a new computer. And so the onus is on computer makers to offer a compelling enough value and price to convince the user to upgrade, as opposed to living with an old machine, or waiting for a better one. At any given point in time, to buy a computer is micro-economically foolish, and yet transactions still manage to take place, because buyers and sellers eventually meet in the middle.

The trick to this, of course, is that this model only works if the currency deflates at a stable rate. BitCoin still has yet to establish itself in this regard; time will tell.

<comment><USER10> Good point on technology.

I also doubt that the deflationary spiral is a big issue as it's made out to be. To prove it, I assumed that it is a big issue, and then proved that it isn't -- proof by contradiction.

I think ultimately it all depends on how well a currency is designed or managed. An inflationary currency could outcompete an autonomous cryptocurrency in theory if it could introduce price stability as well as manage the macro economy. In practice this seems like an exercise in futility. Time will tell what happens with the current QE practices.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> Most of this discussion is centered around the definition of anarchism. It seems difficult to debate undefined ideas.

<comment><USER10> He starts out by saying that Anarchism is a truism. That is, we by default live in Anarchism and cannot escape it.

Then perhaps the discussion is really about what the world should be like in the future. If the discussion brings together Anarcho-* of all sorts to dismantle the current regime, it would be time well spent.

In the very least he helps us disassemble the (often conflicting) definitions so that we can better understand the origins and biases, thereby allowing us to reason more clearly about our reality.

But yeah, I get your point.

</comment></comment><comment><USER12> I find it completely ironic that anarchism itself cannot even escape the left- right paradigm. Going down lower Manhattan on a weekdays, you can find heated arguments between an anarcho-communist vs another anarcho-capitalist.

But I also find the term anarcho-communism to be an oxymoron.

<comment><USER13> Well it's not that they are binded by the left-right paradigm, it's just that they've already agreed that authoritarianism (up?) is worse than libertarianism (down?), and then proceed to debate the finer points of collectivism versus individualism (right and left).

Much better than ignoring the up and down completely like neo-liberal and neo- conservative politics. From the mainstream, we see plenty of disheartening policies that both left and right anarchists would agree on.

That said, the problem I think about it what you've mentioned. How do we convince collectivist thinkers that taxes are the use of force? How do we convince individualists that _some_ services need to be established by the collective itself and not an offshoot (syndicate/business)? Our problems are what you've mentioned. Too much left-right and not enough up and down talk. This opens us all up to authoritarian centrists who believe that title to be impossible because of the golden mean...

<comment><USER8> >> How do we convince individualists that some services need to be established by the collective itself and not an offshoot (syndicate/business)?

We don't, because you're talking about _imposing_ some "services" on other people, which _always_ happens through coercion. We should both be aware that coercion is bad, mm'kay?

"But who will build the roads?!" is like asking an anti-slavery activist "but who will pick the cotton?!" \- the correct answer is: "who fucking cares? Slavery is immoral and needs to stop."

Punishing people for attempting to keep their property (ie. taxation) is immoral too. It's plain to see, for everyone who allows himself to.

<comment><USER14> So you're saying public education is immoral, and public health care is immoral. Maybe for the taxpayer it is. I doubt it seems immoral for the child of some poor parents. Or for someone seriously ill who can't afford to go to the private doctor, no matter if they chose not to plan for the future and be financially sound.

<comment><USER15> Yes, public education & health care are immoral because they are funded through means of coercion. Helping the poor is a noble end, but even noble ends can't justify violent means.

It does not matter how fair theft seems to the recipient of the services that are funded in such a way, it's still theft.

It is cool that you care about the unfortunate ones though. You are not alone in that — most people do, too. If we go from the assumption that democracies really represent the opinion of the majority, it can be seen that a lot of people _do_ care.

So, if the majority of people really want to help the poor, they would still help ( _voluntarily_ this time) without a coercive apparatus, right?

 _(Obviously, private charities will be better at helping the poor, and private schools and medicine are doing a better job that the government does, but my reply focuses mainly on the moral side of things.)_

<comment><USER13> This is the main issue between left and right minded anarchists IMO. I've felt like both sides have been right at different times in my life.

That said, you are right. Taxes of any kind, if forced, is theft. If liberal minded individuals can't offer me a voluntary way to function from within society, than they'll be considered authoritarians to me. The only solution I can think of is a type of confederacy of anarchist collectives, where all collectives agree on basic freedoms (freedom to travel, freedom to leave, completely fair representation, etc) where some are bastions of socialism, and the other bastions of individualism.

The only problem I have as a right leaning anarchist with anarcho-capitalism, is wage labour. While I agree that the contract may be voluntary, the conditions leading to it may not have been, and we see exploitation in our current system regarding this, with some people becoming indentured servants.

<comment><USER15> > The only solution I can think of is a type of confederacy of anarchist > collectives, where all collectives agree on basic freedoms (freedom to > travel, freedom to leave, completely fair representation, etc) where some > are bastions of socialism, and the other bastions of individualism.

Voluntarily-entered socialist bastions can indeed exist in an anarcho- capitalist system. As there is no state and hence nothing particularly limiting, they can aspire any kind of community — including a socialist one. As long as participation in this kind of community is voluntary, there is nothing wrong with it. Members of socialist communities may vary well subject themselves to community's rules and socialist dispute resolution providers in their internal matters.

<comment><USER13> Agreed. But it doesn't look like our brothers/sisters in arms would afford us the same luxuries does it?

To me it sounds like THEY want the monopoly on force to be governed by a majority or consensus, which would then impose taxes, threaten opposition with force, and centralize power.... Sounds like a government to me...

I truly don't think anarcho-capitalism is an abomination. I've come to this stance FROM being a socialist because it's the only moral way of doing things. Thoughts?

<comment><USER16> >I've come to this stance FROM being a socialist because it's the only moral way of doing things.

Deontology <: morality. That is, deontological moral codes are a strict subset of all moral codes people actually find compelling.

<comment><USER13> Could you explain what you mean?

What I meant was that while I believe equality and guaranteed access to basic needs as the goal of humanity and necessary for future growth at some point, I also believe that it's impossible to achieve these goals through institutions and governments.

My moral change from being a left leaning anarchists to a right leaning anarchist was because I was swayed by Stefan Molyneux's Philosophical arguments for Individualist Anarchy. The only sticking point I still have is the question of wage labour, which I am opposed to. But then it's hard to imagine a society where you can't work for a wage...

I think there's a saying about how cows in the slaughterhouse can't truly argue about what's outside. They both have no experience.

<comment><USER8> >> I was swayed by Stefan Molyneux's Philosophical arguments for Individualist Anarchy. The only sticking point I still have is the question of wage labour, which I am opposed to

Good job so far. Next, you need more knowledge of economics: [http://mises.org](http://mises.org) and you'll get around to reaching full sanity :p

<comment><USER16> Except that Austrian "Economics" takes an ideologically anti-empirical stance, so it can't actually be called part of any reputable _science_ of economics.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> Anarchism is pure nonsense. There is literally no such thing as "no government."

In a "state of nature," every government is size 1---you. You are responsible for retaliatory and initiatory force, and every interaction is foreign diplomacy with a gun behind it.

From there, things can (and will) evolve so that there are larger governments, but you cannot get "no government."

<comment><USER8> Your profile looks like you're an objectivist. But as others pointed out, you can't just dismiss anarchism off-hand.

As an objectivist, you're against coercion, but in favour of centralized power? Is that correct? If so, don't you think it's a contradictory idea?

<comment><USER18> There's a large body of literature that attempts to justify centralization of power. Social contract theory, to name one. You may not agree with it, but the possibility of a justifiable centralization of power can at least be argued.

<comment><USER8> >> There's a large body of literature that attempts to justify centralization of power

 _Of course_ there is. We've lived under centralized power for hundreds or thousands of years, after all. Why would The Powers That Be suddenly cease wanting to Be?

But centralized power cannot be _justified_ , it can only be rationalized. If you think about the idea of the social contract for a moment, you should realize that it's not a contract at all, but merely a shackle for our minds, to keep us from finding a way out of the maze of our brainwashing.

For starters, _you were never even asked_ , you never had a chance _not_ to participate, and you can't just opt out of the system either. You're either inside a country, or you're freezing to death in Antarctica. It's not much of a choice, but a _voluntary_ choice is necessary for any contract to be binding.

Read this: [http://spot.colorado.edu/~huemer/book3.htm](http://spot.colorado.edu/~huemer/book3.htm)

<comment><USER19> Sounds much like the system of property.

<comment><USER8> What does?

<comment><USER19> _" For starters, you were never even asked, you never had a chance not to participate, and you can't just opt out of the system either."_

<comment><USER8> That may be, but _people_ are not property. Or, well.. _should not_ be.

<comment><USER19> You do realize that the system of property doesn't apply to actual objects? My house doesn't become green if I write so in the deed. The system of property is a set of obligations imposed on _people_ , not things. Much like the social contract.

<comment><USER8> Hm. Yeah, I guess that's a good point. The idea of property can be thought of as "obligations between people", but it's not the same as the idea of the social contract. For starters, it's between individual people, not some collective entity and a given individual, and second, it involves no force as long as no one _aggresses_ against anyone.

Another difference is that the "system of property" actually works to everyone's favour - quite the opposite of "the social contract", and imposes no other obligations on you than to not aggress against anyone else's property. If "aggress" sounds out of place, let's say "interfere with someone's (exclusive) control of their property". There's lots of room for pointless bickering in moral issues, etc.

There are various sensible ways to view the concept of property. One is Murray Rothbard's idea of something becoming your property by "mixing your labour" with it. For example, if you build a house, or plow a corn field, they're your property.

<comment><USER15> All good, but even though Rothbard mentions it, this theory of property is actually Lockean, not Rothbardian. [1]

[1]: [https://en.wikipedia.org/wiki/Labor_theory_of_property](https://en.wikipedia.org/wiki/Labor_theory_of_property)

<comment><USER8> Alright. Keep up the good work, fellow sane person :p</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment>


Ubuntu & HTC collaboration unveiled today? - <AUTHOR>

Both companies have a countdown on their frontpage that looks suspiciuosly similar:
www.htc.com
www.ubuntu.com
Is an HTC tablet with Ubuntu preinstalled coming today? <comment><USER2> Some reports have there predicting early release of new HTC One series of phones since the unit second counter turns green every time it hits one.

Also an interesting observation is that HTC's clock is leading by a minute.

Looks like a new partnership is brewing.

</comment><comment><USER3> Good find, it certainly does look like that.</comment>


Ruby Plus – Coding in the Browser - <AUTHOR> http://rubyplus.biz/

<comment><AUTHOR> Learn C, C++, C#, Elixir, Go, Haxe, Java, Node.js, PHP, Python 2, Python 3 and Ruby on a browser in any device for free.</comment>


Google Ejects Open-Source WireGuard from Android Play Store over Donation Link - <AUTHOR> https://www.phoronix.com/scan.php?page=news_item&px=WireGuard-Ejected-Play-Store <comment><USER2> [https://news.ycombinator.com/item?id=21268389](https://news.ycombinator.com/item?id=21268389)</comment>


The best way to build a dev team: Go where the devs aren't - <AUTHOR> https://medium.com/@cameronp/the-best-way-to-build-a-dev-team-go-where-the-devs-aren-t-d3f226cfe749#.njhck2dzf <comment><USER2> Upvoting this since I'm also very excited about Elixir these days :)

Hopefully, more jobs will be posted to the Radar very soon.

<comment><USER3> I'm the author. I actually think it may be awhile before we actually start to see jobs posted, unfortunately. I think the pattern is largely:

1) Let's try it out internally 2) Hey, this is cool. Let's go to the meetup. 3) Wow, this meetup is full of fired up devs. 4) Let's hire someone we met there .... time passes .... 5) Ok, now we seem to have become at least partly an Elixir shop. Let's post an ad.

Ironically, because it's so easy to find great devs who want to work with new technologies like this, you don't actually have to go to the trouble to post an ad, leading to the (incorrect) impression, that there are no jobs at all.</comment></comment>


How NOT to write pandas code - <AUTHOR> https://towardsdatascience.com/how-not-to-write-pandas-code-ef88599c6e8f <comment><USER2> I've never seen a Pandas tutorial advocating the for-loops the author describes and I have rarely come across for-loops in other people's Pandas code. I think the article could remove the for-loop "how not to" examples and it would be a typical "getting started with Pandas" article.

<comment><USER3> Obviously nobody would advocate for them. But for a new Pandas programmer, loops are incredibly obvious, familiar, and do work. So, tons of people use them.

</comment></comment><comment><USER4> The title is a bit clickbaity, so here's a TL;DR:

Try to use the built-in vectorized Pandas functions instead of writing explicit loops.

<comment><USER5> It's wild how much faster iterrows and itertuples are. I wish I'd known a year ago.

This is one of those cases that really isn't just pedantic, "pythonic", nagging. It matters.</comment></comment>


My Journey from Zero to $10k MRR – SaaS Growth Case Study - <AUTHOR> https://www.whalepages.com/saas-growth-case-study-my-journey-from-0-to-10000-mrr/ <comment><USER2> Thanks for some great advice Karsten! :-)

<comment><USER3> Thanks for thanking ;D</comment></comment>


Hacker News Radio - <AUTHOR> http://hackernewsradio.com/ <comment><USER2> You might want to add a warning about this requiring flash - I had to turn flashblock off to get this to work.</comment>


A Biotech Company Changed Its Name to 'Riot Blockchain' and Its Stock Is Surging - <AUTHOR> https://www.bloomberg.com/news/articles/2017-10-04/from-biotech-to-bitcoin-bioptix-shifts-focus-to-blockchain <comment><USER2> Haha this is all so dumb. Greater fool theory at work.

...not that I don't own some Ethereum, tho...</comment>


Test submission, please ignore - <AUTHOR> I'm trying to figure out why my hn-replies subscription isn't working.  Ignore this post. <comment><AUTHOR> test comment

<comment><USER2> Hi.

<comment><AUTHOR> Hi Dan. My notifications are still not working :-(

<comment><AUTHOR> Oh wait, I seem to have spoken too soon...</comment></comment></comment></comment>


Million Song Dataset - <AUTHOR> http://millionsongdataset.com/ <comment><USER2> This dataset doesn't have audio if that's what you are after

<comment><USER3> They do, however, provide derived features of the audio, as well as code for downloading associated audio from 7digital: [https://github.com/tbertinmahieux/MSongsDB/tree/master/Tasks...](https://github.com/tbertinmahieux/MSongsDB/tree/master/Tasks_Demos/Preview7digital)</comment></comment>


PayPal's Incentive Problem - <AUTHOR> http://stratechery.com/2014/paypals-incentive-problem/

<comment><USER2> I think the author made the core point in a footnote: "2\. Thus the acquisition of Braintree"

PayPal (though newly independent) is such a behemoth that no amount of product rearchitecture or rebranding is going to "beat" Stripe. But, it may be able to happen with Braintree, and since PayPal = Braintree from a business perspective PayPal's fine with that.</comment>


"I can tell you which startups will succeed, without even knowing what they do" - <AUTHOR> https://m.signalvnoise.com/silicon-valley-arrogance-i-can-tell-you-which-startups-will-succeed-without-even-knowing-what-89aa8ea35d23 <comment><USER2> So now we know why she took those idiotic decision which drove Yahoo into the ground at like a fighter jet - Decision fatigue of over worked mind.

It is either that or she is either completely incompetent or had malious intentions.

But in the end she did very well for herself financially while ruining Yahoo investors, employees, and customers equally.</comment>


Pissed off about functional programming - <AUTHOR> http://perlmonks.org/?node_id=450922

<comment><USER2> I get the feeling the author's total knowledge of functional programming comes from reading -1 comments on Slashdot.

      > It's theoretically possible that the 'f()' or 'x' might
      > change between the lazy evaluation of step one and the
      > lazy evaluation of step three million. Trying to prevent
      > that is what we programmers call a 'hard' problem.

I don't know of any functional language that allows values to change during evaluation.

      > Trying to run functional logic in two or more
      > simultaneous threads can raise serious problems, for
      > instance.

The whole point of using functional programming in multithreaded applications is that it _doesn't_ raise such problems. The data are immutable; one thread doesn't affect the other, unless the programmer explicitly uses stateful constructs like pointers.

      > Myth #2 - Functional programming is 'different from' imperative programming.
      > [...]
      > On the other hand, 'imperative' is also used as a
      > contrast to 'functional', on the basis of really poorly
      > defined terms. The most common version runs something
      > like, "imperative languages use assignment, functional
      > languages don't".. a concept I'll jump up and down
      > upon later.

I've never heard this definition. Ever. It simply doesn't make any sense; of course functional languages use assignment.

      > No, functional programming is not some kind of magic
      > pixie dust you can sprinkle on a computer and banish
      > all problems associated with the business of
      > programming. It's a mindset-altering philosophy and/or
      > worldview, not a completely novel theory of
      > computation. Accept that. Be happy about it. At least
      > friggin' COPE.

This paragraph is completely ridiculous. Functional programming:

1\. Is not magic problem-solving pixie dust (obviously).

2\. Is not a philosophy, mindset-altering or otherwise.

3\. _Is_ a novel theory of computation.

Whether a Turing machine and the lambda calculus are equivalent in power is irrelevant; they _are_ , undeniably, different. Otherwise, there would be no point in using one over the other.

      > Myth #3 - Functional programming is referentially transparent

This entire section deals with Perl, which is not a functional language (as it is not based on the lambda calculus). I don't know enough about Perl to state whether the authors assertions are correct, but regardless, they have nothing to do with functional programming.

The second half of section #3 claims referential transparency is impossible because it requires two values with some external equivalent meaning, such as "three" and "3", to be considered equivalent by the program. At this point I'm wondering whether the entire article is some elaborate prank.

      > Myth #4 - Functional programming doesn't support variable assignment
      > [...]
      > According to Church's thesis, all programming languages
      > are computationally equivalent. They all do the same
      > things, and only the syntactic sugar is different.

Section #4 is already off to a bad start...in fact, reading further, it's just more of the same. Abject confusion, combined with a belief that Perl can be used to demonstrate functional concepts, yielding incomprehensible conclusions.

<comment><USER3> I'm decently sure "functional programming has been called programming without assignments" comes from Ravi Sethi's programming languages book.

<comment><USER2> In that case, I'd be very interested to know what Dr. Sethi meant by it. Even the simplest Haskell program requires one assignment (to `main`), so the idea of programming without them seems like an exercise in futility.

<comment><USER4> The "=" in the declaration of Haskell functions isn't considered an assignment, but a binding of a name to a definition. Usually when you have assignments, you are permitting a name to be rebound to a different value than it had originally. In Haskell you cannot rebind a name. Essentially "=" in Haskell is more like the operator's use in defining mathematical functions.

Due to how scoping and how do-comprehensions actually work, you can do some things that look like assignment, but are really creating a new binding in a nested scope. But if you try to bind a name twice in the same scope, the program fails to compile.

<comment><USER2> What definition does "assignment" have other than "a binding of a name to a definition"?

Keep in mind that there are at least three popular definitions of the = operator:

1\. In C or C++, = handles assignment and mutation (no re-assignment):

      int x = 1; /* assignment */
      x = 2; /* mutation */

2\. In Python, = handles assignment and re-assignment (no mutation):

      x = [1] # assignment
      x = [2] # re-assignment

3\. In Haskell, = handles assignment (no re-assignment or mutation)

      let x = 1 in

Just because Haskell's = operator does not behave like C's = operator is no reason to claim Haskell does not support assignment. In fact, that is the _only_ thing Haskell's = operator supports.

<comment><USER5> This is now a semantic discussion - which is not necessarily a bad thing, we should just know that up front.

To me, the statement "programming without assignment" implies programming without making multiple assignments to the same name - mutation or re- assignment. I don't know if the programming language community has agreed-upon semantics for the word "assignment," but to me, it implies re-assignment and mutation. What you call "assignment" I think of as binding - that is, the lvalue gets bound to the rvalue.

<comment><USER6> The technical terms are 'variable assignment' and 'single assignment'. 'Assigment' is generally considered to mean the former unless it's qualified.

From a pedantic perspective, you can't really disagree with the statement 'haskell supports single assignment.' But equally as pedantic, most actual text reads 'FP doesn't support _variable_ assignment', including the original article above.

I don't have a copy of the Ravi Sethi book to see what the exact text of his exact statement was.

Now that I've seen this image on wikipedia:

[https://secure.wikimedia.org/wikipedia/en/wiki/File:Fp_no_de...](https://secure.wikimedia.org/wikipedia/en/wiki/File:Fp_no_destructive_assignment.png)

I'm really tempted to waste some more money at Cafe Press.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> The author of this article appears never to have used a functional programming language in his life. He misunderstands several key terms in the realm of functional programming and makes up a few of his own terms so that he can construct a strawman of what people claim functional programming is, which he can subsequently prove that it isn't. There are plenty of misconceptions about functional programming, but quite a bit more than combating them, he demonstrates how many he himself holds.

      > Myth #1 - Functional programming is Lambda Calculus

Of course functional programming is not just lambda calculus—there are actual data types besides functions—but I don't think this is what he's arguing. He's trying to argue that "simultaneity" (which I think he made up) isn't satisfied, and somehow conflicts with lazy evaluation. This is only a valid argument about lazy, impure languages, and I don't think any exist. All of the differences he's claiming exist between lambda calculus and functional programming don't exist between lambda calculus and Haskell.

      > In point of fact, the simultaneity constraints inherent in lambda calculus have been shown to make it unsuitable for issues involving concurrency (like multithreading), and for problems like that, it's better to use pi-calculus.

Whaaat?! Purely functional programming is much _better_ suited to concurrency, because it completely satisfies "simultaneity constraints", which makes race conditions, etc. impossible.

      > Myth #2 - Functional programming is 'different from' imperative programming.

... > Church's thesis, upon which we base our whole definition of 'computing', explicitly states that Turing machines, lambda calculus, while programs, Post systems, mu-recursive grammars, blah-de-blah-de-blah, are all equivalent.

The entire point of the Church-Turing thesis is there can be completely different models of computation that can compute the same problems, not that they're all the same concepts underneath.

      > For those quick thinkers out there who've noticed that this puts Myth #2 in direct contradiction with Myth #1, KA-CHING! you've just won our grand prize!

I'm not even sure what he means here. The statements that functional programming is based on lambda calculus, that it is different in significant ways from imperative programming, and that all models of computation can solve the same problems are not at odds in any way.

      > Myth #3 - Functional programming is referentially transparent

This is where I get the impression that he has never actually used a functional programming language in his life, and he completely misunderstands what "referentialy transparent". Haskell, barring its unsafe* functions, is indeed referentially transparent, which means that expressions will always have the same value no matter when they're evaluated. His use of Perl examples, using features of Perl that directly violate referential transparency, to demonstrate that functional languages aren't referentally transparent, is bizarre.

      > Myth #4 - Functional programming doesn't support variable assignment

I don't even know what he's trying to prove in this section. Yes, functional programming does allow you to iterate functions on values to produce new values. Haskell's standard library provides several utility functions for doing just this. The point of not supporting assignment is eliminating mutable state, which results in referential trasnparency, and not letting the effect of one part of your code leak out in hard-to-predict ways and affect other parts of your code, both of which it does successfully.

His example makes no sense at all. He shadows a variable and then pretends that it's the same as the parent variable so that he can prove that variables don't always stay the same, because `x == y` is true in one scope and false in another. This doesn't change anything about referential transparency or having assignment, because referential transparency doesn't say that if you copy and paste an expression into a different scope it will have the same value.

I wish that instead of getting angry at other people for misunderstanding functional programming, he should try a little harder to actually understand it himself.

</comment><comment><USER8> This seems confused.

"Lazy evaluation brings time into the picture"

Lazy evaluation changes the programming language semantics; but _time_ , per se, is not "brought into the picture" in any visible way - if the specific _time_ of evaluation were visible, the program wouldn't be referentially transparent.

"In point of fact, the simultaneity constraints inherent in lambda calculus have been shown to make it unsuitable for issues involving concurrency (like multithreading)"

Functional programming is particularly suited to parallelization, but concurrency demands a different approach, but it depends on the application domain.

</comment><comment><USER9> You might want to put "(2005)" in the title.

I'll pick a nit. He writes, "According to Church's thesis, all programming languages are computationally equivalent. They all do the same things, and only the syntactic sugar is different. In this case, the syntactic sugar is so different that people can end up using variable assignment without knowing they're doing it ....."

I think this is incorrect. Church's Thesis (and correct me if I'm wrong) says that all programming languages are "functionally" equivalent; as in, if you look at a program in a language as a black box, you can replace the contents of the box by another program in another language, and the results would be identical. Now, whether one language uses variable assignment and other doesn't, is irrelevant; and it doesn't flow from the claim.

<comment><USER10> Corrected in a followup comment:

<http://perlmonks.org/?node_id=451236>

</comment></comment><comment><USER11>
       > On one hand, 'imperative' means 'a list of operations
       > that produce a sequence of results'. We contrast that
       > with 'declarative programming', which means 'a list of
       > results which are calculated (somehow) each step of the
       > way'. Mathematical proofs and SQL queries are
       > declarative. Most of the things we think of as
       > programming languages are 'imperative' by this meaning,
       > though, including most functional languages. No matter
       > how you slice it, Haskell is not declarative.

Either I'm confused or the author is.

SQL is declarative, sure, but I don't think mathematical proofs are. Sometimes you do declarativeish things like, "Let x be the solution, known to exist, to this unsolvable problem." But I think the structure is fundamentally very imperative; ignoring the syntactic sugar, formal mathematical logic is a sequence of logical transformations that leads from hypothesis to conclusion.

I can chalk that up to confusion or inexperience on the author's part, but the bit about Haskell really confuses me.

I'm still quite new to functional programming -- just emerging from the "how the heck do I do that without for loops?" phase -- and the experience really has been one of learning to program all over again. As in the CS 101 days of yore, I have to spend a lot of time trying to figure out how to accomplish simple things, dealing with alien logical consequences and constructs.

Now, Scala lets me do things either way, and I like to think I'm getting to the point where that's an advantage rather than a crutch. But looking back over the last little while, I can't imagine how you could _possibly_ slice Haskell in such a way that it's declarative at all. I mean, I've been banging my head against the relentless declarativity of it for a while. I'd like to think if there were crutches, I'd have found them.

Unless the author is talking about something silly, like the fact that it's ultimately compiled down to an imperative language to run on a real processor . . . I can't figure out what he's talking about. And I don't know whether to chalk that one up to confusion on his part or inexperience on mine.

<comment><USER2> The author is very confused.

"Declarative" and "imperative" are just syntactic styles. Haskell supports both For example, the following Haskell expressions are equivalent:

      dec :: [Integer]
      dec = map (* 2) [1..10]

      imp :: [Integer]
      imp = do
              x <- [1..10]
              return (x * 2)

Due to their origins, stateful languages are traditionally imperative, and functional languages are traditionally declarative. But please don't confuse the two sets of terms.

                    Stateful
                       |
                       |
                       |
                       |
      Imperative       |       Declarative
      </comment></comment><comment></comment><comment>-----+</comment><comment></comment><comment></comment><comment>
                       |
                       |
                       |
                       |
                       |
                   Functional

<comment><USER11> Thanks for that.

I did know about the do syntax in Haskell, but I didn't think it really made the language usefully imperative. Sure, it's okay if you're trying to kick off two or three unrelated calculations, maybe if you're after side effects. But I didn't find it much to fall back on when trying to do something complicated and having a hard time thinking about how to break down the problem without for loops and case statements and value accumulators. Maybe it's more robust than I know, but it seems like such a small piece of such a big language. Calling Haskell imperative on account of that seems like calling perl OO because you can kinda sorta do basic OO stuff most of the time.

I am also having trouble understanding the claim that the ideas are orthogonal. I can see plainly enough that they aren't literally the same thing: an imperative language breaks a problem down into simpler _steps_ , while a declarative one breaks a problem down into simpler _definitions_. And a stateful language accumulates progress in changing values, while a functional language does so by resolving expressions.

So far so good, but I can't really decouple the concepts. How would you break a problem down by time, but track results through logical resolution? Isn't the point of a step or instruction to leave something about the state of the program changed--I mean, isn't it inherently stateful?

It seems to me the advantage of imperative programming is that the steps are logically decoupled, just as the advantage of declarative programming is that the pieces are temporally decoupled. So how in the world would you make a declarative program accumulate progress via state?

<comment><USER2>
      > But I didn't find it much to fall back on when trying to
      > do something complicated and having a hard time thinking
      > about how to break down the problem without for loops
      > and case statements and value accumulators.

When I'm porting some stateful code to Haskell, I usually start by transcribing it as closely as possible. That means using for-loops, using stateful accumulators, and usually putting the whole thing in an IO. Once you've got a bit component compiled and working, you can start factoring out chunks into smaller, pure functions. It's the same as working out a big tangle in some string: keep working and the mess will just sort of evaporate.

Of course, the difficulty here varies wildly depending on how good the original code is. Most stateful code is pretty ugly (lots of mutation, lots of tangled side-effects), but a few developers are competent enough to write good code in any language.

      > Maybe it's more robust than I know, but it seems like
      > such a small piece of such a big language. Calling
      > Haskell imperative on account of that seems like
      > calling perl OO because you can kinda sorta do basic
      > OO stuff most of the time.

I think you're looking at the word "imperative", and assuming a lot of properties that simply aren't included. A language is not imperative or declarative; a language can _support_ writing in imperative or declarative _style_ , and some languages support certain styles better than others, but it's not like one is much better than the other.

      > And a stateful language accumulates progress in
      > changing values, while a functional language does so
      > by resolving expressions.

Every program progreses by changing values. The difference is scope; a stateful language progresses by applying a sequence of mutations to a single state. Every procedure is a sequence of mutations to that state. Haskell programmers (somewhat snarkily) call that state the World.

In functional programs, state is still accumulated, but in fragments. Each function has access only to what state it needs, and its contribution to the program's progress is restricted to its returned value. Only IO values have access to the World.

That last sentence means that Haskell is also a stateful language. It is, effectively, a superset of C.

      > So far so good, but I can't really decouple the
      > concepts. How would you break a problem down by time,
      > but track results through logical resolution? Isn't the
      > point of a step or instruction to leave something about
      > the state of the program changed--I mean, isn't it
      > inherently stateful?

The point of imperative programming isn't necessarily state, but dependency. Step 10 depends on the results of step 9, which depends on the results of step 8, and so on. It might be that no step actually requires access to the whole state -- only one little corner of it.

In well-designed programs written in stateful language, a great deal of effort is spent on how to separate state from dependency. In Haskell, the compiler handles it.

      > So how in the world would you make a declarative
      > program accumulate progress via state?

With parameters and the call tree, primarily.

<comment><USER11> That was . . . really enlightening.

Thank you for taking the time to educate me. I've found the Haskell community uncannily welcoming to newbies, and you seem to be continuing the tradition.

<comment><USER11> To complete the exercise (in no particular language),

Imperative and stateful:

       var factorialResult = 1;

       factorial(n) {

         if(n <= 1) {
           factorialResult = 1
         } else {
           factorial(n-1)
         }

         factorialResult = factorialResult * n
       }

Imperative and functional:

       factorial(n) = {

         var lastResult = 1

         if(n <= 1)
           lastResult = 1
         else
           lastResult =  factorial(n-1)

         return n * lastResult

       }

Declarative and stateful:

       var factorialResult = 1

       factorial(n) {
         if(n > 1)
           pointlessFunction(multiplyBy(n), factorial(n-1))
       }

       multiplyBy(n) {
         factorialResult *= n
       }

Declarative and functional:

       factorial(n) {

         if(n <= 1)
            1

         else
            n * factorial(n-1)

       }

Everyone learns imperative and stateful. Some people also learn declarative and functional. Good programmers tend to do functional when working imperative. And declarative stateful is possible, but just plain silly.

No value judgement can be made between imperative and declarative; do what suits the problem. Sometimes you're following a recipe for cookies, and other times you're parsing XML.

Functional _is_ better than stateful, but not in a competitive sense -- not like ipods are better than walkmans. More in a pragmatic sense -- more like laws are better than wars . . . when the law will solve your problem. Sometimes you really do need to talk to the world. For everything else, there's functional programming.

</comment></comment></comment></comment></comment><comment><USER5> Comments link to a good LtU discussion that counter his claims and confusions: <http://lambda-the-ultimate.org/node/678>

</comment><comment><USER12> The goals of functional programming are to abstract away as much detail of the calculation as possible where as the goals of imperative programming seem to be providing a lot of transparency as to how the problem will be calculated. If you get away from being in one camp or the other you'll actually see that the two camps are moving closer together. Personally, I think the best example of this is C# (an imperative language with functional extensions) and F# (a functional language with imperative extensions).

Sometimes I really want to specify that a calculation must take place via a certain method (lets say newton's method and the performance improvements that are possible with certain hacks) and sometimes I don't care (such as partial application of a function).

The best situation is to put both tools in the hands of a programmer and let them decide what to do. Neither the functional nor imperative camps are 'right' but there are a lot of good ideas in both camps.</comment>


Verizon Suffers Cloud Data Leak Exposing Data on Millions of Customers (2017) - <AUTHOR> https://www.darkreading.com/cloud/verizon-suffers-cloud-data-leak-exposing-data-on-millions-of-customers/d/d-id/1329344 <comment> Stefan-H I find the reporting of this as a "data leak", as opposed to a more aggressive term, rather concerning.

"UpGuard in its data estimated that up to 14 million customer records were exposed, but Verizon stated that data on 6 million of its users was affected."

I don't see how this would be considered a "leak" rather than a "breach". Are we starting to become desensitized to these security issues, leading to the reporting on them similarly weakening? Is the downplay of language a form of shielding the companies responsible from backlash?

<comment><USER2> "leak" might be used to denote the release in a more passive / accidental sense, whereas "breach" in its traditional usage denotes a purposeful attack, such as if the release stemmed from a wilful actor.

From the Oxford English Dictionary: Breach. A gap in a wall, barrier, or defence, especially one made by an attacking army.

</comment></comment><comment><USER3> Interesting, but [2017]

<comment><USER4> Thanks. Added above.

<comment><AUTHOR> My Mistake. I try to be good about that, for some reason it just popped up in my aggregator and I did not look closely at the date.

</comment></comment></comment><comment><USER5> This article is from 2017. How is this news?</comment>


State Supreme Court Says Digital Cameras Can't Be Searched Without a Warrant - <AUTHOR> https://www.techdirt.com/articles/20170821/10485338053/state-supreme-court-says-digital-phones-cant-be-searched-without-warrant.shtml <comment><USER2> It doesn't matter what any court says when we don't do anything about officers violating someone's rights.

<comment><USER3> The court system is the check against officers violating rights.

In this article, the officers violated the suspect's rights and the court system decided against indicting him on the crime he was charged with.

<comment><USER4> And like the parent says...

What, exactly, were the consequences of the officers violating his rights?

And I don't think you read the article:

> the court system decided against indicting him on the crime he was charged > with.

Absolutely he was indicted, AND convicted.

It had to be appealed to the Supreme Court before the convictions were vacated.

</comment></comment></comment><comment><USER5> Article is about digital cameras, not phones. The article indicates that there was already a decision about cellphones and that it applies to cameras as well.

<comment><USER6> Title fix is needed here

</comment></comment><comment><USER7> But feel free to search those analog phones all you want.

</comment><comment><USER8> Does this mean that passengers on international flights into Logan can no longer have their devices searched at customs?

<comment><USER2> You don't have rights if you're within 100 miles of the border.

<comment><USER9> That's disingenuous, at best. Yes, the rights to be secure in your papers and property are diminished. No, they can't make you house troops within that zone.

I'm also pretty sure they can't just come in my house on a whim. I live within that zone. I still have a right to express myself, bear firearms, etc...

</comment></comment></comment><comment><USER10> _Massachusetts_

</comment><comment><USER11> It's not the supreme court of the US but the supreme court of MA. They are different and have jurisdiction over different things.

</comment><comment><USER12> But the blochain can, eh? (IRS)</comment>


Riak - web-shaped data storage system - <AUTHOR> http://highscalability.com/blog/2009/10/8/riak-web-shaped-data-storage-system.html

<comment><USER2> It looks really, really interesting - the presentation they have here <http://riak.basho.com/nyc-nosql/> explains CAP, Amazon's dynamo concepts, Map/Reduce beautifully. Definitely going to try it.

Only thing I am concerned here is that since the data is accessed over HTTP Stack, won't it become the performance bottleneck.

<comment><USER3> Glad you enjoyed the presentation, and I hope your experimentation goes well.

Re: HTTP Stack: "Performance" is such a slippery word here. If what an application is most sensitive to is the time it takes to fulfill one request, then yes, the HTTP stack may be an important component to keep track of, as it plays a role in defining the lower bound of request fulfillment time.

However, since all HTTP requests are handled in parallel, and since any node can handle any request, applications whose main sensitivity is not latency, but is instead throughput or availability, are less likely to find the HTTP stack to be a performance bottleneck.

There is also the option of doing away with the HTTP stack completely, and speaking Erlang straight to Riak.

</comment></comment><comment><USER4> What are the differences, advantages, dis-advantages between Riak and CouchDB? I haven't found a comprehensive comparison till date.

Edit: Yes, I have been a bit lazy not to go and try Riak first hand. I think I will do it sooner then later but as of now, any light on the subject will be appreciated.

<comment><USER5> They serve different purposes. Riak is a DHT (for key sharding) layer intended to run on top of different backends, just like Dynamo. CouchDB has an integrated storage engine. Riak doesn't support range queries (nature of DHT). I'm pretty sure CouchDB does, which is easy for a single node. CouchDB doesn't scale in the sense that it doesn't do any automatic sharding yet. It's quite possible to use CouchDB as a backend of Riak, though it implies crippling some interesting features in CouchDB.

Technically, I found CouchDB more interesting as it's more "different" from other approaches.

<comment><USER6> The Scalaris DHT supports range queries, I think.

<http://www.zib.de/CSR/Projects/scalaris/>

<comment><USER5> DHT can certainly support range queries by broadcast and merge. The keyword in the context though is _efficiently_. Scalaris not a typical Dynamo like DHT. It's based on Chord, so it's possible support range scan with O(logn) metadata RPCs (n is number of nodes.) per query, which I wouldn't call _efficient_ as these RPCs are expensive in that they not very cachable (nature of the DHT, unless the range query is exactly the same.) Typical Bigtable implementation can easily cache portion of metadata with clients and resulting 0 metadata RPC in many cases.

</comment></comment></comment></comment><comment><USER5> Yet another Dynamo/DHT. Now I've seen implementations in Java (Dynamo, Cassandra, Voldemort), Perl (Blekko) and now Erlang. I wonder if any of these has been used on >1000 node cluster? I haven't seen any mention of checksums (besides those in TCP) to ensure that switch/router would not corrupt data (including cluster states), like Amazon learned the hard way.

Of course, they don't support range scan efficiently either. A proper Bigtable implementation is much harder, I guess :)

<comment><USER7> > I wonder if any of these has been used on >1000 node cluster?

1000 nodes is a _lot_. Facebook's inbox search Cassandra cluster was at 150 nodes 6m ago; probably around 200 now given their growth trend -- so unless you have more data than FB you probably don't need to worry too much. :)

IIRC the largest OSS bigtable-clone cluster is Baidu's at around 120. (maybe 150 now?) So same ballpark really.

> they don't support range scan efficiently either

Cassandra does.

> A proper Bigtable implementation is much harder

It's actually much easier to get something that works when all your hardware goes well with the bigtable single-master model; it's just that having those single points of failure is a bitch (as even google with their vast engineering resources and 3 year head start is still working out, as in the appengine outage a couple months ago -- [http://groups.google.com/group/google- appengine/msg/ba95ded9...](http://groups.google.com/group/google- appengine/msg/ba95ded980c8c179)). Once you put the engineering in, the fully distributed model is much better.

(For those following along: I'm a Cassandra developer; vicaya is a developer of hypertable, a bigtable clone.)

<comment><USER5> 1000 nodes are nothing if you crawl the entire web and keep a history. I'm sure someone already has a modified version of an OSS bigtable cluster at 1PB+ and close to 1k nodes :)

>> they don't support range scan efficiently either > Cassandra does.

Only if you call order preserving hashing efficient when all the keys can be _easily_ hashed in to one bucket, no matter what hash function you choose :) Cassandra's range query is an occasionally useful hack, which is _not_ comparable to Bigtable like implementations, in terms of efficiency, scalability and robustness.

The Bigtable's single-master with standbys is never a problem in practice. The AppEngine outage was a due to a bug in GFS master protocol decoding. If this bug is in every node, all nodes would be crashing instead of one with some service (reads and some writes) still available. The so called "fully" distributed model is only better when you assume that you don't have bugs in your code and that node failure is random. I argue that separating different responsibilities/functionalities into different components is good for fault isolation, a sound software engineering practice. Naive fully distributed model is much more brittle in practice due to code bugs.

BTW, although Hypertable is inspired by Bigtable in design, the implementation and features set (support a dozen languages via Thrift and full read isolation via MVCC) are different enough that I wouldn't call it a clone :)

</comment></comment></comment><comment><USER8> the basho guys are insanely smart and I've been dying to get a look at riak, really excited to see it released.

</comment><comment><USER7> What's the 10,000 ft summary of how Riak is different from Dynomite, PowerSet's Dynamo-clone-in-Erlang?

<comment><USER3> Disclaimer: I've never actually run Dynomite, so I'm going on what I can find in docs.

In any case, a few of differences I found:

1\. Configuration of N/R/W parameters. Dynomite appears to set these properties once, at startup time, for all values. Riak configures N per-bucket ("bucket" being like a 1-level directory structure), and R/W per-request. Specifying N/R/W so "late" means that applications can put different CAP requirements on different sets of data and tune those requirements at request time.

2\. Dynomite's main external interface seems to be thrift-based, with also some RPC exposure. Riak's main external interface is JSON-based, over ReSTful HTTP.

3\. Hinted handoff is a target for Dynomite's 1.0 milestone (according to <http://wiki.github.com/cliffmoon/dynomite/roadmap>). Riak already fully supports hinted handoff.

<comment><USER7> Thanks!</comment></comment></comment>


KDnuggets newsletter (good supply of links about data analysis/mining/discovery) - <AUTHOR> http://www.kdnuggets.com/news/2009/n01/index.html

<comment><AUTHOR> I've been subscribed to this newsletter for a long time and found it useful because it's a quick to read, broad overview of academic and commercial goings on in the data mining and knowledge discovery community. I'm not affiliated with them in any way.</comment>


Ask HN: How will a service with try[service].com change it when it scales? - <AUTHOR>

If a service (I just saw Keep and thought about this: trykeep.com) has a website like that, what'll happen when they want to make it large scale? Will they move to keep.com (given it's available?) Or is it an indication that they are not confident of becoming big? <comment><USER2> They want to build something quickly and see if people want/use it, also as early adopters don't really care about URL/Domain. If it becomes big, they will try buying keep.com</comment>


Why developers are dumping PHP for ColdFusion - <AUTHOR> http://www.techpluto.com/coldfusion-over-php/

<comment><USER2> there are php frameworks that replicate coldfusion's declarative benefits.

Anyway, this article is awful. Last I heard, the trend is for CF developers to migrate to .Net/PHP and not the other way

</comment><comment><USER3> seriously? those are the reasons?</comment>


20 Year Usenet Timeline (2001) - <AUTHOR> http://www.google.com/googlegroups/archive_announce_20.html

<comment><USER2> Already appeared on HN about 4 years ago: <http://news.ycombinator.com/item?id=318537>.</comment>