Academia, Love Me Back - <AUTHOR> https://vivatiffany.wordpress.com/2016/10/27/academia-love-me-back/ <comment><USER2> Interestingly this happened to me and to my younger brother.

There was no racial component to it although we do belong to a religious minority, come from low-income backgrounds and were homeschooled. I personally felt some prejudice at various times on account of these characteristics.

In my case I was a 16 year old high school dropout taking a history class at a Junior College. My prof had a lengthy conversation basically trying to get me to admit my parents wrote an essay for me and eventually gave up, although clearly unconvinced.

In my brothers case a few years later (different college, different prof) I believe he was given a zero on a test and had to file a grievance to be able to redo the work or retake the test. This is circa 2000 so my memory is a bit fuzzy.

In both our cases there was no evidence - just the perceived mismatch between our presentation and ability prompted the accusations. (We were bookworms and in a family of bookworms despite being poor and lower class.)

My only comment and the only relevance to the linked story is I remember how OUTRAGED I felt at the accusation. It wasn't even the accusation of cheating - that I would be dishonest. It was the condescension, the implied "obviously you are not good enough to have produced this" attitude that hurt the most. In my case the professor was obviously trying to give me a break and be friendly but I remember how hurt I was and in my brother's case... I don't think he would ever speak to the prof again.

Being judged incapable of producing your own work BURNS.

<comment><USER3> > My prof had a lengthy conversation basically trying to get me to admit my > parents wrote an essay for me and eventually gave up, although clearly > unconvinced.

Couldn't she have spent the time talking to you about history and confirming your analytical and literacy skills were up to the required standard?

</comment></comment><comment><USER4> We (as outsider observers) lack the evidence to judge this story objectively. It sucks to be unfairly accused of plagiarism (it happened to me once in high school).

However, we don't know why the professor came to the conclusion that she plagiarised. Maybe they jumped to conclusions on limited evidence, or maybe this paper was written with vocabulary and sentence structure which was inconsistent with the student's earlier work. Either way, it's incredibly unlikely that an academic professor of sociology would be a closet racist.

tl;dr It's unfair to be accused of plagiarism for submitting a paper which sounds "too intelligent", but it's also unfair to be accused of racism for attempting to enforce academic standards.

Unfortunately, this a story which is going to press the internet's hot buttons.

<comment><USER5> Maybe not an active KKK-rally attending racist, but a soft racist? There's a reason why two identical resumes for post-graduate work get rated differently when the names are "Chad" vs "Shaniqua".

The fact is, when you really get down to it, most people are subconsciously racist. Not a joke - most people. There have been studies on it and everything.

<comment><USER4> "The fact is, when you really get down to it, most people are subconsciously racist. Not a joke - most people."

If this is true, why the moral outrage towards the professor?

<comment><USER6> Because everyone has some amount of racism whether they realise it or not but it still is not acceptable to display that racism. Sure having some racist thoughts isn't great but just like with almost every other unacceptable topic, everybody thinks about it sometime. It is only a problem when it becomes verbal/affecting others. That is why people are upset with him.

</comment></comment></comment></comment><comment><USER7> I side with the student. This is beyond a jerk move.

To be clear, the _beyond a jerk move_ part is doing this in front of the class with such flimsy evidence. I have see this before. It happened to a black woman friend. In this case the professor's flimsy evidence was a shift in tone which happens in editing all the time.

That said, _hence_ is an adverb and not a conjunction. Hence there is no comma after hence. BTW, this punctuation mistake argues strongly _against_ plagiarism.

<comment><USER8> This is not a uniformly accepted rule. Commas are common after adverbs such as therefore, nevertheless, and hence. There is no mistake here.

<comment><USER7> You might be right. She may have used _hence_ in the conjunctive adverb sense.

Having looked it up a bit, it still goes both ways. The argument against the comma is that if _hence_ is a transitional word then you would omit the comma. If you feel the need for a pause then you would insert it.

So it's a fielder's choice and not a mistake.

</comment></comment></comment><comment><USER9> This is a truly well-written piece, and it accomplished everything it set out to describe without a diatribe or personal attack on the anonymous professor.

Considering how viciously accusations and personal vendettas can be carried out online, I think it's a great mark of character that she refrained from such tactics and explained the injustice she felt in a levelheaded way.

</comment><comment><USER10> It somehow feels that we are missing the other side of this emotionally charged post. I just used the word "hence" in a previous HN comment, but realize that I rarely use this word in verbal conversation.

However, this doesn't excuse the alleged behaviour of the professor in the article. When making a plagiarism accusation, is it common to call this out openly without attempting to address 1-1 first in academic settings?

<comment><USER11> I've never heard of such a thing, but perhaps that's because I am in a UK university. We have a formal procedure for dealing with such things, which certainly doesn't involve public humiliation. It also provides students the chance to defend themselves, which is obviously missing here.

<comment><USER12> There's a formal process at this girl's school as well, I'm sure.

</comment></comment></comment><comment><USER13> I've always been, and to this day continue to be, a poor speller. Hence my prepared writing is always a bit more erudite than my casual spoken communication or even my in class exam answers. I simply don't trust that I'm spelling a word like "erudite" properly without the help of spell-check. Is it really so odd that someone would use a phrase in academic writing unusual in their other forms of discourse?

Its a terrible thing to be falsely accused. Especially so when it is by a person in authority and in front of others and when the accuser has the power to issue a grade. I hope that this article helps professors deal more sensitively with their concerns over plagiarism in the internet age.

</comment><comment><USER14> I've had professors do this to me. I don't understand what's wrong with this. This is a common thing I even do when grading programming assignments. When I notice a stylistic choice that is outside of the norm for the students I can usually tell exactly who the work came from. Every semester I catch at least one group of cheaters (This semester I caught two!) and they admit to it after I confront them with it.

"Hence" is fairly common in modern English, but is it common in the writings of this student?

One such example of this is that one of my students used a "while not in" loop which they had never done before. I looked at it puzzled for a second and then went "wait a minute, one of my other students used this in every lab since we introduced the while statement". I pull up the code, compare it, and BAM! I've got a match.

Just because something is possible/correct in a language and that the person wiring the piece has every bit of ability to use the complex structure at had it doesn't mean they would normally do so. Sure it's not accurate, and sure you may not be right with every call (hasn't happened to me yet but other TAs have called people out and have turned out to be wrong), but it's something that I'd call a "clue" that something is different.

Either the student put more time into the work and found a better/alternate way of approaching the problem, the student asked a friend to help with describing the structure (or in this case what wording to use), or in the only case that presents malice: they copied it.

The only way you can find out is by going to the student and confidently saying something to the effect of "I know you've engaged in some questionalbe academic behavior, this is your time to explain yourself before I contact my boss and have them run a more though investigation". Cheaters usually crack instantly. (pro-tip for anyone on the outside who _is_ cheating: just lie to us, we're probably not going to follow up unless it's extremely obvious if you've got a 1/4 way decent explanation).

I see nothing wrong with this. Just talk to the professors boss and they'll sort it out. This is especially true for cases where you actually did write the work.

Also, I highly doubt anyone questions the intellect of the writer. They are obviously a gifted person.

<comment><USER15> part of the problem is public shaming

        > This morning, my professor handed me back a paper (a literature review) in front of my entire class and exclaimed "this is not your language."

While it's reasonable to ask someone if they've plagiarized, asking that person about it in public in front of their classmates is mean-spirited.

<comment><USER16> There is also something about the extreme emphasis on 'this is _not_ your word'. Much like the public shaming, it suggests the professor is not just displeased, but viscerally outraged.

<comment><USER14> You may not know this but it is an offensive thing to see a student fail or cheat to get out of failure. It's not right to get mad about it but that doesn't change the fact that it does happen.

I try not to get mad and I try to be as objective as possible but after I know for sure it is very disappointing. It's a reflection of my abilities I feel.

</comment></comment></comment></comment><comment><USER11> This is crazy. If the story is true as written, the academic in question needs to be fired. I would expect nothing less at my own University.

I find the story hard to believe only because it is so terrible -- and that an academic would be stupid enough to accuse a student of plagiarism without concrete evidence (in particular, the text from which they plagiarised). However, even if the academic in question had such evidence, they still should give the student the chance to defend themselves.

<comment><USER16> There is a lot I feel like I can get done at my university. Outrage will do a lot.

That said, I know of a few professors that could get away with this shit. There are always a few professors that are thoroughly entrenched.

</comment></comment><comment><USER17> First rule: Institutions can never love you back.

[http://clincancerres.aacrjournals.org/content/5/9/2281](http://clincancerres.aacrjournals.org/content/5/9/2281)

Simone's Maxims are possibly the best piece of career advice anyone ever gave me.

At least she's learning this early in the game. She's clever and tough and will either go far or be smart enough to make much more money at a real job.

Her prof? Not so much.

<comment><USER18> That's a great link, thank you!

<comment><USER17> Please share it with everyone who might benefit.

Be the change you want to see in the world ;-)

(Ha ha only serious. Being an idealist made me cynical. Then it brought me back again. In the end we only have each other, those who we live and love and work with, in this world. It may as well be the best world we can make it.)

</comment></comment></comment><comment><USER19> It looks like the professor pulled a jerk move, jumping to conclusions based on scant evidence. Yet the writer seems to conclude that happened because of her ethnicity, without offering any evidence to support that. Maybe the professor is just a presumptuous jerk who pulls that with students regardless of their race, sex or ethnicity.

I'm not saying that's the case, because I have no evidence either way.

<comment><USER20> Maybe the professor pulls that stuff on people without _conscious_ regard to gender, socioeconomic status, or ethnicity, but in the context of longstanding explicit and implicit discrimination, I'm not inclined to give him the benefit of the doubt that all of his students are equally likely to get called out. Ask your brown friends if they've ever been pulled out for a "random" screening by the TSA.

</comment></comment><comment><USER21> Wow. Just, wow. "This is not your word"??? Even when I stretch my charitable suspension of disbelief to its absolute limit I cannot come up with any interpretation of that phrase that is even remotely defensible.

<comment><USER22> Well, if taken out of context supplied by the author of the article, "This is not your word" could be interpreted as an observation that there is a phrase that does not fit the style of writing of the piece as a whole. That would make it constructive criticism.

</comment></comment><comment><USER23> Never trust just one side of a story. Never.

</comment><comment><USER24> Her post is largely vacuous and superficial. Why? Because it doesn't really provide any evidence that would help the reader understand what really happened. Most of it consists of descriptions of her feelings, uncorroborated accusations, tired ideological slogans, a pervasive self-pity, and the presumption of racism. Add to that the other interesting bits in the article such as how she demands to be recognized as an "academic" even though she isn't even a graduate student. What delusion! The topic of the essay also suggests she's been indoctrinated with the ideology of resentment, one that recently has taken on a frightening degree of irrationality on college campuses in the US. Many of these people require psychological help and should not be roaming around college campuses where they quickly find convenient scapegoats for their issues and rationalize their choices.

Also, it's not like she's going against the grain here. Academia, especially where the humanities are concerned, is overwhelmingly leftist in this country. An accusation of racism in one of these departments is socially damning. If she were a conservative Catholic arguing in defense of sexual morality during a gender studies class at one of our glorified ivy league universities or women's colleges, then I'd say she's got courage. However, this is not the case. Her choice of language indicates that she is riding briskly and blandly on the profitable coattails of the oppression gravy train. Whether it is calculated or merely the result of intellectual error upon intellectual error is another matter altogether.

Bottom line: these shows of support she's receiving are not only inappropriate, but presumptuous. This idea that we must believe what people say on their own authority, or based on their feelings (constrained by the prevailing ethos, of course), with no evidence is a worrying trend in left- leaning humanities departments.

I've come across jerk teachers myself. I have been suspected of plagiarism when my essays exceeded the average length for the class. I've had arrogant teachers presume things in ways that defy common sense. Just because some teacher is an ass (and for the purposes of this post I'll grant her that much) doesn't mean there's any reason to invoke all of that baggage. Victimization ideology is a pathological condition that's being propagated by leftist academics that does severe harm to those concerned. It's used for political gain in the Alinsky style and has the effect of entrenching poverty and cultural pathologies. Sure, upper middle class white (or occasionally black, etc) academics can generate this vile substance with no effect on them, but it does have a real affect on those whom it concerns.

<comment><USER25> I'm puzzled by your comment, especially the first few sentences. What did you not understand from her blog post? What kind of evidence would help you understand? What would you ask her to clarify, what would you like to know, if you were told this story in person instead of as a blog post?

</comment></comment><comment><USER26> I am amazed that a professor would be so surprised by a student using the word "hence". Most professors I've had would probably have taken off points for using it over a simpler alternative like "therefore" or just a different sentence structure that incorporated the two sentences without needing an adverb to tie them together.

<comment><USER27> In every humanities class I've taken, the professor wouldn't grade us based on word choice nitpicking.

</comment></comment><comment><USER28> She doesn't say what happened after she got the paper back but going by what she wrote there is no evidence that the prof singled her out because of her ethnicity.

If he or she just handed her the paper back with that scrawled on it that's a dick move, absolutely, but she's reading a lot into "those aren't your words" and doesn't provide any information that backs up her claim her prof is racist.

I checked out her blog and the comment section is full of posters piling on the prof for being "racist".bHow do they know this? Became a stranger on the internet says so? The mob mentality that self-righteous liberals exhibit doesn't help them much in the credibility and integrity department.

I

<comment><USER28> Whoops...please pardon the typos and stray letters

</comment></comment><comment><USER29> And I remembered, when I was a freshman, in my Chinese writing class, my teacher insulted me in front of the class like this, "good novel, but never copy and paste", why? because he thought this kind of writing style can't be accomplished by a freshman. I just simply answered, "Thanks. But I never plagiarize" The teacher shocked for few seconds, then he apologize and said "I will recommend to the publication". And my classmates all can't wait to see my novel since the curiosity of human being. If I just accused them without fighting back, things will went to another direction undoubtedly.

</comment><comment><USER30> I think academia has a big problem related to diversity and what the author described was deplorable behavior by the professor, but that said, I don't think this specific example can be fairly used to generalize. Every academic institute I've interacted with takes accusations of plagiarism seriously, requiring formal procedures and evidence, and possibly expulsion. Maybe things are different elsewhere, but I wouldn't attribute this awful treatment to academia as a whole.

</comment><comment><USER31> It is comforting to know. the world is same! As a student from a government run poorest rural village school in India, my college professors constantly questioned how I was outperforming and even suggested that I might be using the bribery system to get good grades in standardized final exams. TBH, I actually enjoyed that experience.

</comment><comment><USER32> This is ridiculous. I'm not at all a native speaker, but after a few years in academia, I learnt to use "hence," in that exact same way

</comment><comment><USER33> Maybe he just couldn't get over someone using an exclamation point in a paper.

From Strunk & White: "Exclamations. Do not attempt to emphasize simple statements by using a mark of exclamation.

The exclamation mark is to be reserved for use after true exclamations or commands."

<comment><USER34> Strunk&White is full of bad advice tho.

</comment></comment><comment><USER35> What an admirable post!

</comment><comment><USER36> That sounds too much like l'esprit d'escalier. We all encounter jerks and most often there is not much more behind it than that some people are jerks.

</comment><comment><USER29> I just saw an interesting attitude when a people encountered a tough-time. They didn't fight, instead, they accused, they accused every people dared to question them, accused them as racist, as mean, as not polite, as any possibility they could think of. Come on, this is how the society goes and this is happens every time. And if you tends to have this kind of mindset. Every time you face a challenge, you accuse someone else. To be honest, you are just a lamentable loser. Life is hard and everyone will face their low tide, fight back, not only accusing.</comment>


Is The Key to Happiness Being Busy? - <AUTHOR> http://www.farnamstreetblog.com/2013/02/is-the-key-to-happiness-being-busy/

<comment><USER2> I think that matches up pretty well with the past year of my life. I'm almost always busy but never in a rush. I definitely feel the happiest I've ever been.</comment>


We Should Be Able to Take Facebook to Court - <AUTHOR> https://www.nytimes.com/2019/01/06/opinion/facebook-privacy-violation.html <comment><USER2> I wish the NYT gave Equifax half the scrutiny they give FB.

</comment><comment><USER3> Does anyone have any good examples of concrete privacy harms that people have suffered?

Generally, the larger an effect is, the easier it is to measure. If we don't have any ideas on how to measure it, that makes me think the effect isn't as real as proponents want to claim.</comment>


The 34-Year-Old Mega-Rich Facebook Investor You've Never Heard Of - <AUTHOR> http://www.businessinsider.com/the-34-year-old-mega-rich-facebook-investor-youve-never-heard-of-2011-1#

<comment><USER2> god I hate headlines like that</comment>


MIT Entrance Exam from 1869-1870 - <AUTHOR> http://libraries.mit.edu/archives/exhibits/exam/

<comment><USER2> They did not want me, so they raised their standards since 1869.

</comment><comment><USER3> I am confident that at least 75% of students at the average US public university would score 60% or less on that test.

<comment><AUTHOR> A couple of the questions were surprisingly easy (e.g. find the area of a circle, given its radius), but you're right about most of it.

Interesting how on the English section, the questions were basically trivia. There wasn't much at all in that part testing skills, as opposed to specific knowledge.

<comment><USER4> The English section, in addition to being beset by trivia, also misspelled 'Bismarck'. And what the hell does Otto von Bismarck have to do with English, anyway?

</comment></comment></comment><comment><USER5> So easy my girlfriend could do it.

</comment><comment><USER6> maybe erasers weren't allowed or something...</comment>


Truck driver has GPS jammer, accidentally jams Newark airport - <AUTHOR> http://news.cnet.com/8301-17852_3-57597971-71/truck-driver-has-gps-jammer-accidentally-jams-newark-airport/ <comment><USER2> I'd call it "accidental" when, for example, a selfmade TV antenna with amplifier starts to oscillate and thereby blocks GPS reception in a harbor:

[http://gpsworld.com/the-hunt-rfi/](http://gpsworld.com/the-hunt-rfi/)

But the jamming device owned by the grossly negligent truck driver performed even better than designed (if one would call these brute-force wideband jammers "designed."). And the fine imposed on the guy for operating those deliberately is well earned.

{edit: spelling, wording}

</comment><comment><USER3> [https://news.ycombinator.com/item?id=6123535](https://news.ycombinator.com/item?id=6123535) GPS signals are surprisingly easy to disrupt (economist.com) 208 points by douglasfshearer 13 days ago | comments

[https://news.ycombinator.com/item?id=4122654](https://news.ycombinator.com/item?id=4122654) Illegal GPS Jammers Are Widespread, Study Finds (techweekeurope.co.uk) 65 points by baha_man 421 days ago | 52 comments

</comment><comment><USER4> And this is why GPS and cell phone jammers are illegal...

</comment><comment><USER5> Looks like the company was Tilcon.</comment>


The Origins of the Early Medieval State - <AUTHOR> http://www.historytoday.com/paul-fouracre/origins-early-medieval-state

<comment><USER2> I don't think you can discuss the Early Middle Ages, otherwise known as the _Dark Ages_ without talking about what we now call Venice, the oligarchy of which transplanted itself later into London, where it still resides to this day.

"The islands of the lagoons provided an invulnerable refuge, comparable to Switzerland during World War II, for Roman aristocrats and others fleeing the paths of Goth, Hun, and Langobard armies. Already between 300 and 400 A.D. there are traces of families whose names will later become infamous: Candiano, Faliero, Dandolo. Legend has it that the big influx of refugees came during the raids of Attila the Hun in 452 A.D. Various areas of the lagoons were colonized, including the present site of Torcello, before the seat of administration was fixed at a group of islands known as Rivus Altus ("the highest bank"), later the Rialto, the present location of the city of Venice. The official Ab Urbe Condita is March 25, 721 A.D. Paoluccio Anafesto, the first ruler of the lagoon communities, called the doge (the Venetian equivalent of Latin dux or Florentine duca/duce, meaning leader or duke), is said to have been elected in the year 697.

The most significant fact of this entire period is that the whelp of what was later to become Venice survived and grew thanks to its close alliance with the evil Emperor Justinian in Constantinople, an alliance that was underlined in later years by intermarriage of doge and other leading Venetian oligarchs with the nobility of Byzantium, where a faction embodying the sinister traditions of the Roman Senate lived on for a thousand years after the fall of Rome in 476.

Venetian families are divided into two categories. First come the oldest families, or Longhi, who can claim to prove their nobility substantially before the year 1000. The Longhi include many names that are sadly familiar to the student of European history: Dandolo, Michiel, Morosini, Contarini, Giustinian (perhaps related to the just- mentioned Byzantine emperor), Zeno, Corner (or Cornaro), Gradenigo, Tiepolo, and Falier. These old families held a monopoly of the dogeship until 1382, at which time they were forced to admit the parvenu newcomers, or Curti, to the highest honor of the state. After this time new families like Mocenigo, Foscari, Malipiero, Vendramin, Loredano, Gritti, Dona, and Trevisan came into the ascendancy."

"between 180 and 280 AD, the empire collapsed. It was reorganized by Aurelian, Diocletian, and Constantine with a series of measures that centered on banning any change in the technology of the means of production, and very heavy taxation. The Diocletian program led to the depopulation of the cities, serfdom for farmers, and the collapse of civilization into a prolonged Dark Age.

The Roman Empire in the West finally collapsed in 476 AD. But the Roman Empire in the East, sometimes called the Byzantine Empire, continued for almost a thousand years, until 1453. And if the Ottoman Empire is considered as the Ottoman dynasty of an ongoing Byzantine Empire, then the Byzantine Empire kept going until shortly after World War I. With certain exceptions, the ruling dynasties of Byzantium continued the oligarchical policy of Diocletian and Constantine.

Venice, the city built on islands in the lagoons and marshes of the northern Adriatic Sea, is supposed to have been founded by refugees from the Italian mainland who were fleeing from Attila the Hun in 452 AD. Early on, Venice became the location of a Benedictine monastery on the island of St. George Major. St. George is not a Christian saint, but rather a disguise for Apollo, Perseus, and Marduk, idols of the oligarchy. Around 700 AD, the Venetians claim to have elected their first doge, or duke. This post was not hereditary, but was controlled by an election in which only the nobility could take part. For this reason, Venice erroneously called itself a republic.

In the years around 800 AD, Charlemagne, King of the Franks, using the ideas of St. Augustine, attempted to revive civilization from the Dark Ages. Venice was the enemy of Charlemagne. Charlemagne's son, King Pepin of Italy, tried unsuccessfully to conquer the Venetian lagoon. Charlemagne was forced to recognize Venice as a part of the eastern or Byzantine Empire, under the protection of the Emperor Nicephorus. Venice was never a part of western civilization.

Over the next four centuries, Venice developed as a second capital of the Byzantine Empire through marriage alliances with certain Byzantine dynasties and conflicts with the Holy Roman Empire based in Germany. The Venetian economy grew through usury and slavery. By 1082, the Venetians had tax-free trading rights in the entire Byzantine Empire. The Venetians were one of the main factors behind the Crusades against the Muslim power in the eastern Mediterranean."

<comment><USER3> What is that quote from? That was amazingly interesting and I want to read more!

<comment><USER4> [http://www.abjpress.com/tarpv1.html](http://www.abjpress.com/tarpv1.html)

<comment><USER5> Ah - Webster Tarpley. Kind of nutty, lots of conspiracies, but some good information nevertheless.

That said, the (very long) history of the Venetian Republic is both fascinating and important. The city of Venice (in the lagoon) is built entirely on wooden piles - astonishing both for the engineering and effort involved. Both words bank and ghetto are from Venice. Etc. Bank was originally a banca or bench on which proto-bankers or money lenders or whatever would sit.

</comment></comment></comment></comment><comment><USER6> I believe most of the issues in Western Europe were due to the reorganization of the Roman Empire by the Emperor Diocletian after the time period known as the crisis of the third century. The reforms he implemented basically helped turn the west to Feudalism. He tied people to the land, and forced people to work in the same trade as their parents. He also tried to reform the economy by fixing prices which of course didn't work out well. Since the coinage was reduced to near nothing, people had to resort to barter, which I'm going to guess stifles trade.

<comment><USER7> He also saved the Empire from total disillusion and established a system that might have led to stability had Constantine not broken up the Tetrarchy.

Also he did not 'tie people to the land', landowners did that as a means of escaping paying taxes.

<comment><USER8>
      ...as a means of escaping paying taxes.

Taxes levied and enforced by an emperor.</comment></comment></comment>


'Anti-authority' tech rebels take on ISPs, connect NYC with cheap Wi-Fi - <AUTHOR> http://www.cbc.ca/news/technology/wifi-nyc-mesh-new-york-city-1.4617106 <comment><USER2> There's an impressive amount of negativity in these comments. A community-run mesh network? HN should be all over it, but it appears to have turned into a bunch of network engineers telling us why they are very smart and it won't work, or is "insignificant".

More power to you, NYC Mesh.

<comment><USER3> It's OK to be optimistic about something different. It's also ok to discuss the reality, that's not necessarily negativity towards a project. Are we only supposed to discuss one side of the argument and ignore reality just because it makes us feel nice inside?

There are bazillion reasons that this is nothing more than a cute side project that will fall on its face when its time to scale in any meaningful way. Unless they have discovered a new technology they are not taking on ISPs by any meaningful way. Especially in NYC.

There is a difference between, just "connecting to the internet" and having a "reliable connectivity to the internet", the difference is a massive investment in infrastructure. Wifi p2p mesh networks don't work on magic. They are very unreliable and hard/impossible to scale to 100,000 users, let alone millions of users. In some places, wifi p2p connection hardware actually costs more money than a traditional wired network.

Mesh networks have been around for a while, technical and reliability aspect of this is well understood within the networking industry. There is a reason why it's not widely used, especially in cities. In remote areas, this kind of network makes sense, if you don't have a choice or an alternative.

Don't brush of educated criticisms from people within the industry. I own an ISP for 5+ years (mostly wired). I am not in US and I am not at all worried if this kind of network comes in my country/area. We are not worried because we have tried and deployed this kind of network in cities and they are extremely unreliable and costs real money.

I also don't understand the "Anti-authority" angle of the argument, unless I live in a different universe, they still have to get license/permit from the local government to do this business (non-profit or not), they still have to follow the same rules enforced by local and federal government when it comes to log/customer information sharing to LEO. So not sure how they are Anti- authority? Unless they mean anti-establishment?

<comment><USER4> Once upon a time the network, storage and data center guys thought Facebook/YouTubes one to many broadcasting of content would never scale too. Now they fall over themselves to work there.

<comment><USER3> They scaled by making a huge investment in their network. They didn't make a fundamental new technical discovery that didn't exist before. They had money, they threw money at the problem, as expected it worked. The problem was an investment (which takes time), not technical.

[https://peering.google.com/#/](https://peering.google.com/#/) [https://code.facebook.com/posts/565767133547005/steering- oce...](https://code.facebook.com/posts/565767133547005/steering-oceans-of- content-to-the-world/)

Any ISP (if big enough), can host their own Google, Akamai, Facebook, Netflix nodes on their premise.

</comment></comment></comment></comment><comment><USER5> Shameless plug: I run a website that helps people do this kind of thing: [https://startyourownisp.com](https://startyourownisp.com) (although it's free and ad free, so not much of a plug really.)

Also have a matrix chatroom on this topic: [https://riot.im/app/#/room/#startyourownisp:matrix.org](https://riot.im/app/#/room/#startyourownisp:matrix.org)

</comment><comment><USER6> _The group also knows there will be growing pains as they challenge the status quo and that it 's only a matter of time before the big ISPs take notice, which could bring new challenges._

I'd like to imagine this would be competition on price and service. Sadly, that won't be the case. They can expect everything from spurious CALEA requirements to rooftop lease shenanigans to unwarranted radio interference claims.

Eventually, though, this mesh model will take over the world, from both a network perspective and a commercial one. We just have to squelch our authoritarian impulse.

Of course, technical challenges may still exist, but those certainly don't discredit the model.

<comment><USER7> I don't think mesh makes sense from an economics perspective. The number of edges scales O(n^2), where n is the number of people in the network. That's not very efficient. Even if you think wireless internet is feasible in dense urban areas (I think this is still an open question) it seems like a star topology connected by wireless backhaul to fiber makes more sense. This scales O(n).

<comment><USER8> > The number of edges scales O(n^2), where n is the number of people in the > network.

I don't see how this is true for any reasonable definition of "edge". It's certainly not the case that every node directly communicates with every other node; if they did, you wouldn't need a mesh.

<comment><USER7> A mesh network is defined as a network where the nodes connect to as many other nodes as possible. In the limit this is a complete graph with O(n^2) edges. It's true that in practice you won't have a complete graph but it's still less efficient than a more hierarchical, tree-like network structure.

<comment><USER9> Am I missing something here?

Let me approach your math with a reasonable argument:

Assume a 1 sq mile area (1mi x 1mi).

Assume each node has a 50 yard diameter reach.

If you space a node every 50 yards, you will have 35 nodes on a side, for a total of 1,225 total nodes.

Each node will only be able to physically connect to 3 other nodes on the edges, and 4 other for interior nodes.

That means the node connectivity is only .00326 for an interior node.

The longest path is diagonal across the square. Given the 35 modes on a side, and A^2 +B^2 = C^2, that means 35^2 = 35^2 = C^2. And c^2 = 2,450, meaning C = 50 (rounding up).

So I'm not sure how O^2 even applies in this example. Yes, it's theoretically a max, but theory >< the real world.

Is this not correct?

</comment></comment></comment></comment></comment><comment><USER10> I really don't understand the HN reaction here. Why all the presumed negativity? This is tech, the best encyclopedia in the world is built on volunteer work and donations. A genius builds the de-facto version control system on his own in 3 months for fun. 2 guys in a garage build revolutionize search.

Certainly there are logistical differences between these things, but I don't understand why one would assert with confidence that such a project (that already seems to be working) couldn't work.

<comment><USER11> I love HN but there is a very healthy level of suspicion for developing different ways of attacking already-solved problems. May not be the best example but look at the comments on announcing Dropbox.

Doing > Wanting to do in nearly every instance. So to borrow from popular culture, Just Do It, and Haters Gonna Hate.

<comment><USER10> Yeah, I just figured that a pro-anonymity, pro-startup, pro-charity, anti- monopoly, anti-comcast community would see this as a win on all fronts.

At least that was my read on this community.

<comment><USER12> My reading of the negativity in this thread isn't "here is why I am _opposed_ to your project" but rather "here is why your project seems like it will fail". The latter isn't anti-startup or pro-comcast at all.

<comment><USER13> ...but why The Facebook? We already have MySpace.

...but why Dropbox? We already have google drive/OneDrive/WebDAV.

...but why an Airbnb? We already have BnBs, hotels, and hostels.

<comment><USER12> Which are all questions that the founders of Facebook, Dropbox, and Airbnb needed to answer to be successful. Questions are a contribution.

</comment></comment></comment></comment></comment></comment><comment><USER7> This stuff is cool and maybe useful in far-fetched scenarios, but it's not going to replace ISPs anytime soon. Mesh networks don't make economic sense in most circumstances.

Also, I'm guessing that users share the same IP address, which can cause problems. Plus, if one user is doing something illegal and causes the supernode to get shutdown then no one on the mesh network will have access to the internet.

Addition: quotes like "The internet doesn't really cost you anything, it's just the connection [that has a fee]... Nobody owns the internet, there's no one to pay." Are shockingly naive. Who does he expects pays for the fiber that runs across the country and along the ocean floor? The marginal cost of a new user on the internet is effectively zero, but that doesn't say anything important. The infrastructure that makes the internet work is enormously expensive.

<comment><USER14> We are replacing ISPs in NYC for our members. This is now, not the future. The main thing holding us back is scaling up the number of installs.

All networks, including us, add to this expensive infrastructure. The difference with our infrastructure is it is owned and shared by members.

<comment><USER15> Sure thing. And I offer free WiFi access to all my household members, houseguests, and next-door neighbors. The main thing holding me back is scaling up the number of installs too. I'm hoping a billion dollars of donations can square that away for me and my wider-area neighbor.

<comment><USER16> I get that you're joking, but this is basically how public wifi works.

A bunch of businesses and nonprofits offer free wifi to their customers, and over time you build up a big enough blanket that pretty much anybody can walk into a coffee shop or library and check their email.

Tons of security risks, of course, but... I mean, I've used public wifi before during personal emergencies, and I was pretty grateful it existed. If you're gonna pick a comparison to be derisive with, maybe don't pick something that's widely useful and appreciated?

Comcast has even turned this into a selling point (I think somewhat unethically) by turning all of their customer access points into semi-public routers for other customers. It actually seems to scale pretty well.

<comment><USER15> Comcast doesn't do it _for free_. That is financed by bill-paying customers

<comment><USER16> Yes?

Comcast provides its wifi by allowing you to connect to any other customer's router. (Roughly) a mesh network provides its wifi by allowing you to connect to any of the other nodes within the mesh.

What's your point? Is the money/infrastructure less legitimate because it was donated? Is the public wifi down at the library fundamentally worse because its cost wasn't bundled into the price of a coffee?

Hardware is hardware. If it works, who cares where it came from?

</comment></comment></comment></comment></comment></comment><comment><USER17> This can work for small scale networks but it needs to be a paid service model to scale it(at least for usable performance).

A spine/leaf arch would be great for this imo. The leaves can be local cells of wifi networks. The spine can be operated by payment collecting individuals that maintain a high speed(10g+) and low latency uplink to the internet and network gear that can handle the traffic from the leaves. In spine leaf, leaves connect to multiple spine nodes as well (don't have to worry about spine operator reliability too much,could potentially load balance over spine)

That being said,I hope any modern wifi network assumes wifi encryption is not reliable and instead implements a layer2 or 3 tunnel (macsec/.11ae and wireguard respectively)

</comment><comment><USER18> When you compare comments here a few themes emerge:

1\. The internet has been captured by the telcos, who operate the internet like telcos have operated the circuit switched voice network.

2\. We are all overlooking the fact that smart radios make spectrum ownership obsolete. That means that the potential of mesh wireless connectivity is orders of magnitude larger than can be realized in current unlicensed spectrum.

3\. The assumption that anyone is owed a business model in internet service is going unquestioned. Of course it costs a lot of money to run the internet the way the Bell System ran phones. But there are plausible alternatives, not all of which are compatible with investor-owned near-monopoly "markets."

This is an economic tragedy and a case of local optimization holding a global optimum hostage. ISPs are a sweet business, in a way that costs the rest of the economy dearly. Kill the incumbents and reap an across the board boost in GDP.

</comment><comment><USER2> This sounds neat. Is anyone working on something like it in the SF bay area? I know there's Webpass and Monkeybrains, which are great, but I'd love to be part of a community-owned, actual mesh network.

<comment><USER19> There's the People's Open Network: [https://peoplesopen.net/](https://peoplesopen.net/) co-ordinated by the SudoRoom folks and the wider community.

</comment></comment><comment><USER20> Can anyone comment on the utility of mesh networks in rural areas, where the big ISPs say "Sorry, not interested", and the distances between potential nodes is high? I have read articles in the past about hardware hackers setting up systems to broadcast WiFi signals for exceptional distances. Could a system of mesh networks provide rural America (or other countries) with low cost / high speed internet service?

<comment><USER21> In Catalunya we've had such a network for many years and it is specifically used for providing connectivity to rural parts of the country. It's called Guifi.net and has over 30,000 nodes. I was surprised it wasn't mentioned in the article actually.

<comment> Paul-ish It was mentioned.

> While there are mesh networks dotting the U.S., she says the best working > example of what mesh technology can do is in Spain. Guifi.net has more than > 34,000 nodes covering an area of roughly 50,000 square kilometres across the > Catalonia region.

</comment></comment></comment><comment><USER22> This is a great ideal. Sticking it to The Man is never a bad thing :) The question seems to be one of level of service and reliability. Collectives (if you will) can certainly work. But at this scale? And necessity?

The ultimate benefit, I would think, would be to cause downward price pressure on the establishment ISPs. But wasn't that also Google Fiber's quest? Which we know didn't last long. Profits or not, there is still a certain level of cost to maintain sustainability of the model.

I hope I don't sound cynical. I honestly just curious and will to share my thoughts out loud.

</comment><comment><USER23> Nice. Our 200+ unit condo building in Portland just got gigabit internet via line of sight radio on the roof from another building with fiber.

This is a trend that will definitely take off and removes the huge barriers of entry that laying cable requires. 1ms latency too.

<comment><USER24> I've got to know more about this. Who set it up? Local start up? Bored engineers?

<comment><USER23> Wave G. I think its a spin off a small isp.

</comment></comment></comment><comment><USER2> It looks like they use a lot of Ubiquiti gear, which I'm a fan of (at least as a "prosumer", haven't used it professionally)

</comment><comment><USER25> I'm curious about the technology they are using. Is it an actual mesh network or point-to-point wireless?

<comment><USER2> Here's a map of their network: [https://nycmesh.net/map/](https://nycmesh.net/map/) (uncheck everything except "Link" to see the network)

<comment><USER26> Down here in South Africa we have a similar network. We have had quite shite connections down here for so long that a mesh network was one of the few ways to get a good connection. Link for our map of Cape Town. [https://wind.ctwug.za.net/index.php/nodes](https://wind.ctwug.za.net/index.php/nodes)

</comment></comment></comment><comment><USER27> I love seeing the growth of mesh network.

"We don't have any use cases that show how nice it can be if you have enough users."

They are going to need software. The supernodes are still a symptom of the thinking that you need the signal to go to another stage in order to communicate locally.

I think there should be more software that works on a local network, like things used to work before broadband.

This is what I'm talking about:

[https://www.youtube.com/watch?v=pZ1O_gmPneI](https://www.youtube.com/watch?v=pZ1O_gmPneI)

</comment><comment><USER28> "Anti-authority" seems like a pretty good marketing campaign that WISPs have figured out.

I mean: good for them.

</comment><comment><USER29> Actual ISP network engineer here: I'm sure these people are having fun and all, but it's insignificant in the larger scheme of things.

(correction to original post: I did not at first see an ASN for them, or announcements of their own IP space. They have an ASN and announce two /24s, and peer at one physical location in metro NYC).

Real five nines network infrastructure takes significant capital investment.

This part is so much unadulerated bullshit:

""The internet doesn't really cost you anything, it's just the connection [that has a fee]. So however you can get plugged in — then you're on the internet. Nobody owns the internet, there's no one to pay.""

Actual, reliable internet costs real money, both in the salaries of people to engineer and architect it, salaries for 24x7x365 clued-in NOC staff, equipment, salaries and expenses for field technicians to build it. And that's before you get into things like establishing colo at major IX points, serious core routers that cost $15k each (do you really want to deploy something in the year 2018 to take several full BGP tables that doesn't have a 4 million FIB capacity?), etc. The Internet is a significant construction project at OSI layer 1, whether you're putting PTP radios on rooftops, running aerial fiber, or underground fiber. Otherwise you're just piggybacking on something that another, larger entity has already built.

I give them an A for enthusiasm and effort. It just needs to be channeled the right way so that they can figure out what it actually costs to run a reliable ISP. I'm all in favor of new startup ISPs.

<comment><USER30> > Actual ISP network engineer her

So then you must intimately understand the depth of the lies and anti-consumer business practices that spur such networks.

> Real five nines network infrastructure takes significant capital investment.

5-9s is 5 minutes of downtime a year. I've never had a commercial, let alone residential ISP hit that mark.

> Actual, reliable internet costs real money, both in the salaries of people > to engineer and architect it, salaries for 24x7x365 clued-in NOC staff, > equipment, salaries and expenses for field technicians to build it. ...

Going back to my first point, almost no one is unwilling to pay for their internet connection.

They're unwilling to pay less to get TV service they have no use for and internet than they are internet alone.

They're unwilling to look at the cost of network access in other countries and be happy when comparing that to the cost of internet in a major, densely populated area in the US.

They're unwilling to accept transfer caps and accept the lies they they help the network by making people share when they know that transfer is irrelevant and bandwidth is the limited commodity.

They're unwilling to accept lies of bandwidth speed that isn't attainable even under the best of conditions.

They're unwilling to accept shady and underhanded business practices such as random, "accidental" charges.

> It just needs to be channeled the right way so that they can figure out what > it actually costs to run a reliable ISP. I'm all in favor of new startup > ISPs.

However, as an "actual ISP network engineer", you must understand the (very arguably unfair) regulatory issues that the big players have put in place to make this impossible to do. Google failed at starting an ISP, and it's not like they are tight on either cash or lawyers.

<comment><USER29> Google did not fail at starting an ISP. They decided that the ROI for doing individual residential home FTTH was not worth it, considering the cost per house passed. Then they went and bought Webpass, which has a totally different condo/apartment based business model, which has been wildly successful.

Starting a small ISP based out of extreme dissatisfaction with huge, evil incumbents is something that I encourage any plucky upstart to do. If you don't like Comcast, Charter, RCN, etc, if you have sufficient clue, go do it yourself. Just don't try to do it with $20,000 and some good wishes, because something with much deeper pockets like Webpass will come along and obliterate you.

The point that I am rather bluntly trying to get across to these folks is that if they're going to invest significant amounts of their personal time/engineering resources into it (unpaid labor?), that they should treat it like a serious business and not a nonprofit. Raise sufficient funds, somewhere in the $500,000+ range to start, to build a network that actually does meet five nines through sufficient diversity in topology and routing.

Compared to random YC companies that seem to raise $1m to $5m to do some vaguely defined web2.0 SAAS thing, the amount of money that a group of people need to raise to build the core of a serious ISP is not huge. Elsewhere in this thread I threw out a figure like $750,000 in beginning CapEx if they're serious. For some good reasons, however, it's easier to get $5m of series A funding from a VC for a pure software development business than a wholly- owned-facilities-based startup ISP.

Even if you completely ignore things like colocation costs, IP transit costs, IX/peering costs, office rental, office utilities and overhead, running a serious ISP with live, clued in humans watching the network takes a lot of money per year for salaries.

3 to 5 NOC persons for 24x7x365 NOC phone coverage. At a certain point if you peer with the big boy ASNs they will expect a live human to answer the phone at 3am when something unusual happens in traffic between your mutual networks. That live human needs to be somebody who has the equivalent of 'enable' on your peering router and sufficient clue to wield it.

2 network engineering/architecture positions, multiply by expected fully loaded yearly salary and benefits package for each person.

at least 2 field technicians

vehicles, vehicle insurance, tools for technicians

At NYC salaries? In a greenfield scenario, I could easily spend $900,000 a year in just payroll and benefits, just to _begin to attempt_ to duplicate what some other serious ISPs already have in place for manpower.

And again that is before you spend one dollar on equipment.

<comment><USER30> > Google did not fail at starting an ISP. They decided that the ROI for doing > individual residential home FTTH was not worth it, considering the cost per > house passed.

~Kind of sounds like it failed.~ Apparently Google Fiber is starting to expand again. I still think it serves as a good case against why the current climate makes it near-impossible for a "plucky upstart [with vc funding]" to start a large, traditional ISP.

> Starting a small ISP based out of extreme dissatisfaction with huge, evil > incumbents is something that I encourage any plucky upstart to do. ... Just > don't try to do it with $20,000 and some good wishes, because something with > much deeper pockets like Webpass will come along and obliterate you.

You can't be a plucky upstart while simultaneously having pockets deep enough to fend against the current major ISP players.

I'm not going to sit here and reply line-by-line to your post. The issue isn't that people want a free network. The issue is that people are fed up with the large ISPs and that the large ISPs have made it all-but-impossible to start an ISP.

You seem to think that VC-sized money is the only thing people need to enter the market. You're entirely wrong. Entrenched players have made it next-to- impossible to compete in terms of capital and regulatory hurdles.

<comment><USER29> > The issue is that people are fed up with the large ISPs and that the large > ISPs have made it all-but-impossible to start an ISP.

that's the thing, they haven't. Not in the service territory this new nonprofit is operating in. There's something like 14 different ISPs which overlap service area with them in their part of New York City, _none of which_ are big nationwide Comcast or Centurylink sized entities.

What they're trying to do on a shoestring is on super hard mode, they've picked the single most competitive market in the entire USA to try to compete with independent, non-LEC, non-major-telco companies that _do_ have a few million dollars to throw at their own facilities-based network infrastructure.

<comment><USER30> NYC, and potentially a few other urban areas, have the capability to run fiber easily.

I'm also just not sure why you're acting like this. You're being simply dismissive of the project. There are many successful meshnets.

You've yet to give a specific complaint about this project and have simply ranted about how they're just small, underfunded, and not a real ISP without ever actually substantiating any of that or specifically what they're doing wrong.

<comment><USER29> NYC is actually the textbook example of most expensive places to run new fiber, due to needs for nearly 100% underground, and extensive pre existing utilities. I'm going to guess you've never been involved with an outside plant fiber project.

<comment><USER31> With your knowledge of the subject, how would you suggest they overcome this challenge?

Do they need to run their own fiber - is there a way they could rent fiber space from an existing line? At least for a little while?

Don't be worried that a new player has entered the space; be worried that the existing ones haven't improved the space. The total market space can be increased by creating new connections, and someone has to do it.

<comment><USER14> There are about 10 fiber providers in the city. We've spoken to most of them. It's pretty much impossible for us to lay fiber, so we have to lease it. We'd like to eventually lease a fiber ring across the city so we can cut costs when connecting multiple buildings. If you don't have a ring then you have to pay for the run all the way back to a coloc.

<comment><USER31> That's a good start to having "your own" network, and then you can choose to have packets traverse your network or the peering networks. You could shift egress points around, based on cost and other factors as well. Each link you add can multiply your routing options.

Get large enough, and you can start carrying other traffic too, and eventually start buying/laying fiber. But you know this, it's probably part of your business plan.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER32> Well, it seems this idea has touched a few nerves. In this case, it may indicate you're on the right track. Keep it up!

</comment><comment><USER33> Förderverein Freie Netzwerke e. V. has been doing this, rather successfully, in Germany since 2003 [0]

Similar projects exist in Austria (FunkFeuer) and Switzerland (Openwireless).

[0] [https://en.wikipedia.org/wiki/Freifunk](https://en.wikipedia.org/wiki/Freifunk)

<comment><USER34> There are several distributions for mesh-networks:

\- Gluon is the most popular: [https://gluon.readthedocs.io](https://gluon.readthedocs.io) \- Libremesh is more flexible: [https://libremesh.org/](https://libremesh.org/)

Like NYC Mesh both are based on OpenWRT.org a wonderful distribtion for wireless devices from 4mb flash / 32mb memory up to 64bit x86 support.

</comment></comment><comment><USER35> 'Anti-authority', even with scare quotes around it.

God forbids somebody uses the word "anarchist"...

<comment><USER6> Synonyms are fine. Those with knowledge know what's going on. Those with prejudice can remain in the dark for a time.

</comment></comment><comment><USER36> Is there an inexpensive point to point technology that isn't hampered by rain?

<comment><USER7> 2.4 and 5 GHz aren't too affected by rain fade. And they're cheap.

<comment><USER37> I thought wifi couldn't go through water? (Or is rain just so sparse/diluted that it doesn't really affect it?)

<comment><USER7> I'm not sure about water, but rain doesn't have much effect at those frequencies. At higher frequencies (e.g. 24GHz) rain can make the link useless.

</comment></comment></comment></comment><comment><USER38> [Ron Paul voice] It's happeniiiiinnnnng

</comment><comment><USER39> NYC public housing has had rock bottom cost, high speed Internet for years.

Have to now do something with it.

</comment><comment><USER40> I'm in NYC. I know a few people who played with it. It barely works.

I'm all for this kind of tech solutions, but for crying out loud pretending that this is a replacement for ISPs in NYC really discredits techies.

<comment><USER14> Our average speeds connecting to our supernodes is around 100Mbps. I'm not sure what you are referring to.

<comment><USER15> What's the throughput?

<comment><USER41> What's the latency?

</comment></comment></comment></comment><comment><USER42> Hate to sound stupid but how does DNS work on a mesh?

<comment><USER14> We originally used the qMp/OpenWRT package that included mdns, which is a version of zeroconf for the internal network. It isn't scaling so we now use our own DNS server to resolve internal addresses on our 10/ network.

<comment><USER42> Mesh networks are going to be incredible when they're widespread!

It's truly superior to centralized control by ISPs/Governments and the world will be grateful for this after this whole net neutrality debacle recently.

You guys are brilliant and doing an amazing job and I'm grateful people like you guys exist!

Thank you for the response!

</comment></comment></comment><comment><USER43> Who's responsible when someone torrents copyrighted material through that Internet exchange point?

"NYC Mesh will comply with all federal laws in the countries it operates, however, as policy, no data is collected and therefore no data exists to provide requestors."

This is cute, but if you aren't hunting down people abusing the mesh, the whole connection to the internet will go away for a ToS violation.

<comment><USER44> I've worked for an ISP for about six and a half years. In that time, I have received I-don't-know-how-many e-mails about people torrenting and so on. I've had one conversation with a Special Agent from DHS investigating credit card fraud.

Over this same period, I have given out a customer's identifying information _zero_ times (if more than 24 hours have passed, there is a _very small_ chance that I am able to identify the customer). In every instance, it was "sorry, I am not able to identify the customer".

Zero is also the number of times I have heard anything from any of my upstreams regarding any ToS violations or similar.

<comment><USER45> > if more than 24 hours have passed, there is a very > small chance that I am > able to identify the customer

Hmm ??? Could u please explain.

I always assumed that all ISPs will store all DHCP (PPPoE? PPTP? IANANE) allocations by IP & time.

<comment><USER44> I'm sure that most do. Those records are kept for systems I manage as well, but not for any longer than I need them.

</comment></comment></comment></comment><comment><USER46> Mesh networks are too complicated to be reliable over a long term. They are useful in disaster situations or places with no infrastructure. Project Byzantium ([http://project-byzantium.org/faqs/](http://project- byzantium.org/faqs/)) is a great way to quickly stand up a mesh network as a temporary thing.

<comment><USER30> > Mesh networks are too complicated to be reliable over a long term

I mean, this isn't true. I'm not even sure to begin. Perhaps specific points instead of nebulous nonsense would help us better understand you.

<comment><USER46> Sure. The protocols, the routing, the interference, backend and gear maintenance, regulatory requirements, expense and complexity of multiple WAN links, and ad hoc setup of said links in an uneven physical environment. To say nothing of managing uneven customer load.

Running fiber is expensive but less complicated and less error prone. And unless I missed something, there aren't any initiatives by big companies (that obviously don't want extra expense and headache) to go mesh. Big ISPs use their customer nodes as wifi hotspots but that is obviously not mesh.

<comment><USER30> > Sure. The protocols, the routing, the interference, backend and gear > maintenance, regulatory requirements, expense and complexity of multiple WAN > links, and ad hoc setup of said links in an uneven physical environment. To > say nothing of managing uneven customer load.

I think you have an overly narrow view of what meshnets can be.</comment></comment></comment></comment>


Ask HN: If you are to start a project today, what would be your stack? - <AUTHOR> I had this legacy app with a mesh of jquery, boostrap and php. Moved things to use django a year ago. Recently moved the frontend to use Vue.js. While moving things, I am always confronted on what is the 'current' way of doing things. Was curious to know what people would use if they started today. <comment><USER2> Golang, Postgres, Bootstrap + whatever JS I can hack into it abusing the hell out of template/html.

Seems all that verbose error handling and static typing leads to writing fairly bulletproof code. Have a 5k sloc codebase that doesn't seem to blow up in prod. Parts of it run on a linux server doing fairly CRUDy things, a cross compiled windows client handles the application logic, a virtual filesystem bundles all my assets into the executable and net/http serves it all via localhost.

</comment><comment><USER3> Since I build things to last...

I'd pick something that hasn't changed much in popularity for a couple decades. If this has to be a web app, then plain C on the server and plain unadorned javascript on the client, but only to the extent that actual code is required at all. If this doesn't have to be a web app, obviously ditch the javascript.

It also performs well and has more build-time error checking than the typical stack.

</comment><comment><USER4>
      Front: Elm
      Back:  Elixir/Phoenix
      Store: PostgreSQL

</comment><comment><USER5> Golang, Vuejs, Redis, and Mysql</comment>


Hyperdrive v10 – a peer-to-peer filesystem - <AUTHOR> https://blog.hypercore-protocol.org/posts/announcing-hyperdrive-10/ <comment><USER2> This is really cool, but why reinvent the wheel? For instance SQLite already has tons of years of optimization regarding storing and accessing files on disk.

To make SQLite decentralized (like Hyperdrive) you can put in a torrent. Index it using full-text-search [https://sqlite.org/fts5.html](https://sqlite.org/fts5.html) for instance. Then let the users seed it.

Users can use sqltorrent Virtual File System ([https://github.com/bittorrent/sqltorrent](https://github.com/bittorrent/sqltorrent)) to query the db without downloading the entire torrent - essentially it knows to download only the pieces of the torrent to satisfy the query. This is similar techniques behind Hyperdrive I believe just again, using standard tools and tech that exists and highly optimized: [https://www.sqlite.org/vfs.html](https://www.sqlite.org/vfs.html)

Every time a new version of the SQLite db is published (say by wikipedia), the peers can change to the new torrent and reuse the pieces they already have - since SQLite is indexed in an optimal way to reduce file changes (and hence piece changes) when the data is updated.

I talk a bit about it here: [https://medium.com/@lmatteis/torrentnet- bd4f6dab15e4](https://medium.com/@lmatteis/torrentnet-bd4f6dab15e4)

Again not against redoing things better, but why not use existing proven tech for certain parts of the tool?

<comment><USER3> The big asterisk is, this only works if your database never changes.

<comment><USER2> What do you mean? The author (say wikipedia owners) can change the db as they usually would change (using UPDATE queries say). Those write queries will result in the least-amount of disk-pages updates. In the torrent world this equals a minimum set of pieces modified and needed to be downloaded by users.

<comment><USER4> Last I checked you can't update a torrent. So if Wikipedia changes even a single letter, you'd need to download all the data once more

<comment><USER5> Actually, there is a solution against this. Just combine [https://www.bittorrent.org/beps/bep_0030.html](https://www.bittorrent.org/beps/bep_0030.html) (Merkle-tree-based hashing) with [https://www.bittorrent.org/beps/bep_0039.html](https://www.bittorrent.org/beps/bep_0039.html) (Feed-URL based updates), and in some settings also [https://www.bittorrent.org/beps/bep_0047.html](https://www.bittorrent.org/beps/bep_0047.html) (Specifically the padding files, so that flat files inside a torrent can also be efficiently shared in arbitrary combinations of non-partial files.).

<comment><USER6> All those BEPs are in "Draft" status. Okay, libtorrent implements two of them. But also, BEP 39 (Updating Torrents Via Feed URL) doesn't really fit very well into the fully distributed setting because of the centralized URL part.

So now to update the torrent file you need a mechanism for having a mutable document you can update in a distributed but signed way. Or you could make an append only feed of sequential torrent urls... oh wait.

My point is: Hyperdrive's scope is sufficiently different from your proposed solution that yes, you could probably rely on existing tools (and I have much love for bittorrent based solutions!) but it starts feeling like shoehorning the problem into a solution that doesn't quite fit.

<comment><USER5> The distributed-but-signed way is there in [https://www.bittorrent.org/beps/bep_0046.html](https://www.bittorrent.org/beps/bep_0046.html) (Updating Torrents Via DHT Mutable Items).

That draft status is of little practical relevance, though, if nothing changed for years, and no one voiced well-founded critic on the technical details.

I do agree though that Hyperdrive is different from what the bittorrent ecosystem has to offer. I too like not reinventing the wheel where that's not necessary, as you recommend there. I'll leave you the list of BEPs for further reading, in case you're interested: [https://www.bittorrent.org/beps/bep_0000.html](https://www.bittorrent.org/beps/bep_0000.html)

<comment><USER6> I've been keeping an eye on that list for a long time. There's some really cool stuff in there, and I _think_ bittorrent has really been within reach of being "simply good enough for most applications" for quite some time now. And the massive user base is of course a good thing there, especially if you're talking more about archival projects.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> Been following this for years, congratulations on your release. I looked at dat for building P2P services, but found that ZeroNet was actually more capable for "real" services - maybe now is the time to reevaluate that, especially given the improvements to large (both deep and wide) archives and the new hole punching.

Can you please mention your thoughts on:

\- Discoverability of content in Hyperswarm (DHT Search/"Superpeers"/???)

\- What happens to the old DEP proposals? (There is a critical feature that I need for my service that's still an open DEP proposal! [https://github.com/datprotocol/DEPs/issues/61](https://github.com/datprotocol/DEPs/issues/61))

\- What uses cases do you have in mind for this service?

Thanks!

<comment><USER8> Thanks Mizza!

Not sure if I'm answering your first point, but Hyperswarm is baked into the Hyperdrive daemon, so daemon users should have their drives swarmed/available automatically. There are a few CLI commands to toggle this behavior too, in case you don't want to add your drive's key to the DHT.

The Hypercore Protocol org's creating a similar proposal repo called HYP [0] (we couldn't resist the name), scoped tightly to the core protocol. We're still solidifying the proposals plan, but yours would add lots of value, so we don't want to lose track of it.

Since your proposal is about peer identifiers, you might like that Hyperswarm uses the Noise protocol [0] to handshake each connection, and the Noise key can be used as a stable peer ID.

As for use-cases, check out Beaker (launched today too). Paul's made a whole bunch of example applications that take advantage of Hyperdrive features, like drive mounts.

I'm personally really interested in using Beaker to make personal document indexers + search (kinda like an amped-up Dropbox), but that's for another post!

[0] [https://noiseprotocol.org/](https://noiseprotocol.org/)

<comment><USER8> In the flurry of the launch I got all my links wrong. The HYP repo can be found here: [https://github.com/hypercore- protocol/hyp](https://github.com/hypercore-protocol/hyp)

</comment></comment></comment><comment><USER9> Quick note: it would be nice to ship the hyperdrive daemon as something more self-contained than a npm package.

I tried to install it and got some c compile errors. Probably my nodejs version is too old, but you want to have the absolute minimum of friction for users to install it.

Beaker comes as an appimage for linux, which worked flawlessly at the first try. Maybe do that for the hyperdrive daemon as well?

<comment><USER10> Yes, we'll definitely be doing this. Thanks for the feedback.

</comment></comment><comment><USER11> Total noob question, but:

Last time Dat was on HN I tried to follow the "simple chat application" tutorial[0], but got stuck at the stage where 2 instances were supposed to automatically discover each other because they only intermittently managed to actually discover each other.

Will this new version of Hyperdrive improve this? Or is it something completely different?

[0] [https://docs.dat.foundation/docs/kappa](https://docs.dat.foundation/docs/kappa)

<comment><USER8> The new version has a totally revamped networking stack that should be a lot faster and more reliable. Part of it includes a new UDP holepunching algorithm that's had a super high success rate for us in practice.

There are some diagrams describing this on the Hypercore Protocol [0] site, and the repo's can be found here.

[0] [https://hypercore-protocol.org/#hyperswarm](https://hypercore- protocol.org/#hyperswarm)

[1] [https://github.com/hyperswarm/hyperswarm](https://github.com/hyperswarm/hyperswarm)

<comment><USER10> The new holepunching algorithm also runs distributed on the our DHT, meaning any peer can help you holepunch to other peers.

We also talk a bit more about this on the hypercore website

</comment></comment></comment><comment><USER12> > Hyperdrive is a POSIX-like filesystem implementation

How POSIX-like is it?

From a filesystem I would expect some more rigorousness than just stating this without any extra information, especially as implementing an actual, to-spec, fully distrubuted POSIX file system is known to be a difficult problem and is generally solved by not wanting a full POSIX-compatible implementation instead (ie. no locking, append-only files, etc.).

</comment><comment><AUTHOR> Hyperdrive is part of the peer-to-peer networking stack that Beaker Browser uses for websites. Today is release day and this tech update is the first announcement.

<comment><USER13> What is your relationship with Hyperdrive & the Beaker Browser?

<comment><USER14> He lives inside of a beaker browser. It's his home.

</comment></comment></comment><comment><USER15> Color me interested. I've been wanting to use IPFS for similar use case but there's always something that annoys me or documentation that's lacking.

While reading Hyperdrive/Hyperswarm docs I actually managed to find everything I wanted to know without much effort (most of all it seems like setting up this on private network should be doable)

</comment><comment><USER16> Does it support private clustering and encryption? My use case is to have shared volumes between servers, laptops but without all the networking hassle (VPNs, opening firewalls).

<comment><USER17> My go-to example for 'private cloud' (which as soon as we hit the Trough of Disillusionment on this Hype Train, we will all be talking about) is having a set of people in your social or family group maintaining a file share for your semi-private photos and videos.

You don't have to worry about a flood or tornado taking out all the photos of first birthdays, or Grandma when she was younger than you are now, and you also don't broadcast family dynamics, whereabouts (people have been robbed when social media made it clear they were not at home) or pictures of minors.

You, your uncle who Moved to the City and your cousin by marriage who wants to be a game designer all set up a file server and share the photos (your cousin is gonna throttle traffic while he's playing CoD or Dota of course, which it turns out he does all the time but at least he's an extra backup copy).

I don't know how you keep that one relative from uploading funny things they found on the internet that keep trying to install spyware or back doors though, but I suppose you'd have that problem now.

<comment><USER18> Do you have any good solutions for this exact use case yet?

<comment><USER16> Checked Tahoe-LAFS, Perkeep, but they are too low level. Haven't tried Resilio Connect yet, although I would prefer something open-sourced.

I think we have all the tech needed already (WebRTC, DHT, NAT hole punching, decades of p2p, encryption, onion security, StorJ/Filecoin) etc but what is lacking is dead simple UX and wide support of operating systems - Windows, Mac, Android, iOS, Linux (Raspberry, Synology, cheap VPS backup)

<comment><USER17> Yeah there are no turnkey solutions that I'm aware of. Which is why every time there's a thread like this I come to see what the rumpus is.

I'm still waiting for a 'Drobo' like device without the proprietary physical layout. A light or dial goes into orange territory, you head to Best Buy or Amazon and buy the biggest drive that doesn't give you sticker shock, you push a button, out pops your worst drive and in goes the new one. Some lights flicker for a while and then go green.

I thought ZFS would have given us almost everything but the hardware ten years ago, but it turned out they oversold a few of the features back then, and then Larry happened.

Custom hardware is too expensive for small run consumer hardware, and Apple might have gotten into that space but never did. I wonder how many PCIe lanes you could shoehorn onto a Pi clone...

<comment><USER19> Synology hardware is pretty close to what you're describing. Light on front goes from green to orange and you get an email, you plug in another drive, click a button in the GUI, and wait for the volume to reshard/resilver/remirror.

A freenas box does this as well, but won't have the pretty drive light indicator if it's a home-built box, but then you're not limited to proprietary hardware.

<comment><USER17> I had been meaning to look into Synology more and I watched a few reviews after this exchange. Sound good except I'm not happy about having to link to their servers. But everything works that way these days :/

<comment><USER20> Why would you have to? Their devices work fine without any sort of account with them.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER21> So regarding the multi-writer problem, exactly how far away is this from implementing a central (bare) git repository? Seems to me that all you need is some kind of way to take care of authentication and possibly some way to make git's (mostly) append-only data-structure play nice with Hyperdrives'. I realise I'm probably missing some details, so my question is which details?

</comment><comment><USER22> Congratulations to the team !

2 questions:

\- What is the difference between dat and hyperdrive ?

\- I see hyperdrive manages hyper:// urls -- what about dat:// urls ? Will they be still managed ?

<comment><AUTHOR> Thanks!

\- "Hyperdrive" was formerly the internal data-structure name of Dat archives. With this release, the team decided to rename the protocol from Dat to Hypercore Protocol. Subsequently, "Dat Archives" are now "Hyperdrives." The Dat community will post some updates about this soon.

\- The hyperdrive-daemon does not support dat:// URLs. I don't know what the future of dat:// URLs will be but Beaker is phasing them out with a converter tool.

<comment><USER23> Another comment which is now gone asked what your relation to the Beaker project is, and how Hyperdrive is related to the Beaker browser?

<comment><AUTHOR> Just replied there -

Beaker uses Hyperdrive, it's basically the source of its novel features. The Beaker team works on Hyperdrive (the Hypercore Protocol) but we maintain a separate org at [https://github.com/hypercore- protocol](https://github.com/hypercore-protocol)

</comment></comment></comment></comment><comment><USER24> How soon until this can replace Dropbox for my little distributed working group consisting of 3 people wanting to share a single file system?

<comment><USER8> Our little team has been doing just this, both for dogfooding and because it works well for us.

Take a look at the "mounts" section of the blog post where we describe a group pattern you can set up. You can create a group directory called "team-drive", then mount each user's drive into the group.

If you're using FUSE, this will feel similar to Dropbox, but with one directory per-user.

We're starting with these kinds of simple mounts, and brainstorming ways to extend them soon.

</comment></comment><comment><USER25> Could this be used to create a kbfs (keybase file system) competitor?

<comment><USER10> Yea totally. The FUSE support is already very close to being elexactly that. We've been using it internally during our beta as an in office P2P dropbox using the mounts feature

<comment><USER22> In kbfs when I create a shared drive between you and me, the space is automatically available in read and write for both of us. Can this be done directly on top of hyperdrive or do we need to modify how hyperdrive (or the daemon I suppose) work ?

<comment><USER10> We can mount each others drives and collab that way.

For a full "union" mount experience we still have some research to do but we are working on it.

The mount setup is really good though and fully p2p

</comment></comment></comment></comment><comment><USER26> Interesting, thanks. Not clear to me when reading the announcement is: will hyperdrive replace dat? From reading the comments here, it seems that the answer is "yes".

Now, onto the content, you touch on de-duplication. I am quite concerned with the cost associated to updating a large file. Is something like rolling hashes investigated, to chunk files independently of their size? I guess it is, given you seem to be working hard on de-duplication.

But then, that kind of trick best works on uncompressed data, which is inefficient for transmission. Is data compressed before transmission? Whole chunks, or whole files at a time? Ahead of time? Interactively based on what the peer needs?

The trade-offs are many, and complex to investigate. I'm wondering if this could be used as an OS image, like OSTree does?

And lastly, I did not get if multiple peers having the same private key identity could modify the structure simultaneously. What would happen?

Also, nodejs gave me a kneejerk reaction that may be unwarranted, but that's quite a huge dependency to pull in for something that wants to be a ubiquitous building block. Does it have a C API? Also, twitter, discord, github (node to a lesser extent)... it seems somewhat ironic to build a ultimate decentralized filesystem while relying on these hypercentralized offerings, and I am afraid it could turn some contributors off.

<comment><USER9> I kinda share your kneejerk reaction about nodejs for foundational tools, even though I must admit that dat/hyperdrive works surprisingly well for a js tool.

But dat/hyperdrive is a well-documented protocol, and there are several ports to other languages.

I am very interested in the rust port, but I am not sure in what state it is: [https://datrs.yoshuawuyts.com/](https://datrs.yoshuawuyts.com/)

I have seen some tweets about progress being made on this. Does anybody know more?

<comment><USER10> The rust port of Hypercore has been very active recently and they are making good progress. Part of the latest Hypercore release was to move some of the transport crypto to be easier to port to other languages such as rust.

The wire protocol works now: [https://github.com/Frando/hypercore-protocol- rs](https://github.com/Frando/hypercore-protocol-rs) and the community is active in #datrs on freenode

</comment></comment></comment><comment><USER27> I really liked Dat (thanks, Knight Foundation). And I think it gets a ton of things right. However, I wish it had two things that we need for our purposes... can anyone chime in and say how AND IF they can be accomplished with the current Dat and hypercore?

1\. Consensus. If I submit conflicting updates and sign both, how does the swarm resolve what is the latest state?

2\. Migrating the swarm. If all the machines in a swarm get corrupted, can I migrate to a totally new swarm?

</comment><comment><USER28> What are the main differences between this and Syncthing?

<comment><AUTHOR> I'm not deeply familiar with Syncthing but here are some differences based on what I do know:

\- Hyperdrive includes mass-publishing as a usecase, so it uses public-key URLs and a bandwidth-sharing mechanism among its active peers (like BitTorrent)

\- Hyperdrive is built on a general-purpose protocol called Hypercore which is a signed append-only log. These logs can be used for other datastructures. Some examples [1] [2]

[1] Kappa-core, a general-purpose db built on the logs [https://github.com/kappa-db/kappa-core](https://github.com/kappa-db/kappa- core)

[2] Cabal, a chat network [https://cabal.chat/](https://cabal.chat/)

</comment></comment><comment><USER29> what version of node.js does it require? i get this with v10.19.0 from Ubuntu 20.04:

        node_modules/hyperdrive-daemon/node_modules/hyperdrive-daemon-client/bin/commands/create.js:8
          static usage = 'create [path]'
                   ^

        SyntaxError: Unexpected token =

<comment><USER8> Oops, give it a shot in v12. Think static class properties were added after 10.

<comment><USER29> v12 seams to work, v14 crashed. Would be nice to have that info in the README.

<comment><USER10> It should work on 14. Could you open an issue on the repository?

<comment><USER29> filed hyperdrive-daemon #47 problem with hyperdrive-daemon/node_modules/fuse- native/prebuilds/linux-x64/node.napi.node

<comment><USER10> Thanks, appreciate it

</comment></comment></comment></comment></comment></comment><comment><USER30> Great work. Planning to read about it. I, and I am sure others, are continually intrigued about this space and I hold out hope that the definitive solution appears soon (along with the solution to micropayments !!).

On a slightly related note, is anyone interested in having a discussion about how to layer on top of all these "drive" systems a `HDFS` like drive that would have n block replication across different sources, and trying to interpret any given source (dropbox, hyperdrive, google drive, etc..) blocks to make sense of what is being stored there would render the person confused?

:)

<comment><USER30> One more question... How effective would this be for a `network questionable` environment, if I were let's say doing a mobile app and wanted to incorporate this type of solution in for an offline first type of experience?

<comment><USER8> Hyperdrive (and the stack it's built on more generally) works great offline. All the drives you own are still editable without the network, and changes will sync when you come online.

As for 'network questionable' and mobile, here are a few of the other projects building on Hypercore that have made those a priority [0] [1] [2].

The Hyperdrive daemon as it's currently built wouldn't fare too well in a bandwidth and/or battery constrained environment (wasn't designed for that), but a mobile solution is definitely on our radar.

[0] [https://cabal.chat/](https://cabal.chat/)

[1] [https://www.digital-democracy.org/mapeo/](https://www.digital- democracy.org/mapeo/)

[2] [https://github.com/consento-org/mobile](https://github.com/consento- org/mobile)

</comment></comment></comment><comment><USER9> Congrats for the release. Mostly familiar with IPFS, but learned about DAT at dtn.is last year. The hypercore protocol is pretty well designed and documented.

How would hyperdrive deal with the following scenario: you got a large dataset such as wikipedia. Lots of people have browsed it, but most of them only have a tiny fraction.

How do yo know which peers to connect to to get a particular bit (offset?) you are interested in? The DHT only tells you which nodes participate in the hypercore, not what they have in detail, right?

<comment><USER10> The peers gossip using compressed bitfields in regards to what data they have. These bitfields are super small so we can pack quite a bit of information.

At the moment we don't do anything special in regards to discovery, but as we scale that's something we want to investigate. Since everything is running on append-only logs we can group the data into sections quite easily so there is some easy wins we can do there with announcing to the dht that you have data in a specific region.

<comment><USER9> So do you opportunistically gather info about what other peers have via this gossip protocol, or just when you need something?

I had looked at [https://datprotocol.github.io/how-dat- works/](https://datprotocol.github.io/how-dat-works/) , but I don't remember anything about a gossip protocol or a peer building a "view of the world". Is that new?

<comment><USER10> It's only between the peers in your subset of the swarm for now. They exchange a series of WANT and HAVE messages where they subscribe to the sections of each others logs they are interested in.

We are working on expanding this scheme so peers can help discover peers that have the section you are looking for.

Due to the compressed bitfields these section are quite large. In most cases using a few kilobytes you can share WANT/HAVE for millions of blocks

<comment><USER9> Yes, I saw the compressed bitfields. Most amount of bit twiddling I have ever seen in a pure javascript library...

Being able to identify a piece of content by an integer instead of a hash makes things more efficient compared to content-addressed storage a la IPFS.

</comment></comment></comment></comment></comment><comment><USER31> How do you handle merge conflicts? Since you aren't calling it a blockchain I take it that Hypercore is targeted at the trusted (writers) and semi-trusted (readers) use case?

<comment><USER10> Hypercore is a single writer append-only log. The website has a bit more info about how it works, but's basically a merkle log signed by a private key / public key scheme. We build collaborative data structures by combining multiple Hypercores.

Hyperdrives builds a p2p filesystem on top of Hypercore for a single writer. Using mounts you can mount other peoples drives so merge conflicts don't happen since there is no overlapping writes.

We are working on a union mount approach as well for overlapping drives (we talk a bit about this in the post)

<comment><USER31> So it sounds a bit like you've replicated git.

Do you agree that for the collaborative data structures side of things (like the chat app) users of the hypercore-protocol will likely run into clock trust problems?

PS I'm a big fan of your work/repos.

<comment><USER10> Yea you have to trust the original writers atm. I have some ideas for reducing this trust in the future through some consensus schemes but nothing fully baked yet. Def something I wanna hit tho, so we can get better security in something like a massively distributed chat system.

</comment></comment></comment></comment><comment><USER32> Would Hyperswarm be useful for setting up mesh-like Wireguard networks automatically? We need some solution for this ASAP - IPv6 not coming, CGNATs everywhere.

<comment><USER33> That'd be zerotier or tailscale

</comment></comment><comment><USER34> Sharing an append only log can be quite treacherous for users that are unaware of it. For example, when accidentally including some confidential stuff. A not so security aware person may think a quick delete fixes it and depending on the situation this might event be true, but its still in the log... Not familiar with Hyperdrive so please somebody correct me if I am wrong and this case is handled.

</comment><comment><USER35> This sounds good!

But I would not call it finished, as of today, apparently only one person can make changes to the filesystem. That does limit the use cases.

"In v10, we don't go all the way to a general multi-writer solution; solving multi-writer scalably, without incurring major performance penalties or confusing UX, remains a research question for us. "

.. so, give them some support, so they can solve this.

</comment><comment><USER36> Would it be possible to build a service that resembles email using this technology? Like dropping a message file within a local folder and making it appear on someone else's local folder?

<comment><USER8> @pfrazee threw together a demo app called PaulMail [1] that you can view in Beaker [0]. It's very simple and is just a proof-of-concept (messages are unencrypted, for example), but it does hint at possibilities there.

[0] [https://beakerbrowser.com](https://beakerbrowser.com)

[1] hyper://1bc1faf01a22270fb5698a60e63ef7a596ad976457e6d9914a8fd56d87281917/

<comment><AUTHOR> Your recipients also need to be in your beaker address book for it work. It's a very rough PoC.

</comment></comment></comment><comment><USER37> I've been following dat since a few years and I've been hearing about multiwriter since a few years but not anymore. Is it still something you are working on ?

<comment><USER10> Yes we are continuously exploring and research this. The mount support is our first stepping stone towards this. See the union mounts section of the post

</comment></comment><comment><USER38> I'm not able to view the link due to HTTPS errors.

The page you are trying to view cannot be shown because the authenticity of the received data could not be verified.

<comment><AUTHOR> That's odd, what browser?

</comment></comment><comment><USER39> Can it be that you moved from the early days of dat which was written in Go afaik to primarily node/js? Why the move?

<comment><AUTHOR> I think dat has always been node/js

<comment><USER40> They may be thinking of IPFS, which is written in Go.

</comment></comment></comment><comment><USER29> does it scale now? previously i was not able to import large folders (several TB) or many files (+100k files)

<comment><USER10> Yep! That is one of the things we've worked the hardest on. Completely new indexing structure, using an append-only hash trie which scales really well. We've tested it with many big datasets including importing all of Wikipedia as files in a single folder. Worked like a charm :)

<comment><USER29> this one? [https://dumps.wikimedia.org/other/static_html_dumps/current/...](https://dumps.wikimedia.org/other/static_html_dumps/current/en/) how long does it take to import it?

<comment><USER10> I think it was that one yes. Can't remember the exact time it took, as we ran it over a couple of days due to some unrelated computer issues.

</comment></comment></comment></comment><comment><USER33> Can you limit the synced size on clients (like an LRU cache) ?

</comment><comment><USER41> Would there be an advantage to implementing this using WebRTC as part of the networking, so you could use it in browser?

I don't know much about this space, but always been interested in some sort of web socket/WebRTC p2p fs.</comment>


Ask HN: Are there any drawbacks to using S3 for website assets? - <AUTHOR> <comment><USER2> [https://www.theverge.com/2017/3/2/14792442/amazon-s3-outage-...](https://www.theverge.com/2017/3/2/14792442/amazon-s3-outage- cause-typo-internet-server)

</comment><comment><USER3> It depends on the global reach of your website, but please remember S3 is not a CDN. If you host your files in the US, clients in Europe will see a significant latency, for example. If possible place a CDN in front of your bucket.

<comment><USER4> If you use CloudFront (AWS CDN) they don't charge for the bandwidth transfer between S3 to it which is good but here's the drawback, setting up permissions and configuring access in anything but the simplest cases can be quite frustrating and even involve some trial and error in development to test it in my experience. I've had to whitelist http headers in the past that weren't clearly documented as being required to be done.

</comment></comment><comment><USER5> What are you trying host? One drawback for websites is it makes deploys more complex as you've got to keep your S3 files and website files in sync. Maybe a website host only with a CDN in front would be enough.</comment>


Every Country's Top Tourist Attraction – According to Tripadvisor - <AUTHOR> https://www.vouchercloud.com/thomson-holidays-vouchers#tripadvisor-map <comment><USER2> Poland has many cities with "main market squares".

They probably meant this:

[https://en.wikipedia.org/wiki/Main_Square,_Krak%C3%B3w](https://en.wikipedia.org/wiki/Main_Square,_Krak%C3%B3w)

</comment><comment><USER3> Instant Bucket List.

<comment> qu4z-2 Something about the concept of a crowd-sourced bucket list is just depressing. But I guess it's an apt metaphor for the modern world, hah.</comment></comment>


37signals on taking outside investment and rewriting halfway through a new product - <AUTHOR> http://www.37signals.com/svn/posts/687-ask-37signals-any-big-risky-decisions

<comment><USER2> Amazon isn't well known for their culture. I remember couple months ago where I heard and read awful things from employees and ex-employees that works/worked there.

Picture this: they'll give you a pager so they can page you at 3 AM in the morning to fix bugs. The development culture is more like patch-n-release without thinking the long-term design.

No wonder Steve Yegge bolted to Google (and then praise Google for having "Clean Code Base")</comment>


Music recommendation by real people on Songza - <AUTHOR> http://mashable.com/2010/01/06/for-music-lovers-curated-setlists-for-free-online/

<comment><AUTHOR> I've just spent the last 5 hours use Songza's new 'curated' setlists and have to say I absolutely love them. I'm a huge Pandora fan, but this is a great source of not only similar music, but often culturally significant music. Short writeups for each song tell a tale about the people behind the song and the music itself. Really great stuff.</comment>