Tumblr Has Lost Almost a Quarter of Its Value Under Yahoo's Ownership - <AUTHOR> http://www.buzzfeed.com/williamalden/tumblr-has-lost-almost-a-quarter-of-its-value-under-yahoos-o <comment><USER2> This is a misleading, bait-y title. Yahoo took a write-down on Tumblr, but that doesn't mean it "lost value" post-acquisition. A far more likely reason for the write-down is that Tumblr was overvalued at acquisition. Paying too much for something is not the same thing as reducing its value.

The title leaves the impression that the way Yahoo has managed Tumblr has harmed the site. It may have, but the article presents no evidence to back that argument up.

<comment><USER3> What did you expect from a fair, balanced and accurate source like Buzzfeed?

<comment><USER2> Nothing, but this warrants a better title. My suggestion:

"Yahoo writes down 230 million dollars on Tumblr".

</comment></comment></comment><comment><USER4> >Yahoo, which bought Tumblr in 2013, said it had reduced its valuation of the blogging service by $230 million, or about 23%. The move was basically an acknowledgment that Yahoo overpaid in the $1.1 billion deal.

Either it lost value, or Yahoo overpaid. Both can't be true at the same time. Make up your mind, buzzfeed. Was Tumblr worth $1.1 billion and has now declined, or did Yahoo overpay?

<comment><USER5> If the "correct" valuation was halfway between the 2 valuations, then they could have overpaid at valuation, and still have declined in value.

Of course, value is relative and only based on what anyone will pay for it.

<comment><USER4> The title implies the entire decrease was after the purchase. That's inconsistent with the actual article.

</comment></comment></comment><comment><USER6> Yeah, and how much value has Twitter lost since the IPO? Or Snapchat since not selling to Facebook for $6B. Surely nobody believes these valuations were reasonable in the first place.

</comment><comment><USER7> This is a write down, they could change the value on the books to basically any number they want assuming they can get an accountant to say it's not crazy.

This has everything to do with Yahoo messing with its balance sheet and manufacturing an earnings number and very little to do with the value of an independent Tumblr.

<comment><USER8> Once again journalists writing about things they know nothing about and coming to conclusions they shouldn't be, such as standard wall st shenanigans/accounting games (not expecting much from buzzfeed). Another example: Every tech IPO that pops on day one is spun as a company in huge demand and doing so well when in reality it's just that the whole IPO was orchestrated in the first place for a huge first day pop so bankers could hand hefty returns to their most valuable clients.

</comment></comment><comment><USER9> Prepping for a sale or spinoff. Yahoo's hoping that by writing Tumblr down by $230 million they can slip the new valuation past auditors.

</comment><comment><USER10> When companies lose value (or are overvalued) why don't we ever highlight it when they use Agile / Scrum? Since those techniques are expressly marketed as value- and productivity-enhancing techniques, I think it's very fair to call out attention to cases when they aren't working.

Of course there are confounding factors. Maybe Agile/Scrum are working well in Tumblr but other things are keeping them from achieving their higher value. Or, maybe Agile/Scrum bleed away productivity and fail to extract value from engineers. Or maybe it's a mixture.

In either case it seems useful to point it out, ask questions about it, and so on. We shouldn't let Agile-adhering failures go un-called-out, whatever the post-mortem analyses might bring.

<comment><USER11> What methodology do you subscribe to?

Agile, when implemented and executed correctly, really just shortens delivery cycles and allow for small corrective changes along the development path. What it's best at is delivering a product that more closely aligns with needs, than something like Waterfall.

In many cases the overall delivery time might be longer with agile, but the correctness of the project is better. Which means less QA at the end of the cycle, and more throughout the entire process.

<comment><USER10> I've only experienced Agile lengthening delivery cycles, but more critically Agile also greatly reduced quality, because the lack of global context and long term planning caused the products to suffer. All of the micro-planning to make tinier-scale adjustments merely _sounds like_ it should incrementally improve quality. But when your short term sprints aren't informed by long-term planning, what happens is more like a random walk. Some sprints randomly add quality; some randomly reduce quality, and overall quality grows slowly, like sqrt(n) due to this random walk effect.

I also see a lot of this No True Scotsman fallacy of saying "when implemented correctly" that Agile, seemingly by definition, cannot fail.

The teams I've seen be most effective at delivering quality products quickly always just created their own methodologies, and they would scrap them and change them often particularly based on changing needs of the project, new personnel, and so forth.

Instead of the Agile approach of trying to say there is a one-size-fits-all / cookie-cutter solution (e.g. just use this timeframe for every sprint, just use this way of dividing up work for every project, just use this way of measuring progress for every team, ...), productive teams seem to negotiate this stuff organically, and with common sense.

Even teams that nominally used Agile but still managed to be productive did so precisely by only adhering to Agile as a ritual to appease managers, and outside of what managers wanted to see, the teams just eschewed Agile and found ways to be productive _in spite of_ it. That may not be true everywhere, but it informs my experience in three different Agile organizations of varying sizes and ages.

<comment><USER11> > saying "when implemented correctly" that Agile, seemingly by definition, > cannot fail

Poor choice of words, you are correct. And I actually agree with what you're saying in the rest of that response. Allowing scrum teams the ability to decide how to work together is huge, and I think correct as you point out some of the benefits of that. Rigid inflexible systems are horrible.

One thing that I know puts me generally in conflict with most agile purists is that a sprint should only have a time estimate component, and not a firm end date. Shoot for two weeks,but if it's three so be it.

<comment><USER10> Sometimes things need to be on a different order of magnitude than a few weeks. Sometimes a single sprint should be months, especially when the subtasks involved require novel research and pilots / prototypes (which is way more often, I find, than what typical "Agile thinkers" believe is true about business projects). You just cannot know what will be hard or what reasonable time estimates or how to divide up the work until you begin trying things for a significant amount of time and hitting the unanticipatable failure cases.

Agile is great in small, well-controlled cases where the path to a solution is easily seen by everyone to consist of commoditized, been-done-a-million-times- before tasks. But the issue is that once you use the Agile hammer, now everything looks like a we-must-solve-it-in-only-the-been-done-a-million- times-before-and-fits-into-sprints nail.

The other big thing is that so, so few Agile implementations are remotely like the Agile ideal that we should probably stop pretending there even is an Agile ideal. If everyone using a given tool consistently messes it up in the same ways, we should just stop trying to defend the tool and admit it's the tool's fault for being intrinsically easy to misuse/abuse, and seek to invent better tools that make it very hard to misuse/abuse them.

<comment><USER11> I don't think I can agree that things should be months long. You build in discovery at different points in the project, aka spikes. It should be possible to come up with tasks that can be accomplished in roughly two weeks.

If you break it down, you probably already do this: 1) get build for project going, 2) get initial config/entrypoint for system up, 3) implement stubs for API, etc... you can break it down, but each of those is a constrained set of work. It might be part of a very large project that can't "ship" at the end of two weeks, but you can verify different components along the way. It also is a helpful exercise to allow for multiple people to collaborate on different portions of the problem.

<comment><USER10> The trouble with this is that it assumes very particular kinds of projects. For example, I once worked on a user-facing mobile and web application that did forecasting and prediction to dynamically steer students through online college course materials.

The portions of the app that were very "customer facing" were exactly as you describe -- things like the user interface, APIs for educators who made use of the tool, etc.

But the statistical modeling and "science" code at the core of it was completely different. You could not propose a series of step-wise changes that could be pre-planned over short-term horizons. You might propose something and after 12 days of working you might discover it was _mathematically_ not going to work, regardless of engineering effort, and you could not have foreseen this (or at least our experienced team of stats people could not).

If you say something like "solve this hard cryptography problem" or "find a way to forecast X with Y% accuracy" or "determine an architecture that will boost performance by Z%" \-- these questions are hilariously ill-suited to someone just pre-specifying a set of steps for a 2-week period. There's nothing magical about 2-weeks that implies you can _always_ make meaningful progress on _every_ problem in that time frame. Sometimes you can't, and the extra constraints of trying to pretend like you can and structure work like you can act as significant hindrances to the real processes of discovery underneath.

I agree this is _somewhat_ rarer than the types of work that do fit neatly into 2-week chunks. But a lot of Agile evangelists want to politically recast the engineering efforts as if it's _always_ that way, and that _any_ request for more of a research approach to a problem must just be coming from whiny engineers who don't want to deal with real-world deadlines. And, predictably, this leads people to try to solve totally non-Agile-suited problems in rigid Agile-only ways, which ends up bad for everyone.

<comment><USER11> > You might propose something and after 12 days of working you might discover > it was mathematically not going to work

This is perfect though! In two weeks, you discovered the wrong thing. That's as valuable, IMO, as finding the right thing. The goal I find in my work is to try and fail fast... in other words, discover as quickly as possible whether or not a particular solution is feasible or not.

For truly complex issues, I have found that it often can take upwards of three failures before you discover the 'right' way to solve complex problems. You might have worked in an environment that didn't allow for failure and/or changing of dates. That definitely sucks.

<comment><USER10> > This is perfect though! In two weeks, you discovered the wrong thing.

Maybe I didn't describe it very well. 12 days was perhaps a poor choice because it is convenient to a sprint cycle, but in my comment it was meant to function as a random placeholder.

This type of work happens like Poisson shocks, or like lightning strikes. You work and work and it seems like you don't get a single thing done, zero story points completed, etc., and then boom, in one fell swoop you figure out a huge advancement in the problem. That's the defining aspect of the work I am talking about. The burndown graphs will look like (and _should_ look like) a flat line that suddenly drops to zero near the end.

It has nothing to do with "failing fast" because it's not "failing." It is the process of discovering what the actual problem is. This is very different than a situation where you, say, try out a MVC architecture but later realize it's not optimal for a business use case and need to change it to MVVM or something. You can _always_ make measured progress on the first attempt, and if it doesn't work, that would constitute valuable failure-case knowledge.

But with more abstract questions it's more like you are saying this: we tried to solve it, and we are not even sure whether or not we've made any progress yet, and we won't be sure until right at the moment we've either solved the problem completely or demonstrated why our whole approach cannot be used to solve it.

That's perhaps a very idealized version; most problems don't fall that far on the research spectrum. But the point is that many problems still do fall _pretty far_ on that spectrum, and if you set up your approach to them based on doing Step A then doing Step B then reporting about metric C, etc., it actually _hinders_ your ability to solve it, since the very nature of the problem is not amenable to that sort of thing.

<comment><USER11> At this point we should probably agree to disagree on some of these finer points.

But I do think that you are tending to blame agile for what really sounds like poor management. Management wants to see what progress is being made, so they use burndown graphs. Development, as you point out goes through spurts, except for the most basic tasks. So to management it can look like there is no progress being made.

What I'm trying to point out is that trying to divide the problems into as small of a set of isolated components as possible may help get to a solution. But that doesn't say your wrong. It might take months of time to find that solution, the problem is you need to explain to someone over that time what you're doing, otherwise the money and project will be cut. This is where it can be good to try things out and show what you've learned over that time, this is important for managing up.

<comment><USER10> I agree with this, except that I think in the presence of Agile what you are calling "poor management" is the norm, largely _because of_ Agile and the way it structures thought and planning. I think it's very important to make that distinction, so that people stop defending Agile as if it wasn't to blame. When a tool is politically subverted by management and results in this problem over and over almost everywhere it is used, it's time to just admit it's mostly _the tool 's fault_ for not having codified ways of avoiding that management subversion.

Basically, what I'm looking for in an engineering process is one that raises engineering concerns above manager and executive political concerns. In any organization you always have a struggle over who gets to use the phrase "business priority" \-- do engineers get to say that the engineering work is the priority, or do managers get to say that political circumstances are the priority? Agile makes it _easier_ for managers to promote political concerns _while at the same time acting and talking as if they promote engineering concerns more_. I'm perfectly happy being shown real business evidence about the cases when the engineering concerns are not primary -- but Agile offers ways for managers to _never_ provide such evidence, to keep people busy generating more and more progress metrics that give them more political surface area with which to manipulate.

I guess you could say this is just "poor management" but then I think we'd have to agree that 99.9%+ of management is "poor management" because this all repeats itself constantly in organizations of all ages, sizes, and stripes. And Agile is a hallmark indicator of such a dysfunctional engineering culture.

<comment><USER11> Politics... is there anything that prevents that other than working at a company that is too small for them to exist?

For me Agile is the least bad solution, it's definitely not ideal, and it's a million times better than Waterfall.

> agree that 99.9%+ of management is "poor management"

I'm not sure about the percentage, but good managers are rare.</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment>


You think it and a robot sees it: The future is here with mind-reading AI - <AUTHOR> https://thenextweb.com/artificial-intelligence/2018/01/10/you-think-it-and-a-robot-sees-it-the-future-is-here-with-mind-reading-ai/ <comment><AUTHOR> When I obscure the true image on the left side, it's very hard to predict the truth from the computer/brain imaging on the right. With one exception: the tech does a mighty fine job with geometric shapes.

(I wonder how it would do with playing cards.)

Oddly, with non-geometric images, I see a lot of faces (especially dog faces). Is the brain prioritising facial recognition? Is the subject a dog lover? Do the two recurring dark spots represent the eyes' blind spots?

I dunno, but I find these demos remarkable.</comment>


I Want Decentralized Version Control for Structured Data - <AUTHOR> https://jonas-schuermann.name/projects/dvcs-for-structured-data/blog/2020-03-22-manifesto.html <comment><USER2> > ...there are great decentralized version control systems like > Pijul/Git/Fossil and many more that check every requirement that I have, but > they are built to work with textual data and are therefore unsuited to be a > database backend of a graphical application.

> I basically want a DVCS that doesn't operate on text files, but on a proper > data model like relational algebra or algebraic datatypes.

Prolog + git.

Storing a text file of Prolog rules defining your DB is not as wild as it might sound. It's IMO much nicer to work with than SQL.

\- - - -

TerminusDB (in Prolog)

> TerminusDB started life as a quality control system and only later morphed > into a full database because we couldn't find a storage layer that was fast > enough to support quality controlled transactions at scale.

> Terminus uses the W3C's OWL language to define the schema of its databases. > This gives Terminus a uniquely powerful and expressive way of defining rules > for the shape and structure of the data that it stores.

> OWL is a very rich language based on first order logic and set theory.

[https://terminusdb.com/docs/schema](https://terminusdb.com/docs/schema)

\- - - -

CQL (in Java, I believe)

> The open-source Categorical Query Language (CQL) and integrated development > environment (IDE) performs data-related tasks — such as querying, combining, > migrating, and evolving databases — using category theory,

[https://www.categoricaldata.net/](https://www.categoricaldata.net/)

<comment><USER3> Similarly, if you are merely destructuring databases into text formats, most SQL database engines are pretty good about round-tripping their "dump" formats. For certain sizes of, for instance, SQLite databases, it is entirely possible to git store the SQL dumps and rebuild the database from them to query/work with them. (Whether it is reasonable to do it that way is another question.)

</comment></comment><comment><USER4> We needed to solve a similar problem - version control & synchronize .json files from different machines (annotations for ML models).

Writing a custom git merge driver was quite painless - a cmdline script (written in Python), which has task-specific logic on how to merge data from these .json files. Load these files, parse them, decide how to combine, detect unresolvable conflicts, etc.

It seems one may need custom logic to merge structured data, there is not a single best solution. This could make creation of a generic tool harder.

git is not a bad base technology for this. I'm not sure what other things are we missing (e.g. better diffs for structured data?), because .json is still text; it is just merges which are unreliable if you treat .json as text. There are also caveats - e.g. you can't install a custom merge driver on github, so "merge" button becomes dangerous. But overall for .json this approach works fine.

<comment><USER5> I wanted to write a git diff for files like KiCad or even Word. I didn't know custom git merges were a thing. Do you have a link for how to get started?

<comment><USER6> Both custom diff and merge drivers are described at a high-level in gitattributes(5)¹. They're pretty useful even in really basic ways such as adding a textconv with "jq -S ." or "xmllint --pretty 2" to pretty print JSON or XML before calculating diffs.

Plus, if you've already dipped in to those docs to see the diff options be sure to check the funcname attribute too. It allows you to add custom diff(1)-style `--show-function-line` options. For example, you can use an ugly regex such as `^\\\\[\\\\(.*\\\\)\\\\]$` to guess section names in .ini file diffs. Or the wordRegex option to make CSV files break on fields with `git diff --word-diff`. Or... well, thousands of other things. There are tonnes of things you can do to improve diff and merges for textual data in addition to the things you may want to do binary blobs.

1\. [https://git-scm.com/docs/gitattributes](https://git- scm.com/docs/gitattributes)

</comment></comment></comment><comment><USER7> [https://www.dolthub.com/](https://www.dolthub.com/) ?

Never used it myself, but it's one of the many "git for data" projects I've seen go past. I'm interested that none of them meet the authors need.

<comment><USER8> For being decentralized, dolt is really not good at getting the message across, I don't find anything in the documentation how I can get the collaboration/repository stuff to work without DoltHub, which seems to be their centralized service.

One of the neat things with Git and others is that it doesn't really care about what type of endpoint you're fetching data from, you can `git clone` from a unix directory, NFS shared directory, over ssh, via Keybase and so on. Dolt doesn't seem to support anything else than using DoltHub (if it does, they really need to update the documentation).

<comment><USER9> Where did you look? I clicked the "GitHub" link at the top, which is the standard way to bypass marketing fluff and get to an actual program. The README has instructions for installing from source, verifying the installation, configuring, creating an empty local repo, adding data to it, etc. It barely mentions DoltHub.

<comment><USER8> I think you misunderstood my message. I'm not asking how to install dolt, I'm asking where the whole "decentralized" feature-set is described and implemented in dolt, as I've scoured the docs, both marketing page and the GitHub repository, and the collaboration features seems to be locked to dolthub, which is very different from for example git.

So I create a local repo with dolt, and now I want someone else to collaborate on this dataset with me, without using dolthub. How I do this?

The article in the submission has "decentralized" as the first requirement, so I assume that dolt is mentioned here because it's somehow decentralized. But I cannot for the life of me find where that's mentioned

<comment><USER10> > So I create a local repo with dolt, and now I want someone else to > collaborate on this dataset with me, without using dolthub. How I do this?

The best I could find is utils/remotesrv in the repository [0], which seems to enable collaboration.. but also seems bit basic and it's not clear if https is supported.

[0] [https://github.com/liquidata- inc/dolt/tree/master/go/utils/r...](https://github.com/liquidata- inc/dolt/tree/master/go/utils/remotesrv)

<comment><USER8> Seems to confirm my suspicion that dolt is not made with decentralization in mind and their collaboration is mostly tied to dolthub and a specific protocol that currently just work over http. Thanks for digging that out!

</comment></comment></comment></comment></comment></comment><comment><USER11> I'm surprised nobody has mentioned datalad/git-annex. It might be relevant. It stores references to binary data in a git repository. And depending on the backend, like bup, it benefit from differential compression. And it is highly peer to peer, and has git-annex-sync[1], which will synchronize branches with its remotes.

Datalad is based on git-annex, and already being used for sharing large scientific datasets[2].

[1]: [https://git-annex.branchable.com/sync/](https://git- annex.branchable.com/sync/) [2]: [https://www.datalad.org/](https://www.datalad.org/)

<comment><USER12> I think git-annex is pretty much the right answer if you are already a linux saavy person that expects a mature battle tested tool that integrates well with your existing linux workflow systems. It sits on top of git in a sensible discernible way.

[https://www.datalad.org/for/git-users](https://www.datalad.org/for/git-users)

Peoples non text data is going to be some form of indiscernible binary blob to whatever system is managing it. Think about what a diff looks like for a jpeg, excel file, or an audio file.

[http://docs.datalad.org/en/latest/metadata.html](http://docs.datalad.org/en/latest/metadata.html)

What about git-annex vs git-lfs?

1\. """LFS Test Server is an example server that implements the Git LFS API. It is intended to be used for testing the Git LFS client and is not in a production ready state."""

2\. git-lfs seems to want to store your files in the github/microsoft cloud, its not really ready to be deployed inside an existing systems workflow? If you need to check a tickbox and just assume Atlassian/Microsoft has you covered it might be a good choice.

3\. git annex has a ton of integrations with multiple backends -

[http://www.chiark.greenend.org.uk/doc/git- annex/html/special...](http://www.chiark.greenend.org.uk/doc/git- annex/html/special_remotes.html)

4\. Poke around the git-annex website. I think you'll see a tool that's being used in a variety of workflows successfully for a decade plus. That's not true of really any of the other tools others have noted in this thread.

<comment><USER11> I've worked with both over the last few years. Git LFS is really not p2p like git-annex is. And it is less flexible. But maybe more stable, and git works with it out of the box.

You really have to pay for specialized LFS hosting, and you can't remove hosted LFS objects. And while it is possible to only download some LFS objects, it is not easy or intuitive. Once a repo gets to be beyond a certain size, it an be really impractical.

Git annex lets you fairly easily get only the objects you want. And it is strongly p2p. You can just set up another repo as a remote, and git-annex-copy whatever files you want to directly over ssh. Or, you can use Google Drive or DropBox or any number of other hosting services that git-annex knows, and stash your files there. Sibling repos will be updated about where they can find the objects. And it is really easy to push your binaries to as many backends as you want.

Finally, you can find massive multi terabyte repos of medical image data[1]. There's no way that LFS could handle these terabyte repos. It is really easy to fork one of these data repos, change it, put your changed files into your own backend, and then reshare it.

[1]: [http://datasets.datalad.org/](http://datasets.datalad.org/)

<comment><USER12> Also of note (from my understanding) is that git lfs local repo is double the size of the underlying data because it doesn't hardlink.

</comment></comment></comment></comment><comment><USER13> You may want to have a look at qri "query" \- [https://qri.io](https://qri.io) (disclosure, I work there).

Free & open-source tools for dataset versioning built on IPFS (the distributed web). Qri datasets contain commit histories, immutable hashes for each version (commit), schema info, metadata, readmes, & transform scripts, all of which ride together with the data (or, body).

Latest versions of our CLI tools support SQL & version diffing. We also have an electron app, Qri Desktop ([https://qri.io/desktop](https://qri.io/desktop))

<comment><USER2> PgUp PgDown Home and End keys don't work for me on your site. (I'm using Firefox 75.)

<comment><USER14> Thanks for letting us know. I opened an issue here which we will address as soon as possible: [https://github.com/qri- io/website/issues/202](https://github.com/qri-io/website/issues/202)

<comment><USER2> Cheers!

</comment></comment></comment></comment><comment><USER15> Is what they want just the same thing as a Conflict-free Replicated Data Type (CRDT)?

[https://en.wikipedia.org/wiki/Conflict- free_replicated_data_...](https://en.wikipedia.org/wiki/Conflict- free_replicated_data_type)

The only thing on their list of requirements that might not be covered is the last item, "collaborative". (If replication is a solved problem, it removes some obstacles to that, even though it doesn't give you collaboration for free.)

</comment><comment><USER16> [https://irmin.org/](https://irmin.org/) seems like it it may fit the bill:

> A distributed database built on the same principles as Git

I haven't used it myself, but it has been around for years and seems pretty mature.

</comment><comment><USER17> >A recent paper suggested a new mathematical point of view on version control. I first found out about it from pijul, a new version control system (VCS) that is loosely inspired by that paper.

[https://jneem.github.io/merging/](https://jneem.github.io/merging/)

<comment> dan-robertson I think this is quite relevant to the problem.

Whatever your data is, you need a way to merge it, but the format of allowable patches and the algorithms used must surely depend on the data structure.

Imagine first a set of strings. This may be merged more easily than the list of lines a dvcs usually deals with but you still get conflicts (I'm not sure what the merge of "add foo" and "add foo; remove foo" should be. It could reasonably be "add foo; remove foo" or a conflict of maybe adding foo)

Now imagine some kind of recursive structure (say a unix filesystem structure except the files are all empty). It seems hard to come up with a model for patches that will work well with things like removing directories or moving subtrees. Are these different patches/

move the contents of a directory too to another newly created directory and then delete the first directory

move a directory to a new place

How would these merge with creating a new file under the directory before things are moved?

For a set of relations, you want merging to preserve properties of primary and foreign keys. I'm not sure what merging that would look like.

These all sound like interesting hard problems. I'm not so convinced that their solution would be particularly useful.

<comment><USER18> > Imagine first a set of strings. This may be merged more easily than the list > of lines a dvcs usually deals with but you still get conflicts (I'm not sure > what the merge of "add foo" and "add foo; remove foo" should be. It could > reasonably be "add foo; remove foo" or a conflict of maybe adding foo)

My imagination is pretty confused by the application of a binary operator with only one argument. Would you please clarify? I think you'll find that when you explicitly think about the unstated arguments the problem makes sense, but I could be misunderstanding your point.

<comment> dan-robertson The point, as per the article mentioned in the parent comment, is to talk about merging patches rather than snapshots. This allows you to force certain nice properties of merges like associativity.

Sometimes you need information about deleted things in your patch to be able to merge correctly though I don't have an example from the top of my head

</comment></comment></comment></comment><comment><USER19> That's exactly what we have developed at [bohr.app]([https://bohr.app](https://bohr.app)).

As far as I can tell, our solution checks all the requirements mentioned in the post. We have a decentralized data storage system that supports delta syncing, sends/receives data over encrypted P2P communications between multiple devices/users without a "central" data storage, and leverages some concepts taken from blockchain technology to ensure data integrity and immutability.

</comment><comment><USER20> A few projects I'm involved with that might be worth sharing here:

\- Redwood ([https://github.com/brynbellomy/redwood](https://github.com/brynbellomy/redwood)), a realtime, p2p database. Data is structured in state trees that evolve over time. Merge algorithms are configurable at any point in a given state tree, but the default is a CRDT algorithm called "sync9".

\- Braid ([https://github.com/braid-work](https://github.com/braid-work) and [https://braid.news](https://braid.news)), a draft IETF spec that specifies certain extensions to HTTP that make it much easier to build systems like Redwood. The Braid spec is under active development on Github, and we welcome input from anyone interested in the idea.

\- Axon ([http://axon.science](http://axon.science) and [https://github.com/AxonNetwork](https://github.com/AxonNetwork)), marketed as a platform for making it easier to collaborate on scientific research, but under the hood, it's basically just some extensions to git that allow you push commits over a peer-to-peer network bound together by a DHT.

I would also highly recommend getting involved in the Internet Archive's DWeb meetups and conferences, where you'll find hundreds of people interested in solving exactly these kinds of problems.

</comment><comment><USER21> I went to a meetup on TerminusDB. It seemed like a cool project and quite mature. [https://terminusdb.com/](https://terminusdb.com/)

</comment><comment><USER22> I'm using "topic-based versioning"[0] and it solves most of the issues presented here. The principles are quite similar to what you would find in a database, with a central data repository and a write-ahead log.

Each "table" is organized in ordered, write-only topics (or folders) containing immutable messages (or files). Each operation is also adding a message into a special topic we use as log. Each participant have to remember the message id ("cursor") at which it was on the last sync and fetch new messages on the log first. Then, it's only a matter of applying the new ops until the desired point-in-time.

\- decentralized, private, efficient: you own a complete copy of all versioned datasets, and you can store deltas between versions since all messages are ordered in topics. Each sync is done like a "git pull --rebase".

\- reliable: If you follow the principles, conflicts are not possible.

\- collaborative: Here's the slightly difficult requirement. You can choose to defer message id ("cursor") allocation to another distributed service (like zookeeper, for example) and accept to be online to register new write operations on a topic ; or force the system to be linearizable.

[0]: I wrote an article describing this design pattern here : [https://medium.com/bcggamma/topic-based-versioning- architect...](https://medium.com/bcggamma/topic-based-versioning-architecture- for-scalable-ai-application-c926ffa92c1) ! Any feedback welcomed !

</comment><comment><USER23> Any immutable database? Datomic is one example.

On a related note: the idea of a database that loses data is perplexing, why would anyone want that? Why isn't data retention a default and limiting that retention an opt-in choice? When was the last time you rewrote your git repo to delete some old commits that you no longer need?

</comment><comment><USER24> A proper "DVCSD" would help a lot with software packaging, configuration packaging and deployment, immutable infrastructure.

It could allow fine-grained control and vetting of code and configuration.

(and overcome the security disasters called containers and "configuration management")

It's sad that few people understand this.

</comment><comment><USER25> I'd like to see a system like this that treats the type system (aka schema) for the data as just more data and puts _that_ under DVCS too.

Then I'd like to add a robust security/permissions model and build an OS around it.

<comment><USER26> You might be interested in this: [https://www.categoricaldata.net/](https://www.categoricaldata.net/). Categorical Query Language (CQL).

<comment><USER25> I'll take a look, thanks!

</comment></comment></comment><comment><USER27> Wasn't this IBM Lotus Notes?

<comment><USER28> why use the past tense? It is still being sold and used.

</comment></comment><comment><USER29> I'm a little concerned that the author is basing their approach on on Pijul given that its data loss issues are exactly what you wouldn't want for the "reliable" part of their priorities, but hopefully that's just an issue with Pijul's implementation of the ideas. It seems plausible that there aren't any major pitfalls in the theory itself instead of the code.

</comment><comment><USER30> It's super unhelpful to write a post like this while showing very little evidence that you Googled the thing you want to see if it exists already. What about [https://docs.dat.foundation/](https://docs.dat.foundation/) or everything else in these comments?

<comment><USER31> I like Dat, but right now it fails on the author's reliability criterion. And it's not clear[1] whether multi-writer is still a work in progress, which would mean a failure on "collaborative", too.

1\. Looks like it's still not finalized, but it might be—and the uncertainty about the answer is a failure in and of itself.

<comment><USER32> Dat multi-writer already exists, here's a demo of it: [https://dat-shopping- list.glitch.me/](https://dat-shopping-list.glitch.me/)

<comment><USER33> Unfortunately that demo was built on a protocol prototype which was deprecated.

Multiwriter is still being worked on, but it's not in the current release schedule. The upcoming release focuses on performance, scaling, reliability, and "mounts" (effectively symlinks across drives). Mounts can be used to create a kind of multiwriter, as the mounts stay in the control of the author, but we don't yet have "multiple authors of a shared folder."

In user-space, I've been able to create unioned folders like Plan9 did which is a _serviceable_ multiwriter scheme. A more sophisticated EC approach would use a vector clock in file metadata to track revisions but would need some approach to tombstones, which I don't have solution for yet. It's solvable, it'll just take time and performance tuning.

<comment><USER32> I see, thank you

</comment></comment></comment></comment></comment><comment><USER34> You can take a look at this talk: [https://www.youtube.com/watch?v=DEcwa68f-jY](https://www.youtube.com/watch?v=DEcwa68f-jY). It describes how to build a dapp with sqlite + CRDT

</comment><comment><USER35> What about Noms?

"The versioned, forkable, syncable database"

[https://github.com/attic-labs/noms](https://github.com/attic-labs/noms)

<comment><USER36> Unfortunately development has stalled out

[https://github.com/attic- labs/noms/blob/master/README.md#sta...](https://github.com/attic- labs/noms/blob/master/README.md#status)

> Nobody is working on this right now. You shouldn't rely on it unless you're > willing to take over development yourself.

<comment><USER37> FWIW, we (replicache.dev) have begun working on it again. We're part of the original team that built Noms.

Unclear what the roadmap for Noms will be yet, which is why we've not updated the README.

</comment></comment></comment><comment><USER18> I also want a widely adopted purely functional distributed data structure in the Okasaki style. Persistence is wonderful.

</comment><comment><USER38> Doesn't this map nicely onto functional data structures, where each new "version" is just a new top level reference?

<comment> dan-robertson The hard part is merging different changes.

</comment></comment><comment><USER39> How would one merge updates from all those offline clients?

</comment><comment><USER40> git-lfs+torrents (over SSH if you need authentication?)

</comment><comment><USER41> IDGI, just store the structured data in Git.</comment>


Numerous celebrity/company twitter accounts hacked - <AUTHOR> http://www.techcrunch.com/2009/01/05/either-fox-news-had-their-twitter-account-hacked-or-bill-oreilly-is-gay-or-both/

<comment><USER2> Look at the twitter feed: <http://twitter.com/foxnews>

Looks like the feed is updated automatically from RSS feeds or some other source. The latest tweet doesn't fit the template, so I'm guessing the "hacked" theory is correct.

</comment><comment><USER3> One does not necessarily invalidate the other.

</comment><comment><AUTHOR> looks like its a widespread problem...I guess there is some vulnerability being exploited

</comment><comment><USER4> Once you start to get big, hacking and spam surely follow... It'll be interesting to see what the story is here. Is every twitter account hackable? Are they going to start implementing some sort of spam filter? or a method to report spam?

<comment><USER5> You can report spam to the @spam user. That's only effective against users who are spamming, however. It doesn't really help in the situation where an account has been compromised. I strongly doubt "every twitter account is hackable"; someone probably just got hold of the passwords of the accounts in question.

EDIT: looks like more accounts are being compromised. Maybe someone _has_ found a hole in twitter's servers. Or, a disgruntled employee decided to have some "fun".

</comment></comment><comment><USER6> So does that mean I'm _not_ invited to Natalie Gulbis's birthday party (<http://twitter.com/natalie_gulbis/status/1072283758>)?

Bummer.

</comment><comment><USER7> In fairness, both could be true :)

</comment><comment><USER8> @barackobama was hacked as well.

</comment><comment><USER9> Probably a simple MITM attack. I think it's time twitter.com invested in a SSL certificate.

<comment><USER5> <https://twitter.com/> does exist and work. Is there a problem with their certificate?

EDIT: honest question, I'm not trying to be snarky.

<comment><USER9> I guess I meant to say that Twitter should default to using secure cookies. Once you login via twitter, it defaults to unsecured HTTP.

<comment><USER5> This is of course a fair point. I haven't seen the SSL version of the site encouraged or even documented anywhere. I suspect it would take down their current server infrastructure if all users suddenly switched to SSL access.</comment></comment></comment></comment>


Show HN: Running Sublime Text on a Raspberry Pi 2 - <AUTHOR> https://taoofmac.com/space/blog/2015/02/27/2210

<comment><USER2> That is way cool!</comment>


Eric Schmidt Testifies Before Congress - <AUTHOR> http://www.c-span.org/Events/Google-Exec-Testifies-Before-Congress/10737424248-1/ <comment><USER2> The central question seems to be the following, articulated most precisely by Mike Lee (R-UT): even if the organic search results are entirely unbiased, Google also programmatically (via "Google Universal Search", the system that organizes the overall search page) inserts specific links to other Google pages--such as Places, Maps, Shopping, or News--in very prominent positions. There are competitors for all of those sites (Yelp, Nextag), but Google systematically (it seems) prefers their own version.

This behavior is relevant to an antitrust principle that a firm with a monopoly in one area/market should not be allowed to use that monopoly to get inordinate market share in other markets.

The senators do not appear to have concluded that Google is doing anything illegal, though several have used the words to the effect of "I'm concerned with what is happening here."

My take: I agree with other commenters here that the senators have at several points seemed to have gaps in their understanding of Google vs. the average HN commenter, though I didn't hear any overtly wrong claims with respect to the central question. Maybe this analysis is better done by the FTC (there is an investigation in progress there), but it does not seem ridiculous that Congress, which is somewhat more transparent, is asking these particular questions, which do not seem incoherent or pointless.

<comment><USER3> Google is free for users and free to anyone wanting to get in their index, so they do not owe any self proclaimed competitor anything and that is the crux of it, the law is on their side, you cannot monopolize "free" and all this politicians' propaganda is waste of time and tax payer dollars..

<comment><USER4> You clearly have very little understanding of how antitrust laws work.

<comment><USER5> He shares the point of view of the person defending Google (she's a former anti-trust investigator?). If you can freely and instantly switch from Google to Bing then you cannot claim they have a monopoly. If you don't like Google's results there is nothing locking the consumer into Google except their own choice.

<comment><USER6> Not only can you switch instantly from Google to Bing, you can do the same with the indirect competitors such as yelp.

Yelp claims Google is hurting consumers because often Yelp's results are better than what Google displays as their own Location results above.

Is it really hurting a user to have to scroll slightly down? Or if Yelp's results were really better would users not go directly there or use their apps as an alternative?

I find it very interesting you can claim Google has a monopoly on the web + mobile, which seems to be the necessary premise to make these claims.

<comment><USER7> _Not only can you switch instantly from Google to Bing, you can do the same with the indirect competitors such as yelp._

That argument is quite compelling for search, where switching is easy and other search engines are going to give you fairly comparable results most of the time. But search isn't the only market Google is in. In the advertising market the "just switch" argument doesn't hold as much weight in my opinion: what good is switching to an advertising network that doesn't generate any views? If Google can dominate advertising and analytics due to their search monopoly, does any other advertising platform stand a chance?

Disclaimer: I've never been a Google customer, and I don't really know how much competition they have in the advertising market, so I don't know how much ability they have to jack up prices etc.

</comment></comment></comment></comment></comment></comment><comment><USER8> I think Google would be well served if Schmidt had a little more Steve Jobs in him. Not a lot, as I think Jobs might get so frustrated he'd melt down, but a little. Eric makes everything sound somehow slightly sinister, even when it isn't.

When asked about something like maps and places the correct answer is:

We realized that is what people wanted in response to their queries. We looked around, and the other map products sucked. Also, we couldn't use them in the way we wanted to serve the users, even if we had wanted to. So we spent a ton of money developing our own -- and it absolutely clobbered the competition. It was magical, far better than anything that existed before it. Way more information, way easier and faster to use. And the users have responded overwhelmingly positively to it.

If you're asking if we favor our own maps properties, damn right we do. Because it is better. Were the people that made other maps products happy about it? Of course not. We didn't build it for them. We built if for our users.

As a part of any online maps product, users want to use it to find information tied to the locations and businesses they are looking at. We call this places. We realized we needed to build something so that we could serve up this information in a highly integrated way, because we believed users wanted a unified experience. So we spent a ton of money, and we built out our maps product with Place information. Business hours, phone numbers, pictures, and reviews. We tried to partner on reviews. It didn't work out. So we built our own review system.

And it is brilliant, consumers love it. Maps and Places absolutely blow away anything else that exists in terms of ease of use and functionality and richness of information. It is one of our greatest feats. Again, consumers love it.

Were existing business directories and review sites happy about it? Of course not. We didn't build it for them, we built if for our users.

Maps, and Places, and Product Search -- they are all part of search. They are all part of finding information, and answers. And that is what we do, and we'll continue to push forward on that front. Because that is what consumers want.

<comment><USER9> Disagree. Eric Schmidt is being very careful and respectful, knowing that his quotes will be taken apart later, and it's hard when you have congressmen some of which with their own agendas who want to show their constituents that they are asking Google the tough questions and are not afraid to take action.

Whenever I see a person and think, "man, that would be really hard for me to do, and I am a CEO!" I think they are doing an excellent job.

I would say that a programmer's carefulness is serving him well here. Even though he's not one, _as far as I know_. :)

<comment><USER10> Eric Schmidt is no longer a programmer, but he was one of the authors of lex: <http://en.wikipedia.org/wiki/Lex_(software)>

</comment></comment></comment><comment><USER11> I hope the politicians don't confuse plain old search result rankings with smart searches.

<http://www.google.com/landing/searchtips/>

When I enter "1+1" into Google, I expect back "2", not a link to a calculator website. When I enter a street address, I want the first result to be a map, not a link to a mapping website. When I enter a stock symbol, I want the current stock price at the top of the results, not a link to a slow loading website.

Google has had these smart search results since day one and the functionality keeps expanding much to the displeasure of some companies. One could argue that this is an example of Google "favouring" its own services, but as a consumer, that's what I expect and that's why I keep using Google! I don't want links, I want answers. Bing likes to talk about being a "decision engines", but Google did it first.

</comment><comment><USER12> at D9: According to eric at D9, these are the top consumer companies 1-4-the gang of four(appl,google,amazon,facebook) 5-6(paypal,twitter) Schmidt said he believes Microsoft is "not driving the consumer revolution." He doesn't count Microsoft's Xbox business because it's "not a platform at the computational level."

today before congress: Microsoft's Bing has continued to gain in popularity, perhaps because it comes pre-installed as the search default on over 70 percent of new computers sold. Microsoft's Bing is the exclusive search provider for Yahoo! and Facebook. Microsoft recently signed a deal for Bing to power English language search on the fast-rising Chinese search engine Baidu, which Baidu has acknowledged will help it become more competitive in markets outside of China. In addition to Internet Explorer, Microsoft has integrated Bing into its popular gaming console, the Xbox 360, which it is in talks with cable companies to convert into the set-top box of the future. Microsoft?s Bing launched in June 2009 and has grown so rapidly that some commentators have speculated that it could overtake Google as early as 2012.

Read more: [http://news.cnet.com/8301-1023_3-20109615-93/schmidts- writte...](http://news.cnet.com/8301-1023_3-20109615-93/schmidts-written- testimony-before-congress/#ixzz1YcCQbU6L)

<comment><USER13> Indeed, but one is comparing the consumer tech market as a whole and the other is just in the search market, so this isn't apples to apples (so to speak), no?

<comment><USER14> Really? Microsoft does:

Bing -- second largest search engine

XBox -- one of the largest gaming platforms, including Indie games.

Windows -- most popular consumer OS

Halo -- one of the most popular game franchises ever

Kinect -- hottest selling consumer electronics gadget ever

Windows Live Messenger -- most popular IM platform in the world

Hotmail -- one of the largest email sites in the world

IE -- one of the most popular web browsers in the world

Metro -- one of the most interesting consumer UIs currently available

And all this puts them behind -- Twitter? Really?

<comment><USER15> I guess that's the whole problem, Microsoft wants to be everything to everybody.

Now just compare the same with Apple. They don't have 1/10th the product line as Microsoft and are still more successful.

Focus is important here, when you do so many things and none of them properly you are ultimately in a situation where all your products are weak quality wise. There are always better alternatives in the market and people will feel compelled to switch to them. Imagine this happening with most of your products. Ultimately the whole ecosystem gets weaker.

Even though your numbers may be looking good due to volume, that doesn't tell the whole story. Slowly your competitors one by one will drive you out of the market.

<comment><USER14> _Now just compare the same with Apple. They don't have 1/10th the product line as Microsoft and are still more successful._

The Apple comparison is a red herring. Apple is unique, not only to the computer industry, but virtually all industries. No other company has risen so quickly, from so low, on a few products.

To put it another way, if you take Apple out of the equation, MS appears favorable to just about every other company in the industry. From SAP, to HP, to Google, to Facebook, to Twitter, to EA, to DropBox, to Oracle.

</comment></comment></comment></comment></comment><comment><USER16> So.. who is the asshole, sitting behind (visible to the left of) the ex-FTC commissioner, that's constantly laughing, shaking his head, and rolling his eyes at what she's saying?

He is extremely irritating.

<comment><USER17> I noticed that, too. Not only did I find that annoying, I think it was disrespectful.

<comment><USER18> I wouldn't be surprised either. I'd like to know how do spectator get access to the room? Can anyone show up to a hearing like this?

Placing someone would be hard, involving knowing where the cams are and how to position yourself and getting there first.

</comment></comment></comment><comment><USER19> The lack of eloquence and understanding by those congresspeople is very, very disappointing.

Mr. Schmidt answered those questions in an excellent manner.

The racetrack analogy was particularly unnerving.

<comment><USER20> +1

Did anyone else notice that Al Franken was almost entirely incomprehensible? I'm happy that Schmidt has the presence and patience to pleasantly attempt to correct these scary misinterpretations.

</comment></comment><comment><USER21> The antitrust regulator just asked about sending in a third party to review google's search algorithm. Haha. Hahahahaa.

Not a chance.

<comment><USER22> And why not?

Google could put up a lot of resistance but if the government wants something bad enough they generally get it. The other option is not to do business in that particular country, and for google that option is not on the table.

<comment><USER23> Not sure if it was the GP's point, but it seems incredibly naive to think that the search algorithm itself is such that someone can just go in and "look" at it and gain any meaningful understanding. Even assuming Google is guilty of outright manipulation there's never going to be a smoking gun of "if(website == google) rank = rank +1, it is going to be an abstract machine learning algorithm with hundreds of inputs and dozens of outputs, all themselves the results of complex statistical properties gleaned from the web. There would be zero insight available to any mere mortal who might gain access to the code driving it.

</comment></comment></comment><comment><USER24> I'm going to get burned for this comment but ...

Microsoft has had a desktop monopoly for 20 years. They have 90% desktop market share and people don't complain about that. They destroyed their competition yet people complain because Google has 65% market share on a product that people can leave overnight. I'm not sure why people get bent out of shape over Google.

<comment><USER25> > Microsoft has had a desktop monopoly for 20 years. They have 90% desktop > market share and people don't complain about that

You don't count being fined billions of dollars over that monopoly, being forced by governments to make changes to Windows to help competitors, and being sued by several state governments and others as people complaining!?

</comment></comment><comment><USER20> We need more technically savvy congresspeople.

<comment><USER26> Maybe we should start electing websites as our representatives.

</comment></comment><comment><USER27> Last I checked it wasn't possible to switch the default search engine on Android phones (whether it was set to Google or otherwise). Is this still the case? I couldn't find anything conclusive in my searching.

<comment><USER28> I'm not sure what you mean. You can't try a new widget instead of the Google Search one? Or are you saying that widget should allow you to switch to Bing?

</comment></comment><comment><USER29> Seems to me that one way you can tell Google has competition is the facebook.com/cspan tag at the bottom of the screen.

<comment><USER23> Yes, I find it odd that Google gets so much attention while Facebook never rates a mention. I find Facebook way more creepy and actually much harder to switch away from - I can use bing any day of the week and the results are comparable. I can use G+ today and the results are useless. To my mind there has to be some recognition of whether a "monopoly" is due to exploitation of natural barriers to entry or not, and Google seems one of the least problematic in this regard.

</comment></comment><comment><USER30> They keep pounding the same misconceptions again and again in endless repetition, and I can't help but wonder, who hired these guns? (that's not to say that some of their arguments don't hold water, but they are going to incredible lengths to cast Google as a monopoly, why?)

The cynic in me doubts that the cause is a lack of understanding.

</comment><comment><USER31> Jeff Katz (NexTag) is slamming Google right now for copying his business.

<comment><USER31> Yelp is up now.

<comment><USER32> I can't wait until he starts talking about why Google is bad, I wonder if any one knows about Yelp's "pay us for advertising or you'll have your good reviews deleted" policy.

<comment><USER33> This was my first thought as well. I completely disagree with him that Yelp would not have been able to start up right now. Google would have helped drive traffic to yelp now just as it did then. Edit: I think I remember it being "join Yelp so you can delete the bad reviews".

<comment><USER32> I really thought he was going to talk about how he couldn't create Yelp now because of Google buying Zagat but it seemed like his entire point was "Google is evil."

I'm glad the person currently speaking who actually understands technology (seemingly) and worked on antitrust law suits knows what she's saying.

</comment></comment></comment></comment></comment><comment><USER34> Holy shit, they're are hammering down on Eric Schmidt accusing Google of tampering with search results and what not for pretty dumb reasons imho. Seems really unfair to me.

<comment><USER35> They expect him to be able to say "We know that we are not tampering with search results because of x, y, and z"--a strong statement--instead of "Search is a magic box that is imperfect. Shrug?" which is kinda what he said.

</comment></comment><comment><USER36> Something that kind of irked me about Jeremy Stoppelman's testimony: He failed to mention that Yelp provides an API and, in essence, gives away their data.

If Google was consuming their API (vs. scraping content directly from their site), would this even be an issue?

Let's assume, hypothetically, that Google used the API provided by Yelp to enhance their search results. If Yelp said, "We don't like what you're doing, so stop because we said so", I feel that it would totally be within Googles rights to say, "Fine, we no longer wish to do business with you and will exclude you from our index".

Of course, I'm not entirely sure what the Yelp API of 2010 looked like, nor have I read their TOS. However, I still feel that Stoppelman's testimony did more to remove my desire to ever use Yelp than it did to convince me that Google is this big evil monopoly that's destroying the little guy.

<comment><USER37> <http://www.yelp.com/developers/getting_started/api_terms> [http://www.yelp.com/developers/getting_started/display_requi...](http://www.yelp.com/developers/getting_started/display_requirements) <http://www.yelp.com/static?p=tos>

"5. Restrictions

You agree that you will not, and will not assist or enable others to:

...

B. modify the Yelp Content, or use it to update or create your own database of business listing information;"

"As between you and Yelp, you own Your Content. We own the Yelp Content, including but not limited to visual interfaces, interactive features, graphics, design, compilation, computer code, products, software, aggregate user review ratings, and all other elements and components of the Site excluding Your Content, User Content and Third Party Content. We also own the copyrights, trademarks, service marks, trade names, and other intellectual and proprietary rights throughout the world ("IP Rights") associated with the Yelp Content and the Site, which are protected by copyright, trade dress, patent, trademark laws and all other applicable intellectual and proprietary rights and laws. As such, you may not modify, reproduce, distribute, create derivative works or adaptations of, publicly display or in any way exploit any of the Yelp Content in whole or in part except as expressly authorized by us."

republishing of Yelp's content, except as allowed by its Terms of Use, Terms of Service, and Display Requirements, is against the terms of your agreement with Yelp, and illegal. even though someone grants you an API to remix their data, their intellectual property rights to that data are not waived.

</comment></comment><comment><USER38> Susan Creighton knows her shit.

<comment><USER31> She is smart, but she did skate around the question of scraping. They asked whether Google had scraped content, but instead she attempted to change the question to whether or not it is scraping when they show the excerpt about the site in results - not whether they had scraped Yelp reviews.

<comment><USER39> I think that's the crux of it though. Of course Google does scraping. They'll even give you a full cache of a page if you ask for it.

<comment><USER31> A cache is a bit different though I think - that is a reference. The Yelp case was them using content in their own competing product and then not representing the source clearly.

I deal with this as an affiliate all the time. I don't mind Google cacheing my web pages to increase our search ranking and help drive traffic to our site. However, if Google made a competing site about the same products and then took our reviews to use in that site without our permission, then I would have a major problem.

I'm not in favor of regulating Google, but I am in favor of some discipline and guidelines being laid down. The government clearly cannot decide how this should work, but I think Google is responsible enough to know that things need to be adjusted and just how much is needed.

<comment><USER38> Not representing it clearly? When the places pages included Yelp reviews it would put the top review with a link to "View more reviews on Yelp" and very clearly had a Yelp logo next to it. Same with TripAdvisor reviews. Now the places page is only showing Google reviews with "Reviews from around the web"

</comment></comment></comment></comment></comment><comment><USER40> Is there a recording available?

<comment><USER41> <http://www.c-spanvideo.org/program/301681-1>

</comment></comment><comment><USER42> For once Google are being honest and the others are being slimey. In one image they covered the sponsored ads in a box which hid the fact there is a clear differentiation between Sponsored and algorithmic results, they also say "Ads".

<comment><USER43> >For once Google are being honest and the others are being slimey.

For once? I hardly get the impression that Google is slimy and duplicitous, and their competition not so.

</comment></comment><comment><USER44> They still haven't open sourced Android 3.0, in which they said they would never release that part of Apache licensed code to the public.

So there could then be a possibility, again be hidden APIs so that their tablet app offerings could be better than others.

I personally think that, if you claim open source, some should be able to obtain the code if they are using the software itself. I clearly don't think a normal android tablet user would reasonably grab a copy of that source code.

<comment><USER45> Do you have a reason to believe the source code will not be released for Ice Cream Sandwich?

<comment><USER44> If ICS would run nicely on tablets, it would have some meaning.

</comment></comment></comment><comment><USER46> It sounds like both questions and answers are scripted.

Edit: that was my impression in the beginning of that show. After warming up the questions became deeper and more interesting.

</comment><comment><USER47> Watch the recording here: <http://www.c-spanvideo.org/program/OversightofG>

</comment><comment><USER48> Also available here: <http://sunlightfoundation.com/live/>

</comment><comment><USER49> > "Jeremy Stoppelman, CEO of Yelp and also a witness at today's hearing, said > otherwise. He said Google demanded that Yelp comply with Google demands to > be included in Google's search engine, including providing its content to > Google."

Is that quoted out of context? If not, that sounds worrying.

<comment><USER50> From Jeremy Stoppelman's written testimony, as posted on Yelp's blog (here: [http://officialblog.yelp.com/2011/09/seeking-a-level- playing...](http://officialblog.yelp.com/2011/09/seeking-a-level-playing- field.html)):

"In 2010, Google began incorporating the content that it indexed from its competitors into Google Local without permission. Although Google had previously acknowledged that it needed a license to use Yelp's content, it was now using it without permission to prop up its own, less effective product. In some instances, Google even presented this content to its users as if it were its own. [See Exhibit C]

Google's Offers a False Choice

In response to our objections, Google informed us that it would cease the practice only if we agreed to be removed from Google's web search index, thereby preventing Yelp from appearing anywhere in Google web search results. This, of course, was a false choice. Google's dominant position in the market prevents services like Yelp from exercising any sort of meaningful choice in the matter: it is a choice between allowing Google to co-opt one's content and not competing at all."

<comment><USER49> That sounds evil.

</comment></comment></comment><comment><USER51> Is this not similar to "Free-to-Air" TV stations promoting their own shows rather than competing stations shows.

Seems identical given the fact they both generate revenue from advertising, are "anti-competitive", and free (to the viewer of ads)

</comment><comment><USER52> Senator: "Ok Mr. Schmidth, tell me what I`m thinking."

</comment><comment><USER31> Simply put, IMHO what needs to happen is Google needs to be pushed to stop blatantly favoring it's own properties vs. others. At the same time, it cannot become so regulated as to become a utility.

<comment><USER31> Ms. Creighton is claiming that competition needs to flourish - one important aspect is that competition in an ecosystem you control falls under this statement.

</comment></comment><comment><USER53> Senator, you have so many hypothetical !

</comment><comment><USER3> For fuck's sake senator Blumenthal just asked if Google "Has ever scraped content?" !!

<comment><USER35> That had a very specific meaning in context. He was asking if Google used Yelp's content to build its Places content, in which case Google would be using its capacity as a Search company in order to grant its Places division a competitive (anti-competitive?) advantage against Yelp. If simultaneously it can be shown that Search was favoring bringing attention to Places over Yelp, there might be an anti-trust case.

There's a lot of posts on here about how uninformed the politicians seem to be. We seem to forget that _we're_ the ones who are uninformed of the vast technicality that is anti-trust law and the oceans of oratorical precision required in order to establish a legal case.

<comment><USER3> He asked both question, that was a separate one in general meaning. As for Google showing snippets of Yelp or anyone else that is still lawful, the whole search business is built of snippets and fair use.

<comment><USER35> again, you're using 'snippets' out of context and taking the letter of the politician's word at the expense of his intent.

</comment></comment></comment></comment><comment><USER3> These senators seemed ignorant about pretty much everything involved.

And as for Mr. Schmidt he should have started by saying that Google is free, it's free for users and for those wanting to be indexed by them so they don't owe any self proclaim competitors anything.

<comment><USER54> They actually just did. They argued that Google is free, and there's nothing to prevent users from using a different search engine. The argument seems to be that a large market share does not constitute monopoly power. In order for monopoly power to exist, there have to be barriers preventing users from switching search providers.

<comment><USER55> > In order for monopoly power to exist, there have to be barriers preventing > users from switching search providers.

That is incorrect. A "monopoly" is simply having a large enough control over the market that you dictate how others access that market. Monopolies are not illegal. Google does have a monopoly on search, Schmidt admitted that much today by saying they are "in the area" of having one.

Abusing your monopoly power is however illegal, and that's what Google is being investigated for.

<comment><USER23> > Abusing your monopoly power is however illega

However the principle that underwrites all of it is harm to the consumer. If you can't prove harm then it is hard to justify action. If consumers are choosing the monopoly provider willingly with zero compulsion by factors other than that it serves them well then you have a good argument that the monopoly is not harming the consumer.

The best argument would be that the purchasers of advertising are harmed because Google's ad prices are bolstered by their monopoly on search. However even there, Google uses a quite transparent and open bidding system for ads. They have no direct control over what people pay, they don't set their own prices - Google only shows ads at the prices people voluntarily bid. This doesn't put them in the clear but it makes it pretty hard to prove they are gouging or any of the other obvious signs of a monopoly abusing their power.

</comment></comment></comment></comment><comment><USER56> Eric Schmidt: I am not a cook.

So who cooks the algo Eric?</comment>


French ISPs Ordered to Block Sci-Hub and LibGen - <AUTHOR> https://torrentfreak.com/court-orders-french-isps-to-block-sci-hub-and-libgen-190331/ <comment><USER2> This was demanded by Elsevier and Springer, as the copy of the order shows [https://torrentfreak.com/images/scihuborder.pdf](https://torrentfreak.com/images/scihuborder.pdf) . Every researcher who still is an editor of one of their journals is responsible for this. Any academic manager who supports these journals, either by buying them or by counting impact factor points, is responsible of this. People should know that all of these people who support this are paid from their pockets.

<comment><USER3> Any chance of getting a SFW version of that link? I can't click on a torrentfreak.com link, even if it probably is benign.

<comment><USER4> What do you mean? TorrentFreak is a news site.

<comment><USER5> His employer's filters might be blocking it since it includes "torrent" in the url.

That's a pretty common rule in Schools and Universities, for example.

</comment></comment></comment></comment><comment><USER6> Those 2 libraries are the closest thing to the modern internet equivalent of the "Library of Alexandria". I hope they don't manage to burn in down.

Yes, for now governments only caught up to DNS level blocking, but more effective measures will follow if we don't change the political willingness to withhold knowledge from the masses.

<comment><USER7> > _Those 2 libraries are the closest thing to the modern internet equivalent > of the "Library of Alexandria". I hope they don't manage to burn in down._

This just reminds of how in the late nineties and early 2000s we _used to have_ torrent sites for music enthusiasts with an archive of all the different versions, editions, bootlegs, all painstakingly added and catalogued, representing tens if not hundreds of man-hours worth of work by people who simply _love music and wanted to share that love with other enthusiasts_ , creating some of the richest catalogues of music and music metadata the world has ever seen. And then it all was destroyed.

And this was true for many other topics of interest too.

We have burned many digital libraries in the last few decades, and we will probably continue to do so for a while longer.

<comment><USER8> Perhaps someone should painstakingly make a catalog of the internal web stacks of SV companies for people who love code.

I'm sure _that_ "library of Alexandria" would be burned quite quickly.

<comment><USER9> You're trying to make a snarky comment, but this is exactly what patents are _supposed_ to do...

Reveal how inventions and large companies work, as opposed to the old cloak- and-dagger method of 'secret formulas' and such. Then allowing anybody to see the method and licence it from the company, or be liable for a lawsuit.

What we used to have is if somebody somehow stole the secret to an invention it became fair game. Nowadays though companies try to game the system by having intentionally labrinthian patchworks of patents that do their best to not reveal anything about how the invention works while still being able to be used to sue people.

</comment></comment></comment></comment><comment><USER10> >The court order targets a total of 57 domain names, including various mirror sites. The academic publishers had asked the court for a more flexible blocklist, which they could update whenever new domains would become available, but this was denied. If the publishers want to expand the blocklist, they will have to go back to court. This ensures that there remains judicial oversight over local website blockades. Also, a request for a specific IP-address block was denied.

So a DNS level block... This should help to educate some French users on what a DNS is, and steer some of them away from using their ISP provided DNS servers.

<comment><USER11> Good luck trying to make Cloudflare (126.96.36.199), Google (22.214.171.124) or Quad9 (184.108.40.206) to make exceptions for a single geographical area. =)

<comment><USER12> If a court orders it, what are they going to do? Do you think Google would give up the EU market over Scihub?

<comment><USER13> You mispelt give up the entire internet. If every country dictates how every server in every other country is supposed to function then it will be pretty much impossible for anyone to operate.

Any filtering gets to happen inbound to that country unless you want to only view content that is legal in turkey, Israel, Pakistan,Uzbekistan, south africa, Russia, the US and _insert another 60 names here_.

We already have a way to handle different DNS results for different sub sections of the populace. They can run their own DNS servers and mandate that their users use them blocking alternatives if they so desire.

<comment><USER14> Returning different query results bases on the IP of the query sender is not magic. Couple that with location bases routing to thw nearest instance of a DNS server responding at 22.214.171.124 or 18.104.22.168 and you have nice and reasonably reliable segmentation based on the user's location. Nothing about that tech is magic and most of it is likely already implemented. So your doomsday scenario does not hold.

</comment></comment></comment></comment></comment><comment><USER15> Bought a book on GooglePlay once. Wanted to download it as an ePub. ePub wasn't available due to a technical error on Google's side (apparently affecting all epubs). Had to use Adobe Digital Editions to "download" it only to figure out the download wasn't available. + DRMs (i.e. can't read the ebook on a smartphone).

Went to libgen instead, will never buy a book on GooglePlay again.

<comment><USER16> How long did the ePub error last?

<comment><USER15> Seems like it is gone now (you still have to juggle with DRMs though). However when it happened I searched for the error code and found some posts on Google forums. They were 6 months to 1 year old, with no provided solution.

<comment><USER17> When did this happen? Maybe it affected limited number of users. I use google play books all the time and this was never an issue for me.

<comment><USER18> Here we are in the "age of information" and there is still no way to distinguish between the shill and/or troll and the genuine information provider. Glorious.

This age of electronics and logic and AI should have been able to provide a reasonable benefit to consumers at the very bottom. Instead, corporations have found new ways to rig things, impersonate people, and generally be dishonest about the quality and effectiveness of their offerings.

</comment></comment></comment></comment></comment><comment><USER19> And with this news, millions of French now become aware of these tempting sites they wouldn't otherwise even know about. (And Google methods of alternative access.) #streisandeffect

<comment><USER20> With a lot of proxies (in the linked document (french) [0]) I didn't know about; perfect!

[0]: [https://torrentfreak.com/images/scihuborder.pdf](https://torrentfreak.com/images/scihuborder.pdf)

<comment><USER21> Also it feels so old-school to see the same list 4 times in the juristic document.

<comment><USER20> Yeah.. which will be printed a couple dozen times (with I guess most of the copies ending up never being read?).

</comment></comment></comment></comment><comment><USER22> How difficult would it be to host one or more IPFS nodes with the files on Sci-Hub (and a directory thereof)? My understanding is that would be infeasible to block, and quite probably cheaper to host, as oft-requested papers would get mirrored.

<comment><USER23> If you have access to a cloud service/servers, you don't even need to do that. Your server is on the other side of these blocks, just rent a VM for a few minutes, RDP/VNC into it and surf the web without any ISP constraint.

<comment><USER22> Yeah, my concern is for if/when Sci-Hub's servers get taken down in their hosting country.

</comment></comment></comment><comment><USER24> Note that French universities use the public RENATER provider, which do not block, for now. More great technical details in a blog post by Rémy Grünblatt [1]. He use the RIPE Atlas network to probe.

[1] [https://translate.google.com/translate?sl=fr&tl=en&u=https%3...](https://translate.google.com/translate?sl=fr&tl=en&u=https%3A%2F%2Fremy.grunblatt.org%2Fla- censure-de-sci-hub-et-libgen-vue-depuis-le-reseau-ripe-atlas.html)

</comment><comment><USER25> Why do the French have such a hard-on for copyright enforcement? (Honest question)

<comment><USER26> I guess because it's the law, and governments are supposed to enforce the law.

<comment><USER27> I'm slightly baffled by the difference, though, as some other laws are not enforced. For instance, the case where directors of a tyre manufacturer were forcibly taken hostage and physically abused by the workers. Normally, you'd send in the police, arrest hostage-takers and give them long imprisonment.

<comment><USER28> You're implying that the hostage takers where not arrested or given imprisonment, but in the case of the Goodyear employees they were arrested and sentenced to 24 months imprisonment (15 months suspended).

That's what the victim party had asked for too, so I don't understand what problem there is with this judgment?

<comment><USER27> Thanks - I did not know that. The hostage-taking did make headlines internationally, any subsequent trial less so.

</comment></comment></comment></comment></comment><comment><USER29> Western countries are becoming increasingly like China and Russia. UK, Australia, Germany, Canada, France, Germany, New Zealand... So far, only the US still holds some resemblance to "protecting the freedom of speech" — sure, copyright violators are pursued, but at least nobody is trying to censor the Internet... yet.

<comment><USER30> the US? COPPA brought down a whole piece of the Internet. the US is no better than any other country.

<comment><USER31> > COPPA brought down a whole piece of the Internet.

You mean FOSTA/SESTA?

</comment></comment></comment><comment><USER32> I hope the Indian govt. doesn't follow suit. Libgen and Scihub serve a very great resource in a country that can't afford the absurd rates to go through the paywall; rare books that are not popular, and thus not part of the South- Asia-only reprints, are absurdly expensive.

Sadly, the current idiots in power think following the diktats of any and every passing foreign bureaucrat is an order from the Devas themselves. I don't think anyone who'll be coming in May will be any good either (if not worse).

<comment><USER33> Indian govt doesn't have to - Reliance Jio is proactively censoring all sorts of "problematic" sites without any government mandate. they took down reddit for some states already

</comment></comment><comment><USER34> More of a push to open access like this:

[https://gatesopenresearch.org/](https://gatesopenresearch.org/)

</comment><comment><USER35> FYI, for anyone not in France, this site automatically checks for the latest working Sci Hub links: [https://sci-hub.now.sh/](https://sci-hub.now.sh/)

</comment><comment> Funes- Access to knowledge should be open and free. Especially if it has been produced by an institution that receives public funding.

<comment><USER36> Note that not just Sci-Hub but Libgen was blocked as well. Libgen hosts academic works, sure, but it also hosts literature, including popular literature like everyone's favourite science-fiction and fantasy series, children's books, etc. So, the "access to knowledge should be open and free; the public might have paid for it anyway" isn't a suitable defense of it here. As a big fan of Libgen who has scanned and uploaded many books myself, I would rather take an overall position against copyright in general, regardless of whether the text matches what people think of as "knowledge" or not.

<comment> Funes- I was strictly referring to research papers and other academic publications; maybe I should have said "access to academic works" instead. Other than that, I could do away with Libgen's copyrighted literature, since I've never downloaded any of it. Also, I personally wouldn't be so fast to take a position against copyright in general, to be honest, but that discussion will have to take place somewhere else.

</comment></comment></comment><comment><USER37> Would be interesting to measure the negative impact on a French science. Not sure how much though.

<comment><USER38> I would expect there to be basically none. Most of the journals that people are after their university still pays for access for. Failing that, they email the author and get a copy directly. And if that isn't possible, there are a few people with wider ranging interests or wider ranging technical skills who know about sci-hub and libgen, and they will pass out the new domain name as necessary. Or even convince the department's IT to use a non-French DNS provider as the DHCP setting.

Sci-hub and LibGen are services that those of us outside a university or at something besides rich research universities depend on more.

<comment><USER39> Not true. Most people I know of or personally in academia in Germany and other countries rely on scihub. It's an incredibly valuable resource.

<comment><USER38> I'm a few years out of date, then, and I'm glad to hear that the situation has changed.

</comment></comment></comment></comment><comment><USER40> > Also, a request for a specific IP-address block was denied. The court sided > with the ISPs, who argued that they should have the freedom to choose their > own blocking method, including DNS blocking. That does mean, however, that > the ISPs will also have to bear the costs.

There are several issues that make blocking a website ineffective:

\- if you intercept and replace DNS responses for these sites then users with DNS-over-HTTPS or DNS-over-TCP will bypass the block. So let's promote using DoH/DoT and enabling it by default instead of relying on ISP's DNS servers.

\- if you examine the domain name in SSL Server Hello message then the site can switch to TLS 1.3 where it is encrypted.

\- if you resolve domain names and block access to those IPs automatically without human review then the owner of the blocked site can block any other site by adding its IP address to their DNS server. This can be funny, and we had such cases in Russia.

\- if you resolve domain names not very often, for example once every day, then the owner of the site can avoid blocking by regularly changing IP address.

Let me give a hint to French court: ordering the search engines and advertisement companies to stop dealing with the site is much more effective.

</comment><comment><USER41> SciHub and LibGen seems like very different things. They're both extremely useful. However, surely the ethical situation is very different? For SciHub I see little ethical problem: the authors of scientific papers never expect royalties and will almost universally welcome free access to their publications. No-one's going to shed tears on behalf of scientific publishing companies; they had their time, in the era of type-writers, but nowadays we can create our own PDFs, thanks very much.

LibGen on the other hand I'm fascinated by. Most people I meet have absolutely no idea that a vast proportion of all commercially-produced (more or less technical) books are available for convenient free download. And while I would love for that human knowledge to be freely available to everyone on the planet, we do have to concede that many of those books were written by authors on the understanding that it was a commercial project and they would receive income for it.

Do people generally agree with the above distinction? I feel that this thread (and perhaps the French government / courts) should be making it more clearly.

</comment><comment><USER42> As Western countries are all slowly drifting towards what amounts to government-sanctionned censorship by ISPs - especially after the adoption by the EU Parliement of the copyright directive, but it's just one more step in the general movement towards controlling who can publish what on the web -, we need more initiatives that bypass those ISPs, which are currently in the Western world the only ones capable of enforcing censorship.

I'm no expert on this and the ways of achieving it, but as a citizen I feel more and more concerned about this - in particular as my (French) government seems eager to move in the not-right direction.

<comment><USER43> I wonder how much time will pass before using a debugger becomes illegal unless you have a license.

</comment></comment><comment><USER44> Didn't know these sites. Thanks to the French ISP for helping to spread the word.

</comment><comment><USER45> I don't understand why many people here seems to agree blocking is a bad thing. If some content is illegal in the country, isn't it normal that the state is enforcing the law? I'm mean that's one of the things it is supposed to do I believe

On the other hand, I understand disagreements with the law exist, but that's a normal thing and we (in France) are still able to vote for some parties with different point of view on copyright, so I'm not sure why some people are speaking of "authoritarian" or "censorship"

</comment><comment><USER46> Scientists etc are probably smart enough to find ways around this (or already have access). For example, there was a Twitter convention to request journal articles, and researchers with free access would retrieve them and send them via email as PDFs. Of course, I am not suggesting you do this if it contravenes the law.

On a brighter note: cosmic rays are coming at you; and Elsevier and Springer are making bank.

</comment><comment><USER47> wonder why SciHub was not already moved to distributed hosting system like [https://ipfs.io/](https://ipfs.io/) or [https://zeronet.io/](https://zeronet.io/)

<comment><USER48> Because of the very large amount of data.

<comment><USER47> could be solved by splitting into several sub sites by topic and further thematic distinctions.

zer0nets youtube pedant solves this by creating a site for each "channel" und just aggregating them

</comment></comment></comment><comment><USER49> Great way to revive the storage industry.. am now buying 400TB of space to make mirrors...

</comment><comment><USER50> Does this mean other countries in the EU and at international level could follow suit?

<comment><USER20> The linked document (in french) [0] says that there have been similar requests in the USA, Italy, Germany and UK.

[0]: [https://torrentfreak.com/images/scihuborder.pdf](https://torrentfreak.com/images/scihuborder.pdf)

</comment></comment><comment><USER51> I'd really recommend the Sci-Hub Button script. It places buttons for the open paper on Sci-Hub just about anywhere you encounter a science paywall:

[https://greasyfork.org/en/scripts/370246-sci-hub- button](https://greasyfork.org/en/scripts/370246-sci-hub-button)

</comment><comment><USER52> Streisand effect. Now everyone will know about those two websites, get curious and visit them from behind a VPN.

</comment><comment><USER35> Would using Tor bypass this block? Or changing DNS servers to Google DNS?

<comment><USER53> Yes, both would work. Other countries have started with DNS blocks and eventually moved to IP blocks, which would prevent Google DNS from bypassing the block. For now though, changing your DNS will be sufficient.

I'm not aware of any block that is effective against TOR. Even if that were possible, I bet sci-hub has a .onion mirror, which is by definition not blockable.

</comment></comment><comment><USER54> How do we get widespread adoption of a uncensorable DNS alternative?

</comment><comment><USER55> So, after pirate sites we will have to see pirate ISPs too.

</comment><comment><USER56> proxy to afghanistan for free books <3 yalla yalla

</comment><comment><USER57> These paywall bypassers are revealing a problem and offering a temporary solution. However this situation wherein scientists are both providers and consumers of editors, and forced to go to pirate websites in order to work is not sustainable. I feel like a permanent solution needs to be found.

I'm not a scientist myself, and would be happy to hear anyone else's opinion, but alternatives like peercommunityin.org look more promising.

All in all, this ban can be easily be circumvented (it's just a DNS block), so I'm not really sure it will help editors. It may cause an opposite effect, helping people to move to different solutions. But maybe I'm over optimistic.

<comment><USER58> Or alternative like [https://gatesopenresearch.org](https://gatesopenresearch.org) maybe ?

</comment></comment><comment><USER59> OH man. This is not good.

</comment><comment><USER60> Is elsevier French ?

<comment><USER61> It is Dutch.

</comment></comment><comment><USER54> Why do researchers publish with these companies that lock their papers behind paywalls? Do the researchers get a part of the paywall profits? Are there not any alternative publishers that have free academic journals or release papers for free?

</comment><comment><USER62> first they came for the torrent listing sites, then they came for the open research databases, and then it was too late.

</comment><comment><USER63> I wonder what exactly they will be blocking?

$ nslookup [http://sci-hub.tw/](http://sci-hub.tw/)

Server: 127.0.0.53

Address: 127.0.0.53#53

Non-authoritative answer:

Name: [http://sci-hub.tw/](http://sci-hub.tw/)

Address: 188.8.131.52

It is trivial for the user to appoint another DNS server, or even to add the site to his /etc/hosts file manually, if they try to block at that level. It is also trivial to tunnel over another ip address to reach the one wanted. How is "blocking" supposed to work?

In the court document, they asked for an injunction against a list of domain names. Therefore, I assume the "block" will be limited to blocking them at the DNS level.

"Toutes mesures propres à empêcher l'accès" ("All appropriate measures to prevent access"), means everything and nothing, especially, when it is most likely impossible to do. What they really want is to block particular content. In my opinion, it is not possible for an ISP to support that.

</comment><comment><USER64> How long would it take the cryptocurrency miners to use all their hashing power to find the deterministic key that generates the content of libgen?

That might be an easier solution than trying to achieve minimally sane copyright reform.

<comment><USER65> It doesn't work like that. Hashes aren't compressed versions of the hashed content.

<comment><USER66> I think theyre suggesting hashing all of libgen and distributing and working backwards from that hash, as a joke

<comment><USER67> It's not totally ridiculous, just wildly impractical. There _is_ a hash out there that if brute-forced would yield a Windows 10 version of Half-Life 3.

<comment><USER68> There are an infinite amount of things that hash to a given hash.

For something of the size of half life 3 (about 2GB? no idea), the number of collissions would be far more than the number of atoms in the universe.

<comment><USER16> To provide a much tighter bound, you reach "atoms in the universe" levels of collision once your hash inputs are 260-270 _bits_ bigger than the hash output. No need for even a single kilobyte.

Almost a "Graham's number" level of understatement.</comment></comment></comment></comment></comment></comment>


Printed Circuit Board (pcb): All You Need to Know - <AUTHOR> http://dev.emcelettronica.com/printed-circuit-board-pcb-all-you-want-know

<comment><AUTHOR> A PCB (printed circuit board) has more functions than just being a power facilitator as it provides mechanical support for the many components that are attached to it; it is the backbone of the electronic device, conforming to overall shape and dimensions, provides for signal fan out opportunities, etc.</comment>


The Rise of the Data Scientist - <AUTHOR> http://flowingdata.com/2009/06/04/rise-of-the-data-scientist/

<comment><USER2> Recommendations for becoming a data scientist? Sounds like a wonderful job.

What books would be on the required reading lists? What is the best set of tools to learn and know?

An alternative approach: how much of this could be automated and delivered as a web application/service? Provide easy ways for people to parse and upload their data, algorithms to run with good descriptions and recommendations for lay people, and well designed visualization tools for presenting the output of the data. This would be a very valuable service, but would take a broad range of knowledge and expertise to implement well (experts in the CS, Statistics, and Visualization fields as described in the article).

<comment><USER3> As for your latter suggestion - I think its unrealistic to build a general web-based solution. As you are describing it, you would essentially be hosting R, some visualization tool, and a scripting tool. That's absurd enough and doesn't even address the bandwidth issue - these datasets are usually huge.

You'd be better off writing enterprise software in this field. Especially considering anyone who would pay for data munging and visualization tools/services probably doesn't want their data out on the internet.

<comment><USER4> Hosting R is already implemented - see <http://predict.i2pi.com/>

<comment><USER2> Yes, this looks a lot like what I had in mind. Reading his blog entry, I like that he is getting something out there, even though it is not all of what he originally had in mind. If he keeps iterating, he should do well. I wonder if he has any thoughts on business models.

</comment></comment></comment></comment><comment><USER5> This link has tons of data visualization examples and resources

[http://www.meryl.net/2008/01/175-data-and-information- visual...](http://www.meryl.net/2008/01/175-data-and-information- visualization-examples-and-resources/)</comment>