Tagsistant: semantic filesystem for Linux - <AUTHOR> http://www.tagsistant.net/ <comment><USER2> Hierarchical file systems lack expressiveness and are awkward in places. In my day-to-day computing this has become more apparent and problematic with each passing year. For example, I dislike that you are forced to give unique names for each file, that you can classify each file in only one way, and that you can't tell how many copies you own of a piece of data.

The smallest step up from a hierarchical file system is to allow any file to have any number of tags, where each tag is just a simple string (no hidden IDs or anything). I believe most tag proposals implement this concept and not much further. I briefly looked at Tagsistant and numerous software and papers.

I sketched some of my own ideas about identifying immutable files by hash and creating arbitrary tags that reference such files. It turns out that this way of organizing files goes really deep, and I haven't explored all the implications yet. It yields a completely different landscape than the file system that we are used to today - the concepts of path, mutability, attributes, etc. are replaced with different mechanisms.

The article is long, but I would appreciate hearing if the concepts resonate with anybody else: [https://www.nayuki.io/page/designing-better-file- organizatio...](https://www.nayuki.io/page/designing-better-file-organization- around-tags-not-hierarchies#contents)

<comment><USER3> I remember for a while Google Drive worked on a tags metaphor and it was just too much friction for users. The UI for the traditional file tree is just plain cleaner. Trees provide clear delineations of ownership and categorization. They suffer from the limits of hierarchies, but you can break them down and get a nice tree-view of them, for example.

Having spent time categorizing my photos, adding a tag layer on top of the existing filesystem is a great idea, but using it as a _replacement_ for the traditional directory structure isn't.

Simple operations like "delete everything in here" becomes complicated under a tag structure now that we no longer have a strict concept of A is within B.

<comment><USER4> As long as the tags themselves can be hierarchical, can't you recover all the benefits of a tree-like file structure?

<comment><USER5> This. I believe a hierarchy of tags is a great solution.

<comment><USER3> But then don't you have the same probelm of the limits of taxonomies and you start thinking "well is this tag a member of A or B? It should be a member of both" and then you need to tag the tags. Turtles all the way down.

<comment><USER4> That's the advantage of tags vs true directories. A tag could have mutiple parent tags.

Of course, that raises the question of what happens when you encounter a cycle in the tag graph.

</comment></comment></comment></comment></comment></comment><comment><USER6> Great project!

I feel that the problem of archiving files is not well served by the POSIX file system, and deserves attention. Gaps are in data safety and backup capabilities (dropbox is a huge leap forward here) and document retrival. Usually external idices (which go out of sync) are used to query file names. Also there is no way to attach semantic metadata to files (appart from date stamps, and permissions).

This tool provides an interesting stab at the latter problem. I have always thought that semantics would be layered ontop of a POSIX file system. This project fips the logic and implements tagging within the fs.

I wonder how compatible this is with NFS/Dropbox/git. Can I use this to tag files on a Mac via Dropbox Sync? _digging..._

I have been using some home grown tools that allow me to put sematics into filenames for a while now. And it has served me quite well. Files look like this:

      2017-04-04 #S4907 #Choir List of names.pdf
      2017-04-05 #EXCITE #S5005 Notes on Data Repositories.pdf
      2017-04-06 #ARAG #S5031 $Amount=14.2EUR Invoice.pdf

And I have command line + CGI scripts that allow me to manage and query folders which contain properly formatted files. I just begun writing a second version in python:

[https://github.com/HeinrichHartmann/pile](https://github.com/HeinrichHartmann/pile)

(waaay to early to use, yet. But the README elaborates a little). Has anyone aware of similar approaches?

E.g. using JSON documents as file names seems another obvious way to layer semantics ontop of POSIX/fs. Is anyone doing this?

EDIT: Formatting fixes.

<comment><USER7> Thanks for sharing your system!

I'd like to give you some HN formatting tips, if I may:

Add two new lines for a new line. Otherwise the formatter will put your text on the same line (eg. with your list of files, it looks like one big filename now)

 _italics_ work by putting asterisks (*) around the word or phrase. Though maybe you meant to use underscores in this case :)

Best wishes!

<comment><USER6> Thx. Fixed now.

</comment></comment></comment><comment><USER8> Nice! An equivalent for comparison: [https://tmsu.org/](https://tmsu.org/)

<comment><USER9> This should be higher up, I think TMSU is the best known product in this space, and a comparison between the two would be useful.

For one thing I think TMSU's achilles heel is handling renaming of files gracefully, I wonder how Tagsistant does it?

</comment></comment><comment><USER10> On Github: [https://github.com/StrumentiResistenti/Tagsistant](https://github.com/StrumentiResistenti/Tagsistant)

<comment><USER11> And if you're having trouble accessing the page (like I am), Google's cache can help: [http://webcache.googleusercontent.com/search?q=cache:http://...](http://webcache.googleusercontent.com/search?q=cache:http://www.tagsistant.net/&num=1&safe=active&strip=1&vwsrc=0)

Usually I prefer to link archive.is, but it didn't manage to capture the page before it went down this time around.

</comment></comment><comment><USER12> I used this years ago. I had a few scripts that would copy files to the appropriate tags, but it ended up being super slow. Eventually I wrote my own tools around the sqlite database it created (it's a pretty simple schema) but I've since lost most of those and ended up rewriting a bunch in a system I've been working on.

This was all back for the 0.2 release and I think the author changed a bunch of this stuff in later releases. I wish there were more options for tagging file systems in Linux. If I ever get my stuff out of the "thrown together" phase, I'll probably publish them.

<comment><USER13> See also: [https://news.ycombinator.com/item?id=14538060](https://news.ycombinator.com/item?id=14538060)

</comment></comment><comment><USER14> I've been thinking increasingly of something along these lines myself, with Tagsistant being among the systems which have come up in my own research.

The problems, variously, are that fixed-name hierarchical-storage filesystems meet the needs of document-based storage, projects, workflows, sharing, and lifecycle exceedingly poorly.

The problem is coming up with a better option.

A filesystem-based approach has the advantage that it's low-level, _not_ tied in to a single application or toolsuite, and may be extensible.

Among the questions I've turned up include identifying what specific problems this is trying to solve, distinguishing between _public_ and _private_ information, and what levels of standardisation might apply. There are also some very significant questions about privacy and data leakage.

My current thoughts are largely grouped around a documents-based system (provisionally, "/docfs"), and an online or Web-oriented system (provisionally, "/webfs"), both under an umbrella context system, KFC (for KFC's Fine Context). Mostly considering the domain space, workflows, and possible solution-shaped objects. Part of that (largely focused on Web access) discussed here:

[https://www.reddit.com/r/dredmorbius/comments/6bgowu/what_if...](https://www.reddit.com/r/dredmorbius/comments/6bgowu/what_if_the_web_was_filesystemaccessible/)

</comment><comment><USER15> Humans have to tag the files, though. This is the same problem which kept the "semantic web" from going anywhere.

<comment><USER16> _Humans have to tag the files, though._

Only once, if done properly. Look at music tagging. A good system ought to have canonical tags for everything.

User-created files could also have a lot of auto-generated tags too. I'm thinking along the lines of email address/URL origin, Exif metadata, source code tags (ctags/etags), keyword extraction from prose text (via machine learning models)...

Beyond all that, though, would be your standard date and timestamps, your document name (which could be non-unique), and a project-based tagging scheme.

Take a look at the Library of Congress's MARC project [0][1]. It's the most ambitious tagging project I'm aware of.

[0] [https://www.loc.gov/marc/](https://www.loc.gov/marc/)

[1] [https://en.wikipedia.org/wiki/MARC_standards](https://en.wikipedia.org/wiki/MARC_standards)

<comment><USER2> Could you clarify what you mean by "canonical tags for everything"? Do you mean an online database like freedb, MusicBrainz, and such?

<comment><USER16> _Do you mean an online database like freedb, MusicBrainz, and such?_

Yes, as well as the Library of Congress and doi.org.

</comment></comment></comment></comment><comment><USER17> I often wanted a tag system, but this is not what I was picturing, I think. I'll rather like to have a database and an entry in the context menue to tag files without moving them. It should also be able to identify the file if is being moved or renamed.

<comment><USER18> It must be possible to get the same effect without copying the files by using links. If it is all on the same file system then hardlinks would work and be very space efficient.

<comment><USER19> Extended attributes can do this for most unixy operating systems. See fsetattr and fgetattr. It's a bit tricky in that the standard tools, like find, don't support them directly. But, the tags stay with the files, no separate DB needed. You do have to pay attention to copying, backups, etc, to make sure they are preserved, and no separate DB means all "queries" are full scans.

</comment></comment></comment><comment><USER20> I've never been able to figure out what people mean by "tags" and what the benefits are, perhaps someone can enlighten me?

I feel like the only problem with standard hierarchical file systems is that sometimes you want files to show up in multiple places. I think this is only a subset of files, typically "media." A photograph or a song often has multiple categorizations. However, a lot of things, like my tax documents, source code, Word documents, and notes typically only have one place they need to go to. It seems like symlinks or hard links are an already-existing solution; other than lack of great options on managing them, how do tags improve this?

As far as I can tell, tagsistant does have hierarchical tags, so it isn't doing away with hierarchy, which is good. One problem with some tag systems (like Gmail's original) is that under "notes" I have "lectures," "stuff I want to remember," "things like list of books I want to read," "temporary." If I'm looking for photos, I really don't want to see a bunch of notes tags.

<comment><USER21> I think tags only prove beneficial when you have a very well defined purpose. More concrete, I think the number of tags has to be fairly limited and the logic behind the tagging scheme should stay consistent forever.

One example of tags being superior to links is that you can search files with logical queries. For instance, a family member asks you to send them a good portrait of you for whatever reason. You may want to make sure that it is somewhat recent. However, you're willing to take one that is two years old if it looks better. Then you could search for: #portrait & #me & #2017 & #2016 & #2015

</comment></comment><comment><AUTHOR> I suspect the site is suffering the HN "Hug of Death".

As linked elsewhere[0], on github:

[https://github.com/StrumentiResistenti/Tagsistant](https://github.com/StrumentiResistenti/Tagsistant)

[0] [https://news.ycombinator.com/item?id=14537805](https://news.ycombinator.com/item?id=14537805)

</comment><comment><USER22> Great work OP! Had done something similar as my master thesis in 2005 basing on one of Hans Reiser's papers. It was more of a quick-hack having the code written in three days so not much have survived to this day, but you can see some of it in action on an adobe flash[1] video at [https://adam.kruszewski.name/assets/static/mtfs- demo.htm](https://adam.kruszewski.name/assets/static/mtfs-demo.htm)

[1] don't judge me, mp4 in the browser wasn't main-stream back then ;)

edit: for those without flash like me -- even as a quick hack it had manual and automatic file tagging (based on file metadata) and you could query it using logical expressions. It also had pretty nasty memory leak I didn't care to find out :) Still, without a first-class, built-in support from file managers like BeOS had for its filesystem the idea is not fully realized I think.

</comment><comment><USER23> It isn't explicit about this on its main page description. This project depends on FUSE to work.

</comment><comment><USER24> My mind read this as tagistant - which is cool and awful name at the same time.

</comment><comment><USER25> Now we just need a redesign of Unix tools and principles/practice to accommodate these :-S

<comment><USER14> That's actually a fairly reasonable conclusion.

You'd want to have tools which are aware of a semantic filesystem, or new tools which can make use of them. Probably a mix of both.

Wrappers around extant tools might be a reasonable migration path toward the former.

<comment><USER25> There was a concept of "db-fs", whereby rather than files in a hierarchical folder structure, you just had blobs/collections of data you could search via the usual search-queries. I suppose similar structure to the average no- sql/JSON db, but as a file-system.

The problem is, it isn't even compatible with the usual interface for a fs driver (read/write, permissions etc). There is no "search" concept in the fs conceptual model. The model assume, to some extent, a finite-size mount, duplication/multi-reference is only poorly supported/emulated via hard/soft links.

plus, existing tools would, as you mention, have no support for what would now be build into the fs - functionality such as provided by 'find' would be now built in, such that you would need a shell syntax/dsl to utilize.

My best guess would be a 'special' query command that would result in a virtual folder popping up on a special virtual fs mount, e.g:

        > mk_qdir *some-search-query*
        /proc/srch/023013/

but if ls'ing the folder kicked of a proc in the background that initiated the query, you would have to be careful no process (a mount counter for 'df', for example) crawled the partition.

<comment><USER14> Good point on mount counts.

Doesn't that problem already exist to an extent for remote FS mounts (NFS, etc), especially over automount?

<comment><USER25> Yep, if the external mount is large.

However, it's a little different for an fs system that can quickly (w/o network-speed limits) generate recursive fs structures.

for example, what if a created a vfs that created a 'foo' folder, with a 'foo' folder inside, and so on. The system would crawl an infinite descent of /foo/foo/foo/foo... and so on, which would eventually fill some cache or another.

<comment><USER14> Good thoughts, and I'm thinking of possible hazards and pitfalls of the approach.

There's already a concept of setting flags to avoid traversal of certain filesystems, and given the proliferation of virtual and networked filesystems, this seems useful: /proc, /sys, /dev, /udev, and a few others (it's getting to the point I don't recognise a full mounts listing readily anymore under Linux).

With the concepts I'm considering, in particular, of the tree as being essentially _search traversal_ rather than _a static filesystem_ (not even one with lots of symlinks all over the place), the potential to create some deep dives or recursive tangles is pretty high.

Another example: tools such as 'locate' or Spotlight shouldn't attempt to traverse this tree when generating indices. Instead, they should _query_ it when requests are made.

Information and metadata leakage is another key consideration when moving data off local host and/or caching among hosts.

<comment><USER25> Yep, in fact I think a lot of vfs are like this - doesn't /proc/ query the kernel when returning parts of its tree (e.g. representing processes).

wrt the fs representing queries: one of the tag systems is like this, it represents tags as folders, and the search for term 'X' and 'Y' can be found under either folder ./X/Y/ , or ./Y/X/ ; obviously, every new tag combinatorically expands the virtual space.

<comment><USER14> Right. My vision is of a /docfs under which you might travers /au/stephenson/ti/snowcrash _or_ /ti/snowcrash/au/stephenson, as an example. Either way works.

If a search terminates in multiple (or no) results, it's a directory, if in a single result, it's that file. Plus a few twists (virtual / dynamically generated file formats, summaries, synopses, metadata).

<comment><USER25> Of possible Interest is the new article:

[https://news.ycombinator.com/item?id=14550060](https://news.ycombinator.com/item?id=14550060)

and [http://www.sqlite.org/src/doc/trunk/src/test_onefile.c](http://www.sqlite.org/src/doc/trunk/src/test_onefile.c) for an idea of turning SQlite into and _actual_ vfs.</comment></comment></comment></comment></comment></comment></comment></comment></comment>


Trump Fires Acting Attorney General - <AUTHOR> https://www.nytimes.com/2017/01/30/us/politics/trump-immigration-ban-memo.html <comment><USER3> If I understand it right this is the first time an AG has been relieved for purely political reasons. If the AG doesn't think it's likely they'll win a defense, that usually doesn't cause an issue.

As an edit: it probably wouldn't have been reasonable for her to put up an intentionally weak case, but if the order isn't able to stand up in court, then what?

The major worry for me is Trump's seeking to punish anyone who shows disloyalty. If Trump thinks he can have something done, I worry he'll demand people below him do so, or replace them with someone properly spineless.

<comment><USER4> >The major worry for me is Trump's seeking to punish anyone who shows disloyalty.

This shouldn't be surprising nor worrying. This is how hierarchical leadership works and it's fully expected that anyone under the President may be forced to do whatever the President wants. It's the entire reason there are 2 other branches of government.

<comment><USER5> Would you write an app that stores important passwords in plaintext client side if your boss demanded it?

<comment><USER6> This is a great question, and one that should be asked of any engineer.

If you answer yes to this question, you should find another career. Engineers have ethical responsibilities as well.

<comment><USER7> The vast majority of developers are not engineers even if they like calling themselves Software Engineers.

</comment></comment></comment></comment></comment><comment><USER8> From Sam Altman on Twitter: ".@SallyQYates, @ycombinator and many others would be delighted to give you a job and resources to continue your work defending American law."

[https://twitter.com/sama/status/826262739202908160](https://twitter.com/sama/status/826262739202908160)

<comment><USER9> Most likely response: who the fuck is Ycombinator, and why on earth would I want to work there?

<comment><USER10> She'd get to hang out with Peter Thiel.

</comment></comment></comment><comment><USER11> For the curious, the line of succession for AG after deputy goes:

1\. U.S. Atty for E. D. Virgina (Now the current acting AG)

2\. U.S. Atty for D. Minnesota

3\. U.S. Atty for D. Arizona

Interestingly, this is set by executive order so it may be changeable - <USER2> limited to those eligible under the Federal Vacancies Reform Act of 1998, I think. (And I guess we can argue all day over the constitutionality of that under the appointments clause)

Source: [https://obamawhitehouse.archives.gov/the-press- office/2010/1...](https://obamawhitehouse.archives.gov/the-press- office/2010/11/04/executive-order-13557-order-succession-within-department- justice)

<comment><USER12> interestingly there seems to be a new executive order 13762, from the final days of the obama whitehouse, that changed the Justice Department order of succession, yet Trump seems to have followed the order that you mentioned/linked

saw a tweet raising this point after having just read your comment. not sure what it means but thought your comment was very interesting, so... here's some more detail.

<comment><USER11> That's really strange! I wonder if he may have followed the wrong order, or or if the vacancies reform act gives him some leeway here...

[https://obamawhitehouse.archives.gov/the-press- office/2017/0...](https://obamawhitehouse.archives.gov/the-press- office/2017/01/13/executive-order-providing-order-succession-within- department-justice)

</comment></comment></comment><comment><USER13> "I think a lot of the voters who vote for Trump take Trump seriously but not literally. And so when they hear things like the Muslim comment or the wall comment or things like that, the question is not 'Are you going to build a wall like the Great Wall of China?' or, you know, 'How exactly are you going to enforce these tests?' What they hear is 'We're going to have a saner, more sensible immigration policy.'"

-Peter Thiel, 2016

I guess to Thiel, a saner, more sensible immigration policy involves invoking 9/11 to ban people from 7 countries not involved in that attack from the U.S, including permanent residents on a legal path to citizenship and holders of valid visas. All done with zero warning via executive order and without conferring with DHS or DOJ.

<comment><USER14> * The ban is a temporary measure and has nothing to do with religion. President Obama signed a list of countries into law for enhanced visa scrutiny and President Trump used that exact list without modification, directly referencing the statute in which it was codified. They'd already been identified as terrorism hotspots that warranted additional scrutiny from immigration officials by the Obama administration. Trump just took it a step further and temporarily halted immigration from these nations while his administration reviews the policies to ensure they're sufficient.

* Alien permanent residents from the affected countries are not currently banned. They never were. Neither are dual citizens.

* It is true that non-diplomatic visas have been temporary suspended (not canceled), but this applies at border crossings only; persons currently in the country on those visas do not have to leave.

* There has been 18 months of warning in the form of the Presidential campaign. Trump promised something like this up and down. He was taken seriously, not literally, and he seriously implemented a 90-day stay on the travel of foreign nationals from areas with high terrorist activity, using the exact list that President Obama signed into law. He did _not_ literally implement a ban on Muslims, or indeed, any other religious group, despite the literal meaning of his prior comments that did explicitly mention Muslims. Likewise, his statements about "the wall" seem to be flexible to the point of something more like multiple layers of well-patrolled fencing.

To me, "seriously but not literally" appears to be holding up really well. Trump is not betraying the intent of the voters who elected him, but he is also consulting with his advisors and legal experts and implementing things that accomplish the intention within the constraints of practicality and legality.

<comment><USER15> _> President Obama signed a list of countries into law_

 _> using the exact list that President Obama signed into law_

Yes, it's the same list of countries... with much, much broader restrictions placed upon them in this instance, to the point that some of the restrictions may violate the Immigration and Nationality Act of 1965 (point[0], counterpoint[1]).

0\. [https://www.nytimes.com/2017/01/27/opinion/trumps- immigratio...](https://www.nytimes.com/2017/01/27/opinion/trumps-immigration- ban-is-illegal.html)

1\. [http://www.nationalreview.com/article/444371/donald-trump- ex...](http://www.nationalreview.com/article/444371/donald-trump-executive- order-ban-entry-seven-muslim-majority-countries-legal)

<comment><USER14> It's a matter for the judiciary to determine exactly whether an obscure law is contradicted or not and whether that law supersedes the President's authority to issue EOs to the contrary. Not the AG's role, especially since the matter at hand is _not_ prosecutorial discretion but merely the DoJ's ability to prepare a fair legal defense for President Trump's order. Why should a judge not be able to hear what the DoJ can trudge up? Would the hearing be fair if Ms. Yates' rule was allowed to stand?

No one denies that President Trump's order is more restrictive, nor that it's inconvenient for the people whose travel plans have been temporarily disrupted. The point is that Trump did not pull this out of thin air. The countries had been previously identified as potential sources of malicious actors. It seems perfectly sensible for a new administration with a significantly different perspective to temporarily suspend travel to/from such high-risk blocks while the vetting procedures are evaluated.

<comment><USER16> > It's a matter for the judiciary to determine

I'm pretty sure any oath of office, including those in the executive, include the protection of the constitution and bill of rights. To leave the defence of the constitution solely in the hands of the judiciary is terribly dangerous, especially considering how politicised that branch has become.

> It seems perfectly sensible [..] to temporarily suspend travel to/from such > high-risk blocks while the vetting procedures are evaluated.

There are enough articles out there why it's not reasonable (mostly the fact that there haven't been any attacks from citizens of those countries in the US, that it would have been reasonable to continue the SQ while any review is ongoing, that the review could have been done during the transition etc.)

The acting AG's issues where not about "reasonable", but "legal". Specifically – with regards to the actual cases she would have had to defend – the legality of removing people holding permanent residency permits.

The illegality of removing green card holders is pretty obvious. To quote the Department of Homeland Security:

As a permanent resident (green card holder), you have the right to:

\- Live permanently in the United States provided you do not commit any actions that would make you removable under immigration law

\- Work in the United States at any legal work of your qualification and choosing. (Please note that some jobs will be limited to U.S. citizens for security reasons)

\- Be protected by all laws of the United States, your state of residence and local jurisdictions

Your Responsibilities as a Permanent Resident

As a permanent resident, you are:

\- Required to obey all laws of the United States the states, and localities

\- Required to file your income tax returns and report your income to the U.S. Internal Revenue Service and state taxing authorities \- Expected to support the democratic form of government and not to change the government through illegal means \- Required, if you are a male age 18 through 25, to register with the Selective Service

[https://www.uscis.gov/green-card/after-green-card- granted/ri...](https://www.uscis.gov/green-card/after-green-card- granted/rights-and-responsibilities-permanent-resident/rights-and- responsibilities-green-card-holder-permanent-resident)

<comment><USER14> Nitpicks over specific laws do not justify invocation of the oath of office. Despite what this thread is trying to pretend, it is not conventional for the AG to play SCOTUS and pre-emptively declare the administration's position indefensible.

Ms. Yates has not offered any legal rationale for her position. She is failing to uphold the Constitution by undermining the operation of the executive branch and the authority of the chief executive without a compelling constitutional cause.

In principle, you're correct that AGs should decline to participate in the implementation of clearly unconstitutional instructions. However, no such plain and obvious unconstitutionality exists in this case. There's nothing even close. All actual legal complaints raised to this point involve potential collisions with obscure, decades-old laws. It's the judiciary's role to handle that type of conflict.

<comment><USER16> See above: Green Cards confer full protection of law, including entry, to the extend of rights afforded to citizens. The exceptions are limited to people committing crimes or leaving the US for prolonged times. That is an exhaustive list (other laws include a generic clause like "and as deemed necessary by the Department of...").

EO cannot break laws, only modify their implementation. There is no room for interpretation. You could argue that the right of entry is not in the constitution – but I'm pretty sure even Scalia would have found a way to argue that US citizens have constitutional right to entry into the US (It's probably not in the bill of rights because it's too obvious)

See also the Watergate precedent: the AG refused to fire the special prosecutor investigating Nixon, in defiance of a direct order by the President. His replacement also refused and was fired, only the third in line complied. Those sacrifices went a long way to create the political climate in which the (now universally recognised as such) crook Nixon had no choice but to resign.

<comment><USER14> You're going all over the place here. I understand your opinion that Mr. Trump's EO is illegal. That's fine, but it's not relevant, unless you're a federal judge who likes to post on HN in his free time? And even then, it's irrelevant until this case comes before the bench and gets a fair hearing.

In the Saturday Night Massacre, the two AGs in question _did_ resign rather than carry out Mr. Nixon's order. They didn't claim a fundamental constitutional conflict that would justify defiance; rather, they claimed personal reservations and recognized that those reservations made it impossible to observe the established chain of command, so they stepped aside.

The issue here is whether it's appropriate for the acting AG to hijack the DoJ based on their private opinion on matters of nuanced legal interpretation, of which reasonable people can be on either side, and which have not yet been given a fair hearing in a court of law. That is absolutely _not_ the convention and the oath of office does not require it (and indeed, Ms. Yates does not claim it does). Until such time as the constitutional violation becomes blatant, the DoJ has a constitutional obligation to serve the people's elected chief executive.

Ms. Yates was actively involved in blocking such fair hearings based on nothing but her personal proclivities. Following the precedent of SNM, she should've resigned over personal reservations. In this instance, there is no justification for defiance.

Ms. Yates's behavior looks very questionable and unprofessional to me. She stayed on board for Obama's EOs, which everyone knew were illegal at the time of signing, and was happy to allow the judiciary to make its call there, but is not willing to do so in Mr. Trump's case, where the legal justification appears much more sound.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> The kick in the balls to Trump was real. I was so happy to read Sally Yates' thoughts on his immigration ban. Obviously, her days were numbers, so she had nothing to lose. Might as well go out like she did in an honorable fashion. :)

Edit: fixed pronoun

<comment><USER18> Boente is the replacement. Yates was the AG who was fired.

</comment></comment><comment><USER19> From 0 to Saturday Night Massacre, in 10 days. That's not really a record one ought to be proud of.

[0][https://en.wikipedia.org/wiki/Saturday_Night_Massacre](https://en.wikipedia.org/wiki/Saturday_Night_Massacre)

<comment><USER20> I feel like the comparisons to Nixon's actions are quite a bit strained. At the very least the magnitude of the actions is vastly different. Nixon was shopping around for a AG that would remove a special prosecutor investigating him.

<comment><USER21> Trimp is shopping around for an AG that won't investigate him.

<comment><USER22> Do you have a reference for this? The last I heard, Sessions was still his pick.

</comment></comment></comment></comment><comment><USER23> Almost my whole adult life, I looked up to America. Whatever I know and my life's value system is mostly because of the books I read by American authors. I couldn't belive the news when I heard that Americans chose Trump. When did America become so stupid to vote in these people?

<comment><AUTHOR> We didn't choose Trump. He trailed the leading candidate by almost three million votes, an unprecedented amount. Making such statements is a slap in the face of the majority of American voters. There are also still substantial questions remaining as to how much assistance Trump may have received from foreign governments in achieving his victory.

<comment><USER24> > He trailed the leading candidate by almost three million votes

i.e., he _won_ the popular vote, _except for California_.

<comment><USER25> Is California not part of the United States?

</comment></comment></comment></comment><comment><USER26> If I understand the American system correctly, this is indeed something that Trump has mandate to do. However.

I wonder whether I will ever stop to be surprised and horrified over the development in the US.

<comment><USER27> Trump and the GOP have a mandate to govern, ie. to conduct the normal affairs of the government in accordance with statute and internal policy.

A "mandate" makes it sound like Trump, with a minority of the adult voting-age adult population of the United States has received the right to transform the country wholesale, which simply isn't true.

</comment></comment><comment><USER28> Shocking, and also politically/tactically very stupid. Why not just use his base's newfound dislike of Yates (the fired AG) as leverage to get Sessions (his AG nominee) confirmed? Which confirmation _was_ all but assured; but now...

I thought his campaign was brilliant, tactically; but this is amateur hour.

<comment><USER29> > Why not just use his base's newfound dislike of Yates (the fired AG) as > leverage to get Sessions (his AG nominee) confirmed?

Because Sessions was already a shoe-in with the Republican Senate, but there are Republicans in the Senate opposed to the immigration order, and they've specifically, among other issues, cited the failure to legally vet the order. Making Sessions confirmation about Yates opposition to the order wouldn't make it smoother.

</comment></comment><comment><USER30> If I am not wrong this sets the wrong precedent, although it is hardly surprising that he has done this. I get a sense of Hitler and the Nazi party idelogy from this, they killed their opponents, now it is just firing & shaming & blah blah that their trolls do. But these things start innocuously, nobody knew that Hitler would turn out that way, not even thise who were mesmerized by his charisma, they staunchly supported him and later woke up to the horrors too late when Berlin was bombed ( I recently watched Man in the high castle, hence Hitler Nazi)

<comment><USER31> Request for feature on HN: anytime you write 'Hitler', your comment starts out at -1 karma.

<comment><USER30> Do you get the irony?

</comment></comment></comment><comment><USER32> Please, take two seconds to read the statement. There's third graders who literally have a better grasp of the English language than Trump.

<comment><USER33> Let's not fall into the trap of thinking that Trump is stupid. He may be a lot of things, but stupid people don't make it into the White House.

And speaking of people who may be smarter than we think: [http://keithhennessey.com/2013/04/24/smarter/](http://keithhennessey.com/2013/04/24/smarter/)

<comment><USER34> > He may be a lot of things, but stupid people don't make it into the White > House.

No, he's like a smart person. (Trump in speech to the CIA).[1]

Really, if this isn't proof that stupid people _do_ make it into the White House I don't know what is. If Trump was really smart he would be in control of the situation and not headed for an impeachment (hopefully sooner than later). The guy is a living trainwreck and he's taking America with him, that his main advisor has actually stated that his goal is to burn it all down[2] and does not seem to care if he destroys the civilized world as we know it as collateral damage doesn't help one bit. For every day that this passes it will take a year or more to undo the damage, assuming it can be rectified at all.

A couple of months of this and it will have gathered momentum to the point where the divisions have become so enraged that a shooting war is a real possibility, which could be one of the goals, to purposefully lit the fuse so momentum will take over and they can enact even more ridiculous measures with the cover of an actual emergency situation. It's not like that trick didn't work countless times in the past.

[1] [https://www.youtube.com/watch?v=3OG6itojBiI](https://www.youtube.com/watch?v=3OG6itojBiI)

[2] [https://www.google.com/search?q=bannon+wants+to+burn+it+all+...](https://www.google.com/search?q=bannon+wants+to+burn+it+all+down)

<comment><USER35> > headed for an impeachment (hopefully sooner than later).

Not a rethorical question, I just don't know. If there is impeachment and Trump is removed from office, what's supposed to happen? VP replaces him? That doesn't really sound like an improvement.

<comment><USER36> > That doesn't really sound like an improvement

Perhaps not to Americans, but to the rest of the world the VP seems like the stabler choice. The administration's alignment would move from Chaotic to Lawful. I happen to value predictability and forces that can be reasoned with.

<comment><USER34> I suspect this is part of someone's game plan. Get the disaster elected then impeach him so someone else who would never in a lifetime manage to be elected president on his own power can take over to then do a whole bunch of stuff that would have never happened otherwise.

Trump was using the GOP just as much as the GOP was trying to use Trump, the question now is how is the GOP going to control the problem they created without ending up in the gutter themselves. They are all scared and probably with good reason, any impeachment proceedings with Trump as the subject will automatically include a horde of very angry and potentially dangerous people. Right now they might be wondering if it is better to let things go to hell under Trump than to risk the wrath of a good 30% of the population.

</comment></comment></comment></comment></comment></comment><comment><USER37> His base will love this

<comment><USER9> He could shit on a plate and they'd eat it up. "Masterful distraction! Wow! He's been planning this for weeks! MAGA"

<comment><USER38> Crudely put, but accurate. It shows the emptiness of this seriously versus literally sophistry. The more norms he breaks, the more certain segments of his base eat it up, seriously, literally, or other.

</comment></comment></comment><comment><USER39> Imagine a world in which officers of the executive branch were completely free to defy the president. Would there not be chaos? It makes sense for a president to fire insubordinate subordinates.

The AAG was likely trying to score political points by going out the way she did. I respect her acting according to conscience, but do think making a show of it like that dilutes the message a bit. Probably more effective for her to have made a statement outlining her objections to the EO and unwillingness to defend it, and then resigned.

</comment><comment><USER40> "Ms. Yates (new AG) was the only person at the Justice Department authorized to sign applications for foreign surveillance warrants. Administrations of both parties have interpreted surveillance laws as requiring foreign surveillance warrants be signed only by Senate-confirmed Justice Department officials. Mr. Boente was Senate-confirmed as United States attorney and, though the situation is unprecedented, the White House said he was authorized to sign the warrants."

</comment><comment><USER41> I'm outraged, but like .. this is not hacker news at all.

</comment><comment><USER42> He acts like Erdoğan.

</comment><comment><USER43> The fact that this is flagged is ridiculous. The USA is undergoing historic turmoil - history is literally being made; meanwhile on Hacker News: here's some JS tips for your next webapp!

<comment><USER44> You can't judge this accurately by a single post. HN has had a significant uptick in political stories lately. Please see [https://news.ycombinator.com/item?id=13522433](https://news.ycombinator.com/item?id=13522433) and [https://news.ycombinator.com/item?id=13516969](https://news.ycombinator.com/item?id=13516969) for more about this if you're interested.

Edit: looks like a moderator unflagged this one. That's how the system is designed to work: upvotes, flags, and moderation all play a role.

<comment><USER45> We all appreciate the work you folks are doing keeping up with the moderation, I'm sure there's a lot of it.

But I hope I don't detect a note of dismissal here. I think it's reasonable to question examples of flagging such as this one, which kept a trending story off the front page for a critical period that almost allowed a much inferior duplicate to take over. Considering these situations carefully will keep the community standards healthy.

</comment></comment></comment><comment><USER46> Respectfully, can we maintain the political detox on Hacker News? [https://news.ycombinator.com/item?id=13108404](https://news.ycombinator.com/item?id=13108404)

I'm personally as engaged in reading and debating politics as I ever have been, but elsewhere. As dang put it: "Political conflicts cause harm here."

<comment><USER47> From news.istambulcombinator.com.tr , circa 2012 :

>Respectfully, can we maintain the ban on political articles on Istambul Hacker News?

Sure, we can close our ears, and also we can argue that here we only discuss tech all the time. But as long as the world we live in influences tech, and all of us, there should be a political discussion here.

HN comments are one of the sharpest for online forums, and it is a shame that we are not allowed to a medium to read what the tech community feels about the current issues

<comment><USER46> I'm happy to debate the intersection of technology and politics here. But not pure politics. This community has an identity and norms and topics that are and aren't in scope.

An alternative would be to use the same software, same group of passionate people to start the discussion, and the backing of Y Combinator in a different forum with a different identity, norms, etc.

I'm also happy to be wrong / downvoted. It's for the community to decide, but it has to be a choice.

<comment><USER47> I agree that ultimately the forum reflects the joint will of both YC the company and its users. For instance, a bird-watching forum might have a a strict ban on anything not related to birds, while the YC ban on politics is softer.

What I don't like at all is people abusing flagging. You can flag any new article with ~4 votes, so less than a handful of active users can shape what we end up discussing.

</comment></comment></comment></comment><comment><USER48> is there an option like "i don't really care about politics, US especially" anywhere on this site?

<comment><USER22> If you feel a submission is inappropriate for HN, you can flag it. Edit to add: Or hide it, an option that doesn't get a lot of attention.

</comment></comment><comment><USER49> Anyone in the private sector would be fired under similar conditions, except probably without an article in the NY times.

<comment><USER50> Considering it is illegal for companies to do what Trump ordered[0], I'm not sure that would be the best analogy.

[https://www.eeoc.gov/laws/types/nationalorigin.cfm](https://www.eeoc.gov/laws/types/nationalorigin.cfm)

<comment><USER49> She was fired for outright refusing to do her job [1], encouraging her subordinates to the same, and for going public about it.

[1] "to prosecute and conduct all suits in the Supreme Court in which the United States shall be concerned"

<comment><USER51> No, her job is to help follow the laws of the country, not to do whatever the President says. The President is not a dictator or a CEO.

<comment><USER52> The president is a CEO. He is quite literally the chief officer of the executive branch.

Though he is a CEO of a public entity. Something I'm pretty sure he has no experience with.

</comment></comment></comment></comment></comment><comment><USER53> Remember when there was a ban on political topics for like three days? Remember how nice that was?

<comment><USER54> At this point, I think we should just upvote _all_ new incoming political threads. Either allow the angry people to get it out of their system, or clearly illustrate that this stuff doesn't belong on HN.

</comment></comment><comment><USER55> She had two legitimate options: execute the President's instructions if she thought they are legal and moral, or resign if she thought they are illegal or immoral. She doesn't have the option to refuse to do her job.

Ministers and government officials resign all the time when they believe that their government's position is in error.

N.b.: I did not vote for Trump; I am not certain that everything in his immigration-related executive order is consistent with the law (but I am not certain that it's _not_ , either: I am not a lawyer).

<comment><USER56> > _She doesn 't have the option to refuse to do her job._

Oh, yes she does. The head of the DOJ, acting or actual, should be relied upon to raise a notable fuss when an executive is determined to be stepping outside the bounds of legality. Same goes for every other official who takes the oath of office. They take an oath to defend and uphold the Constitution, not an executive's agenda. Resigning is not defending and upholding the Constitution. If, however, an order belived to be illegal and unconstitutional is challenged and subsequently upheld as legal and constitutional, and a person continues to object to carrying out their duties accordingly (based on moral or other grounds), then they should likely follow their conscience and resign. But not beforehand.

</comment></comment><comment><USER57> Have I ever had a strong disagreement with my boss? Always. Every day. But did my boss fire me? No. But should my boss considered to fire me? Probably. Normally you don't fire someone from your cabinet even if you are just an acting member. You make the person to resign.

I don't know the full story, but this goes to show Trump is truly an unique president. He doesn't care what people think of him. He's running the White House like a real business. Disclaimer, I like the idea of running government like it's a business (think Singapore, salary is on par with industry standard and subjected to yearly review). But I strongly disapprove Trump's ruthless dictatorship. He just doesn't understand that his business decision impacts nationwide. You need to actually think through before making any decisions or giving any public comments. Even Joe Biden learned to shut up (although he did make some dangerous comments in the past). It's okay to have a strong opinion (some people don't like compromise), but use reason and earn people's trust. He makes up stories and excuses, with little to no real proof for many of his past and current commentaries. No one really trust him except the people who voted him into the office. His cabinet members - trust me, none of them really trust him. I know that from my gut. I know my 9-month old niece can tell that as well. His mind is juggling and wandering all around the space. He's intellectually challenged, despite being a ruthless and tricky business man. He went broke but made so much from bankruptcy. I will give him the credit of being a smart asshole and a good salesman, but other than that, he's a terrible leader.

Trust me, he would be the best Secretary of State of U.S. to the Martians, because he speaks nonsense.

<comment><USER58> While Yates is a hero, she didn't merely disagree. She actively countermanded a directive from her boss.

<comment><USER57> I do that if I know it isn't right. You may argue she should just resign. But perhaps, and I don't want to make any assumptions, somebody just want to give him a hard time and stand up for his/her belief against Trump for as long as possible. The old saying, you can't do anything unless you are in the position. You can resign and next person take over can kiss Trump's ass. Do something while you are in the position that can do something.

</comment></comment></comment><comment><USER59> I despise Trump, but I don't think it's it's acceptable for a civil servant to insert themselves into the news. If you're getting an unlawful or unethical order, you follow the process, suck it up or resign in protest.

By going rogue and making yourself the story, you're attacking the office of the president. It's a bad idea, bad precedent and will have bad outcomes.

<comment><USER19> Bollocks. Presidents are not above the law; most of them don't need to have their nose rubbed in the carpet in order to learn that.

It is eminently clear that the muslim ban executive order was drafted, written, and released without any of the normal communication, forewarning, or legal reviews. The AG is 100% within their duties to the nation and the constitution to say that they will not enforce something of that nature.

<comment><USER60> Did Obama, Clinton, or Bush have an AG that willfully showed insubordination and encouraged it to their subordinates? I'm surprised her action didn't get more news coverage, well not really.

<comment><USER61> In March of 2004 John Ashcroft's Justice Department determined that the Bush Administration's Stellar Wind surveillance program was illegal. While Ashcroft was in the hospital being treated for acute pancreatitis, President Bush sent White House Counsel Alberto Gonzales and Chief of Staff Andrew Card to tell Ashcroft to sign a document overturning his department's ruling. He refused.

He remained Attorney General until February 3rd, 2005, when he resigned the office.

<comment><USER62> The Acting AG at that time? The current FBI director, James Comey. Truth is stranger than fiction.</comment></comment></comment></comment></comment>


Pinboard 2014 Expenses - <AUTHOR> https://docs.google.com/spreadsheet/ccc?key=0Au7yiR802qQmdE5seXMzTlFDTl80QTFNaWpmTHBfdlE&usp=sharing#gid=0 <comment><USER2> He runs a really, really right ship. Off the top of my head I think the SaaS budget alone at my companies is larger than that.

That said, by the standards of physically extant businesses, it is ridiculous how far you can get on $1k or $2k a month.

<comment><USER3> I am actually jealous of the level of control. I need to work harder !

<comment><USER3> Sorry that was meant to comment on the top, not yourself.

Edit: And was barely worth commenting at all, let alone this comment only drawing attention to the information-free nature of my opinions.

Oh dear ...

But the OP has shamed me into realising I do need to upgrade my financial controls - "worrying" does not seem to count as a system!

</comment></comment></comment><comment><USER4> I wonder what other services would suit this small, "artisinal", paid-for internet service model?

There's some RSS readers and email providers. Photo hosting and blogging are too "bulk" and high-profile for this business model, probably.

I suspect the abuse/takedown request workload is tiny, which helps a lot in keeping the cost down.

<comment><USER5> There are a lot of niche opportunities in larger segments. The hard part is always getting attention / traction, getting those first thousand customers to pay, getting enough money to afford some marketing and support (if necessary).

Just a simple example: HomeAway's huge vacation rentals business. Within the larger super category ('for rent by owner'), there are long term rentals, and that's a niche compared to the much larger vacation rental market. Several years ago, you could have created a modest business around that smaller niche, charging $30 or $50 per year for a listing. 10,000 listings at $50 per year becomes a nice, likely quite profitable, small business.

Real estate listing concepts are great if you can get traction and listings, the costs of hosting and serving up the data and images might as well be zero.

</comment></comment><comment><USER6> They use Hetzner for dedicated servers, slicehost for VPS, 2 colocation services, and AWS cloud.

For a relatively small site, why do they need so many different hosting providers?

<comment><USER7> I run admin + staging stuff and my own email on the dedicated servers. The Hetzner box has some offsite backups, too. I want something that won't die at the same time as the website.

I use two colos for redundancy. One is in Sacramento, and it mitigates the earthquake risk of the one in Fremont.

The AWS costs consists of stuff I'm too lazy to go find and delete. Everyone has an unused EC2 instance or S3 bucket somewhere; that's what keeps Amazon in the black.

<comment><USER8> Interesting. Are colo prices generally that high in the US? May I ask what's included at 600$ for 2(-3?)U of rack-real estate there?

      * What level of support do you have (can you call someone at the
           DC at 4am and have him hot-swap a failed HD for you)?
      * How many shared or own gigabit or 100mbit uplinks are included?
           (if >1: can do bonding on your interfaces?)
      * With how many other servers do you share the same circuit-breaker?
      * I guess you have at least 1 UPS + 1 non-UPS power supplies?
      * exceptionally good peering?
      * network/power SLA?
      * 24/7 access to your server?

I worked for a DC in europe some years ago and i'm genuinely intrigued how the price in your spreadsheet for colo-hosting is so much more than what i was used to (~100-150$/U).

<comment><USER7> $600 is for an entire cabinet, 15A, 100 Mbps bandwidth. HE doesn't offer half- cabinets in that facility, or I would have gotten one.

Note that 15 A is nowhere near enough power to fill it to even 1/3 capacity.

<comment><USER8>
      > Note that 15 A is nowhere near enough power to fill it to even 1/3 capacity.

Yes that's pretty standard, afaik. The dc I worked at had individual circuits even for quarter- and half-cabbinets (2x 16A each), but i don't know how common that is in general. (instead of just putting 4 quarter-cabinets on the same circuits and have 1 customer blow his and 3 others' power)

</comment></comment></comment></comment></comment><comment><USER9> I've read so many terrible budget spreadsheets that I first I assumed it was all in thousands. Nice work on keeping the costs down!

<comment><USER10> Those figures aren't in _thousands_? That's some good cash management!

</comment></comment><comment><AUTHOR> Source: [https://twitter.com/Pinboard/status/494238943894700032](https://twitter.com/Pinboard/status/494238943894700032)

</comment><comment><USER11> Is it me, or does $60-70/month for DNS seem pretty high?

<comment><USER7> Seems high to me too (I use DNSMadeEasy). But I'm too lazy to shop around for alternatives right now. The current setup Just Works™ and is not a big proportion of my costs.

<comment><USER12> CloudFlare offers DNS for free, or $20/mo if you want to pay them.

[https://www.cloudflare.com/dns](https://www.cloudflare.com/dns)

</comment></comment></comment><comment><USER13> Pinboard is one of a few services I happily pay for. Honestly, I'd probably pay a subscription price for it. I've got 1041 bookmarks, it's fast, searches and tags flawlessly, the unread functionality is super useful, and I've got some (unofficial?) android app that makes it a snap to use with my phone. Keep up the great work.

<comment><USER14> Start paying for archiving then. :)

</comment></comment><comment><USER15> Does Hetzner has any proper competitor somewhere, with similar power/price ratio? Especially interested in some on the US side of the pond. They seem to be a go-to company for everyone that wants to host stuff on his/her own.

<comment><USER16> Looking at their VPS offers, digitalocean is about the same.

</comment></comment><comment><USER17> This is great information on just how little it takes to run a service like Pinboard, with hundreds of thousands of active users, on your own hardware. Just imagine how much more it would cost on Heroku or even AWS, just so you can avoid having to think about ops.

<comment><USER7> The site only has about 25k active users. But I'm pretty sure it could handle about 10x the traffic without changing this spreadsheet much.

<comment><USER18> Hmm. For comparison, historious has around 7k active users and I only pay $10/mo for a 1 GB RAM linode.

<comment><USER7> Good for you!

</comment></comment></comment></comment><comment><USER19> still happy i paid for pinboard a while ago.

the "no-nonsense, speed first" policy and the fact that it's actually being executed are my favorite things about the service.

</comment><comment><USER20> Maciej finally bought an iPhone! When I met him last year in Australia he uses his 15" laptop to check his tweets.

</comment><comment><USER21> About how much were the AWS expenses before moving to your own servers?

<comment><USER7> I never ran the site on AWS. Here's a spreadsheet from 2011, when I was running the site on leased servers:

[https://docs.google.com/spreadsheet/ccc?key=0Au7yiR802qQmdHU...](https://docs.google.com/spreadsheet/ccc?key=0Au7yiR802qQmdHU2ZXJfVDY0c0JSbHd6aTd0M01BWEE&usp=sharing)

I had to change providers numerous times that year for various reasons, hence the confusing number of services.

</comment></comment><comment><USER22> slicehost? :)

</comment><comment><USER23> wat [https://twitter.com/Pinboard/status/494239839349268481](https://twitter.com/Pinboard/status/494239839349268481)

<comment><USER24> You've stumbled upon the greatness that is Maciej's sense of humor. Just click the follow button, and consider yourself lucky.

</comment></comment><comment><USER25> Don't confuse Pinboard with Pinterest, which is thousands times bigger.

Pinterest active monthly users: 60 million

Pinboard active monthly users: 24 thousand

<comment><USER7> By all means confuse Pinboard with Pinterest if you are depositing that sweet VC money.

<comment><USER26> Just don't confuse it with pinboard-dot-com which looks pretty sketchy :)

<comment><USER7> Pinboard dot com is run by a hilarious, crotchety Linux guy who has been around forever. He did me a huge favor by not selling that domain despite great temptation.

<comment><USER27> Looks like he recently moved to pinboard.jp

pinboard.com is currently not responding, and was last cached on 29 Jul at 1:47 GMT (roughly 5 hours ago).

<comment><USER28> It's working here (redirecting to .jp)

</comment></comment></comment></comment></comment></comment><comment><USER29> I'm building a bookmark site as well and I'm looking for seed users.

If you have a such need, please try [https://linklet.io/](https://linklet.io/)

It's not officially released yet so there might have problems.

<comment><USER24> I clicked, ready to chastise you for blatantly advertising on a competitor's thread, and for launching yet another business without charging users money.

I'll still chastise you for the blatant self-promotion on a competitor's thread, but kudos for actually charging money!</comment></comment>


CIA 'Angelfire' covert Windows malware system - <AUTHOR> https://wikileaks.org/vault7/#Angelfire <comment><USER2> There are multiple companies in the industry where you could get interns to build a project like this. Implantation (what we used to call "rootkit") tech is more interesting than banking trojans from a technical perspective, and at the very high end can reach the level of generating new research results --- but this isn't that level.

People continue to be enthralled by low-on-the-food-chain software security work. There are teenagers that have written stuff like this. It's sort of embarrassing that the CIA commissioned this at all (if that's how it happened), rather than pooling with NSA and getting a proper, reconfigurable, deniable tool built.

(But then: leaks of NSA tooling in the last 2 years have demonstrated that we've all been a bit generous regarding NSA's technical reputation, too).

Virtually everything that Wikileaks has published about the CIA has been like this. To the extent that it damages national security, it does so by making CIA look clownish. And yet, despite the extraordinarily low stakes of publishing, Wikileaks is still milking a drip-drip from the original cache they obtained of CIA warez. It's cynical, and speaks to a general contempt they have of their audience.

<comment><USER3> If you're the CIA/NSA don't you want to use this kind of low level stuff most of the time because it blends in with the background malware? If it's discovered which it may almost certainly be at some point there's little to point at and say "this is malware created by a nation state" vs "this is malware made by a bored teenager".

Seems like a positive advantage?

<comment><USER4> Seems like the difference of assassination by hit-and-run/"robbery gone wrong" vs polonium. One screams "state actor", the other is almost entirely deniable..

</comment></comment></comment><comment><USER5> jesus christ. the damage our own government has done to microsoft is just insane. if i was a shareholder I would be so pissed.

big companies go along with this behavior and collude with the government, but if this has a big enough effect on the bottom line, then it will happen less.

<comment><USER6> You must see it in a broader context. I'll paraphrase you:

"jesus christ. the damage our own government has done to [Iraq, Syria, Libya, North Korea, Vietnam, Serbia, Afghanistan...] is just insane. if i was a [citizen of these countries] I would be so pissed."

At least, the shareholder has his life and his property (sans the missed opportunity due to "the damage our own government has done to microsoft").

<comment><USER5> i agree with you completely, but it seems irrelevant to this particular conversation.

</comment></comment></comment><comment><USER7> Is the WikiLeaks site not responding to requests?

<comment><USER8> It loaded for me. Here's the text of the post:

\</comment></comment><comment>

Angelfire 31 August, 2017

Today, August 31st 2017, WikiLeaks publishes documents from the Angelfire project of the CIA. Angelfire is an implant comprised of five components: Solartime, Wolfcreek, Keystone (previously MagicWand), BadMFS, and the Windows Transitory File system. Like previously published CIA projects (Grasshopper[1] and AfterMidnight[2]) in the Vault7[3] series[4], it is a persistent framework that can load and execute custom implants on target computers running the Microsoft Windows operating system (XP or Win7).

Solartime modifies the partition boot sector so that when Windows loads boot time device drivers, it also loads and executes the Wolfcreek implant, that once executed, can load and run other Angelfire implants. According to the documents, the loading of additional implants creates memory leaks that can be possibly detected on infected machines.

Keystone is part of the Wolfcreek implant and responsible for starting malicious user applications. Loaded implants never touch the file system, so there is very little forensic evidence that the process was ever ran. It always disguises as "C:\Windows\system32\svchost.exe" and can thus be detected in the Windows task manager, if the operating system is installed on another partition or in a different path.

BadMFS is a library that implements a covert file system that is created at the end of the active partition (or in a file on disk in later versions). It is used to store all drivers and implants that Wolfcreek will start. All files are both encrypted and obfuscated to avoid string or PE header scanning. Some versions of BadMFS can be detected because the reference to the covert file system is stored in a file named "zf".

The Windows Transitory File system is the new method of installing AngelFire. Rather than lay independent components on disk, the system allows an operator to create transitory files for specific actions including installation, adding files to AngelFire, removing files from AngelFire, etc. Transitory files are added to the 'UserInstallApp'.

[1]: [https://wikileaks.org/vault7/grasshopper/](https://wikileaks.org/vault7/grasshopper/)

[2]: [https://wikileaks.org/vault7/#AfterMidnight](https://wikileaks.org/vault7/#AfterMidnight)

[3]: [https://wikileaks.org/ciav7p1/](https://wikileaks.org/ciav7p1/)

[4]: [https://wikileaks.org/vault7/#](https://wikileaks.org/vault7/#)

\</comment><comment>

And here are the documents linked beside the post:

Angelfire 2.0 -- User Guide: [https://wikileaks.org/vault7/document/Angelfire-2_0-UserGuid...](https://wikileaks.org/vault7/document/Angelfire-2_0-UserGuide/)

BadMFS -- Developer Guide: [https://wikileaks.org/vault7/document/BadMFS_Developer_Guide...](https://wikileaks.org/vault7/document/BadMFS_Developer_Guide/)

Wolfcreek Docs -- Angelfire User Guide: [https://wikileaks.org/vault7/document/Wolfcreek-Docs- Angelfi...](https://wikileaks.org/vault7/document/Wolfcreek-Docs- Angelfire_UserGuide/)

Wolfcreek Docs -- Angelfire Test Matrix: [https://wikileaks.org/vault7/document/Wolfcreek-Docs- Angelfi...](https://wikileaks.org/vault7/document/Wolfcreek-Docs- Angelfire_test_matrix/)

Wolfcreek Docs -- NotesSee more: [https://wikileaks.org/vault7/document/Wolfcreek-Docs- Notes/](https://wikileaks.org/vault7/document/Wolfcreek-Docs-Notes/)

<comment><USER9> > It always disguises as "C:\Windows\system32\svchost.exe"

If I ever meet the manager who decided bundling half the services in the OS into svchost was a good idea, I'll give him a piece of my mind. I've lost more hours to that (both cleaning malware and troubleshooting performance or crashes) than anything else on Windows since the 9x days.

<comment><USER10> Seems like a kneejerk reaction.

The reason services are bundled into SVCHOST is that it offers reduced memory footprint and lower startup costs, which is still significant on resource constrained systems (which Windows embedded still targets)[0].

If malware didn't copycat SVCHOST they would just copycat one of a dozen other common Windows processes like conhst, dllhost, csrss, etc.

As an aside, in Task Manager if you go to the details tab, select columns, command line. You'll see exactly what each instance of SVCHOST is running. Process Explorer gives you even more information than that.

Process Explorer allows you to turn on Digital Signature checking. Run it as administrator. Select Columns -> Verified Signer. Options -> Verify Image Signatures. But also check your CA store to make sure no custom CA has been injected into the OS.

[0] [https://blogs.msdn.microsoft.com/oldnewthing/20030918-00/?p=...](https://blogs.msdn.microsoft.com/oldnewthing/20030918-00/?p=42423)

<comment><USER9> Now that there are nice tools like Process Explorer, it's less of an issue. Back in the day, you'd pull your hair out.

Even so, the original premise of performance doesn't hold water to me. I load a process that has 10 services inside so I can use two of them, and that's somehow reduced memory compared to just running the two, just in case I want to run the other 8 later? Some of those services I have disabled and never want to run. It's still taking up the memory.

<comment><USER7> Still, if MS is going to do it, then the onus seems to be on them to surface it in built-in tools.

</comment></comment></comment></comment></comment><comment><USER11> Can someonebody please help me understand what this is? Is it software which Microsoft has included in Windows maliciously for such exploitation, or is it the fact that the CIA has identified a Windows exploit and taken advantage of it?

Did Microsoft know that this problem existed, and continued to let it exist at the CIA's request?

<comment><USER12> Neither. This was not written by Microsoft. This is not an exploit. Further, you never "identify an exploit" \- you identify a vulnerability, and the exploit is what you build to take advantage of that vulnerability.

This is software that has been written by presumably US govt to be installed on a windows machine to maintain control if it once they've compromised it. Think "really advanced malware". It is supposed to be hard to detect and/or remove, and allows long term access to that machine.

Edit: hopefully that helps clear up. Also your line of questioning seems to suspect Microsoft colluding with the govt. zero evidence of that.

<comment> wu-ikkyu >Also your line of questioning seems to suspect Microsoft colluding with the govt. zero evidence of that

Snowden leaks indicated otherwise

<comment><USER12> Everything I wrote is referring to this Wikileaks release. There is zero evidence Microsoft colluded to help the CIA write this malware.

</comment></comment></comment></comment><comment><USER13> Had to update [https://www.npmjs.com/package/nsaname](https://www.npmjs.com/package/nsaname) again. It never stops...

<comment><USER14> Why? Was one of the Angelfire names used on the NPM usage examples?

<comment><USER13> No. I added the new names.

</comment></comment></comment><comment><USER15> I wonder what political news story this is trying to take attention away from...

<comment><USER16> I don't understand what you are implying here. Would you kindly tell us which political news story are those pesky russian hackers at wikileaks trying to cover up?

<comment><USER15> Discussion from last: [https://news.ycombinator.com/item?id=14920664](https://news.ycombinator.com/item?id=14920664)

<comment><USER16> What's the point of saying 'Wikileaks is trying to distract us from happenings in Russia' when you are not even specifying the things we are supposedly being distracted from?

If you are trying to proliferate the 'Wikileaks are Russian agents' mantra at least back this up with some facts and observations. Don't simply link to the circlejerk from previous thread - that's kind of lazy.

<comment><USER15> >What's the point of saying 'Wikileaks is trying to distract us from happenings in Russia' when you are not even specifying the things we are supposedly being distracted from?

Am I required to have omniscient knowledge of the subject prior to commenting? You asked me what I was referring to, and I answered your question.

It has been painfully clear, with almost every release in recent history, that wikileaks uses releases as distractions to take the spotlight away from other stories. That's why it's worth bringing this up on every post.

Just because wikileaks wants the story to be about the release, doesn't mean that's where the real story is. I'm curious as to what the real story is. I do not know the answer to that question, hence my initial comment.

</comment></comment></comment></comment></comment><comment><USER17> WikiLeaks was taken down by hackers last night

<comment><USER18> DNS was hijacked, that's it.

<comment><USER19> Except I believe people were still showing a successful HTTPS connection, were they not?</comment></comment></comment>


KiK's Side of the breaking-the-internet story - <AUTHOR> https://medium.com/@mproberts/a-discussion-about-the-breaking-of-the-internet-3d4d2a83aa4d#.95priw8ta <comment><USER2> already submitted 15+ times, discussion here [https://news.ycombinator.com/item?id=11346845](https://news.ycombinator.com/item?id=11346845)

Please check before submitting!

</comment><comment><USER3> The World Wide Web is a subset of the Internet.</comment>


Mercurial 1.0 released - <AUTHOR> http://www.selenic.com/mercurial/wiki/index.cgi/WhatsNew <comment><USER2> I've been using Mercurial for personal projects and I've really been liking it. I do a lot more branching and merging with it and it feels faster because of local commits.

<comment><USER3> I've noticed it feels faster than SVN even when used like svn (pushing/pulling from a remote source)

</comment></comment><comment><USER4> We looked at few solutions at our startup and we decided to go with git instead of Hg after we launch (we'll use svn until then). I actually liked both and liked Hg a bit more due to its simplicity but since the Rails world has adopted git and everyone's distributing code using git and github, the decision was basically made for us.

<comment><USER5> We're in the same boat. We've been using Subversion, but will switch to git once our tools support it (we eat our own dog food, and Subversion is the only revision control system our products support right now). The determining factor for us was similar, as well...the Perl core developers appear to be moving to git for all Perl development. We're a mostly Perl shop, so we'll go where Larry and Co. goes. It just insures that the tool-chain for working with our codebase with git will be really solid, if the majority of Perl developers are also using it.

<comment><USER6> You know about the git svn bridge, right? It comes right in the standard git distribution, works intuitively and easily, and is the only thing keeping me sane as I work on two projects managed in Subversion. Basically, I use git in my development environment, with all the slick branching, merging, and great performance. Then I push code out to Subversion, and it all just works.

</comment></comment></comment><comment><USER7> It seems as though Linus (he wrote git right?) has successfully performed his coup of the version control systems. Ok, not really, but its surprising that he could change what was thought to be an essentially solved problem with cvs and then svn. I would have never thought that someone could come up with a better CVS, but then SVN came out. Then came Git, and then this. Just goes to show that even problems that seem "solved" might have better solutions given a different set of situations.

<comment><USER8> > Then came Git, and then this.

Git and hg were both started in April, 2005, in response to the Bitkeeper fiasco.

<comment><USER7> Ahh... I knew Git was basically started with that Bitkeeper war, but since I don't really follow linux politics much as I don't use it all that often, I didn't realize that hg was started then as well. They seem very similar in their paradigm shift away from the central repository style of CVS and SVN. Is that a coincidence or was Bitkeeper also in this everything is a branch method?

<comment><USER8> I think it was Bitkeeper's solid decentralized workflow that convinced Linus that VC would even be feasible for kernel development; hg and git were written after people realized that they could no longer do without the features that Bitkeeper had.

</comment></comment></comment></comment><comment><USER9> I haven't used Mercurial, but I highly recommend darcs over git. The only advantage git has (which may be important to you) is it is much faster. Interactive recording on darcs is a joy to use.

<comment><USER10> darcs user interface is a joy to behold - it's far easier to get up and running than git, and easier on a day-to-day basis. However even on a one man project, I found that darcs performance problems started to bite after a few hundred check-ins (darcs unrecord being an obvious problem). IMHO this isn't a rare problem with darcs - any reasonably sized project is likely to hit this at some point. I also managed to get my darcs repositories into an unusable state several times just by pushing cherry-picked patches between them, which wasn't fun.

I miss darcs in some ways, particularly as getting running with git is a pain in the rear end. Fortunately darcs-git [http://ftp.frugalware.org/pub/other/pacman-tools/pacman- tool...](http://ftp.frugalware.org/pub/other/pacman-tools/pacman-tools/) emulates enough of darcs user interface to sweeten the pill, and is definitely a good way to get started with git. I've also found that some of the functionality available in git but not in darcs has been very useful (even if it is typically a pain to work out how to use it).

<comment><USER9> any more info on using darcs-git? Looks like a python script that implements darcs user interface functionality on top of git without any need for darcs.

</comment></comment></comment><comment><USER11> Mercurial rocks. We use it at the startup I work for, and it has been invaluable. Being able to swiftly and easily make your own branches is fantastic for prototyping.

<comment><USER12> I'm an hg fan, too.

I pondered darcs, since there are a number of Ruby projects using it, but landed on mercurial.

</comment></comment><comment><USER13> I'm currently considering switching from Darcs to Mercurial or possibly Git. I've had problems with Darcs taking vast amounts of time to commit larger files and I'm assuming that both Merurial and Git (especially) will be much better at that.

I'm leaning more towards Mercurial at the moment because while I don't use Windows much myself I do want to be able to work with people who do.

Any thoughts would be appreciated....

<comment><USER6> Both git and hg are very nice, you will not go wrong with either one. If Windows support in the near future is a priority, you should probably use Mercurial. It works well today. The Windows port of git is coming along, but I don't know how long it'll take to reach first-class citizen status.

That said, if you don't need to deal with Windows, I suggest using git. It has a more power-user feel to it than Mercurial, and it has a few spectacular features I can't live without: rebase, stash, and the git-svn bridge. I'm also rather fond of git's lightweight branch mechanism. I know hg supports a variant of rebase and stash using the Transplant and Queues extensions, but the last time I looked at hg svn, it wasn't ready for use.

Also, when you read about git, you'll hear a lot of first-time users complaining about "the index." Just remember to think of it as a staging area for commits, and you'll be fine. It was a bit confusing at first, but then I realized that I like using it for making granular commits and diff checking.</comment></comment>


Why whales sing: it's a question of culture - <AUTHOR> https://www.spectator.co.uk/article/why-whales-sing-it-s-a-question-of-culture <comment><USER2> "Animals are more human than we'd like to admit"

I take issue with this subtitle of the article. Humans are animals. This false dichotomy between Homo Sapiens and "animals" has been foisted upon us by religious and homo-centric ideology. We need to embrace Darwinian evolution and our biological roots, we are a branch on the tree of organisms. We should be surprised that "other" animals may have aspects of culture;of course they do, as do macaques, wolves, chimpanzees, crows, eusocial insects, singing birds, anything that lives in colonies, etc.

<comment><USER3> I don't think the intent of the article is to indicate that we are not animals but to point out that we should apply more of the way on how we treat other humans to how we treat animal because the difference we thought were between animals and humans are really much smaller than we thought and in many cases we are much closer than many of us want to belive.

<comment><USER3> The subtitle even says: "Animals are more human than we'd like to admit"

So all humans are animals but not all animals are humans which is correct. You could also say "human" in this context is used as in "human behavior". Not sure.

However, I as well would have greatly have prefered: "Humans are more animals than we'd like to admit"

</comment></comment></comment><comment><USER3> On every turn we discover that animals have more cognitive ability's and are closer to the specific primate branch that is us than we previously imagined. But a mix of convenience, prejudges and ""culture"" will make sure nothing about the way we handle animals will change for a long time.

Sometimes I feel like on staying in the middle of Rome looking around at all the slaves and thinking why the f### does nobody care?

<comment><USER4> Ignorance, fear ... and nature ... i understand needing to hunt for food, when food is scarce. But we dont stop and continue because of habbit, ritual, and culture.

Basically cargo-culting until it risks our own skin.

I am filled with a deep sense of apology to everything we touch ... very few things we touch are truly better. Its a shame, and a gloomy, powerless conclusion.

I hope .. that we change, and transcend, soon.

<comment><USER5> Read about some microbiology. Bacteria and fungi are all about drastically changing their environments to suit themselves while at the same time making the area toxic to their competitors.

Penicillium is a famous example of a fungus that makes toxins to destroy bacteria. On the other hand, the bacterial cultures in sourdough produce lactic acids which make the environment hostile to other, slower-growing things.

Then on a macro scale, look at beavers. They undertake massive ecological engineering for their own sake. They can wipe out countless other small animals and ground-nesting birds with the floods they cause.

It's another kind of human exceptionalism to think we're unique in altering the environment to suit ourselves at the expense of other species. We're not. Perhaps the more distinguishing feature is that some of us actually care about it.

<comment><USER6> I think that makes the difference, the fact that we are (becoming) aware of the consequences of our actions. It gives it the ethical/moral responsibility that we don't expect from fungi, trees, or beavers.

On the other hand, I like your comparison to other species. At an essential level, human culture has lot in common with ancestral life forms, like a culture of slime mould. There's collective behavior over generations, of which individuals are largely unaware.

"Exceptionalism" is an apt word, because not only do we treat other species as _our_ resources, we do it to our own. Humanity eats its children. The fact that we're doing this knowingly, is why we carry shame, a guilty conscience in our collective psyche. We expect better of ourselves, because we should know better.

</comment></comment></comment></comment><comment><USER7> While this is always an interesting topic, this article is barely more than an advertisement for a book. I think the advertisement would have been more effective if it picked one example, such as whale song, and stuck with it a bit. Then ended with 'learn more about this and other interesting animal behavior in the new book.'</comment>


A global map of wind, weather, and ocean conditions - <AUTHOR> https://earth.nullschool.net/ <comment><USER3> I forked this and put plate tectonic velocity vectors (in an earth-centered, earth-fixed coordinate system) in place of wind vectors but didn't have the .js knowledge or time to fix all the annotations and menu stuff. I'm hoping to get to that this fall (but I've been saying that for a year or two...):

[http://earth-analysis.com/vels/public/](http://earth- analysis.com/vels/public/)

<comment><USER4> Nice! Lucky the annotations aren't accurate, I wouldn't like to have 68mph plate tectonics... :)

Interesting that there seem to be the plate tectonic versions of cyclones, like off Baja California.

Also, is the absence of data at the poles due to a coordinate system limitation?

<comment><USER3> The velocities are in mm/yr. 68 mph plate tectonics would definitely be terrifying; that's an order of magnitude faster than faults move during an earthquake.

There are two phenomena that look cyclonic.

The larger ones have to do with motion on a sphere: Euler showed that any velocity vector on a sphere can be represented as a rotation about a point (called an Euler pole); tectonic plates move basically rigidly, so the whole things rotate around a point. When that point is very close to (or within) the plate, the rotation is quite visible. You can see this in the Antarctic plate, halfway between Antarctica and Madagascar.

The second is an artifact of the vector visualization that Beccario used. The atmosphere is a continuum, so particles move smoothly between one location and another (more or less). But tectonic plates have rigid boundaries, and dive under each other or slide rigidly past one another. But the visualization algorithm doesn't know about these boundaries, so it creates particle paths that cross the boundaries and appear to swirl around. This probably happens to some degree in the mantle below the plates, but not really at the surface, with minor exceptions that I won't get into now unless people really want to nerd out on microplate rotation.

I'm not sure about the data gap at the poles. That's probably a real gap in the dataset I used; I haven't looked into it.

<comment><USER5> The point where velocity field vanishes seems to be a good example of the hairy ball theorem; even if the motion on the surface is not a rotation, if the field is continuous there has to be at least one point on the surface where velocity is zero.

[https://en.wikipedia.org/wiki/Hairy_ball_theorem](https://en.wikipedia.org/wiki/Hairy_ball_theorem)

</comment></comment></comment></comment><comment><USER6> I used this site to check wind conditions during the 2015 Tianjin explosions. I was 90 miles away (Beijing) and was considering leaving the country had wind been blowing in the wrong direction. At the time the risk of the hazardous chemicals released by the explosions was completely unknown, so wind conditions were the only objective piece of information I could base my decision on. Fortunately the wind was blowing north/east.

</comment><comment><USER7> It's especially interesting to watch Hermine travel up the US east coast right now. NHC forecast track here for the next 5 days: [http://www.nhc.noaa.gov/graphics_at4.shtml?5-daynl#contents](http://www.nhc.noaa.gov/graphics_at4.shtml?5-daynl#contents)

Right now is especially fascinating because the Euro and GFS models have recently begun to agree on the storm re-forming into a hurricane in a couple of days off the New Jersey coast. It's a fascinating time to watch wind patterns in the US right now!

Edit: Another really interesting tool is the Total Precipitable Water product: [http://tropic.ssec.wisc.edu/real-time/mimic- tpw/natl/main.ht...](http://tropic.ssec.wisc.edu/real-time/mimic- tpw/natl/main.html)

<comment><USER8> I have found the Euro model quite good for the Northeast US.

<comment><USER9> Ecmwf are the best medium term forecasts, 3-7 days.

<comment><USER8> Do you know a place to view ECMWF? Weather Underground isn't showing it any more.

</comment></comment></comment></comment><comment><USER10> Thanks, but I prefer this [https://www.windyty.com/](https://www.windyty.com/)

<comment><USER11> The interface on windyty.com is better. It does not have the problem of filling up your browser's history with every move you make. I would like to be able to see the color scale key with out also having to open the full control panel.

<comment><USER12> Every new view should get a URL. That makes every view linkable. People who do not do this are actually violating the spirit of the HTML/URL specs. The fact that you see your browser history as polluted is an implementation detail that could be rectified by simply collapsing all consecutive URL's from the same domain over N url's into a hierarchical dropdown.

<comment><USER13> It can also be (sort of) rectified through judicious use of history.pushState vs. replaceState. The latter just replaces the URL in your URL bar but _doesn 't_ add an entry to your history - so every new view can get a new URL without polluting history.

I say "sort of" because everyone has their own idea about which things should be pushed vs. replaced. One person's "history pollution" is another's useful tool. But in this case it seems pretty clear that simply moving the map shouldn't push to history.

</comment></comment></comment></comment><comment><USER13> If you're interested in playing with this visualization technique (particle advection in vector fields) outside the context of geospatial applications - <USER2> built this tool called "Vector Toy" which allows you to visualize any given (2D) vector field this way: [http://dandelany.github.io/vector- toy/](http://dandelany.github.io/vector-toy/) \- note that the functions on the right side are editable.

</comment><comment><USER14> If you use 500mbar visualizations, you can visualize the "reverse" weather pattern from last summer, with the wonky jet stream, high pressure baking the Pacific Northwest, and cool weather in the midwest and on the East Coast: [https://earth.nullschool.net/#2015/07/03/1800Z/wind/isobaric...](https://earth.nullschool.net/#2015/07/03/1800Z/wind/isobaric/500hPa/orthographic=-122.01,47.50,651)

Another example from July 2014: [https://earth.nullschool.net/#2014/07/14/1500Z/wind/isobaric...](https://earth.nullschool.net/#2014/07/14/1500Z/wind/isobaric/500hPa/orthographic=-122.01,47.50,651)

Conversely, you can see how the "heat dome" of this summer smothered almost the entirety of the contiguous United States, except for the West Coast in general and the Pacific Northwest in particular: [https://earth.nullschool.net/#2016/07/22/1500Z/wind/isobaric...](https://earth.nullschool.net/#2016/07/22/1500Z/wind/isobaric/500hPa/orthographic=-132.88,45.66,651)

</comment><comment><USER15> Whoa, check this out, 121 KM/h winds over the Antarctic Peninsula: [https://earth.nullschool.net/#current/wind/surface/level/ort...](https://earth.nullschool.net/#current/wind/surface/level/orthographic=-71.49,-68.20,3000/loc=-61.478,-72.375)

It doesn't look like the other storms, in Hawaii and Florida. I wonder if these are normal conditions for that ridge?

<comment><USER16> I think it's called a katabatic wind, see the bottom of [http://www.doc.ic.ac.uk/~kpt/terraquest/va/science/climate/c...](http://www.doc.ic.ac.uk/~kpt/terraquest/va/science/climate/climate.html)

</comment></comment><comment><USER16> This is fantastic!

Be sure to click the menu, and browse the chemical pollution or particulate data, and the alternative projections. XKCD fans will be pleased :-D

</comment><comment><USER17> That's really cool, but does it really have to reload data on every viewport move/change?

I use somewhat similar app for android made by Croatian Ministry of Maritime Affairs, Transport and Infrastructure [https://play.google.com/store/apps/details?id=hr.mppi.nis&ah...](https://play.google.com/store/apps/details?id=hr.mppi.nis&ah=69Ap9mDbQ_49TEg0t0Iuisb69UY) There's also iOS version [https://itunes.apple.com/us/app/nautical-info- service-croati...](https://itunes.apple.com/us/app/nautical-info-service- croatia/id1134146644?ls=1&mt=8)

Really handy when you're out there.

<comment><USER13> It doesn't reload data - it just stops the particle system advection and restarts it when you're done moving.

And yes, this is pretty necessary (at least it's quite hard to work around). To maintain the same view during viewport movement, it would have to keep track of all the particle trails in lat/long space and re-project them all to the new view. But this is implemented in canvas, so the trails (ie. particle histories) are likely not stored in their entirety - more likely, the system keeps track of only the particle's _current position_ , and the trails are accomplished by drawing new line segments over the old on each frame. Yes, it would be technically possible to keep track of each particle's full history, and to re-project them all during interaction, but it would be pretty slow.

(I didn't make this, but I've written a very similar particle system in canvas and ran into similar issues)

<comment><USER18> I imagine you could accomplish the trails with some triangle strips in WebGL and let the GPU do the heavy lifting. I wonder if that would perform any better.

</comment></comment></comment><comment><USER19> Similar, but Arctic only: [http://www.arctic.io/#simulation](http://www.arctic.io/#simulation) Changing dates and hours with a slider is quite informative.

</comment><comment><USER20> Another is Ventusky [http://www.ventusky.com/](http://www.ventusky.com/)

</comment><comment><USER21> Does anybody know what this is? [https://earth.nullschool.net/#current/wind/surface/level/ort...](https://earth.nullschool.net/#current/wind/surface/level/orthographic=-143.19,19.58,3000/loc=-149.272,19.983)

It shows over 300F in the middle of it.

<comment><USER11> A storm. Look at the wave height. Also appears to be moving. It likely won't be there later.

Oh and that's wind speed not temp. In K/h.

Edit: actually the degrees you are seeing is the wind direction.

<comment><USER16> Here's a link with a time, so it continues to work: [https://earth.nullschool.net/#2016/09/02/1500Z/wind/surface/...](https://earth.nullschool.net/#2016/09/02/1500Z/wind/surface/level/orthographic=-143.19,19.58,3000/loc=-149.272,19.983)

</comment></comment></comment><comment><USER22> This is also good [https://www.windyty.com](https://www.windyty.com)

</comment><comment><USER23> this is amazing! so trippy. You can really track the two storms (US East Coast and right of Hawaii). The eye of the storm is relatively calm at 0-6 km/hr.

</comment><comment><USER24> This is cool! I would love to see a similar global car traffic visualization. I suppose that data is much harder to come by.

</comment><comment><USER25> Very nice but strangely it's becoming slower on Firefox when I'm moving the mouse cursor around.

</comment><comment><USER26> This is very cool but, wow it takes a lot of cpu cycles. My desktop tripped out after about 5 minutes.

</comment><comment><USER27> Anyone know what the 100km/h ridge of high speed wind over Antarctica is from?

</comment><comment><USER28> Very impressed at how well this works on mobile!

</comment><comment><USER29> I love the options for different projections.

</comment><comment><USER30> Way to go Cambec!

<comment><USER31> thanks!

</comment></comment><comment><USER32> Awesome.</comment>