Flexbox and Grids: your layout's best friends - <AUTHOR> https://aerolab.co/blog/flexbox-grids/ <comment><USER2> Some quick rants.

CSS is almost as bad as PHP when it comes to naming. Most flex related properties are prefixed with flex-; but some aren't (align-items, etc) so you now have tons of non prefixed properties that actually only work within a very specific context (vertical-align, etc) and there are no compilers to tell you which combinations don't make sense or whether you have redundant properties.

IE11 has tons of bugs with flexbox; I had to remove a flexbox layout earlier because of that crappy browser (it ignores explicit/inline width/height set on flex children).

Otherwise it's a great tool if used sparingly (heavily nested flexbox elements with lots of special properties start to become a bit hard to follow)

<comment><USER3> I was using VS Code earlier today and used vertical-align in some case where it didn't make sense and VS called it out. Such things do exist!

But I agree with your points. Despite its flaws, it is probably the best thing out there for creating visual layouts from code.

<comment><USER4> Webstorm is absolutely incredible. Everyone should try it. Seriously, `cd your-app && brew cask install webstorm && webstorm .`

Best IDE ever. Better than emacs. And I've loved emacs for a couple years, and Vim for 10 years before that. Webstorm even comes with a Vim plugin.

IntelliJ is really rocking it. Webstorm for webdev, Cursive for clojure ([https://cursive-ide.com/](https://cursive-ide.com/)), PyCharm for python ([https://www.jetbrains.com/pycharm/](https://www.jetbrains.com/pycharm/))... every one of those IDEs feels amazing out of the box.

It sounds dumb, but just try it. You'll get hooked on Webstorm and never want to go back. Example: press shift twice. It pops up a "Search Everywhere" box, and you can type filenames, symbol names, actions (Rename, etc)... it's basically helm M-x, but way better.

You won't ever feel like "CSS names are bad!" because you won't ever need to remember anything. Want to center something? Just type "align" and usually webstorm pops up autocomplete with the right thing. And that's true in every context: [https://i.imgur.com/jCAAesF.png](https://i.imgur.com/jCAAesF.png)

All of these IDEs are specializations of IntelliJ IDEA ([https://www.jetbrains.com/idea/download/#section=mac](https://www.jetbrains.com/idea/download/#section=mac)). They specialize it for web and call it WebStorm, for Python and call it PyCharm, etc. But all the plugins and settings work in all the IDEs, so really the underlying awesomeness is IDEA.

At this point I trust IDEA enough to know that if I open any codebase, I can start navigating it right away. Doesn't matter if it's C++, ocaml, or VueJS. Typing shift-shift will always let me jump to definitions.

Yeah it costs money, but $13/mo is super reasonble. Dropbox was similarly impactful on my life, and that was $10/mo.

<comment><USER5> It also has 6000+ unresolved issues reported, a lot of them open for 3-4 years with little hope of getting fixed. They keep rolling out half-baked support for the trendiest new JS frameworks, while basic parts of the IDE are falling apart every day. (For example, when opening the mentioned "Search Everywhere" dialog, sometimes it just doesn't receive focus. Good luck with that productivity.)

Meanwhile, after launch, the IDE grabs focus about six times while loading. Symbol navigation sometimes picks up random unrelated functions from ignored folders instead of the one sitting right beside the file, because it has the same name. Want autocomplete for that vertical-align's value? It just throws up every possible value for any CSS property.

Pretty bad when code editors running in a browser engine are getting more usable than your paid product.

<comment><USER4> Yeah, the unresolved bug reports are pretty annoying. I ran into similar problems trying to get Live Edit working with AngularJS.

 _Symbol navigation sometimes picks up random unrelated functions from ignored folders instead of the one sitting right beside the file, because it has the same name._

Even after excluding a directory? If you have autogenerated build folders (e.g. dist/, out/, etc) you have to mark them as Excluded: [https://i.imgur.com/E2uXJzu.png](https://i.imgur.com/E2uXJzu.png)

Odd. I haven't noticed that, but I'll watch out for it.

<comment><USER5> This usually happens from node_modules, marked as a library folder, so it's indexed and might even be relevant, but it shouldn't really be a better match than the one in the source. And it's not even consistent.

Hope they'll get their shit together, it used to be a pretty great IDE before these annoyances started piling up.

<comment><USER4> Ah yeah, I've always excluded node_modules. That's true though, it should probably be indexed.

</comment></comment></comment></comment></comment></comment></comment><comment> c-smile "flexbox horrors":

"flex-basis" \- defines mysterious "main width" (WTF is wrong with just width?)

"flex-grow" and "flex-shrink" \- grow and shrink relative to what?

How to define flexible margins? Like I want left margin to be twice larger than right one and all this to be flexible:

        |---m:2*</comment><comment>-|--1*---|--m:1*-|

?

Effectively flexbox breaks CSS box model that postulates only width/height to define box dimensions.

Why I cannot say just this:

        { width: 1fr;
          margin-left: 2fr;
          margin-right: 1fr; }

?

Where 'fr' are __flex units __introduced in CSS Grid[https://developer.mozilla.org/en- US/docs/Web/CSS/flex_value](https://developer.mozilla.org/en- US/docs/Web/CSS/flex_value)

Why two specifications define the same entity (flex weight) by two different mechanisms - by property and by unit?

CSS looks like Christmas tree where each passer-by (Google and Microsoft here) adds its own shiny stuff ...

Is this what we name as modularity of CSS?

<comment><USER4> Just use Bulma. You can drop it into any project. [https://bulma.io/documentation/columns/sizes/](https://bulma.io/documentation/columns/sizes/)

Your example can be accomplished like this:

m:2*

1*

m:*1

There's a good reason to limit yourself to 12 columns: the layout is consistent across all common screen sizes, whether it's tiny mobile screens or huge desktop screens. If you have sizes like 1.374fr, you lose a lot of that automatic layout functionality.

<comment> c-smile Bulma, as any other grid systems based on % units, is quite limited in real life.

Check this: [https://sciter.com/images/wt- langs.png](https://sciter.com/images/wt-langs.png) You see the same HTML markup there but in different languages. Text in most of European languages is longer.

This is HTML/CSS Sciter UI that uses _flex units_ and _flow property_ : [https://sciter.com/docs/flex-flow/flex- layout.htm](https://sciter.com/docs/flex-flow/flex-layout.htm)

<comment><USER4> Hmm, why is it limited? I'm interested in hearing your experiences.

<comment> c-smile Check screenshots above. There is a block having 4-columns layout. Each column there a) is flexible and b) uses min-content constraint. So widths of columns are unknown upfront - determined by actual content in grid cells.

That last part (use of actual content) is what you cannot reproduce using % units in principle.

</comment></comment></comment></comment></comment><comment><USER6> _> If you wish to fully support all browsers (yay!) then I'd highly recommend using feature queries to provide an alternative layout for browsers that don't support Grids or have an older syntax._

How does that not translate into "Just do all your work twice!"?

<comment><USER7> Since CSS grids are mostly used for the global page layout, the ideal use case is using it to design complex pages that would be very annoying to make with other techniques, and on older browsers fall back to a much simpler design with flexbox or floats.

Since those fallbacks only require a handful of lines of CSS, it only takes a few minutes to get the compatibility to almost-perfect, and you can get a better design in front of most of your users very quickly. To me at least it's a solid mix of design freedom and development speed.

<comment><USER6> If you have a good reason for complex pages your solution is only viable once the share of old browsers has dropped below, say, 5% to 10%, depending on your niche.

What's actually going to happen is that most people will layer JavaScript frameworks on top of HTML and then we're going to complain about the huge performance and memory overhead of JavaScript heavy websites.

</comment></comment></comment><comment><USER8> These "Grid by Example" videos by Rachel Andrew are an excellent intro to CSS Grid:

[https://www.youtube.com/watch?v=Dz9BzY21Zks&list=PLQkVA6z3dF...](https://www.youtube.com/watch?v=Dz9BzY21Zks&list=PLQkVA6z3dFvbnBJetfYDAF3-cG_ubgdZR)

Highly recommended for anyone who's not familiar and wants a quick introduction -- each video is just a few minutes long. :)

</comment><comment><USER9> I agree that flexbox and grids complement each other, but I doubt that grids are ready for product. I mean it is nice that all major browsers have a decent support in the newest versions, but production ready includes for me that older browsers (without support or major bugs) are not common anymore (~ <5%).

As far as I can see it it is currently scored at ~75%: [https://caniuse.com/#search=grid](https://caniuse.com/#search=grid)

<comment><USER10> You don't have to make pages that look identical in every browser. Make something that _works_ for everyone and then use newer browser features to make it as good an experience as possible depending what features are available. CSS even has the "@supports" feature for doing it - [https://developer.mozilla.org/en- US/docs/Web/CSS/%40supports](https://developer.mozilla.org/en- US/docs/Web/CSS/%40supports)

<comment> c-smile If to speak about layout methods @supports will not help you but will force you to do the job twice.

If you can do your layout without flex boxes why do you need to create second version with them at all?

<comment><USER10> _will force you to do the job twice_

That's one way of looking at it. I prefer to see it as writing features that only certain users will benefit from. In the case of layout, I tend to give users with older browsers a much more basic, linear flow (just making the semantic HTML work really), and then add in more complex layout using what tech is available. That's not really doing the same work twice in my opinion.

</comment></comment></comment></comment><comment> purple-dragon I was able to finally start using grid in production for a new project a couple of weeks ago. It made page layout a breeze and my html and scss are so much tidier and easier to read. Flexbox still has its place as the article points out, but abusing it to do 2-dimensional layout didn't feel any better than doing the same with older techniques.

<comment><USER11> My understanding is Grid is designed for whole page layout, while Flexbox is designed for sub-page or component layout.

<comment> purple-dragon Yes, but before grid was available people did some funky things to build responsive grids (like using a flexbox column containing flexbox rows or using a lot of repetitive media queries).

</comment></comment></comment><comment><USER12> Is it now possible to do a responsive CSS-only layout a la Masonry/Pinterest, with the correct order of elements using this?:

        [1]  [2]  [3]
        [4]  [5]  [6]

<comment><USER13> No, you can't, because the position of each item depends on all the previous items.

Flexbox only deals with a single row at a time, whereas masonry doesn't really have a notion of a row (given the 4th item in each column can be at very different places, depending on the sizes of the items above). Grid requires you to bind things to an actual grid, so you still can't do masonry (for similar reasons). The closest you can do is using multicol, but that's then top-to-bottom then left-to-right instead of left-to-right then top-to-bottom.

Note that this is mostly thinking about unequal height items (which is the common case of masonry); equal height you can do with floats yet alone more recent layout modes.

<comment><USER12> Thanks for the rundown. That's unfortunate. This might be another case where browsers will eventually implement yet another layout method (how many are we up to now with floats, tables, columns, flex box, grid?) as soon as these layouts are replaced by something new.

<comment><USER13> Masonry is something somewhat fundamentally unique given it relies on how things flow across both dimensions in a specific way; there's been some vague talk about extending Grid to support it (but nobody's actively working on things like that for now, even subgrids have gone a bit quiet), but probably more practically we'll stick to doing complex layout modes with a limited scope in JS (and hopefully soonish with Houdini and avoid much of the costs associated with current ways of doing layout in JS), given solving things like Masonry in a general way is hard (and relatively unclear that the gain is much for the amount of work involved).

As for number of layout modes, it depends on how you count. Roughly: block, inline, table, positioned, flexbox, and grid. Multicol doesn't actually introduce a new layout mode, it just functions in a similar way to fragmentation when doing layout for printed media, just fragmenting across columns instead of across pages.

</comment></comment></comment></comment><comment> crush-n-spread Most everything with CSS Grid is broken on Edge 15, which is what most Edge users are running. Thus Grid is not something you should be putting in production.

[https://caniuse.com/#feat=css-grid](https://caniuse.com/#feat=css-grid)

<comment><USER14> The same site says virtually nobody, aprox 2-3% is using edge.

You either have to go all the way back to supporting IE11 and ignore everything not supported by 2013 microsoft to capture another shrinking single digit percentage of possible users until they progressively die of old age or ignore Microsoft stuff.

Alternatively start providing a slightly degraded interface with fewer features and a banner. Our website works best with a modern browser with links to up to date edge, firefox, and chrome at top of page. Preferably with a notice regarding dropping support for internet exploder sometime next year.

<comment><USER15> It also doesn't work in the Firefox fork I use. But that's probably an even smaller market share.

I noticed this when Mozilla recently resigned their add-ons site and starting using CSS Grid support sniffing to determine weather to show the real site or the gimped mobile site. As it is all browsers without CSS Grid are forced to use the mobile site.

It's super annoying.

<comment><USER14> The new nightly is a huge improvement, maybe you should try it.

</comment></comment></comment></comment><comment><USER16> I had the chance to see Rachel Andrew speak at laracon. Great presentation and got me excited about flexbox and grid. I'm still wondering is the browser support wide enough to implement this in production for the average site?

<comment><USER11> It all depends on if you have to support IE or not:

[https://caniuse.com/#search=grid](https://caniuse.com/#search=grid)

[https://caniuse.com/#search=flexbox](https://caniuse.com/#search=flexbox)

<comment><USER16> Has everyone decided to drop ie? For tech related sites that might make sense but I think not supporting ie means turning off support to the users who are older in some cases but more likely to buy.

Where are people drawing the line? Bootstrap 3 still supports legacy browsers but version 4 and other popular frameworks like bulma use flexbox.

<comment><USER17> _Has everyone decided to drop ie?_

Maybe some consumer sites aimed at smartphone-wielding younger audiences can now get away with dropping IE completely. _Maybe_ some sites aimed at very small businesses too.

If you're aiming at older individuals or at medium to large organisations, not a chance.

</comment></comment></comment></comment><comment><USER18> this is great info, [https://gridbyexample.com](https://gridbyexample.com) is also good resource

</comment><comment><USER19> The article does not really say what to use if something does not support grids? should your fallback be flex box? Also IE 11 does not support the @support.

<comment><USER20> My personal favorite is lazy loading a traditional grid system with Javascript if there's no css grid support...

</comment></comment><comment><USER21> If you enjoyed this article you may find this one interesting too: [https://tech.nine.com.au/2017/10/25/14/16/unlocking-the- powe...](https://tech.nine.com.au/2017/10/25/14/16/unlocking-the-power-of-css- grid)

It's centered on grids but it has 15th Century analogies. ;)

</comment><comment><USER8> Be careful using CSS Grid - IE11 supports only an old version[0] of the spec, and thus is missing many major CSS Grid features like repeat(), grid-gap and span.

[0] [https://www.w3.org/TR/2011/WD-css3-grid- layout-20110407/](https://www.w3.org/TR/2011/WD-css3-grid-layout-20110407/)

</comment><comment><USER22> Recently used CSS Grids for layout control and Bootstrap v4 for element styling. Grids are insanely easy to use and do a really good job of separating content and presentation.

And I used flex boxes within grids to fine tune layout and it's really awesome and simple. Verbose, but simpler than using frameworks like BS, Semantic or Materialize for layouts.

</comment><comment><USER23> Fantastic piece. Clarity, breadth, depth, working examples... this is what "howto"-style, practical posts should look like.

<comment><USER24> I agree, it's a nice piece of work. It makes a better flexbox tutorial than most flexbox tutorials.

</comment></comment><comment><USER25> When using conditionnal arrival of dom object (I mean like using ngIf in Angular or simply using switching display none/block on some block) flexbox layout seem very difficult to anticipate. For that kind of things I still prefer using inline-block, that is more predictable.

A tip: Strangely Safari tend to need flex-basis value everywhere

<comment><USER13> > A tip: Strangely Safari tend to need flex-basis value everywhere

Most of the Safari issues should have been fixed in Safari 11, FYI. (The implementation of flexbox was largely copied over from Blink.)

</comment></comment><comment><USER26> Is anyone aware of any attempts/plans to bring CSS-grid style layout to React Native? Either officially or through some library/plugin.

I'd love to try out Grid on the Web side of my app, but it's hard to justify if it'd mean I have to maintain a separate Flexbox-based layout for the React Native side anyways.

</comment><comment><USER27> Sure, grid and flexbox are cool, but when is the CSS Cute Cartoon Faces spec coming out!?!?

</comment><comment><USER28> i've just fell in love with Grid's pragmatic approach!

even if it doesn't cover specific edge cases, its simplicity pays off.

</comment><comment><USER29> this thing does not open in safari.

</comment><comment><USER30> Every page doesnt have to look the same in each browser.

</comment><comment><USER31> > If you wish to fully support all browsers (yay!) then I'd highly recommend > using feature queries to provide an alternative layout for browsers that > don't support Grids or have an older syntax.

So basically, write two versions of your layout: the old way, and the new way. Flexbox and grids aren't ready for production yet.

<comment><USER32> Well, it depends on context. More and more, web tools are being used in a way where you can control what browser they're running in. Not just Electron, but also when making UIs for things like industrial tools, where you control not just the browser, but the hardware it runs on.

<comment><USER33> _> Not just Electron, but also when making UIs for things like industrial tools, where you control not just the browser, but the hardware it runs on._

Wow, is that becoming common these days, using web browser engines to run UIs on specialized devices? Sounds a little off in the context of industrial tools, where I'm thinking embedded systems and relatively low-performance SOCs.

<comment><USER32> Sorry. I realise that I combined words that will sound frightening to some. I mean non safety critical systems in an industrial environment. For example, mobile robot health and KPI monitoring.</comment></comment></comment></comment>


Jesus Christ was also misunderstood, Masayoshi Son tells investors - <AUTHOR> https://www.ft.com/content/d01fe70a-598f-4e6f-becc-2a002d6187b8 <comment><USER2> It is incredible how wrong this man is. He predicted a 20 year long stock market crisis and it ended in 3 months.

</comment><comment><USER3>
      The man said get out of here I'll tear you limb from limb.
      I said you know, they refused jesus, too. he said: you're not him.

[https://www.youtube.com/watch?v=kbdF4hBfQiE#t=190](https://www.youtube.com/watch?v=kbdF4hBfQiE#t=190)

</comment><comment><USER4> [http://archive.is/rL8Tj](http://archive.is/rL8Tj)

</comment><comment><USER5> So was "Hong Xiuquan", the brother of Jesus Christ.</comment>


Firework launches a short-form video storytelling app, backed by Lightspeed - <AUTHOR> https://techcrunch.com/2019/03/14/firework-officially-launches-a-short-form-video-storytelling-app-backed-by-lightspeed/ <comment><USER2> Perhaps I am underestimating its creative applications, but the "flip reveal" thing seems like a weak differentiator at best? Although perhaps it is a durable one if they are granted their patent.

The other bits about video length and meaningful engagement seem like positive differentiators, but those also seem easily replicable by competitors.</comment>


Zika virus: Tracing its origins in Uganda [video] - <AUTHOR> http://www.bbc.com/news/health-35436517 <comment><USER2> I haven't seen any references to the film "Children of Men"; that film freaked me out deeply, and is a bit similar to this issue with the Zika visus. Basically, it depicts a world in the near future where no human children have been born for over 18 years (and europe full of refugees, society breaking down into a police state dystopia)

Hearing entire countries advise women to not get pregnant is really scary!

<comment><USER3> > film "Children of Men"

From a P.D. James book, for those more print-oriented

( clicky [https://en.wikipedia.org/wiki/Children_of_Men](https://en.wikipedia.org/wiki/Children_of_Men) )</comment></comment>


Some gripes about nacl - <AUTHOR> http://www.tedunangst.com/flak/post/some-gripes-about-nacl <comment><USER3> It's quite interesting to study why crypto_(secret)box requires zero padding. It's not to avoid memory allocation, as tedu suggests, but to make the implementation stupidly simple. For every invocation of crypto_secretbox, NaCl needs to generate a single-use 32 byte key for the Poly1305 MAC. Since the first 32 bytes of the message are zero, NaCl can simply encrypt the entire message with Salsa20, and the first 32 bytes of the resulting ciphertext conveniently become a suitable key for Poly1305. (So when the documentation says the padding has to actually be zero, it's not kidding.)

By making the implementation more complex (some memcpys and special casing of short messages), as libsodium has done for its crypto_secretbox_easy[1], you can avoid the need for this padding.

Personally, I think djb should have avoided exposing this implementation detail in the interface. To simplify the implementation, he could have used the first output block (stream position 0) of Salsa20 only for the Poly1305 key, and then encrypted the message starting with stream position 1, instead of trying to use the first output block for both the Poly1305 key and the message. The implementation would have been only marginally more complex (not as complex as the libsodium implementation, as you wouldn't need special casing). Unfortunately this can't be changed now without breaking interoperability with messages encrypted with the current version of crypto_secretbox.

[1] [https://github.com/jedisct1/libsodium/blob/17932c782e5900ac8...](https://github.com/jedisct1/libsodium/blob/17932c782e5900ac8cee42fe43e3455db85351e0/src/libsodium/crypto_secretbox/crypto_secretbox_easy.c#L18-L65)

</comment><comment><USER4> Is nacl deprecated in favor of the libsodium _fork_? Or are both projects strong and continuing in parallel?

      [libsodium] is a portable, cross-compilable, installable,
      packageable fork of NaCl, with a compatible API, and an
      extended API to improve usability even further.

Source: [https://github.com/jedisct1/libsodium/blob/master/README.mar...](https://github.com/jedisct1/libsodium/blob/master/README.markdown)

I see DJB (original nacl creator [http://nacl.cr.yp.to/](http://nacl.cr.yp.to/)) appears to contribute to libsodium: [https://github.com/jedisct1/libsodium/blob/master/AUTHORS](https://github.com/jedisct1/libsodium/blob/master/AUTHORS)

The website for nacl seems to be a bit outdated (many dates seem to refer to 2013), although there is still this tantalizing bit about features in the 'next' nacl release:

      Major features in the next release of NaCl: full PIC support,
      for easy integration into other languages; Ed25519 signatures
      (currently available in SUPERCOP); NEON optimizations.

Thanks for any help clarifying this.

<comment><USER5> Being listed in the AUTHORS file does not imply active participation. libsodium is based on NaCl's algorithms and code, both of which are publicly reusable, but I think they were placed in libsodium by other people. If you clone the repo, `git log --author=bernstein` and `git log --author=djb` return no results.

It's probably worth treating NaCl as something like research code or a reference implementation. It's certainly way more usable than most research code is, which is extremely commendable, but it's still not quite libsodium in terms of its goal as a product.

<comment><USER4> I noticed that and assumed that DJB may just not be a 'social coder' with a github account :-)

</comment></comment></comment><comment><USER6> As someone who didn't really know what they were doing, I used NaCl recently to write a program to tunnel packets over UDP.

I started by reading the NaCl source and all the introductory material available—the web site, the two NaCl papers ("Cryptography in NaCl", "The security impact of a new cryptography library"), and quickly reviewed a couple of other papers (e.g. to understand deterministic encryption and D-H key exchange). I think the biggest problem I had was to understand nonce generation and handling properly. The "Cryptography …" paper does contain some advice that I ultimately implemented. But I had to think very hard about what it said before I was confident that I was doing what it said. For example, it says:

«…the nonce can be chosen as a simple counter: 0 for Alice's first packet, 1 for Bob's first packet, 2 for Alice's second packet, 3 for Bob's second packet, 4 for Alice's third packet, 5 for Bob's third packet, etc. Choosing the nonce as a counter followed by (e.g.) 32 random bits helps protect some protocols against denial-of-service attacks. In many applications it is better to increase the counter to, e.g., the number of nanoseconds that have passed since a standard epoch in the local clock, so that the current value of the counter does not leak the traffic rate.»

I managed to figure it out, but I would certainly have welcomed a more detailed explanation, and would have been very happy to have help from the code to do the right thing.

In contrast, the cryptography functions were easy enough to figure out and use (with the C API). The only mistake I kept making was specifying the secret key first and the public key second in all my function calls. Once I got used to doing it the other way around, it was fine. Zero-padding the messages was slightly ugly, but I didn't develop any especially strong feelings about it. (Aside: I actually ended up using TweetNaCl, but of course all the documentation is the same.)

I'm very pleased with the resulting code, anyway.

P.S. I looked at libsodium, but greatly preferred the unadorned library.

<comment><USER7> A former colleague wrote a tool for just such a thing (Encrypted UDP tunneling) [http://nardcore.org/ctunnel/](http://nardcore.org/ctunnel/)

<comment><USER8> This looks good, but from a brief glance I can't seem to find if/how it implements windowing in net.c. I'm guessing it just waits for an ACK before continuing? I'm searching for something very much like this, where I could bind to a TCP socket that uses UDP internally, something like ZeroMQ, for punching through NAT with STUN or something similar.

</comment></comment></comment><comment><USER9> The tl;dr is: use libsodium: [http://doc.libsodium.org/](http://doc.libsodium.org/)

</comment><comment><USER10> The easiest way to get round the slightly odd build environment for NaCl is to just use TweetNaCl ([http://tweetnacl.cr.yp.to/software.html](http://tweetnacl.cr.yp.to/software.html)) which just requires you to drop a single .c/.h file into your project - <USER2> isn't as performant as the reference code but in most cases this probably doesn't matter. The libsodium implementation seems overly complicated to me for most uses.

The NaCl API is reasonably simple and is fairly easy to wrap (plug - if anyone is looking for an example have a look at jim-nacl [1] which embeds into Jim TCL)

[1] [https://github.com/paulchakravarti/jim- nacl](https://github.com/paulchakravarti/jim-nacl)

<comment><USER11> Might also want to checkout libsodium ([https://github.com/jedisct1/libsodium](https://github.com/jedisct1/libsodium)).

<comment><USER10> Except that this is the opposite of my point - the libsodium repository is 9MB with 326 .c/.h files and c.25000 lines of code.

Tweetnacl is 2 .c/.h files and c.1000 lines of code.

</comment></comment></comment><comment><USER12> Paging @tptacek for comment

<comment><USER13> I like nacl.

I like the author of this article and think he knows what he's talking about.

I like libsodium but do worry about the project getting away from them; cleaning up the API is one thing, adding new crypto primitives is another. Still: use libsodium.

The person whose opinion you really want on this article is 'pbsd.

<comment><USER14> I've been worried about that too. Recently I learned that libsodium includes a way to generate crypto_box keys from a fixed argument entropy chunk. It sounds really convenient but also circumvents all of the labor around centralizing entropy generation.

Is that a good thing to have? Maybe! It's convenient feeling. Is it dangerous? I have no idea—probably? Should it be part of the interface to libsodium? No clue.

<comment><USER15> It was added after a popular request: providing a simple way to derive a key pair from a secret key, itself derived from a password.

By not providing a simple way to achieve something, users end up copying/pasting random code and inventing their own crypto.

Additional functions that libsodium provides over NaCl were added after observing how developers were using NaCl in actual projects.

When you see projects that allocate temporary buffers to store signed messages, and immediately discard everything because all they actually need is the signature, you realize that developers have to work around limitations of the NaCl API.

When the maintainer of the Go bindings reports memory corruption, which turned out to be due to sign_open() requiring an output buffer larger than the unsigned message you might blame him for not having read the fine print in the NaCl documentation. But maybe the API should better match what feels intuitive to developers instead.

And when developers systematically write wrappers (for example around box() and secretbox() functions) to work around the same issues, it probably means that API could be improved.

Adding convenience functions addressing very common needs to libsodium instead of having everybody rewrite their own implementation makes it less confusing and eventually less dangerous.

</comment></comment></comment></comment><comment><USER16> These are all very legitimate gripes, and even dangers.

One wrapper I used (think Python/Perl/Ruby), the author did not properly understand how the 0-padding was supposed to be implemented. I was scratching my head for a few hours wondering why messages I encrypted were not being received properly. Turns out, bugs in the C padding code.

The discussion of nonces really needs big flashing warning text and icons. I've seen total disasters with code trying to do nonces correctly, or not even understanding why nonces are important (one VPN package I used had the option of using null nonces, as if it were a feature!)

<comment><USER15> The ZEROBYTES padding is a source of confusion and bugs, including in bindings. Having to allocate more bytes than the unsealed message in crypto_sign_open() was also the root cause of quite a few unfortunate bugs.

And even when used properly, people always ended up writing wrappers allocating temporary buffers and moving data around in order to add/trim these bytes.

And when people start writing wrappers, it usually means that there is something wrong with the API.

For this reason, libsodium fully supports the crypto_box and crypto_secretbox in a compatible way, but doesn't document them in favor of the "easy" interface, which basically does what everybody was reinventing when writing wrappers for these functions.

How to generate nonces for each operation is described in the libsodium documentation. There are legitimate cases for having full control over the nonce, such as the dnscurve and dnscrypt protocols.

Some bindings such as the Swift bindings (Swift-Sodium) have the box() operation generate and return a nonce in addition to the ciphertext. Less flexible than leaving the responsibility to the developer, but definitely safer, and something that might be added to the library itself. On the other hand, adding more and more functions to do the same thing (because breaking backward compatibility is not an option) isn't good either.

</comment></comment><comment><USER17> Can anyone comment on the use of formal verification in NaCl? Is it used at all? Would it even be valuable? [0] doesn't mention anything about it.

[0] [http://nacl.cr.yp.to/valid.html](http://nacl.cr.yp.to/valid.html)

<comment><USER18> There's a section in the tweetNaCl paper that discusses some limited formal verification.

For some interesting commentary on the difficulty in formally verifying c code (with crypto examples) see: [https://www.imperialviolet.org/2014/09/07/provers.html](https://www.imperialviolet.org/2014/09/07/provers.html)

and the followup

[https://www.imperialviolet.org/2014/09/11/moveprovers.html](https://www.imperialviolet.org/2014/09/11/moveprovers.html)

</comment></comment><comment><USER19> Simply put, use the reference nacl implementation only when you _really_ know what you're doing. As with all things crypto it's easy to shoot yourself in the foot, and libsodium makes it a bit harder than reference nacl.

<comment><USER20> If you really know what you are doing you aren't even using NaCl, you use the individual algorithms.

For example: [https://github.com/orlp/ed25519](https://github.com/orlp/ed25519) .

<comment><USER21> If you really know what you're doing, you'd use a fast assembly implementation that's guaranteed to run in constant time, like curve25519/ed25519-donna, both of which you get for free if you just use libsodium.

</comment></comment></comment><comment><USER22> Isn't the fact that nacl is written by such highly reputable guy(in the crypto community) and that it sees relatively lots of review by experts important here ?

<comment><USER5> He's a highly reputable cryptographer, and a ... controversial ... software developer. I would trust my life to the algorithms. I wouldn't trust my appendix to the code.

The software is delivered over HTTP (not HTTPS) and not cryptographically signed. I've always interpreted this as a statement that, if you're not qualified to review what you just downloaded and make sure it's not malicious and that it faithfully implements the algorithms, that download isn't for you.

For what it's worth, Debian considered qmail "so buggy that it is not supportable": [https://bugs.debian.org/cgi- bin/bugreport.cgi?bug=510415](https://bugs.debian.org/cgi- bin/bugreport.cgi?bug=510415) There's a fork called netqmail that's more actually usable.

In a similar way, libsodium is effectively a nacl fork that's actually usable by normal people. I think I've seen more people review libsodium than nacl's code itself (especially the custom assembly versions of various algorithms).

<comment> X-Istence For years I ran vanilla qmail with two patches (AUTH support for sending authentication, and a patch to allow for a database of valid supported emails so you get rid of the backscatter issue).

It was absolutely fantastic, it ran without issues for years. It was very fast and I never had issues with losing mail or behaviours that baffled me or left me scratching my head.

I currently have a postfix setup, and while it works well I am very wary of upgrading it for fear of breaking something and causing mail to bounce or not be delivered. With qmail it would just queue mail if you accidentally misconfigure something (such as the delivery program is unavailable), with Postfix I've had it simply drop mail on the floor. Sure they are issues that could be solved with better testing, and verification of systems, but qmail is a lot more forgiving without bouncing.

<comment><USER23> I also quite prefer qmail to postfix in terms of its flexibility and resilience, but it really does highlight one of the biggest problems with long term use of djb software: It always seems to reach a point where djb feels it's done (or he's done with it) and it sits there for years with no updates, while still appearing to be the authoritative version of that product.

Vanilla qmail is genuinely unusable on the internet today (because of the backscatter issue in particular), but netqmail, which is still maintained, still looks like a secondary fork to someone who doesn't know the history. It doesn't help that djb for a long time used licensing that was incompatible with reasonable distribution of these forks as well.

So these days I actually tend to go for the forks and clones of djb software where possible. I use netqmail on my mail server, runit instead of daemontools, etc.

<comment><USER15> Unmaintained software is a problem and vanilla NaCl also left unfixed issues.

For example, the signature system was a prototype that shouldn't be used any more. The portable AES128 implementation produces incorrect output on some other platforms, one of the Curve25519 implementations performs out of bounds memory accesses, and one of the poly1305 implementations will produce incorrect output if your application changes the FP rounding mode. CurveCP was also a fantastic idea, but the NaCl implementation was just a proof of concept that cannot really be used in actual projects.

But would you rather have djb spend time addressing qmail compilation issues on Ubuntu 14.10, or keep making significant advances in applied cryptography (and security in general) instead?

In addition to organizing competitions, the amount of game-changing publications he made or contributed to is very impressive. And it might not have been the case if he didn't move on from software he wrote years ago.

</comment></comment></comment></comment></comment><comment><USER24> Somehow these and other "gripes" about djb software over the years have never been as interesting or entertaining as the ones djb himself has been sharing with us since the 90's. But complaints are to be expected.

Personally I do not care for libsodium, but to each his own.

Instead I chose to study curveprotect, tinysshd (tweetnacl), qremote, etc. as examples to learn from. Realizing that everyone is capable of making mistakes, but some are less likely than others.

</comment><comment><USER25> Good API design is, like "installability," a critical and often neglected aspect of software engineering. I find that bad APIs are the norm in every area, not just crypto. Audio and video stuff is just hideous.

I used the NaCl crypto for ZeroTier One but basically refactored the code into C++ classes that build like normal modern C++.

</comment><comment><USER26> I thought this was going to be complaining about salt...

</comment><comment><USER27> This title a very subtle trap for those who do not read articles. :P

<comment><USER28> It's amazing how people can comment without reading the article. It's obvious this happens to some degree, but I was amazed that the first few posts in this thread have exclusively been about a completely different thing.

<comment><USER29> The 3rd and 4th comments were on topic. Now half the comments are boring meta discussion of the confusion.

</comment></comment></comment><comment><USER30> > If nacl picks up that your new CPU has some advanced instructions, it will > use them, resulting in a library that won't work on older CPUs. It's hard to > ship binaries when you're uncertain of what CPUs they will run on.

Uhm.. What? Isn't this going exactly against all that nacl was ever designed for?

Also, why can't instructions for one CPU be translated/compiled to instructions suitable for another CPU?

<comment><USER29> The article is about [http://nacl.cr.yp.to/](http://nacl.cr.yp.to/) .

Are you thinking of [https://developer.chrome.com/native- client](https://developer.chrome.com/native-client) ?

<comment><USER31> Ah, quite an unfortunate clash of names given the context. I confused it for Chrome's NaCl too. Although, Chrome's NaCl is short for NativeClient and this one is pronounced Salt based on their description.</comment></comment></comment>


"Egor, stop hacking Github" - <AUTHOR> http://homakov.blogspot.com/2012/03/egor-stop-hacking-gh.html <comment><USER2> Given the recent story on HN about how former YouSendIt founder had taken their servers down to prove their vulnerability [1] [2], I'm surprised how little reverence these "lol-hackers" (that's going to be my term for them) give to showcasing these vulnerabilities by exploiting them in the real-world and messing with people's real things.

I know as hackers, we feel a duty to show people how serious these things are and that we get impatient and annoyed when ignored. And I also know that it's hard for us to reconcile the idea that when we _show_ the owners rather than _tell_ , it's suddenly considered a crime. But it is.

Let's try an analogy. Door locks on houses are ineffective. Think about it. Your house is covered with windows, which are made of glass. Glass is really easy to break. I mean really easy. If you found out your neighbor didn't have a house alarm, you might talk to them and tell them they should get one. If they didn't get one, would you then break into their house one night and walk into their bedroom to show them how dangerous it is?

OK, who knows, maybe you have a weird relationship with your neighbor. Furthermore, this is an imperfect analogy, because here, Rails and Github are both responsible for other people's property.

But now imagine it's a business across town and that you don't actually know the business owner. If you broke into their business to show them their building's security vulnerabilities, you bet your ass they would press charges and I don't think anyone would blame them. Even if you're doing it with the best intentions, it's still vandalism at best.

All of that being said, this is a very effective way of making your point and getting people to fix the problem. That doesn't make it right. But if you're willing to put yourself in harm's way and essentially become a martyr to get these security vulnerabilities fixed, more power to you.

[1] <http://news.ycombinator.com/item?id=3643102>

[2] [http://www.inc.com/magazine/201203/burt-helm/a-silicon- valle...](http://www.inc.com/magazine/201203/burt-helm/a-silicon-valley-tale- of-humiliation-and-revenge.html)

<comment><USER3> Door locks on physical houses are just a small part of a much greater security picture involving your community, observant neighbors, other monitoring systems, the police, even dogs. And yes, houses do get broken into causing a massive amount of aggregate economic loss every year.

But on the internet everyone's front door is accessible from even the most remote and hostile places in the world and you're pretty much on your own for securing it. Internet-facing systems also tend hold far more valuable things than your typical home.

For these reasons, I don't think the household lock analogy works very well.

<comment><USER2> EDIT 2: Hmm, maybe I shouldn't have removed my original response, I just didn't want to derail the conversation upon second thought. For posterity's sake, my original response was something along the lines of this:

\---

I think you may be overestimating the effects of "the greater security picture" involving community, observant neighbors, other monitoring systems for the average community. For example, my first startup was RateMyStudentRental.com, and as I recall adding automated motion-activated sensors and lighting actually had a negative effect on break-in protection. As in, adding such a system actually increased your chances of having a break-in. According to the study, neighbors don't actually pay much attention to someone standing at a well-lit door (mileage varying based on the specific community obviously).

\---

ORIGINAL EDIT: I won't comment on the technicality of the differences between the analogous situation and the actual situation at hand; the differences have nothing to do with the point being made. Don't get caught up in the analogy, because it's just an analogy.

In case it wasn't clear the first time around, the entire point of the analogy was that, breaking into and vandalizing someone's business just to show them that it can be done is generally a bad idea. Though I don't think anyone would argue it isn't effective.

<comment><USER4> _> [...] automated lighting systems actually have a negative effect on break- in protection [...]_

Do you have a source for this? I've always been under the commonly held impression that this has a preventative effect, and it would be very interesting if the opposite really is true.

<comment><USER2> Good question. It was a long time ago, and I'm having trouble googling for it, because all I get in search results are articles and how-tos by motion-sensor companies selling their products. I'll search though and update if I find it.

UPDATE: Hey, awesome, turns out we blogged about it back in the day: [http://blog.ratemystudentrental.com/2008/03/05/a-safe- home-i...](http://blog.ratemystudentrental.com/2008/03/05/a-safe-home-is-a- happy-home-well-it-helps-anyway/)

Here is the original article: [http://www.schneier.com/blog/archives/2007/09/light_and_crim...](http://www.schneier.com/blog/archives/2007/09/light_and_crime.html)

 _Much so-called security lighting is designed with little thought for how eyes -- or criminals -- operate. Marcus Felson, a professor at the School of Criminal Justice at Rutgers University, has concluded that lighting is effective in preventing crime mainly if it enables people to notice criminal activity as it's taking place, and if it doesn't help criminals to see what they're doing. Bright, unshielded floodlights -- one of the most common types of outdoor security lighting in the country -- often fail on both counts, as do all-night lights installed on isolated structures or on parts of buildings that can't be observed by passersby (such as back doors). A burglar who is forced to use a flashlight, or whose movement triggers a security light controlled by an infrared motion sensor, is much more likely to be spotted than one whose presence is masked by the blinding glare of a poorly placed metal halide "wall pack." In the early seventies, the public-school system in San Antonio, Texas, began leaving many of its school buildings, parking lots, and other property dark at night and found that the no-lights policy not only reduced energy costs but also dramatically cut vandalism._

I remembered it incorrectly, as apparently motion-activated lighting helps, but on-all-night floodlighting makes the situation worse.

<comment><USER3> Peter Gutmann reported some interesting studies in his keynote talk at Shmoocon this year. It's probably available on Ustream or the Shmoocon site.

In short, sometimes darkness helps. Consider a tall glass office building. If all the lights are out, nothing stands out to a security guard more than a flashlight waving around in the darkness.

</comment></comment></comment></comment></comment></comment><comment><USER5> This reminds me of how PHP used to turn HTTP request variables directly into global programming variables by default. Now it only happens when you enable the register_globals option. I don't think I've ever met anyone who didn't consider it a huge security issue.

This rails behavior is actually even more powerful than the old PHP one for hackers because with this you get directly into the model and then the DB when everything is still left as generated, not just the temporary variables. It's actually pretty surprising how much resistance there is to fixing the issue.

It could be that the proposed whitelisting isn't the only solution. It does require annoying configuration. With PHP, nowadays, most people just access a particular array when they want their request variables. Similarly, maybe Rails could have a request model object and a DB model object with simple methods for copying state between the two. Maybe combine it into some sort of validation logic with user friendly error messages being specified. I guess it is still more work that default overwriting of the DB with request variables, though.

<comment><USER6> I was also thinking of PHP's register_globals. I was tempted to make a snide remark, so I'll make it now. The difference here is that the PHP group realized register_globals was a bad idea, deprecated it in 5.3 and removed it in 5.4. Furthermore the default has been OFF since 4.2.0. The resistance to fixing the Rails problem just makes me ever less likely to give Rails a shot, it should be really bad PR when you ignore security issues.

<comment><USER7> Welcome to the new decade, where PHP has secure defaults and Rails apps get pwn'd left right and center.

But seriously, PHP still has lots of problems to fix.

<comment><USER8> > But seriously, PHP still has lots of problems to fix.

Like what?

<comment><USER9> <http://phpsadness.com/>

<http://www.reddit.com/r/lolphp/>

Among others...

<comment><USER8> If the best you can do is close to a year stale and doesn't apply to the latest version, I'd say PHP is doing a fine job.

Also, reddit? Really?

<comment><USER10> > If the best you can do is close to a year stale and doesn't apply to the > latest version

Really? 5.4 has fixed the retarded associativity of the ternary and all error reporting?

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> For what it's worth, we'd like it if security vulnerabilities in Rails were disclosed to <http://rubyonrails.org/security>.

Obviously this situation is a bit more complicated, as a ticket was opened up, and a lot of community discussion occurred. In general, emails to the security list are taken extremely seriously.

<comment><USER12> People are reading this as a vulnerability in rails when it is actually a vulnerability in GH's code. Your comment unfortunately is going to add to it :(.

What should he have reported to rails security team? Shouldn't he have been contacting GH security team instead?

Edit: Rails guides discusses the root cause and counter measures against these type of vulnerabilities <http://guides.rubyonrails.org/security.html#mass- assignment>

<comment><USER13> When I read "this vulnerability affects tons of Rails apps," I read that as a security vulnerability in Rails. I'm not a user of the framework, but I've heard "convention over configuration" often enough to think that if this was brought up in the Rails issues tracker, it should be prevented by convention.

<comment><USER3> If it affects "Hello Rails" the canonical example Rails application, then maybe it is arguably a vulnerability in Rails.

[http://guides.rubyonrails.org/getting_started.html#say- hello...](http://guides.rubyonrails.org/getting_started.html#say-hello-rails)

</comment></comment></comment></comment><comment><USER14> Are people misunderstanding this?

It is not a bug. It's an acknowledged part of the framework (<http://guides.rubyonrails.org/security.html#mass-assignment>), although one that could get a developer into trouble without knowing to protect attributes where necessary.

If he'd known that there was something the GH team missed, he should have just brought the issue to them directly. If I realize my neighbor's house is in danger of collapsing because the contractor used the wrong type of wood, I don't bring the issue up with a lumber yard and then knock over my neighbor's house to prove a point when they ignore me.

I agree that it's a problem that many developers aren't aware that they need to protect against mass assignment, but it seems like this dude is totally misunderstanding the entire ecosystem here, and now people are calling him a "hero" because he took advantage of something that everyone already knows.

Big whoop.

<comment><USER8> > If I realize my neighbor's house is in danger of collapsing because the > contractor used the wrong type of wood, I don't bring the issue up with a > lumber yard

Yes, you do. Because the lumber yard sold your neighbor the wrong wood.

If you're going to make a framework for a language, do so in a manner that discourages stupidity.

PHP learned this the hard way quite a few years back. You never make things easier for the end-user at the expense of security. It's time for Rails to learn the same lesson.

<comment><USER15> You can't chop off half of someone's sentence and then debate the leftover part. Not at an "and", at least.

</comment></comment></comment><comment><USER16> If this vulnerability is really due to attr_accessible, then that's got to sting for GitHub as this is a well-known "insecure by default" issue in Rails, and GitHub is probably the most (or one of the most) public Rails app out there.

Would be kind of scary if he had injected some nastiness into the rvm repo master branch, for instance, because I know some people do ride on the master version (`rvm get head`). Or, some gems that are built from Gemfile's pointing at the git repo on GitHub.

Luckily, git itself is quite resilient to attacks on the repo integrity so I don't think there could be much long-term harm done (no rewriting repo history would go unnoticed, for example).

</comment><comment><USER17> Can't decide whether to love or hate this guy. For his young age, he seems to have an impressive skill set, but you can totally tell the douche growing inside

<comment><USER18> I 'm not that douche you want to show me. But it seems so, sorry.

<comment><USER3> Dude, you're a hero.

Others just don't realize it yet because they're going through the stages of grief (Denial, Anger, Bargaining, Depression, Acceptance). <https://en.wikipedia.org/wiki/K%C3%BCbler-Ross_model>

If no one else says it, I will:

Thank You!

</comment></comment></comment><comment><USER19> 'Bout time someone took the Rails "rockstar ninjas" down a peg or two.

<comment><USER20> Have they wronged you in some way?

<comment><USER21> Some people are jealous of success.

<comment><USER22> Rather, note tenor of comments at <http://news.ycombinator.com/item?id=3651958> .

</comment></comment></comment></comment><comment><USER23> This is not a rails bug, but a github bug. The discoverer reported it in the wrong place, the rails people (correctly) told him to report it to github, he ignored them, and this hilarity ensued.

While this is a legit bug on github, and Egor deserves credit for finding it, he also deserves a scolding for the classic noob mistake of not reporting it to the right place.

<comment><USER24> That's a matter of opinion. He intentionally reported it against rails because he thinks it is rails' fault for making it so easy to have this security problem. Given that the #1 rails app out there written by professionals suffers from it, and reportedly others do too, he has a point. You can disagree if you want but it's definitely not a "classic noob mistake".

</comment></comment><comment><USER25> Alright, this did a pretty good job of winning me over to Egor:

<http://homakov.blogspot.com/2011/07/octocat-tattoo.html>

That tattoo is legit.

<comment><USER26> Maybe not? [https://github.com/rails/rails/commit/b83965785db1eec019edf1...](https://github.com/rails/rails/commit/b83965785db1eec019edf1fc272b1aa393e6dc57#commitcomment-1041295)

</comment></comment><comment><USER27> sometimes it's better to do bold moves when some nastiy thing needs attention and noone is really hurt.

</comment><comment><USER28> Dont quite think this is what I'd call "responsible disclosure", and not exactly sure this is "disclosure" either.

<comment><USER29> He made an attempt to patch upstream, was shot down, and then proved his point without hurting anyone. Maybe not the best way to get the issue out, but we are talking about it and it wasn't especially harmful. Mission accomplished?

I don't know how else he could have done this. He was being _ingored_!

<comment><USER30> > He made an attempt to patch upstream, was shot down, and then proved his > point without hurting anyone.

Assuming no economic impact to github you mean. If paying users leave because they feel github is no longer safe for private repos because of this, that is harm.

I imagine a bank would be far more grouchy if someone exploited a vulnerability and deposited one cent into someone's account "just to show there was a vulnerability" then publicized it, without talking to them about it first. <sarcasm>No harm done right?</sarcasm>

Now, I am not saying that it is bad that this github vulnerability was found and fixed. I am very glad! But I think it could have been far more responsibly done.

<comment><USER15> If github users quit because a hacker deletes their stuff, maybe that's harm the hacker did to github.

But if github users quit because a hacker shows them github is insecure, that's harm github did to itself. Don't blame the messenger.

</comment></comment></comment></comment><comment><USER31> tl;dr; - Learn about: attr_protected[1]

[1] <http://guides.rubyonrails.org/security.html#mass-assignment></comment>


Things I wish someone told me when I started with Go - <AUTHOR> https://github.com/monsooncommerce/golang-tripping-hazards <comment><USER2> > and that carries a MAJOR limitation, because the instance of ImplementsFooer > (f) inside your function is a copy of the object you called it on. That > means you cannot manipulate data-members in interface-functions, even inside > your own struct.

You can, by doing it like this: [https://play.golang.org/p/5yXfOf6Y_3](https://play.golang.org/p/5yXfOf6Y_3)

EDIT: I'll just file an issue as I doubt the OP is the author.

<comment><USER3> @rtz12: This is excellent, thank you so much! I'll make the edit. :)

EDIT: (ps, I'm the author)</comment></comment>


"Thank You For Playing" film review - <AUTHOR> http://arstechnica.com/the-multiverse/2015/09/thank-you-for-playing-film-review-a-beautiful-tragic-attempt-to-press-pause/

<comment><USER2> This film steers into "Grave of the Fireflies" territory, something that may be great, but you have trouble watching it once without weeping.</comment>


Swiss to end secret bank accounts - <AUTHOR> http://www.upi.com/Business_News/2009/03/14/Swiss_to_end_secret_bank_accounts/UPI-61721237062256/

<comment><USER3> Switzerland has NOT decided to "end secret bank accounts".

From a BBC News article:

"while it will now abide by international rules on bank data sharing, it said it would only respond to "concrete and justified" requests." <http://news.bbc.co.uk/2/hi/business/7941717.stm>

<comment><USER4> Plus - This isn't really huge news, and the reputation of Swiss bank secrecy is a little outdated - e.g. The US and Swiss authorities have been co- operating around Tax information for some time. Similarly, it's still incumbent on the Swiss to track and act on illegal flows of money (and they do).... It's still far from transparent, but it's not the citadel that some people think.

<comment><USER5> The main reason for this major change is that even in the presence of information-sharing provisions of treaties, Switzerland refused to provide any information on most account holders that were evading US (or other countries') taxes, on the rationale that evading non-Swiss tax was not a criminal offense in Switzerland. Cooperation was virtually nil, especially in the tax arena.

Depending on the scope of the changes, this could be huge news.

</comment></comment></comment><comment><USER6> This is sad. One of the few places where banks still acted in their clients' interest rather than their own or the government's.

All interaction between your lawyer and you or your doctor and you is private? Why not the same for you and your banker?

<comment><USER7> If you conspire with your doctor or your lawyer _to commit a crime_ , then your discussions in furtherance of that conspiracy are not privileged.

<comment><USER6> Most (good) lawyers will keep their mouth shut even if they knew the details of a crime that you committed. They will happily burn any documentation before they hand any over to the government.

Every conversation between you and your lawyer, doctor or priest is confidential – and none of them will rat you out.

Banks (in my country at least) will hand over details of your bank statements without your knowledge to the government.

\---

It is only one part of my problem with banks. Banks generally act in their own interest. When you discuss a contract with a lawyer he will act in your interest. Your doctor will also act in your interest when he prescribes you medicine. A banker is the only entity that will give you "advice" that is more in his interest than yours. A good example is contracts with banks - <USER2> "standard" contract is extremely one sided.

<comment><USER8> I see no reason for secret banking other to subvert laws. Secret banking is an enormous global problem which at its worst enables terrorism and drug trade. At "best" it enables the very wealthy to not pay their fair share. I hope these new changes are the tipping point and within 10 years, there will be no tax havens.

If you have a problem with being overtaxed, the only respectful approach is to be active in managing how your government works. Subverting the process only makes things worse.

And a "good" lawyer should never cover for crimes. It truly is not their purpose and perverts their privilege.

<comment><USER6> > I see no reason for secret banking other to subvert laws.

Replace the word "secrecy" with "privacy". I think there are a lot of good reasons to keep your personal banking details private. For example – in my country there is a law that the bank have to inform a government department if there is any deposit above about $1000. Thinking of donating to a political party?

You must show proof of fixed residence to open an account (even if you always have a positive balance). How would a poor person without a fixed address (as defined by government) open an account?

> and drug trade.

Because people voluntary give money to buy drugs and people voluntary sells drugs. What is the problem?

> at its worst enables terrorism

It is a stretch to blame private banking for terrorism. Terrorism will occur regardless of banking laws.

> At "best" it enables the very wealthy to not pay their fair share.

If there was a flat tax regime you can (maybe) try to make that believable. Why is it that rich people pay a higher percentage of their income in tax? It is simple – ten wolves and a sheep deciding who is for dinner. The whole idea of progressive taxation is the mob taking what it wants because of its majority – there is no inherit "fairness" in it.

A politician buys the vote of the poor with the money from the rich (coerced through taxation).

> If you have a problem with being overtaxed, the only respectful approach is > to be active in managing how your government works.

In my country there are 4 million tax payers and about 14 million people receiving government "grants" (this will probably quadruple in a year's time in a plan to give everyone a "basic income grant" (BIG)). Is it fair that 4 million productive people subsidise 14 million unproductive people? Is it fair that a young tax-paying couple puts off having a child – yet their tax dollars is used for other peoples' child grants?

> And a "good" lawyer should never cover for crimes.

A "good lawyer should always act in their clients' best interest – whatever it may be.

In summary my argument is as follows:

1.There are plenty of reasons why banking should be private. 2.IMHO tax evasion by overtaxed people (i.e. when the tax percentage is progressive) is not morally wrong.

EDIT: Downmod all you want.

<comment><USER9> I generally agree or fail to disagree with most of your post, but on the particular topic of progressive taxation, it's not fair to simply dismiss it as "10 wolves and a sheep": marginal utility is at play.

With all goods, the more of it you have, the less valuable it is. Individuals allocate goods to satisfy their most critical needs first. Ipso facto, the first unit of the good is worth the most, the second unit less so, and so on until the final unit is worth the least.

And money is simply a good, more fungible but otherwise no different than any other. The greatest utility of money is achieved by providing the necessities of life: food, drinking water, shelter. The next greatest utility is achieved by providing the basic comforts of life: showering water, clean laundry, furniture. After that comes longer-term goals, and so on (roughly mirroring Maslow's hierarchy of needs).

Now imagine two people, A and B, who earn $20,000/year and $80,000/year respectively. It's more unethical to take $200 from A than it is to take $800 from B. Why? While taking money from either one should be minimized (or preferably eliminated), in all likelihood A's $200 was going to meet a more critical need than B's $800.

As an argument ad absurdum, imagine that instead of taking away a flat percentage (1% of annual income for both), a flat absolute amount were taken instead ($500). The total amount taken would be the same, but now it would be evenly divided among both A and B, regardless of income. This would be the most fair arrangement of all, would it not? If a progressive tax is less fair than a flat percentage tax, then surely a flat percentage tax is less fair than a flat absolute tax. The flat percentage tax is still "punishing" B for making more money than A, when instead it "should" treat them equally and not care what their incomes are.

Clearly, though, this line of thinking is absurd, because A will be harmed disproportionately. Whereas taking $200 from A might prevent A from paying the electric bill, taking $500 from A might instead prevent A from paying the rent. The result is that, relative to the flat percentage tax, A has lost the fulfillment of a more critical need, while B has gained the fulfillment of a more minor need.

<comment><USER6> > Now imagine two people, A and B, who earn $20,000/year and $80,000/year > respectively. It's more unethical to take $200 from A than it is to take > $800 from B. Why?

Just taking money is wrong. The usual argument is that the government takes money from people to provide a service. If the government takes more money from you and provides less of a service (i.e. it is busy with wealth redistribution) it is wrong - it is simple theft by the masses.

> As an argument ad absurdum, imagine that instead of taking away a flat > percentage (1% of annual income for both), a flat absolute amount were taken > instead ($500).

The argument for flat tax is usually that in order to make money a person uses the infrastructure (roads, ect...). If he makes more money the argument is that he used the public services more. While this is not correct (lower income groups disproportionately use public services) it can be argued as such. There is however no defence for progressive taxation - other than wealth redistribution.

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> Intersting fact: Many US customers want to get a "local" nationality to be no more American so that they can escape from all the US regulation.

</comment><comment><USER11> Off to Singapore, I suppose.

<comment><USER12> You obviously didn't get the memo last week.

</comment></comment><comment><USER13> Whoa ... this means I need to change my account as soon as possible! Where else do they have secret accounts ?!

<comment><USER14> I hear Nigeria is quite the place...

No, on a more serious note, if you need a secret account, something about your attitude towards finances is wrong in the first place.

<comment><USER10> If in some countries you work 6 month (per year) only for taxes, there is something wronger...

If you work 1-2 month (per year) for taxes, the need for a secret account will be null...

<comment><USER14> I don't get it - taxes are a percentage of your overall income... so how can you have to work 6 months for taxes?

I do realize that you have to work more to be able to make a living, but keep in mind that:

\- By working in any state, you are also subject to their rules/laws/customs. That means you have to pay taxes, and you don't pay them for nothing, but you get something in return.

\- If you disagree with the tax system in your country of choice, feel free to move somewhere else. But don't be a jerk and take the advantages (health care, free education up to a certain degree, ... the list goes on) without the disadvantages (giving a (admittedly quite considerable) part of your income to the state in the form of taxes).

So, I rest my case. If you need a secret account, something with your attitude towards finances is wrong. Period.

And on a more anecdotal level, let me say that I am by all means not someone who encourages a very restrictive government in terms of economy. E.g. I'm going to university in Sweden (though not from Sweden), but I'll never work here, simply because, while living here is great, taxes get ridiculously high when you earn more than the average. But tell you what, I'm not going to open a secret bank account, cheat the government and deceive the Swedish people.

<comment><USER15> "I don't get it - taxes are a percentage of your overall income... so how can you have to work 6 months for taxes?"

Well, if the percentage is 50%, then...

More concretely, in Maryland, my last paycheck had about 31% taken from it before I got it, and there are many places with higher tax burdens on the state and local levels, even in the US. Add sales taxes, property taxes if you own a house, and you could easily get to 40% in the US. We're constantly hearing about how much higher taxes are in many other parts of the world, so 50% doesn't seem unlikely for some countries.

The only reason that we can afford this much government, of course, is that we're all so incredibly rich by the standards of yesteryear that the losses have been sustainable so far. We'll see if that continues to be the case (my bet is that it will).

</comment></comment></comment></comment></comment><comment><USER16> I don't get it, how is this relevant to HN audience?

<comment><USER17> It's important to keep in mind that--in the absence of downvotes--scores only mean that content is relatively more relevant than other content surrounding it, not that it is absolutely relevant to our audience. The current voting system encourages diversity, and it's diversity we get.

<comment><USER18> Weekends, in particular, are a whimsical time on HN.

</comment></comment></comment><comment><USER19> Will it help in reviving the economy?</comment>


Should Companies That Use OSS Pay a Tithe? - <AUTHOR> http://www.technologyreview.com/blog/mimssbits/26020/

<comment><USER2> I don't agree. If you want people to pay, give the software a licence that requires people to pay a fee. You can still release the source, you don't need to do anything about it.

There's a thread over on reddit about how major vendors haven't donated to OpenSSH. OpenSSH comes with a highly unrestrictive licence. If the authors don't like it, then that licence might not be right for the project.

If you feel a debt to OSS it's better to contribute back. Looking at what Gabriel is doing in this respect is that he's effectively swapping his time for money, which is fine. But I don't think companies should be required to do this unless it's clear upfront.

<comment><USER3> OpenSSH is a bit of a special case, since (AFAIK) if you donate to OpenSSH some fraction of that will actually be spent on OpenBSD instead. You can imagine that Linux users and vendors aren't interested in funding OpenBSD.

</comment></comment><comment><USER4> OSS isn't a company. If there is a foundation or a person who will do more if you give them money: Then give them money if that will help you.

Lots of OSS is just stuff dumped in github or the like.

Honestly the best contribution is paying your devs to give back patches!</comment>


Rethinking Files - <AUTHOR> https://www.devever.net/~hl/objectworld <comment><USER3> Nice to see folks rethinking files, as they're a scourge on the planet and an antiquated anti-pattern that has been holding back the industry pretty much since its inception. I don't know how anyone could take a look at /etc for example, and consider it anything but archaic. The adduser command is some 1130 lines long, and all it does is do CRUD on files, to name just one example. Then there are countless config files that just have to be edited by hand and happily accept syntax errors and logical errors. No modern system would tolerate this.

The root of the problem with files is that they lack an information model, beyond just a sequence of bytes. They are unopinionated to a fault. All files have structure. Even if that structure is a "non-structure" like "all these files are just a random sequence of meaningless bytes", then that is their structure. But this information isn't present in the system, nor can it be enforced or constrained when that is desirable.

To me, the obvious alternative is the database, aka "everything is a row". We have used the database (relational or otherwise, but mostly relation) to successfully model many many domains, and bring coherence and clarity to them. The cool thing about the relational database is that it's based on an underlying relational algebra. The syntax of data in an RDBMS is really just one manifestation of a deeper layer of structure that is syntax-free, and these abstract structures can be (and are) manifested in multiple coexisting syntaxes.

I'm exploring this pattern ("datafication", headshake) with Aquameta ([http://aquameta.org/](http://aquameta.org/)) and written a lot more about why file-centric is holding us back ([http://blog.aquameta.com/intro- chpater2-filesystem/](http://blog.aquameta.com/intro-chpater2-filesystem/)). Boot to PostgreSQL! :)

<comment><USER4> files can be modified by any text editor in /etc it sounds like you are advocating for a system similar to the windows registry. it can easily corrupt and can't be fixed by live cds or other operating systems that have the filesystem driver. it would be a massive step backwards.

<comment><USER3> I agree the system you describe would be a massive step backwards. :)

No, I'm saying the OS needs an information model as a first-class citizen. But since you brought up data corruption: This hypothetical OS could also benefit from having transaction support up in application space -- to avoid data corruption -- something most "modern" programs don't even have, even though most file systems do.

To be fair, I do think the database needs a way to edit the contents of a field using your favorite text editor. But we've got a pgfs plugin that uses FUSE to make the database accessible as a filesystem as well.

</comment></comment></comment><comment><USER5> > People like Unix's "everything is a file" approach because what it really > means is "everything is exposed to the same nexus". It means you need only > ssh to a system and you have all the power to reshape all aspects of that > system with a single interface, the command line, using a common set of > highly composable tools

But at least in Linux there are ton of files that are _not_ exposed to the "same nexus", i.e. filesystem. The most common example would be network sockets. They are files, but do not exist anywhere in filesystem. In Linux file is more of an object handle.

[https://yarchive.net/comp/linux/everything_is_file.html](https://yarchive.net/comp/linux/everything_is_file.html)

[http://events17.linuxfoundation.org/sites/events/files/slide...](http://events17.linuxfoundation.org/sites/events/files/slides/fd_0.pdf)

<comment><USER6> Yes, people call it the "Unix philosophy" not the "Linux philosophy." And the Plan 9 folks (nee Unix folks) also thought everything in Unix should be a file and not everything was.

[edited for clarity]

<comment><USER7> Instead of calling everything a "file", you might as well call everything an "object" then. It also seems more correct.

<comment><USER8> Calling it object would mean you have different methods of accessing it. But files have a fixed set of methods/properties.

<comment><USER7> Not sure what you mean, because e.g. ioctl() call has many functions which are not sensible for regular files.

In any case, the article has 17 occurrences for "file(s)" versus 28 for "object(s)", so the author seems to agree with me :)

<comment><USER8> Not every file respond to ioctl() and that's not what people mean when they say "unix philosophy". Yes, in a sense ioctl() does model the object nature, but they aren't discoverable and rather adhoc, as the article points to powershell and capability for reflection for how a OOP based resource access would look like.

</comment></comment></comment></comment></comment></comment><comment><USER9> I found the URL addressing scheme of Redox to be fascinating, if perhaps slightly less user-friendly compared to files and file paths.

[https://doc.redox-os.org/book/design/url/urls.html](https://doc.redox- os.org/book/design/url/urls.html)

<comment><USER10> Personally, Redox's use of URLs seemed like really bad design to me. It doesn't get simpler than the Unix path syntax.

Having a scheme:// makes sense for URLs because you don't otherwise have any contextual information indicating _how_ to access a resource. But this isn't the case for something like a virtual filesystem, where the total set of filesystems mounted under it - and their types - are all known to the system. There's no need for disk://foo when you can just have /dev/disk/foo.

<comment><USER11> That's true when the namespace covers objects that are very similar to access, ideally identical.

If that's not the case, I have found the scheme to be helpful to indicate what's going on.

<comment><USER10> On *nix, you can always figure out what type of filesystem is mounted at a given prefix by typing `mount`.

What the use of schemes does is make things needlessly inflexible, and embeds a dependency on the name of a filesystem provider inside consumers of that filesystem. It's akin to a Unix where filesystems can only be mounted in top- level directories /mnt, but not /mnt/foo, etc.; I don't see the appeal.

<comment> yjftsjthsd-h I prefer to use `df -T /path/to/mount`, personally.

<comment><USER12> > I prefer to use `df -T /path/to/mount`, personally.

Why?

<comment> yjftsjthsd-h Lets me specify a single file system; I don't think `mount` does that (unless I'm blind; possible).

<comment><USER12> Mount lets you do this as well.

mount -t type device destination_dir

Unless I am missing something in your use case.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER13> I recommend opening up developer tools and adding this before reading this article:

      body {
          width: 40em;
          margin: 0 auto;
          font-size: 1.4em;
          line-height: 1.4em;
      }

<comment><USER14> Or use the Reader View/Reader mode in Firefox/Safari.

</comment></comment><comment><USER15> What if the interaction were more like OOP - the File class wouldn't necessarily make sense as the top parent.

Would be kind of interesting to call methods on objects rather than read/write files, but it's not immediately obvious to me that that really gains anything over the status quo.

And now that I've written that, I wonder is that what powershell's verb-object does anyway? I've never come close to proficient enough (nor wanted to!) to know.

<comment><USER10> This is pretty much what I mean. Essentially I'm proposing that the "base class" shouldn't support anything other than close(). You could have objects which don't support read()/write(), but custom verbs with different semantics appropriate to their type. Tape drives (and only tape drives) could support a rewind(), etc.

<comment><USER16> I think it might be more helpful o think of what semantics could be supported by the everything is a file operations. A tape drive rewind() is just a specialized version of seek(), which any random access object would need to support.

The file metaphor is soooo flexible, so it's hard for me to think of examples where it breaks down. So, what are some good examples where the file metaphor breaks down? Maybe that's helpful?

<comment><USER10> The trouble is, a tape drive can support seek(). But can it support it performantly for all arguments? seek(0, SEEK_SET) is easy. seek(1024, SEEK_CUR) is easy - just read forward a little. But seeking to some arbitrary fixed offset? As far as I'm aware, tape drives are designed to use filemarks for 'searching', not precise offsets.

Of course seek(n, SEEK_SET) could be implemented anyway, in a very un- performant (and tape-wearing manner): by rewinding, and then reading forward n bytes. There's a question of whether the utility of this is desirable when weighed against how surprising it may be to people who don't realise just how bad the performance will be, especially when a tape drive which only supports seek(0, SEEK_SET) can easily have this behaviour emulated on top in userspace by seek(0, SEEK_SET) followed by dummy reads, if you really want it.

read() and write() and seek() prove remarkably versatile, but the niggles come with the fact that different types of file/device on POSIX can have subtly different gotchas with these verbs which, on the face of it, appear to be the same verb. Essentially, I might argue they're not the same verb at all - <USER2> just seem similar.

For example, read() from a UDP socket and read() from a normal file have extremely different semantics. If you read() with a 64 byte buffer from a UDP socket, the message is truncated and the remainder of it is lost. This is a very different semantic to reading from a file, where you can read in whatever chunks you like.

I wrote the article upon reflection of precisely this attempt to force everything into the straightjacket of everything-is-a-file that we've had for decades with UNIX. How much code correctly deals with short write()s? "Everything can be expressed as an object on which you can perform read()/write()" can only be true if you ignore the details of a verb's precise semantics, but the precise semantics are important. I think it's fair to argue that write() isn't one verb at this point, but an overloaded verb referring to a set of verbs. Which verb in that set you're invoking is dependent on the type of "file".

</comment></comment></comment></comment><comment><USER11> That's kind of the point of stores:

[https://github.com/mpw/MPWFoundation/blob/master/Documentati...](https://github.com/mpw/MPWFoundation/blob/master/Documentation/Stores.md)

and Polymorphic Identifiers:

[http://objective.st/URIs/](http://objective.st/URIs/)

Hierarchical paths were a good idea, let's use them. Objects were also a good idea, let's use those. A small set of verbs (GET, PUT, POST, DELETE) was also a good idea. Let's combine these!

Abstract from:

       Path    + File       + POSIX I/O
       URI     + Resource   + REST Verbs

Get:

1\. Polymorphic Identifiers, which subsume paths, URIs, variables, dictionary keys etc.

2\. Stores, wich resolve URIs, subsume filesystems, HTTP servers, dictionaries, etc.

3\. A small protocol that essentially mirrors REST verbs in-process

See also: In-process REST, [https://link.springer.com/chapter/10.1007/978-1-4614-9299-3_...](https://link.springer.com/chapter/10.1007/978-1-4614-9299-3_11)

<comment><USER10> The whole point of my article was that constraining things to a small, limited set of verbs is actually a bad idea.

Theoretically you can just make up your own verbs for HTTP and use those. In practice people stick to the common ones because they're well supported. This leads to people massaging a problem domain into the straightjacket of GET/PUT/POST/PATCH/DELETE, regardless of how well it fundamentally fits that set of verbs. (I'm also convinced nobody actually knows what "REST" means, but that's another rant for another time.)

<comment><USER17> What other verbs could be needed?

<comment><USER10> Let's suppose that I have an HTTP endpoint for a doorbell (for some reason). The doorbell resource is represented as [http://example.com/doorbell](http://example.com/doorbell).

There's no RING HTTP method, and I could invent one, but heaven knows if various HTTP middleware would be happy with that. In practice, people do something like

        POST http://example.com/doorbell/ring

The problem with this is that you now have a hierarchy of verbs; you have first class verbs (GET, PUT, POST, PATCH, DELETE), and second class verbs which have to be represented as distinct resources. This feels like a hack to me.

<comment><USER17> Ah okay.

But isn't this basically what RPC vs REST boils down to?

As far as I know people tried the RPC way for years then gave up on it and started doing REST. Seemingly because inventing a whole bunch of methods was inherently flawed.

</comment></comment></comment></comment></comment><comment><USER18> Your not solving anything, at best you are getting maybe one extra level of abstraction by shifting _potential_ complexity in the application. It may or may not care about internal file "schema" and thus has no code for it. Shifts to _concrete_ complexity in the system; Your application doesn't utilize file schema but some applications might so everyone gets a schema field and there is a bunch of extra code and complexity to support it.

From a security/reliability standpoint it sounds like a nightmare combining the worst of things like NTFS alternate data streams and share library loading into one.

</comment><comment><USER19> See my earlier comment, [https://news.ycombinator.com/item?id=14542595](https://news.ycombinator.com/item?id=14542595) .

Lotus Agenda/Chandler [https://en.wikipedia.org/wiki/Chandler_(software)](https://en.wikipedia.org/wiki/Chandler_\(software\)) is another part of this long Grail quest.

</comment><comment><USER20> Files are too finite, low-level and lose generate/parsing knowledge that is implemented N times in N places. OSes should read and write message-oriented streams of records (pb, capnp or similar.) that are invisible to the user, while tools and code see data and data structures. This solves many problems of unnecessary and repeated effort parsing log files, log file rotation, proprietary file formats, portability, compatibility and extensibility.

Also, programs should be able to dynamically-serve the contents of "files" as well with an "activation symlink", i.e.,

        /etc/resolv ->* resolvconf

The "the everything must be plain text" refrain is obsolete and unnecessary because it's trivial to serialize anything to any format since it would already be an universally-supported data structure both in tools and code.

It's not 1978 anymore.

<comment><USER21> Sounds a lot like WinFS [https://en.m.wikipedia.org/wiki/WinFS](https://en.m.wikipedia.org/wiki/WinFS)

</comment></comment><comment><USER22> Two sic projects that can help managing files until we get another system

TMSU - tags your files and then access them through a virtual filesystem from any other application

[https://tmsu.org](https://tmsu.org) \-- [https://github.com/oniony/TMSU](https://github.com/oniony/TMSU)

Tagsistant - Semantic filesystem for Linux, with relation reasoner, autotagging plugins and deduplication

[https://www.tagsistant.net](https://www.tagsistant.net) \-- [https://github.com/StrumentiResistenti/Tagsistant](https://github.com/StrumentiResistenti/Tagsistant)

</comment><comment><USER23> The examples given at the end, where verbs are commands at certain paths, looks a lot like a special file system. All the printers are under `/print` and all the print commands are under `/print`. One could imagine all the database tables being under `/db` and all the commands being under `/db/bin`.

</comment><comment><USER24> Another site, can we please just add something like this:

<style xmlns="[http://www.w3.org/1999/xhtml">](http://www.w3.org/1999/xhtml">) body{ max-width: 600px; font-family: "Calibri"; margin-left: auto; margin- right: auto; }

</style>

</comment><comment><USER25> I've done some silly things [0] with python's pathlib recently that seem related to the issues discussed here. Given that smalltalk message passing finally clicked for me durnig the process, I am attracted to an object-like solution for everything (or a file-object-like solution for everything, since the practical performance advantages are undeniable). That said there are some considerations both for the low level implementation, and for high level things like affordances for 'file' operations.

In direct response to the suggestion about file paths for verbs. Allan Kay says in one (possibly many) of his talks something along the lines of 'every function should have a url.' The one of surely many challenges is how to ensure that the mechanism used to populate file system paths with nested functionality (e.g. /usr/bin/ls/all to `ls -a`) don't trigger malicious behavior during service/capability discovery. Being able to more deeply introspect file data and metadata as if the file were a folder could potentially be implemented as a plugin, and I worry about the complexity of requiring a file system to know about the contents of the files that it hosts, or that the files themselves be required to know about how to tell the file system about themselves. Existing file systems adhere to a fairly strict separation of concerns, since who knows what new file format or language will appear, and who knows what file system the file will need to exist on.

Said another way I think that the primary issue with the suggested approach is that it is hard to extend. The file system itself needs to know about the new type of object that it is going to represent, rather than simply acting as an index of paths to all objects. If there is a type of object that is opaque to the current version of the file system that object either has to implement a file-system-specific discovery protocol (which surely would have fun security considerations if it were anything other than a static manifest) or the user has to wait for a new version of the file system that knows what to do with that file type.

Some thoughts from my own work. (partially in the context of OJFord's comment below)

Treating files and urls as objects that have identifiers, metadata, and data portions and where the data portion is treated as a generator is very powerful, but the affordances around the expression local_file.data = remote_file.data make me hesitate. When assignment can trigger a network saturating read operation, or when setter doesn't know anything about how much space is on a disk, etc. then there are significant footguns waiting to be fired and I have already shot myself a couple of times.

The more homogeneous the object interface the better. However, this comes with a major risk. If the underlying systems you are wrapping have different operational semantics (think files system vs database transactions) and there is no way to distinguish between them based solely on the interface (because it is homogeneous) then disaster will strike at some point due to a mismatch. To avoid this everything built on top of the object representation has to be implemented under the assumption of the worst case possible behavior, making it difficult to leverage the features of more advanced systems. As with the affordances around local.data = remote.data, if I have a socket, a local file, a remote web page that I own, a handle to an led, a handle to a stop light, a database row in a table that has triggers set, the stdin to an open ssh session, and a network ring buffer all represented in the same object system, I have as many meanings for file_object.write('something') as I have types of objects, and the consequences and side effects of calling write are so diverse (from flipping bits on a harddrive to triggering arbitrary code execution) that it is all but guaranteed that something will go horribly wrong. At the very least there would need to be a distinction between operations where all side effects could be accounted for beforehand (e.g. writing a file of known length to disk has the side effect of reducing free disk space, but that is known before the operation starts), and operations where the consequences will depend on the contents of the message (e.g. DROP TABLES), with perhaps a middle ground for cases with static side effects (e.g. the database trigger) but that would not immediately visible to the caller and that might change from time to time.

The distinction between files and folders is quite annoying (non-homogeneous), especially if you want to require that certain pieces of metadata always 'follow' a file. This is from working with xattrs that are extremely easy to loose if you aren't careful. Xattrs are a great engineering optimization to make use of dead space in the file system data structure, but they aren't quite the full abstraction one would want. It is also not entirely clear what patterns to use when you have a file that is also a folder -- do you make the metadata the outer file and the data the inner file? Or the other way around? Having the metadata as the outer file means that you can change the metdata without changing the data, but that the metadata will always 'update' when its contents (the data) changes. However, when I first thought about using such a system, I had it the other way around, and a system with that much flexibility I suspect would have even more footguns than the current system.

Another issue is the long standing question around what constitutes an atomic operation. Everything is simple if only a single well behaved program is ever going to touch the files, but trying to build a full object-like system on top of existing systems is a recipe for leaky abstraction nightmares.

While I was working on this I came across debates from before I was born. For example hardlinks vs symlinks. There are real practical engineering tradeoffs that I can't even begin to comment on because I don't understand the use cases for hardlinks well enough to say why we didn't just get rid of them entirely.

0\. [https://github.com/SciCrunch/sparc- curation/blob/master/spar...](https://github.com/SciCrunch/sparc- curation/blob/master/sparcur/paths.py)

<comment><USER10> >and the consequences and side effects of calling write are so diverse (from flipping bits on a harddrive to triggering arbitrary code execution) that it is all but guaranteed that something will go horribly wrong.

This is why I suggest we really need the ability to dynamically add new verbs. POSIX has one write() but in terms of semantics it's really a whole family of verbs as one overloaded method.

>The file system itself needs to know about the new type of object that it is going to represent, rather than simply acting as an index of paths to all objects.

What I had in mind was that a given filesystem driver (e.g. a userspace FUSE process) would provide object types it supports. So for example, a "printer FS" process, printerfsd, would provide printer objects under e.g. /printer/. But the vfs - the layer that does prefix matching on mount()ed filesystems wouldn't need to know about new object types, as it's just a dispatcher.

One shortcoming of this is that you can't mv /printer/foo to another filesystem. That's also a shortcoming of e.g. today's /proc or /sys, but there still seems to be enough that's worthwhile about this approach.</comment></comment>