The end of Framework Churn - <AUTHOR> http://blog.ionic.io/the-end-of-framework-churn/ <comment><USER2> I do not believe Web Components will stop framework churn.

The big problem I have with web components is that they aren't very useful. Everyone seems to think the component is the key to victory with modern web applications. It's true that it's sometimes valuable to have turn-key UI components, but they almost never stand completely on their own. They need to know about the rest of the application and what's happening - and this is where web components fall short.

Imagine a shopping cart indicator on an E-commerce site. (A so-called "mini cart"). These are ubiquitous. When a user adds something to the cart from other places on the page, the indicator (usually in the upper right corner of the page) increases the count displayed, and maybe adds a small picture/description to a dropdown list showing what's in the cart.

Components help with a vanishingly small portion of that interaction. Sure, they allow you to drop a single tag on the page and have the UI look the same. But how does the cart component know to update? Global event bus? Click event handlers? Polling? Callbacks? That mechanism is undefined with Web Components.

Communication _between_ components is the key to winning the web application war, and this is why I think React has done so well. Their data flow/communication pattern with Redux is really understandable and easy to reason about. They've got first class support for passing application state around, and ensuring an action in one place on the page cascades to all places that care about it. To me that is infinitely more important for productivity than stamping out the same custom element over and over.

<comment><USER3> I think the point is more that by using web components, whatever framework you're using becomes more interchangeable. Web components only solve a piece of what, say, React does – the component piece. For state management, you can still reach for Redux (which tries to be framework agnostic), or even still embed them in React if that makes sense for your app.

So maybe web components won't _stop_ framework churn, but they'll make it a lot easier to transition between frameworks.

<comment><USER2> I dunno, I'd rather use "vanilla" React/Redux/JSX than try and get those tools to fit with custom elements, shadow DOM and the html import/template stuff.

The web components specs look poorly thought out to me. They seem superficially like good ideas, but when you actually bother to build something with them you hit the rough edges and missing pieces. I do not appreciate that Chrome rammed them unilaterally down our throats, or that their Polymer evangelists continue to flog these technologies as though they are somehow an answer to the vast majority of problems web developers face.

<comment><USER4> React is an ideal version of web components in a world where ui = f(state)

</comment></comment></comment></comment><comment><USER5> In my experience, the biggest cause of framework churn is bad architecture and development patterns - no clear separation of business logic and 'framework', which forces these all-or-nothing rewrites with the sexy framework of the day in the hopes of a massive productivity boost.

If you can manage to get >50% of your core logic in a 'vanilla' language(js, python, good ol' C) with minimal external dependencies, these yearly framework shifts become much less of a big deal.

I just wish I had internalized this lesson way, way, way earlier in my career...

<comment><USER6> I was once using angular, and I wanted to use a common thing I had done in the past (autocomplete for a form I think it was). I already had a simple and effective way of doing it in jquery, but getting angular and jquery to work together is kind of a pain.

at the time, it wasn't implemented in angular very well, and there were some specific things I wanted to do that was incompatible to the way angular did it.

when I posted to the community how to get it working, their response was 'using jquery is not the angular way'. my way is getting things done with code I have used time and time again.

it took a bit of wrestling, but I was able to get it done with jquery and angular.

</comment></comment><comment><USER7> As I've said before, universal web components are a pipe dream.

Developers disagree on even the most trivial things, like the best way to parse a query string. What makes anyone think those disagreements will magically disappear once web components become a standard?

Sure, give us native APIs that allow developers to build web components. But even then, people are not going to be abandoning their preferred frameworks. Frameworks can change their internals to target the new native APIs, but on the surface, they probably won't change much.

<comment><USER8> The goal of what folks call "universal web components" isn't (or shouldn't be) to create a single universal UI library -- that would indeed be a pipe dream.

Instead, the goal is (or ought to be) to create a native, app-directed API for creating and manipulating UI components. HTML and the DOM are a universal content format and API, and we need its equivalent for web apps (as opposed to content).

The web is a strange dualism between content and apps[1], and unfortunately, the app side has been forced to implement its constructs on top of an API that at its most fundamental level is built for presenting content.

[1] I like the diagram on page 33 of Jesse James Garret's seminal _The Elements of User Experience_ : [http://www.jjg.net/elements/pdf/elements_ch02.pdf](http://www.jjg.net/elements/pdf/elements_ch02.pdf)

</comment></comment><comment><USER9> It's mid of September so this can't be an April fool. Having read this headline I was hoping that they are doing something more 'vanilla': just good old Javascript for the core logic, HTML5 with data attributes describing the markup of an app and CSS3 to for styling.

Instead they announce switching from Angular to Web Components before they ship a stable version of their current framework. It's great for doing really basic applications - but if it's getting into serious business their framework is just too buggy and the incredible demand to rewrite the whole app in pure JS becomes overwhelming.

I disagree that Ionic has found the end of the framework churn. What stays is a mixed feeling of a click bait. At least to me.

</comment><comment><USER10> One of the reasons that React, Vue, and other reactive frameworks have become so prevalent is because of Virtual DOM reducing the number of Physical DOM updates.

Web components are a nice standard, but don't force reactive DOM updates. Web components can contain their own script block and can do whatever updates they want, breaking reactive patterns in apps. In order to keep a reactive app behaving properly, developers will have to vet web components to ensure they work properly with their reactive app. In all likelihood everyone will continue re-implementing components in the reactive framework their app relies on.

<comment><USER11> You can use vdom, lit-html or hyperhtml to force that (so yes everyone will use their favourite tool I guess, and that is a good thing). But individial components can be shared and reused.

</comment></comment><comment><USER12> > _To end Framework Churn once and for all, we need a way to create components > that speak a language every developer and framework can understand. A > component language that can be standardized but is also compatible with > existing framework component languages._

Components are just one thing frameworks do -- among many other things. So the existence of a 'standardized component format' wont do anything to reduce framework churn.

Nor are Web Components going anywhere.

</comment><comment><USER13> The reason for the churn is not lack of a standard for components. It's that the ecosystem is both new and large, so many experiments are going on, some of which become popular until the next best thing comes along. This won't be solved by components. It will be solved slowly over time as innovation levels off.

<comment><AUTHOR> To me custom components feel like one of the "innovations" that bring us more to a state where one part of the equation will be "solved" and there won't need to be any more experiments. It will just work. With all frameworks. And that is great.

<comment><USER14> I look forward to see the Web finally catch up with 90's.

Maybe in a couple of years we get our Delphi and VB component designers and toolboxes for the Web.

</comment></comment></comment><comment><USER15> As much as hacker news frowns on humour replies. I really frankly laughed out loud when I read

"To end Framework Churn once and for all, .... Thankfully, there's an answer: Web Components. "

I was expecting something like "Plain Javascript!" or something fundamental.

This is a content free reply, but I can't help but think this is the beginning of another churn.

<comment><USER16> I had the same reaction because it's ridiculous to claim that Web Components can stop the Framework Churn when they predate most of the frameworks and didn't stop the churn from happening.

There are fundamental shifts happening in UI development, but Web Components are definitely not it.

<comment><USER17> > it's ridiculous to claim that Web Components can stop the Framework Churn > when they predate most of the frameworks

How on Earth do you figure that? I'm well aware that e.g. the <video/> element is basically implemented as a Web Component in most (all?) browsers, but the reason people are excited is that it's now possible for _non-browser_ developers to use Web Components and have those components work just like native elements.

<comment><USER16> > How on Earth do you figure that?

We were discussing Web Components at the time the Angular was getting popular and before React was even released.

> for non-browser developers to ... have those components work just like > native elements

Honest question, how do Web Components help native developers in any way? Because from UI devs perspective, Web Components are useful only in a single edge case - when you must make sure than it's impossible for Cascading Style Sheets to cascade to your component.

For a developer, Web components are solving problems that almost no one has or that were solved better by React and Vue.

<comment><USER17> Do you not count e.g. jQuery as a framework? Angular and React are also pretty late entrants. Before that there was Backbone, Ember, etc.

> Honest question, how do Web Components help native developers in any way?

Being standardized, i.e. any old framework that can work with the DOM can with them, i.e. it reduces the number of truly necessary components to an "N" problem instead of an "NxM" component. Simple as that.

(Let's say we're looking for a "scrolling/sortable table" component. N is the number of different components along the 'features' axis. M would be the number of frameworks (Vue, React, etc.). It's a _massive_ waste of resources that M > 1.)

EDIT: Yes, frameworks will still be around and that's fine.

<comment><USER16> > Do you not count e.g. jQuery as a framework?

Not really, it was used as a library to make JS work consistently across browsers and add missing features like querySelector and ajax fetch. But it had no opinion on how you should structure your app.

Angular, Backbone and Ember is from the same post-jQuery era when talks about Web components started.

> Being standardized, i.e. any old framework that can work with the DOM can > with them

But does a developer really care that it's possible to use the same component in different frameworks, when you need it to work with just one (yours)?

I think the biggest fallacy is that people expect a technology to appear where you could just drop the component and it would work perfectly. From my experience, that will never happen - external component libraries are 90% what you need and 10% of hacking around missing features. The easier it is to safely and predictable hack the last 10% the better the framework/component library is.

Web Components provide nothing that help in UI development:

Helps manage multiple components state? Nope.

Provide a declarative API for components? Nope, imperative - the one we are trying to get away from.

Prevent CSS from affecting components? Cool, or I could just not write global styles, use BEM or CSS modules.

Easy to share components? NPM still works just fine for that.

> It's a massive waste of resources that M > 1

Agreed, that's something I thought a lot about. So I have created a cross- framework visual component editor - one component exports to many frameworks: [https://github.com/UgnisSoftware/ugnis](https://github.com/UgnisSoftware/ugnis)

<comment><USER17> > Angular, Backbone and Ember is from the same post-jQuery era when talks > about Web components started.

Really? Maybe I was just out of the loop. I hope we can agree that Web Components were little more than a gleam in somebody's eye at that point, right?

(I'd love if you could point to any relevant mailing list/google groups discussions or whatever around this time, because apparently I missed 'em.)

> But does a developer really care that it's possible to use the same > component in different frameworks, when you need it to work with just one > (yours)?

Yes! I often have to work with multiple frameworks. Also: It'd be _fanstastic_ if I had a choice matrix of "N" vs. "MxN" when I had to choose an implementation of, say, a scrollable-live-loading-table. That would be amazing, especially since we'd already decided on the "M" axis and written 20k LoC! I could choose the _very_ best feature set for me! Ok, I might have to write a thin "shim" to have it interact with React or Vue.js or whatever, but the point is that this would be a trivial shim. I wouldn't have to rewrite the whole frontend when $BOSS decides that they want SuperLongUberTable.

The mere fact that the contents of such a table are just DOM nodes would also mean that I would be able to publish (as OSS) our own implementation of said beast... to much acclaim and prestige.

I also appreciate the fact that changing "NxM" to "N" means that (eventually) we'll get higher-quality components. For example, the components shipped in your browser are actually quite good. How many "good" video players did you see before HTML5 <video/>? My point is: The fact that anyone can write them means that there'll be a sort of evolutionary competition which tends towards an (admittedly, local) maximum -- but it's not as if the framework world has done much better, so... (Of course this is _still_ at least a little hypothetical. We'll see, but I'm cautiously optimistic.)

EDIT: I might not have articulated it, but I think it goes without saying that I obviously want _all_ components to evolve towards the "stateless-and-emits- events-to-container-components" model of e.g. React... and I think that _will_ happen because it's simply easier to integrate into _any_ framework.

<comment><USER16> > I'd love if you could point to any relevant mailing list/google groups > discussions or whatever around this time

I heard about Web Components spec in late 2012. There are some blogs from around that time: [https://hacks.mozilla.org/2013/05/speed-up-app- development-w...](https://hacks.mozilla.org/2013/05/speed-up-app-development- with-x-tag-and-web-components/) In five years not much has changed.

> Yes! I often have to work with multiple frameworks.

There is an easier solution to this than Web Components - using a single framework. Unless you are a consultant same as me, than we can only blame ourselves for having to deal with this.

> My point is: The fact that anyone can write them means that there'll be a > sort of evolutionary competition

That's not what happened with React component libs (material-ui, react- toolbox, etc.) - none of them has "won", all of them are pretty similar in quality. I would say that quality of a component is less important than how easy it is to adapt them to your business needs is.

My point is that having more ways to create a component is irrelevant - Framework churn existed not because we couldn't share components, but because writing components and managing state was awful. Web Components are still terrible at both of these things.

The good thing is that there hasn't been any new JS innovations in three years since React, Cycle, Elm, ClojureScript boom. And I would claim that there is not going to be because we are limited by the textual representation of our programming language. So the Framework churn ended not because everything is perfect, but because we couldn't make UI development better.

<comment><USER17> > There is an easier solution to this than Web Components - using a single > framework.

Yes, we all know how easy that is. I mean, really, come on -- that's not realistic in any reality.

EDIT: We all know the "n+1" standards XKCD joke, but the reality is that standards (HTML, CSS, HHTP, DNS, TCP, UDP, IP, etc.) brought us the fucking Internet, flawed as those standards may be. As I said, I'm cautiously optimistic that standardizing "custom DOM" will eventually get us to a standard slightly above raw HTML.

<comment><USER16> > I mean, really, come on -- that's not realistic in any reality.

Huh? Most projects use a single framework in a project, sometimes two if they are migrating. If I had multiple frameworks in my project, sharing components between them would be the least of my problems.

> standardizing "custom DOM" will eventually get us to a standard slightly > above raw HTML

I understand this sentiment, but DOM is irrelevant to front-end work right now. DOM is just a render target for frameworks, making DOM smarter changes nothing.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER18> I've been using the same JS framework for 3+ years. I don't agree with all of these complaints on JS churn being too much. Web components aren't the answer either.

</comment><comment><USER19> Web Components solve a number of problems with other component libraries such as React.

React components are brittle. The brittleness comes from the global nature of HTML, CSS, and JS. The DOM tree inside a React component isn't encapsulated from the rest of the page. This lack of encapsulation means your document stylesheet might accidentally apply to parts inside the widget; your JavaScript might accidentally modify parts inside the widget; your IDs might overlap with IDs inside the component; and so on. This means React components cannot be safely reused.

Web Components solve this problem. What's more, you can use JSX with Web Components (to implement Web Components as well as to use Web Components). See here: [https://github.com/wisercoder/uibuilder](https://github.com/wisercoder/uibuilder)

<comment><USER20> I was really pumped about Web Components at first, largely because this sort of argument made a lot of sense to me. A few years on, I'm a lot less convinced that these are the problems that need to be solved.

Element IDs, for example, are useful if you're doing a kind of old-fashioned select-and-mutate approach to DOM management. But if you're using React idiomatically, you're never running query selectors or ever reading from the DOM at all (modulo trivial things like getClientHeight in lifecycle methods). It's hard to imagine when you'd have an ID conflict with React because it's hard to imagine when you'd need an ID.

Similarly, over the past few years I've had maybe two or three cases where CSS leaks due to under-encapsulation were a problem, and I was able to pick up on them and solve them with very little effort. Over the same period I've devoted much, much more effort to solving problems with over-encapsulation in cases where I needed to style a supposedly reusable component but some clever attempt at encapsulation (e.g. inline styling) prevented me from doing so and forced me to do a lot of work or re-implement the component wholesale.

I've known a few projects that were excited about things like Shadow DOM and went through a lot of work to incorporate them only to end up stripping them out (see Atom: [http://blog.atom.io/2016/11/14/removing-shadow-dom- boundary-...](http://blog.atom.io/2016/11/14/removing-shadow-dom-boundary- from-text-editor-elements.html)) for related reasons.

So at this point I see a lot of the touted benefits to things like Web Components as a potentially painful set of solutions to a set of problems I've never really had. I don't really get it, but maybe they scratch some itch I don't have.

<comment><USER19> Here's an example of why you need IDs: [https://stackoverflow.com/questions/29420835/how-to- generate...](https://stackoverflow.com/questions/29420835/how-to-generate- unique-ids-for-form-labels-in-react)

CSS name clashes are an issue if you have to incorporate independently developed React components.

<comment><USER20> I guess I'm not saying that there aren't conceivable situations where you'd need an ID - it's just that they're typically small issues that are easily worked around. A duplicated "for" attribute problem is going to come up when you have two forms on a page and both of them use labels in this way and use conflicting IDs. There's a potential collision, but it's relatively easy to foresee and prevent, I've never seen it come up in practice and the worst outcome when it goes wrong would be something like the user's input focus going somewhere unexpected if they click a label instead of a control.

W.r.t. CSS name clashes, yeah they can come up especially in the contexts of independently developed React components. But in my experience over- encapsulation comes up far more often in the same context. I'm almost never in a situation where I want to slot a component in as-is. Instead, I want some find grained control over its styling or behavior, and if that's hidden behind an encapsulation barrier, that makes re-use even harder.

I do think the React component paradigm has some issues, but IMO they're the same sort of issues that come up in a lot of OO designs where abstraction and encapsulation happen in the wrong places. I still feel like Web Components solves some re-use problems I rarely if ever see in practice while doubling down on ones that I do.

</comment></comment></comment></comment><comment><USER21> Web components are syntactic sugar and a registry (yes, Java coming to a browser near you!) around the following:

        interface Component { void render(); }

If you think this somehow solves all JS framework issues, I have a bridge to sell you. Or there was never a meaningful problem in the first place.

</comment><comment><USER22> OK, so we'll spend a year moving the unwieldy multitude of frameworks to instead be inside web components that will be distributed just as wantonly as frameworks are now. What's really gained there?

</comment><comment><USER23> As long as there's something to gain for an aspiring developer by creating a _new_ framework, there will always be churn as there will always be new frameworks. Rather than reducing churn we should aspire to make it less and less painful to the point of irrelevance.

Some ideas:

Standardization of certain APIs (see the craziness in the Javascript world regarding promises, though that night not even matter with 'async')

</comment><comment><USER24> >The era of Framework churn is coming to an end, and now we can all go back to working on what makes our apps unique instead of spending precious time, energy, and money jumping to something new every year.

I believe this to be naively optimistic, but I'm all for Web components or any other development that help simplify things a bit. It's just that there will never be any silver bullet. I can make a list of the types of "The era of <Problem X> is now over and from here on it will be smooth sailing forever" that I've witnessed followed by the introduction of <Problem Y> that couldn't be anticipated. In a world of exponential complexity and creativity, I don't think this cycle is avoidable. It seems that the best we can hope for is that the rate that initiatives of abstraction, consolidation and standardization will manage to keep up with the forces (market and other) of defragmentation and growing complexity.

</comment><comment><USER25> Web components don't take care of the full ecosystem that is the front-end and will prove irrelevant to framework churn.

If anything, web components make framework hopping marginally easier resulting in a more likely (marginally) exodus of a framework when something new and shiny comes by.

Front-end framework churn is ultimately a result of the ever-growing Javascript community and its relatively fast-paced development. Because we now treat the front-end as seriously (and formally) as we treat the back-end and server/service stack, we will see the constant invention/innovation/adoption cycle that we see in these aspects of technology.

Framework churn is here to stay until browsers change the way the front-end works altogether.

</comment><comment><USER26> I've been wanting to use Web Components since Polymer was pre 1.0. But, browser support still has a ways to go. I think we would all love a solution that has all our dependencies baked into the browser.

But, until that actually happens, I'll carry on using Vue.

</comment><comment><USER27> They say history repeats itself, and I feel we have been here before. Is the web framework landscape not a little like the Windows desktop in the 90s? What does history say will happen next?

</comment><comment><USER28> Considering how jarring the change from Ionic to Ionic 2 was (we haven't yet had time to migrate our 30k line app) I'm kind of worried how the upgrade path looks from Ionic 3 to 4. Moving forward is good, I suppose. But I would have appreciated some information on what this means for us that have yet to migrate to Ionic 2/3\. Should we wait for version 4 to be completed?

Anyone have any input for a confused developer?

<comment><AUTHOR> From what I have seen and heard, upgrading to v4 will be a bit more work than from 3.5 to 3.6 - but not nearly as big as the v1->v2 transition.

Here is a currently working link to a branch where this is developed (pre- alpha, so can change of course): [https://github.com/ionic- team/ionic/tree/core/packages/ionic...](https://github.com/ionic- team/ionic/tree/core/packages/ionic-angular) & [https://github.com/ionic- team/ionic/blob/core/packages/ionic...](https://github.com/ionic- team/ionic/blob/core/packages/ionic-angular/BREAKING.md) As I see it mostly the syntax and attributes of components were cleaned up.

Ionic Framework is still fully based on Angular, it will just be using Custom Components (bundled in a package called @ionic/core) inside instead of the "Angular built Components" (no idea how to name these...) in v2+3. These custom components are built using the Stencil compiler by the Ionic team, but as a Ionic Framework user you won't (have to) come into contact with that at all.

These new components can be used in other frameworks like React etc as well - or even AngularJS (Ionic v1!) or plain Javascript.

An alpha of v4 is planned pretty soon I hear.

<comment><USER28> Thanks, that's comforting to hear.

</comment></comment></comment><comment><USER29> We have the revolutionary answer to framework churn! Nothing will ever be the same! Ya'll ready for this!

Behold! Web Components!

Can someone please queue the Picard facepalm meme?

</comment><comment><USER30> Or you could just ditch HTML and XML, and use only JavaScript and CSS. You still use elements like input, button img, but under an abstraction layer.

</comment><comment><USER31> Tag based interfaces? Sounds kind of like client-side Coldfusion.

</comment><comment><USER32> You should never be using frameworks. [http://www.catonmat.net/blog/frameworks-dont-make- sense/](http://www.catonmat.net/blog/frameworks-dont-make-sense/)

<comment><USER33> This is an extremely short sighted article. It points out potential weaknesses of frameworks and then concludes unilaterally that they should never be used. This doesn't help anyone. Without the Rails framework my job would be significantly harder since I'd have to piece together several libraries for ORM, HTTP requests, DB Connections, etc. This would be significantly _more_ cumbersome than just using a framework, since any one of those libraries could go defunct just like a framework could. My only other option if I can't use a framework is to write my own implementation of _every single core feature_ of rails. No thank you.

Use frameworks when they are appropriate, stand on the shoulders of others who have made significant contributions to the community.

</comment></comment><comment><USER34> _To end Framework Churn once and for all, we need a way to create components that speak a language every developer and framework can understand. A component language that can be standardized but is also compatible with existing framework component languages._

[https://xkcd.com/927/](https://xkcd.com/927/)

<comment><USER35> I'm downvoting this because this is a knockout argument: People use this comic way too often to shush those of us who want to have a debate about whether a standard would be useful in a particular situation. I don't think this was Randall's intention when he did this comic.

<comment><USER12> > _People use this comic way too often to shush those of us who want to have a > debate about whether a standard would be useful in a particular situation_

And more often than not, they are right.

In this particular case Web Components only solves part of what frameworks do, and solves it badly. As badly as CSS solved the layout problem the last 2 decades.

I wouldn't mind a nice Web Components solution, but the current design reminds me of the over-engineering of things like Swing and JSF.

<comment><USER36> > And more often than not, they are right.

And yet, every time that the number of standards is reduced, it is by this same process.

So, yes, it is the one way to do it. And yes, more often than more, it fails completely. So, let me ask you, what is the point of complaining that somebody is going through that?

<comment><USER12> > _And yet, every time that the number of standards is reduced, it is by this > same process_

Is it? I'd say it's usually rather because:

(a) Some of the competing standards wither and we get rid of them.

(b) Some standard emerges as the winning one and is adopted by everybody.

But very rarely by the addition of a new standard.

<comment><USER36> So, some standard emerging does not require creating standards?

And for competing standards to wither, there somebody must have created the other standards too.

<comment><USER12> > _So, some standard emerging does not require creating standards?_

I wrote "emerges as winner". Not "emerges period".

Which does not require creating a NEW standard on top of existing ones (N+1), since an already existing standard (of N) can emerge as the winner.

</comment></comment></comment></comment></comment></comment></comment><comment><USER37> This is totally a shame.

I'm watching the conference where there are saying that "that is not yet another frameworks" but come on, that IS TOTALLY A NEW JS FRAMEWORK.

Let tell to a recruiter that you work 3 year on angular 1, 2 year on angular 2, but no really you feel you are ready for web component but you already failed your job interview, because you are a distancied old programmer working on old'good angular 4.1.3

Guys at ionic, you should work to correct your current version totally buggy (I know what I say, I will soon release my first app that I worked 5 month full time on)

Next time for sure I will stick with angular cli and a naked cordova

<comment><AUTHOR> Ionic Framework users won't come into contact with Stencil at all, just with the custom components built using it. The old `ion-textarea` (for example) will be replace with a new one that was compiled with Stencil.

So no, this is not a new JS framework that will be relevant to framework users.</comment></comment>


Writing a file system from scratch in Rust - <AUTHOR> https://blog.carlosgaldino.com/writing-a-file-system-from-scratch-in-rust.html <comment><USER2> I recently wrote a very simple and naive filesystem in rust for a toy OS I'm building and it was quite an interesting thing to do: [https://github.com/vinc/moros/blob/master/doc/filesystem.md](https://github.com/vinc/moros/blob/master/doc/filesystem.md)

Then I implemented a little FUSE driver in Python to read the disk image from the host system and it was wonderful to mount it the first time and see the files! [https://github.com/vinc/moros-fuse](https://github.com/vinc/moros- fuse)

</comment><comment><USER3> I have read down to the implementation section, but for my money, this is the best way to describe the high level function and behavior of a filesystem that I have ever seen.

<comment><USER4> A very accessible (though dated) intro to filesystems is Practical File System Design, by Dominic Giampaolo.

PDF link: [http://www.nobius.org/practical-file-system- design.pdf](http://www.nobius.org/practical-file-system-design.pdf)

<comment><USER5> Is he the guy who did the BeOS filesystem?

<comment><USER6> Yes. Later went to Apple and did Spotlight.

[https://en.wikipedia.org/wiki/Dominic_Giampaolo](https://en.wikipedia.org/wiki/Dominic_Giampaolo)

<comment><USER7> And APFS: [https://developer.apple.com/videos/play/wwdc2016/701/](https://developer.apple.com/videos/play/wwdc2016/701/)

</comment></comment></comment></comment></comment><comment><USER8> There's also this file system chapter from a series on writing an OS in Rust: [http://osblog.stephenmarz.com/ch10.html](http://osblog.stephenmarz.com/ch10.html)

<comment><USER9> And for code there is TFS [https://github.com/redox- os/tfs](https://github.com/redox-os/tfs)

</comment></comment><comment><USER10> Shameless plug. I did similar in my OS course. But, in C. Github: [https://github.com/immortal3/EbFS](https://github.com/immortal3/EbFS)

Warning: Terribly written. many hacks.

<comment><USER11> Soooo... how does it work?

I'm not asking about the structure or how it's organized. I mean... is the filesystem in a file or... how?

Background: I mostly do embedded stuff so at a glance I would have expected low level primitives (like, HW interactions, registers and stuff) but I see none. So maybe, my expectation, when tacking a problem, of interacting with the HW directly, does not stand in modern environments.

Even better, but unrelated question... how the heck does a x86 OS request data from the HDD?

<comment><USER12> You'd presumably have some "block device" abstraction between your filesystem and your device driver. Don't want to re-implement a FS for each type of hardware. On a Linux system, you can read, eg, /dev/sda1 from userspace, which is what it looks like this filesystem probably does.

As for how you actually request data from the hard drive: There's older ATA interfaces, and BIOS routines from them, which I suspect is what most hobbyist OSes would use.

A more modern interface is AHCI. The OSDev wiki has an overview, where you can see how the registers work: [https://wiki.osdev.org/AHCI](https://wiki.osdev.org/AHCI)

</comment></comment></comment><comment><USER13> Always fun to see this type of work. I notice the usage of OsString, and it made me wonder: does the way an OS encodes it's strings potentially make this FS non-portable between OSes? If I want to mount a drive formatted with this FS, would the OsString be potentially non-portable?

There was a lot of discussion in the past around TFS [https://github.com/redox-os/tfs](https://github.com/redox-os/tfs), my understanding is that effort has kinda lost steam.

<comment><USER14> This is really cool, I wish someone would fund it.

<comment><USER15> That dead[1] project?

Actually everything around "Redox" looks like:

[https://gitlab.redox-os.org/redox-os/tfs/issues/66](https://gitlab.redox- os.org/redox-os/tfs/issues/66)

[1] [https://gitlab.redox-os.org/redox-os/tfs/issues/80](https://gitlab.redox- os.org/redox-os/tfs/issues/80)

<comment><USER16> Redox is still very much active...

[1] [https://gitlab.redox-os.org/groups/redox- os/-/activity](https://gitlab.redox-os.org/groups/redox-os/-/activity)

[2] [https://www.redox-os.org/news/](https://www.redox-os.org/news/)

</comment></comment></comment></comment><comment><USER17> It would be nice if the intro had a brief explanation of why a disk needs to be divided into blocks. Otherwise, I really enjoyed this read from the perspective of a lay person.

<comment><USER18> > It would be nice if the intro had a brief explanation of why a disk needs to > be divided into blocks.

One reason is that HDDs simply don't have a byte-wise resolution, so there's little point talking to HDDs in sub-sector units. Sectors are usually 512 bytes to 4k.

A second reason is being able to simply address the drive. Using 32b indices, if you index bytewise you're limited to 4GB which was available in the early 90s. With 512 bytes blocks, you get an addressing capacity of 2TB, and with 4k blocks (the AF format), you get 16TB. In fact I remember 'round the late 90s / early aught we'd reformat our drives using higher-size blocks because the base couldn't see the entire thing.

<comment> jagged-chisel > HDDs simply don't have a byte-wise resolution

Sufficient explanation for code. Now why is it that disks lack byte-wise resolution?

<comment><USER19> .. And does it hold true for ram disks?

<comment><USER20> Yes it would, but you would be working in cache lines rather than disk blocks.

When working with RAM your PC will transfer a cache line of memory from RAM to your CPU caches. Again this is a feature of hardware limitations and trading off granularity against speed.

Your CPU operates many time faster than RAM so it makes sense to request multiple bytes at once, then your RAM can access those bytes, and line them up on it's output pins ready for your CPU to come back and read them into cache. This give you better bandwidth. (It's a little more complicated than this because the bytes are transferred in serial, rather than in parallel, but digging into the details here is tad tricky).

On the granularity point, the more granular your memory access pattern is, the more bits you need to address that memory. In RAM that either means more physical pins and motherboard traces, or more time to communicate that address. It also means more transistors are needed to hold that address in memory while you're looking it up. All of those things mean more money.

And before we start looking at memory map units, that let you do magical things like map a 64 bit address space into only 16GB of physical RAM. Again the greater the granularity, the more transistors your MMU needs to store the mapping, and thus more cost.

So really the reason we don't have bit level addressing granularity is ultimately cost. There's no reason you could build a machine that did that, it would just cost a fortune and wouldn't provide any practical benefits. So engineers made trade off to build something more useful.

</comment></comment></comment></comment></comment><comment><USER21> Is there any advantage in writing a custom file system for a niche purpose? It seems like most file systems are just different variations of managing where/when files are written simultaneously. Could a file system written specifically for something like PostgreSQL cut out the middle-man and increase performance?

<comment><USER22> Yes. Oracle has done this (ASM) to eliminate overhead, implement fault tolerance and provide a storage management interface based on SQL, for example.

I once made a 'file system' to mount cpio archives (read-only) in an embedded system. Cpio is an extremely simple format to generate and edit (in code) and mounting it directly was very effective.

<comment><USER23> I suspect operating on block storage directly may both be easier and more reliable for databases, since about 75 % of the complication in writing transactional I/O software is working around the kernel's behavior.

<comment><USER24> Kernel's fsyncing behavior is one thing, but just relying on a massive amount of fragile C code running in kernel is a significant liability, especially if your software is a centralized database and crashes, panics will bring down everything.

<comment><USER23> Yes, and also the traditional answer was that the kernel handles weird and complicated hardware and can talk to RAID controllers properly, but nowadays hardware has much less variance, and RAID is rare (and arguably unnecessary for a direct-IO database).

I think it'd be viable for an enterprise-y database to do IO directly over NVMe. Imagine the efficiency and throughput gains you could get from a database that (1) has a unified view of memory allocation in the system (2) directly performs its page-level IO on the storage devices.

</comment></comment></comment></comment></comment><comment><USER25> We've attempted this as well and it's not as simple as it seems. The issues we've run into have made us reconsider porting our FS handlers to Rust, although we are cautiously optimistic about later results.

<comment><USER26> Any more specifics?

</comment></comment><comment><USER27> My dream is to add enough type parameters so in-memory collections can also work as (not horribly tuned!) on-disk datastructures.

It's a nice ambitious goal which can really drive language and library design.

</comment><comment><USER28> I'd be curious to experiment with a file system where all of the file and path metadata is centrally stored in a sqlite blob. Is sqlite fast enough for dealing with file system metadata requests?

</comment><comment><USER29> Something like bcachefs could have been written in Rust.

</comment><comment><USER30> Once you have the file system, and a scheduler, don't you have a basic rudimentary operating system?

How soon until someone builds an Operating System developed in Rust? Maybe make it microkernel-based this time.

<comment><USER31> > _How soon until someone builds an Operating System developed in Rust?_

Redox[1] has been around for almost as long as Rust has. I first heard about it 4-5 years ago.

They had an interesting competition a while back challenging people to figure out how to crash it.

1\. [https://www.redox-os.org/](https://www.redox-os.org/)

<comment><USER30> Yeah, I heard about this project. But there's a graveyard of dead OS projects out there.

What's the progress and potential of Redox?

<comment><USER32> Redox is being worked on continuously. They just added support for gdb.</comment></comment></comment></comment>


SQL is Agile - <AUTHOR> http://lucumr.pocoo.org/2012/12/29/sql-is-agile/ <comment><USER3> At a past job we had a really fun interview question. What we did was laid out a concrete problem, and asked the interviewee to lay out a schema. Since we know the problem well, for virtually any schema we'd be able to invent reasonable requirements that would force them to change their schema. At the end we'd verify a basic knowledge of SQL itself by asking them to answer a couple of basic questions.

(The actual problem we had them tackle was reports to use in making picks in a fantasy sports game. But it could be anything simple to explain, fairly straightforward to solve.)

The reason why the question worked so well is that this was, in fact, pretty much how we build CRUD applications. We'd decide the schema, and once the schema was known, the CRUD application was fairly obvious and straightforward. And a large part of the programming model would be immediately provided by your ORM once you had done so.

<comment><USER4> There's no difference between this and designing the classes first.

Designing the classes first in an explicitly typed language actually works a lot better than designing a schema in a DB as you're not constrained by relational requirements and it's easier to change imo. Make a change to your classes and you can usually just right-click refactor, try and make a change to your SQL schema and it won't accept it at all until you unhook all the dependant tables.

Basically, I don't really get what point you're trying to make.

You _never_ know your domain well enough to make a decent schema unless you've already written the program once before. Which is a pointless thought exercise where you get to point at flaws in someone else's nascent design. Like it sounds like you do in an interview, which is great for seeing their thought processes, but don't fool yourself that it's because of that design method.

In other words, the reason it worked so well is because you had already built it once, nothing at all to do with your choice of schema first.

In the end do what works for you, but schema first vs classes first is a trivial and pointless argument as both work fine.

<comment><USER5> Having done this for a while, I can solidly say that the relationships and structure of data has been consistently better when designed as a set of schema tables than as a set of classes _because_ they are especially constrained. The constraints forced people to actually think about the relationships between facets of their data, and the resulting classes were much cleaner and more maintainable. Should we ever decide to move aspects of the classes into a database, the transition requires little code refactoring; this comes up fairly often for us, since we are trying to give users more customization powers.

People _should_ be doing that when they design classes, but the flexibility tends to work against them. It could just be a mindset that makes it work well, and this might be a more appropriate approach for larger projects than for smaller types.

<comment><USER6> In my experience, relational modelling works fine for domains containing simple logic.

As soon as you start having constraints such as "from date must be before to date" and "the quote must have at least one line, and these lines should sum to > 0 and < 1000000" then relational modelling fails. Hard.

I like to use a Quote with QuoteLines as an interview question. Relational modellers make two tables, both having unique identifiers. Domain modellers create two classes, sometimes exposing only one to the outside world. Only one of those classes (Quote) has an identity.

Domain/class models tend, from what I've seen in the wild, to fail faster under load and under change. Ironically, experienced relational modellers tend to build cleaner class models. Although that might be more a function of experience than anything else..

<comment><USER7> > As soon as you start having constraints such as "from date must be before to > date" and "the quote must have at least one line, and these lines should sum > to > 0 and < 1000000" then relational modelling fails. Hard.

Really? Both those problems are insanely trivial to solve with one or two SQL statements. Your second and third criterium are _perfect_ candidates for relational modelling; the first is so trivial it beggars belief.

<comment><USER8> Enforcing that a quote has at least one quote_line is pretty tough, at least in Postgres, and involves triggers and locks.

How do you enforce that two transactions don't delete the last quote_line at the same time?

And how would you enforce that the sum of the quote_lines is correct with multiple writers?

Not saying it can't be done easily, but I'd love to hear how.

<comment><USER9> "Enforcing that a quote has at least one quote_line is pretty tough, at least in Postgres, and involves triggers and locks."

The reason it seems simple when using a few classes is because there is an implicit assumption that concurrency is a non-issue and there's only one application involved. But those aren't good assumptions, so the approach using classes will start to look more complex (and involve locks, etc.).

In Postgres, the trick is to use SERIALIZABLE transactions everywhere (should be the default, eventually), which avoids the need for explicit locking. Then, add a trigger that is fired before changes to either table, and it would simply check that the condition holds for the quote that was modified.

Here's some code, since you asked:

      create table quote(
        quote_id int8 primary key,
        customer_id int8,
        valid daterange
      );

      create table quote_line(
        quote_id int8 references quote(quote_id),
        amount numeric(10,2)
      );

      create or replace function check_quote() returns trigger language plpgsql as $$
      declare
        line_count int;
        line_sum numeric;
      begin
        select into line_count, line_sum
          count(*), sum(amount) from quote_line where quote_id=NEW.quote_id;
        if (line_count < 1 OR line_sum < 0 OR line_sum > 1000000)
        then raise exception 'invalid quote';
        else return NEW;
        end if;
      end;
      $$;

      create constraint trigger quote_check_trig after insert or update or delete on quote deferrable initially deferred for each row execute procedure check_quote();
      create constraint trigger quote_line_check_trig after insert or update or delete on quote_line deferrable initially deferred for each row execute procedure check_quote();

Note that I did not need to add the CHECK constraint, because it's much better to use the appropriate data type -- DATERANGE -- instead of hacking it together from parts.

<comment><USER8> I figured that serializable transactions would be necessary. That's going to be the default?

If you forget to use serializable transactions every time you update these tables, is it possible to store incorrect data?

<comment><USER9> I think that SERIALIZABLE should be the default eventually, but others may disagree and I won't make a prediction.

However, it isn't necessary to solve the problem. It would be relatively easy to use a row lock in this case to solve the problem, as well, but I like to avoid those unless there's a reason.

If you want to have a mix of SERIALIZABLE and other transactions, or you are worried about making a mistake (or some malicious user), then you need to use the row lock. Eventually there should be a way to force users into serializable transactions.

EDIT: actually, in the trigger, you could explicitly check if the transaction isolation mode is serializable. That would be the best approach:

      if current_setting('transaction_isolation') <> 'serializable' then
        raise exception 'serializable mode required';
      end if;

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> There are no schema-less databases. There are databases where the schema is applied at write and there are applications that apply a schema at read.

<comment><USER11> Very true. I always chuckle when some NoSQLer tells me how much time he's saved by not having to define and update a schema. Then he tells me how all he has to do is check whether fields exist or not in his code, and enforce constraints, and provide default values, and so forth. He has to do this in the Ruby code that he's writing, and his colleague has to do the same in the PHP code she's writing, and then there's the analysis team who has to write the very same code in their Python scripts...

<comment><USER12> I'm not a big fan of either of those approaches.

The cross-implementation schema constraints in SQL DBs are pretty weak. I can only express a small fraction of my application logic there unless I want some serious vendor lock-in.

And then regardless, once I'm referencing the same schema from multiple code bases, schema improvements are basically impossible.

If I need to get at the same data from multiple code bases, I think an API is a lot cleaner. And then I'd rather put the constraints in my API code. The tools are so much better, and programming languages are much more flexible.

<comment><USER11> I find your arguments somewhat contradictory.

So you want to avoid vendor lock-in, but you see NoSQL databases as being better in some way? They're far worse. If you initially target MongoDB, that's what you're going to be stuck with. You aren't going to be moving to CouchDB easily.

It's far easier and much more reliable to use a subset of SQL that's common to the major relational database systems commonly used today. This subset of SQL still ends up being extremely powerful.

I'm not sure exactly what you're getting at with the incorrect claim that "schema improvements are basically impossible" when accessing a single database from multiple code bases. There are many trivial ways to offer or retain compatibility. Using views is one common way. You then have much more freedom to change the underlying schema, while still supporting existing queries, for example. Stored procedures, functions and triggers offer other possibilities.

An API doesn't magically get rid of compatibility issues, either. You'll need to maintain compatibility within your API, or you'll have to run multiple versions of your API simultaneously, or you'll have to deal with it some other way. All that you've done when using an API is added yet another layer that you'll have to maintain. Duplicating the functionality offered by relational databases within your API's code will only end up taking more of your time.

At this point, I think that we've got enough experience as an industry and a craft to know that it's a good practice to use a schema, it's not difficult to change your schema while still maintaining compatibility, and it's better to keep your constraints as close to your data as possible. Relational databases enable this, while NoSQL databases make it much more difficult than it needs to be, if it's even actually possible.

<comment><USER9> "It's far easier and much more reliable to use a subset of SQL that's common to the major relational database systems commonly used today."

I think that is generally a mistake for non-trivial applications. MySQL and SQLite are so far from the standard that it's just not worth trying to find a subset of SQL that works the same on those systems as the others.

The similarities between the SQL implementations do make porting easier, however. But I consider it a port rather than using a common subset.

</comment></comment></comment></comment></comment><comment><USER13> The thing I noticed about NoSQL is that, at least with Mongo, you just end up putting your schema in your app. You probably do it by saying a particular class has a bunch of fields or instance variables that you expect to be a certain type.

I wrote a portion of an app that relies on Redis, and despite the fact that it's _really nice_ to work with, especially in Ruby (that practically mimics Redis' native API), it was incredibly difficult to remember the structure. What keys do I need? What type are they? Oh I'll have to do `KEYS *` and look I guess. Then I'll remember to document it (and thus create the schema).

Of course, the problem on my part was, in part, that I probably didn't need to use Redis. And when I needed to use Mongo, well, I wasn't storing things that look like documents. I think this is an easy mistake to make, but one you learn from quite easily once you've experienced it yourself.

<comment><USER14> I recently switched from Mongo over to CouchDB. The reason that I favor Couch over Mongo is primarily because I felt like Mongo's querying capabilities suck when dealing with embedded documents and that Mongo's MapReduce is not their first-class focus. Second, though, was because with CouchDB, I'm forced to basically put all of my query logic on the database server, which keeps it out of my client code. This cleans up my client code a ton, and querying from views gives context to what the data that you will be getting in return is. I don't think I'll ever look back to Mongo as long as I can choose CouchDB.

</comment></comment><comment> d--b This post is not addressing the right question. Think of databases as storage space, like wardrobes. The sql database will allow to store your clothes in spare parts each with its own dedicated space, allowing you to quickly calculate the sum of all stripes on all your socks. That's great but on the other hand, if you want an outfit you have to put it together, starting by sewing the sleeves on the shirt. On the other the NoSQL wardrobe will have various boxes where you can put any cloth you want. Now it's up to you to tidy the whole thing. If you want you can put full outfits in the box so that you don't have to think much about what to wear. However to count the number of stripes on your socks, you'll have to go through the whole wardrobe. The point is: It's not the fault of the wardrobe if your room is a mess

</comment><comment><USER15> Mmm, there's so much less code when everything is in JSON though. You can often ditch half the classes in the system. Sometimes I just don't need a client side class, a server side class, a schema, etc. just to store a few fields. It violates Do Not Repeat Yourself, especially when client and server are running different technologies like NodeJS server vs. Objective-C client. It is so much simpler to have just JSON in node and a dict on the client than to have a set class JS file with schema in node and a class on the client, and have to change them all whenever the data format changes. Also violates You Ain't Gonna Need It, from the agile rule house. A lot of times you build all that boilerplate, maintain all that bolierplate, and you never end up needing it anyway. Or have to change it all before you would have needed it.

<comment><USER16> What's the difference between a server side class and a schema? And isn't the necessity of creating a class just a complexity required by some statically typed OO languages. Some data defined by a schema is still just data -- it can be used similarly to JSON.

</comment></comment><comment><USER17> It seems to me that of all the advantages of NoSQL, schemalessness is probably the most minor and dubious. Most of the pain of dealing with SQL comes from the relational part, not from needing a schema.

The problem with SQL is that it's _clever_. To do things efficiently and reliably, you often must do very clever combinatorial operations on different tables, producing elaborate structures which are then winnowed down to a simple list of information. So right off the bat, you encourage programming logic that is difficult for future maintainers to wrap their heads around. Of course, most programmers _aren't_ clever, so their queries end up being both difficult to understand and subtly wrong.

Then, of course, your data comes out as a list of rows. Well that's just great. If your needs involve a more complex data structure than that, well, you're going to need code to rebuild that outside the database.

<comment><USER18> Yes, and not even only this. The really big difference is the eventual consistency, used as a core concept in most horizontally scalable data stores. If you once design your application with heavy reliance on hard consistency, you will have a _huge_ problem scaling it later on.

<comment><USER19> Very few applications need to scale past that point. I know plenty of business where if they would scale past the point of being able to run on a single SQL database instance they would make billions and be larger than their theoretical target market. Not everyone is building the next Facebook (which coincidentally uses MySQL for a lot of things).

<comment><USER18> It's not only about being able to serve lots of users. It's also about availability and geographical distribution. Even if you don't have millions of users, they will gladly benefit from you being split across several datacenters (serve everyone's requests from a location near to them).

<comment><USER11> It's quite easy to do this using traditional relational database technology, and various forms of replication. We've been doing it for many years, likely before many of today's NoSQLers were even born.

In the relational world, we don't really see such capabilities as anything special. They're merely one of the many basic features we've come to expect from any relational database system worth using.

</comment></comment></comment></comment></comment><comment><USER20> Mongo is like the PHP of the database world. Many important features are dropped in the name of "getting started quickly".

What you gain in short term wins, are nullified by all the long term headaches you suffer.

<comment><USER21> I don't think I would quite agree with that.. I would say that mongo is very accessible in terms of getting started... also, you can index nested properties via dot notation, unless you are really deeply denormalized, collections of objects with collections, I haven't been aware of any issues there.

A few things that surprised me about mongo... first that if you do a sort, you either need an index on your sorting fields or your sort info needs to fit in 32mb. I'm using skip/limit for paging, and it really became an issue once I was running against the full datastore. Second, only one single index is used in a query, unike most sql databases that will collate results of mutiple indexes under the covers... this caused me to rethink my index strategy, as well as add both asc/desc indexes for sorting results.

For me it was a matter of getting away from an over normalized data set (30+ queries for a single vehicle classified ad display, 23-joins if you wanted to reduce 15 of those queries (my main VIEW created for migration)... denormalized data makes more sense for this use case. For others it may or may not make sense.

<comment><USER22> Relational systems use multiple indexes for queries, though I'm not aware of an implementation that uses multiple indexes for a _single table_.

Remembering that a query in Mongo is basically a query on a single table it's acting in the same way a relational db does - ie, check statistics, choose best index, perform lookups.

</comment></comment></comment><comment><USER23> This matches my experience from the last few years as well. I was burned by denormalizing too early and not fully understanding the trade-offs I was committing to for the sake of not dealing with 'schemas' (which I actually have no problems with). And just like the OP, I have found far better success sticking with SQL early on for many of the same reasons mentioned. Later, as products and data schemas matured I found very specific uses for data stores like MongoDB. But, they are never the core data store.

</comment><comment><USER24> Well, another thing that is not mentioned here is that SQL and the original relational model were effective designed from the start to be "agile", to allow ad-hoc queries and other "quick changes".

The original relation database model was, from the get-go, intend to decouple the database from large-scale programming. The DB administrator would make queries and evaluate their effect on the system without having to be a programmer.

</comment><comment><USER25> How can you have a project without models and models without schema? And so you have to change a schema; is there usually much difficulty? More than supporting multiple schema across a single set?

Even if DBs were still bad at migrating a schema (eg. table locking horrors), the problem usually only presents with millions of rows, which one presumably doesn't have in the early stage of a project -- "start[ing] a project." Nonetheless, even with millions of rows, there have been solutions for all the major RDB since before MongoDB.

<comment><USER26> data mining : you dont know how exactly how your data will look like , and how much datas you'll have to handle. And you are opened to new types of datas. data logging : you dont need schema databases for that. file upload : GridFS , ... User generated content : save semi structured documents ( like a configuration , a spreadsheet , ... ) that need to be parsed but dont need a "schema".

<comment><USER27> True, typically you don't know in advance of a data mining operation. Nonetheless, you have some invariants that you know you'll need and these can be modelled with a schema.

Additionally, when attempting to put a model into production, you'll normally know what it is you need, and this can be put into a schema for ease of interoperability with other requirements (CMS, booking etc).

</comment></comment></comment><comment><USER28> I don't think that schema or schemaless should change your app structure. Your entities/models should contain your data structures. If you structure your application right, schema doesn't matter. In fact, your persistence mechanism shouldn't either. Mongo, Mysql, Redis, they're just different ways to store and retrieve your data.

The biggest reason to avoid schemas is to avoid schema migrations. On big enough datasets, they take a long time. That is not every app's problem, so it depends on your app. Also, not having a schema doesn't mean you completely avoid data migrations as a result of changing your schema in all cases.

<comment><USER29> The problem with avoiding schema migrations is that you end up with those migrations in your code. If you add a new field or re-arrange your data structure, you have to add code to handle that old data. In many ways, it's simply easier (and less permanent) to do a migration.

I've done it both ways and I wouldn't go schema-less again. Right now I have decade old code in my models that handles old data structures that more than likely don't even exist anymore. With a schema I can just make the change and perform any data transformations in SQL and not be required to have any code to handle legacy data.

<comment><USER28> I agree that avoiding schema doesn't solve the problem. My point is that your schema exists in your code one way or another (explicitly or implicitly), and even without a database schema, you still need to either handle the schema differences in your code, or you need to migrate your data into the new schema in some way or another.

The best use of schema-less data storage I've heard of is Craigslist of MongoDB for their archives. Apparently migrations on their MySQL servers was taking way too long, and switching that system to Mongo made a lot of sense and apparently has worked out pretty well.

In the end it's about understanding the use of the right tool for the right job.

<comment><USER29> I have experienced slow migrations and it is a pain -- but it's not a fundamental conceptual problem -- it's just a tool problem. I believe postgres will or already has the ability to do background incremental schema changes without locking the table.

My point is that you have a schema in code and a schema in the database whether you have a fixed schema or an implicit schema based on the structure of your data. I prefer to be explicit.

<comment><USER28> I agree.

</comment></comment></comment></comment></comment><comment><USER30> mongo is easy to start with but it sure makes your project more complicated and painful as time goes on. it was to the point, i actually stopped enjoying work.

the power of sql is just too great to ignore, had gone through too many times where i wish i was on sql.

i will probably use mongo for those times where i absolutely need to store a schema-less data (custom attributes), but will only do so in conjunction with a sql database.

there is also that reliability and maintenance factor of securing it, making it redundant, and correcting problems when problem arises. all of which takes time off development.

and just like OP, i probably won't ever start another project with mongo from ground up. there are just too much things to worry about and problems come out from all different angles when you least expect it. but in sql, everything just works.

<comment><USER26> impedance mismatch , that's why SQL is uber non sense in OOP architecture, especially when you end up using slow and buggy ORMs...

<comment><USER31> If an ORM is slow or buggy, you're using it wrong or its crap. Most things that claim to be ORMs are really not very good quality.

Try hibernate / nhibernate with ehcache cache backend. We find both to be an order of magnitude faster than hand tuned SQL and testable to boot.

<comment><USER32> I despise hibernate. I will never use a self managed 'session based' ORM again, it creates far too many messes.

<comment><USER31> When you get over the learning curve, which is quite large, the mess doesn't exist. I've never seen an issue with hibernate which isn't caused by poor understanding. You really do have to know your shit before you start - <USER2> can't just waltz in django-style and write code. You have to think about session management strategies and architecture.

<comment><USER32> It has nothing to do with understanding and everything with having to create a bunch of excess code to deal with the way it works.

As an example: I shouldn't have to write a filter to wrap the template rendering stage of request processing in a transaction just to make lazy loading work.

To make hibernate useful you almost always need to have another framework manage it (SEAM, Spring, etc). I'm not going to use an ORM that requires an entire other framework just to make it reasonable to work with.

Hibernate also has the worst query building framework I've seen. Its far and away the most verbose piece of crap I've ever worked with. As a result most people just drop to writing queries in JPL which removes one of the big niceties of an ORM.

<comment><USER31> Well you've conclusively proven my point because its obvious you don't know how to use hibernate properly.

You _never_ lazy load stuff from a view. That is a recipe for disaster. You use a projection or set of projections and populate your model up front. The transaction should be concluded way before the view is rendered. If you get that wrong, your architecture is broken.

The only bit of infrastructure we use is an annotation + AOP which wraps a service method in a transaction and unit of work. This is all container managed. To be honest, it's literally 3 lines of config and works universally.

Verbosity - that's a load of crap. We have 3 lines of AOP config, a mapping file per class and nh config which is about 12 lines of XML. All the config was cut and paste.

The whole thing gives us a model which is portable between engines, proper transaction management, cache layer, schema generation, fully testable data API (we materialise the whole model into HSQL in integration tests) and documentation which is actually good.

It's a no brainer. I suggest you go read POEAA and have a think.

<comment><USER32> Sounds like a complete mess and I see no reason to build an application that way. I certainly can't see a reason to create a separate query for every place in a view a model may be used just to avoid lazy loading. Maybe on performance critical views but the rest of the time, that is an unneeded premature optimization.

You've also proved my point, by needing an entire 'container' managing things to keep everything straight. I'll happily keep hibernate in my rear view mirror, I'm quite happy with Ebean.

<comment><USER31> Each to their own. Our architecture handles 22,000 reqs/sec, is scalable across our 80 man dev team and pulls in approx $30mil on revenue. Does yours?

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER33> For me the process of designing my schema and planning my project are almost one in the same. I have a hard time thinking about how I'm going to build something without thinking about the schema.

</comment><comment><USER34> I spoke at Code Generation 2010 at Cambridge in the UK on my experiences implementing code generation in various parts of an existing shop's development process.

One of the low-hanging fruits in that regard is selecting existing and under- utilized models and making the most of them.

I found that SQL schemas were a great way to model a business domain even if you need to migrate to another database ... since you have so many tools (including free cross-platform visual ones, like MySQL Workbench) at your disposal, validation, decent type selection, options for more complex validations using triggers, everyone knows it (unlike UML), etc.

Another great 'model' that people are usually sitting on top of, also ripe for code generation, is the software dependency databases in many distributions.

Another is your RCS/VCS system.

Another is - and here comes the kicker - the filesystem, which with modern tools (inotify, etc.) can give an impressive amount of RCS/VCS style functionality (COW, event notifications, etc.) at kernel level.

What was that quote again? "Unix is simple, it just takes a genius to understand its simplicity". They weren't lying.

<comment><USER31> If you're using code generation, your tooling or architecture needs rethinking. There is 100% no reason to do it.

<comment><USER35> Sorry, to say there's 100% reason not to do anything is rather silly. By your own definition, what if your tooling lets you do codgen? Regardless, I've found code generation to be particularly useful in many scenarios and I would never dismiss it out of hand.

<comment><USER31> By my definition, I'd avoid it. The pitfalls, additional abstraction and debugging complexity just aren't worth it.

</comment></comment></comment></comment><comment><USER36> I've found that the primary value for me in NoSQL implementations is in allowing for customization at the data level for the end users without having to explicitly build structures for it. So if User A wants to add a tracking field to all her data, she can do so without my having to build an extensible data structure on the back end; it's already there, and it doesn't affect any other user records.

</comment><comment><USER37> There is something to be said about the ambivalence of using SQL or NoSQL for OLTP-esque applications (loosening the definition of a transaction, that is), but SQL is very useful when working with OLAP-esque applications, i.e. writing a SQL query and avoiding drafting a map-reduce job or building the logic into a custom script, and still using good stuff like secondary indexes, etc.

</comment><comment><USER38> More than often, reading the schema is the lowest hanging fruit when you are trying to understand the domain knowledge of a startup. This becomes even more true when the codebase is not that readable. Now this is not an enough of a reason to shy away from NOSQL data stores, but should make you pause and consider what you will be losing when you invest in a schemaless DB.

</comment><comment><USER39> As long as you have the data then you can calculate it. It may be easier with SQL and a normalised form but that forces a lot of other constraints.

With a non-relational DB the idea is to attack it with map reduce and possibly in app calculations. This may not be ideal, but the benefits of easy placement and access of the documents outweigh the aggregation problems on most of the problems I have dealt with.

Neither solution is perfect you just need to make a prediction based on the projects requirements and make a choice.

</comment><comment><USER40> I've been using MongoDB to "start" with for quite a while (and used AppEngine before that). Instead of using the low level mongo libraries, I use a wrapper like Mongoid to provide a high level object mapping.

With MongoDB + Mongoid, the schema is in the code in the form of Mongoid models and data stored in MongoDB mimics the in-memory application structures. I've found that this makes it much easier to visualize data flow - although it does force one to think carefully about their schema up front.

</comment><comment><USER41> I still like GridFS for simple distributed file systems and Redis for N-length lists of string and/or binary content. That said I have to agree with the author whole heartedly.

</comment><comment><USER42> _At one point you need to have an understanding of what you're dealing with: you need to know of what type a value is._

If we agree with this then why is it so controversial to assert that types are also useful at the language level? For me working in a dynamically typed language is just as awkward as working with a schema-free datastore. Writing code is hard. Why not let the compiler help you?

<comment><USER43> Typed languages add a ton of boiterplate to what would be simple operations. That slows you down, more than most people notice.

Now, that said, explicity typing makes it easier to catch a lot of mistakes, and helps you keeping you data consistent. It's very usefull... But slows you down.

As a conclusion, you want to explicitly type anything that is important, but also wants to keep the explictly typed data to a minimum. (What makes optional typing, like in Haskel very interesting.) And, in nearly all the applications all the important data (and very little unimportant data) goes to the DB.

</comment></comment><comment><USER44> The title is very correct. SQL is agile.

When can we PLEASE stop messing up names? Everybody who's talking about "NoSQL" should be thrown into a deep black pit.

SQL is a query LANGUAGE. It's excellent at what it does. To be honest, I like HQL, even though I'm a ruby guy since about 5 years.

The whole scalability, consistency, acid, schemaless discussions are actually not the point.

</comment><comment><USER45> I'm not sure about your point on schemas. When using relational database you must have your schema in two places: 1. database and 2. ORM - the second one is enough to avoid most of problems.

What's the point of having schema in database then?

<comment><USER46> 1\. Your ORM can derive it's schema from the database's.

2\. I think the main point of the post was that you can run ad-hoc SQL queries no matter how much you've denormalized. You can't necessarily do that with a NoSQL database.

<comment><USER26> Yet you are still using schemas in 2 places : not DRY , SQL just doesnt fit OOP design. Most of languages today have strong functional capabilities , that makes SQL obsolete , they have functions , event systems , RDBMS exist because people use to query those systems directly , does your users log into your database directly ? no they connect your database through middleware, that's where the job should be done.

<comment><USER47> DRY is "Don't Repeat _Yourself_ ", not "Don't Repeat Ever". If your ORM fully correctly derives everything it needs from your database schema, that doesn't count as a repeat. If your ORM needs a couple of hints, but those hints are indeed extra information that your DB can't have ("this isn't just a string, it's an IP address") that doesn't count as a repeat. If _you personally_ typed your schema into a database creation script and then _you personally_ also typed the same schema into your ORM specification, only then are you violating DRY.

Code generation is a powerful DRY tool, not something it suggests avoiding!

I say that independent from the question of whether you should be using an ORM at all. If you are going to use it, either your ORM should come out of the DB or be responsible for creating the DB, either is fine, as long as you're not saying the same thing twice.

<comment><USER48> > DRY is "Don't Repeat Yourself", not "Don't Repeat Ever".

It's actually "Every piece of knowledge must have a single, unambiguous, authoritative representation within a system."

(this advances your point even further)

<comment><USER49> And makes clear that DRY is just a handy catchphrase for the design principle of normalization.

</comment></comment></comment></comment></comment></comment><comment><USER50> I avoid SQL not for the notions, or concepts behind SQL Databases, but for the SQL itself; the LANGUAGE of it all (specifically the syntax).

Nowadays I work with things like ActiveRecord and Mongoid, which add an abstraction layer on top of the literal interface language. However, at the very beginning of my intro to databases (quite recent, last 6 months), the prospect of learning a new language just to talk to databases was off putting. The notion of documents and fields made more sense to me, as it was more comparable to a file system. SQL tables, rows, and columns were daunting and seemed like a relic of paper spreadsheets.

The ecosystem around SQL solutions if clearly far more developed though, however I don't think we should avoid non-SQL solutions for that reason.

Give the alternatives a chance to develop.

Don't look at MongoDB through SQL tinted lenses.

<comment><USER51> Having to know yet another language is annoying, but mongo has one too (and one that's less consistent and powerful).

A good SQL library (like SQLAlchemy) lets you easily build queries without looking at SQL. The only case you might have to is debugging, which is the same time you'd be looking at mongo queries.

</comment></comment><comment><USER26> SQL is not DRY, especially when people stuff their apps with bloated ORMs. SQL exists because people used to query RDBMS directly , now most of the time databases are simple data repos , it makes sense to put most of the logic in the application itself.

One should not have to deal with types in the app and types in the database, exceptions in the app and database errors , events in the app and triggers , functions in the app and stored procedures , or it means one doesnt trust it's own middleware.

SQL makes apps insecure, and hard to refactor, and is a non sense when doing OOP design ( here comes the bloated ORMs again which usually come up with their own query langage ... linq , dql , hql ... ).

Doesnt mean data modelling is bad , on the contrary that's the first thing to do , but data modelling doesnt need to fit the SQL mindset ( thinking in tables and columns ). Objects are parts of collections that can hold collections of other objects themself. Why should one try to fit that model into SQL ? it does not.

MongoDB may not be the best NoSQL solution out there neither , there are better products.

<comment><USER19> One major advantage of using SQL databases is that they provides excellent guarantees for data integrity. This means I do not have to think much when refactoring my data. And I do not have to worry about production systems crashing due to old data being different from new.

When modifying the schema in NoSQL databases I always worry about if I mess up and leave old cruft which will cause my code to crash, or even worse if I lose data (data created between the last backup and the upgrade).

With proper SQL constraints there are many fewer ways where you can screw up when refactoring. And easy refactoring of data is key to agile development of systems which have been put into production.</comment></comment>


Adobe vets build rival to Flash for iPhone apps  - <AUTHOR> http://venturebeat.com/2009/06/23/adobe-vets-build-rival-to-flash-for-iphone-apps/ <comment><USER2> _"In the Corona SDK, you'll write code in Lua, an intuitive and powerful scripting language."_

In case you were wondering (like I was) if it was ECMAScript based.

</comment><comment><USER3> Then you just have to hope Apple won't care / notice that you're using a scripting language. So far, Apple has been reasonably lenient (there are apps running on smalltalk for instance), as long as you don't allow users to run custom code, but who knows when they'll go and be more strict?

<comment><USER4> But will you be using a scripting language? If you can compile something to C, you can call it Objective-C and pass it off to Xcode, right?</comment></comment>


A Fast General-Purpose Lock-Free Queue for C++ - <AUTHOR> http://moodycamel.com/blog/2014/a-fast-general-purpose-lock-free-queue-for-c++ <comment><USER2> This is an interesting showcase of lock-free programming. Thank you for this!

However, I see the following concerns:

1\. The code is so complicated that (imho) an actual formal proof would be needed to show that this actually works.

2\. How portable is this? Will this work on other architectures than i86, which have different ordering guarantees?

3\. This reduces the time for enqueueing and dequeueing by a factor of x, which is immense (near 100x or 1000x?), but we have to keep in mind that the code uses only a small fraction of total time spent by an application. Therefore, it would be nice to see that this reduction is worthwhile in a practical application.

4\. The code for this queue is actually so large (2800 lines) that a large amount of cache memory is needed to keep this code in active memory, and keep it running as fast as it could.

Again, thanks, this will be interesting to study.

<comment><AUTHOR> 1\. It is complicated, and, of course, with any complex code comes the possibility of bugs (especially when threads are involved!). I'd love to have a formal proof myself, but it's not a simple undertaking -- the sheer number of possible codepaths at execution time would make an exhaustive check infeasible. (And expressing the _entire_ thing as a formal model is beyond my means.) What I've done instead is to try to test individual subsystems in isolation (you can check out the tests if you're interested): There's a free- list, a thread-local storage map, an add-only lock-free list, two block mapping methods, and the core algorithm itself. Then I wrote a series of unit tests (that are really more like integration tests, I suppose) that test individual functions, and everything together (and boy did those tests ever catch bugs during development!). There's never too many tests. There's never too much model-checking. But I only have limited time :-) So in the meantime, I made an initial release when I thought it was stable and correct to the best of my knowledge, with a reasonable amount of testing that I feel is reasonably thorough. I'd gladly accept more tests contributed by somebody else.

2\. It is meant to be 100% portable to any system that has a C++11 compiler. I was very careful about the orderings I wrote, so yes, it should work on CPUs with weaker memory models.

3\. The factor depends on what part of the API is used, and with how many threads. I absolutely agree that a fast queue itself probably won't buy you much in terms of application performance. But if you're going to write a data structure, why not write a high-performance one? Much more fun, IMO, and certainly can't hurt. 4\. Here I must disagree. The number of lines of code of a heavily templated data structure are entirely unrelated to the compiled size :-) In particular, if you use either only producer tokens or only no producer tokens, then a large portion of the code is unused and will be optimized out. Same for the bulk methods.

</comment></comment><comment><USER3> Please don't be upset with me for asking ... but I am sincerely curious: I know that an MPMC lock-free queue is a vastly challenging thing (the best I could do lock-free myself was one producer thread, one consumer thread), but is this really the minimum level of complexity needed to implement this idea? There's a lot of comments (which is great), but even with that, the code just seems incredibly _massive_ to me for what it is (over 2800 lines and 100KB, of course it has a lot of great commenting in there as well.)

If there really is no way to reduce the complexity (or at least without ruining the performance), then I understand. I know code size isn't a particularly important metric, I just find code that's been reduced to its raw essence a lot easier to understand and learn from than over-architected, over- abstracted code.

I ask because this is something I'm interested in understanding (I want to write a nice thread-safe allocator), but the code even with the comments and article is fairly overwhelming to me at the moment.

Either way, great job on making this! Really impressive performance metrics, too.

<comment><AUTHOR> You're certainly right that it's not the minimum amount of complexity. But it's not that hard to implement a simple MPMC queue (e.g. if you can assume DCAS on your platform) -- what's hard is making it fast. So part of the complexity derives from a design that calls for blocks of elements instead of a simlpe linked list, for example; but a lot of the speed comes from that design too.

The rest of the code size (and complexity) arises from the feature set (which I intended to be fairly complete): Users can provide tokens for better performance -- but don't have to. There's bulk methods. There's a move constructor and assignment operator (for the tokens too!). There's traits so that you can override the malloc/free that's used, and all the constants. There's memory debugging machinery (hidden in an #ifndef NEBUG). It all adds up, but not without reason, I think :-)

I will add a "how to read the code" section to the README later on when I have a minute. I think it would be instructive for those curious.

</comment></comment><comment><USER4> OK, now demonstrate that this doesn't require fence instructions to prevent problems on superscalar CPUs (anything bigger than a low-end ARM today).

<comment><AUTHOR> Of course it requires fences. It wouldn't be correct otherwise, and would probably not pass the tests even on x86, thanks to compiler re-ordering.

</comment></comment><comment><USER5> This doesn't seem to correctly implement a queue to me?

Suppose thread A enqueues an item, signals somehow to thread B that thread B may proceed, and then thread B enqueues an item. There's no guarantee that thread A's item will be dequeued before thread B's.

Of course, non-linearizable doesn't necessarily mean useless (this is in fact an active research area), but it makes me think the performance comparisons aren't really fair; you're comparing genuinely linearizable queues with one that isn't bound by that performance-hampering requirement.

<comment><AUTHOR> I consider it a queue, despite its non-linearizability. I'll quote from my README where I addressed this concern:

One particular consequence of this design (which seems to be non-intuitive) is that if two producers enqueue at the same time, there is no defined ordering between the elements when they're later dequeued. Normally this is fine, because even with a fully linearizable queue there'd be a race between the producer threads and so you couldn't rely on the ordering anyway. However, if for some reason you do extra explicit synchronization between the two producer threads yourself, thus defining a total order between enqueue operations, you might expect that the elements would come out in the same total order, which is a guarantee my queue does not offer. At that point, though, there semantically aren't really two separate producers, but rather one that happens to be spread across two threads. In this case, you can still establish a total ordering with my queue by creating a single producer token, and using that from both threads to enqueue (taking care to synchronize access to the token, of course, but there was already extra syncrhonization involved anyway). I expect this use case to be fairly rare, though!

<comment><USER5> This isn't really a great solution though, since it pushes the problem of ensuring FIFO ordering outside of the queue and onto the user; but being FIFO is entirely the point of the queue data structure. If your queue doesn't provide ordering across multiple threads, it can't meaningfully be called a multi-threaded queue, so it's not very fair to compare it to genuine multi- threaded queues. Unless I've misread the documentation, this is just a collection of SPMC queues, together with a read operation that checks all the queues for an item to dequeue, right? But there is no synchronization that occurs to tie any of these SPMC queues together. Even if you wrapped every single access to this data structure in a lock, for instance, you would not be able to ensure FIFO ordering of elements inserted into it.

I think the estimation that having inter-thread ordering requirements is uncommon is incorrect. Probably the most common use of MPMC queues is as a concurrency primitive in systems using a CSP-like abstraction, where such requirements are extremely common. But you can't implement, say, Go's channels in terms of this data structure.

<comment><AUTHOR> Yes, it is essentially an abstraction over a collection of SPMC queues (when using tokens to dequeue, it is actually a bit smart in that it tries to pair producers with consumers where possible).

If you wrapped every enqueue operation in a lock and used the same producer token, then you'd essentially have a single SPMC queue, and it would be fully FIFO. But obviously this defeats the purpose somewhat, so yes, it's not a good solution if you need that guarantee.

I honestly didn't consider external inter-thread ordering at all while I was designing the queue. I guess I was picturing more of a producer-consumer model, than a message passing system (where this dependence between messages across threads/producers is far more common, it seems).

I'd change this constraint if I could, but I can't. It's the fundamental assumption the very heart of the entire design. I'd have to start from scratch (which I'm not prepared to do, at this point). But I've changed the README to make this limitation crystal clear (you're not the only one with reservations with regards to this aspect).

I think it's still useful, perhaps not for all applications, but for many. Sorry if I came across as trying to purport it as something that it wasn't; that was not my intention at all.

</comment></comment></comment></comment><comment><USER6> This idea is very similar to the Java Disruptor pattern ([http://lmax- exchange.github.io/disruptor/](http://lmax-exchange.github.io/disruptor/)). So is this an implementation of this pattern in C++ of sole independent creation of the author?

<comment><AUTHOR> Yes. I hadn't looked at the disruptor pattern until yesterday, and I still don't fully understand it. I designed this queue from scratch.

</comment></comment><comment><USER7> May I ask why you choose to use inline on the non-template class-methods? Did it make any difference in the code produced? For the rest: this seems very well done. Love all the proper comments and the huge amounts of static_asserts.

<comment><USER8> Not the author, but since the whole thing is defined in a header, everything has to be declared inline to avoid violating the one definition rule.

Inline hasn't had anything to do with performance for years (compilers just do what they want anyway), but it's somewhat confusing that the meaning was changed.

<comment><USER3> I found a cool trick around that recently. Use anonymous namespaces when you want to have "standard-library/boost"-like template header libraries. Anything inside of an anonymous namespace has no external linkage, and it can transparently use anything inside its parent namespace.

For instance:

        namespace nall { namespace {
          void foo() {
          }
        }}

You can include this in multiple compilation units, invoke it with nall::foo(), and have no issues with multiple symbols.

(and of course as rav said, as long as you put the function definition inside the class definition, that can also work. But my trick will work if you want the functions declared outside the class, or want non-class functions.)

<comment> T-zex Wouldn't that become a separate function per translation unit?

<comment><USER3> I want to say that GCC will do the same thing with the inline trick. It has the option to inline the function (where there won't be any actual functions in the resulting binary), or declare it as a unit-unique function (where the actual function will be repeated in the binary, one for each object that includes our header and uses the function), or recognize it's the same function in each unit and merge it back to just one function for the entire binary (which seems even more likely to happen with LTO.)

Theoretically, the same thing should happen here. The only difference is you don't _have_ to hint that you want the function inlined (and maybe you really don't), but you can still mark the function inline (or always_inline) if you want to ask GCC to please do that, and it doesn't ignore you.

However, I haven't actually analyzed the resulting binaries comparing the two approaches, so I can't say for sure if GCC can optimize the multiple functions to one in the former case (inline) but not the latter case (anonymous namespace). It would seem like an oversight if that were true.</comment></comment></comment></comment></comment>


'Don't forget to use your semicolons' - Douglas Crockford - <AUTHOR> https://plus.google.com/118095276221607585885/posts/74ZcpCQcio7

<comment><USER2> It really is funny how people like Mr Crockford can dish out insults left right and centre, but cannot EVER allow any honest and open discussion.

I made a comment about ASI, and was immediately barred from the thread, and my comment was deleted.

Note, there was no profanity, or insults, just an alternative view.

So, I guess that that bigger brother is alive and well, and living on Mr Crockford's left shoulder.

Liam Carton

</comment><comment><USER3> So Douglas Crockford didn't read how it was the Google Closure minifier that added the commas and that it was a bug in Opera's JavaScript parser. So in Douglas' opinion Google Closure team are "idiot web developers".

<comment><USER4> Old man on porch yells at kids to get off lawn, mutters something about eating more fiber.</comment></comment>


Israel to Bring Orthodox Jews Into Tech Boom - <AUTHOR> http://online.wsj.com/articles/israel-aims-to-draw-ultra-orthodox-jews-into-tech-boom-1402352311?mod=WSJ_hpp_sections_business

<comment><AUTHOR> A very good read.</comment>


16 ways to market your bootstrapped startup - <AUTHOR> http://peterharrington.info/post/59941557/16-ways-to-market-your-startup-for-free

<comment><USER2> I actually really like this list. A lot of it is (or should be) common sense, but it's still a good reminder.

I also appreciate that he thinks some effort should be put into SEO. I tend to agree with that even though I know some here don't.</comment>