PCB Design Tutorial for Startups - <AUTHOR> http://www.nepsu.com/nepsu_ultimate_pcb_design_guide.html <comment><USER2> A couple things that I wish people had pointed out before I got started with PCB's:

1) You will spend your life reading datasheets.

2) Datasheets lie.

3) You will spend way more time in component selection than schematic design or board layout.

4) You will spend way more time creating component footprints than in schematic design or board layout.

5) Lay out the board by hand. You will waste more time fighting an autorouter than if you just laid it out.

6) A 6 layer board is not that much more expensive than a 4 layer board and is _VASTLY_ easier to lay out. 2 layer boards should be avoided unless you you are a masochist or every fraction of a penny _really_ counts.

Some of the given advice is a bit off.

1) He doesn't put enough emphasis behind avoiding BGA. BGA should be avoided like the plague. Period. It will only cause excruciating pain.

2) Small QFN isn't that hard to deal with ... if you tweak the PCB footprints a touch (make sure you have .5 mm of bare copper outside the package outline and you can treat them like QFP).

3) 0603 is actually pretty small. Most people should stick with 1206/1210.

<comment><USER3> Agree on most parts. I would say you can learn to love 0603 and even 0402 if you have the right equipment. You need a proper stereo zoom microscope and then everything is big, and smaller components can save a lot of space. But don't go there until you've done the bigger stuff and actually felt a need for more room. Definitely 1206 and 0805 are plenty fine for people who aren't super in to it.

Another hard lesson I have learned is that PCB assemblers lie. Even supposedly reputable community friendly ones. If they can save money by swapping out a component for unknown alternatives, there's some nonzero chance that will happen.

<comment><USER4> I was a solder monkey for a summer a few years back, and all I had was a magnifying glass lamp [1], I was soldering resistors down to 0603.

Last semester at university I did a hardware design course (sort of by accident), at the lab, they had a stereoscopic microscope similar to [2]. I never realised what I was missing out on until I used one, apparently they're up to 10 grand a pop, so I can see why my old work didn't have one, being a pretty small shop.

[1] [http://uk.farnell.com/lightcraft/lc8093led-euk/led-table- mag...](http://uk.farnell.com/lightcraft/lc8093led-euk/led-table- magnifier-1-75x-4x-mag/dp/2442796)

[2] [http://www.visioneng.com/products/stereo-microscopes/lynx- dy...](http://www.visioneng.com/products/stereo-microscopes/lynx-dynascope- inspection-microscope)

<comment><USER3> I've seen that style but never used them. I will say that a $200 AmScope is well worth the investment. Not as nice as a high quality microscope, but vastly superior to a magnifier.

</comment></comment></comment></comment><comment><USER5> A long article that says almost nothing. Instead of choosing trace widths and layer count and component sizes at random, take a week to read and understand the very useful book "High Speed Digital Design" by Howard Johnson. Despite the name it concerns itself more with the analog domain than with the digital. Very useful information about the behavior of traces and holes in boards.

<comment><USER6> I didn't want to be critical of it but as someone who went through the process of designing and manufacturing my own hardware product to sell a couple years ago that was my thought as well. Really long and very little insight into the real difficult parts.

<comment><USER7> As someone who has actually designed manufacturable boards I'd say the same. In addition the article has some strange recommendations:

* don't use QFN — I see no reason to, on the contrary, these are great packages (see my other post in this thread)

* 0603 minimum size for passives — huh? I see no reason not to go down to 0402, which is both easy to manufacture (assembly costs are the same, component costs are usually lower), and easy to prototype (just get a large desk magnifying glass and a good pair of tweezers and you're good to go).

</comment></comment></comment><comment><USER7> Here's something I don't get: why discourage people from using QFN packages?

QFNs are only a problem if you are hand-soldering your boards with a traditional soldering iron. Which is nuts. These days everyone should have a soldering station with both traditional iron and hot air — and soldering QFNs using hot air is straightforward and easy.

Besides, the article is about manufacturing, so you aren't likely to solder those QFNs yourself. So why discourage them?

I think QFN and DFN packages are fantastic. They save lots of board space (and believe me, board space matters a _lot_ in all designs, more than you think!), have good thermal properties, lack of leads makes for better low-inductance connections, less material means lower price. And most still have pitch of 0.5mm or more, so you can use the same design rules and same PCB pricing. Everything is great about them.

BGAs are a different matter, but not for the reason most people think: they are problematic not because they are difficult to solder, but because you need much tighter PCB parameters to be able to route all the signals. In particular, via size is the biggest problem. You need really small vias that can be placed between four BGA balls.

I dislike the hate being poured onto QFNs. I think it is a relict of the past: back when hot air was expensive and "strange" and people tried to solder everything by hand using a soldering iron, QFNs were the enemy. But these days they should not be.

The unfortunate effect of hobbyists hating QFNs is that some manufacturers (Freescale/NXP, notably) later class certain chips as "hobbyist" and don't provide QFN packaging at all. And believe me, if you're designing a wearable device, the difference in size between a QFN-64 and an LQFP-64 is monstrous. Providing no real advantages, because both have 0.5mm pin pitch.

So, while I would agree with the author that BGA packages are best avoided, I would encourage everyone to actually go ahead and use QFN/DFN packages. They are better in every respect.

</comment><comment><USER8> Nice overview article. I have been doing PCB's for about 3 decades and here are just a couple of little tricks I have learned over the years, some are too long to go into here, but just a couple quickies maybe will help someone out:

1) Often the chip manufacture provides BSDL files which often times contain the foot print info and a lot of CAD tools have macros that will read them in and create your foot print for you.

2) There are often foot print libraries already out there you can download or are provided by the CAD software. But in either case, don't trust them, there are often errors, get your parts first, do the PCB layout, print it out on a good printer (usually the CAD SW will print it to scale and test fit everything.

3) There is a big difference between doing prototype boards and production boards. For prototype boards, make them big because you will have mistakes and will probably have to black wire (cut a trace and fix) it. I often use 2 layer or 4 layer for prototypes because it is easier to do fixes. If you CAD SW does auto layout then it is not that big of a deal as long as the board is large enough for it to route everything for you. For Production PCB's it is a different story and a lot more work to do right and I recommend farming it out if you can afford to.

4) I agree that part selection is critical but also, if you have simulation software either a separate program or in your CAD system, doing simulations can save you a lot of time. If you don't have simulation SW check into it, there are some decent free ones to learn on but to get serious you have to pay a lot.

5) For complex designs, I do separate prototype boards, one with the FPGA on it, another with the MCU and another one with the analog on it and so or, sort of a unit test approach (sort of like dev-ing SW) that way you can swap out sections that aren't working for you without throwing away a board full of parts, it is more effort but if you plan it out it can save you time and money.

6) I agree with the BGA comments, don't use those types of components unless you have a reflow oven and know how to use it, it is easier to debug QFN and QFP type packages since all the leads are exposed for inspection. You can often get parts in either package type and when you go to production you can always swap out to the smaller package later. If you have to use those for prototyping check if the manufacturer has a prototype board for it already it can be easier to just use an off the shelf one.

A lot of people say the first board is a throw away, but that can be expensive for a production one so I have learned these tricks for prototyping so when you get to production you can avoid that bad first board.

<comment><USER9> > There is a big difference between doing prototype boards and production > boards. For prototype boards, make them big because you will have mistakes > and will probably have to black wire (cut a trace and fix) it. I often use 2 > layer or 4 layer for prototypes because it is easier to do fixes

Not only this, you can also print your own boards if you stick to two layers. Way, way, way faster than waiting days to try something out and notice you swapped Vcc and GND pins on a chip by accident.

<comment><USER8> Heck ya, if you have the capability to do your own boards, even if only 2 layer, I would do that. So much faster t !

</comment></comment></comment><comment><USER10> I'm in the Bay area and a hardware hobbyist (software is my full time gig). I see tons of meetups for SW but almost nothing for electronics hardware. Any recommendations?

</comment><comment><USER11> Also, ChipQuick is great. It's super-low melting point solder you can use to easily rework SMD components. This thing has saved my ass too many times to count as a hobbyist :) Was able to get away without a reflow station for the small stuff I've been working on.

[http://www.chipquik.com/store/index.php?cPath=200](http://www.chipquik.com/store/index.php?cPath=200)</comment>


CS Unplugged: Teaching material for CS using cards, string and crayons - <AUTHOR> https://www.csunplugged.org/en/ <comment><USER2> Several decades ago, "without a computer" was the norm, because they were expensive, noninteractive, and/or difficult to access. You had to write your entire program before being given a chance to run it, and if there were any errors, you had to wait possibly a very long time before you could run it again.

Now that computers are ubiquitous and nearly everyone can immediately start writing code, I still think there are many advantages of doing it the "old school" way. Many of the highly-skilled developers I've worked with started learning with no constant access to a computer, so I definitely think this form of teaching is very useful --- without a machine "doing some of the thinking" for you, as it were, it forces you to actually understand the problem more deeply, so that you can tell the machine what to do. Those developers are also the ones who will spend more time whiteboarding or writing with pencil and paper than they do writing or debugging code.

Also, I thought this image depicted students being handed stacks of 5.25" floppies in their protective envelopes:

[https://storage.googleapis.com/cs- unplugged.appspot.com/stat...](https://storage.googleapis.com/cs- unplugged.appspot.com/static/img/general/how-do-i-teach-this.png)

<comment><USER3> If your country had CS for 5 to 10 year olds several decades ago you're lucky.

I think for that kind of age range physical is better than digital anyway. But as you point out it's still useful for us bigger children too.

<comment> a-saleh This would be the norm for ~15 year olds in my country, ~15 to 30 years ago?

</comment></comment></comment><comment><USER4> We did a deep-dive on CS Unplugged with the program's creator, Tim Bell, on The Changelog last year.

Link to the transcript, for those curious: [https://changelog.com/podcast/302#transcript](https://changelog.com/podcast/302#transcript)

</comment><comment><USER5> I learned doing ACSL exercises on paper starting in middle school. We had excercices for things like Boolean algebra, graph theory, base conversion ect. As well as 5 hacker rank style programming questions throughout the semester, in addition to our more regular CS coursework. Teacher made us keep an organized notebook with printouts of all our excercices/tests ect. Thought it was rediculous and old school but wasn't even 5 years before I looked back with gratitude. Learned the fundamentals needed to get me through 2/300 level courses at college. The teacher was amazing and in large part the reason I'm remotely successful today. There were 13 - 20 kids each year that took more then the intro courses. He would always treat us like family, and would do things like have us all meet after school once a year in fancy clothing for a "mock" fancy meal. It was so he could teach us (some autistic) how to be proper at a dinner and which fork is used for what. He taught us how to be skilled programmers and also people. Random middle of nowhere public high school in PA, very grateful. The teacher was deathly terrified of birds tho.

</comment><comment><USER6> I've run these activities with kids (middle and high) and adults (teachers) and they have always been remarkably engaging and surprisingly effective at conveying the concepts each lesson is supposed to teach.

I would 100% recommend using them if you ever get a chance to construct a CS fundamentals curriculum.

</comment><comment><USER7> Do not use string, index cards, and multiple CS PhD students holding the cards to illustrate how garbage collectors work, particularly if most (all?) of the people in the groups haven't written a garbage collector. Trust me on this.

<comment><USER8> As someone who hasn't written a GC, why is this?

<comment><USER9> As someone who hasn't written a GC, how well do you think you could illustrate how one works in a live presentation?

</comment></comment></comment><comment><USER10> A thread from last year: [https://news.ycombinator.com/item?id=17661707](https://news.ycombinator.com/item?id=17661707)

</comment><comment><USER11> Wow, this is an amazing pursuit. I would love it if kids were introduced a bit earlier to more analytical thinking methods thru ways such as this. Learning about computer science has benefits in changing the way you approach concepts and problems, but kids shouldn't have to wait until middle-high school!

</comment><comment><USER12> Tim is also a great lecturer (I had him as my lecturer many years ago), and one of the authors of Managing Gigabytes, - which for many years was apparently basically required reading at google - which is a great intro to all the normal compression systems, as well as large scale text search

</comment><comment><USER13> And here I was, thinking 'huh, they finally condensed down deep CS topics so that my dumb ass can understand it! Neat!' Imagine my disappointment looking at the ages section.

Still, a neat idea! Just not for my dumb ass.

</comment><comment><USER14> I have been building a course for my 5 year old using Scratch. She has a ton of fun working on the material with me.

<comment><USER15> Do you plan to publish the courses?

<comment><USER14> Yes, I will publish it near the end of summer.

I have all the lessons written, but shooting videos and post production takes time.

My daughter also does some of the video shoots. We have to do a few different shoots of each scene and then choose among the best.

</comment></comment></comment><comment><USER16> This reminds me of dorkbot and runme.org. :)

</comment><comment><USER17> does the United States have any such programs?</comment>


Why we moved from NodeJS to Ruby on Rails - <AUTHOR> http://blog.targeterapp.com/post/22984987832/why-we-moved-from-nodejs-to-ror <comment><USER2> I really do think that, as time wears on, we will see more of these incidents of people switching from Node to another, more familiar development environment. It seems to me that a lot of people started using Node simply because of the amount of hype that surrounded it, and many of these decisions were not as well researched as they should've been.

Reading through this gentleman's blog post, it seems that, originally, Node was chosen for just one reason – he wanted to develop the entire application in JavaScript. Other than this, it does not seem that he took the time to research the Node development environment before deciding to develop his application in it.

With respect to the author of this post, the testing environment and the ease of developing a CRUD web application are things that should have been looked into before a single line of code intended for production had been written.

In addition to this, as several other commentators have already noted, Node is not directly comparable to Rails. Node is basically an event framework and standard library that bolts onto JavaScript. Node would be more comparable to something like Python's Twisted or Ruby's EventMachine. The author was probably using one of the frameworks available in Node such as Express.js, Railway.js, or Geddy.

<comment><AUTHOR> Actually since the app was developed at startup weekend which gives you 54 hours to make a presentable demo, I'd say testing was not the first thing on their mind.

<comment><USER3> And it's generally nowhere near 54 hours... more like 42. We nominally started at 6pm on Friday night, but teams weren't formed until 9-10pm. Pitches were demoed Sunday afternoon, and 48 hours from the start on Friday - Sunday at 6pm, it was all over. I've no clue why they SW fixates on the "54 hours" number. 42 would be far closer to the truth, but perhaps more intimidating for people?

</comment></comment></comment><comment><USER4> I only played with Node.js and I didn't like it. The event loop is cool for long-lived requests. It scales and so on.

However for the project I worked on, I decided to go with Ruby on Rails + Java Servlets.

You see, only some parts of an API needs to scale. But what about simple web pages like the viewing/editing of a user profile? What about viewing some stats that are auto-generated? What about the freakishly boring admin interface that every web app must have? What about deployment automation? What about just searching for a library that already does what you want and actually finding a good one?

Therefore I've built a Ruby on Rails app. Everything except the API that needed to scale was built on top of Rails. Then the API with scalability concerns was built as a couple of Java servlets. And I deployed the whole thing on top of Jetty.

The wonderful thing about Jetty is that it has support for continuations, so requests don't have to block on processing in case you're doing something expensive. You can just push that processing in an Akka actor and release the request until a response is ready. And Jetty may not scale as well as something custom built on top of JBoss Netty or Apache Mina, however a single Jetty server does scale to ten thousand requests per sec easily. And in case that doesn't satisfy me at some point, Netty and Mina are there, waiting for me to tap their potential.

And then with the wonderful JRuby-Rack integration, I could configure Jetty to select between the pure Java implementation and Ruby on Rails for serving, based on the URL. So everything, like the Rails server, the Java servlets for the API and the queue processing (Akka) is running in a single process. Which really, is freaking awesome.

So why use Node.js, when I can get the Rails maturity and ease of use, while escaping to Java in case I have special needs, such as extreme scalability or flexibility?

</comment><comment><USER5> I could write a similar post, having used Node on a couple of projects before nodding to reality and switching back to Rails for a larger project.

For a "typical" web project, Rails simply has a ton of mature libraries that don't exist in Node, or are very early days. Sure, there might be a basic data mapper, but does it have support for testing? Automatic counter caching? Workaround on StackOverflow for a hundred things the framework doesn't cover by default?

Rails versus NodeJS isn't apples-to-oranges. Rails is an opinionated framework for web development, Node is a low-level evented framework. This is important because right now, Node has no Rails equivalent. It has Express as the equivalent of Ruby's Sinatra, but for a higher-level web framework, there are several possibilities and none of them the outright leader. This is the situation Python found itself in circa 2006, before Django became the de facto standard for web apps, and (speaking as a mostly-outsider) its seems to me the Python community is better for it. I truly hope a high-level framework establishes itself as the clear standard for Node development, presumably one which embraces the benefits of dual-side JavaScript.

<comment><AUTHOR> There is a project coming up - railwayjs. I hope it changes the way people use nodejs!

<comment><USER6> That's part of the problem in a way. Rails-style large applications are kinda poor by-design. Even Express suffers from this more than I'd like (express- specific plugins etc). Ideally Express "plugins" or extensions would just be thin wrappers around things that the rest of the Node community can utilize. I'm guilty of this too of course but that's largely due to time constraints.

Anyway, writing Rails for Node won't be anything revolutionary, it would just be... Rails on a different platform. Frankly lots of developers including myself prefer these smaller frameworks, sure some things might be a bit slower to get started with but ultimately you're not fighting a framework, it's just another block of lego.

I used Drupal for about 3 or 4 years, and you can get things going much faster than Rails even, but you'll spend a lot of time fighting the thing.

</comment></comment></comment><comment><USER7> When it comes to prototyping, anything goes - you should be optimizing for the speed with which you can deliver and market-test a product, and you should choose the tools that allow you to do so. The trick is having the discipline to pay back the technical debt that accumulates as a result of the fast, informal prototyping work.

I think the OP did exactly that, realizing that Rails is the better long-term solution for their engineering activity. I wouldn't say they "moved" from NodeJS to RoR, because that implies that that they chose a new technology to solve the same problem - they didn't... they chose NodeJS for prototyping and RoR for production.

I had the same experience transitioning from a prototype RoR app to a production app also written in RoR - I still did significant rewriting to improve testability/test coverage and make it more RESTful, but I didn't need to switch frameworks to do that.

</comment><comment><USER8> > NodeJS is suitable for apps that do plenty of short lived requests.

I'm confused. Isn't Node's event-loop style programming ideal for long-lived requests? I.e. ones that, under a synchronous i/o, block other requests?

<comment><USER9> The OP probably meant short-lived with reference to CPU bound requests.

<comment><USER10> OP explained things improperly, then. It's an important detail, because Rails currently defaults to single-threaded. This makes an operation such as fetching and returning a twitter feed in a single request (assuming it takes hundreds of milliseconds to get a response from twitter's servers) expensive.

<comment><USER4> Only rookies do that inside the web server process.

The job of fetching a Twitter feed can be offloaded to a background jobs queue. With a little help from Nginx, you can free the Ruby process to take care of other requests until the response of that Twitter feed is ready.

Or you could simply deploy your Rails app on top of a Java server, by means of JRuby and forward that request to a servlet that uses the continuations support in EE 6, offloading the request to an Akka actor and freeing the pipeline until it is ready. Works great and you can even write everything in Ruby ;-)

Also, Rails does work with multi-threading.

<comment><USER11>
       Rails does work with multi-threading.

Ruby has no support for concurrency, no matter how many threads your interpreter is using. 1.8 had no OS threads at all, and 1.9 has a global interpreter lock. This is not solvable in the application layer (for example, by a framework like Rails): this is a problem inherent to the runtime.

<comment><USER4> JRuby is fully parallel with no global lock and many people choose it for deployment. Rubinius 2.0, which is in development, will also lack the GIL.

You're also very confused about how threading on Ruby MRI 1.9 works. First of all, pure Ruby code in 1.9 can and does execute on multiple OS threads, in parallel.

Also, the problem that the Ruby VM still has is that while executing native code, it does not allow a context switch unless that code explicitly informs the VM that it can do a context switch. This is in effect how the global interpreter lock works. The gotcha here is that native extensions that are well behaved, can inform the VM that a context switch is possible. For instance, the older "mysql" gem was NOT well behaved, blocking context switches between threads, but the newer mysql2 gem does behave well and works correctly in multi-threading.

Right now, if you start a new Rails 3 app, it will work in a multi-threading environment correctly and modern Rails servers are taking advantage of that, unless you install some older gems that haven't been fixed. The biggest problem is that you can't know what libraries are well behaved, but if that's too much of a burden, JRuby is a fully supported platform for Rails and doesn't share the same issue.

So like, seriously dude, do some reading.

<comment><USER11> Pure Ruby code in MRI 1.9 can run on multiple OS threads, but it can't run on multiple OS threads in parallel. Here's some reading you can do if you're unsure:

<http://merbist.com/2011/10/03/about-concurrency-and-the-gil/> [http://www.igvita.com/2008/11/13/concurrency-is-a-myth-in- ru...](http://www.igvita.com/2008/11/13/concurrency-is-a-myth-in-ruby/) [http://jabberwocky.eu/2010/10/02/where-is-the-global- interpr...](http://jabberwocky.eu/2010/10/02/where-is-the-global-interpreter- lock-/) <http://en.wikipedia.org/wiki/Global_Interpreter_Lock>

Yes: Ruby can allow native code to execute in parallel with Ruby code (although it doesn't always do so, as you note). But if you're under the impression that multiple Ruby threads can execute in parallel, you're wrong. That may or may not be a problem, depending on what you need Ruby to do.

JRuby can be a good option for parallelism, but it's also slower than Ruby MRI 1.9 and has an ecosystem that most Ruby developers will be unfamiliar with. Regardless, my point was that Rails doesn't magically "work with multithreading," at least for standard Ruby deploys.

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> This post would benefit from concrete examples rather than vague complaints. For example, if something is "very immature", why not include specific examples of bugs or issues that required upgrading to "the latest version"? Likewise, if Node's testing frameworks are "good" but "no match for … Django", explain what's missing or how other testing frameworks are better. Being precise with your criticism helps the reader put them in context, and also helps maintainers improve their projects. Unsubstantiated criticism merely generates fear, uncertainty and doubt.

</comment><comment><USER13> Wow, Node is being turned away from because it's too new, and Rails is the destination because it's established?

I remember a few years ago Rails using the argument of re-inventing the wheel was the only way because there was presumably no other capable web framework out there.

We all know better now and I hope celebrate choice that's relatively equally capable :)

<comment><USER14> Maybe some day people will finally realize what's so awesome about Ruby on Rails. It's not the libraries and the fact it's been around for almost 8 years now. No, it's simply the language. Ruby is a fantastic language and an addictive one at that. Once you get a feeling of how productive and expressive it is, it's enough of an incentive to keep at it. The same goes for Python by the way, another GREAT language. One day people will realize that Javascript just doesn't cut it.

<comment><USER13> I think you totally missed my point.

I was just speaking to how Railists too often, do not openly consider or think anything productive, well structured, or maybe even great could have existed before, or at the same time as Rails for the web.

We forget the difference between a language and a framework as well. Too many folks short sightedly compare a language to a framework. Frameworks will always evolve and change, and few rarely do anything new or novel, just like languages may, but they're entirely different things.

</comment></comment></comment><comment><USER15> _The data is being written to the db, or being crunched, while the request is ending, and the browser can continue with the important stuff._

It sounds like a good fit for NodeJS for sure, but for what it's worth, even in a synchronous web framework nothing forces you to keep the connection open until the request has been processed.

I don't know off-hand how to do it in Rails, but say in Java Servlet-based frameworks you'd just call HttpServletResponse.close().

<comment><AUTHOR> The point is that the rack middleware would be called for everything, which is very heavy for something so trivial

<comment><USER15> I didn't mean to imply that NodeJS isn't the best fit for the problem at hand. I was just trying to clarify that it doesn't take an asynchronous framework like Node to keep processing a request after the connection is closed.

<comment> mike-cardwell Yeah, even with a simple Perl CGI, all you have to do is close STDOUT if you want the HTTP response to end, but continue processing the request.

</comment></comment></comment></comment><comment><USER16> I suspect -- and I may be wrong -- that the advantages of using Node differ between large teams and small teams. For my first startup, I had the budget to hire a single full-time developer. The was a substantial amount of work to do on both the frontend and backend -- with the backend development needing to get underway first, but the frontend work being much more experimental and needing to go through many more iterations as part of the development process (we understood this upfront).

My developer did his homework, and came to exactly the same conclusions about Node that this article highlights. So he began developing the backend in Python/flask. I'm sure that was a good choice for the backend, on general principle.

However, we began hitting problems when it came time to start developing and iterating the frontend in parallel with the backend: it proved to be almost impossible for a single person to handle the task-switching between two completely different languages, testing frameworks, etc. So backend development lagged a little, and frontend development lagged a _lot_. I'm fairly sure this set the project back by 3-4 months. It wasn't until we brought on a dedicated front-end programmer that development was able to pick up pace a bit.

So I can see how for larger companies, it is probably best to optimise the backend with something other than Node -- you will have enough people working for you that backend and frontend programmers can specialise. But for companies with two (or fewer) developers -- which know that they also have to be doing a quite substantial amount of javascript on the front-end -- then I suspect that Node's comparative small task-switching penalty is probably _hugely_ valuable.

That's just my hunch, but it's lasted a long while. So as I get ready to hire a couple of developers for my second startup, I'm leaning towards using Node. It's a frontend-intensive application which is unlikely to ever have to deal with "web-scale" issues that more mature backends are needed to handle. And if it _does_ ever approach "web-scale", then it'll _certainly_ have the budget to hire additional programmers to re-implement the backend in something else -- or maybe, by then, Node will have evolved into something sufficiently robust. Either way, it seems better than causing ourselves a bunch of task-switching pain upfront.

I'd be really curious to see a critique/comparison of Node which addresses the relative merit (or lack thereof) of its low task-switching barrier -- particularly for small teams.

<comment><USER17> What it this "low task-switching barrier" ? I mean, even though, the language is the same, back end coding and front end coding are really different. For example, your front end development may have lagged because of lack of front end development knowledge, not because of this mythical task switching barrier that everyone is talking about !

<comment><USER18> Maybe not true for everyone, but I find when working on quick prototypes of web ideas, where I have to write both front and back-end code, the fact that they're both in the same language with node.js is actually a huge advantage -- much greater than I expected.

<comment><USER17> IMHO the main problem is that usually front end development implies XHTML/CSS integration and/or UI and/or UX, and that's a very different job from back end development.

</comment></comment></comment></comment><comment><USER19> I read this post as "your mileage may vary", gravitating toward "it doesn't work like we expected". One particular line caught my eye:

"NodeJS' async everything wont have any affect on a CRUD app. Popular frameworks, provide very good caching and everything that is needed for your app to work fine even synchronously."

There is a lot to this statement that implies how an application is architected. Node.js is async at heart, and one can only build a successful application if you follow the programmatic model as it's designed. It doesn't matter how simple or complex an application may be, if a framework is being bent to work in ways in which it wasn't designed, the application may not work so well.

Reading between the lines, what I'm getting is a picture of a team that has encountered productivity issues with the current stack and is migrating to a more familiar stack to increase productivity. Nothing wrong with that tradeoff, but I always look to what future tradeoffs are being made: "cache everything and gets the work done ASAP" is great _until_ you need finely- grained control of your queries.

I'm sure they thought well about their decision, but if asked to make a wager -- I'd guess that certain problems are being traded from one framework to another.

<comment><USER20> I think the point wasn't that "asynchronous is hard", it's that "asynchronous is just incidental complexity". CRUD apps don't need to be asynchronous, and node.js has a relatively impoverished ecosystem for such apps.

There are undoubtedly tradeoffs no matter what you use, but this seems like an eminently practical decision.

<comment><USER19> I'm not so sure about that (the async-is-hard part.) If one is making a decision of architecture change due in part to CRUD capacities within a framework, I'd guess the asynchronous programming model in Node.js is a little more than just incidental complexity.

CRUD operations within R/R might be just a few lines of code, but it's not _that_ much more to accomplish the same with Node, Express/Railway, and any one of a bunch of template schemes.

The change certainly sounds appropriate for the team, but the basis of CRUD as a reason to change an entire architecture leads me to question how the team approached their solution with Node.

<comment><USER20> Unless you're taking the sunk cost of the node.js development into account, I don't understand your point.

If the core programming model for node.js doesn't help them in any way, and the ecosystem is less mature, then it kind of misses the point to say that everything is doable in node.js. Whether something can be accomplished is not a good metric for whether a platform is productive.

<comment><USER19> > If the core programming model for node.js doesn't help them in any way, and > the ecosystem is less mature, then it kind of misses the point to say that > everything is doable in node.js. Whether something can be accomplished is > not a good metric for whether a platform is productive.

Hmmm, don't recall saying everything is doable in Node.js. I was speaking of CRUD operations.

I've had experience with many teams who have made similar architectural decisions based on broad, abstract data points, i.e. programming model, ecosystem maturity, etc. Frankly, making these types of decisions on those over-arching points rarely leads to good decision making. Most often, problems of one type are simply exchanged for problems of another, and it's usually caused by lack of real evaluation of system/architecture needs.

I'm not arguing this team shouldn't make this switch, just that the stated reasons don't pass the sniff test. I think the team prefers to programmatic model of R/R to Node, and feel they would be more successful going forward. But I would be hard-pressed to believe this transition is being made for most any reason beyond comfort and familiarity, as opposed to significant system limitation.

</comment></comment></comment></comment></comment><comment><USER21> This is why it's good to have a lot of tools in your toolbox. Everything has its strengths and weaknesses.

</comment><comment><USER22> The author seems to be confused about what Node.js is.

> NodeJS is a very young framework, and packages around it are very immature.

Node.js is not a framework. It is a JavaScript engine and standard library. Node is comparable to the MRI.

<comment><AUTHOR> First thing is that NodeJS is not a JS engine, V8 is. Second, <http://notinventedhe.re/on/2011-7-26>. That is from <http://nodejs.org/community/> which makes the terms quite right, i'd say.

<comment><USER22> Node _is_ V8 and a standard lib. Node is not an abstraction around http (which is what RoR is).

<comment><AUTHOR> "Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications."

From nodejs.org

<comment><USER22> Right, and "fast, scalable network applications" !== "http abstraction layer". Node doesn't even come with an http abstraction, it comes with the standard request/response objects that nearly every standard lib ships with.

</comment></comment></comment></comment></comment><comment><USER23> Maybe I'm missing something but isn't node for chat/messaging, API's and other stuff that you might have used something like Twisted for?

Why would you use it over Rails/Django etc. for a content-heavy site (i.e. 95% of websites)

<comment><USER24> I use node for everything now myself for a number of reasons. The "problem" with node is that its still very young. And its a different programming model then people are used to. So its similiar to the whole RDBMS/document store argument. You use what works for your applicaiton.

The thing that annoys me personally is that they make it seems like there something wrong with node. And there isnt at all.

<comment><USER23> Does it provide any of the functionality that you get with a Django/Rails style framework? Or is it at a lower level?

<comment><USER24> Yes of course via express.js and railway and several other frameworks. Again node itself is just a set of libraries on top of of v8. So its not a proper comparison.

Personally i use express.js and roll my own framework essentially because i prefer to work that way.

</comment></comment></comment></comment><comment><USER25> The important point in posts such as these is that execution matters, not the tool you use. If the tool you decide to develop with fits, then use it. If you find out later it doesn't work, then switch. Sure, the tool you use can affect your execution, but that only means you have to understand how you want to execute before even selecting a tool.

I think it would be a big mistake to read such a post and come to the conclusion that Rails is somehow superior to node.js.

</comment><comment><USER26> My take on this is that it is more about getting the MVP out with "whatever" (in this case Node), than anything else.

I do this all the time. Some ideas get mocked up in python, others PHP, some in C#, etc. Then if it sticks, I look at the needs it has and pick the best tool for the job. Sometimes it Rails, others its .NET.

Though it is quite interesting to read about it from someone else's point of view.

</comment><comment><USER27> This reminds me of Derek Sivers switching from Rails back to PHP 5 years ago :-)

[http://www.oreillynet.com/ruby/blog/2007/09/7_reasons_i_swit...](http://www.oreillynet.com/ruby/blog/2007/09/7_reasons_i_switched_back_to_p_1.html)

<comment><USER28> Haha great find. Thanks. I agree, I think it would have been important to also say just how quickly Node.JS has developed.

I'm a firm believer it has a very bright future, but this is not a zero-sum game.

</comment></comment><comment><USER29> The argument he makes is that it boils down to maturity. One framework is still evolving and the other is more mature.

<comment><AUTHOR> Exactly the point being raised i'd say!

</comment></comment><comment><USER30> I'd be curious which things were missing in Node. I realise gratuitous detail doesn't make for a great blog post but if Node.js is missing things that stopped development of an app it would be useful for the Node community to know what they are.

Mentioning "immature packages" and "testing" is fine, but I'd really like to know what makes testing on RoR better than, say, testing with Mocha.

I'd also be really interested in the experiences of trying to cache with Node. I think while it's possible to do it's definitely a place that could be improved.

<comment><AUTHOR> IMO, things like a good ORM(or ODM) are still way too far off in NodeJS. I am aware of mogooseJS, but it is very far from the power provided by ActiveRecord and Mongoid.

Also mocha is powerful, but if you have a look at capybara and rspec, they are much more powerful and friendly.

The final thing in my opinion is the speed of development. With rails you go to devise, and you have authentication. I am aware of the presence of a similar authentication system for nodeJS but I am not very sure if that works with mongoose js and is as extensible as devise is. And then there is the rails console, which makes trying out things real easy and fun.

<comment><USER31> I started looking into libraries for a Node-based model layer this weekend. The two most interesting after my initial reading about are Tubbs [1] and Resourceful [2].

I won't have a chance to actually play with them properly until this weekend unfortunately. Anyone else have any experience with these two (or other options)?

[1] <https://github.com/dandean/tubbs>

[2] <https://github.com/flatiron/resourceful>

</comment></comment></comment><comment><USER32> Anyone else find it ironic that this post is about moving from Node to Ruby in order to produce the admin interface to a _Java_ hosting service? (This was a blog post written by "Jelastic" devs, who make a Java PaaS product).

We seem to be seeing more of this "don't eat your own dogfood" approach to software development lately. The other example that comes to mind is the Play Framework, written in Python and also targeted to Java devs.

<comment><USER33> I see what you mean by ironic, but it might be a bit less ironic when you consider the history of the name "JavaScript." Its original market positioning was as a lightweight complement to the heavier Java. Now we have other languages serving in that capacity as well: Python, Ruby, Scala, etc.

I don't know if there will someday be a language that is as easy to work in as a Python or Ruby for simpler programs but which lets you get (almost) as fast and fine-grained as C as your features stabilize and you start to optimize, but while we're waiting for Godot, it's not too ironic to use different tools for different projects, even if the project relates to one language in particular.

(And I don't know why you were downvoted for this. Sheesh, downvoters.... I'm contradicting you a little with my post, but I see no reason why you shouldn't say what you think. So, after contradicting you, I'll now go and cancel your downvote with my upvote. How's that for irony?)

<comment><USER34> >I don't know if there will someday be a language that is as easy to work in as a Python or Ruby for simpler programs but which lets you get (almost) as fast and fine-grained as C as your features stabilize and you start to optimize

Have you looked in to Cython? It seems like it tries to do what you wrote.

From <http://cython.org/>

"The Cython language is a superset of the Python language that additionally supports calling C functions and declaring C types on variables and class attributes. This allows the compiler to generate very efficient C code from Cython code. The C code is generated once and then compiles with all major C/C++ compilers in CPython 2.4 and later, including Python 3.x. PyPy support is work in progress and is mostly usable in recent developer versions."

The latest Gevent dev uses libev (same as node.js) I'm curious to know how Cython and the new Gevent will perform vs the alternatives.

</comment></comment></comment><comment><USER35> First of all I would like to say that I'm no fan of these 54/whatever hour coding rushes, I think it makes people go too fast with poor planning and NO time for testing anything, even for developing looks short, I would rather take my time to plan anything before writing a single line of code.

About Node I have played with it and it looks nice but at the end of the day I just returned to Java and PHP (can't talk about ruby never tried and not sure when I will), why? First, all the Node talk seems to revolve around the hype it generated recently, why I don't know. The idea of the async is nice but stall it with something that takes long to process (and needs to send a response) and you are screwed just like any other language, but I think node may be worse as far as my understanding goes its a single thread dispatching stuff (am I right? please correct me on this if I'm wrong).

Libraries for it, there seem to be too many of the same kind all of them with different ways of working that is not easy, and when I tried it none did what I wanted.

</comment><comment><USER36> Does anyone know 1 major consumer-facing site that was able to scale with Rails?

I know Twitter kind of had to move away from Rails due to scalability issues.

GameBattles.com also had to back away from Rails after their uptime dropped to 80%.

I am not trying to tarnish Rails, I am just curious to learn about some success stories involving Rails at very large scale.

<comment><USER37> Github, Airbnb, Groupon.

Three reasons people think Rails doesn't "scale":

(1) Earlier versions were adopted by applications with anomalous scaling requirements. So, the project had to "learn to scale" on its feet in a very short amount of time. There were predictable hiccups. Or, put even more simply: it's a young framework that got adopted quicker than people expected.

(2) There's an expectation that web platforms can accomodate near-automatic horizontal scaling with maybe just a couple config-file tweaks. That expectation is unrealistic on any platform for which you can easily hire developers. The reality is that scaling to Twitter or even Github's volume is extremely difficult, and the developers who can reliably accomplish it on _any_ platform are extremely hard to find and extremely expensive.

(3) Rails attracts a kind of junior/casual developer that Python just doesn't have, and (weirdly enough) also I think attracts systems developers like myself that aren't steeped in web scaling folklore, which long story short means lots of projects start out by making lots of embarrassing mistakes.

In our practice we see much more Rails today than we do any other framework (there may be some selection bias there, but probably not too much) and for the most part Rails just works. It is definitely _not_ the case that we see lots of startups go out the gate on Rails, go "oh shit we can't scale", and switch to something else.

<comment><USER38> > There's an expectation that web platforms can accomodate near-automatic > horizontal scaling with maybe just a couple config-file tweaks. That > expectation is unrealistic on any platform for which you can easily hire > developers.

I can attest to this. I did consulting work for Match.com a few years ago back when they ran around 20 million uniques/month. They are a .NET shop here in Dallas, TX where you can throw a stone and likely hit a .NET developer.

After a few years, their stack looked nothing like a typical ASP.NET stack. It included things like custom database replication, memcached, config pulled out of files and stored in actual code, and lots of other things that went way off the beaten path from standard practices for .NET. New developers often had to learn "The Match Way" before they could be effective. Many things that worked with typical .NET enterprise apps would break down spectacularly when massive scale was thrown at them.

</comment></comment></comment><comment><USER39> It saddens me that so many developers need to learn why it's a mistake to build a system in node.js right now, just because it's "cool" right now. A variation of this post showed up on HN about 20 times already.

The lesson is simple - don't use node.js to build big apps, it's still immature.

</comment><comment> sasha-dv This reads like my life story. Falling in love with a new technology, trying to use it in production, only to find out that it just isn't there yet :)

Node.js will become a great option for web development, but I don't think that node, at this moment, is ready for "mainstream" use.

<comment><USER40> To me that's the value proposition actually. Invest in learning and collaborating now, then be ahead of the curve in a few years (if interested in getting hired) or be ready to capitalize and know how to hire and make software that only could've been made with it when it eventually reaches some level of maturity. I mean, those that grow it reap the best rewards.

Software is a product of the platform(s) it was built with, in the same way a novel is a reflection of the writer's language more-so than the writer. Microsoft's software is what happens when you use C and .NET; Google with C++, Java, and Python; Facebook with PHP; 37signals with Rails; etc. There are apps that will be built because they could only be built with a unified JS platform. There will be new kinds of software that directly result from the new possibilities of Node, CoffeeScript, Meteor, Firebase, Parse, etc.

Different tools yield fundamentally different results. Once a platform is "mainstream", it's to late, the big opportunities have already passed and the innovation is elsewhere. There's still a chance to be the "DHH of Node"; that spot's obviously already been taken in the Ruby world.

<comment> sasha-dv Learning new technologies is something I do, not because I'm interested in getting hired or in becoming the DHH of node. I would do it even if I were by profession a truck driver.

This may sound "romantic", but when your love of technology interferes with you actually being productive and getting things done, ... it's frustrating.

If you are a student, then jumping on a new platform and making a name for yourself is a great thing to do. But I'm an old fart, I was there when Java applets were "the thing that will change the world".

>There will be new kinds of software that directly result from the new possibilities of Node, CoffeeScript, Meteor, Firebase, Parse, etc.

Like what? (not rude, just curious)

As far as I can see there's nothing new under the Sun, so to speak.

<comment><USER40> I hear you, and can relate to the romance and frustration! I was in grade school when JS was released... but've been writing it for over 10 years now. I tried "everything" else, and it's made be a better JS hacker, but I feel like I'm doing it wrong when I use anything else.

I didn't mean to suggest riding the wave of new, distinct, non-progressive technologies. Their _is_ a progression; it seems clear to me that PG was right, that we (as an industry) are slowly moving toward Lisp. He also said that he felt Lisp and C represent the two "clean, consistent" programming models, and I agree. And that's the reason that JS isn't just another passing wave (although my previous comment did seem to make that suggestion).

JS is something different; it's (1) a good-enough balance of C and Lisp and (2) available on every platform. To me, the situation is clearly that JS will form a solid, durable layer over C. Then the language designers and industry hype machine will shift to langs that compile to JS. This is already happening, naturally, it's just not evenly distributed.

> Like what?

We're still in the early days of compiling to JS, and being able to use it as a modern server-side environment. I don't know what the results will be, but I think the difference will stem mostly from development time. Even though it could be done, in practice you don't end up with the same app if using Fortran, Java, and CoffeeScript, because doing so would take a month versus a week versus a day. To paraphrase Linus from his Tech Talk on Git, speed doesn't mean you do the same thing faster, it changes behavior.

Apps will get written that wouldn't've. I'd argue that Facebook succeeded mostly because of PHP, and in turn, that they'll eventually fail because of it, too. They beat MySpace because MySpace used the MSFT stack; it's like England's victory over Spain due to more nimble war ships. Same thing'll happen to FB unless they evolve when necessary.

This is what Yegge was getting at back in '06 [Dreaming in a Browser Swamp]. He mentioned "Scheme on Skis" and "JavaScript on Jets", which might turn out to be ClojureScript and Express or Railway. Well on our way, and he was entirely right in retrospect (even though a shocking number of smart hackers don't want to accept it and keep on with archaic tech that's becoming rapidly endangered, but that's life).

Light Table is an early example of an app that's happening because of ClojureScript. And many exiting CoffeeScript apps are quite impressive: <https://github.com/jashkenas/coffee-script/wiki/In-The-Wild>

I think it comes down to being able to focus on design and make fast changes, without needing to worrying as much about the lower layers of abstraction.

</comment></comment></comment></comment><comment><USER41> I wonder what testing functionality, specifically, they see missing in the Node ecosystem...

<comment><USER42> I'm writing a server in Node and to date I'm still trying to figure out how to control the main loop from Node. I want to write tests like this:

        1. register some timers and create some sockets
        2. send some stuff over the sockets
        3. wait until the socket handlers received some data which cause the handlers to unregister the sockets and timers
        4. at this point, the main event loop has ended. go to next test and setup a new event loop

How do you do (3) and (4) in Node? I have no idea, and I couldn't find any clear answers on the Internet. I ended up writing my tests in RSpec which launch Node processes, and I just check whether the stuff printed to stdout matches what I expect. RSpec also happens to have a much much nicer syntax than any Javascript testing framework I've encountered.

<comment><USER43> That sounds like a complicated approach at it. Here's an example of some NodeUnit tests that involve communications with a Redis-based message queue:

[https://github.com/bergie/noflo/blob/master/test/MQ_SendMess...](https://github.com/bergie/noflo/blob/master/test/MQ_SendMessage.coffee#L46)

More BDD-ish version of something similar:

[https://github.com/nemein/kckupmq/blob/master/test/001_kckup...](https://github.com/nemein/kckupmq/blob/master/test/001_kckup- mq.coffee#L111)

<comment><USER44> Does that test you link actually talk to a live Redis-based message queue that must exist for the test to run?

If so, while that's a nice test there's also a need to be able to fully control the event loop so you can test arbitrary event orders, some of which you may not be able to synthesize (or not easily, or not reliably) but may be able to occur in the real world. Testing event loop code sucks hard enough when you can fake events perfectly correctly, take that away and it sucks even worse.

<comment><USER32> _> there's also a need to be able to fully control the event loop so you can test arbitrary event orders_

Hmm, something doesn't sound right there. I'm not really sure why you'd want to "fully control the event loop" for testing, but if you did you can install a callback on every tick: [http://nodejs.org/api/process.html#process_process_nexttick_...](http://nodejs.org/api/process.html#process_process_nexttick_callback)

If you want to test arbitrary inbound event sequences, you'll need to run at least two processes. You could run two node processes, one generating events for the other. This would probably give you a better test environment then hacking the event loop, too.

<comment><USER44> "I'm not really sure why you'd want to "fully control the event loop" for testing,"

Because while I'm actually one of the first to tell you that not everything needs to be a unit test and I tend to prefer integration tests over unit tests if given a choice, I will also say that unit tests still have their place, and the way you unit test an event-driven program is to feed the events you specify to an event loop and ensure that the proper actions taken, using whatever is necessary to ensure proper independence.

It's nice to _be able to_ do integration tests, it is not nice to _have_ to do integration tests.

You know, this may explain a great deal of the gap between my opinion of event-based programming and the conventional Node opinion. Testing event-based code isn't necessarily _that_ hard (testing is never really easy in a language where you're always in the local IO monad equivalent, though cracking it up into events does make things challenging, especially if you try to take advantage of closures in your handlers, which unfortunately is advantageous in other ways but hurts testability hard) but it doesn't help that the frameworks never seem to ship with the testing support I just specified and I'm always having to bash it together myself, which tends to show a certain... disconnect in expectations between me and the authors.

</comment></comment></comment></comment></comment></comment><comment><USER45> If you think how the whole web development has been changing from server-side generated views to more "produce data and send it over as JSON" style services, Node.js and others start to make a lot more sense. That said, the post has point that on more established platform you can expect to be more productive since you have more established work flows and standard libraries.

Btw. note the terminology: Node.js should not be called "framework", it's a platform. "Framework" is something that runs on platform and you extend/modify it's functionality by writing code. "Library" is something which you call/use from your code.

</comment><comment> brown9-2 _Since our stack involved MongoDB, it only made sense to live in a JS only environment._

I've never used MongoDB; can someone explain to me why this would seem like an advantage, and/or what it has to do with MongoDB?

Is the MongoDB API simply JavaScript?

<comment><USER24> Mongodb is a json document store. I use it myself. It works very well with "node" because node of course uses javascript as its language.

<comment><USER46> It's not JSON, it's BSON. BSON supports more data types, which are needed for a proper database (such as object IDs, longs, doubles, binary data, references, etc.)

You still need a driver and a bson serializer if you use node.js.

<comment><USER47> Sure. But it's certainly easier than most other languages. It's not a great reason to switch for that, but working with Mongo is fairly trivial in Node.

</comment></comment></comment></comment><comment><USER48> Basically they switched for... personal opinion. That's all I can really gleam from the post. You can do it all in Node JS, they just didn't want to.

Screw all the frameworks, I am working with bare-metal Node and loving every minute.

<comment><USER49> definitely agree with this. I can understand why 37signals use RoR, because they built it and know it inside out. But for a newcomer diving into rails now is difficult because the ecosystem is so huge it's hard to navigate, and everything is so integrated and you usually end up bundling a ton of thin that you don't need. With node everything is modularized, it is much more easier to understand what's happening under the hood, and streams are freakin awesome.

<comment><USER50> I disagree with this. By the ecosystem being huge do you mean the modules and different gems? That in itself is modularized for having the choice not to use them. The availability is what makes it powerful on top of the simplicity to get it all running.

Node is by far not simple for a new developer or new teams running node. Projects tend to drift toward unreadable due to poor design, just try following people's callbacks. I am not saying this is Node's fault but it being young results in developers' lack of experience.

<comment><USER24> Im sorry but you make it sound like everybody that does rails is a perfect programmer. Rails itself has changed over the years and if you look at an app from years ago im quite sure it will be very different then a current rails app. So again this is just not an appropriate comparison.

<comment><USER50> I don't think they are perfect programmers. Heck, I use several languages and frameworks, whatever is right for the project. Knowing a language isn't a perfect programmer (we all know that). And to your point, that was the comparison I would make. Node is young so there is a lack of experience with it.

</comment></comment></comment></comment></comment><comment><USER51> >I am huge fan of NodeJs and I believe it's a very technology and we will see it getting

I think you accidentally word.

<comment><USER52> Thanks. Fixed now.

</comment></comment><comment><USER53> Ok. So for a beginner coder looking to build web sites and mobile apps on weekends. They are not looking for a career or job coding. What language?

Assume they know html, CSS, Javascript, SQL. Where do they go and spend their next 12 months?

When do the differences matter?

</comment><comment><USER54> i think what i'm enjoying most about node.js and the new frameworks, is that they require me to think differently. i'm also optimistic about coding end-to- end in the same language (client, server, database). this is a big learning curve for many folks. i remember when folks transitioned from C to C++ and from C++ to Java -- it takes awhile. i think this is going to be a lot of fun to see where it takes us. also i think work like coffeescript and future IDEs might make this really interesting. Also meteor -- lots of creativity and cool things.

</comment><comment><USER55> I'd love some elaboration on these points, especially the one about testing as it's an area that interests me.

</comment><comment><USER24> For anybody wanting to learn about node the NodeUP podcast is excellent.</comment>


The China Bust: Tic Toc - <AUTHOR> http://mises.org/daily/5701/The-China-Bust-Tic-Toc

<comment><AUTHOR> One sign of the bust, "It's A Boat, It's A Plane, It's The Great Wall Of China: Part Of Symbolic Chinese Landmark Collapses".

[http://www.zerohedge.com/news/its-boat-its-plane-its- great-w...](http://www.zerohedge.com/news/its-boat-its-plane-its-great-wall- china-part-symbolic-chinese-landmark-collapses)</comment>


Odoo: The new OpenERP - <AUTHOR> https://www.odoo.com/ <comment><USER2> If you look around there is no "beautiful ERP" solution out there and the status quo is "you can make a very capable ERP or a "pretty" ERP but not both". I understand the the difficulty of striking that balance but I think by applying the right kind of modularity to the app design (sort of like openERP but even more granular and forgiving) you can arrive at a solution which invites businesses of all-sizes but can grow with the company into very complex processes.

There is HUGE opportunity in this market and I believe one big reason is that a new generation of userbase will expect and demand a better UX than any current offering.

In fact, I suggest a great startup idea would be a to take the openerp codebase and wrap a new UI/UX around it starting with one vertical ("niche") market and working out from there. (i.e. "A beautiful ERP for medium-sized businesses focusing on your growing warehouse, sales and purchasing needs built on the solid foundation of openERP."). Not saying it would be easy, but just worthwhile. I have a feeling the Odoo rebranding points to an effort for a UI overhaul which acknowledges the opportunity of a great UX.

<comment><USER3> This is nonsense. An ERP has to run your business and operations. It has to be extremely reliable. Users couldn't care less about it's UX, in fact, the standard ERP solution, SAP, looks terrible, but it is also quite ok to use. In a big company an ERP error/fault can generate millions worth of damage, are you still thinking a prettier UI is a great start up idea for an ERP?

Finally, you forget that IT departments in big companies are just a tad above trained monkeys and your software solution has to take that into account, in fact, it's the first piece of your service: it has to be easy to maintain. It's probably here where you want to focus on to compete...

<comment> UVB-76 _> This is nonsense. An ERP has to run your business and operations. It has to be extremely reliable. Users couldn't care less about it's UX [...]_

Indeed. When a person's livelihood depends on using your software, they have a pretty strong incentive to learn how to use it effectively, and put up with all its imperfections.

I'm sure some employees actually revel in how backwards and difficult to use their business software is. If everything were automated, effortless, and required no expertise, they'd be out of a job.

</comment></comment></comment><comment><USER4> By the way, the core and base modules (and in theory, all community modules, since they're derived works) of Odoo are Free Software (APGL) and available on Github: [https://github.com/odoo/odoo](https://github.com/odoo/odoo)

Disclaimer: I work for a company that implements solutions based on Odoo for small and large companies. So yeah, it's possible to make a living working on FOSS, if anyone still doubted it :)

<comment><USER5> they still use their custom-own-built orm ?

<comment><USER6> I cannot understand why anyone would not use SQLAlchemy if they're building a database-backed Python application.

<comment><USER7> True today, but not 10 years ago :)

Plus Odoo ORM and SQLAlchemy have different purposes.

</comment></comment></comment></comment><comment><USER8> OpenERP could've really benefited from some usability expertise, and if this site is any indication, Odoo may suffer the same fate. If you're going to put this much time/money into a new brand, how could you miss so badly?

In my experience with OpenERP, the end users always took a back seat to the developers. Granted, ERP software is inherently complicated, but with each new version you could tell they made an attempt at a simplified interface and never hit the mark.

<comment><USER9> OpenERP is still leagues above the million dollar paid apps of Oracle and SAP. I'm not saying you are wrong. Just how bad this particular segment of software is. It is wripe for disruption.

<comment> UK-AL ERP software is huge, simply because there so many different ways businesses works, all the regulations you have to comply with etc etc

It would be very hard for a startup to build it from scratch.

</comment></comment></comment><comment><USER10> Not sure the rebrand is what will attract new customers to OpenERP.

A couple months ago, I evaluated ERPs for our business. OpenERP was one of the main contenders but ultimately the manufacturing module failed to meet our requirements and custom development would have been too costly.

I was also shocked to get a glimpse at OpenERP's internal corporate structure. Despite the open source nature of the project, their business model puts them on track to become yet another big, slow, expensive ERP company pushing massive service fees.

Unfortunately there's just no great solution in the ERP world. Requirements for this category are so complex that by the time a system becomes adequately feature rich, improvements in software technology make it feel obsolete. I wonder if someone will crack that nut.

<comment><USER11> Exactly. Every company's business process has it's own uniqueness, so if you try to evolve it into a mass-market tool (as opposed to designing for it up- front), you end up with a Frankenstein system that is enormous and complicated. Then your company starts morphing to match what you're selling.

If some YC'ers want to make some serious cash, go build the JIRA of ERP. Make the system: pluggable, VERY configurable, give it a sexy UI and don't require an army of consultants to implement. Want some reasons? Pretty much all your potential customers hate their existing ERP system. Your competition is staffed by sleazy salespeople that sell nothing but snakeoil. Your market is filled with companies of all sizes who will literally throw cash at you to make them more efficient.

</comment></comment><comment><USER12> What are the alternatives to OpenERP today ? (I mean, free and open sources ones, with acceptable UI)

<comment><USER13> There is [http://tryton.org/](http://tryton.org/)

however I don't know what you mean by "acceptable" UI

<comment><USER12> For me, an acceptable UI in an ERP means this ERP can be used to do basic tasks by someone not having any PhD ni ERP Usage :)

</comment></comment></comment><comment><USER14> Icons!

As a designer, I've probably never seen so many icons on a page.

</comment><comment><USER15> I've considered working with OpenERP and looking at becoming a consultant in it (ERP is $$$ and very useful), but last I checked the OpenERP site (2? years ago?), the demo site was half broken. Glad to see it's gotten a refresh and a genuinely web 2.0 name. ;)

Apparently the devs have churned it into a SaaS offering, which is also very cool.

<comment> UK-AL is OpenERP contractor fees that large?

I know SAP and Oracle Fees are high, simply because its quite hard to get opportunity to work on it, and all the certifications. So there aren't that many people around.

<comment><USER15> Well, I'm not sure what the rate is. But I know that it supplies a _very valuable service_ , and other things in the space are very lucrative. I imagine that part of the lucrativity is the value that it provides to businesses.

<comment> UK-AL Salary does't always mean value provided.

Salary = Supply/Demand.

</comment></comment></comment></comment><comment><USER16> They also discovered how to succeed as a tech company: Better use at least three 'O's in your brand name.

"Choosing the right name is hard. To find the perfect name, we analysed the names of top internet companies. We discovered a direct correlation between the company valuation and the number of 'O' in its name. The graph below shows the average valuation of the top 10 internet companies according to the number of 'O' in their names.

With three 'O' in its name, 'Odoo' is in phase with our ambitions."</comment>


Ask HN: I just wrote an O(N) diffing algorithm – what am I missing? - <AUTHOR> Hey folks,
I've been building a rendering engine for a code editor the past couple of days. Rendering huge chunks of highlighted syntax can get laggy. It's not worth switching to React at this stage, so I wanted to just write a quick diff algorithm that would selectively update only changed lines.
I found this article: https://blog.jcoglan.com/2017/02/12/the-myers-diff-algorithm-part-1/
With a link to this paper, the initial Git diff implementation: http://www.xmailserver.org/diff2.pdf
I couldn't find the PDF to start with, but read "edit graph" and immediately thought — why don't I just use a hashtable to store lines from LEFT_TEXT and references to where they are, then iterate over RIGHT_TEXT and return matches one by one, also making sure that I keep track of the last match to prevent jumbling?
The algorithm I produced is only a few lines and seems accurate. It's O(N) time complexity, whereas the paper above gives a best case of O(ND) where D is minimum edit distance.

  function lineDiff (left, right) {
    left = left.split('\n');
    right = right.split('\n');
    let lookup = {};
    // Store line numbers from LEFT in a lookup table
    left.forEach(function (line, i) {
      lookup[line] = lookup[line] || [];
      lookup[line].push(i);
    });
    // Last line we matched
    var minLine = -1;
    return right.map(function (line) {
      lookup[line] = lookup[line] || [];
      var lineNumber = -1;
      if (lookup[line].length) {
        lineNumber = lookup[line].shift();
        // Make sure we're looking ahead
        if (lineNumber > minLine) {
          minLine = lineNumber;
        } else {
          lineNumber = -1
        }
      }
      return {
        value: line,
        from: lineNumber
      };
    });
  }
 RunKit link: https://runkit.com/keithwhor/line-diff
What am I missing? I can't find other references to doing diffing like this. Everything just links back to that one paper. <comment><USER5> Diffing means computing the longest common subsequence (LCS); the edit script is everything that's not part of the LCS. The proposed algorithm greedily finds a matching element. However it may be that it would be better to skip this element, and not incorporate it into your sequence.

Consider the following sequences (here each letter represents a line).

       AAAB
       BAAA

Where the first one is "left" and the second one is "right".

The LCS is AAA. The minimal edit script from left to right is:

1\. Insert B at beginning.

2\. Delete B at end.

The proposed O(N) algorithm finds B as the first matching line, and thereby exhaust `left`. So it finds the common subsequence of just "B" and its edit script is:

1\. Delete 3 As at beginning

2\. Insert 3 As at end

which is longer.

<comment><USER6> Is it subsequence or substring? Don't you want to find the longest common substring, i.e. contiguous block? I always understood subsequence to be non- contiguous.

<comment><USER7> It's subseqence. For instance, in

ABXCDYEFGZ QARBCSTDEFG

You want to find [Q]A[R]B(X)C[ST]D(Y)EFG(Z)

ABCDEFG is the longest common subsequence, QRST are the additions and XYZ are the deletions.

</comment></comment></comment><comment><USER8> I spent a bit of time learning the Myers Diff algorithm, (I made this page: [https://blog.robertelder.org/diff- algorithm/](https://blog.robertelder.org/diff-algorithm/))

Like someone else said, I think the key is that you're not getting the shortest edit script, you're just getting _a_ edit script.

Interestingly, even git's implementation of diff (which uses a variant of the Myers Diff algorithm) doesn't actually find the shortest edit script by default. It has a hard-coded heuristic that just quits early if it finds a 'good enough' solution:

[https://github.com/git/git/blob/master/xdiff/xdiffi.c#L143](https://github.com/git/git/blob/master/xdiff/xdiffi.c#L143)

Same this for unix diff (unless you specify the commandline flag to find shortest):

[https://github.com/Distrotech/diffutils/blob/distrotech- diff...](https://github.com/Distrotech/diffutils/blob/distrotech- diffutils/lib/diffseq.h#L255)

Having said that, if you're still confident that you've made an improvement, you can port the function to python and try it out as a test case in my unit test file that I wrote when I developed my variant of the Myers Diff algorithm:

[https://github.com/RobertElderSoftware/roberteldersoftwaredi...](https://github.com/RobertElderSoftware/roberteldersoftwarediff/blob/master/myers_diff_and_variations.py)

On that topic, I'm pretty sure my variant of the Myers Diff algorithm is actually slightly faster/uses less memory than the original (I'd be happy to be proved wrong). At least, I've had my blog post up for a few years and no one has corrected me, but then again, I think the original paper has a few mistakes/obvious flaws and it still has hundreds of citations and no one has written much about improvements.

<comment><USER5> Nice work on your visualization! Thank you for rebutting the Fraser article; I came to the same conclusion as you (that Myers algorithm is in fact correct) but had never written it up.

I wrote a vectorized Myers implementation for diffing binary files. Perhaps the most interesting aspect is the progress bar: it's not obvious how to create a monotonic progress estimate for Myers diff.

[https://github.com/ridiculousfish/HexFiend/blob/master/frame...](https://github.com/ridiculousfish/HexFiend/blob/master/framework/sources/HFByteArrayEditScript.m)

<comment><USER8> Cool. You've got a fairly impressive following on that tool. I wish more people took an interest in studying these sorts of algorithms. Most people assume that some genius figured out the optimal way to do things years ago, but more often than not if you spent a few weeks focusing on a specific paper/algorithm, you'll come up with ideas for improvement (although it might not always be in the way you initially expected). Sometimes the improvement is just figuring out a better way to explain it and that in itself can lead other people to create breakthroughs.

I think there are still a few improvements that could be done the the Myers algorithms. One specifically, is that I'm pretty sure it's tail recursive. You won't get any big O improvements off of that, but less stack space is still an improvement. Plus better cache locality. I'd work on some demos, but too many things to do these days...

</comment></comment></comment><comment><USER9> Since the problem has already been resolved, I'll go on a slight tangent and mention there are some very interesting probabilistic diffing algorithms invented by the bio-informatics people.

E.g [https://en.wikipedia.org/wiki/BLAST_(biotechnology)](https://en.wikipedia.org/wiki/BLAST_\(biotechnology\))

</comment><comment><USER10> Slightly off topic, but one great way to diff is to compress the new file using the old file as a dictionary. Both zlib and zstd support this (but zlib is limited to a 32KB file).

You can tune how much CPU you want to spend getting a better diff by adjusting the compression ratio, the compressor is well tuned for speed and is generally more efficient than a diffing algorithm, it can find repetitions in the new content, and you get entropy coding.

<comment><USER11> That's really neat, do you maybe have any links on this?

</comment></comment><comment><USER12> You're most likely not getting a minimal diff, but it's highly likely that your diff is good enough for your use-case. React and most other similar VDOM libraries also fail to achieve minimal diffs, however, that's not necessarily a bad thing. Just because a diff isn't minimal doesn't mean it's wrong.

The ideal trade-off (diff-size vs space-time characteristics) depends on the use-case. It might be more important for a VDOM library to obtain a good- enough diff in O(N) time, as opposed to an LCS in O(NM) space and time. Linear diffing for VDOMs works well, especially with heuristics and things like explicit and implicit key matching. Sometimes the asymptotic characteristics of a program are irrelevant if your system is bounded.

<comment><USER13> React doesn't compute diffs, and it would be wrong for it to do so since components are stateful, and so it matters how they are matched (the idea that VDOM diffing is just an optimization is a very common misconception).

Instead, you just specify keys for elements (either explicitly or by index in the sequence) and it matches elements with the same key in the old and new trees.

<comment><USER12> > React doesn't compute diffs

I think this is a bit pedantic. React computes what is essentially an edit path from one tree to another, so yes, it is computing diffs for all intents and purposes. That said, you're right that implicit and explicit key matching are not _only_ for optimizations; they're for persisting state as well.

</comment></comment></comment><comment><USER14> I just found an O(M + N) diffing algorithm:

      $ my-diff yours mine
      --- yours 2019-10-28 15:11:07.150118090 -0700
      +++ mine  2019-10-28 16:46:42.093923066 -0700
      @@ -1,50 +1,75 @@
      - <USER4>
      - <USER4>
      [...]
      - <USER4>
      + mine
      + mine
      [...]
      + mine

Spit out the diff and hunk header, then dump yours preceded by "\- " and then mine preceded by "\+ ".

<comment><USER15> There's a non-trivial sense in which you'd be justified calling this O(1), inasmuch as the edit script "delete yours, insert mine" can be yielded up as an edit script without even reading "yours" or "mine", simply yielding "yours" and "mine" by reference. That's in a more abstract sense than something "diff" could take. However, with at most a slight change to "diff" (and I'm not sure it's even that, I think it may have this already I'm just not checking for an HN post), you can yield an O(1) "delete yours" and then insert mine, so it's O(N) without the M [1].

In UNIX terms you can just spec out "cp mine yours" as the edit script, which is O(1) from the perspective of the diff algorithm. Applying that script is O(N), of course, but generating it is constant.

1: Super pedants will note that O(N + M) isn't terribly meaningful here since adding two linear terms means you can reduce that to O(N) under the big-O rules. However, I've always like keeping the information like that in the O notation, in some sense precisely _because_ O(N+M) is-equivalent-to O(N) in big-O terms, but if you have additional knowledge like for some reason, M is exponential in a term you care about you can generate the new big-O term that you care about. In graph theory, my professor tended to want to express O(edges) as O(nodes^2), but I always liked retaining the O(edges) because O(edges) yields useful information if you know you have a sparse graph, for instance, whereas O(nodes^2) removes that. To his credit, he never marked me down for it, which puts him ahead of a lot of professors and their personal preferences...

<comment><USER16> O(N + M) is not equivalent to O(N) if you make no assumptions about the relative growths of N and M, that's why it's actually meaningful to keep both terms around. You can only reduce it to O(N) when N dominates M (M = O(N)).

<comment><USER15> In the case of file size inputs to a diff algorithm, it would not normally make sense to discuss how one file is going to be "exponentially" larger than another or something. They just are what they are, the bare inputs to the function in question.

By contrast, graphs can have significantly different relationships between nodes and edges, for instance, with sparse vs. dense graphs.

One could construct a situation in which file size inputs are varying related to each other in some O(...)-significant way, but it would be very strained, especially in the context of a _diff_ algorithm (why are you trying to diff things significantly different in size at all?). You can fiddle with almost any O(...) pronouncement that way, e.g., "adding two integers isn't O(1)". True enough, but generally, more detail then we are looking for.

<comment><USER17> You're wrong, the sizes of both input files are commonly considered when analysing algorithms that work on two files. Consider a problem that could be solved by sorting one of the files and then stepping through both files item by item (not saying that any meaningful problem with this property exists, it's just an example). Then you would have a complexity of O(n log n + m) assuming n is the size of the file being sorted.

There is valuable information in running time being linear in the size of both files. You could define _n_ as the combined size if you wanted to, but that could be confusing.

</comment></comment></comment></comment></comment><comment><USER18> Does this algorithm find the _shortest_ edit script?

Finding any diff is not really the problem.

<comment><USER19> I think shortest is probably a good proxy if we can't find something better but I think in an editor "easiest to understand" is probably what's actually important.

Not that I have any particular expectation that this algorithm does unexpectedly well at that.

<comment><USER20> Yes. That reminds me of lossy image compression: the real goal is to find a representation that looks as close as possible to the input image to the human eye. In practice, we use proxies.

Not all proxies are equally good. Eg just going by Hamming distance or L2 norm distance would be awful. Proxies that take more features of our visual system into account are going to fare better.

</comment></comment></comment><comment><USER21> This doesn't answer your question but you really should read up on Needleman- Wunsch for matching up strings.

Love this post by the way, wished we would see more of these on HN.

</comment><comment><USER22> Ah, no, the Myers algorithm is also known as the "Sortest Edit Script" and your algo does not follow the shortest path or produce a script (just -1 for mismatches).

More specifically, it's not just something at the end appearing at the beginning causing skipping like others have pointed out. The skipping can happen in the middle. Consider the two sequences A B B C and B C A. Your algo will match the first Bs and then the Cs and skip the common B C together.

Even if you could somehow make it work, it would be O(N) but note that with all of the split() and map() and array indexing, it starts to be more like O(N+NX) where X is that potentially non-insignificant overhead.

Don't let me pooh-pooh you from trying but if you do give-in and use the Myers' algo, here's my C diff implementation:

      http://www.ioplex.com/~miallen/libmba/dl/src/diff.c

This is actually pretty compact as diff implementations go and would probably be a reference for implementing diff in JS.

<comment><AUTHOR> Thank you: I'll definitely reference this if (and when) the bottleneck becomes our diffing implementation, or if as part of the editing experience we want to be able to show diffs between files that aren't stored in git.

Keep in mind that right now the implementation is for an in-browser code editor. The importance of the diff algorithm is primarily to quickly determine which HTML elements to re-render on a line-by-line basis. The changes a user makes between frames are at best single-grouping insertions / deletions (cut, paste, delete, insert). For a 2k LOC file, the render step (pre-diff) was on the order of 200ms: I was repopulating thousands of DOM elements every time a line was changed. Now it's at like 3ms, so under the 16.67ms frame cutoff.

I'm floored by the responses here though, this has been super helpful! I knew I was missing something academic here. :)

</comment></comment><comment><USER23> Question: why do you ensure that you are only moving forward? Does it matter, for line highlighting, that a line has moved up, or now appears twice?

</comment><comment><USER24> Your approach is called hash join in databases.

Not sure about the diffing research though.

</comment><comment><USER25> React has absolutely nothing to do with that.

</comment><comment><USER26> hash map worst case insertion/search is O(N) so this is potentially worst case O(N^2). IIRC a typical implementation would be more likely to be O(N log N) for some high log factor in the limit, but still worse than O(ND). And this is before we account for the fact that hashing a line probably has an implicit factor of D in it.

<comment><USER27> I have just been interviewing at several companies that have highly technical interviews in London, and about 80% of the interviewers who asked questions about hash maps expected me to assume that hashmaps imply constant-time lookups.

People tend to dismiss the statistics related to hash collisions because they think they are absolutely irrelevant to "real-life scenarios".

<comment><USER28> This is because hash tables resize when they are filled with too many elements. Assuming a suitably good hash function, this gives O(1) lookup w.h.p. and insertion stays O(1) amortized. The mathematics of hash collision probabilities is factored into the resizing thresholds.

Many standard library implementations of hash tables (such as rust's) also include special hash functions which are salted with random values to prevent DoS attacks which create large numbers of hash collisions.

<comment><USER20> If you are really paranoid, you can even use a salted cryptographic hash function. But the constant factors for those are usually worse than for simpler salted hashes, so they are not worth it for hashtables.

Your fallback for collisions could also be something with O(log size_of_bucket) runtime, instead of linked lists. But again, when you don't have many collisions, that's going to be slower than something simpler.

(I half remember a result that if you tune your parameters right, you can get away with a hashtable that only has enough size for storing one element in each slot; and if a collision happens, you replace that element with a tomb stone, and store put the elements involved in a single global linked list.

Basically, for that to work you need to keep your total number of collision smaller than a constant with high probability.)

<comment><USER29> Once your hashes collide, you have no other mechanism to use besides equality. How can you use something other than a list in order to find the actual value? This being for the general case where items don't have an "order/rank/sort" property between them.

<comment><USER20> Most of the time, when you talk about collisions we mean that two different keys get hashed to the same bucket.

What mechanisms you have available depends on your implementation.

For example Cuckoo Hashing ([https://en.wikipedia.org/wiki/Cuckoo_hashing](https://en.wikipedia.org/wiki/Cuckoo_hashing)) relies on having two different hash functions available.

And yes, for having something like a balanced tree, having a comparison would be useful.

In general, most hash functions work by consuming something like a stream of bits. So for your implementation it makes a lot of sense for the datatypes you want to use as keys to export a way to convert themselves into that stream, and leave the actual hashing into a fixed size as a detail of your hashtable.

That way you can eg do fall-back comparisons directly on stream of bits (including for equality). Or you can transparently support multiple hash functions.

Even in languages where the hashable interface works by giving you a method to spit out eg a 64 bit number only, you still have to map that number to one of your buckets. So for your fall-back, you can choose a different mapping.

<comment><USER29> My point was "what else can you use besides a linked list to store hash- colliding values?"

If you have a second (or a third, or a fourth...) hashing algorithm, then make it hash tables all the way down. At the end, you still need some data structure to store hash-colliding values. And if so, what other structure could you possibly use besides a list (linked-, array-, etc.) ?

<comment><USER20> > If you have a second (or a third, or a fourth...) hashing algorithm, then > make it hash tables all the way down. At the end, you still need some data > structure to store hash-colliding values.

Why? You can have just two layers: the primary hash table and your buckets are made up of one small secondary hashtable each. If there's a collision in the bucket hashtable, pick a now random hash function for that bucket and re-hash everything in the bucket.

If that fails after trying a few times, pick a new random hash for the primary hash table and consider resizing.

I bet you can make that scheme workable in O(1) expected amortised time for inserts and lookups.

Cuckoo hashing ([https://en.wikipedia.org/wiki/Cuckoo_hashing](https://en.wikipedia.org/wiki/Cuckoo_hashing)) is a related idea: you just have to hash functions. If you had three elements that collide for both hash functions each, you just re-hash your entire table.

(From Wikipedia:)

> When a new key is inserted, and one of its two cells is empty, it may be > placed in that cell. However, when both cells are already full, it will be > necessary to move other keys to their second locations (or back to their > first locations) to make room for the new key. A greedy algorithm is used: > The new key is inserted in one of its two possible locations, "kicking out", > that is, displacing, any key that might already reside in this location. > This displaced key is then inserted in its alternative location, again > kicking out any key that might reside there. The process continues in the > same way until an empty position is found, completing the algorithm. > However, it is possible for this insertion process to fail, by entering an > infinite loop or by finding a very long chain (longer than a preset > threshold that is logarithmic in the table size). In this case, the hash > table is rebuilt in-place using new hash functions: [...]

You say:

> And if so, what other structure could you possibly use besides a list > (linked-, array-, etc.) ?

Any datastructure you feel like. You can also use a balanced search tree, if you want to. See eg [https://en.wikipedia.org/wiki/Hash_table#Separate_chaining_w...](https://en.wikipedia.org/wiki/Hash_table#Separate_chaining_with_other_structures)</comment></comment></comment></comment></comment></comment></comment></comment>