Deploying ECMAScript 6 - <AUTHOR> http://www.2ality.com/2015/04/deploying-es6.html <comment><USER2> I know I'm far from the only one but I just wanted to throw out there that my company is now using babel to transpile our ES6 to ES5. The conversion (which only touched the code we've written in the last year or so b/c the old code is just spaghetti so we only converted what was already JS "classes") took less than a week of 1 dev's time and has been flawless so far (been using it production for a little over a month now). We are using gulp to facilitate this process. Just wanted to add a data point to those considering using it.

I've found the JS to be much cleaner and easier to read and I've heard similar statements from the rest of the dev team. I highly recommend it. The best part is you can convert your code 1 file (or "class") at a time and your old JS will still continue to work perfectly.

Feel free to ask me any questions about the process and any potential issues and I can respond with how we dealt with it (if we did).

<comment><USER3> How did you do one file at a time? Enabling it with a different extension e.g. script.es6 or similar?

Did it live side by side with existing code?

Were your watchers efficient or did it slow down over time the more you added?

Looking to do this on a large JS code base soon. Thanks for your input, Josh

<comment><USER2> Running non-converted files through babel is not an issue, they will just come out looking the exact same. That said we have a high number of JS files (that I am slowly pruning down) so it was a worry (time-wise it takes ~30 sec to transpile all our JS) to run them all through babel. I considered trying to add a:

      "use babel";

at the top of files that should be transpiled but I couldn't find a good "gulp way" to check for that. I also considered using a "filename.es6.js" structure but following file renames in SVN is a bitch and a half (not my only reason) so I didn't want to have to change those all back when es6 was widespread enough to use directly. I settled on running them all though so our "gulp build" takes 30 seconds or so but for our "gulp watch" (which starts off by building everything) I implemented caching so that only changed files had to be re-transpiled before getting concated into our "app.js" file. With this change it takes <1sec to transpile changed JS. Since our JS dev's use "gulp watch" everything works out quite nice so that they can edit a file and reload their browser to see the change immediately (30 sec times would have killed babel in it's crib for us and I assume most other places).

For caching we used the gulp-cached [0] plugin. It's important to note that if you plan on concat-ing your files that you are caching then you will also need to use gulp-remember [1] which re-introduces cached files into your file stream after the intensive parts of are done so for example:

      var customJSStream = gulp.src('webroot/source/js/**/*.js'))
      	.pipe(cache('app-custom-js'))
      	.pipe(babel())
      	.pipe(remember('app-custom-js'))
      ...

(NOTE: I've left out stuff from our gulpfile and simplified it for display here)

We then combine that stream with 2 others and concat them all together but that's not important for this example. Let me know if you have any other questions!

[0] [https://www.npmjs.com/package/gulp- cached](https://www.npmjs.com/package/gulp-cached)

[1] [https://www.npmjs.com/package/gulp- remember](https://www.npmjs.com/package/gulp-remember)

<comment><USER4> I'm always amazed at what people go through to use gulp/grunt/etc instead of make.

<comment><USER2> _sigh_ I'm really not sure how to reply to you, I want to say don't feed the trolls but.... make does not suit our needs and so we don't use it. Period.

Also good luck getting source maps + babel + bower + etc to work under make, I honestly don't believe it's possible or if it is it would be so convoluted that it would be impossible to extend or understand.

Gulp may not be a perfect tool (I don't believe such a tool exists) but it's leaps and bounds above make and a better choice than grunt IMHO.

</comment></comment></comment></comment></comment><comment><USER5> Wow Spartan (IE replacement) is really kicking some major ass with its ECMAScript 6 support:

[https://kangax.github.io/compat-table/es6/](https://kangax.github.io/compat- table/es6/)

<comment><USER6> Not really. Compared to, say, Firefox' future versions, it's barely ahead, with Firefox putting some features counted multiple times behind a flag (let's remember that `let' has been available in Firefox for ages, regardless of user settings, assuming the script tag has suitable type attribute).

Further than that, it's silly to compare browsers that aren't stable yet.

<comment><USER7> Not to mention that IE is usually a bit ahead at launch, but with such a long tail (roughly two years) between releases gets pretty stale, and is in general the boat anchor that holds back being able to use said features without transpiling and/or shims for 4+ years.

<comment><USER8> I was under the impression we are going to see regular updates to Spartan similar to what we get from Chrome now.

</comment></comment></comment></comment><comment><USER9> Babel contributor here. It's great to hear how many people are already deploying projects using Babel, and even better to see how fast ES6 is getting adopted. Many people are still stuck in an ES3 world... ( _shakes head_ )

If anyone has any questions, feel free to ping me or stop by our support chat[1].

[1] [https://gitter.im/babel/babel](https://gitter.im/babel/babel)

<comment><USER10> Is there anything wrong with being "stuck in an ES3 world"?

<comment><USER9> I mean if you never want to improve the language I guess not. The point was that ES5 adoption was really weak at best, and ES6 has been very strong.

<comment><USER11> JS5 adoption was really strong. It was hyped with "JavaScript - the Good parts", and it was lean. Typescript inspired ES6 with syntactic sugar "class" keyword in a language that has prototype based inheritance ...hmm we will see. I dislike the recent forced influence of Microsoft to JavaScript.

<comment><USER12> JavaScript: The Good Parts predated ES5, and certainly predated actual adoption.

IE didn't support basically any of ES5 until 2011 with IE 9, and, for example, Safari didn't support bind() until 2012. Strict mode didn't appear until IE 10, Firefox 4, and Chrome 13. Those dates aren't so great even if you ignore that you'd often have to support the earlier versions for a significant amount of time anyway.

</comment></comment></comment></comment></comment><comment><USER13> This states TypeScript is layered on top of ES6. I did not believe that to be true. Is there some meta-bug or other location tracking the TypeScript move to ES6? I was under the impression there were still a number of bugs that prevent fully using TS with ES6. (Maybe I'm wrong and someone can fill me in :))

<comment><USER14> I'm using TypeScript everyday, so I might can give a good answer to your question. They have just finished support for ES6 modules, template string, let/const etc. ES6 classes have been supported for a long time and most of the syntax sugar of ES6 I can think of is supported. On top of that, they are about to release v1.5 the big thing there is support for decorators.

What is not so good with TypeScript is that they have a rule of not polyfilling things. So if you want any new ES6 methods for array/string etc. You would need to polyfill them yourself.

<comment><USER15> You can compile .ts to ES6 .js and then use babel to compile it to ES5 .js.

<comment><USER16> Doesn't it error-out with the unsupported syntax, for example, generators?

<comment><USER17> Yes, however some features in TypeScript are only supported when targeting ES6 (such as "const"), so there's still some value in double-transpiling from TypeScript -> ES6 -> ES5 (I'm doing that on a couple of projects).

That may have changed somewhat in the TS 1.5 alpha - they're supporting more features on the ES5 target than they did in 1.4.

But yes, you still can't use ES6 features that TS doesn't know about.

</comment></comment></comment></comment></comment><comment><USER18> If you're looking for a ready-made template project to try out ECMAScript 6/7/2015, I created, and have been successfully using my base-node [0] and base-angular [1] projects for a while now.

You can use babel or traceur (just comment/uncomment one line), although the default is traceur. Gulp is used for building, packaging, and lint/transpile on save. base-node contains everything to package your application into a minimal Docker container (<10MB compressed + your minified, transpiled code) with io.js. base-angular will give you a module system, LESS transpiling, live-reload (LESS/CSS changes are injected without reload), and a minified, cache-busted output that you can drop into any web server (minimal nginix containerization in the works).

I'm sure the documentation could be better, but the Readme.md has enough to get you started and the code should be very straight-forward.

Since I'm using git, I can set these master repos as the 'upstream' and whenever I make changes to the base project (like, adding containerization support) I can easily push that change out to all my projects and everything gets tracked nicely in the revision graph. The `create.sh` script in base-node will actually do that rewiring for you.

[0] [https://github.com/blakelapierre/base- node](https://github.com/blakelapierre/base-node)

[1] [https://github.com/blakelapierre/base- angular](https://github.com/blakelapierre/base-angular)

</comment><comment><USER19> Not a fan of transpiling; you go from having one language to having to understand two (ish; ES6 isn't entirely different) while only being able to debug in one. It's nice people can write in and actually use ES6 today but I really don't think transpiling is worth it. I've been bitten by weird bugs in transpilers before that wasted far too much of my time.

> npm may eventually support two versions of the same module, which would > enable you to deliver libraries as both ES5 and ES6 for Node.js, io.js and > client-side module systems that are based on npm.

No, please no. Why would I ever want to deliver a library in both modes just for some better / different syntax? Yes ES6 is nicer than ES5 but doubling the modules isn't worth it at all.

<comment><USER20> I may be wrong, but I feel the anti-transpile crowd hasn't done their research. Comments in this vein have been popping up for years and have been resolved for years. Check out the transpiled Babel code -- it's what you would expect, and sourcemaps take you the rest of the way.

<comment><USER19> I've used source maps before but they're not nearly as good or intuitive as simply debugging ES5. Too much of a pain in the ass in my opinion. I just don't see the value of adding an additional layer of possible bugs / issues just so I can use a little bit better syntax; I'd rather wait until more things get updated with the native implementation.

I still think a better solution is some sort of byte code so people can use whatever language they want instead of trying to force JavaScript to be the "byte code of the web".

<comment><USER7> It's already been tried several times... and unless the big 3 browser makers (google, mozilla and ms) all support a given approach it just won't happen. JS is for better or worse what we get in the browser, and the subset optimizations for asmjs is probably as good as what we will get broader support for.

Transpilation is really the only option... And even using a common bytecode, there will still be a compile step which is separate from the source, and requires mapping to be able to associate the bytecode to the source with the same potential for bugs you are complaining about.

<comment><USER19> > It's already been tried several times... and unless the big 3 browser makers > (google, mozilla and ms) all support a given approach it just won't happen.

Has it really been tried though? I've never seen more than a few half assed attempts without all parties involved. I've love to see them come together to work something like this out; the web as it exists today isn't nearly as snappy or nice as native apps so I feel like something will eventually change here. Whether that's the help of a bytecode implementation or something else I'm not sure.

> And even using a common bytecode, there will still be a compile step which > is separate from the source, and requires mapping to be able to associate > the bytecode to the source with the same potential for bugs you are > complaining about.

Yes there is always potential for compilation to introduce bugs but compiling into a bytecode is vasty different from compiling into JavaScript. JavaScript is a functional language with a lot of quirks; Java bytecode and .Net's MSIL, for comparison, are very raw and fast with as few features as possible. It's significantly better to compile into some form of bytecode for later execution than compiling into another human-readable language.

</comment></comment></comment></comment></comment><comment><USER21> Switching from writing classes in other languages to using the JS prototype made writing object oriented code more fun and much faster! The prototype way was a step forward. I think it's very weird that JavaScript is now planning to introduce classes, and that you guys are so exited about it.

<comment><USER22> The new `class` syntax is just sugar on the ordinary prototypal way in which JS developers have created "classes" for ages (functions which return themselves as objects.) It just makes the code you have to write a little prettier to look at.

</comment></comment><comment><USER23> If I just want to clean up some basic code in a .html file with a terminal script - preferably with a "\--watch" flag, what's the simplest way of accomplishing that?

Also seems like the best way to ease into it.

<comment><USER24> My favourite way is via JSPM and live-server. I got into that after watching this amazing dev video: [https://www.youtube.com/watch?v=iukBMY4apvI](https://www.youtube.com/watch?v=iukBMY4apvI)

<comment><USER25> Webpack is a very good solution too. I really like jspm, and especially the dedication of its main developer, but I found much easier to integrate webpack in my build process than jspm. More features, a bit less bugs.

</comment></comment></comment><comment><USER26> It's a pleasure for me to use webpack with Babel for my projects.

</comment><comment><USER27> to transpile is not an happy idea</comment>


TinkerCAD reopens, purchased by AutoDesk - <AUTHOR> http://blog.tinkercad.com/2013/05/18/autodesk_tinkercad/ <comment><USER2> I'm Kai, one of the founders of Tinkercad. If anyone has questions I'm happy to answer here.

<comment><USER3> What's happening to Airstone? Is the team now back working on Tinkercad instead?

<comment><USER2> This was a pure asset deal, no personnel moved to Autodesk. The team will continue working full time on Airstone.

<comment><USER4> Could you elaborate on the HPC elements of what you're doing? I'm somewhat unfamiliar with the simulation/modeling tooling ecosystem (beyond a fuzzy "mix of computational geometry, physics and numerical computing probably happens here" view point).

What are some good starting points for understanding the landscape of such tools?

Context: I'm a few months away from rolling out (from my company, Wellposed) some pretty interesting HPC grade numerical computing products/tools and it seems like the simulation/modelling ecosystem might be one segment that might find my Wellposed tools intriguing.

<comment><USER2> I think we might have something out of interest for Wellposed. Fire me an email at firstname.lastname@example.com and lets talk more.

</comment></comment></comment></comment></comment><comment><USER5> This is wonderful news! TinkerCAD fits perfectly as a bridge into AutoDesk's 123D family of modeling applications. I wouldn't be surprised to see TinkerCAD integrated with 123D Design in the future. It's also a great way to introduce younger audiences to the AutoDesk brand as a long-term bridge to 3D Max and Maya.

</comment><comment><USER6> Wow....the sheer joy from the commenters is quite telling.

I don't think I have ever seen that sort of reaction for an acquisition of any web property before - except maybe YouTube going to Google and not MSFT.

</comment><comment><USER7> What an unexpected outcome! Wow!</comment>


Firefox Private Network is in Beta - <AUTHOR> https://private-network.firefox.com/ <comment><USER2> Here is standalone client for Firefox Private Network: [https://github.com/Snawoot/firefox-secure- proxy](https://github.com/Snawoot/firefox-secure-proxy)

Also works outside US.</comment>


The Human Cost of Tech Debt - <AUTHOR> http://www.daedtech.com/human-cost-tech-debt/ <comment><USER2> I recently became CTO of a company with two moderately successful SaaS products. The second one is a fork of the first, but has been maintained by more competent people than the original product. Both are still monuments to technical debt.

About 4 million lines of PHP code, written by underpaid, sometimes not well meaning, freelancers and students over the span of 8 years. The CEO wrote a large part, but stopped learning new techniques around 2004.

I'm bringing competent and well paid people in through my network and try my very best to give them as much freedom as possible, I allow and encourage greenfield modules/services that run on separate and new infrastructure for anything that is possible to be rewritten in the timeframes, but the larger part of the job is still mind numbing to my team and that makes me question my wisdom.

If anyone has tips on how to steer such a ship in a direction where the work is less frustrating for my devs I'm very open for advice.

<comment><USER3> I've run across this situation many times (I'm a "senior" team lead, meaning, I've been working for 25+ years). I've witnessed companies that overcame tech debt, and seen companies fail because of it. There's basically three approaches that people can take.

One is the "big re-write". They start on a new code base, and try to develop it in parallel. It takes a very long time, and the teams have to work on two solutions for some time. It's a big bang approach, and it often fails, or drags on for years.

The second is massive refactoring. It requires intensive testing and best practices. This strategy requires that the teams focus on testing intensely. However, often the testing culture is not there, which is why the code became unmanageable in the first place. It's kind of like starting over. And the new focus and discipline on testing is hard for teams to do without strong leadership, training, or new talent.

The last, and most effective in my opinion, is to go with a service-based, incremental approach. If the code base is not already using services, APIs must be built. Frontend/apps must be de-coupled from the legacy components. A clear domain model has to be agreed upon, and then parts of the legacy codebase are put behind APIs, and de-coupled from the rest. Over time, sections are refactored independently, and the APIs can hide legacy away. Maybe the legacy parts are refactored and replaced, or they stay around for a while. But the key is, that this approach allows multiple people or teams to work in parallel and focus on their areas. This is domain-driven design in action, and it works. New features can actually be developed sooner, even though the legacy is not replaced yet.

In the end, overcoming tech debt is about people. And on larger code bases, it's more of an organizational problem than a code problem. Developers need to be able to move forward without having to navigate too much code or too many different people.

<comment><USER4> Do you subscribe to the microservice philosophy? I ask because you come from the era of shared objects, and I personally still consider libraries with well defined APIs to be much simpler than dealing with multiple processes possibly running across different hardware.

I do break my infrastructure apart, but far less aggressively than some advocates.

I'm curious what 25 years had lead you to believe.

<comment><USER5> I'm not the op and I don't have 25 years experience but I'm a manager/architect with 18 year's behind me. My take:

I very much believe in microservices. I've repeatedly seen small library based approaches fail because one of the key and near universal truths of tech debt IME is code being tightly coupled. When you force an API interaction to happen via an outside protocol, you force a clean contract and a culture of coding to a contract. Decoupling the code allows yet team to move faster and more independently.

</comment></comment></comment></comment><comment><USER6> "They know that they're going to have to manufacture endless explanations for why seemingly simple things take them a long time." This is what kills me. You can't say to the boss, "Your beloved senior dev built this arcane and fragile system, so everything I do takes forever." Instead you have to find diplomatic/meaningless explanations for why you're moving so slowly.

<comment><USER7> From the bosses' point of view it is very difficult to tell the difference between "we have a crap codebase" and "the new guy sucks at reading code and is going to want to rewrite everything he touches".

<comment><USER8> If 5 different people are all saying the same thing, that makes it a lot easier.

<comment><USER9> It's not rare that a company hires 5 bad developers (or more) for each 1 good one.

</comment></comment></comment></comment><comment><USER10> I wonder if it is also the case that if the code quality is high, people are happy and they stay. This means fewer openings are available in those teams. Therefore, if you take a new job, it is more probable that the code is bad and there is a revolving door of developers who have tried to make it better, failed, left and created an opening for you. The vicious cycle of bad jobs.

<comment><USER8> Honestly, I would get bored working at a place where everything is "easy". I actually enjoy the challenge of working on awful code-bases, and trying to improve upon it, while still preserving the functionality and not breaking anything.

That said, if I had to work with awful code _and_ I wasn't allowed to touch it or make it better, I would definitely leave in a heartbeat.

<comment><USER11> I used to enjoy the thrill of the refactor, especially on applications where there weren't a lot of users or the product didn't generate much revenue. Or better yet, refactoring before the product has even been released. When the stakes are low, you can be much more fearless, and it is fun to rearchitect and introduce new patterns, etc.

But when the stakes are higher, the risk/reward ratio just isn't there. You can refactor the code to make it more sensible or pleasing to your eye. Two problems with this: 1) Someone else may disagree and refactor it again. 2) When it blows up or has unintended consequences, and you are now on the hot seat, you find yourself working on Saturday asking yourself if it was worth it at all. Thus the fear sets in.

</comment></comment></comment><comment><USER12> Bad management, oversight, and architecting returns bad code. Often team leads know better, but are driven by deadlines and the lack of a skilled workforce (with little time to train them) to produce a product. Of course this leads to turn-over...eventually. People will work for a year, realize that nothing they want to accomplish will happen quickly because of resourcing, then they'll run away. I really don't agree that the human cost of technical debt is a new concept. It's foremost in my mind when I think about giving tools to employees to add new features. Well written article, but not really anything new.

</comment><comment><USER13> Stuff like this is why I wish you could review the companies code before joining. Getting hired, discovering how bad everything is and then starting the job hunt again is awful for everyone.

<comment><USER14> For most job offers I've gotten I've asked to review the source code post- offer. I've even done this in a couple interviews -- can't recall ever being denied.

In one particular instance I came back to review the code for a product that required a level 3 security clearance; the hiring manager happily complied, letting me spend over 2 hours alone studying the code on a workstation in a side office. One reason I passed on that position was due to the code being more spaghetti than I would have preferred.

<comment><USER15> That's an excellent idea. How to ask about it cordially?

<comment><USER14> Inside of an interview, what usually happens is we get into a discussion of the architecture of the product I'd be working on. Then I pepper questions about the codebase itself, and after a few questions I'll say "Wow, that sounds interesting. Mind if I take a look?"

Post interview I'll talk to the hiring manager and express how much I enjoyed meeting with the team and love the product, then ask if it would be alright to stop by to take a closer look at the codebase, watch how the team works in action, just be a fly on the wall.

</comment></comment></comment></comment><comment><USER16> Copying my comment from the OP

> Analysts and project managers might account for technical debt when > discussing slipped deadlines.

No they won't, you cannot quantify technical debt, it's a _metaphor_.

I completely agree with the rest of the post. But "technical debt" is not a cause, nor a valid metaphor to share with management. Other bad metaphors are "building" and "architecture". We are not building anything, we are only writing algorithms that computers will follow. When we write down those algorithms, we are also encoding the inner workings of the company. The true knowledge of how a company works is not only in its documentation or in the heads of their employees, it's also in its automated processes.

The automated processes are the reason why a company can be competitive nowadays. People need to be able to understand those processes, those algorithms, _from the code_ , documentation is never enough, and is never up to date. The code is the primary source of information. If it's not legible, if it becomes arcane knowledge, a black magic that only a consultant can "fix", then the company grinds to a halt, it withers and is crushed by competitors. Be careless about the code quality is not about the suffering of the developers, it means that you are burying vital knowledge of your company, and eventually no employee, no consultant, will be able to dig it out.

If we are trying to figure out how to deal with managers who don't care about the company, but only about absurd deadlines that their managers gave them, then we need to start pushing back, start saying NO. Politics are unavoidable, and often necessary, not all decisions can be made because technical reasons, but we need to start imposing the reality of the code, you cannot rush it, our you are killing yourself.

</comment><comment><USER17> > For a manager, a code base high in technical debt means that feature > delivery slows to a crawl, which creates a lot of frustration and awkward > moments in conversation about business capability. For a developer, this > frustration is even more acute. Nobody likes working with a significant > handicap and being unproductive day after day, and that is exactly what this > sort of codebase means for developers.

The very same holds for managers and teams in companies that need to deal with organizational debt:

[https://steveblank.com/2015/05/19/organizational-debt-is- lik...](https://steveblank.com/2015/05/19/organizational-debt-is-like- technical-debt-but-worse/)

</comment><comment><USER18> The concept of technical debt is easy to convey to management, the hard part is conveying the scale of the debt because it's an abstract value. You don't have x units of debt. And because there's no actual value to measure it's easy to keep adding it on, it's easily viewed as the same debt.

To give you an analogy imagine you have a credit card. As you spend you increase your debt but you never get to know want your balance is. All you know is how much you pay per month. In fact even that number would be blurry and fluctuate. That's what management sees from their perspective. As a result it's easy to keep adding debt because it had no real value, you don't see that you're $1000, $10k, $100k, etc in debt, just that you have to pay something each month. Yes each month you have less to spend but it's a lot less than you spent that month. Last month you had to pay $100 but this month it's $101 and you don't have to pay that $50 one Ike hit. The extra $1mth is easier. You kinda forget that in a year it's now an extra $2-$3/mth. And again you never see your balance, so you don't know how much you have in debt. Your spouse keeps saying your way in debt but you have no idea of the scale. It's very easy to increase your debt this way.

Unfortunately I don't have an answer as to how you can value the technical debt so that the business people can appreciate the scale. I don't think as developers we can even measure it ourselves accurately, it's more of a feeling, a scale if you will.

</comment><comment><USER19> Most articles about technical debt should actually be about _technical default_. Tech debt is a good technical management instrument, it's just need to be paid and everyone involved must understand that. Delaying payments is as bad as delaying regular finance debt payments.

<comment><USER20> Completely agree with this sentiment. Not all tech debt is bad - and it's often inevitable. I wrote some stuff about this notion: [https://medium.com/@MostlyHarmlessD/on-technical- debt-7bac65...](https://medium.com/@MostlyHarmlessD/on-technical- debt-7bac65edf349)

</comment></comment><comment><USER21> This article hits close to home!

What should be done in a company that's basically committed every error you could commit? What's an engineer to do? Just switch jobs?

I joined a company a few years ago that had hit a wall with its tech stack after 8 years. They brought in a new technical lead who convinced everyone to approve a "rewrite everything from scratch on new stack" approach (even though it rarely works).

2 Years and $40-50m later:

\- The thing's nowhere near done. Customers all still using old legacy product that's not been updated in 2-3 years now since resources were diverted to rewrite

\- 88% of the eng org has turned over

\- current eng org doesn't know how 20-40% of the codebase works

\- customers have caught on and started leaving en masse

\- Rest of company has turned over as well as they realized nothing useful was going to come out of engineering for years more if ever

It's a bit confusing. What the heck should we do at this point?

I take some solace in knowing that I've seen firsthand what NOT to do in situations like this.

<comment><USER13> > What should be done in a company that's basically committed every error you > could commit? What's an engineer to do? Just switch jobs?

Sometimes that's all you have the power to do. Why are customers leaving en masse if the old product is still functioning?

It sounds like the rewrite got stuck in the pattern where it has to do 100% of the original and then some. Just like a new product, a rewrite has to hit that MVP mark, just not as urgently. They also have to be able to throw out the cruft and not import every bad idea from v1.

<comment><USER22> Maybe competitors are providing a better product, so customers leave? It's hard to make a new version of a product that does less. Features were presumably mostly added for a reason, and customers probably use them. Removing them is a hard sell.

<comment><USER13> If there are better alternatives then sticking with the current technical debt may be a non-option as well.

>Features were presumably mostly added for a reason, and customers probably use them. Removing them is a hard sell.

Trying to make everyone happy is one of the big drivers of technical debt, a lot of stuff that gets added just to get that one critical contract. Sometimes you can refactor to better handle these variations that weren't anticipated, but sometimes you can't and you have to evaluate the complications versus that one customers business.

You say this most often in businesses that practice sales team driven development.

</comment></comment></comment></comment><comment><USER23> This is a great article. I've also seen tech debt exacerbated by poorly implemented agile practices. One of the worst projects I worked on kept kicking the can down the road because of a misguided idea that everything should be a customer deliverable. So not only were we forced to work with crap, but we were disempowered from making it any better! It was a nightmare, motivation and team cohesion was non-existent and yes, turnover was very high!

<comment><USER24> > everything should be a customer deliverable

And everything should be delivered in a 100% predictable timeframe and that predictable timeframe should be about a day.

</comment></comment><comment><USER25> > To tie this back to the tech debt metaphor, think of someone with mountains > of debt trying to explain being harassed by creditors. It's embarrassing, > which is, in turn, demoralizing.

Seems that a more apt use of the analogy would be "think of someone with mountains of debt trying to explain why they can't afford to do the same things as their colleagues"

</comment><comment><USER19> The human cost _of unpaid_ tech debt.

</comment><comment><USER26> "If you're not already familiar with the concept of technical debt" ohh how I envy thee.

</comment><comment><USER24> As much as I love the metaphor, it's been quite a while since I first noticed the "eye-twitching" among non-technical management types whenever the term "technical debt" comes up. I can almost hear them thinking, "oh, boy, more spoiled programmer bullshit for me to deal with".</comment>


Is uProxy dead? - <AUTHOR>

They appeared to have missed their launch deadlines, the first Google Search return for uProxy redirects to a Google page not about uProxy and I see little sign of it on social media. <comment><USER2> There's significant incentive for us to wait until we feel all the security pieces we have been working on are relatively stable before releasing.

Even if we don't advertise, we really want to have multiple security audits, and to know that it's pretty hard to mess up with the software before it gets in the hands of anyone who could get in trouble for using it.

It's a hard balance to strike, but we're getting pretty close :)

The github link has most of the source public, it's really just the front end UX that's kept private until we're ready to open up a public beta.

</comment><comment><USER3> [https://github.com/uProxy](https://github.com/uProxy) has many changes made in the last 24 hours, so I don't expect it's dead.

<comment><AUTHOR> Why didn't I check there first. Sorry for the bother -- my first Ask HN attempt, and asked-and-answered it was. Thanks fellas, glad to know it's still in motion. Fan of this project.</comment></comment>


Analysis of PS4's security and the state of hacking - <AUTHOR> http://cturt.github.io/ps4.html <comment><USER2> FreeBSD was vulnerable to BadIRET. Oddly, they never seem to have published an advisory, but the fix was here:

[https://reviews.freebsd.org/rS275833](https://reviews.freebsd.org/rS275833)

I thought the handling of that issue was very strange. I notified CERT, who apparently coordinated with FreeBSD, but no one ever really responded. The closest thing to an advisory that I can find at all is my post:

[http://www.openwall.com/lists/oss- security/2015/07/09/1](http://www.openwall.com/lists/oss- security/2015/07/09/1)

which contains a PoC that crashes the system. It's almost certainly possible to turn it into privilege escalation, though.

Go figure. I suspect that the security community just doesn't pay as much attention to FreeBSD as they do to Linux.

<comment><USER3> Probably has to do a lot with timing. At the time of your report, the Security Officer was DES. Life happened, and he wasn't able to keep up or respond to events as quickly.

As of June, Xin Li (previously Deputy Security Officer) has taken over as security officer and things have been handled very promptly and succinctly.

[https://lists.freebsd.org/pipermail/freebsd- announce/2015-Ju...](https://lists.freebsd.org/pipermail/freebsd- announce/2015-June/001646.html)

edit: I'm passing word to FreeBSD security officers to see if they can review this

</comment></comment><comment><USER3> Capsicum is part of FreeBSD itself so it doesn't have to be listed separately in the list of Open Source Software.

I'm also surprised anyone thought it was using ASLR -- it's a huge effort to get that completed and working sanely, which is being handled by the HardenedBSD folks. Their work didn't even exist when the PS4 was released.

I think it's possible for Sony to backport it and use it, but seems unlikely they would do that at this stage.

I'd also like to point out that FreeBSD jails on PS4 means there are ~23 million units in the wild deploying that technology. Will take quite a while for Docker containers to catch up, haha :-)

<comment><USER4> The article says userspace ASLR is used, and indeed presented a speed bump along the way.

<comment><USER3> I think it's unlikely they would fork from the upstream FreeBSD kernel that much without contributing ASLR back. They would end up having to repeat it all again for Playstation 5.

Stranger things have happened, though.

</comment></comment></comment><comment><USER5> Lots of good detail here. Raise your hand if you knew the switch to AMD would greatly benefit hacking vs custom, PPC chips. (raises hand) Now you know why I deployed security-focused stuff on PPC at one point. The economics of hacking alone work in your favor. :)

Personally, I think they should've tried to license Cavium's Octeon III processors: RISC (MIPS) ISA; 48 cores at 2.5GHz; many pre-made accelerator engines; huge I/O bandwidth. A royalty deal that made Cavium money for profit & continued R&D, while letting Sony have them cheap in PS4, would've made for one, badass gaming rig. Cavium might have had upgrades ready for PS5, too, given all the improvements they made going from Octeon II to III.

Everyone went with AMD instead. So, we get the real benefits of reuse of x86 code, low-costs, and reduced production issues. However, we also get all the black-boxes of risk in x86, a monoculture where an attack on AMD CPU's/firmware might break all game systems, inherent inefficiencies of x86, and lack of hardware differentiation (mainly accelerators) that developers can benefit from. Only time will tell if it was a good trade.

<comment><USER6> > Now you know why I deployed security-focused stuff on PPC at one point. The > economics of hacking alone work in your favor.

Perhaps true for a small-time, individual hacker. Over a decade ago, I recall being quite impressed by a professional pentest team whose tools included a lovely exploit authoring DSL embedded in a popular scripting language. That DSL allowed them to write "abstract" descriptions of specific software exploits, ala vs openssh vX.YpZ, then "render" (~~ compile) the exploit code automatically against any of their supported target architectures and/or OSes. (i.e. all of them.) Even though perfectly capable of it, they got tired of manually porting everything around.

This has also been a strong reminder for me that exploits are usually against the software, irrespective of the hardware architecture. It's easy for folks to get mixed up about that.

For something like hacking a PS4, the "obscure platform" logic might apply( __), but never assume it applies to any attacker who can _afford_ (n.b.: not just _build_ ) a sophisticated attack platform.

( __) With the caveat that now it 's a numbers game, and you're up against bored teens/tweens with way too much time to throw at the problem.

<comment><USER7> So, I'm kind of curious: It seems that the whole ROP thing depends on X86's CISC architecture in order to allow turing complete programming. Am I wrong in this understanding?

<comment><USER5> Oh, the attackers in academia have been clever for a long time. Check out this old gem:

Automatic Patch-based Exploit Generation is Possible [http://bitblaze.cs.berkeley.edu/papers/apeg.pdf](http://bitblaze.cs.berkeley.edu/papers/apeg.pdf)

Gave me a sly grin when I saw it years ago. And to think I thought I was clever because I always tried to compromise networks via whatever they trusted for security. These jokers straight-up turned patches into weapons. I realized at that point, along with all the hacks in media, that computer security was fundamentally (censored).

Started focusing on clean-slate approaches where possible with obfuscation, diversity, and strong interface protection everywhere else.

</comment></comment></comment></comment><comment><USER8> FreeBSD's default x86-64 calling convention doesn't pass arguments in registers?

<comment><USER9> Presumably the _function_ calling convention does, but not syscalls.

<comment><USER10> Neither does; the author is linking to the i386 documents, where there is indeed a difference between syscall and C calling conventions on some operating systems. On x86_64, everyone follows the SysV ABI and uses registers for both purposes.

</comment></comment></comment><comment><USER11> The section on ROP is fascinating. Has anyone done a ROP compiler?

Some program that takes a binary + c code -> address list?

If you think of the gadgets as being like assembler instructions, it seems like it'd be possible, though tricky to do.

</comment><comment><USER12> When was this published?

<comment><USER13> Originally on 2015-06-29, updated frequently and as recently as three days ago.

Commits: [https://github.com/CTurt/cturt.github.io/commits/master/ps4....](https://github.com/CTurt/cturt.github.io/commits/master/ps4.html)

Blame: [https://github.com/CTurt/cturt.github.io/blame/master/ps4.ht...](https://github.com/CTurt/cturt.github.io/blame/master/ps4.html)</comment></comment>


DB Superstar Jim Starkley touts NuoDBs new patent - <AUTHOR> http://gigaom.com/cloud/database-superstar-jim-starkey-touts-nuodbs-new-patent/

<comment><USER2> This article is terrible. Not only does it fail to give any technical information about the actual innovation being claimed, but it is also factually incorrect as it lists the wrong date for the patent filing.

Here is the actual patent: <http://www.google.com/patents/US20110231447>

I'll write a longer reply when I have time to digest the legalese.

That said, Jim Starkley seems to be pretty patent-happy. Check out this one from 2008, "Method and apparatus for generating web pages from templates" [1].

[1] - <http://www.google.com/patents/US7039658>.

<comment><USER3> I waded through the patent (morning coffee reading).

After all of that, I am unable to discern the novelty, having designed and implemented quite a few distributed database systems. The patent describes boringly standard internals that can be found in many existing distributed databases. In fact, I stood up a system on an EC2 cluster a couple years ago that had remarkably similar internals to what is described in the patent. Maybe I am missing something but I am skeptical.

The design described in the patent still has the same problems and limitations that have plagued distributed OLTP systems for years. The patent appears to be a minor rearrangement of standard architectural elements common to a lot of distributed systems. What is described will work but I could buy equivalent functionality from other vendors.

I was hoping for an interesting computer science-y technology but that patent doesn't contain one. It appears to be a description of a particular implementation of a conventional distributed transaction processing system. What am I missing?

</comment></comment><comment><USER4> " NuoDB is often lumped in with NewSQL databases, which he finds to be an oversimplification. "SQL is just one personality for us. We can be NoSQL or SQL, the innovation we have is much deeper," Morris said. He prefers to compare NuoDB to BitTorrent in the way it divvies up tasks to any number of processors — avoiding bottlenecks — but somehow managing to keep all that data organized, accessible and safe."

I've been waiting for something like this. NoSQL products took a look at the limitations of SQL DBMS products and (incorrectly) assumed that SQL was the problem. SQL may be a _symptom_ of the problem, but the deeper problem is conflating the logical and physical data model in SQL-based DBMS and also in most NoSQL products. A "record" (however you define it) doesn't need to be mapped 1-to-1 with the physical bits on storage, but for some reason (probably because it's easier) virtually every product does so.

It doesn't matter what language you use to access your data, as long as the underlying DBMS operates on solid fundamental principles of data management (ACID, logical/physical orthogonality), you can have an innovative, pleasing- to-use system.

Very interested to see what these folks come up with.

<comment><USER3> Underneath the hood _all_ database systems contain the same basic elements: an implementation of relational algebra operators and a concurrency control mechanism. Even nominally "non-relational" databases, like graph databases, are built on top of relational operator implementations (you say "graph traversal" and I say "join recursion").

What has traditionally separated the NoSQL databases from the SQL databases is that they implement a smaller subset of relational algebra e.g. the famous lack of join operators. This gave them the latitude to throw out the SQL interface and build interfaces that were more domain specific and a better match for what some applications need to do.

Now we are starting to see new databases that are discarding the SQL orthodoxy but with a relational algebra implementation that is as expressive as anything found underneath a conventional SQL database. They could implement SQL if they wanted to but there is less of a compulsion to do so than there used to be. Hopefully this will lead to the best of both worlds.

<comment><USER4> "NoSQL" projects have thrown out the relational baby with the bath water and instead implemented their own data model (mostly made-up).

The Relational Model is (hugely simplified) predicate logic and set theory combined with logical/physical data independence and integrity.

Relational algebra (selection, projection, union, etc.) are important but just a part (very important, of course, as it gives us a sound theoretical foundation to implement from) of the query language. The "Management System" part of the "DataBase Management System" are the constraints, logical/physical representation, data model, etc. - which IMHO represent the bulk of what a DBMS ought to do.

What sounds interesting about Nuodb is that they've kept the sound relational fundamentals and created something that is what the relational model has espoused from the beginning. Or so it reads.

</comment></comment></comment><comment><USER5> "Now it also has a patent that gives credence to its claims that its elastic database is truly innovative."

Regardless of any innovation NuoDB may bring, can this phrase really be credibly stated, given everything we've seen in the patent arena?

Edit: and the article ends with this paragraph: "We all know technology patents are tricky business — see GigaOM's Jeff John Roberts' continuing coverage of the raging patent troll epidemic — but it does show that NuoDB may really be onto something here with its self-scaling elastic database."

However, no technical details are given as to what the innovations are, just that the underpinning technology is so innovative that there weren't even names for what they're doing.

</comment><comment><USER6> I read the quoted patent blurb and thought, "oh great, so now if I write 'A multi-user, elastic, on-demand, distributed relational database management system,' I have to deal with this dude suing me." I hope patents don't work like that...

<comment><USER7> Use this to be safe: "a multi-user, on-demand, sharded relational database"

</comment></comment><comment><USER8> This is some pretty tone-deaf PR considering all the patent lawsuits that are going on right now. If their database is as great as they say then this patent won't make or break them.</comment>