A web-based Excel/database hybrid - <AUTHOR> http://www.ragic.com <comment><USER2> This isn't what you think it is. It's definitely not an excel competitor. It feels like a cross between an Access and a Salesforce competitor.

It looks like you're close to making a good product. But you're at 90%.

Basically I tried creating a little app and it's very painful. I don't believe you've sat with a user and watched them try to create an app without you intervening and giving guidance.

So sit down with some real users and watch them try to use it. Your front page claims install>customize>import. It doesn't work like that at all. It was confusing. For example the upload excel img looks like it's a button. It's not. When I did manage to create an app I first tried editing the cells directly, but they're not cells like excel. Eventually I realized I had to add a textbox, which was also confusing as the width kept changing and I couldn't stop it. At about this point I gave up. Then got hit with multiple alerts when I tried to close the pages.

Then tried to save it.

The most confusing save mechanism I've ever come across.

Tried to just close the page again. Multiple warnings again.

Argh.

And calm. I'm in the process of stopping smoking so my temper is a little, hmm, Hulkish.

I decided to try again while writing this using the import excel function. It didn't work from the front page at all as you had to already have a project (why?). When I tried to do it via the upload the open file dialog never opened. I gave up again.

Also find someone who knows something about sales and marketing. There's not even a CTA button on the homepage to signup. The copy is terrible. You've made the mistake of going typical corporate million page website which doesn't actually tell me anything at all. The order of the home page is all wrong (you should not be leading with the app concept which doesn't actually seem to be in the product). You're making the mistake of selling features, not benefits.

I'd personally spend a little more time so the sales website didn't look like bootstrap (the irony being you don't even seem to be using bootstrap).

As I said, good, but no cigar _yet_.

<comment><AUTHOR> Thanks, very useful comment, and I apologize for the confusion that you experienced.

Right now most of the Ragic developers first go through 2 hours of training before they start building. So they kind of know what the basics process are.

We are in the transition process trying to move from a tool that we personally teach developers how to use, to a self help type of product that developers can start by themselves. Functions like drag and drop add fields are added just this month.

I will certainly find a group of users, sit by them one by one to make our learning process a lot smoother. Again, sorry for the confusion. I think I will certainly post again after we tune the process. The HN posting feedback is really helpful.

<comment><USER2> I'm curious how many of you there are? As you should try getting funding if there's only a couple of you and you're attempting to bootstrap, get a good UX guy and really try to hit the ball out the park.

There's definitely a market for something like this and personally I think Salesforce is a overcharging by a lot and the field is ripe.

Ignore the thing about the guy moaning about $19.95 p/m.

</comment></comment></comment><comment><USER3> Idea #22 is a web version (realization) of Lotus Improv. He's old enough that he should know better.

<http://en.wikipedia.org/wiki/Lotus_Improv>

I've been waiting ages for this. Desktop, web-based, tablets, however.

Old timers may recall that Lotus Symphony had both database and spreadsheet features, like a multi-app. It was cool. At the time, Lotus Improv seemed like a healthy fusion, next logical step, minimum viable product.

Improv was victim of the Innovators Dilemma plus installed base inertia. I remain impressed that even with their market dominance, Lotus was experimenting, releasing crazy new products, and willing to eat their own children. (Though I can't forgive Lotus Notes, sorry.)

PG's original blurb (<http://ycombinator.com/ideas.html>):

"22. A web-based Excel/database hybrid. People often use Excel as a lightweight database. I suspect there's an opportunity to create the program such users wish existed, and that there are new things you could do if it were web-based. Like make it easier to get data into it, through forms or scraping.

Don't make it feel like a database. That frightens people. The question to ask is: how much can I let people do without defining structure? You want the database equivalent of a language that makes its easy to keep data in linked lists. (Which means you probably want to write it in one.)"

<comment><USER4> I'm interested in what you say about Lotus Improv, but I'm having trouble understanding what it has to do with Ragic Builder.

      He's old enough that he should know better.

Who is old enough, the Ragic Builder creator or PG? That person should know better than to what?

<comment><USER3> Ragic claims to be a fulfillment of Idea #22. I haven't tried it, so can't comment. I'm just observing that Improv predates Idea #22 by about 1.5 decades.

</comment></comment></comment><comment><USER5> Just some hopefully constructive feedback here: for Excel power users, you're not just competing with Excel, you're competing with Excel+ODBC+SQL+Workbench which is basically an Excel/DB hybrid, just not web based.

I read your "how this is different than a spreadsheet" blog, and none of the examples struck me as compelling, even though I use my Excel/SQL toolchain every day. Given how strong the ecosystems are around those tools, I think it would be very challenging to convert power users without some really obvious hook.

That one hook COULD be a good mobile client. For example, I have a dashboard built in Excel with an ODBC connection to a few of my company's DBs. However, I obviously can't see this on my phone, since no mobile Excel client supports ODBC (not to mention the macros I have built in).

I could maybe see a "disruption" angle, where you provide 75% of the functionality of Excel+ODBC+SQL+Workbench in one easier to use package, but I wonder how large the market is for people who want to work with that level of data but don't want power tools. I guess that's one area where you'd have to trust your market research/gut.

<comment><USER6> He could readily target the market of people who use Excel for things they shouldn't be doing, because they don't know how to encorporate SQL+Workbench, and don't have the time to learn.

<comment><USER5> Yea, it seems like this is a pretty market position if you ask me -- if you need DB like tools, why not just use a DB? This product would have to be an order of magnitude easier to set up and use to get people to use that instead, IMO.

Could be wrong though -- would be great if something like this could improve the data/analytical literacy of the general population.

<comment><USER7> "if you need DB like tools, why not just use a DB"

A) people don't really know they need db-like tools. B) even if they did, setting up a database to have even a portion of the "up and running" aspect of Excel is extremely time consuming.

The closest I've seen is some companies that get advanced enough set up a database engine somewhere, then set up Excel to connect to it as a datasource (ODBC or whatever). This still requires them to understand how to set up a database, secure it, and model table structures around their data, which is asking quite a lot from someone who really just wants to create a few short forms to collect data.

</comment></comment></comment></comment><comment><USER8> There are two markets:

1\. The programmers, utilizing an Excel+SQL setup for users in an IT setting.

2\. The business users who use Excel for more than they should be, or use it in a place a DB would be far more efficient.

Like another poster said, the former probably aren't going to find anything above and beyond what they can't already accomplish with a DB and Excel.

The latter, on the other hand, are a _huge_ market and need help. This is the market who hammer spreadsheets, but are completely in the dark when it comes to databases, and to whom, I believe, PG was referring. As one of those users (okay, I know a bit more about DBs than the "average" business person), this looks complicated. I can't determine precisely what it does, and there seems to be a lot of programmer speak. Apps? APIs? "Development" required? It looks hard. In other words, I _want_ an Excel/DB setup, in theory, but don't really know if this is what I think it is.

That said, as a data dabbler, this is intriguing. I'll be following progress.

<comment><USER9> Why do you want "an Excel/DB setup"? In what ways is Excel not enough?

You mention that a DB would be far more efficient. What kind of efficiency do you have in mind?

<comment><USER8> Querying and manipulating millions of rows. Basic usage of Excel doesn't work well there, in my experience.

The technique I've used in the past is to use SQL to query and output the data I need, then manipulate it in Excel. I'll leave it open that I'm not good enough with Excel, but I find MySQL query language more powerful for filtering, grouping, etc. However, there are a few too many steps in there, and it's not something I'll be teaching my mother. Someone actually has to set up the database, and that's no cake walk for the lay person.

Now, I'm going to take a guess that you're implying that Excel already has these features, to which I'll answer: possibly, but nobody I know is using them. That's why I think it's a large market.

But hey, if you know a solution off the top of your head, let me know. I'm always looking!

<comment><USER9> No, I don't think Excel already has those features. It's an open question how to do it in a way that the average spreadsheet user will find intuitive. Sounds like you have some opinions about that, as do I. But it looks like these guys have a useful product and I don't want to hijack their thread. If you want to discuss it offline my email is in my profile.

</comment></comment></comment></comment><comment><USER10> Essentially, your pricing indicates that you offer only the crippled version of the application unless the individual or small group obtains the $199.99 per month account.

The pricing and usage limitations on forms for "personal" (one user / 10 forms / $19.95/month) and "lite" (20 users, 10 Forms, $99.99/month) indicate that you don't understand that full collaboration for both large and small entities is where you will find and grow your market of users, and that small groups are not going to be thrilled with crippled functionality.

I suggest charging only on a per-user basis and not cripple the application with limited forms. Make it easy for even small users to want to rely on the full application for all of their activity. They will tell everybody about your useful application, and won't advertise its silly crippled limitation. Let both small organizations and individuals grow into comprehensive and full usage without a big hurdle ($200 a month) for complete functionality.

<comment><USER11> What redtexture says: all my clients would need the $199/month package because they are collaborating with multiple people (otherwis they could be using Excel with a lot less trouble). But for $2400 a year it's not that much of a cost saving anymore. Please offer "more reasonable" pricing plans.

<comment><USER9> _they are collaborating with multiple people (otherwise they could be using Excel with a lot less trouble)._

What's insufficient about Google Docs for that? i.e. a spreadsheet that supports collaborative use.

</comment></comment></comment><comment><USER12> This is very pendantic I know but in your demo video the line 'They all have tons of features that Emily don't need, all she needed the most is a CRM with attendance records' would be more correct English if it read: 'They all have tons of features that Emily doesn't need, all she needs is a CRM with attendance records'.

In any case, the concept sounds good and is otherwise well presented.

<comment><AUTHOR> Hey thanks! I really appreciate this, my English is not all that good.

<comment><USER13> Front page grammatical issues: "Most of the apps in our library is free" should be "...are free"

Anyway I haven't tried your products, but I'm very positive about what you're trying to do. The HN community is so comfortable with programming that they'll connect less with the value of this concept than many, so take their comments with a grain of salt. For the countless people out there who feel constrained by their knowledge of spreadsheets --- useful but limited in scale --- this could be very useful, _if done right_

</comment></comment></comment><comment><USER14> I poked around for a bit.

I want to reiterate what has already been said; this does not feel like Excel. If you spend some time watching (medium-to-low skill) people who use Excel as a general purpose tool to perform general tracking and calculation tasks, you'll see there's a very different way they work.

They start with a blank sheet, and start putting data in rows. Then they add some formulas to the rows, and do a bunch of sums on some columns. The idiom you've chosen, of dragging and dropping, is a significant workflow change.

A few more things that I haven't seen pointed out already.

Your pricing scheme should differentiate between designer and non-designer users. If I buy a personal account, it's kinda useless when I can't give my url to someone and tell them to create a new entry and edit data.

Also regarding users, the lack of OpenID integration seems an obvious lack. Creating an account just to enter some data is a pretty high bar. Ideally, I should be able to share a form via email, implicitly granting data entry permissions to the recipient, then they can click the link in the email and be prompted to use their email provider's OpenID to authenticate.

Anyway, you've got some good ideas here, but I don't think you've got the right focus on Excel/Access users quite yet. It lacks the _feel_ of Excel, and lacks the features of Access.

<comment><AUTHOR> The idea is like this:

If the user already have data on an Excel, we provide an import wizard so that they can import their Excel and create a database with it.

If the user does not have existing data from Excel, they can first define the data fields on Ragic using the design mode. They can actually create cells as fields just like they would on a spreadsheet, but a lot of users find it hard to catch on so we added the drag and drop model just recently. So both ways should work okay.

And actually we do support OpenID log in, and the ability to click on an URL in e-mail to grant access to some DB entries. I guess we have to work on making these features more obvious to users.

Thanks for the feedback!

</comment></comment><comment><USER15> Nice. I accept everything that decreases the chances I'll have to work with <http://quickbase.intuit.com/> again in the future.

<comment><AUTHOR> Kind of curious what kind of trouble that you ran into when using Quickbase. You can e-mail me if you don't feel like posting it here. (firstname.lastname@example.com)

<comment><USER15> I'm not good at describing the UX parts, but every time I deal with it seems painful. I can't easily get to what I need, update notifications don't work the way I expect (you need to manually mark stuff as viewed), creating custom tables can be blocked by project owners (I understand not editing the shared ones, but if I want to create my own report, I don't see a reason to disable that). I can't point out many big issues, it's just papercuts whichever direction I turn to.

Things that I can point out as missing:

\- no support for other locales (it's dd/mm/yyyy and 2300, not 11pm where I live, US is not the only place in the world)

\- no rich-text on long-content fields

\- no actual changes visible in revisions, history contains only "changed by xxx on yyy"

From the developer's perspective, the API is really silly. It seems to be completely different from normal access path. Tokens need to be authorized by project owners, not all authentication schemes work with APIs. I don't get at all why is API access treated as something additional, rather than a normal usage pattern. If I've got a user account, I should be able to do via the API every action I'm allowed to do via web - without anyone's blessing.

API design also leaves a lot room for improvements. Custom XML only with things like this definition of a query:

        <query>{'5'.CT.'Ragnar Lodbrok'}</query>
        <options>num-4.sortorder-A.skp-10.onlynew</options>

Keep in mind that '-s only separate the arguments to the operators, not the actual items, so for multiple values you have to quote with " inside of a '. Now try to protect that against injections. As a developer I want to cry at that point. Also the HAS operator is special cased to work only on lists of users via their screen name, id, or email... Did I mention that separators for lists of elements are periods (unless explicitly defined as semi-colons in some places).

Trying to figure out the exact meaning of "field_type", "base_type", "role" and "mode" in the results (all are attributes on every value of the results) along with how to resolve them when they reference values in other tables (which can be done in 3 different slightly underdocumented ways from what I've seen so far) is pretty difficult. This may be a good start for a TDWTF article...

<comment><USER16> I like QuickBase for prototyping and departmental applications but it does have it's idiosyncrasies. Unfortunately, I don't get to use it much anymore because I am bootstrapping a startup and it is too expensive (not free enough) for us. If we start scaling I will use QuickBase to automate as much as possible.

I have a working DBAPI compliant Python module that is about 40% complete. It really needs a method to translate SQL to QuickBase's clunky xml query format. The Ruby API has one and it would be trivial to port it to Python.

Would you (or anyone else?) be interested in contributing if I put it up on GitHub?

<comment><USER15> Not sure how much I can contribute, but I'd definitely appreciate it if you ever do publish it.

<comment><USER16> Cool. I'll email you when it's ready public consumption.

</comment></comment></comment></comment></comment></comment><comment><USER17> Small suggestion: Add a stronger CTA (I was thinking after the 3 steps and before the 'next steps'). At that point I was ready to sign-up to try it out but wasn't sure what to do next.

FWIW I think this is an interesting approach. We're trying to approach one specific use case in our own product (I'm in agreement that smaller companies need an alternate solution to larger CRM tools). Good luck!

</comment><comment><USER18> Nice idea. I haven't tried it yet. When I see the word "Excel/database", I image it's a system that can import existed excel files and generates an runnable CRUD app instantly. Does Ragic archive this?

<comment><AUTHOR> Yes, using the import Excel wizard we can do that.

</comment></comment><comment><USER19> Just a tip: when I clicked on the video, I assumed it had loaded a new page and then clicked back in my browser (ending up back on HN). Maybe make the overlay fade in or be partially transparent?

<comment><AUTHOR> Great tip! Thanks! We'll do that.

</comment></comment><comment><USER20> Great idea.

The logo is not showing up for me, or for that matter, neither are any of the images on your Features or Pricing pages.

On your home page, what is your primary call to action? Do you want people to watch the video? Could you have people try something out? How about making the applications clickable to show more details or have people play with the applications? Have your video proofread for grammar. Good luck!

<comment><AUTHOR> We used Google Application Engine as our CDN...turning on billing now!

</comment></comment><comment><USER21> "Most of the apps in our library is free" should be "Most of the apps in our library are free".

</comment><comment><USER22> Anyone else stop reading after seeing the lowest tier was $19.95/month?

Maybe I'm just a cheapskate...

<comment><USER23> I didn't stop reading, but it feels like too high (for lowest tier).

I think there should at least be a FREE tier which allows 1 or 2 forms. Once someone sees the value, they'll jump for more.

I understand that textual data is light, but 250MB seems quite low given current costs of hard drive storage.

Why the user limit too? Your costs seem to be in data transfer/storage.

<comment><AUTHOR> The user limit is to segment large user sites from small user sites. We assume larger users sites have more budget.

But we seem to need a cheaper plan designed for developers and geeks so that they can play around Ragic with minimal investment.

<comment><USER24> It seems to me you might want a free plan suited to app development but not production use (e.g. something that doesn't keep data long-term) to encourage more people to develop for your platform.

</comment></comment></comment></comment><comment><AUTHOR> Disclosure: I'm one of the founders of Ragic.

<comment><USER25> How is the name pronounced? "Rage-ick" or "Rag-ick"?

<comment><AUTHOR> Rag-ick It's suppose to be Rapid + Magic

<comment><USER26> If it's not too late, I think you should consider a name revamp. I get that you want to convey the speed and power/ease/fluidity of your application, but the truncation + concatenation of those two words ends up not bringing either of those words to mind. I don't blame you if English is not your first language; but this is a case where the combo does not work because you've combined two unrecognizable parts of two words to create a third unrecognizable word.

As someone said earlier, people are much more likely to think of "tragic." Actually, I think they'll think of "rage", which is an emotion that you most definitely do not want associated with your software.

Given that you aren't even getting the connotation of "rapid"/"magic", you're better off using an arbitrary word.

For Arrested Development fans, think of the combo-profession of "Analyst" + "Therapist".

</comment></comment></comment></comment><comment><AUTHOR> Some of the content didn't show for a while because we used GAE as our CDN and we exceeded billing limit.

Things should work normally now.

</comment><comment><USER27> Nice set of ideas at <http://ycombinator.com/ideas.html>

</comment><comment><USER28> Please don't end up like DabbleDB.

<comment><USER29> This is the biggest concern - invest a ton of effort into something and then the company gets bought out and the product dies. The least Dabble could have done was release DabbleDB as open source.

<comment><USER28>
      It would be nice for them to have written in the contract to open source their product if they go out of business and are not purchased are liquidated by another company.
      I too spent a lot of time utilizing DabbleDB unique tool-set and have yet to get back to the level of productivity after they went under.

</comment></comment></comment><comment><USER30> You forgot to add the rounded corners to the about page content area.</comment>


Pope Francis: Wasting food is like stealing from the poor - <AUTHOR> http://www.washingtonpost.com/national/on-faith/pope-francis-wasting-food-is-like-stealing-from-the-poor/2013/06/05/d4dc167c-ce13-11e2-8573-3baeea6a2647_story.html

<comment><USER2> Maybe he should sell off some of the massive holdings that the Vatican has and invest in food for the poor?</comment>


Ask HN: Strong typed language for data analysis? - <AUTHOR> Seems that the languages used for data analysis are all interpreted: R Python Matlab / Octave<p>Are there strong type languages used for data analysis? Bonus points if it uses vectorization ala matlab. <comment><USER2> I don't personally use it for that purpose, but Scala has some traction in data science. E.g. from a quick Google search:

[1] [https://darrenjw.wordpress.com/2013/12/23/scala-as-a- platfor...](https://darrenjw.wordpress.com/2013/12/23/scala-as-a-platform-for- statistical-computing-and-data-science/)

[2] [https://www.quora.com/Is-learning-Scala-useful-to-a-data- sci...](https://www.quora.com/Is-learning-Scala-useful-to-a-data-scientist)

[3] [https://www.safaribooksonline.com/library/view/advanced- anal...](https://www.safaribooksonline.com/library/view/advanced-analytics- with/9781491912751/ch02.html)

</comment><comment><USER3> Java has a nontrivial number of modeling/statistics/analytic libraries and frameworks. This would suggests some adoption at least in some circles. Though I'd suspect workflow is a little different for interpreters and Java in this space, i.e. interactive vs non-interactive.

</comment><comment><USER4> F# maybe?

[http://fslab.org/](http://fslab.org/)</comment>


SQL Server on Linux - <AUTHOR> https://www.microsoft.com/en-us/server-cloud/sql-server-on-linux.aspx <comment><USER2> Why would you want SQL Server on Linux in the first place? Are there enough use cases out there for mixed environments where such an admin would want to move this over to Linux vs keeping things homogeneous? Would a Linux Administrator really want to manage this versus MySQL or PostgreSQL?

<comment><USER3> I prefer SQL Server for its ease of administration, I hope that transfers well to Linux. The management tools Microsoft has made (still Windows-only) are unparalleled in my opinion.

I use Postgres and like Postgres too.

A use case that immediately pops into my mind (Although not necessarily beneficial for Microsoft) is companies wanting to save costs on Windows licenses.

Spinning up a new SQL Server instance becomes a lot easier if you can do it in a dockerified way, without license hassles (provided they still support the free SQL Server Express on Linux).</comment></comment>


Top 7 Reasons Why You Should Work for a Startup - <AUTHOR> http://sebastien-arbogast.com/2015/07/02/top-7-reasons-why-you-should-work-for-a-startup/

<comment><USER2> Just right below is another post that says what to avoid in a startup. A stroke of luck that both posts get posted in same time.

</comment><comment><AUTHOR> I know this may seem obvious to most Silicon Valley people, but here in Belgium, it's really hard to find software engineers ready to get away from the usual jobs in big companies and into startups. Here are some of the reasons that make it worthwhile to me, in the hopes that some Belgian people are reading HN.</comment>


Ask HN: What are good examples of "pro" UI? - <AUTHOR> The recent discussion on best books on UI/UX[1] got me thinking: What are some good examples of user interface design for professional applications: CAD tools, audio/video editors etc. Tools which help people do a job.<p>Especially interested in tooling optimized for productivity and "power users" (however those are).<p>[1]: https://news.ycombinator.com/item?id=18662992 <comment><USER2> Well as a CAD/IDE/VIDEO EDITOR _power user_ I liked the following UI concepts:

I LIKE:

\- Keyboard shortcuts!

\- Hide/Add menu items

\- Save a copy of your settings configuration

\- Scripting system in a well documented language

\- Clean icon-based tabbed menu at top, like MS Word.

\- Large work-space, for editing/viewing

\- File browser tree on left

\- Definitely a plugin system/store

I HATE:

\- I hate to see thousands of menu items...

\- I hate to see a lot of space dedicated to small menus that don't get used much/ever. If I don't need it, then hide it.

\- When there's no way to create a shortcut

</comment><comment><USER3> Blender and any other programs that truly embody customizable. A power user adapts a UI to their own needs instead of adapting to the program, in my opinion.

<comment><USER4> Blender and Foobar2000 are my favorite UIs for any end-user program. The potential for customization is endless.

Does anyone else know of programs of this caliber, with a strong emphasis on extendable UI/UX?

</comment></comment><comment><USER5> Sketch is the ultimate optimized UI for power users - designers. Sketch can do what both Photoshop and Illustrator can do without looking bloated and 'laggy'. It's minimalist and spartan UI hides a depth of power and functionality it's like having a la Ferrari engine underneath a Minicooper

</comment><comment><USER6> Ableton Live, Cubase.

<comment><USER7> Seconding Ableton. Live combined with the Push 2 is the best computer interface I've seen for anything. I've often found myself starting a project at 9pm at night and just getting lost tinkering with the sounds. The next thing you know it's 2am and I'll have a headache at work tomorrow.

You can do the same thing with a keyboard and mouse, but the Push makes playful experimentation so much easier, tactile, and fun.

</comment></comment><comment><USER8> matlab, bloomberg terminal, airbus cockpit, the vim keybindings

</comment><comment><USER9> Maya, Avid Media Composer, Pro Tools

<comment><USER10> Oof yeah, Maya. That's a learning curve.

</comment></comment><comment><USER11> Norton Commander and derivatives.

</comment><comment><USER12> The bash shell? Vim?

Photoshop/Illustrator?

</comment><comment><USER13> blender / adobe photoshop / 3ds max

their UIs are nice designed and practical.

</comment><comment><USER14> Stripe is an often cited example of good UI.

</comment><comment><USER15> [https://uimovement.com/](https://uimovement.com/) and [https://mobile- patterns.com/](https://mobile-patterns.com/) are sites i as a mobile dev find interesting</comment>


Does the Web need HTML/JS code signing? - <AUTHOR>

As pure HTML/JS web apps begin to encroach on the turf of native applications, they will inevitably face some of the same security problems that native apps once faced. One of these issues is trusted code delivery. This has traditionally been addressed with code signing, yet it is not currently possible to sign an HTML/JS single-page app in the way that an iPhone app or a browser extension can be signed. Do browsers need a new method for HTML/JS code verification? <comment><USER2> No. Web pages are sandboxed and thus have limited control to the computer and operating system compared to native apps.

<comment><AUTHOR> Yet web page code can certainly be modified to perform malicious actions; XSS is a rampant example. More and more Javascript applications handle information that is of high value to users - passwords, private journals, financial information, etc. Code signing can protect users against an attacker who tampers with the architecture that handles this sensitive information.</comment></comment>


Why Are Geospatial Databases So Hard to Build? - <AUTHOR> http://www.jandrewrogers.com/2015/03/02/geospatial-databases-are-hard?h <comment><USER2> Author of that article here. Feel free to ask me questions about all things spatial (and beyond), I will attempt to answer them.

<comment><USER3> I enjoyed the article, and wondered what are the queries you support? Something like "SELECT * FROM world WHERE (distance_from_me < 500 meters);" ?

I ask because a lot of what you discuss seems to resonate with folks in the graphics business who are, for example, culling object geometry from a render scene by doing what is effectively the above operation. Its also what a 'whisper' or 'yik yak' type of App does trying to find people near itself although in that case the problem is greatly simplified by assuming a sphere and testing for containment in the sphere (or the circle if being planar)

<comment><USER2> You can do much more sophisticated operations than simple point-in-polygon queries.

The geospatial implementation in SpaceCurve is quite sophisticated and in some aspects the state-of-the-art. Complex geometry types and operators are obviously supported, and the implementation is fully geodetic and very high precision by default. Complex polygon constraints, aggregates, joins, intersections, etc on data models that also contain complex geometries are supported and massively parallelized. There is nothing else comparable for big data in this regard that I know of.

It is useful to understand that the entire platform, database engine on up, is fundamentally spatially organized even for plain old SQL "text and numbers" data. The SQL implementation is a thin layer that is translated into the underlying spatial algebra. You can use it for non-geospatial things, it just lends itself uniquely well to geospatial data models because it can properly represent complex spatial relationships at scale.

<comment><USER4> Graphics will tend to want 3D spatial queries, though, and this gives the impression of being optimized for, if not restricted to, 2D or 2D-plus- curvature. The article consistently talks about "polygons" as opposed to "polytopes", for example.

Have I got the wrong impression?

<comment><USER2> The geospatial implementation is a true 3-space model. You can content-address objects and features past low-earth orbit by default. All surfaces upon which geometries are constructed are embedded in this space. Polygons relative to an embedded surface, such as a geodetic 2-ellipsoid, are actually represented as 3-space shapes under the hood; the 2-surface is a convenience for popular use cases but the data is not organized that way.

There are an unbounded number of possible shapes and surfaces in 3-space. Storing and content-addressing them is easy enough. The challenge is that to be useful you need, at a minimum, generally correct intersection algorithms for all of the representable shapes. It is an open-ended computational geometry problem and we continuously extend geometry capabilities as needed. Currently, the most complex shapes can be constructed relative to a number of well-behaved mathematical surfaces embedded in 3-space. Shapes directly constructible in 3-space are significantly more limited but I expect that to expand over time as well, particularly if there are specific requirements and use cases.

The underlying representation was intended to mirror the spatiotemporal organization of the physical world at a data level. We can generate projections but the data lives in 3-space.

</comment></comment></comment></comment></comment><comment><USER5> This is an OK introduction to the issues of geospatial DBs (a meta meta article as it were) but two interesting (to me) points were made in passing:

> Most software engineers assume today, as I did a decade ago, that the design > of scalable geospatial databases is a straightforward task.

You could substitute almost any computing topic for "geospatial databases" and still (sadly) have a perfectly reasonable sentence. Partially that is because our field is so new, and partially it's by simple naïveté which is occasionally powerful but mostly dangerous.

> Algorithms in computer science, with rare exception, leverage properties > unique to one-dimensional scalar data models. In other words, data types you > can abstractly represent as an integer.

Alas, this is painfully true, and the field of algorithms is so complex that the one-dimensional nature of our exploration is easy to overlook.

</grumble>

</comment><comment><USER6> I'm not sure why the author is dismissing space-filling curves so quickly, just based on the age of the idea. I've worked with them extensively, in the context of spatial indexing in a database system, in both research and product settings. They address a number of problems described for interval data types. The techniques I've been working with "decompose" a spatial object into a number of regions by recursively partitioning the space, ending up with a small number of "z-values" for each. A z-value is represented by an integer, which is then used as a key in a conventional index, e.g. a sorted array, balanced binary tree, B-tree, skiplist, etc.

Getting a uniform distribution for sharding is easy in one dimension. I'm not sure what kind of scaling the author has in mind, but if you put the z-values in a B-tree, it scales exactly like a B-tree, i.e., extremely well. I agree there is no exploitable ordering of the spatial objects, but the trick is to order the z-values. If a given spatial object gives rise to 4 z-values, then they -- and the spatial object -- appear in 4 places in the ordering. That's fine. Run your search (in one dimension), get qualifying records, and then get rid of duplicates later.

<comment><USER2> The point was really that space-filling curve indexes do not solve the underlying issues with polygon indexing, their oldness was not a factor. That type of indexing is so old that the patents (yes, they were patented by the big database companies) have long since expired. And yet they are rarely used commercially. There is good reason for that. There is a resurgence in their usage by people new to spatial indexing but it is pretty obvious that few implementors are aware of the extent of the computer science, which goes back into the 1960s.

The assumption of size was "large-scale", which in this day and age usually means massively distributed. A B-tree is not a solution to that problem domain.

<comment><USER6> > The assumption of size was "large-scale", which in this day and age usually > means massively distributed. A B-tree is not a solution to that problem > domain.

Another thought on this one: Space-filling curves decomposes the problem nicely: Scale out your favorite, ordered key/value stored, and then layer spatial indexing on top. No need to develop a new data structure that both scales out and handles spatial data.

<comment><USER7> This is my thought as well. It seems like a perfectly feasible solution to me, but I might be speaking out of the naivete mentioned in the article.

</comment></comment></comment></comment><comment><USER8> Hmm, Do you have to index intervals directly?

Algorithms for spatially indexing points are tractable - see k-d trees etc. R-trees seem pretty lame to be honest, like an ad-hoc data structure to partly meet a number of requirements.

Of course, some kinds of indexing and searching are inherently harder than others. But I don't see why one couldn't program any needed spatial algorithm directly from just an index of all points in the system.

For example, suppose you have an index of triangles. If want to find all triangle intersecting triangle (x,y,z), you could itterative expand a near- neighbor search from each point. Every point you find close than the given triangle-point's further point is a potential neighbor and could checked reasonably quickly. This gives the set S of intersecting triangles to a given triangle in log^n(size(S)), making the process relatively scalable.

K-d tree seem more sensible: [http://en.wikipedia.org/wiki/K-d_tree](http://en.wikipedia.org/wiki/K-d_tree)

And there's more: "Storing objects in a space-partitioning data structure (kd- tree or BSP for example) makes it easy and fast to perform certain kinds of geometry queries – for example in determining whether a ray intersects an object, space partitioning can reduce the number of intersection test to just a few per primary ray, yielding a logarithmic time complexity with respect to the number of polygons."

Lends credence to my argument that a polygon-polygon intersection should be obtainable in log^n or maybe even log time.

Space partitioning in general does cool stuff: [http://en.wikipedia.org/wiki/Space_partitioning](http://en.wikipedia.org/wiki/Space_partitioning)

<comment><USER9> The problem is less with the mathematics, but efficient storage and retrieval of a large number of points (i.e. index won't fit in memory) from disk. Doing a search across a tree rendered to disk is really slow.

Even with mathematically sound storage trees as indexes, locations which are close in space could easily be split between two arms of the tree, making proximity searches quite expensive - you would have to traverse the entire tree to ensure that you do not miss data separated by a poor splitting plane.

There are a number of theoretically efficient solutions to this, but they are all become remarkably inefficient when you add in a hard drive.

<comment><USER10> I think if you're going to be doing sufficiently 'biggish data' with those kind of retrieval charcteristics, you've got to just give up on disk and use flash only.

<comment><USER9> Flash solves the "added overhead for random access" portion of the problem, but the overhead for any access at all is still remarkably high, particularly when compared to main memory.

Perhaps when we get more "flash connected to the main bus" options (there are a few commercial options which sit in PCI Express, AGP, and even DRAM slots), we can begin discounting the overhead more and more, but we're not there quite yet.

</comment></comment></comment></comment><comment><USER11> A possible solution to the firehose problem are stream-databases, the catch phrase is "rather than data waiting for queries, it is queries waiting for data".

The people behind Truviso ([http://en.wikipedia.org/wiki/Truviso](http://en.wikipedia.org/wiki/Truviso) subsequently bought out by Cisco) had an implementation of stream processing, an overview is here:

[http://www.neilconway.org/docs/cidr2009_franklin.pdf](http://www.neilconway.org/docs/cidr2009_franklin.pdf)

We looked at Truviso when we had to solve video-player-analytics and it seemed to us to solve that problem. I imagine a tree-structure of stream-queries where the leaves are the data gathering points, aggregating up the tree for summaries would address the scaling problem.

<comment><USER2> The problem with stream processing is that spatiotemporal analytics are rarely summarizations. Typically, the only aggregates that are computed at ingest are path, vector, and polygon features (e.g. an entity path reconstructed from samples). This is not the real-time analysis part of the application, just the ingest processing, and these features must be continuously indexed and online.

Most spatiotemporal analytics compare events and entity behavior in the present with the past at a pretty fine-grained level, hence why the summarizations or in-memory shortcuts do not work that well in practice. Your working set is frequently measured in days to months of data for common use cases, limited primarily by the storage budget.

In short, spatiotemporal analytics requires fully online indexing of the data in the stream, and a pretty big index at that for most apps.

</comment></comment><comment><USER12> Instead of Vincenty's, there is a very nice Meeus method for great circle distance, which has virtually identical accuracy and no loop inside; [http://www.abecedarical.com/zenosamples/zs_great_circle_rout...](http://www.abecedarical.com/zenosamples/zs_great_circle_route.html)

</comment><comment><USER13> The article seems to lack even one good concrete example. All it came up with was a bunch of high level complaints.

</comment><comment><USER14> I'd love to hear some opinions (from the author?) on how well the current offering of major DBMS-vendors (eg. Microsoft, Oracle, ... ) handle geospatial data. How well do they scale and do they have some inherent limitations?

</comment><comment><USER15> A great article, and very true as MemSQL (w2011) just announced its own real- time Geospatial capabilities: [http://blog.memsql.com/geospatial- intelligence/](http://blog.memsql.com/geospatial-intelligence/)

</comment><comment><USER16> I respect the author's experience, but "Database Engines Cannot Handle Real- Time Geospatial" is probably not true. We (MemSQL) have developed geospatial on top of our existing skiplist indexes, and can deliver thousands of intersection queries per second, over billions of points, in milliseconds per query, on very modest hardware. If you want more performance, just add more hardware.

We previewed our geo code at the Esri Developer's Conference last week. The demo delivers queries in millisecond time over 170M records without any tuning. The naive version worked fine.

[http://blog.memsql.com/geospatial- intelligence/](http://blog.memsql.com/geospatial-intelligence/)

<comment><USER2> With all due respect, 170M points is a trivial data model, it fits on a thumb drive. That is seconds worth of data in IoT; you can brute-force the implementation and look great. Several well-known companies do. It just doesn't scale to anything real.

The location analytics data models I need to support, e.g. mobile telecom, have _trillions of polygons_. Indexing millions of records per second continuously concurrent with sub-second queries is pretty normal for IoT, and easy to support with good computer science. (Location is almost never described as a point by primary sources but as polygons; these are converted to centroid approximations for platforms incapable of doing analysis on the source.)

While I do not know the details of your implementation, I do know that indexing complex geometries using a skiplist index won't achieve the necessary volume or velocity required for IoT-like applications. If you restrict yourself to point data, people have been building extremely fast, extremely large quad-trees for years; you just can't scale high-value analytics with them, which limits their utility.

<comment><USER16> Fair enough; though I'm intrigued by the mention of trillions of polygons. How complex are they? 10 points? 100? 1000? What benefit does the raw data give that centroids can't?</comment></comment></comment>


The creation of the modern laptop - <AUTHOR> http://arstechnica.com/gadgets/2015/06/from-laptops-that-needed-leg-braces-to-laplets-engineering-mastery

<comment><USER2> How can you have a history of portable computing and not say a single thing about LCDs? The innovation that contributed most to adoption of notebook computers was the TFT screens with wider viewing angle and better colors. The article talks about the CNC used to make the sleek computers of today. But it wasn't any deficiency of manufacturing that made old laptops look boxy. They had to build the frames to block as much light as possible from interfering with the display.

<comment><USER3> There were laptops with plasma screens (grid compass 2 = turret from 'Aliens'), or EL. At least plasma ones had no trouble with direct sunlight.

[https://groups.yahoo.com/neo/groups/rugrid- laptop](https://groups.yahoo.com/neo/groups/rugrid-laptop)

<comment><USER4> Who would like a grid compass case for the Novena laptop ?

</comment></comment></comment><comment><USER5> "battery gains have been mostly linear for the last 25 years" After a whole paragraph about how Li-ion improves by ~5% per year. Reporters don't understand what exponential means!

<comment><USER6> Growth of 5% per year is geometric, not exponential.

<comment><USER7> What's the difference? Wikipedia says that geometric growth is a special case of exponential growth, so anything that exhibits geometric growth would, by definition, exhibit exponential growth.

</comment></comment></comment><comment><USER8> This article is a bit lacking in details. The GRiD Compass is not given credit for being the first computer with a clamshell case (also a plasma screen). Also, not mentioning the Tandy TRS-80 Model 100 is a crime in a discussion of how the modern laptop came to be.

</comment><comment><USER9> Previous submission from 2 weeks ago: [https://news.ycombinator.com/item?id=9637015](https://news.ycombinator.com/item?id=9637015), which is funny because the article says it was published yesterday.

<comment><USER10> If you check that submission you'll notice that it's on the European version of ars, while this is on the US edition. It seems the two publications are not aligned.

<comment><USER11> Yeah i was wondering why everyone was buzzing about it now, until i noticed that it had been re-published on their .com site.

</comment></comment></comment><comment><USER12> > It is the result of trillions of hours of R&D over tens of thousands of > years.

Wow, I did not know that Ars was so much into hyperbole. That claim is just ridiculous.

<comment><USER13> It's not hyperbole but you could probably say the same about any modern computing device. Think about all the hours spent by the countless engineers designing all the individual components in a modern laptop. Now think about all the hours spent developing / iterating on previous versions of those components to get them to the point they are now.

It's very impressive.

<comment><USER14> Assuming "trillions" means 3 trillion. Divided by the average work hours during a persons career that'd come up to about 33 million people.

That's quite a few engineers.

<comment><USER15> It's like the Civilization technology tree. To get the laptop, you first have to invent Electronics, and before that Physics, and before that...

Easily adds up to trillions of hours.

</comment></comment></comment></comment><comment><USER16> 25 years of laptops and we still have these damn screens on them that are physically attached to the computer.

Any laptop engineers out there: I want a screen that I can detach from the laptop, velcro it to a wall or to a stand and let me use my laptop with good posture.

<comment><USER17> Buy a Surface Pro 3 and use a wireless keyboard/mouse. That's more or less what you're asking for. I will admit you'd need pretty heavy duty velcro to attach it to the wall, but it likely is do-able if you don't mind having a completely fuzzy-backed SP3.</comment></comment>


Ohio gas well blowout leaked more than many countries do in a year - <AUTHOR> https://arstechnica.com/science/2019/12/ohio-gas-well-accident-last-year-released-surprising-amount-of-methane/ <comment><USER2> I grew up in Noble County, Ohio (next to Belmont County) and still get the weekly newspaper delivered to me via mail.

There were exactly zero write-ups of this in the local papers.

The Radio stations in the area are generally owned by one entity (AVC Communications), and there really isn't a lot on their website about it: not really surprising, because 25+ years ago they went Satellite Only and only someone in the news room from 6A-4P.

My biggest complaints about the Oil and Gas industry taking the SE Ohio area and turning it into a cash cow are two-fold: 1.) We're basically treating nature like a resource and stripping it all back to nothing (like AEP did in the 60's and 70's), and 2.) this has basically zero benefit to the locals: most of the jobs in the Oil and Gas industry are specialized and workers imported (leading to stories of welders making $140/hour in Ohio: just simply not true).

I could go on and on about growing up in the second poorest county in Ohio, but, I'll just leave it at this: not surprised that it happened but very surprised that nobody has really reported on this until now.

<comment><USER3> But to even question this in today's political world makes you a liberal agitator. I wish I knew how we could reframe these issues as "proper stewardship of our commons" vs. "but we need the jobs now!"

<comment><USER4> In American politics you don't even need to deliver on the jobs. You can just make some shit up and blame everybody's problems on something else (e.g. free trade, immigrants) and then whip up a frenzy and never deliver. But even after consistently never delivering you're still on the side of the people. Somehow.

<comment><USER5> It would be cool if there was a (website?) which clearly showed the amount of promises a politician actually keeps. It would be nice to see it put together in a nice, visual way too so everyone can understand it.

Ie, verified number of jobs delivered to said community etc.

It would be also nice to have donations clearly labelled against a politicians profile.

The major issue for me is there is a major lack of transparency and accountability in politics right now, globally.

Honestly, I feel like the old four year term system is just way too out of date with the current crisis we need to solve.

<comment><USER6> It's not possible because it's simply not easy to measure many complicated things, such as job growth over many years due to specific actions or infrastructure development that takes a decade or more, but politicians go in and out quicker and voters minds don't pay attention for that long.

</comment></comment></comment></comment></comment><comment><USER7> "Leaked more than many countries do in a year" is an interesting way of expressing the scale. Many countries have little to no petroleum industry.

> Or, for another kind of context, the researchers point out that this Ohio > well released more methane in 20 days than the oil and gas activities in > most nations around Europe do in an entire year—save only the UK, Germany, > and Italy.

[https://en.wikipedia.org/wiki/List_of_countries_by_oil_produ...](https://en.wikipedia.org/wiki/List_of_countries_by_oil_production)

[https://en.wikipedia.org/wiki/List_of_countries_by_natural_g...](https://en.wikipedia.org/wiki/List_of_countries_by_natural_gas_production)

Most European countries other than Norway have orders of magnitude smaller petroleum industries than the US, and many have near zero production. While leaking oil and gas is bad, this kind of journalistic sleight of hand is neither necessary nor productive.

To better put this leak in perspective, the US alone apparently leaks 13 million tons of natural gas yearly [1], and 440 million tons are annually leaked world wide from both human and natural sources [2]. So this leak accounted for 0.5% of nation-wide gas leaking in a given year and 0.014% of global methane emissions in a given year.

1\. [https://www.scientificamerican.com/article/oil-and-gas- facil...](https://www.scientificamerican.com/article/oil-and-gas-facilities- leak-more-methane-than-previously-thought/)

2\. [https://www.scientificamerican.com/article/methane- siberia-c...](https://www.scientificamerican.com/article/methane-siberia- climate-change/)

<comment><USER8> ==Most European countries other than Norway have orders of magnitude smaller petroleum industries than the US, and many have near zero production. While leaking oil and gas is bad, this kind of journalistic sleight of hand is neither necessary nor productive.==

Isn't your comment an equal slight of hand? The article isn't about total leaks in the US compared to other countries. This is about one specific gas leak in Ohio, which they compare to other countries.

<comment><USER7> The point is, leaking _any_ amount of oil is sufficient to exceed the petroleum leaks of many countries that have no petroleum production whatsoever.

> Isn't your comment an equal slight of hand? The article isn't about total > leaks in the US compared to other countries. This is about one specific gas > leak in Ohio, which they compare to other countries

The comparison to other countries, many of which have little to no petroleum industry, _is_ the sleight of hand I'm addressing.

Putting this leak in respect to the total amount of natural gas leakage in the country and the world is a straightforward was of representing the contribution of this leak to total natural gas leakage.

<comment><USER8> ==The comparison to other countries, many of which have little to no petroleum industry, is the sleight of hand I'm addressing.==

My point is that failing to mention that they are comparing 20 days of leakage to an entire year, is also a slight of hand.

20 days is about 5% of a year, so the comparison still works for any country producing even 5% of the amount of oil & gas as Ohio. Ohio drills about 88k barrels/day [1]. 5% of that is 4.4k.

European countries that produce more than 4.4k barrels/day, according to your Wikipedia link, are: Croatia, Hungary, France, Netherlands, Ukraine, Romania, Denmark, Poland, Serbia, Austria, Albania and Belarus (along with Norway, UK, Germany and Italy already mentioned).

[1] [https://www.eia.gov/dnav/pet/hist/LeafHandler.ashx?n=PET&s=M...](https://www.eia.gov/dnav/pet/hist/LeafHandler.ashx?n=PET&s=MCRFPOH2&f=M)

<comment><USER7> I'm not sure I follow. You're counting nation-wide petroleum production of these countries against 1/20th of the oil production of one US state, which only produces a fraction of the 15 million barrels of oil produced daily in the US. How is that a better comparison?

Let me make my point even clearer: go pour a glass of gasoline out in your yard (hypothetically, don't actually do this). You have now polluted more than over a hundred countries' oil production industries. Because out of the ~200 countries on Earth, only 96 have oil industries. I could turn around and write, "one man contributed more oil spills than the oil industries of hundreds of countries!" and in a purely factual sense this is correct. But anyone who bothers to scrutinize this statement will see that it is very misleading.

</comment></comment></comment></comment></comment><comment><USER9> I guess this is related to fracking? I seen something before that Pennsylvania doesn't allow certain waste, so the companies put it in trucks and pay to put it in injection wells in Ohio by the barrel. Pretty sad Ohio accepts money to dump other states waste.

Looked it up to confirm about how waste from Pennsylvania gets dumped in Ohio:

[https://environmentohiocenter.org/programs/ohc/ohio- fracking...](https://environmentohiocenter.org/programs/ohc/ohio-fracking- waste-dumping-ground)

Even caused a earthquake in Youngstown, which isn't where you'd typically expect earthquakes. [https://stateimpact.npr.org/pennsylvania/2018/10/12/fracking...](https://stateimpact.npr.org/pennsylvania/2018/10/12/fracking- wastewater-from-pa-usually-ends-up-in-ohio-some-residents-say-theyve-had- enough/)

Then it gets into the water supply too. [https://www.nrdc.org/stories/ohio- communities-are-becoming-d...](https://www.nrdc.org/stories/ohio-communities- are-becoming-dumping-ground-fracking-industry)

Remember seeing a video once where someone could light the water coming out of their kitchen sink facet on fire. [https://youtu.be/4LBjSXWQRV8](https://youtu.be/4LBjSXWQRV8)

I don't know why state leadership allow this stuff to happen. Then you have the brain drain problem, Ohio is losing highly educated residents to other states with more opportunities, especially when it comes to tech. I feel like Ohio is moving backwards and not innovating. Maybe in Columbus or Cincinnati, things are better but still probably dwarfs in comparison to opportunities in Silicon Valley or Austin.

<comment><USER10> That "confirmation" link in no way confirms that this leak was related in any way to fracking or dumping.

If I were to bet, it is unrelated. Methane and natural gas are not the waste products you would be pumping into an old well.

<comment><USER9> Was talking in general about allowing dumping, but clarified my post.

</comment></comment></comment><comment><USER11> Incidents like this is why we're not going to save the planet with individual actions. I could spend my entire life eating vegan, recycling, and bicycling everywhere and that entire gain is dwarfed by one incident like this.

The only thing that will save us is decisive and collective action. Think about that when you think about your vote in the next election in your area.

I particularly like the suggestion in one of the comments on the original article:

>I wonder if it would have taken 20 days to stop if they were being fined by the pound.

<comment><USER12> Yes, but without you doing all that it would be _even worse_.

<comment><USER13> No, it wouldn't. The commenter's actions make no difference at all on a global scale. Zero.

That's the point: understanding that individual virtue does absolutely, totally, precisely nothing. Only collective actions count. At all.

Understanding orders of magnitude is not something that comes easily to humans but it must be understood to work on large problems.

<comment><USER12> The commenters actions make a difference _to them_ and in turn may influence people around them.

Collective actions start out as individual actions and like that they set the stage for change that is real and measurable.

By your reading Rosa Parks should have just shut up and sat in the back of the bus because 'individual virtue does absolutely, totally, precisely nothing'.

Understanding orders of magnitude is _indeed_ not something that comes easily to humans but it _must_ be understood to work on large problems, right alongside understanding how social change is engendered.

<comment><USER14> There is a difference between individual action leading to social change (such as political activism or civil disobedience) and individual action that makes the individual feel better and satisfies the need to "do something" while having no measurable effect.

If you take one less flight this year, you've considerably lowered your carbon footprint. Congrats. If every single person that flies reduced their carbon footprint by one flight this year like you did... it would make absolutely no meaningful difference with respect to climate change. You might as well whistle in the wind. On the other hand, if all of those people had actually taken political action, meaningful change would be practically inevitable.

The narrative that encourages recycling, buying green brands, and generally expressing how much you care about the earth by how you choose to shop (making your environmentalism part of your identity _as a consumer_ rather than as a citizen) serves only the status quo, and dissipates energy that might otherwise be channelled into making an actual difference.

</comment></comment></comment></comment></comment><comment><USER15> Yet we don't use nuclear energy because of its "risks"

<comment><USER16> I believe that we don't use nuclear energy because it's not economically feasible in a private energy market. Startup costs and operation costs are very high compared to natural gas, solar and wind. We saw how energy companies plowed right through public outcry and resistance in the Dakota Access Pipeline. I think it's silly to assume they wouldn't do the same with nuclear power if they saw a profit in it. Even with federal subsidies and loan guarantees energy companies aren't biting.

<comment><USER4> > We saw how energy companies plowed right through public outcry and > resistance in the Dakota Access Pipeline.

In the US it got blown into something completely different, about sticking it to the libs or whatnot. So we went ahead and did it because our president has the personality of a toddler.

<comment><USER17> The president doesn't make decisions on pipelines. And as far as public outcry -- showing up to a protest doesn't make that the representative voice of the public. I support that pipeline because it's better than sending trucks and trains overland to ship energy. But was my voice counted? Not if using protests as a proxy for public opinion is how it's done.

The oil company could use Star Trek teleportation technology and people would still protest it. They don't oppose the pipeline, they oppose what runs through it. Yet, they didn't all get to those protests in electric cars on on bicycles. They didn't tweet about it on stone tablets -- they use the very product they don't want people to use.

Don't like oil? Stop benefiting from it while simultaneously hating it.

<comment><USER11> >Don't like oil? Stop benefiting from it while simultaneously hating it.

You don't have a particularly compelling argument, as illustrated in this humorous cartoon: [https://thenib.com/mister- gotcha](https://thenib.com/mister-gotcha)

</comment></comment></comment></comment></comment><comment><USER18> Sensationalist title, as Ohio is larger than many countries.

<comment><USER18> I think the leak is bad, but we should be fair, even with things we don't like, right?</comment></comment>


Rails 3.2.16 and 4.0.2 have been released - <AUTHOR> http://weblog.rubyonrails.org/2013/12/3/Rails_3_2_16_and_4_0_2_have_been_released/

<comment><USER2> Please note these contain security fixes for XSS in `number_to_currency`, `simple_format`, and `i18n`, and a denial of service exploit in `ActionView`.</comment>


Discovery of classic pi formula a 'cunning piece of magic' - <AUTHOR> http://www.rochester.edu/newscenter/discovery-of-classic-pi-formula-a-cunning-piece-of-magic-128002/ <comment><USER2> just posted yesterday: [https://news.ycombinator.com/item?id=10544026](https://news.ycombinator.com/item?id=10544026)</comment>


Drone CI 0.2: GitHub Enterprise, IRC and more - <AUTHOR> http://blog.drone.io/2014/3/18/drone-version-0.2.html

<comment><USER2> Very cool. I'm really glad to see the community get involved in this. I think the enterprise support is a big deal.

</comment><comment><USER3> Author has been extremely helpful and responsive with issues filed on GitHub. Definitely appreciated.

</comment><comment><USER4> I love the wall display tool. How does it gets its data? Is there an api?

<comment><AUTHOR> Right now it connects directly to the database although I think ultimately it should use some sort of API.

You provide the path to the database, as well as the repositories to monitor, as command line arguments:

drone-wall --repos="github.com/drone/drone,github.com/drone/drone-wall" \--datasource="/var/lib/drone/drone.sqlite"</comment></comment>


Here are ~32 ways in which I think US policy could be better - <AUTHOR> https://sideways-view.com/2018/09/08/policy-potpourri/ <comment><AUTHOR> (I'm not the author. Author is Paul Christiano)</comment>


Minimalist, arguably useful HTML/CSS snippets - <AUTHOR> https://nullisnull.blog/html-and-css-reconsidered-1a14e658a7e9 <comment><AUTHOR> I know. You can't use most of these "tips" in your monolithic monolith. But you could.</comment>


Fwupd: Updating Firmware in Linux - <AUTHOR> http://www.fwupd.org/index.html <comment><USER2> > By default, any users are able to install firmware to removable hardware. > The logic here is that if the hardware can be removed, it can easily be > moved to a device that the user already has root access on, and asking for > authentication would just be security theatre.

\- [http://www.fwupd.org/users.html](http://www.fwupd.org/users.html)

But it is not given that a user has physical access to the machine, is it?

Well... I guess that's why it says "By default", and you can configure it? Seems targeted at desktop installations?

<comment><USER3> > But it is not given that a user has physical access to the machine, is it?

Yes, I think the logic here is flawed. The only way to know someone can do something in the physical security theatre is by their doing it. Needing to cajole any normal user into running a script is a tad more optimal than convincing them to physically move devices from the server room to the new machine that they won in your raffle.

</comment></comment><comment><USER4> The big question is, what vendors will use this system instead of their own horrible systems?

</comment><comment><USER5> Interesting development. Is there info anywhere of manufacturers who have undertaken to provide updates through this system?

</comment><comment><USER6> this has come at a good time. nearly all thinkpads are undergoing bios updates because of a security issue.. but cannot be done on Linux.

I wonder if someone can build a howto for Thinkpads on Linux.

<comment><USER7> > nearly all thinkpads are undergoing bios updates because of a security > issue.. but cannot be done on Linux. I wonder if someone can build a howto > for Thinkpads on Linux.

Nonsense. Thinkpads can update the firmware from a bootable CD whose ISO you can download from the Lenovo website.

I did this just last week with my Carbon X1. I can guarantee you that no Windows was ever involved.

<comment><USER6> here's the last one I looked for - [http://support.lenovo.com/us/en/downloads/ds001322](http://support.lenovo.com/us/en/downloads/ds001322)

Did I make a mistake ?

</comment></comment></comment><comment><USER8> This is actually pretty sweet. If you are running a non-linux OS you could just reboot into a live disk and pull down all of the updates.

Take it a step further, you could just PXE-boot into a scripted image that loads up and checks for FW updates, then reboots into the default OS when complete.

</comment><comment><USER9> Why can't this be done using standard ways, like deb packages?

<comment><USER10> Technically, LSB mandates RPM as the standard package format, so deb packages would be non-standard.

For a more serious response, if you look at the architecture diagram, the fwupd daemon is independent of any packaging or download mechanism so there should be nothing preventing you from calling it from dpkg postinstall scripts.

</comment></comment><comment><USER11> This completely ignores the "evil maid" scenario.

<comment><USER12> Not if the firmware updater enforces digital signatures like the ucode update does on CPUs.

</comment></comment><comment><USER9> Is it restricted to gnome?

<comment><USER5> Well gnome-software is obviously restricted to gnome,and it might be the only current gui option but I don't think the system is. The page states that you can interact using the D-bus api and that's not restricted to gnome: [https://en.wikipedia.org/wiki/D-Bus](https://en.wikipedia.org/wiki/D-Bus)</comment></comment>


Convert Facebook newsfeed into Reddit-style trending stories - <AUTHOR> http://apps.facebook.com/additback/

<comment><USER2> Is it me or there is serious astroturfing going on in this thread? There are bunch of new accounts with no submissions, 1 comment and 1 karma posting one- liners like:"Awesome","Pretty cool", "Looks interesting"... Just check last 10 accounts on bottom of this thread.

</comment><comment><USER3> Meh, probably the author should not have categorically asked everyone at plan9 (an incubator he heads) to up-vote this story and make comment like awesome , cool etc .

Umar Saif probably thought that he could use his position of power to get all the up-votes he want , what he didn't know was that HN has specific filter to counter the just the karma gathering activities like this one.

<comment><USER4> Yeah I hesitated to even comment on this story, knowing it would be flagged almost immediately. How could someone put a bunch of time into an app related to social ranking and networking and not be savvier at astroturfing?

</comment></comment><comment><USER5> This is a bold step, I must say, Facebook is known to raise issues with app that provide something that Facebook is already providing but in a better way.

</comment><comment><USER6> Interesting concept, the ranking is definitely different from that of Facebook, its cool to get a different view of the same feed

<comment><USER5> Yes, that's what we are striving for. Not only a different but a BETTER view

</comment></comment><comment><AUTHOR> Do you know that around 50% of your friends' updates are never seen by you? Why: Stories are buried deep in your newsfeed; facebook also only shows stories of the friends who regularly interact with you on facebook. Additback digs deep in your newsfeed to find the most interesting stories

No more endless scrolling of your newsfeed to find interesting post by your friends. Get a ranked list of trending stories every couple of hours

</comment><comment><AUTHOR> The app basically goes around the facebook EdgeRank algo and finds trending stories -- as opposed to only showing the stories from friends with whom you have a good edgeRank.

</comment><comment><USER4> the first premise of this app is flawed:

> _Around 50% of the updates by your friends are never seen by you_

Well yes...I suspect that's the same ratio of HN posts that are never read....because no one liked/upvoted them...and if thhe algorithms are right, that's a _good_ thing. I'm not saying there isn't room for a new way to rank posts (in either FB or HN)...but it's a _good_ thing to not see 100% of all the drek that gets posted to your news feed.

* edit: What would be useful is to see a comparison of how the OP's rendition of the feed compares to the status quo. I've been mostly satisfied with FB (though how do you really know how good it could be?) two of my best friends recently got married. Their friends are also close friends of mine. Theoretically, my newsfeed should be absolutely swamped with notes and photos of their wedding, and a constant circlejerk of likes and comments (I've posted scores of photos and tagged them myself, which creates a mass of FB notifications in itself)...but I've been pleasantly surprised to see that FB has still kept up an interesting variety in my feed.

<comment><USER5> I totally agree, that it's a good thing not to see 100% of posts from either HN or FB

But the updates that you miss are also some which have a lot of likes/shares/comments on FB. That's what we do, we don't dig out crap posts, but only those posts that outrank others.

So we have found out that you also miss those high ranked posts, almost as much as 50%

<comment><AUTHOR> You need to take a look at Facebook's EdgeRank and how this circumvents it: [http://mashable.com/2013/05/07/facebook-edgerank- infographic...](http://mashable.com/2013/05/07/facebook-edgerank-infographic/)</comment></comment></comment>


Ask HN: What resource/thing is most needed to reduce homelessness? - <AUTHOR> Thought to ask after the SF homelessness article.<p>Is it a money problem? An ideas/strategy problem? <comment><USER3> It is a poverty problem. People become homeless for a wide range of reasons, but once there, it's hard to get out. How much harder do you think it is to get a job, deal with essential bureaucracy when you have no fixed address?

It isn't because of a shortage of housing. There are homeless everywhere. The most effective solution to homelessness is literally giving people a home. Not just because it means that technically you aren't homeless any more, but it gives someone a chance to get their shit together again.

[http://www.huffingtonpost.com/2014/03/25/housing-first- homel...](http://www.huffingtonpost.com/2014/03/25/housing-first-homeless- charlotte_n_5022628.html)

</comment><comment><USER4> What would make the most difference in the quality of life for those on the street would be public showers, public restrooms accessible 24/7, and access to secure and affordable temporary storage in order to leave possessions temporarily without losing them. Boxes installed along sidewalks which could accept change like a parking meter and be set to a combination to unlock - <USER2> would remain locked until the time runs out. That would make a huge difference, enormous. The loss of paperwork and documentation as well as clothing and other possessions is a constant problem for those on the street - and the inability to leave the bags behind and not be immediately recognizable as homeless is a huge barrier and trap.

It surprises me that laundromats in the bay area do not have pay showers similar to truck stops. When you must carry all of your possessions it would be nice to be able to be free of dirty clothes (and body) after doing laundry. There are, of course, a few shower facilities which open and close at specific hours and people must stand in line to shower in a shared room with others without control of the water temperature. It is an experience to be avoided. I assure you that 'bird baths' in public restrooms are a far better option. Dignified public showers. That would make an enormous difference in quality of life.

And of course, restrooms. It is absurd and repugnant that the bay area, with the enormous homeless population fails to provide public restrooms accessible at all hours. This, again, is an issue of human dignity and it is also an issue of sanitation and health.

Those are the three things that would make the most difference - restrooms, showers, secure temporary storage - and would prevent homelessness from being an inescapable trap. It would allow people to be functional and presentable in the world, to be able to find employment and have options.

</comment><comment><USER5> For two years I lived in a homeless camp. Those folks ridiculed institutional programs to help them because the programs were embarrassingly naive.

They were "taught" things like how to fill out an employment application. These folks were not stupid nor ignorant. They understood this sort of thing. The programs they were often forced to attend to get help were based on fantasies about who the homeless were and what their problems were.

Homeless people have difficulty leading their lives. They have problems coping. Their problems are deep-seated personality problems which constantly interfere with maintaining a directed lifestyle.

Compelling drug, alcohol, and tobacco habits were common, but this was _not_ the problem. Why did they have these habits, and what was this thing that was interfering with living a direct lifestyle?

I don't know.

I do believe that they need a sheltered life and the help of a psychologist with a deep understanding of what was interfering with their lives.

They were trying to lead the best lives they could and be the best people they could manage. But, that was not enough.

</comment><comment><USER6> I think homelessness falls under the broader category of poverty and that's what should be addressed as a whole. A better wholistic strategy that would involve the following ideas could help reduce poverty and increase human welfare overall: 1\. Population Reduction 2\. Mass Education 3\. Better Allocation of resources through reducing zero-sum games.

I've discussed this in more detail here - [https://www.quora.com/What-is-your- most-controversial-or-unp...](https://www.quora.com/What-is-your-most- controversial-or-unpopular-plan-to-improve-the-world/answer/Tejas- Mehta-10?srid=tRPf)

</comment><comment><USER7> Erm, homes, presumably.

Also mental health care.</comment>


Suggest to HN: A Hacker News Amazon Affiliate Link - <AUTHOR>

I post a fair lot of Amazon links when talking about books. I don't have an affiliate link, as I don't recommend enough books to bother with one, and I don't want any appearance of conflict of interest when I recommend books on a public, community site that I'm just a guest at.
But grinich just wrote in a thread:
> Will you post an affiliate link? If you recommend a good book to me, I'd like you to get a kickback.
Maybe he's got a point - I mean, my no affiliate link links just mean a waste of a small percent that somebody could be getting. To Paul and the rest of the Hacker News/YCombinator team, do you guys want to register an affiliate link or have one already? I'll add it to my posts here, and you can do whatever you like with the credit you get. Use it to buy books to read or to stock a startup library for your funded startups. I don't mind taking a minute to append it to any book recommendations I make, it gives a little coin or resources back to a great site, and I suppose it is a waste of potential revenues not to use Amazon's affiliate program.
You don't need to append it automatically or anything crazy - just let people choose to use it if they want to. If you've got one, I'll use it, and be happy that you guys are getting a bit of extra coin or free books to benefit what you're doing. <comment><USER2> When I link to aff stuff on amazon I usually do "[link] (aff)" so if people want to go out of their way to copy the link and remove the affiliate tag they can.

... but more recently I've been using a link cloaker (ie paulstamatiou.com/random_product_on_amazon ) with a WP plugin called pretty link pro. Boosts conversions a ton, tracks clicks and so on and can even rotate links so you can test out different affiliate codes with the same link and tell it how often you want certain ones to appear. Sly, yes I know but I can use the help to pay my rent. :-)

That being said my best month ever was only $300 on amazon and that was because someone read my review of a cell signal booster and literally bought a pallet of them.

</comment><comment><USER3> My idea is based on the following: If the reader was amazed with the review you wrote and convinced with the book and he "Really" wants to thank you, so he should go more steps to do that.

Put your referral tag in your profile, he add it to the link and give you back commission.

If he liked your review and he doesn't want to kickback even a cent, he won't bother. Like that, only people who were both amazed and convinced of the writer-commission idea will give the commission. People who really don't care about whom was the writer and don't want to support him for writing the review, won't pay. Neither they won't write review , since they'll think people will never bother to help them.

The reason for this non-easy-reward system is to stop the flow of spammers that may result if affiliate links were permitted here. HN has got a potential buyer base (imo), so marketers will come here and write their reviews, even if they don't induce the HN community, they'll still think of the organic traffic that will drive by time. Such a system will kill the odds that organic traffic can make sale, people who find HN through Google don't know that the Tag is in the profile, even if they bother to kickback!

</comment><comment><USER4> HN could do what Metafilter does and automatically rewrite all affiliate links to their own. If they don't want/need the money, they could donate it to charity.

<comment><USER5> why does metafilter think they're in the right to change those affiliate links and steal those rewards? why would anyone bother responding to posts on their site knowing that not only can they not get rewarded for responding with a recommendation, but that metafilter is stealing the reward from them?

i don't understand the aversion to affiliate links. if someone buys a book from amazon based on my recommendation (and link), amazon thanks me by giving me a portion of their profit on the sale. that shopper doesn't pay extra money, the author doesn't receive less money, and i am rewarded for taking the time to respond to that person's question.

<comment><USER6> > why does metafilter think they're in the right to change those affiliate > links and steal those rewards?

Because that is the policy that the moderators and community arrived at. Some people think that allowing affiliate links incentivizes behavior detrimental to the site. You are free to disagree, but calling it "stealing" is begging the question.

</comment></comment></comment><comment><USER7> I think a problem with affiliate links is that they can only reward people who cause sales to occur. This is good if your commentary on the book was so useful it made someone buy the book. But what if your criticism is so useful, it caused a prospective buyer to not buy the book. You did just as much a service to him/her by providing a valuable opinion, yet you get no reward.

Next to every Amazon affiliate link should be a "Donate if I convinced you not to buy" button.

</comment><comment><USER8> I would suggest, as I have seen heavily used on forums, that you provide two urls for readers. One would be a direct link, and the secondary link contains the affiliate id. I personally like to think of it as along the same lines of a "contribute" paypal link on low traffic blogs.

I tend to manually remove affiliate URLs from links if I was not given a choice. I tend to be more favorable of signing up or purchasing with affiliate URLs when:

* I was given a decision to opt in

* I was notified that the link contained an affiliate code up front.

<comment><USER9> That's really probably just you :) I'd just add the link, don't see the harm done.

<comment><USER10> I'm with you on this one. I don't see the problem with affilate links. It doesn't change the value you get out of the product.

Sure there might be a few odd people out there who live of affilate links but is that a bad thing? If someone sees value in the information they've been provided where is the harm?

<comment><USER11> The harm is just that you have to question the motivation for posting the link in the first place.

"Is this person posting this because they really like the product or because they get a kickback?"

</comment></comment></comment></comment><comment><USER12> YC is not a charity.

<comment><AUTHOR> Sure, and a dolphin isn't a tiger. But if I can increase the revenues of a site that I'm getting free access to without any real hassle, I'm happy to. I'd like people who are benefiting my life to get compensated. Two reasons - first, it's a good thing in a general sense for people to get compensated for their efforts and value they deliver. Second, generally having people get paid for delivering a good product makes them more likely to deliver it. Who knows how much the most HN Amazon commissions could be - a couple hundred bucks a month? But that's a couple grand a year, which would be enough to build and sustain a pretty good YCombinator library of technical and business books. And again, it's no skin off my back, so why not?

<comment><USER12> A site which is owned and operated by an (apparently quite successful[1]) early-stage investment firm. Your sentiment is admirable, but I don't think they need a couple thousand dollars extra per year.

But you're right: it's no skin off our backs.

[1] [http://blog.awesomezombie.com/2009/12/analyzing-y- combinator...](http://blog.awesomezombie.com/2009/12/analyzing-y- combinator.html)</comment></comment></comment>


A 'Big Red Button' for AI to interrupt its harmful sequence of action [pdf] - <AUTHOR> https://intelligence.org/files/Interruptibility.pdf <comment><USER2> I feel like an actual solution for this is more literal.

I mean, if a box-carrying robot actually hurts someone, then the proper course of action is to detonate something inside it, and completely immobilize it, cut it off from whatever it's networked to, and preserve everything else so we can dissect what went wrong and prevent it.

This is a really interesting approach but I think a much simpler method would be a bit more practical in actual usage.

<comment><AUTHOR> I like this solution much better when facing less severe situations. Keep in mind that it's also a solution for learning, so it can be used to guide the AI towards the proper action, instead of baby sitting it and redesigning it forever.

<comment><USER2> Which is what is supposed to be happening anyways. That's literally what machine learning is about.</comment></comment></comment>


What We Learned Driving 2,000 Miles Through the South's Startup Landscape - <AUTHOR> http://www.theatlantic.com/technology/archive/2011/11/what-we-learned-driving-2-000-miles-through-the-souths-startup-landscape/247671/

<comment><USER2> Our 20-person team is split between Nashville, TN and Vancouver Island, BC. Staying out of the major startup hubs has definitely helped us to avoid the assumption that funding is a critical component or yardstick of success. For us, I think this was a lucky break; we still haven't encountered a day in which lack of funding is a significant impediment to our growth.

</comment><comment><USER3> The big four cities are SF, NYC, DC and LA? I know Boston is slipping a bit, but in what context is it not "bigger" than LA or DC for startups?

</comment><comment><USER4> I noticed a third of the photos are from Durham, NC.

I grew up there, and used to work for a startup there. In many ways I didn't like Durham so much -- way too suburban and sprawling for me, most of my friends lived in cookie-cutter apartments, the sense of local character was kind of faint. But it definitely does have a lot of things going for it, in particular a critical mass of talented, energetic people.

It's the kind of place where if you want to try out X, you will find a surprisingly large and dedicated group of people doing X -- for just about any value of X.

And, to add to that, I've noticed that the center city has been rejuvenated a lot since 2004 when I left.

</comment><comment><USER5> The south is a wonderful place for start-ups. In my smaller-ish city of Lexington, KY we have a close-knit, vibrant community of aspiring entrepreneurs. The cost of living is low and the University of Kentucky brings great talent to the area (and they actually want to stay).

Also, most of the area businesses are bootstrapped.

</comment><comment><USER6> Ok, so it is cheaper to do business in the south...but businesses that start in the south will get a 250k round of funding while the same one would get 2 million in the valley...which one is going more likely to give a company to get the traction it needs?

<comment><USER7> Because there's definitely no examples of bootstrapped companies which have gained more traction than funded companies, and (gasp) maybe even been more successful...

Come on. Let's end this celebration of funding once and for all, and start celebrating the hard working entrepreneurs who bootstrap.

</comment></comment><comment><USER8> They missed the 15 or so startups from jumpstart foundry, incubated at the Nashville Entrepreneur Center.

</comment><comment><USER9> What I learned from the slideshow: nscrtbl nspllbl names have spread far beyond the valley! (Ibuzzn? Sqord? Wefiends? Scholr.ly?)

<comment><USER10> Really? Scholr.ly is lumped with those? Scholr.ly is probably the result of scholar.ly and scholarly.com both being taken, and likely a domain name acquisition issue. Other than that, it's quite clear in its connotations, and hardly inscrutable. It's only mildly unspellable (an omission that's easy to remember, if problematic). Unlike Sqord, or really the others you listed. Not sure it belongs in that list…</comment></comment>


The perils of Lisp in production - <AUTHOR> http://symbo1ics.com/blog/?p=1321 <comment><USER3> You can write spaghetti code in any language.

Whether enforced compile-time strong type checking is a benefit seems to depend on the programmer. It apparently helps some people, it certainly does not help me.

For what it's worth, I've been building "production" (is "production" something you make money on? I find this word increasingly vague) systems with both Common Lisp and Clojure for quite some time now. I prefer Clojure. But both languages are lisps. My thoughts so far: you can build spaghetti code in any language. You can either use a language that lets you run your spaghetti quickly, or one that whacks you on the head repeatedly thus making your spaghetti stiffer and straighter. But you end up with spaghetti anyway.

I agree that it is difficult to write good lisp code. In my code, I spend a lot of time thinking about contracts and data structures. If I am not careful, I end up with problems later on. But using a language like Java doesn't solve that: you just get the illusion of "better code", because your spaghetti design is now codified into neat factories and patterns.

The advantage of using a language from the lisp family is that reworking your spaghetti into something better is much easier, if only because there is so much less code.

<comment><USER4> Have you ever worked on a 1M line code base? Or with dozens (or hundreds!) of developers.

Once the team gets big, or the code gets so big you can't hold it all in your head, _any_ machine-enforced QC can have a major impact.

(I'm working on a very large Haskell code base now -- the parts that tend to cause trouble are the dynamically typed bits -- they can just go wrong in so many unanticipated ways, and the complexity means its almost unavoidable that devs miss things).

I'm increasingly convinced that to wrangle massive code bases, you need both heavy abstraction abilities (higher order functions, DSLs, algebraic data types, polymorphism), _and_ very strong typing, to keep the complexity under control

<comment><USER5> A 1M line codebase can mean many things. It can mean code that belongs to many systems mashed together as if it were a single thing. It can mean that the code that should belong to different systems is tightly coupled into one monolithic entity that should have been several little entities. It can mean lots of boilerplate code too. It can indicate a lack of timely refactoring on an aging codebase so interconnected nobody has the courage to separate in more manageable pieces. None of those can be solved by clever language choices alone.

You also mention things going wrong in unanticipated ways - this may signal that the problematic code where errors bubble up is not problematic at all - it is called by code written by people who don't really understand what the functions do and who probably didn't write adequate tests for that - <USER2> the tests should catch the unanticipated parts. The problematic code is the one calling the parts where errors bubble up. The canary is not responsible for the gases in the mine.

While you may be right that, in order to deal with multi-million-line codebases you need static typing, I'd much rather split that codebase into smaller units that could be more easily managed.

Wearing a straitjacket is often a consequence of an underlying condition that can, sometimes, be corrected.

<comment><USER6> I would expect that dons is talking about a 1M codebase which already consists out of manageable pieces. It's only, that the pieces have to work together, talk to each other, know about each other (but not too much).

Sometimes software solves problems which provoke incidental complexity because of sheer size and my experience (albeit not above 500k) tells me that indeed, all bits help, also compiler enforced type checks. I would never bet my life on tests. As you write: "because the tests should catch the unanticipated parts", that's the point: tests never catch unanticipated parts by their very nature. Sometimes, by sheer luck, yes.

<comment><USER5> > tests never catch unanticipated parts

No, but calling a function with arguments of the wrong type is something tests would catch.

<comment><USER7> > No, but calling a function with arguments of the wrong type is something > tests would catch.

Or the compiler.

<comment><USER5> A compiler can only go so far. It'll happily compile:

      #include <stdio.h>

      int main(int argc, char *argv[])
      {
        FILE *fp = fopen("outfile", "w");
        fclose(fp);
        do_something(fp);
        return 0;
      }

      int do_something(FILE *f)
      {
        fprintf(f, "This should work if you know what you're doing");
      }

And it won't work.

<comment><USER7> In C++, proper scoping with RAII (similar to CL's with-blah idioms) would alleviate that issue as do_something in the code below won't compile.

        struct WithOpenFile {
            FILE *fp;
            WithOpenFile(const char *fileName) : fp(fopen(fileName, "w")) {}
            ~WithOpenFile() { fclose(fp); }
        };

        int do_something(FILE *f)
        {
            fprintf(f, "This should work if you know what you're doing");
        }

        int main(int argc, char *argv[])
        {
            {
                WithOpenFile file("outfile");
            }
            do_something(file.fp);
            return 0;
        }

But Lisps will happily accept this:

        (defun do-something (foo)
            (1+ foo))

        (with-open-file (file "outfile")
            (do-something file))

<comment><USER5> Fine. But now you'll have to accept both files and network sockets. What do you do?

<comment><USER8> Pass the int representing the FD instead of the FILE*?

<comment><USER5> And then you are back to dynamic typing again, except that, when the integer you send points to something fprintf doesn't like you'll get a segfault instead of a doesNotUnderstand.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> If you presume that it's OK to rely on compiler errors rather than attempt to exercise all code paths, then yes, Common Lisp, Scheme, Clojure, JavaScript, PHP, Python, Ruby, Perl, and bash will be more perilous than C, C++, C#, Java, ML and Haskell. (Which of the former group are Lisps?) I disagree with the presumption.

<comment><USER5> I think the most dangerous piece in such system is, indeed, the programmer who thinks the compiler will save him from himself.

<comment><USER10> Much less dangerous, in my experience, than the programmer that thinks he can keep a huge system all in his head. I've moved back to static languages after over a decade with dynamic languages and I couldn't be happier to let the compiler take care of a lot of low-level consistency issues so I can focus on the code.

<comment><USER11> An alternative approach is to avoid building systems that require you to keep the whole "huge system" in your head at once.

</comment></comment></comment></comment><comment><USER12> Should I guess (from the domain name) that the link is running on Lisp? I can't get past that!

<comment><USER13> Surprisingly no, blub strikes again. But this is just some guy's blog, I don't see any definite connection to the Lisp Machine pioneers.

      Server: Apache/2.2
      X-Powered-By: PHP/5.1.6

HN itself is basically pg's testbed for prototyping <http://arclanguage.org/> though.

</comment></comment><comment><USER14> Summary: use SBCL or CMUCL, declare your types, hire good programmers, and don't commit to a deadline without getting a commitment to a specific set of requirements.

</comment><comment><USER15> Statically checked types are little more than very broad pre/post conditions on your code. Both CL and Clojure offer such conditionals and both can do it at compile-time if necessary.

I don't get it.

<comment><USER16> Pre/post conditions at compile-time in Clojure? Nope. How could you know the value returned by a function at compile time? Or the value of the inputs?

You could only know the types of the input if the compiler was smart enough and define constraints on the types but as far as I know it's not possible in Clojure.

<comment><USER15> You're right, Clojure doesn't support compile-time conditions. Must've been confused.

</comment></comment></comment><comment><USER17> I thought the main peril was that the beards of the programmers would get caught on things...

<comment><USER18> This so childish. If this article was about Python or some other language, this type of comment wouldn't be tolerated. Why is it when the topic is lisp?

<comment><USER17> I like this Larry Wall quote:

"By policy, LISP has never really catered to mere mortals. And, of course, mere mortals have never really forgiven LISP for not catering to them."

I say this as someone who's prototyped several things in Scheme before porting this to other languages.</comment></comment></comment>