From Python to Lua: Why We Switched - <AUTHOR> https://www.distelli.com/blog/using-lua-for-our-most-critical-production-code <comment><USER2> I usually don't comment on these posts, but I felt the need to in this one. A couple of things stood out:

\- Python 2.4? Really? In 2014?

\- Yes, writing compatible code with a 10+ year old release of Python is going to be hard. Who uses 2.4 still? You have to use the lowest common denominator, I mean I don't think Python 2.4 even supports context managers!

\- Yes, in 2014 you should be supporting Python 3, or at least have it on your roadmap.

\- The ethernet address (MAC?) via uuid.get_node(). Or for something a bit more advanced you can use netifaces[1] (or even better psutil.net_if_addrs[3]). One of Pythons core strengths is the number and variety of the packages available. Took 2 seconds of googling to find a cross platform solution.

\- Calling C from Python is really really simple. Use the built in ctypes library, or something better like cffi[2]

I'm obviously firmly in the Python camp so perhaps I'm a bit biased, but I don't see any clearcut reason to switch other than "the current code is not optimal, let's re-write it in X", where X could be anything. I would actually say Go would be a much better fit than Lua in this situation. I guess the memory reduction/CPU usage is a point, but really your program seems to be network orientated. What's wrong with asyncio? How is replacing everything with a C library _actually_ better?

1\. [https://pypi.python.org/pypi/netifaces](https://pypi.python.org/pypi/netifaces)

2\. [https://cffi.readthedocs.org/en/latest/overview.html#simple-...](https://cffi.readthedocs.org/en/latest/overview.html#simple- example-abi-level-in-line)

3\. [https://pythonhosted.org/psutil/#psutil.net_if_addrs](https://pythonhosted.org/psutil/#psutil.net_if_addrs)

<comment><USER3> Hi, I'm the Founder at Distelli and I wrote the original version of the Distelli agent in Python that Brian rebuilt using Lua. The main reason I picked Python 2.4 is because we have large enterprise customers that are / were running Python 2.4 on servers and I had to support Python 2.4 to serve their use case.

<comment><USER4> They were probably running RHEL5, but even in 2014 they would have been in the Production 3 phase which is really should only be used for legacy purposes, and that's definitely not the sort of system you push new software to. Red Hat even pushes those customers towards virtualization because a) hardware rarely lasts 7 years and b) they won't bother backporting drivers for new hardware to the old kernel at that point.

I still use a few RHEL5 machines but nobody would ever dream of pushing new software to them.

<comment><USER5> >not the sort of system you push new software to.

Telling customers that they are doing things wrong isn't really a great way to win customers.

<comment><USER6> This is something I don't understand. If someone is using Java, they'll install the version of Java they need. If they use Ruby, they'll install the version of Ruby they need. If they need Lua, they'll do the same thing.

Why everyone is relying on python that's installed there for RedHat's needs (e.g. yum).

Unlike other languages, Python is probably the best one that can have multiple versions coexist together.

<comment><USER7> As a consultant I have very different experiences. Even in Java development you find shops that refuse to upgrade their JVM for various reasons.

RHEL 5 is really common.

<comment><USER8> A lot (all that I'm familiar with) of the enterprise software packages that I've worked with ship with their own, very specific (read: regression tested against) version of java. Enterprise software usually requires a ton of servers, so it's not like anything else on those servers will be using the jvm.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> The overall tone of the responses here are surprisingly negative and doing a lot of Monday morning quarterbacking, as if this was some kind of failure. There are wild assumptions about their clientele and the desires of the company, completely disregarding the fact that the company did this to meet their own stated objectives.

They wanted something easy to deploy (single executable), easily portable, very small CPU footprint, and very small memory footprint.

Lua is well regarded for all these things, and the company succeeded in meeting all these objectives.

Congratulations Distelli. Nice job. And thanks for sharing.

<comment><USER10> If you rewrite your app you end up with something better, in any language. It's an uncontrolled experiment, and so the headline is very likely misleading - this isn't about Python and Lua. And many of us have been majorly inconvenienced when a manager read a similar headline and declared that we would rewrite our app in language y, regardless of whether that language is appropriate to our circumstances.

Something like [http://roscidus.com/blog/blog/2013/06/09/choosing-a- python-r...](http://roscidus.com/blog/blog/2013/06/09/choosing-a-python- replacement-for-0install/) where the author actually considers multiple candidates and performs a controlled comparison would be far more valuable. Given that the goals sound very similar I'd be very interested to see a comparison between Lua and OCaml.

<comment><USER11> > If you rewrite your app you end up with something better, in any language.

You end up with something that is superficially better, but misses a lot of edge cases that the ugly legacy application covered. Fortunately, you have usually moved on to save other projects, leaving lowly maintenance developers to fix the problems, thereby creating job security. It's a win win situation, really.

</comment></comment></comment><comment><USER3> I'm the founder at Distelli and I just want to add a quick bit of background around the original python decision (2.6 vs 2.4) as well as the decision to "rewrite in Lua".

1\. I started the company as a single founder and wrote the first version of the agent. See this HN post that gave me the encouragement to keep going (thanks HN!) - [https://news.ycombinator.com/item?id=6059481](https://news.ycombinator.com/item?id=6059481).

2\. I wrote the original agent in python 2.6. It was great because I could use things like python-requests (which was awesome!). However my first encounter with an enterprise customer showed me that was not feasible because they were running only python 2.4 and would not be upgrading for another 2 years.

3\. I had the choice of either not getting the customer or back porting the code to Python 2.4. I chose to back port the code to python 2.4 and get the customer. This was a lot of work but worth it because I was able to then raise a round with a16z. - [https://www.distelli.com/blog/distelli-series-a- funding](https://www.distelli.com/blog/distelli-series-a-funding)

4\. When Brian joined Distelli, he actually worked on the Python code for many months and made many improvements and added many features. When it was time to add the streaming logs feature he built the prototype in Lua just to prove that the idea would work and then we talked extensively to figure the direction we should go in.

5\. We decided to move to Lua because Brian was very familiar with it and it was a lightweight language and we could ship a single download with zero dependencies. We also looked at using Go. At the end of the day the decision was Brian's to make and own as the senior engineer on the project. The passage of time has convinced me that was the right decision.

I'm sure we could have stayed with Python with all the suggestions on this page, but we decided to go with Lua. It was a good fit for what we needed and honestly I felt that my original code was not in great shape and would need a lot of work to refactor and a rewrite felt like the right decision at the time. Looking back I still think it was the right decision and I'm very happy with the end result.

edit: grammar

<comment><USER12> I have been programming Python by day and Lua by night my entire career for the past 7 years, and I find the main difference between the two is, Python has the overwhelming advantage in its package manager providing ease of access to thousands of packages[1], and Lua is faster and lighter by a constant factor. In your case, Python's package system is a disadvantage - you want to have as few dependencies as possible.

I think your company made a good decision. The only reason not to have done it was because your existing code was valuable, but as you've said it wasn't in great shape and required tons of work to refactor and rewriting anyway.

[1] Lua has LuaRocks, but it's just not the same.

<comment><USER13> I looked at Lua briefly, but the immediate impression I got was that Python gives you more help in avoiding errors. In Lua, you might be bitten by:

* variable name typos (undeclared vars are nil),

* accidentally confusing array-style tables and hash-style tables (or vice versa),

* tables with "holes" (nils) in them,

* multiple assignment with wrong number of vals or vars (mismatch silently ignored),

* variables are global by default.

Have you found this to be the case at all? Is it more difficult to avoid bugs when writing in Lua?

<comment><USER12> _Have you found this to be the case at all? Is it more difficult to avoid bugs when writing in Lua?_

Good questions.

 _\- variable name typos (undeclared vars are nil),_

There's a linter to catch undeclared variables. [https://github.com/mpeterv/luacheck](https://github.com/mpeterv/luacheck) There's also some runtime checking techniques you can use. [http://www.lua.org/pil/14.2.html](http://www.lua.org/pil/14.2.html)

 _\- accidentally confusing array-style tables and hash-style tables (or vice versa),_

I've had that happen about as often as confusing Python dictionaries returned by different types of functions. (Which is - not often enough for me to remember the last time that happened)

 _\- tables with "holes" (nils) in them,_

When I'm building a vector and I want holes, I use false to fill the holes.

 _\- multiple assignment with wrong number of vals or vars (mismatch silently ignored),_

For me, when I'm thinking about multiple assignment to function returns, it does mean having to read documentation for a library function more carefully than if I was reading the same for another language. If it's as simple as "local a, b, c = 1, 2, 3", then it's no issue.

 _\- variables are global by default._

If I'm using linting, when I can't have global variables implicitly, then it's like C or Java where I must declared all my variables and by default they are local to the scope.

Otherwise it's just like javascript where it's best practice to always declare variables using "var".

<comment><USER13> Thanks for the feedback! Will have a look at the linter.

</comment></comment></comment></comment></comment><comment><USER14> The described issues that prompted the switch don't really have much to do with Python. The issues map back to running on whatever installation of Python the customer might already have.

Lua, in and of itself, didn't solve the problems. Shipping their own, known, Lua interpreter + extensions solved them. (which they could have done with Python)

<comment><USER15> The reason why people don't want you to ship your version of Python is that it takes a lot of space. Lua makes that hurdle much easier.

For example my local Python venv is about 130 MB. It symlinks in another 52 MB of standard Python libraries. And I know it won't run without some unknown amount of additional dependencies in C-libraries. If I wanted to send that to someone I'm starting at 200 MB. After compression that might be 40 MB or so. You're going to have to work to strip that down.

By comparison their download including Lua, code, and libraries is under 6 MB.

I have a minimal running Python application that was started recently and has done nothing. It takes close to 90 MB of RAM just to start.

Theirs is generally under 10 MB while it is running.

So at every step Python requires 5-10 times as much data. If you're trying to get other people to put you in their containers, this can matter a lot.

<comment><USER10> Or it can not matter at all. To the kind of enterprise that is still running RHEL 6, maybe shipping a 200mb install on a CD will show them that you're a serious business.

There are use cases where 200mb of disk space matters, but for the overwhelming majority of applications, a 5% (say) increase in developer productivity would be well worth adding 200mb to the install.

<comment><USER16> The point of the Distelli Agent is to deploy software. Think of it as the "bootstrapper". Serious customers would rather use that 200MB for their own software, not for the "bootstrapper" :). Here at Distelli we optimize for the customer, not for our own internal developer productivity.

<comment><USER10> Good soundbite, but if using 200MB of extra space would let you lower your prices while offering the same service (by increasing developer productivity) then I bet many customers would take that.

And did you try spending a person-week or so of dev time trying to minimize the size of the Python version? That would be a much smaller one-off cost than the costs of migrating everything to Lua, and I could easily imagine you'd get it down by one or two orders of magnitude. (You could do the same for Lua too I'm sure, but if we're talking about say 5mb vs 200k then that's "who cares?" size for most customers).

</comment></comment></comment></comment></comment><comment><USER17> It isn't linked to in the article, but if you're interested in this please do check out luvit by tim caswell ([https://luvit.io/](https://luvit.io/)). It's a very impressive system.

At a very high level luvit looks like node.js for Lua, with some extra goodies thrown in like being able to bake your apps into single binaries. This is awesome in itself and the article discusses these things.

That said, I think the _major_ innovation over using javascript is that it allows the support for coroutine-based I/O. This means you can get the performance benefits of non-blocking node.js without the headache of callbacks/promises. I think the linked article only touches on coroutines briefly but they are a major piece of Lua's awesomeness that you shouldn't miss.

Though you aren't required to use coroutines, the package management server which hosts packages for luvit is called "lit" and it seems to demonstrate this ability nicely:

"Lit is written in lua and uses the same system platform as the luvit project, but is has a very different I/O style. Instead of using callbacks and event emitters, it uses coroutines so that code can be written with simple blocking I/O, but still maintain the abilities of a non-blocking event loop." (from the README on [https://github.com/luvit/lit](https://github.com/luvit/lit))

<comment><USER18> I also suggest looking openresty based frameworks, it is non blocking I/O, but you can write procedural code normally and Nginx will handle events internally. That means _no callback hell_

There's Lapis, by leafo [http://leafo.net](http://leafo.net) And I'm also improving Nginx experience on the framework I develop, Sailor, because it was originally developed for Apache [http://sailorproject.org](http://sailorproject.org)

</comment></comment><comment><USER19> On the python side, this matches my experiences with python. Python is a great language, and I like it very, very much. However, packaging and deployment in python remains such an amazing mess, even after 20 years.

And outside of scientific computing, python doesn't offer any real edge, to me at least. Ruby has better tooling libraries, java/go/c++ have better performance, JS has more powerful website frameworks.

Why should I bother with that packaging mess after having it done once already?

<comment><USER20> > packaging and deployment in python remains such an amazing mess, even after > 20 years.

I'm Lua fanboy, but I've to admit, that I'm missing clean deployments and upgrades with virtualenv and pypi available in Pyhton. I've never had any issue so far, am I being just lucky bastard? It has been like that for many years already.

> Ruby has better tooling libraries

On the other hand, I've honest to maintain one Redmine instance (which is written in Ruby) and this is real nightmare for me. Each time I've to upgrade, I've to drink a lot day or two in advance :-)

<comment><USER21> those handle python deps ok, but not lib dependencies, at least last time I was forced to use it. If your company has a mix of older and newer projects, you'll end up with eg conflicting dependencies on libxml.

My solution was to docker all the things. It made my life so much better; you could simultaneously run multiple services on the same box without awful fragile hacks.

<comment><USER16> Docker is a great tool, but not all customers will have that available to them, and the docker experience isn't great for customers on Windows/Mac since they have to run Linux in a VM, which means the agent would have the footprint of an entire OS on those platforms.

</comment></comment></comment></comment><comment><USER22> Yeah, the only real advantage of lua is that you can link your code into the binary, and that it's super lightweight. But you can still deploy a python binary with your code, so the only real reason for switching is, "I like lua, and I felt the need to make my team switch."

Not that I don't lua. It's like the best parts of javascript, mixed with some AWK and scheme, with all the crap removed. OTOH, from-1 indexing is evil.

<comment><USER17> I find that the only reasonable complaint that can be made about Lua is the from-1 indexing. Like: "I found this really awesome language and it's ONLY wart is that it doesn't index arrays like every other programming language in the world".

Yet, I still admire the Lua creators for making this bold choice. Why should we use from-0 indexing for all eternity? Just because it's always been that way? This is a real question if you have the answer.

<comment><USER23> In addition to Lua, these languages are 1-based: ALGOL, AWK, COBOL, Fortran, Julia, Mathematica, MATLAB, R, PL/I, and RPG.

<comment><USER24> BTW, FORTRAN is the big reason Lua has 1-based indexing. Lua' s first users were engineers, which were more familiar with FORTRAN.

</comment></comment></comment></comment><comment><USER25> Here are some ideas that could work just as well without a full rewrite:

\- [http://nuitka.net/pages/overview.html](http://nuitka.net/pages/overview.html)

\- [https://code.google.com/archive/p/shedskin/](https://code.google.com/archive/p/shedskin/)

\- [https://micropython.org/](https://micropython.org/)

It would result in a similar binary, but save the full code rewrite.

</comment><comment><USER26> As a crazy exercise to learn both Python and Lua at the same time, I've been porting a small Numpy project to Torch. I will say that both the languages are equally nice, but Numpy is much cleaner compared to some of the seemingly hacky things I need to do in torch.

For example. In numpy:

         x = numpy.array([1,2,3,4])
         y = 1/x

In torch:

         x = torch.Tensor({1,2,3,4})
         y = x:apply(function(n) return 1/n end)

In any case, its been a fun and interesting exercise!

<comment><USER27> As ufo said, Lua could also support the 1/x syntax since it does have operator overloading. Additionally, there is nothing hacky about having an apply method taking a function.

</comment></comment><comment><USER28> Any rewrite is going to be superior to the old code (in any language).

It sounds like the author just wanted to move it to Lua to me. I would only go with Lua over Python if you needed something more embeddable. Like the game scripting usecase it's known for. The lost libraries and language features may make some things harder. I package up Python and my code, deploy it to other platforms all the time with under 5MB footprint in a single executable using PyInstaller.

That said, it'll work out fine with Lua. My question would be who will maintain something years down the road. There's far more Python devs than Lua. The blog said he joined in 2014 so it may end up being someone else.

</comment><comment><USER29> Did you ever contemplate Golang as a possible solution? Seems to me like it would have provided a cross platform build environment superior to both Python and Lua in terms of standardization and ease.

<comment><USER3> yes we did consider Golang but Brian' familiarity with Lua was a major factor in the decision. Most importantly however, we foster a culture of ownership and it was Brian's decision to own and he owned it and it was the right decision.

</comment></comment><comment><USER30> Couldn't they have used virtualenv with Python? Also, Python packages have a way of declaring dependencies including specifying the Python version that your package requires. This should've guarded against any incompatibilities. I don't think any of this was specific to Lua.

<comment><USER31> I don't think virtualenv would solve their problems here; they are installing their agent on a customer's machine. I imagine they didn't want to ship the entire Python interpreter or alternatively install whatever version they want to use on the customer's machine.

But you're right, I don't think Lua is unique in being able to fix this issue. I think it's more that the author was already very good with it and liked it.

</comment></comment><comment><USER32> Lua is a beautiful language, I'm glad the switch worked out for you guys!

<comment><USER33> I really don't see the 'beautiful language' part. I tried LUA for the first time making a custom game mode for DOTA2 a while back (so admittedly not a huge amount of experience), and the language just seemed terrible to work with. Its focus on keeping the language simple meant that the standard library and language features are so bare-bones doing the most basic things was a huge chore.

<comment><USER16> Here are some things that I think are "beautiful" about Lua:

[a] Here is the complete syntax of Lua:

[http://www.lua.org/manual/5.2/manual.html#9](http://www.lua.org/manual/5.2/manual.html#9)

...compare that to Perl where there is no "complete syntax" of Perl. This shows how simple it is and thus it is easier to write tools for and teach others how to use it.

[b] The complete default API consists of ~200 functions:

[http://www.lua.org/manual/5.2/contents.html#index](http://www.lua.org/manual/5.2/contents.html#index)

[c] The defacto book on learning the complete language (including the C interface) is 366 pages:

[http://www.amazon.com/dp/859037985X](http://www.amazon.com/dp/859037985X)

Simplicity is beauty IMHO. I would expect any competent developer to get up to speed on the Lua language within a week, it would then take them another couple of weeks to get up to speed on the libraries we use with Lua.

The real power in Lua comes when you couple it with competent APIs that do the "real" work. The Distelli Agent uses libuv for the OS interface, libcrypto (part of openssl) for the crypto primitives, zlib for compression, LPeg for parsing, sqlite for an embedded DB, etc.

The job of the Lua code is simply to "tie" these core components together.

</comment></comment></comment><comment><USER34> From the article: "The Lua standard library uses the POSIX C library, making it portable; but that also means that you can't use sockets, it doesn't support signals or timers, and all OS calls block the thread. Using the POSIX C library is like going backwards in time."

I can't speak to the "Lua standard library" but I can say that I believe every single statement above about the POSIX C standard is incorrect as written.

POSIX Sockets: [http://man7.org/linux/man- pages/man2/socket.2.html](http://man7.org/linux/man-pages/man2/socket.2.html)

POSIX Signals: [http://man7.org/linux/man- pages/man7/signal.7.html](http://man7.org/linux/man-pages/man7/signal.7.html), [http://pubs.opengroup.org/onlinepubs/009695399/functions/sig...](http://pubs.opengroup.org/onlinepubs/009695399/functions/sigaction.html)

POSIX Timers: [http://www.2net.co.uk/tutorial/periodic_threads](http://www.2net.co.uk/tutorial/periodic_threads), [http://pubs.opengroup.org/onlinepubs/009695399/functions/tim...](http://pubs.opengroup.org/onlinepubs/009695399/functions/timer_create.html)

Nonblocking IO: [http://man7.org/linux/man- pages/man7/aio.7.html](http://man7.org/linux/man-pages/man7/aio.7.html)

That doesn't mean that the POSIX features are best in class or easy to use -- rather just that the features exist.

<comment><USER16> That was a mistake in the article. I should have said "ANSI C". I'll see about getting it corrected.

Thanks!

</comment></comment><comment><USER35> Wow, the comments here are amazingly bad. A bunch of people sneering at the author for having to support older versions of Python that his customers use. Suggesting that he bundle hundreds of megabytes of dependencies and force the users to install them. Can you give this guy a break?

</comment><comment><USER36> It's great to see people discussing the use of Lua for systems applications. It has a lot of advantages over many dynamic languages, as far as footprint, performance and deliverability, while offering a comparable development experience.

</comment><comment><USER37> My current job is heavy in Python and a previous job was heavy in Lua. It's hard to tell from the article, but it seems like this is an environment that is embedded in another executable. In that context, Lua is certainly lighter weight and luvi sounds like a fantastic extension to the language (lua scripts are generally single threaded).

That said, anyone who has written Lua knows that scaling is difficult and relies on really high quality code. I'd be wary about replacing python scripts with Lua, especially if the problem with python was users wanted more advanced features. Still, my use case isn't yours and I'm glad the migration went well.

</comment><comment><USER38> It is a bit off-topic, but last time I checked the unicode support in Lua was less than stellar.

Since that was a long time ago, can anyone chime in about the state of unicode support in Lua?

<comment><USER16> Lua 5.3 has a utf8 library standard now:

[http://www.lua.org/manual/5.3/manual.html#6.5](http://www.lua.org/manual/5.3/manual.html#6.5)

Full unicode support is rarely necessary in my experience. Lua "strings" are raw byte arrays. It is the job of the programmer to validate the byte arrays are in the character set of your choosing and doing appropriate conversions and validation when necessary.

<comment><USER39> I would rather like my language natively supports binary string, rather than unicode string.

One can always use library functions to process the binary string as unicode.

</comment></comment></comment><comment><USER40> Lua is great when you will write at most a few thousand lines of code, you can ship or control your interpreter binary _and_ the number of developers who will ever deal with the code is single-digit. Beyond that it is miserable. Hope that Distelli Agent remains within those constraints.

</comment><comment><USER41> i suspect it's more about lua being a really nice, embeddable scripting language. there's a reason game companies all* use it for content developer scripting languages.

language preferences aside, sometimes it's all about the right tool for the job.

* not literally _all_ of them, but lots and lots and lots.

</comment><comment><USER42> My company also uses RHEL5 quite a bit, so in order not to stay stuck on 2.4 I use anaconda: [https://www.continuum.io/downloads](https://www.continuum.io/downloads)

It's a life saver.

</comment><comment><USER43> Minor correction: with the exception of libdl (and its equivalent on various platforms), Lua uses only the C standard library, not even POSIX.</comment>


YC founder looking for tenant to sublet our 510sq ft. SF office for 4 mos - <AUTHOR>

About the unit:
- On 3rd floor of historic building @ 222 Columbus Ave at intersection of highly desirable North Beach / Jackson Square / Chinatown neighborhoods. - Several other startups & small businesses in the building. - Comfortable for 4 people, could accommodate up to 7 - Recently remodeled office - Solid hardwood floors, white walls - 3 separate entrance doors w/ deadbolts - Security door in hallway plus coded front door entry - 3 large windows - Window unit air conditioning - One large & one small room connected by inner door - Access to 2nd floor kitchen - Men's & women's bathrooms on every floor - Elevator - Great building, quiet neighbors - Lots of great coffee shops and lunch places nearby
RENT: $2028/mo for 4 months AVAILABLE: June 1 to September 30
I'm the founder of Little Details LLC, and am moving my business to the NY area in mid-June. After the release of our latest project, Patchmania (http://getpatchmania.com), we're taking the opportunity to do the move sooner rather than later. Our lease is up September 30th, which means we have a 4 month gap during which a suitable tenant could sublet our vacant offices. If you like the building, as I hope you will, any further extension beyond September 30th is definitely possible but would require you to sign your own lease with the landlord, which would likely be at a different monthly rent.
$2028 is extremely competitive for this area and the quality of the office space. The market prices have gone up considerably since we rented the unit, but we're offering the space at our current monthly rent which was set a couple of years ago.
Serious tenants should contact me at email@example.com. Will happily send pictures. Unit will be vacant upon move-in although there is the option for you to purchase some office equipment (desks, chairs etc.) from us should you need it (otherwise it will be moved out of the unit). <comment><USER2> we're interested - png me at email@example.com</comment>


Ask HN: How do you motivate yourself at a boring corporate job? - <AUTHOR> I work at a big software company that treats its workers well.  My product is well-respected in its niche, and I'm working on features that will make our customers happy.  Nonetheless, I find it almost impossible to give a crap about the code I'm writing.
This doesn't really matter when the work is easy; I can do it on autopilot.  But as soon as I encounter a tricky design problem, I want to give up because I don't care about the product enough to put in the effort.
I might switch jobs, even though I've only been here 6 months and would probably burn bridges by leaving.  But, short of that, do you have any tricks to motivate yourself in this type of situation? <comment><USER2> 1\. Appreciate the reliable paycheck and stability you're bringing to yourself or your family.

2\. Enjoy side projects.

3\. Learn new techniques/tools/languages/skills along the way.

4\. Seek more direct interaction with beneficiaries of your work.

5\. Take pride in getting shit done for any value of shit.

6\. Introduce new, advantageous tech to project.

7\. Request employer sponsored training.

8\. Represent employer in standards efforts.

9\. Request different assignment.

10\. Shift entirely or partially to management/sales/marketing/support/operations/research or another tech group.

11\. Mentor junior developers.

12\. Begin to plan to leave (for new a corporate job / consulting gig / startup).

<comment><USER3> Best advice. The common theme here is to focus on personal growth, whether learning new tech or taking on more responsibilities. Use this opportunity to make yourself more attractive for the _next_ job.

</comment></comment><comment><USER4> Don't make the job just about the code in front of you.

Use the opportunity you have right now to learn about working in a team (and possibly taking the lead/managing), managing customer requests and expectations, implementing business processes, etc.

All of this will be valuable whether you keep working at this company, another one or decide to work on your own ideas/dreams.

</comment><comment><USER5> I have a question related to this. Even though my primary role isn't development (I'm systems engineering), there is some programming work involved. If I pour a lot of effort into something good (say a tool kit that makes future code easier), it is very hard for me accept that the code is owned by the company I work for, and I can't use it on any future jobs. Kind of makes me want to do a so-so job, and save the good code for my personal projects. How do professional developers deal with this internal conflict, other than no caring about the code you write for work?

<comment><USER6> Here's what I do: Work at a company that encourages/allows you to do free software development. Then you only have to write proprietary code that's not generally-useful anyway; no conflicts!

</comment></comment><comment><USER7> I'm not even in a software company, but a too-big-to-fail financial company that continues to cut back and treat us as an evil expense with a bureaucracy to put governments to shame. I've been there for 15+ years and will continue to be there as long as they have me, for the simple reason of a paycheck for my family.

Try to get by with side projects - though that sucks right now as I want them to make more money - they at least are something that gives me some passion in developing.

</comment><comment><USER8> "... do you have any tricks to motivate yourself in this type of situation?"

If you're looking for tricks, make sure you find tricks that work for long- term. Otherwise after a few meetings/projects/days/weeks, you'll start wondering again.

"... because I don't care about the product enough to put in the effort."

Here is an alternate thought (call it a trick if you have to): What if you forced yourself to care? Unconditionally. Emotionally. With passion. Like you'd care for your kids or pets or dear ones? Fake it at first if you have to, and then slide into it. Do a mental hack.

Caring for real will make you vulnerable (which is a sign that you actually care). But if the company culture is decent (which it sounds like it is), you will soon start to stand out, more opportunities will come your way and it will be a virtuous cycle.

If after trying for a few months, this doesn't get you anywhere, you know it's very clearly time to leave. And this time, go do something you are sure you will care for.

<comment><USER9> How does one force themselves to care?

</comment></comment><comment><USER10> > as soon as I encounter a tricky design problem, I want to give up because I > don't care about the product enough to put in the effort.

Is this really about the product or do you just not want to do the hard work?

You need to have a side project. For at least an hour a day, do something that you want to do. It doesn't need to be programming but it needs to be something you're interested in and has a goal you can see yourself moving towards.

Also, try to do your job better. Whatever you're working on, spend 30 minutes per day reading books where you can use the knowledge you learn to do your job better.

People are meant to be continuous learners. When we're not learning we get bored and don't care. So a) do some learning of your choice outside of work and b) do some learning at work to get better at what you're doing now.

</comment><comment><USER11> If you're getting bored after 6 months, you might be better off chasing contracts than settling on a full-time job. It can be stressful sometimes, but it beats being a cog in the machine.

Perhaps if the company you were working for belonged to you, you might take more interest in the problems, no?

</comment><comment><USER12> Reminds me of a post I saw on Get Rich Slowly awhile back on ditching a dream job for a boring one. Interesting perspective. Emotional investment can be taxing in its own way. I started in a job with very interesting problems but lacked meaning, then moved to a job with much more meaning but with less interesting problems (that eventually turned into boring, frustrating, and stupid problems). I've since found one that's more balanced, with interesting problems that I care about to a moderate degree.

[http://www.getrichslowly.org/blog/2012/03/18/reader- story-i-...](http://www.getrichslowly.org/blog/2012/03/18/reader-story-i-quit- my-passion-and-took-a-boring-job/)

</comment><comment><USER13> Its always good to teach and share to reinforce your existing skills and discover new ones.

Like many said, mentoring or taking up/ suggesting new projects may be a plausible avenue. You could also internally create some challenges for yourself in relation to work, for example I may want to have codes in portable modular form so I can use for other projects etc.

There is no need to burn bridges even if the work is not fitting, especially when people are nice.

Ultimately, this is admittedly short term. Alignment of interests and incentives are critical in the long run, thats why often people are willing to forgo larger stable paychecks to head over to risky startups with low paychecks.

</comment><comment><USER14> If I were in your situation, I'd simply quit. It's not worth it for me to do something I'm not interested in. That's just me though.

For other kinds of jobs where boredom is frequent but where I at least have some free reign to do side projects and develop things for myself, I'll often spend a lot of time writing tools and scripts to automate tasks for myself and my team. That helps get rid of some of the dreariness.

</comment><comment><USER15> 1\. quit 2\. or moonlight

in general, it doesnt matter what i'm doing, but i'm going to put at least 100% into something, whether's it's something menial like raking leaves or having a service scale to a billion requests a day, and note that my 100% is going to be 120% for most. this is a behavior that needs to be nurtured, and quite frankly a bit of your integrity on the line here.

</comment><comment><USER16> I was in a similar position many years ago. I just translated algorithms from CSLR into real code and toyed with project euler problems.

Leaving is the best option. Never looked back.

</comment><comment><USER17> Slightly related: [https://news.ycombinator.com/item?id=7789014](https://news.ycombinator.com/item?id=7789014)

</comment><comment><USER18> why would they pay you any money if you had another motivation to do it?

<comment><USER19> The two are uncorrelated.

</comment></comment><comment><USER20> Do you have kids or anything like that? I would honestly just leave.

</comment><comment><USER21> Tao of Programming, section 7.1.

<comment><USER22> I think that the tree analogy and "the untroubled ease of programming beneath its sheltering branches" is not applicable in his case. Having been in a similar situation, my best guess is that his company's "tree" is called Java EE and in its shadow there is no trace of any academic or industrial advance from the past 10 years.

<comment><USER19> Then you haven't really absorbed the value of the prior posts.

</comment></comment></comment><comment><USER23> While I do agree with most commenters here, I would also like to provide something that has worked for me in the past, because in the end there can be many reasons for not being able/want to quit.

I suggest that you try to bring your inner engineer (child) out, by trying to make it fun to do some challenge or project from the only possible pieces you can have around, i.e. your work.

What I mean is, unlike Principal Skinner putting Bart Simpson to time how many envelopes he can lick per minute, try to see if any of the boring parts of your job could be automatized, or maybe you could ease your workflow by introducing some tooling of your own. You can even put it as a challenge to implement the tooling in a language you always wanted to learn, or using a tool someone recommended you, etc.

So basically you will turn your boring job into the excuse to do the actually fun project you like (the meta-project if you will).

My best personal example happened when working as a contractor for a bank, while they said I was going to be doing software development, it turned out I had to do "impact analysis" which meant running reports from a custom tool, literally comparing them by hand, and then fill in an ITIL-process form. Every X hours, every day, for a year and a half. Non-stop. I had to run reports for every software change in the systems... and then I had to email by hand (i.e. there was no template, no auto-email system, etc) when I found certain things in the reports. This meant I was sending up to about, 10-15 emails a day because every step was done by hand and thus slow.

Now, remember that I was contracting for a bank, so basically I could not do anything to my computer. Could not install anything, it was firewalled to death, cannot plug anything, etc. Then I used this computer to remote into the bank's terminal which was under similar circumstances.

I ended up building a very Frankenstein-y "system" based on VBA (the only thing installed was Office suite), which basically saved me the hassle of extracting the relevant information from the Excel reports, then composing the email by hand and also keeping track of what info was in which report, which ones were sent to which departments, etc.

I remember how much fun I had when I fired my monstrosity to send over 100 emails, all the emails for that day, in like the first hour of the working day. Then I got like 5 or 6 IM windows open because as my luck would have it, most of the people I had sent those emails were in a meeting room together...

Imagine having a table with 20+ people around and about 10 of them getting email after email... the buzzing from their cellphones continued for like 30 secs to a minute. Of course everyone was wondering what happened until I told my boss "Look! I have this new tool that automates most of my work!"

Of course I could now send more than 200 emails in a day, as I was only limited by my speed to run the reports...

Then I got my boss to approve some QA tools to be installed so I could automate running reports... I was about to achieve close to full automation for my job when my boss realized his team of 7 people would basically be put out of work thanks to my tool... and then she realized how dependent they became on it (by this time everyone was already using it) and how was this not good for her.

Long story short(er), she started preventing me to work on that tool anymore, and that's when I finally said it's too much. I mean, this is a boring job I managed to make it fun, real fun (and make accomplishments along the way) and this was clearly threatening to my boss, so I finally left.

tl;dr My point is, if you absolutely must stay there, try to find a way to bring out your passion for engineering/development/whatever out to play, even if the only toys available are those provided by your working environment. I bet you can always find something that can turn into a fun little side project. Especially if you only need 6 months to a year until you can switch jobs without burning any bridges.

Edit: The formatting is horrible... how can I better make it into paragraphs?</comment>


An opinionated guide to Scheme implementations (2013) - <AUTHOR> http://wingolog.org/archives/2013/01/07/an-opinionated-guide-to-scheme-implementations <comment><USER2> This is a really great little guide for people that want to try out a Scheme but are paralyzed by the options. This will help those in need find a starting point. I highly recommend GNU Guile, the implementation that the author co- maintains.

<comment><USER3> Guile and its entire library ecosystem is GPL infected. I'd choose to pay the Chez scheme guy before having to worry about copyleft implications.

<comment><USER4> Chez Scheme is not currently available for purchase. This has been the case since it was aquired by Cicso Systems.

<comment><USER5> Yikes! That would explain why the language's site is still Copyright 2011 Cadence Research Systems.

Apparently Cisco bought Cadence, and Chez Scheme fell through the cracks.

Ugh. As far as I know, perhaps not counting the 2 JVM Schemes, it was the only one that provided OS level non-green threads.

<comment><USER6> I've never used it myself, but fwiw Gauche also advertises support for native threads. SRFI-18 threads are wrapped around pthreads on Unixy platforms, and Windows threads on Windows. [http://practical-scheme.net/gauche/man/gauche- refe_106.html](http://practical-scheme.net/gauche/man/gauche-refe_106.html)

<comment><USER5> Ah, I was not clear enough: native SMP threads, as in each thread accessing the same shared memory heap. Gauche allocates a new VM for each thread ... ah, but I see it has data synchronization primitives. I believe it has moved forward since the last time I checked it, years ago.

Thanks for the tip!

Repeated from the other reply on Guile: Hmmm, now I'll have to narrow the statement to "The only native code compiling Scheme that I know of" (which is harder to do).

</comment></comment></comment></comment></comment></comment><comment><USER7> What about implementing your own Scheme or Lisp if you want to embed an interpreter for your C/C++ project (the first scenario discussed in the blog)? I recently decided to roll my own Prolog interpreter for my project. With C++11 and shared pointers it is really a breeze (of course I've thought about doing this for a long time so the actual implementation part was smooth sailing but not the only story). The advantage of rolling my own:

1\. No GC. Ref counting is sufficient for my own use.

2\. Integrating with the C/C++ libraries I want to use is a lot easier. I designed my implementation for the purpose of embedding in my project and gluing libraries.

<comment><USER8> > What about implementing your own Scheme or Lisp if you want to embed an > interpreter for your C/C++ project (the first scenario discussed in the > blog)?

I've done this for a Java project many years ago, where it sort of grew out of a config file format, as so many Lisps do. I would probably take the trouble to add a standard embeddable interpreter if I did it again. Otherwise, you end up with yet another 80%-ass Scheme with its own quirks and no libraries, and the world has too many of those already.

<comment><USER7> "no libraries"

I forgot to emphasize that this is not a real problem. You will likely have good C/C++ libraries for most things. The real issue really is to expose these libraries to your embedded language on a as-needed basis. To me the effort is better spent in creating my own targeted solution that makes interfacing easier and that is under my control than learning the minutia of the FFI of a third party implementation, after taking into account the following:

1) we are talking about effort on the scale of one or a few KLOC to roll one's own, and comparing that against the effort needed to truly master the FFI of a third party implementation;

2) my interface is no longer limited to the lowest denominator that a standard implementer would face (e.g. most use C but I can use multiple inheritance and templates of C++ to make the interfacing a lot easier);

3) supporting FFI users is typically not the highest priority on the standard implementers' agenda (if there is any support at all).

<comment><USER8> > You will likely have good C/C++ libraries for most things.

For most things in your domain. Then someone wants regexps, or Windows path handling, or what-not, and you find that you didn't really implement Scheme macros, or some particular style of modules, so you lose.

I agree that if you want C++ integration, you're probably going to want to roll your own. The first person to write an easy FFI that groks C++ name mangling will do a tremendous service to mankind.

<comment><USER9> There's no "someone" in this context, because you're not designing a language for the masses, you're making your personal language. At best, if you put it on Github you'll maybe find a bunch of people wanting the exact same thing so you'll have a bunch of users&co-implementors that know that they're basically on their own if they choose to use your language. This is where small and simple languages such as Scheme, Forth or Lua shine: you can master and hack everything from the first byte to the last.

The real issue with this approach is that, if you need a particular library you have to do the bindings yourself first. You have to deal with the delay required to design, implement and test it. On the other hand, the difference between learning the API for an another popular language and learning the native API so you can make the bindings for your language tends to zero as the complexity and size of said library goes up.

</comment></comment></comment></comment></comment><comment><USER10> I would hesitate to call DrRacket "excellent".

<comment><USER11> What specifically are your reservations? - I haven't played around with many Scheme implementations besides DrScheme/DrRacket so I'm not questioning your judgement, I'm just genuinely curious.

<comment><USER12> To expand a bit on "bloat," DrRacket does use a lot of memory for a text editor (don't have it installed ATM but I remember 100s of megabytes of memory in use in a typical light session of 1 file + 1 REPL), crashes somewhat frequently (a couple times in my test run, usually when trying out some of the crazier features hidden in the menus), and becomes less than responsive due to _very_ frequent GC pauses (that are helpfully indicated by a little recycling icon in the corner of the window).

I still think it's a far superior way to introduce newbies to Scheme than emacs, but it's definitely flawed.

<comment><USER13> It comes with 'batteries included', and the defaults work fine on most modern machines.

If you are really resource constrained it's easy to go to the settings and restrict memory use, and turn off tools you don't intend to use. If you have a really slow machine there is even a minimal DrRacket.

<comment><USER12> My Ivy Bridge i7 with 16 gigs of RAM is not exactly resource constrained. I'll take your word for it that you can disable features to get the resource usage down, but it's kinda sad that the default config can't run without stuttering on a machine like that, especially since the program is intended to be used in _education_ where the average machine is _significantly_ less powerful than that.</comment></comment></comment></comment></comment>


Ninja Threat Modeling - <AUTHOR> http://chargen.matasano.com/chargen/2009/10/20/ninja-threat-modeling.html <comment><USER2> So then here's the deal. In the top 20th percentile of "secure" enterprise dev shops, with teams working on J2EE applications for claims processing and document retention, each dev cycle starts with a "threat model".

Threat modeling is part of the "SDL" (or "SDLC"), which is a powerful voodoo magic word that here means "secure software development". Threat modeling is both notoriously fuzzy --- nobody really knows exactly what it means --- and notoriously boring. The goal of the process is to help ensure that you're designing securely, developing securely, and dtesting securely (I forget what the 3rd of the 3 D's is).

Cory's talking about a methodology for getting teams to do threat modeling that either haven't totally bought into formal SDL development or skip the threat modeling step.

I'm not totally sure how this applies to startups and indies [1], but I am sold on the concept of "Seven Deadly Features", which is to say that any team should be able to start a project with a list of the 7 functional areas that are most likely to screw them over from a security perspective. A very typical list for web apps would be:

* File upload/download (it's a boundary crossing and often requires you to synchronize two different access control systems)

* Email (most apps have some kind of effectively anonymous account and some path for that account to trigger emails with some content controlled by attackers)

* Advanced Query Interfaces (because they're hard to implement using pure parameterized queries)

* Templating and look/feel customization (because you need to expose some, but not all, of HTMLs metachars to end users)

* User signup and password reset

(That's 5; the list on my desk is 7, but the last two --- SOAP/web services and thick clients --- are not likely to be problems for startups).

Can't evangelize enough for this idea, which is a really simple one. People who do web security memorize things like the OWASP Top 10, which lists "Cross Site Scripting" and "Cross Site Request Forgery" and "SQL Injection", but like Cory says, nobody wakes up in the morning thinking "I'll write an SQL Injection today!". Meanwhile, if you know what the scary features are, you can:

* Schedule them for extra-formal team review

* Double up the QA time on them

* Not assign them to junior team members to implement

[1] We wrote about this a few weeks ago:

[http://chargen.matasano.com/chargen/2009/9/24/indie- software...](http://chargen.matasano.com/chargen/2009/9/24/indie-software- security-a-12-step-program.html)

<comment><USER3> _Meanwhile, if you know what the scary features are, you can:_

It's worth adding another point to that list:

* Not implement them.

Some features just aren't worth the added complexity they would bring with them.

<comment><USER2> I used to say that, and every time I did, people would yell at me, so I've been conditioned not to.

Although I'll stick my neck out and say "crypto; don't even bother".

<comment><USER3> _crypto; don't even bother_

I'm supposed to yell at you now, right? :-)

In all seriousness, cryptography doesn't really fall into the "complexity which might introduce security flaws" category: Most cryptographic code is extremely simple. Yes, it's easy to get cryptography wrong; but the ways which people get cryptography wrong make it _useless_ , not _harmful_. It's very difficult to write an HMAC with a buffer overflow.

<comment><USER2> You still think that people know what HMAC is. Most people think HMAC means "hash a key in with the message".

<comment><USER3> My point remains even if people do get confused about that: Their crypto might be broken and useless, but it still won't be _harmful_.

<comment><USER2> Dude. You just saw what happened to Flickr. Stop being argumentative.

<comment><USER3> What happened to Flickr would have happened even faster if they hadn't used crypto at all. Their crypto was broken, but didn't make things worse than they would have been in the absence of crypto.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER4> An excellent summary; a little vague on details. If all server code can be compromised, and all client code, and the man-in-the-miiddle, what are you left to test?

<comment><USER5> Thanks for the feedback!

The assumption is that you should consider your source code open and exposed to inspection by an attacker, not that it has been compromised. As a result, if any security control is dependent on "secret" functions or embedded keys in your source, the threat actor is going to know about them and will attempt to use them against you.

As a result, the test plan will need to take that into account.

</comment></comment><comment><USER6> If there were more people writing this entertainingly and concisely about security testing, there would be a lot more CISAs.</comment>


Alphabet Says It Shut Down Titan Drone Internet Project - <AUTHOR> https://www.bloomberg.com/news/articles/2017-01-11/alphabet-says-it-shut-down-titan-drone-internet-project?cmpid=socialflow-twitter-business&utm_content=business&utm_campaign=socialflow-organic&utm_source=twitter&utm_medium=social <comment><USER2> A few nights ago, over a liberal quantity of beers, my friends and I came up with our latest nonsensical conspiracy theory. Google was started by some of a group of time travelers who were sent back in time from an apocalyptic future bringing advanced technology with them. They would use _just_ enough of the tech to make vast sums of money, then use that money to invest in the tech that destroyed their future world.

By investing in tech they already know would work, they would vacuum up all the resources and talent around that technology so that nobody else would bother (because who could come in cold and compete against a titan like google?) and then some time later kill all of the projects very publicly in order to make us all think that these technologies were nonviable.

Except instead with every piece of world changing technology they killed, they were buying their future Earths freedom from the apocalypse they were fleeing from, even if that future they were creating destroyed their own and left humanity in a just post-agrarian stage of development.

This group has been infiltrating our timeline for decades, first with blimps, then Atari, then Xerox, now Google.

Their motto is "save us before we destroy us"

<comment><USER3> Well the obvious time paradox would prevent them from taking this approach. By destroying their own future they would never come back to save it.

There must be another explanation.

<comment><USER4> Just assume that every jump back in time is also a jump into a parallel universe and you don't have a paradox.

<comment><USER5> Then what would it help to do such a thing?

<comment><USER6> Experience is subjective. If you teleport to another location you're not the same person you were before you left, you're a similar set of molecules reconstructed in the same fashion. In all actuality, "you" died and another "you" took your place. But to you and everyone around you, it seems like nothing has changed, so you carry on with your life.

Likewise, if you travel back in time and change the future, you're not helping the people in the future you left, you're helping the people in the past avoid that future. But to you, it looks the same. It looks just like the past you're familiar with, and it develops into some semblance of the future you were expecting. So for all intents and purposes, from a subjective viewpoint you're avoiding the paradox and helping your ancestors. It doesn't matter if the people you left are still suffering, subjectively they don't exist anymore.

<comment><USER7> In the Star Trek universe, where the matter is converted to information, transmitted and then converted to matter, then your statement may be true. But, in other universes where the matter is actually translated in a dimension then it will not be. Also, we don't actually know what "you" is and whether it would be included in the Star Trek mechanism.

My point really, is that there are varying fictional universes but in ours, we don't currently have a way to do teleportation. So, as you say, experience is subjective.

<comment><USER6> Ah I forget that I'm back in the timeline that doesn't have physical teleportation... I have to start keeping notes on which universe has what.

<comment><USER8> you're in that superset of worldlines that require an Ellis Drainhole held open with a counter-rotating exotic matter "collar".

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> This article has a few details that are very different than the way I heard it. What follows are rumors, but I work in this industry so I have a few sources that I trust.

Google did not win the bidding over Facebook. Facebook actually won the bidding with a bid of about 60 million dollars. Facebook pulled out during the due diligence phase. Google swooped in and picked it up for about 20 million after Facebook pulled out. (Rumors remember, I can't find sources for these numbers right now)

Internally, Google was very quickly disappointed in their purchase. They found out that they had pretty much bought a nice PowerPoint slide deck and not much else. Titan still had a _long_ way to go before they had a viable drone.

Anyway, that is the rumor that I heard.

I'm only somewhat envious of the drone companies that are selling out for tens of millions of dollars, while I sit here and work my butt off to keep making a product. [https://youtu.be/b7SjOOuTct0](https://youtu.be/b7SjOOuTct0)

<comment><USER10> As an ex-Xer, I can tell you that the PowerPoint presentation wasn't even that good.

</comment></comment><comment><USER11> At this rate people will soon use "alphabet" as a verb the same way they made "google" into a verb:

 _" we were doing steady progress but 3 months before release the management decided to alphabet the project and send everyone home."_

<comment><USER12> That'd be great because "google" as a verb already has other meanings ("search with a search engine").

<comment><USER11> Isn't there a legal name for this (i.e. company name/product becoming a common word)?

I think it has legal implication for a trademark, but in the case of Alphabet it was already a common word so it probably has no meaning.

<comment><USER13> You could be thinking of [https://en.m.wikipedia.org/wiki/Generic_trademark](https://en.m.wikipedia.org/wiki/Generic_trademark) Or the related term "genericide"

<comment><USER11> Thats is, thanks!

</comment></comment></comment></comment></comment><comment><USER14> I think what's sad is not that they shut down the project soon after acquiring Titan Aerospace, but that they kept it quiet for almost a year. That sort of PR practice makes me worry about their internal culture.

Edit: It concerns me because I wonder how many other projects they've announced and shut down without telling us, and what that must be like for the employees who are no doubt under NDA who are shuffled around and unable to talk about it. Imagine if your company were acquired by X, and then you get fired, and you can't even tell your friends or potential employers it's because X axed the project.

<comment><USER15> Companies don't typically announce they've shut down a generally unannounced project. I consider Titan one.

<comment><USER16> On the contrary. Blizzard cancelled a project also called Titan which they had never actually publicly announced.

[https://en.wikipedia.org/wiki/Titan_(Blizzard_Entertainment_...](https://en.wikipedia.org/wiki/Titan_\(Blizzard_Entertainment_project\))

<comment><USER6> Also Microsoft's unreleased booklet PC, Microsoft Courier. Never officially announced as anything other than a concept, but they did offer an official notice that the project was canceled.

</comment></comment></comment></comment><comment><USER17> Apple's car

Blizzard's game

Google's drone

Lesson: if you want your project to be cancelled, call it Titan.

<comment><USER18> TitanDB - today forked to become JanusGraph [https://blog.grakn.ai/janus- launch-22be6ac3b197#.q9pme0jfd](https://blog.grakn.ai/janus- launch-22be6ac3b197#.q9pme0jfd)

</comment></comment><comment><USER19> Interesting. Though I wish someone pursued this particular tech (solar-powered drones) further - high-altitude, long-duration drones could be a good replacement for many things we use _satellites_ for, and drones would be _much_ cheaper to launch...

<comment><USER20> There are _lots_ of companies working on light solar-powered drones, and in fact some of them call them pseudo-satellites. Airbus, Boeing, NASA, DARPA... the list goes on and on. This really isn't a setback for the field in general.

</comment></comment><comment><USER21> Google starting to resemble Xerox

<comment><USER22> And yet they still have a huge money faucet.

</comment></comment><comment><USER23> my guess is they realized a LEO constellation is simply a better idea. google invested $1B in spacex after all.

<comment><USER19> That's what I don't understand. If they pushed this technology further (high- altitude, solar powered drones) it seems it could take over a big chunk of satellite industry. If you don't _actually_ need space, then solar drones seem like so much better option - significantly cheaper to make, significantly cheaper to launch, etc.

<comment><USER24> Maybe it's easier to keep things orbiting in space than floating in the air? The earth atmosphere is not as predictable.

<comment><USER19> It's easier to keep things in space, but it's much more difficult to _get_ them there because you have to use a huge rocket to make the thing go fast enough to _stay_ on orbit. If you don't need that speed and the ability to circle the whole globe, high-altitude planes seem like much cheaper options.

<comment><USER25> You'd also need a much larger number of the solar powered drones to cover the same area as a satellite. They don't make it clear but I'd guess one of the main issues was getting the radio and networking gear light, small, and low power enough to work in their drone design with enough throughput to make the network vaguely acceptable while covering a large enough area to make the number of drones required reasonable.

<comment><USER19> OTOH having to have more drones gives you benefits of economies of scale when you move past testing phase; you can mass-produce a platform and drop the price significantly. Satellites, on the other hand, are mostly one-of-a-kind constructions, produced in single or double-digit numbers.

(Also with a fleet of drones, failures are cheaper and less dangerous.)

There are lots of factors affecting economics of both solutions. I guess one of these days I'll have to make a spreadsheet with back-of-the-envelope calculations to sort this all out, but my gut feeling for now is that solar drones are a viable replacement for a lot of current satellite uses.

<comment><USER25> Making more would help but there's still limits on how far down it could possibly drive down the price on this large complex drone.

To me the largest issue is simply power, these drones need to store a lot of energy just to fly overnight. Add in the power requirements for all the radio gear, it'll need to operate overnight too to make the service particularly useful, and battery weight can quickly spiral outside of what can reasonably be done by a small company trying to build a small plane. There's also 2 hard competing design goals solar panel area and aerodynamics. The first wants broad flat surfaces to place as many panels as possible on while the second is pushing you towards long thing wings and bodies to reduce drag.

I'm not saying these are impossible things to build but I do think it's more in the realm of large aeronautic companies rather than a small division of google.

</comment></comment></comment></comment></comment></comment></comment><comment><USER26> They're leaving the market to Airbus Zephyr, which was in the lead already?

[https://airbusdefenceandspace.com/our-portfolio/military- air...](https://airbusdefenceandspace.com/our-portfolio/military- aircraft/uav/zephyr/)

</comment><comment><USER27> Free internet for all remains a dream forever I guess!sad indeed!

<comment><USER28> SpaceX's cubesat project might get it done.

<comment><USER29> My impression is that Titan and Loon are meant to work with phones, whereas a receiver for SpaceX's thing would be the size of a pizza box. No way it's gonna be free.

[https://www.youtube.com/watch?v=hJD0MMP4nkM&feature=youtu.be...](https://www.youtube.com/watch?v=hJD0MMP4nkM&feature=youtu.be&t=50m5s)

</comment></comment></comment><comment><USER30> I guess Project Loon would live on

</comment><comment><USER31> Never hire executives from Wall Street. Not even once.

<comment><USER32> As Wilde had it: "a man who knows the price of everything and the value of nothing."

</comment></comment><comment><USER33> Much like aircraft-delivered nuclear weapons, are these being quickly replaced by impending low orbit micro-satellite constellations under development now?

</comment><comment><USER34> Oh yeah, Loon is going to be so much more feasible. Loon makes the spatial economics of providing internet so much worse since it has to cover the 3/4 of the world that is ocean plus all of the places that are deeply uninhabited.

To get fiber into rural places you just have to parallel the electric lines. Except there is something ritually unclean about wires and we can't do it.

<comment><USER35> Why would you need the balloons flying over the ocean? They can control their longitude.

Plenty of rural places in third-world countries without an electric grid too.

</comment></comment><comment><USER36> Cached version to counter HN effect: [http://webcache.googleusercontent.com/search?q=cache:dtWSiWi...](http://webcache.googleusercontent.com/search?q=cache:dtWSiWi8N-AJ:pascaleggert.de/macpro.html+&cd=1&hl=en&ct=clnk&gl=za)

</comment><comment><USER37> So the Titan Project got Alphabit?

</comment><comment><USER38> I can assure you that a time travel longer than 0.00125 seconds into the past is not possible for the near future.</comment>


Making Sense of Standard Deviation - <AUTHOR> http://amarsagoo.blogspot.com/2007/09/making-sense-of-standard-deviation.html <comment><USER2> A story of my last use of Standard Deviation.

My day to day job deals with credit card processing. A lot of my job deals with ensuring that transactions occur securely and reliably. It's a tedious job, it's not exciting, and involves a lot of testing, but I love the sense of knowing that actual money is flowing through a system I built.

Anyways, one of the things I wanted to do was build an automate alert system that would notify me of problems with processing transactions. Looked at it from high up, the system is fairly stable, and while it would be easy to notice if transactions suddenly stopped on the entire system, this rarely, if ever, actually happens (and hasn't happened except for planned < 1 minute outages).

However, consider the system runs through it many small individual sites, looking at all transactions is fairly useless. Instead, I wanted a warning to notify me when any specific account was suffering. Each account is different, and accounts for a variable number of transactions each day. Some accounts do more, some less. Their are other variables: some accounts do well at different types of the day because of where they are promoted in the world. Weekends generally see an uptick, but this again is variable.

So, I developed a system (using standard deviation) that essentially looked at an accounts history for the past X time period for a certain period of time throughout the day. Some accounts are inspected by looking at the numbers in the past hour (accounts with steady transactions), others are looked at over the last few hours, others are looked at over the day.

Obviously, we don't alert ourselves to certain cases that fall outside standard deviation, and we've adjusted the numbers to look at other areas, but the result of using standard deviation in this way suddenly opened up a new way at looking at our numbers and evaluating the current status of our system, as well as the accounts using it. Even if a problem doesn't exist on our end, we can alert the people that handle these accounts that a problem might exist, allowing them to take necessary action.

Understanding standard deviation, understanding how it can be used (along with other associated tools of math) makes for some really interesting things that you can do to improve your system as a whole.

<comment><USER3> A similar system was put in place by a pharmacy system to detect employee narcotics abuse. They'd throw an alarm when a pharmacist started dispensing pills at a rate outside their SD.

The system worked great except in the case where a pharmacist slowly ramped up usage as his addiction worsened. Seems like that's a feature that works well in your case. As a business sells more stuff, their transactions will ramp up and your model will still hold.

nice.

<comment><USER2> That's one of the cases we don't alert ourselves too: getting a lot of transactions is a good thing. =)

In the reverse, however, is another issue entirely. A site that is slowly getting less traffic won't be alerted. However, this is where we have to draw the line. We aren't looking for long term trends. Rather, immediate issues. Basically, if a site that normally has 20 attempted transactions during the last hour suddenly has 0, is this a problem?

There is still a lot of tweaking that needs to go into the system. We're also exploring how to use this in other areas as well. Since all these accounts are internal, it's easy for me to get feedback.

The end goal, however, is to be notified as soon as possible when something occurs that will negatively affect our bottom line.

</comment></comment></comment><comment><USER4> A small but vital correction is that the "biased" estimator will approach a population's std dev as the sample size increases. This is clear since the adjustment is N/(N-1) which tends to 1 with large N.

It's not often a huge deal, but I'm unconvinced that biased estimators deserve such a derogatory name. Using a unbiased estimate does not actually mean you've got a better estimator. I think this is part of why there seems to be no intuitive description of why the N/(N-1) factor removes bias. You instead need to invoke the powerful, abstract ideas of sufficiency and completeness.

<comment><USER5> The "intuitive" explanation I've heard is that (N-1) is the number of degrees of freedom of a sample with N data points, so when you divide by (N-1) you're normalizing some valuation of the "spread" of the sample against a measure of the _potential_ for the sample to "spread."

I think how "intuitive" this idea is depends on how well you understand thermodynamics. I, clearly, do not.

<comment><USER4> I've heard that description before and find it more ad hoc than intuitive. Is there some valid mathematical object K such that often times you will normalize unbiased estimators by N/(N-K)? Is this K sometimes equal to some measure of the number of equations/constraints you have? Does this seem to hold a lot of force in ANOVA especially?

Sure, sure, sure. I still don't see why this normalization is required to make a _better_ estimate (excepting when you define "better" as an unbiased estimator, maybe the MVUE).

</comment></comment></comment><comment><USER6> I don't have a lot of practical day-to-day use for standard deviation and possess a vague understanding of it. Like the author, I resort to Wikipedia periodically and go away with no stronger an intuitive sense than when I began. I think the author offers a great explanation for the advantages of a more intuitive understanding. This article and its simple graphics definitely provided a more intuitive sense of the concept.

I'll wait a while for the statistics experts to weigh in here before I write it to my cognitive hard disk. :)

</comment><comment><USER7> I explain it like this:

It's 3rd and 4. You have two running backs. Running back A averages 8 yards a carry with a standard deviation of 6. Running back B averages 5 yards a carry with a standard deviation of half a yard.

Which guy do you choose? If you just look at averages, you probably would choose A. But if you think of the standard deviation as an indication of consistency you choose B, because his 5 yard gains are very consistent, whereas A might bust one for big yards, but he's also likely to get dropped for a loss.

<comment><USER2> Ahh yes, but it's not that simple. You can also account for the plays these are on - some people crack under pressure. If you just add up the numbers without accounting for other conditions, results can change dramatically.

This isn't to say your example is wrong, but simply to say that all data is not equal and to make sure you use the right data for whatever you're trying to accomplish.

</comment></comment><comment><USER8> The main reason to use standard deviation instead of mean absolute deviation is that

      sqrt((f(x)-g(x))^2)

is differentiable, while

      |f(x)-g(x)|

is not. Though exaggerating the larger differences is often a desirable property, the ability to do calculus is nearly always a desirable property.

<comment><USER9> You point out a real problem with the article. The given rationale for (.)^2 vs. |.| is just wacko ("there is obviously more variation in the second data set than the first").

The reason it's mistaken is, there are other examples, typically with just one or a handful of outliers, in which the SD is "obviously" too high compared to mean absolute deviation.

Futhermore, these type of examples are common in practice, so the SD is problematic to use if you have outliers.

Often, in robust statistics, the _Median_ Absolute Deviation (which is conventionally labeled MAD, <http://en.wikipedia.org/wiki/Median_absolute_deviation>) is used. As an estimate of scale, it's much more robust to outliers. But of course, you can't differentiate it!

</comment></comment><comment><USER10> The author makes a minor mistake.

You can make an unbiased estimate of the _variance_ using the formula s^2 = (sum of squared deviations) / (N-1). However, taking the square root of this gives you a biased estimate of the _standard deviation_ due to the concavity of sqrt.

<http://en.wikipedia.org/wiki/Bessels_correction>

<comment><USER11> Heh. Of course being a non-linear transformation, bias will creep in if you just naively take the non-biased value, but people rarely think about that. Thanks for pointing this out.

(BTW, the link needs an apostrophe: <http://en.wikipedia.org/wiki/Bessel%27s_correction> )

</comment></comment><comment><USER12> Can think of standard deviation much like a _distance_.

So, if have random variables X and Y, if they have finite means and standard deviations, and if we denote standard deviation by Std(X) and variance by Var(X) and set Z = X + Y, then Var(X) = Std(X)^2 and we get the Pythagorean theorem

Var(Z) = Var(X + Y) = Var(X) + Var(Y)

if we can assume that X and Y are _uncorrelated_ which is true if X and Y are independent. So, uncorrelated is analogous to perpendicular in geometry.</comment>


Goodbye Silverlight - <AUTHOR> http://blog.smartbear.com/software-quality/bid/289660/Goodbye-Silverlight

<comment><USER2> _> To get the same quality and security in HTML5, there is a need to implement video extensions into the players via the use of DRM encryption and JavaScript to the Netflix servers._

This is completely disingenuous. DRM has nothing to do with security, and it is for sure not required to get the same quality.

On the contrary, DRM always compromises security and always reduces the quality of the content (in a sense of crippled usability).

<comment><USER3> I agree that DRM compromises usability. However, I don't think the author meant that DRM would needs to be there in order to help out the quality.

<comment><USER2> Usability should be viewed as a quality aspect. Reduced usability means low quality product.

<comment><USER3> Agreed. I think the author should have said "In order to satiate content owners' demands, developers will need to implement DRM. The quality of HTML5 streaming still needs work."

<comment><USER4> Thank you. You are correct that I should have elaborated here. There definitely seems to be some confusion on my stance to DRM from the way I stated it.

</comment></comment></comment></comment></comment><comment><USER5> > Despite that issue, we want our videos streamed in high quality and securely

Based on the previous paragraph (about getting Netflix to almost work on Linux), I'm guessing that the "we" here is consumers of video, not producers or distributors. In which case, no, "we" do not give a rat's ass about whether our videos are streamed "securely."

"We" want to watch what we want to watch when we want to watch it. Content producers insisted on "security" and Netflix chose Silverlight as a short-term technical solution to that problem. No one else wanted it. We just tolerated it.

</comment><comment><USER6> It seems hard to say goodbye when the thing that is supposedly leaving is going to hang around for another 7+ years.

</comment><comment><USER7> Not sure why this is news now; Microsoft announced this back when Silverlight 5 was first supported back in December of 2011.

[http://www.linkedin.com/groups/Microsoft-support- Silverlight...](http://www.linkedin.com/groups/Microsoft-support- Silverlight-5-until-124074.S.84758245)

      > it actually says 2021 or the support lifecycle of each of the supported
      > browsers, whichever is shorter. When IE 9 is no longer supported you can
      > pretty much consider Silverlight unsupported as well.

... though technically there are +'s next to the alternative browsers (Firefox 3.6+, Safari 4+, Chrome 12+) so there's plenty of wiggle room left for interpretation.

<comment><USER8> The reason it's being talked about is because Netflix's recent transition plan is a milestone--the only Silverlight application I knowingly use is finally going away.

</comment></comment><comment><USER9> _Many window users faced Silverlight quality issues themselves due to their .Net frameworks not being up to date. For the best performance, they needed to download the .Net 4.0 SDK in Windows, which is something the average Netflix customer would not be willing to do._

Silverlight doesn't require the .Net framework on the machine. Not sure what performance issues he is talking about but Silverlight has better streaming features and performance than flash or html5.

</comment><comment><USER10> And good riddance..

<comment><AUTHOR> Indeed.</comment></comment>


Are Newer Games Excluding the Disabled? - <AUTHOR> http://www.lockergnome.com/news/2012/04/11/gaming-with-disabilities/

<comment><USER2> The Kinect might make a game unplayable to someone who can't move their body, but it opens up a world of gaming to people who don't have hands. Why do these able-handed people want games to continue to discriminate against the handless?

</comment><comment><USER3> Motion-focused platforms definitely open a whole can of worms in terms of making them accessible to everyone.

It will be interesting to see how companies evolve interface design further to adapt.</comment>


Ask HN: How can developers learn business? - <AUTHOR>

There are tons of resources for people who want to learn programming and web development but where and how can we learn how to do management, marketing, accounting and how to validate startup ideas? <comment><USER2> "So why were we afraid? We felt we were good at programming, but we lacked confidence in our ability to do a mysterious, undifferentiated thing we called "business." In fact there is no such thing as "business." There's selling, promotion, figuring out what people want, deciding how much to charge, customer support, paying your bills, getting customers to pay you, getting incorporated, raising money, and so on. And the combination is not as hard as it seems, because some tasks (like raising money and getting incorporated) are an O(1) pain in the ass, whether you're big or small, and others (like selling and promotion) depend more on energy and imagination than any kind of special training."

\- pg

</comment><comment><USER3> I recommend University of Michigan's finance course on Coursera: [https://www.coursera.org/course/introfinance](https://www.coursera.org/course/introfinance) There's also a Business Foundations series from Wharton [https://www.coursera.org/specialization/whartonfoundations/3...](https://www.coursera.org/specialization/whartonfoundations/38) (I followed a previous version of some of the courses).

</comment><comment><USER4> A good place to start, read Josh Kaufman's book The Personal MBA.

Here's his presentation @ Google> [https://www.youtube.com/watch?v=fp8H8EMm464](https://www.youtube.com/watch?v=fp8H8EMm464)

</comment><comment><USER5> There are so many sides of business that it is hard to say learn "business". It isn't unlike technology in that regard, as people specialize in specific areas to get the most of their skills, even though many of those skills are similar.

Marketing, Accounting, Sales, Business Development, Leadership, Public Relations, Customer Service, etc. All these are different, some seem overly similar and share common skills but there are subtle nuances that make them all unique. Small businesses a lot of times collapse Marketing & Sales and don't have a PR or BD team at all. Small to Mid-sized businesses will combine Marketing and Business Development many times as the skills overlap keeps it cost effective. Large enterprises have the luxury of knowing (and the ability to afford) that while the skills may overlap, you can't simultaneously be good at it all, so they break it up.

IMO, the best way to learn "business" is to recognize you can't do it all and be good from day 1. So pick 1-2 to work on, read and study. I personally suggest Sales and Marketing because without those you have nothing to Account for, employees to lead or customers to service. Read Crossing the Chasm (3rd edition), it is a great book on getting your product from the early adopters into the mainstream purchase cycle. It is also a quick read overall. As for Accounting, outsource it, when you are small it is cheap to outsource and you don't need the headache of jacking it up come tax time. But when you outsource it, spend time every week or month with the person doing the work and learn from them what they are doing and more importantly why. This lets you stay focused but still learn the basics of Accounting.

As for validating startup ideas. There are lots of methods, you should check out patio11's blog write ups as he talks about it when he started Appointment Reminder. In general though, and IMO, validating a startup idea is when people actually pay you for it, not just when someone says "cool idea". The difference seems minor but is huge, lots of prospects will tell you that it is an awesome idea, but when you say ok, fork over $50 or $5000 or whatever all the sudden they are like, yea, well I mean.... Same if you are seeking venture money, if you are not getting meetings or are being blown off you have to reevaluate your idea to make sure that you are pitching it correctly and that you are actually pitching something worth while. That doesn't mean you change ideas after the first 10 no's, just that you keep evaluating it to make sure you aren't too far off track and that you are communicating things correctly.

</comment><comment><USER6> Sheesh, that's a bit like being asked "how can I learn computers?" It's a very broad question. Better questions might be "what within the domain of business knowledge do I need to understand in order to achieve my goal(s)?" And then naturally, the most important question feeding into that is, "what are my goals?"

I think if you want to have a better basic understanding of how large economies work and how that effects political policy, I'd recommend studying macroeconomics.

If you want to understand the basic functioning of a business and how it relates to customers, I'd recommend marketing.

If you want to understand how businesses are evaluated, how they borrow and spend money, how they evaluate internal processes, I'd recommend finance.

If you want to understand how businesses optimize their processes, and design efficiency into their processes, then you should study operations research (aka operations analysis).

If you want to understand how a business keeps track of what it's doing internally, and how it's performing internally and how resources are allocated, and how it handles loans and taxes, then study accounting.

If you want to understand how to acquire talent and insure that employees are being treated in a way that respected laws on fairness, and how to manage worker's comp, pensions, health insurance, etc. then study human resources.

If you want to know how businesses handle their relationships with their communities and the media, then study public relations.

I hope this helps a bit.</comment>