Concurrency I can finally understand and write correctly - <AUTHOR> http://jordi.inversethought.com/blog/advent-of-d/#day18 <comment><USER2> People always used to ask me for advice on multithreading, because I had a reputation for multithreaded code that worked. I recommended they use Retlang (there's a Java version called Jetlang), which works on a shared-nothing message passing basis (and is blindingly fast). They tended to say they didn't want a library and just wanted to use locks and threads. I said that not doing that was how come I had a reputation for correctness.

<comment><USER3> They're in Java and writing locks and threads. Sigh.

java.util.concurrent is probably the single best concurrency library I have ever used. Bar none.

When I'm writing in Java, and I find myself writing a lock, my first reaction now is: "This is okay for exploration, but which data structure from java.util.concurrent am I going to have swap in here when it's all done. And can I do it now and save myself a lot of grief."

It only took me 7+ years to come to this realization. Apparently I'm a slow learner. :(

<comment> cup-of-tea They're using Java instead of Clojure. Sigh.

<comment><USER4> I'm saying this as a massive clojure fanboy: java.util.concurrent is an amazing set of concurrency utilities and is the building block for many of clojure's concurrency features. It's definitely worth knowing when writing performant java AND clojure (you can wrap the little bit you need so you don't have to scatter java interop all over your codebase). Sometimes a (swap! atom fn) doesn't cut it.

</comment></comment></comment></comment><comment><USER5> When Steve Jobs was away due to his illness, Bertrand Serlet presented Mac OS 10.6 (Snow Leopard) at the WWDC June 2009 with the startling slide that said "0 new features". There was sustained applause from the crowd. But in fact there was a very significant under-the-hood new feature called "Grand Central Dispatch". It is a low-level API that removes a lot of the pain of multithreading. Essential for OS X and iOS developers, as it turns out.

<comment><USER6> Dear God I wish they would do another 0 new features release.

It used to be that the excuse was bug fixes don't sell. Maybe not, but buggy software does lose customers. They may bring in the newbies with features, but you retain with bug fixes.

<comment><USER7> I really don't think the general population abandons entire operating system ecosystems, because of buggy software.

<comment><USER8> I think in part they do, otherwise OS X wouldn't have gained as nearly as much marketshare on the promise that "It just works"

<comment><USER7> OS X gained market share because of the insane popularity of the iPod, and subsequently the iPhone. Gained nearly 4% post iPhone launch.

</comment></comment></comment></comment></comment><comment><USER9> In my 30+ years experience of concurrency, the shared-nothing message model makes it relative easy to reason about concurrency (since every actor is single threaded).

I have worked with Java, C# and Python, and for each I have developed a concurrent object framework, making it quite easy to work with multible channels of incoming events, for example for process control systems with barcode-scanners, scales, I/O-busses etc. connected to the system

I think ponylang does a pretty good implementation of the Actor model.

Feel free to have a look at the Python impl. of the framework (www.github.com/pylots/pyworks)

<comment><USER10> In a way it is ironic that we went full speed into threads, shared memory and plugins, only to go back into processes, messages and sandboxing as a means to tame multi-core programming and safety.

<comment><USER11> Concurrency abstractions are still built on primitives that involve locks and memory barriers.

<comment><USER10> Just like the old UNIX process IPC or micro-kernels communication is.

</comment></comment></comment></comment><comment><USER12> I'm surprised nobody's mentioned Clojure in the comments. Its concurrency abilities are top-notch, straightforward to use, and immutable data makes it dead-simple to avoid issues.

<comment><USER13> It also occupies a different subspace. D is still relatively close to the metal.

<comment><USER12> That's fair. Though the JVM is more performant than most people think.

</comment></comment></comment><comment><USER14> In the distant past, I did low-level multithreaded programming in the patchwork style of synchronizing access to various data. Partly because the codebase already did it that way and partly because i didn't know better. These days I would never torture myself with such a thing. I still do concurrency sans framework, but like in the article - lock free with no sharing. Copying a block of data when it's ready is cheap compared to computing it.

<comment><USER15> In addition, you can break your program down into flows of tasks up front. Thinking about the synchronization explicitly helps too, often there is too much unnecessary communication.

I think that the data structures used by D for message passing are the real interesting part here. Akin to not using raw pointers, don't use raw data structures to convey state updates and let the owner of that state do it themselves with a queue or some abstraction for messages.

</comment></comment><comment><AUTHOR> Hey, if you're curious about anything else I wrote in that blog post, just ask me. I thought the concurrency was a big highlight, but there's lots of other parts of D that I would love to talk about.

As I said, the whole thing was very enjoyable. I had lots of fun using D.

:-)

<comment><USER16> yeah but what about recursive named emacs kmacros ?! jk

nice post

</comment></comment><comment><USER17> I like concurrency in Rakudo Perl 6. ([https://docs.perl6.org/language/concurrency](https://docs.perl6.org/language/concurrency))

Especially the .hyper method that turns a regular sequence into hyper sequence that gets processed in parallel. First 5000 prime numbers:

        # single-thread, takes 14.893s
        put ^∞ .grep(*.is-prime).head: 5000

        # multi-threaded, takes 8.853s on my 2-core box
        put ^∞ .hyper.grep(*.is-prime).head: 5000

<comment><USER18> Concurrency is not a first class citizen in Perl 6 though. As with Perl 5 the best you can hope for is actor model implemented on top of promises/futures.

<comment><USER17> What would make it a first-class citizen?

Even an empty Rakudo Perl 6 program is a multi-threaded program, as the dynamic optimizer runs on a separate thread. I'd figure if that's possible, concurrency/parallelism would be quite a first-class citizen, not a best hope on top of something.

<comment><USER18> Language support for actor model as opposed to shared memory multithreading would make concurrency a first class citizen.

<comment><USER17> Thanks.

</comment></comment></comment></comment></comment><comment><USER19> BeOS had an threading implementation like this in C++ in 1999: [https://www.haiku-os.org/legacy- docs/bebook/TheKernelKit_Thr...](https://www.haiku-os.org/legacy- docs/bebook/TheKernelKit_ThreadsAndTeams_Overview.html)

<comment><USER20> I used to program in this, and it was clearly inspired by the actor model (in retrospect, now that I have been programming in elixir for a while). Lots of void* passing, though.

Initially the actor model was designed in the 70s.

</comment></comment><comment><USER21> For a C++ implementation of actor patterns, see: [https://www.actor- framework.org/](https://www.actor-framework.org/)

</comment><comment><USER22> A shout out to the GPars framework [1] which makes actor based coding stunningly easy in Groovy / Java. I'm kind of shocked by the performance I can get out of it, to be honest. I once assumed that for trivial tasks (reading lines from a file, transforming them, writing out result etc), a parallel library would lose a lot of overhead in thread synchronisation. However I find that I can trivially construct data flows that massively outperform single threaded C code in Groovy. Which is to say, whatever overhead there is from thread synchronisation is dwarfed by the benefit of parallelising reading, computation and writing. If I had to think hard about concurrency to do it (as I would in C) I would probably never attempt it for simple programs, but GPars makes it so easy, safe and completely portable that there's really no reason not to.

[1] [http://www.gpars.org/](http://www.gpars.org/)

</comment><comment><USER23> On the other hand an Actor modelled implementation can - will - quickly get out of hand if you start using it to directly represent a business domain. Actor signature is Any -> Unit which essentially means "computes anything" plus there's internal state (become), message queue size, problems with deterministic testability and so on.. I think Actors as an alternative to Threads, particularly valuable for its distributed supervisor trees, and used to build more abstract consumers for a strongly typed model of business representations.

</comment><comment><USER24> It amazes me to no end that CML-style concurrency is not used more often. (Even though Go is sort of almost there)

I have been using guile fibers for some time, and it is a marvel! Not only can I write most code as if it was single threaded, but I can mix fibers code with pthreads and use fibers message passing when I need it.

</comment><comment><USER25> If anyone is looking for another language that also uses the actor model for concurrency and has very similar ways of handling it like D, then checkout Erlang. It's language lessons are clearly still relevant today. :)

<comment><USER26> Would any sort of actor model language have the raw throughput for a AAA game? I get how this model is nice for the web, for example, but I'm wondering why it doesn't get used for high performance computing (that I know of).

<comment><USER27> The Erlang VM (BEAM) is unusual in that it, like the languages it hosts, is very opinionated.

It is designed for robustness, scalability, concurrency, distributed environments. And immutable data. So far as I know, you literally cannot implement a language with mutability on the VM.

So, raw performance will never be its thing.

In general, I think the actor model could achieve high performance, but perhaps only if messaging is syntactic instead of truly distributed with mailboxes, network transparency, etc.

<comment><USER28> I'm a beginner, so by all means correct me if I'm wrong, but as I understand BEAM is relatively good when it comes to 'soft realtime', and especially when 'latency' is a concern (in part due to per-process GC?).

Am I correct in thinking that this would be pretty okay for most games, AAA or not? Would an FPS be possible (quick updates, small messages)? Or a World Of Warcraft or Sea of Thieves style game with many actors that need sort-of- realtime performance but don't rely on it entirely?

I've been looking into creating a game, and I'm also learning Elixir, so I'm curious what would be realistic when combining both.

<comment><USER29> In performance terms, Erlang is broadly speaking a scripting language, in the 10-20x slower than C range. It would be unusable for a AAA game because even using Erlang without any concurrency, it's too slow. It will get even slower if you do what you might be inclined to do for a game and make a separate process for every entity in the game and communicate entirely by message passing. It will be a beautifully clean architecture, but while Erlang may have cheap concurrency, it does not have _free_ concurrency, and if you work realistic math on the sheer number of messages you'd have flying around the system it should become clear that it will not be practical to have literally manifested "messages" in that quantity being continuously created and destroyed.

If we tune our sights down from "AAA game", there are two possibilities. One is you can create a less computationally-intensive game that can run Erlang on the desktop. I suspect you'll find you're a bit short of libraries for that use case, but with motivation you can pound through that. I'm not sure if this has ever been done. The other thing you can use Erlang for is being the backend server of a game system, and _that_ is eminently practical, in the sense that it has been done: [http://erlang.2086793.n4.nabble.com/Erlang-Who- uses-it-for-g...](http://erlang.2086793.n4.nabble.com/Erlang-Who-uses-it-for- games-td2279186.html) You'd encounter some bumps if you tried to scale it up, but that's not a very strong criticism since it's constant regardless of what tech you'd end up using.

<comment><USER28> Thanks for the info. I was definitely not thinking of using Erlang/Elixir for the actual game, but rather as a back-end. Glad to hear that wouldn't be a bad choice.

</comment></comment></comment></comment></comment></comment><comment><USER30> I don't see what's so good with that. Don't we have multiprocessing and threading pools that does that stuff automatically ?

</comment><comment><USER31> Obligatory, _but Haskell_ , comment.

Nice post, looks like a decent API. I wonder how similar each implementation of this model in Language X is and how easily it would be to test their implementation meet the spec.

<comment><USER32> What about Haskell? Haskell has great support for writing code in the Actor style if you wish. But you can also have traditional locks and simple mutable pointers in Haskell too.

<comment><USER31> It's great for writing concurrent code that is both easy to understand and be confident in its correctness.

I just didn't want to derail the conversation away from D and the authors' discovery.

</comment></comment></comment><comment><USER33> Try actors.

</comment><comment><USER34> I'm not a professional programmer, but I had no problems with Java's multithreading libraries. Lock handling, synchronization etc are a pain in the ass to get used to but not so much of a big deal, and I don't see how D is any different than that.

<comment><USER35> With the actor model you don't share state between threads, hence no need to lock anything.

Each variable can only be updated by a single and unique thread. If another thread (or actor) wants to modify it, it has to send a modification request messages to that one thread to do the job.

That's quite a difference!

Edit: Java threading model is fine for limited amounts of concurrency. Complexity escalates quickly with highly concurrent applications; the actor model makes reasoning about those programs far easier.

<comment><USER34> So it's like a lock, except you call it "modification request". Great, got it.

<comment><USER36> The difference is that it doesn't halt your execution unless you want it to. With shared memory and locks you have to wait when you set a value if anyone else is interacting with that object or that part of the object (if you have fine-grained locks). So you wait both for getting and setting.

If your system behavior does not _immediately_ depend on the value of what you're changing, then actors are fantastic. You don't wait, it's async. You send the data to the controlling actor (which is an object in the OO sense itself). Your process continues on as if nothing changed _until_ you need the updated results.

      Process A
        B ! {update, x, newVal},
        %% A bunch of work
        B ! {lookup, y, A}, % where y perhaps depended on x
        Y = receive % Here we finally wait, if B is processing a lot of requests
          {B,y,Val} -> Val
        end,
        %% more work

      Process B
      loop(State) ->
        receive
          {update, Var, Val} -> loop(update(State,Var,Val));
          {lookup, Var, Pid} -> Pid ! lookup(State, Var), loop(State)
        end.

Where B may be much more complex than that, in this case it looks like a simple KV store.

The receive at the end of Process A is blocking and is the only time we end up with a lock in Process A. So if A never needs to do anything but send values to B, then A can run undisturbed by everyone else's data requests. This lends itself well to certain pipelined and dataflow architectures.</comment></comment></comment></comment>


The <header> tag is an expansion on HTML4's <head> tag. - <AUTHOR> http://www.netmagazine.com/opinions/how-html5-affects-seo

<comment><USER2> ".net magazine would like to apologise to its readers for the temporary removal of this article. As a result of a production error, this article was posted without editing."</comment>


Major Outage Reported at Slack - <AUTHOR> https://www.reddit.com/r/programming/comments/c6kgei/major_outage_reported_at_slack/ <comment><USER2> Slack was also hard down 8 hours ago and been intermittently degraded since. [https://downdetector.com/status/slack](https://downdetector.com/status/slack)

</comment><comment><USER3> Why anyone uses this glorified "hello world" app, let alone bases anything serious on it, is beyond me.</comment>


Startups high pre-money valuation is a killer - <AUTHOR>

Startups like to raise fund at a high pre-money valuation. It's understandable; founders want to give up little % of their company for a lot of money. But that could be very dangerous, even fatal for startups/founders. When the company is new and unproven, it's better to raise capital at a low pre-money valuation. Founders know when the company is a few thousand line of code, or barely bringing revenue, it shouldn't be valued as high, but they raise the valuation bar anyway. Let's say the founder's wish come true, and the company gets acquired, which is the new business model for most startups now these days. Investors will get the money they invested and more, before founders get anything. Sometimes founders get nothing. It's never about founders get 45%, investors get 65% of the company's stocks. Investors always want to get their investment back, and possibly more... If taking 65% is higher than taking 10X their investment, they'll take 65%; if taking 10X their investment is higher than taking 65%, they'll take 10X their investment. Investors get paid before founders. In an acquisition, the higher the pre-money valuation is, the more likely investors will take all, if founders don't sell as high as their company pre-money valuation. Now days, before a startup gets out of beta phase, it's already valued at millions. Facebook barely bringing $200 million in revenue is valued at $15 billion. Even Mark Zuckerberg doesn't think his company worth that much. Digg.com barely has a business model, is valued at $200-$300 million. What happen to the good old fashion of buying low to sell high? Startups need to buy investor's money at a low pre-money valuation to exit the company high to make a profit. <comment><USER2> it could be interesting proposition if founders know how much to value, if they have an experience mentor then they can calculate all kinds of permutation combination. Sometime it is better to keep the control and give away stake and vice-versa.

But following is what i learned from one of the venture blog, its very interesting and educating:

Entrepreneurs focus on valuation when they should be focusing on controlling the company through board control and limited protective provisions.

Valuation is temporary, control is forever. For example, the valuation of your company is irrelevant if the board terminates you and you lose your unvested stock.

The easiest way to maintain control of a startup is to create good alternatives while you're raising money. If you're not willing to walk away from a deal, you won't get a good deal. Great alternatives make it easy to walk away.

Create alternatives by focusing on fund-raising: pitch and negotiate with all of your prospective investors at once. This may seem obvious but entrepreneurs often meet investors one-after-another, instead of all-at-once.

Focusing on fund-raising creates the scarcity and social proof that close deals. Focus also yields a quick yes or no from investors so entrepreneurs can avoid perpetually raising capital.</comment>


Getting started with Activity Stream - <AUTHOR> https://medium.com/@rohitkhatana/getting-started-with-activity-stream-d7d5a528394c <comment><USER2> Good post</comment>


Free Speech Matters, Even When It's Not Protected by the First Amendment - <AUTHOR> https://quillette.com/2019/08/18/free-speech-matters-even-when-its-not-protected-by-the-first-amendment/ <comment><USER2> Its quite simple, really. There are laws in EU against hate speech, by race, religion and so on. For some reason there is no such laws in US.

<comment><USER3> How does this have anything to do with the article? It's about private/non- government entities.

I honestly find it weird the EU has such laws because it is the government policing speech. I just don't think that is a power that any government should have or wield for any purpose.

A quick glance of the article seems to point/hint to we should hold other entities to the same standard we hold our governments. Honestly, that is something I don't really see a government can legally do or probably should do. It also something I don't think we will see socially pushed. As socially why would people protect toxic or things of a vitriolic nature.

I think this really just comes down to principle vs reality. People don't follow their principles completely & perfectly. It can make people seem hypocritical, logically inconsistent or compromising. However, people are allowed to do that. However, I think we should hold our governments/institutions to a higher bar than that for sure.

<comment><USER2> "Hate speech in the United States is not regulated, in contrast to that of most other liberal democracies."

I think any civilized country should have these laws, and most do.

[https://en.m.wikipedia.org/wiki/Hate_speech_in_the_United_St...](https://en.m.wikipedia.org/wiki/Hate_speech_in_the_United_States)

<comment><USER3> I don't think that is a role or a power a government should have. I am aware other countries tend be less certain about this than the US. However, looking at data here you can see that it's still not a small amount of people in other Countries. Even at times exceeding 50%.

Image with some interesting data. [https://www.pewresearch.org/global/wp- content/uploads/sites/...](https://www.pewresearch.org/global/wp- content/uploads/sites/2/2015/11/Democracy-Report-80.png)

Honestly there is more discussion, about this coming out of the US because we consider a wider area of things as free speech. It really comes down to the fact we see other entities not following the same principles we hold our government too. However, like I said earlier people or other private entities do not have to have the same standards or be consistent like a government.

Also that is one the biggest problems with giving government the power to police speech is how to be consistent.</comment></comment></comment></comment>


What computer and software is used by the Falcon 9? - <AUTHOR> https://space.stackexchange.com/questions/9243/what-computer-and-software-is-used-by-the-falcon-9 <comment><USER2> Comments moved to [https://news.ycombinator.com/item?id=23368109](https://news.ycombinator.com/item?id=23368109).</comment>


Ask HN: As a developer, how can I take advantage of time spent driving to work? - <AUTHOR> I work as a developer (Java full stack if it makes any difference) and live quite far from my workplace: it usually takes me one hour and a half to and from work by car. Moving closer to my workplace is not an option nor is it using other means to go back and forth (eg train/bus). I feel like my time spent on the road is mostly wasted: I'd like to find something IT-related to do (both listen to and reason about) that can help me learn new stuff.
Since I'm passionate about Linux, I usually have a few Linux (or BSD) podcasts to listen to while driving but I found that to be not really instructive, just more enjoyable than listening to the radio or some music playlist.
I'm looking for any kind of suggestion from fellow developers who are in the same spot and have found any interesting way or resources to 'spend wisely' their travel time.
What resources would you suggest (if any exists) to learn something while driving (much like you would do with a foreign language audio course)? Is it even possible? Has anyone tried or can share an experience of 'audio learning' IT related? <comment><USER2> I suggest leveraging the meditative process driving has on the brain.

I went through a process where I was trying to be extremely productive every minute of every day. That meant filling every minute with productive tasks including commute time with Podcasts and educational material.

But you get into this productivity loop without any real productivity.

In the end I found the best use of my drive time is just relax and let your brain run. Enjoy the sights and smells, the air coming through the window. Observe your brain how it notices cars and avoid potential disaster.

If you play it right, you'll have 2 hours per day to just do nothing. Let your brain work freeform while focusing on something simple like deep breaths.

<comment><USER3> I second this, but would also add that I've started motorcycling instead of driving. Even a commute on a motorbike is a thing of wonder, using parts of my brain and fight-or-flight responses I've left dormant for years.

But to the primary point, I also thoroughly agree that downtime is important, and there's a danger of trying to optimise every waking hour too much. Learning the art of structured procrastination (neither too much nor too little) should be part of every pragmatic programmer's toolkit.

<comment><USER4> I started bicycling to work earlier this year, and it's much less stressful than driving. My commute is about 7 miles when riding my bike, or about 10 miles when driving. Driving takes about 25 minutes on the way to work, and about 35 on the way home.

Cycling is about 40 minutes both ways. I take it a little easier on the way to work because I don't want to sweat too much, and then change in to my work clothes and do a hobo shower in the single stall bathroom. On the way home I can choose my pace, unlike when I'm driving. I can mash the pedals and let off some steam when I need to, or I can mosey home when I'm getting burned out. Stopping along the way is much easier. I'll grab coffee, or a snack, or check out a used book store I've never seen before. Plus, you are much more a part of the world when you are cycling, and you can look around more and notice things you wouldn't when driving. I'm sure a lot of these things hold true for motorcycling as well.

I'm not much of a morning person, but I find it much easier to concentrate when I arrive at work after cycling in. I also find myself much more energized when I get home, even if I'm pretty tired when I leave work. I find taking public transportation provides some of the meditative aspects, since I can reflect on my day more when I'm not driving in Chicago traffic, but bicycling allows me to combine my commute with exercise, which is amazing from both a mental health and physical health perspective.

That being said, riding a bicycle is certainly more dangerous than driving or taking public transportation. While I can take steps to mitigate the risk, I need to cross a number of bridges and go under a number of viaducts on my way to work, and the risk is substantially higher while on a bike than in a car, bus or train. I don't think I'd be comfortable riding my bike on my current commute if I had a kid.

</comment></comment></comment><comment><USER5> Do you ever solve problems or have other insights while not working on those things? Like when you sleep, or in the shower?

Your brain needs rest and distraction, both for professional health and general health.

If I worked, say, 8 hours a day, and then added in your three hour commute every day, that would be borderline death march.

Besides all the utilitarian suggestions, you could _consider_ something unrelated to your work or profession, at least part of the time.

I sing. I put a couple hundred singable (by me) songs on a usb stick, and play them. When I land on one that I want to learn, I put it on repeat, and learn it.

I didn't used to be able to sing, from lack of singing at all and self- consciousness. I didn't used to even sing in the car alone, I was that self- conscious. Now I've improved (my voice has been complemented, singing and not), and I've sung songs among friends and co-workers.

Sometimes I don't sing, and think about something at work or elsewhere. It's not all or nothing.

Maybe listen to old radio shows like The Shadow Knows, The Honeymooners, Dragnet, or whatever you find that appeals.

<comment><USER6> This guy has a very solid point which is especially important in todays world with the immense amounts of distractions and so on.

I would actually recommend not doing anything specifically work related, but maybe listing to a developer/problemsolving/relevant podcast could be one way to entertain yourself with somewhat relevant content if you really feel the need to. Being "bored" is time better spent than you would think :-)

</comment></comment><comment><USER7> I've tried listening to university lectures and varies conferences talks. But, I find them distracting on the road. I've found, consuming really technical stuff to be much easier. When I can give it my full attention.

Personally, I like to listen to podcasts that cover new and upcoming technologies in the field. They could be about high level stuff or even the nitty and gritty, like new language syntax or upcoming improvements to the GC.

Though, I prefer listening to audio books just as much. And, I find them to be a great supplement for professional growth along with podcasts. When I'm not listening to fiction or sci-fi books, which isn't much. I try and listen to books about soft-skills and anything relating to software development that doesn't involve having to read code.

I can't really give you any hard suggestions here. Because, I mostly, work with languages like Ruby & JS/ES6. But, there should be similar options available for at least the JVM, if not the Java ecosystem.

<comment><USER8> What are some of your favorite more technical podcasts?

<comment><USER9> Not technical but CGPGrey from Youtube has "Hello Internet" which is fun and relaxing.

<comment><USER7> That and "Cortex" by CGPGrey. Both of these can be quite entertaining and relaxing.

</comment></comment></comment></comment><comment><USER10> I think you're solving the wrong problem here. Let's start with some math. With your current arrangement, you're losing 3h of productive time a day, 15h each week, and a staggering 750h yearly. That's the equivalent of 18.75 40h workweeks of lost productivity. Pause for a moment and consider what you could do with that time.

If you feel your current employer is compensating you fairly for that sacrifice, then you likely have money to spend on a car service, use your commute time to work or study and still come out ahead.

If your compensation is not inline with the sacrifice you're making, it's time to start looking for a different job. There are plenty of remote gigs for full-stack java devs. Start looking for one of those.

<comment><USER11> It's not lost time if you are doing something productive. I find driving is good for the type of thinking I can't do with a computer in front of my face. (although I like walking better, given a choice)

</comment></comment><comment><USER12> After more than two decades as a developer, I have just now begun to realize all the other valuable skills, outside of technology, that I have not developed. And consequently, I've been very hungry for this additional education.

I understand you're looking for IT education, but I would recommend audio books (or language courses). If you haven't done any of these, give some a try:

* If you don't know a second language, try the Pimsleur courses.

* How To Win Friends and Influence People, by Dale Carnegie

* Getting to Yes, by Bruce Patton, Roger Fisher, and William Ury

* How We Choose to be Happy: The 9 Choices of Extremely Happy People, by Greg Hicks and Rick Foster

* Managing Oneself (Harvard Business Review Classics), by Peter Drucker

It's easy to learn the technical things with hands-on time and night time efforts, but sometimes the tech is so much fun that some of us just simply ignore topics that have longer term value and frankly can enable us to really make the most of our technical abilities.

<comment><USER13> >Pimsleur courses

I was wondering if that would show up in this thread. I have a fairly frequent, though not regular, ~45 min commute, and I've been working on one of their Korean courses. Each lesson is 30 min long, so it fits pretty well.

The only concern I have is that it can be very mentally taxing to, say, correctly respond using Korean to a question asked in Korean. I'd rather not die on the highway than learn a new language, so fairly frequently I have to drive first and ignore the audio for a few seconds.

To help fix this I've been doubling up, playing the same lesson on two commutes. More practice is always good in a language, I guess. I'm also fairly young and have only been driving for about six years, so it's not all autopilot yet.

The lessons are a bit dry, tbh. I've been wondering if there are any other audio-only language lesson publishers I should try that might provide some different styles.

<comment><USER12> Oh I would listen to the same CD over and over, and now I cannot get it out of my head if I tried :P. I definitely was not working toward a goal; I just found it surprisingly fun (and pleasantly distracting) while suffering commutes.

I didn't investigate alternatives for audio. If I were riding a bus, I would use DuoLingo app (which I found almost addicting).

</comment></comment></comment><comment><USER14> You can't. I tried. It is impossible to concentrate enough to absorb anything in rush hour traffic.

I ended up quitting and taking a job to which I can take public transportation.

<comment><USER15> I love coding on a train.

</comment></comment><comment><USER16> I have a nice "life hack" which I use to keep entertained while doing dishes or cooking: I make the computer read to me HN articles (not all of them... god no!), but the select few long reads that seem interesting. I select the text of the article and press a shortcut to enjoy a "custom audiobook" for the next 5-15 minutes. I use the built-in Mac OS X text to speech functionality[1], but I'm sure you can adapt this to other OS.

I have to go back to the computer and "play" a new article every 4-15 minutes which is not good while driving, but you can probably script the whole process and make longer recordings:

        manually select a list of articles that seem worth reading
        urls list --> main content recognizer --> html2text | `say` --> .wav --lame--> .mp3
        concatenate all the mp3 files to form a 2h long podcast

[1] [https://docs.google.com/document/d/1mApa60zJA8rgEm6T6GF0yIem...](https://docs.google.com/document/d/1mApa60zJA8rgEm6T6GF0yIem8qpMmnaBFYOgV32gdMc/edit?usp=sharing)

</comment><comment><USER17> My life changed when I started to listen to audiobooks while driving to/from work. However, the audiobooks are usually not pure technical, but always topics that can help me become better in many aspects in life in general, including professionally.

<comment><USER18> Indeed. Exposure to different areas is a great way of expanding one's perspective. The mind is a great associative device, sometimes connecting ideas we wouldn't otherwise. Plays well with the "I get my best ideas in the shower"-style phenomena.

</comment></comment><comment><USER19> Here are the two things I do during my commute:

      1. listen to programmers podcasts (e.g. Full Stack Radio)
      2. listen to entertaining/news podcasts and audio-books.

Everything else just doesn't work.

<comment><AUTHOR> Thanks, can you suggest me one or more particular podcast or a site to download podcasts from?

<comment><USER20> We do our best to produce high quality and interesting podcasts for developers. Give us a listen: [https://changelog.com/](https://changelog.com/)

</comment></comment></comment><comment><USER21> Focus on driving, you could kill someone.

<comment><USER22> Came here to say this. Take advantage of the time by making sure you get to work in one piece.

</comment></comment><comment><USER23> I'm sure you have some very good reasons for explicitly stating that moving closer to work is not an option, but I'm really dying to know why you're trapped in such a miserable circumstance.

<comment><USER24> Yeah, this seems odd to me, considering that a reasonably successful software engineer can afford to live pretty much anywhere.

<comment><USER25> Not in the Bay Area. Every developer or ops person I know either has roommates or has a similar 1.5-hour commute.

<comment><USER24> I live in the Bay Area too, and I only know one or two people in the tech industry who have that kind of commute. Many of my coworkers rent their own apartments without roommates.

</comment></comment></comment></comment><comment><USER26> 1\. Install Duolingo on your phone 2\. Change your phone's microphone default language to Spanish 3\. Practice by dictating spanish directly to your phone without typing

</comment><comment><USER27> Does it have to IT-related? I find audiobooks, both fiction and non-fiction, to be a great way to pass time during commute. The Great Courses offer a bunch of interesting lectures, for instance.

<comment><USER28> The OP is right to consider professional development during their commute, but it's not the only thing they could do. Instead driving time being professional development time, it could be hobby time. Listening to the news or audio books could free up time spent doing that at home for other things. This assumes, of course, that the OP has hobbies that they pursue at home that could be shifted to car time.

<comment><AUTHOR> My hobbies are mountain biking and windsurfing but I'm not particularly interested in listening to news or stuff related to those. As for books, I find it far more enjoyable to read from paper rather than to listen to someone reading, although it also depends on who's reading... I really enjoyed a few audio books narrated by Stephen Fry.

</comment></comment></comment><comment><USER29> I know a group of colleagues who used to car pool and discuss about IT related topics from their previous experiences.

A group of 3-4 like minded people on a conference call would be a good way to share topics about what you know and learn from others.

This may work best while in cars (as opposed to when using public transit)

</comment><comment><USER30> Pay attention to the road. Distracted driving is the ultimate jerk move, its saying that other peoples lives don't matter.

</comment><comment><USER31> Try listening to some 'non technical' podcasts. That is what I do on my 1 hour walks in the morning. It is strangely refreshing not to have to focus on technical talks all the time, but I find creative talks tend to inspire me and get me thinking in different ways about my work.

Added bonus is that you don't have to be 100% focused on what you are hearing, you can kind of drift in and out as something interesting catches your attention and still not lose the thread of the topic.

Two of my favourite podcasts are "99% Invisible" (creativity, architecture) and "The Tim Ferris Show" (biohacking, inspiration, interesting people etc.)

</comment><comment><USER32> I memorize a certain algorithm/code snippet and recall it while driving. My favorite is Paul Graham's ANSI Common Lisp[0]. From the code snippet, I choose a certain function to recall. It would be hard at first (I suppose that only few that can) but once you've repeatedly recall those functions everyday then you'll be amazed on the benefits when you've got into the flow. I got this idea from reading about Richard Feynman[1] that cause this to divorce with his wife since he is doing calculus in his head while driving: _the appointee 's wife was granted a divorce from him because of appointee's constantly working calculus problems in his head as soon as awake, while driving car, sitting in living room, and so forth, and that his one hobby was playing his African drums._

[0] - [https://sep.yimg.com/ty/cdn/paulgraham/acl2.lisp](https://sep.yimg.com/ty/cdn/paulgraham/acl2.lisp)

[1] - [http://gizmodo.com/5916502/richard-feynmans-fbi-files- make-f...](http://gizmodo.com/5916502/richard-feynmans-fbi-files-make- fascinating-reading)

</comment><comment><USER33> I have an Audible subscription, so I listen to audiobooks on my way back and forth to work (43 miles one way). The only IT related stuff I listen to is podcasts and that's mostly news. I'm not sure its a good idea to listen to anything super, technical that might require more attention than just listening to songs on the radio. You are driving after all.

I guess the trick for me is that the audio substitutes for other entertainment time so I can watch a training video when I am not occupied by driving.

<comment><USER34> I would pay double or triple for Audible if they didn't wrap their audiobooks up in an awful DRM that doesn't work with my ipod-knockoff.

<comment><USER18> They put the DRM in place so they can continue to charge lower rates that more people are willing to pay because the content isn't as easily pirated (for which they make no money at all).

What device do you have? What prevents that device from supporting the DRM Audible uses?

<comment><USER34> Oh, it's some sort of $20 SanDisk copy of the original iPod Nano from ten years ago. Plays MP3s beautifully, but that is about it.

So I ended up buying some files from Audible, then had the choice of resorting to torrenting the ripped versions or using some dodgy converter to extract something useful. Or cancelling the subscription and getting a refund, which is what I did.

<comment><USER18> Sounds like Audible has pretty good customer service!

I see a little incongruity here.

You're willing to pay $20 10 years ago for media player.

Audible subscriptions are about $15/month, with additional titles approximately $15 (or more) a piece. You're willing to spend perhaps $30–$45/month for a subscription or $30–45 per title?

      1 year Audible subscription: $360 (at $30/month)
      10 years: $3,600.

      Apple iPod nano: $150.
      1 year Audible subscription: $180 (at $15/month)
      Total for first year: $330.
      Each additional year: $180.
      After 10 years: $1,950

If you don't care for Apple, I'm sure there are comparable choices available.

There are likely other trade-offs you're likely taking into account as well (e.g., some people are opposed to DRM on principle, but it's not clear to me that's your argument). But if money can solve your problem which you imply by offering to pay more, it looks like you have options to use Audible for the price you're willing to pay.

If I've done the math wrong, or made some assumptions you don't agree to, please feel free to correct me.

<comment><USER34> No, you're correct; I ought to just knuckle under and spend the money, if I want to consume from Audible. But I picked up the current MP3 player on a flier, and have become used to it - it does what I want it to, in a very simple and straight-forward way.

The content that I've spent money on already dwarfs the purchase of the player. Although that content is DRM-free, so I am free to use it on this player, or any of my various laptops and desktops, and any future devices that I buy. And it is in a standard format that virtually any audio playback application understands. And I can just download those files normally, without having to install the weird Audible download manager.

<comment><USER35> I believe the selection is smaller but there is also Downpour [0]. DRM-free which means you should be just fine playing on your MP3 Player. The cost is comparable to Audible ($13/mo instead of $15/mo) and you get much the same deal: 1 audio-book per month with the "credit" you get.

[0] [http://www.downpour.com/](http://www.downpour.com/)

<comment><USER18> Hadn't heard of this! Thanks for the link. I wonder what their business model is like. It looks like they've been around for at least 3 years, so that's a positive sign that they're doing something right.

<comment><USER36> Just testing.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER34> I listen to podcasts or audiobooks on my commute. This summer I went through all of Dan Carlin's Hardcore History. It's true that it's not IT-related, but it's learning, and it is a good break from work. Now I've moved on to going through Game of Thrones once more.

It helps if it is something that you can enjoy without paying rapt attention - it's kind of important to have most of your brain focused on the road if you are behind the wheel.

<comment><USER37> If you're into the subject matter, there's the History of the Crusades podcast by Sharyn Eastaugh. In depth content, some levity, and I found it to be totally engrossing.

</comment></comment><comment><USER38> I really enjoy listening to Tech podcasts that contain a little humor; I would highly recommend:

\- Hansel minutes: [http://hanselminutes.com/](http://hanselminutes.com/)

\- This Developers Life: [http://thisdeveloperslife.com/](http://thisdeveloperslife.com/)

\- Coding Blocks: [https://www.codingblocks.net/](https://www.codingblocks.net/)

</comment><comment><USER39> I listen to the bbc's 'in our time' programmes [0] whilst driving. They are (much) more general than you want, but they are excellent programmes covering the whole spectrum of human knowledge. I've become a more broadly educated person by listening to those podcasts.

[0] [http://www.bbc.co.uk/programmes/b006qykl](http://www.bbc.co.uk/programmes/b006qykl)

</comment><comment><USER40> Relax and clear your head. Listen to some music, audio books.

<comment><USER41> In my experience, audiobooks are excellent for wide open freeway miles, and much less beneficial in traffic. One small event(close call) becomes an attention tangent and I miss last 10 or more seconds of content. Which leads to fiddling to scan back which becomes a distraction/irritation.

Try driving in silence. It allows me to process yesterday and anticipate the coming day.

<comment><USER40> Processing yesterday or the day just gone is not always a good idea, especially if you had a bad day. This used to happen to me, I'd mull over things I should have said or done and basically torture myself for an hour until I made it home. Fuck that.

<comment><USER41> Yup, different strokes. If I don't take time to reflect my life becomes messy and my decisions become reflexive instead of weighted. When I am heated, I do go through the thoughts and let emotions run(alone) to allow the release long before I act.

edit:removed unnecessary anecdote.

</comment></comment></comment></comment><comment><USER42> I live in LA, traffic is an issue here too. I see other comments mentioning take a brain break from IT stuff. They have a point, but up to you how much you need or want this.

 _Another option: instead of learning something new during this time, use it to teach or mentor someone over the phone._

Whether talking to a student or beginner dev, a couple of the commutes you can schedule weekly mentoring sessions while you drive. Similarly, one could offer tutoring over the phone if relevant. Up to you if you want to charge $ for your time or do it just for the good karma of helping someone.

Either way, chatting on the phone while mentoring/tutoring in my experience affects the brain differently than learning something via audiobook, and it is still possible to concentrate on driving while giving good advice.

</comment><comment><USER43> I highly recommend Clarke Ching's "Rolling Rocks Downhill" audiobook [1] which is a novel that teaches/refreshes agile software development in a really engaging way. I'd love to know if there are more novels like that out there - I haven't managed to find any.

I'm getting bored with podcasts these days because most of them have too much idle chat. Shows like "99% Invisible" that are well produced and low on chit chat are fine but I think audiobooks are the way forward.

[1] [http://www.rolls.rocks/podcasts-video/2016/2/23/rolling- rock...](http://www.rolls.rocks/podcasts-video/2016/2/23/rolling-rocks- downhill-audiobook-part-1-free)

</comment><comment><USER44> Listen to software conference talks! It's true that some of the more technical ones will not translate as they require too much concentration. Ideally, you can find some that are more conceptual or do not rely directly upon watching the slides.

I posted a few of my favorites from last year here: [https://www.kevinlondon.com/2015/09/10/10-software-talks- to-...](https://www.kevinlondon.com/2015/09/10/10-software-talks-to-listen- to.html)

I, too, generally dislike software podcasts in the car. For me, they usually are paced more conversationally, so I find that I get disinterested.

<comment><USER45> I've been working on a catalog of tech conference talks, which you might find interesting - [https://www.findlectures.com/?p=1&type1=Tech%20Conference](https://www.findlectures.com/?p=1&type1=Tech%20Conference).

I'm hoping soon to have additional facets for things like programming language / framework.

</comment></comment><comment><USER46> I have a commute of between 45 minutes and 1 hour where the majority of it is a straight shot on the highway.

I try to learn something new on every drive, and for me this comes from two activities.

1) try to work out design of something I have not done before or something that needs to be done. Iterate over some use cases in my mind, look for leaky abstractions etc.

2) listen to technical podcasts or podcasts about programming or developing some skill related to programming. examples podcasts The Changelog, Go Time, SE Daily, Software Engineering Daily, Full Stack Radio, YC Podcast

</comment><comment><USER47> I have 4 hours of commute(train + tram/metro) daily..., read a dozen of novels from Isaac asimov and Arthur C. Clarke in 2 months. After work, I don't even have time to open my computer or do anything besides eating, shower.

<comment><USER47> Sometimes I'm so absorbed by a novel, I seriously consider to arrive 'late' at work :D.

</comment></comment><comment><USER48> I have enjoyed making tech podcast especially for the 20-30 minute commute. They are fun and informative with a kick of inspiration and have been a side project to help me network with cool people in the tech space. So far I have had on the show Grammy & Emmy award winning talent, space experts, angel investors, Billionaires, AI, AR and VR leaders and a few new YC founders to boot.

Tomorrow's Tech - [https://soundcloud.com/user-925097294](https://soundcloud.com/user-925097294)

</comment><comment><USER49> > (much like you would do with a foreign language audio course)

How about trying just that? It's a lot of fun if you get a good course (e.g. Pimsleur), since you can really feel the progress. I've never managed to learn much from technical audio stuff; I feel like I need pen&paper/computer at hand to grasp new technical cocncepts, while language actually works better when i'm not able to distract myself :-) (Step Two: use newfound language skills to move to a place where you don't need the commute.)

</comment><comment><USER50> One useful way to spend dead time like that is to plan what you'll be doing at work, especially if you have a hard problem to solve. One of the ways you can improve your skills even if you're working limited hours ([https://codewithoutrules.com/2016/11/10/work-life-balance- so...](https://codewithoutrules.com/2016/11/10/work-life-balance-software- engineer/)).

</comment><comment><USER51> Consider partial work-from-home for a couple days a week. I don't miss commuting at all, and for me, it would only involve a 45 minute bus-ride during which I can read all I want, since its an end-to-end ride. But working with a cat in the lap sure does beat those cold winter commutes.

</comment><comment><USER52> Audio books. But not software audio books (that sounds like torture). Listen to books on things that broaden your horizon. I listen to a lot of books on business and marketing even though my day to day job has nothing to do with that. I also listen to a lot of science fiction (old stuff and new) because it helps me appreciate where we have been and where we might strive to be as a thinking society.

</comment><comment><USER53> I'm typically thinking about what I have to do at work, or complaining about what processes is horrible and how I can propose to make it better. If I charged for those brainstorming hours during driving I would have a very nice car every year but I feel it makes me better at work and I've come up with some of my best ideas while stuck in traffic.

</comment><comment><USER54> Learning takes a good amount of your attention. While driving your attention should be on the road and what's going on around you. Please don't try to do in depth learning while you are driving.

\- Pedestrian who doesn't wanna get smoked by a distracted driver learning about some new Java framework

</comment><comment><USER55> It might not be possible, but could you share the ride with someone from your workplace that has the same interest?

<comment><AUTHOR> Not possible, none of them living nearby. And I suspect the majority of them would rather not talk about programming or similar stuff outside of work hours. Maybe it's for the best because honestly I would not survive one hour or so drive of small talk everyday.

</comment></comment><comment><USER56> You are asking the wrong question. The real question is why are you still driving to work? What is special about your office that requires you to be there in order to get work done? Do you not have internet at your house?

</comment><comment><USER57> I have a side project and usually in the morning I have a pile of emails waiting in my inbox, so I reply to as many of them as I can on my bus ride to work. Basically doing support.

</comment><comment><USER58> Invest in a Tesla vehicle? Probably sounds like overkill but if you can afford it the autopilot feature could get you your hands/hours back

</comment><comment><USER59> I find TED Talks tend to help inspire new ideas more often than other media I consume. Maybe listening to some of those instead of podcasts.

<comment><USER35> Just make sure they are TED talks and not TEDx talks. TEDx talks carry the TED branding but are local events and aren't always quality controlled. They're also predictable to a fault. Case and point, Sam Hyde trolling an entire TEDx talk [0] with bullshit was both somehow (1) approved and (2) people actually let him finish.

I used to listen to TED talks but after a while I had to admit to myself a majority of them are painfully predictable in how they presented and so many of them are really someone propping themselves up as some sort of smarter/holier-than-thou savior of humanity.

[0] [https://www.youtube.com/watch?v=-yFhR1fKWG0](https://www.youtube.com/watch?v=-yFhR1fKWG0)

</comment></comment><comment><USER60> How about podcasts that aren't directly relevant to your work? There's a lot of other interesting stuff out there.

</comment><comment><USER61> No ones mentioned twit.tv - lots of good programs there. Try triangulation - interviews with IT luminaries

</comment><comment><USER62> You're just like me. Except for I don't drive, I take a bus, then subway and then bus again.

I do read HN, Medium, twitter (all IT-related) and other resources. Though sometimes it's really pretty hard to concentrate and read articles carefully. I also use headphones to not get distracted by the engine noice.

</comment><comment><USER63> When diving your primary focus should be on... wait for it... DRIVING!

</comment><comment><USER64> Meditate.

<comment><USER65> This. But keep awareness of your surroundings while you do :)

</comment></comment><comment><USER66> Audiobooks.

<comment><USER67> When I'm not consuming my weekly tech podcasts, I'm listening sci-fi or fantasy on Audible. Books are really cheap if you pay a subscription, and you also get access to "channels", I sometimes put some comedy channel to lighten up my day.

</comment></comment><comment><USER68> commute by bike. 3 hours in the car every day is brutal.

<comment><USER69> 3 hours by car is likely minimum 60 miles which is 4 hours by bike every single day minimum. If they're getting to highway speeds at all then it's even worse. It'd be like black mirror episode 2 every day

<comment><USER70> Another way of thinking about this is that the marginal cost of biking is only 1 extra hour per day. And there's no reason why this needs to be done every day - he could just bike in once a week.

<comment><USER69> He's already complaining about the actual cost.

If he manages 60mph the whole time are you saying he should do 180 miles once a week? That's not really possible without a lot of training. What if he lives in issaquah and commutes to Seattle. How about on the oil fields in nd?

Bigger question what else does he do to better himself during the bike rides?

</comment></comment></comment></comment><comment><USER71> Call your Mother.

</comment><comment><USER72> voice controlled command line? audio text book? maybe just accept you're not going to get much done when you're on the road. do you want a programming course for when you're sleeping too? give it a rest guy

</comment><comment><USER73> I love the FaiF podcast for this:

[http://faif.us/](http://faif.us/)</comment>


China's lending bubble is history - <AUTHOR> https://www.ft.com/content/45bd8052-59dc-11e8-bdb7-f6677d2e1ce8 <comment><AUTHOR> Paywall bypass here [https://outline.com/VFqdrY](https://outline.com/VFqdrY)

Plugging this FF plugin [https://addons.mozilla.org/en- US/firefox/addon/bypasspaywall...](https://addons.mozilla.org/en- US/firefox/addon/bypasspaywalls/)

<comment><USER2> Clicking the |web| link under the title and then selecting the boxed listing at the top of the Google results page usually provides full access Financial Times articles. Sure I'm probably giving Google data.</comment></comment>


At what time will the YC S18 invites/rejection emails be sent? - <AUTHOR> <comment><USER2> Last time we received an email on April 18 3-4am CEST, which is April 17 6-7pm PST. Still quite some time left ;-) Btw, the Metachat community covering chat on this subject is back, just like last W18 batch. You can find it here: [https://communities.metachat.com/communities/ycombinator](https://communities.metachat.com/communities/ycombinator)

</comment><comment><USER3> Around 24 hours remaining. Come join the unofficial chat room for yc s18 applicants! Link : [https://www.hackerschat.net/](https://www.hackerschat.net/)

</comment><comment><USER4> The suspense begins..</comment>


Employer Waiver of IP Rights related to an employees new venture - <AUTHOR>

<comment><AUTHOR> Can anyone help with a small but important issue I'm facing. I need get my current company to sign a waiver on the IP rights new business I'm creating and want to present a legal document for them to sign. If anyone can point me to a resource, please post the link here or contact me at: hasan (at) promoterforce . (com)

Many Thanks!</comment>


Show HN: Desktop software to secure your wifi - Wifi Protector - <AUTHOR>

Seeing the article on Ars technica about wifi hacking, I feel this could be relevant for other folks here who want to stay secure on wifi networks.
Few weeks ago we launched Wifi Protector - http://www.wifiprotector.com. Its a windows software that will scan your wifi networks and asses how secure they are. It will  show you intruder alerts if somebody new appears on the network and it will also offer VPN protection.
We have been working months on it and got some decent reviews already. I would be interested in your feedback since we still working out what useful features to pack into the product.
Warning! Its a free product that is ad-supported, if you do not want to see any ads - untick the Browser Extension during installation. <comment><USER2> How does it secure anything? It just tells you about the existing security. What exactly does this do?

<comment><AUTHOR> When you click on "Get protected" it routes all your traffic via secured VPN. That way no one on your network can see your traffic.</comment></comment>


Thredis is Redis + SQL + Threads - <AUTHOR> http://thredis.org/

<comment><USER2> Interesting! But not where the original Redis project is headed to (other than Cluster I do not want to add any major feature for a while), so it made a lot of sense to fork and hack on it as a separated project.

<comment><USER3> Agree completely, this is not where I want Redis should go, I was just tinkeing with the idea.

Antirez: are you considering a Sorted Map of some kind with a way to iterate over it? (ZSET is almost it, but semantics are a little different and Redis does not have iterators, at least not exposed to the protocol). With SQLite4 a Redis Cluster with such a type could serve as the underlying storage engine, basically creating an infinitely scalable database.

BTW - kudos on Redis, I LOVE the clean C code. I must note that in the process of hacking on Thredis I couldn't help noticing there are many "cultural" similarities between SQLite and Redis, e.g. fondness of Tcl, integers as strings, etc.

<comment><USER4> you can make your own iterator over sorted sets pretty easily with zrange

<comment><USER3> True, but not quite. The first thing zrange does is look for the starting point, which is an approximately O(log(N)) for a skiplist (which is how it's implemented). An iterator's next() operation should be O(1). If I need to iterate over a gazillion elements, it would make a huge difference. Or course you could zrange chunks, and it would be more efficient, but what's wrong with a simple START and NEXT semantic?

BUT this is all a moot point - a Sorted Set is not the type of structure that something like SQLite4 needs anyway. The keys should be sorted lexicographically, not by score. I think a Skip List fits the bill perfectly, and Redis already contains an implementation of it (as part of Sorted Sets), why not expose it as a standalone type?

</comment></comment></comment></comment><comment><USER5> I think this approach manages to combine all the disadvantages and almost none of the benefits of SQL and NoSQL (and threads?). It's cute but seriously, where would this ever be practical? Consider the alternatives:

If you need concurrent client transactions over sets wouldn't you be better off just using a SQL database?

If you need fast KV, why not just use NoSQL over SQL ala memcache or something newer like memcache/innodb[1] ?

If you want complex updates over redis why not just use redis 's Lua interface[2]?

I think sqlite4's approach[3] (SQL above NoSQL) makes a lot more sense than this since it will presumably let me write a SQL application and deploy it with any KV store.

1- [https://blogs.oracle.com/MySQL/entry/nosql_to_mysql_with_mem...](https://blogs.oracle.com/MySQL/entry/nosql_to_mysql_with_memcached)

2- <http://oldblog.antirez.com/post/redis-and-scripting.html>

3- <http://www.sqlite.org/src4/doc/trunk/www/design.wiki>

</comment><comment><USER6> Oh wow, this is awesome. I wish to put the bitcoin block chain into thredis tonight.

</comment><comment><USER7> Probably a good idea to have antirez feedback on it, and see if this is something that will make sense in Redis overall. I think the dump as a SQLite DB file is a great idea since it can be potentially used by some other system by simply copying it and have plain old SQL applied to it.

</comment><comment><USER8> What are the threads used for? It's mentioned in the page but not explained. Looks awesome.

<comment><USER3> They are explained here: <https://github.com/grisha/thredis/blob/master/README-THREDIS>

Initially Thredis was just threaded Redis, and I had a very specific use case for that. The idea of adding SQLite came later.

</comment></comment><comment><USER9> It's duct-taped together but cute as all hell, like a lil' Frankenstein's Monster

</comment><comment><USER10> Redis used event loop model to handle requests. So basically, at any time, there is only 1 thread change data. This model make redis fast and still guarantee data is atomic. I wonder if thredis use thread how thredis solve that issue?

<comment><USER11> A single-threaded event loop gives you low latency... when you're not under load and servicing multiple concurrent connections. As the amount of load on Redis (or for that matter, any single-threaded program) increases, the amount of time Redis spends servicing other requests increases, and so the average latency increases proportional to load.

If Redis could _actually_ process requests in parallel instead of one-at-a- time, this would not be an issue and Redis could actually have a reliable latency profile under load.

About the only solution in this case is to turn Redis into a distributed system via sharding/slaves, and as soon as you do that, you lose Redis's guarantees around atomicity. Furthermore, Redis provides few tools that are really essential in a distributed system, like read repair/quorums or a failover system that isn't a joke (i.e. sentinel, although redis-failover could fit the bill)

</comment></comment><comment><USER12> I like it. I don't know that I would ever use it, but I like it.

</comment><comment><USER13> Really cool little project. Kudos.

Someone posted about one of their projects here the other day, Akiban [0]. While not exactly the same it was also bridging the SQL/NoSQL worlds. Not 100% sure on how it works but it has table-groups to let you pre join collections together (I think).

[0] <http://www.akiban.com/>

<comment><USER14> No source code?

</comment></comment><comment><USER15> I don't know if I need this, but bravo for being crazy

</comment><comment><USER16> That's insane. In an awesomely insane kinda way.

</comment><comment><USER17> <http://www.quickmeme.com/meme/3spmcc/></comment>


An Analysis of 155 Postmortems from Game Development [pdf] - <AUTHOR> http://research.microsoft.com/pubs/262301/washburn-icse-2016.pdf <comment><USER2> "To avoid schedule slippage, game developers need to spend more time to plan out all the work that needs to be done so that no tasks are overlooked when giving estimates. By planning out tasks, you can also estimate each task individually, and give a more accurate prediction, instead of an optimistic one."

When things don't work out according to the schedule, there are generally two reactions :

1\. We didn't estimate well enough, we need to estimate better next time.

2\. Maybe this estimation thing is impossible to get right, let's not put too much trust in being able to estimate accurately, but rather find ways of working that don't require estimating all work up front.

I've seen that 1 gradually becomes 2 with experience (the experience of getting the estimates wrong every time).

<comment><USER3> I think you're talking about "first year of work" levels of experience. Estimation is hard - but a major goal of Agile development is to make it possible.

I thought the same thing as you, but a month ago someone explained it to me in a way that makes a lot of sense [https://news.ycombinator.com/item?id=11641316](https://news.ycombinator.com/item?id=11641316)

Actually a lot of the comments about this PDF can be summed up with "practice Agile development"

<comment><USER4> How does that possibly solve the problem being discussed here? Are you the god who can define ALL tasks (in the agile meaning, a sprint consisting of a number of tasks) that a large project consists of up front, all the way to the end of the project 3 years later (for example)? Agile delivers because it doesn't attempt the impossible.

<comment><USER5> The problem seems to be that although companies want to pretend that they are all agile, the business and planning side really is not, because resources are not unlimited. So if you can't (even if admitting that it's faulty) put a number on time and effort, someone else higher up the ladder will do, and then it will hurt you.

I think that toolchains should have better support for estimated vs used time and then learning from that. You could classify tasks by field (DB, frontend, etc) and then calculate correction values based on past failures. Learning from wrong estimates seems to be even harder than estimating, because it hurts to admit that you were wrong, so some automated, blame-less version of it (per sprint or so) might be a good start.

<comment><USER6> The tricky part is that agile is supposed to help those people sleep better. But it all kind of depends on your ability to explain to them how agile does that.

Of course budgets and deadlines has to be set. Agile is not a way to ban those, it's a way to allow them, even if forecasting is impossible. It's about saying, well at that date I can promise to have kept under budget, and will have something that more or less reaches the goal of the budget. It will not, however, be what anyone is imagining right now, let's have fun discovering what it'll be.

</comment></comment></comment></comment></comment><comment><USER7> Funny sidenote: the paper mentions a game titled _Age of Ornithology_ under 5.1.g (Scope). Naturally I got excited at the idea of a game where you play some kind of bird collector, or an intrepid late 19th-century magnifying- glass-and-butterfly-net explorer and cataloguer of birds.

But unfortunately this title is some kind of cruel joke:

[http://www.phobe.com/sfi/ornithology.html](http://www.phobe.com/sfi/ornithology.html)

[http://www.gamasutra.com/view/feature/130925/schandenfreudia...](http://www.gamasutra.com/view/feature/130925/schandenfreudian_slips_age_of_.php)

<comment><USER8> I have a feeling this, and the rest of the games made by "Schadenfreude Interactive" aren't real, as much as I would love to play Grand Theft Ottoman or Accordion Hero, or Nazgul Thunder. Hannibal Crossing looks particularly enjoyable.

<comment><USER9> I saw the word 'Schadenfreude' and was reminded of a satire talkshow on a radio station in the PC game Grand Theft Auto IV

Found it: Pacemaker - putting the heart back into the healthcare industry. [https://youtu.be/jG9BgjnpGGY?t=940](https://youtu.be/jG9BgjnpGGY?t=940)

</comment></comment></comment><comment><USER10> The conclusion:

> Finally, based on our analysis of the data we collected, we make a few > recommendations to game developers. First, be sure to practice good risk > management techniques. This will help avoid some of the adverse e ects of > obstacles that you may encounter during development. Second, prescribe to an > iterative development process, and utilize prototypes as a method of proving > features and concepts before commit- ting them to your design. Third, don't > be overly ambitious in your design. Be reasonable, and take into account > your schedule and budget before adding something to your de- sign. Building > o of that, don't be overly optimistic with your scheduling. If you make an > estimate that initially feels optimistic to you, don't give that estimate to > your stake- holders. Revisit and reassess your design to form a better > estimation.

It's amazing how common-sensical this all his. But then again, we could all use a little more common sense sometimes.

As Charlie Munger would put it: "It is remarkable how much long-term advantage people like [Warren Buffett and myself] have gotten by trying to be consistently not stupid, instead of trying to be very intelligent."

<comment><USER4> When the results look bad it doesn't mean people don't follow common sense. For example, when the project would never get approval and a budget with realism but will get both when risks are left out when asking for money it's common sense to ignore risks. The team has a budget for another year, and what happens then is a different problem. The main objective of employees is not to get projects done, it is to get paid.

Unfortunately our system has created perverse incentives: What manager or entrepreneur can afford to take a step back and say "what I'm proposing may not work, let's stop the project"? We have chosen to work under a pressure as if our lives are at stake (and they are - not our bodies, but our livelihood). The guy selling worthless shit, getting people to buy stuff they don't need, that may even be bad for them (or for those producing them) does not have problems. The same guy voluntarily giving up his job where he would have to do things he thinks are actually bad and/or useless is seen as a "parasite" on society - he's not working and we still have to feed him???

You bet all my estimates show the project is worthwhile if I don't have an (equally good) alternative.

</comment></comment><comment><USER11> Reading this, it's clear that game development is now a mature technology. I was working on physics engines in the late 1990s, when nobody really knew how to do that very well. There were Gamasutra postmortems where the game physics went badly wrong and things flew apart.

In the early years of the PS3, game projects were failing because the Cell architecture (a small PowerPC CPU plus six auxiliary Cell processors with tiny memory) required completely new game engine architectures. That was finally figured out, but development on the PS3 seemed to run about two years late, which set Sony back.

Now there are mature game engines, mature physics engines, the graphics pipeline is in good shape, and the hardware is powerful enough. Problems in game development now are more about the game than the underlying technology.

(Pre-1990s, game development was a cram job, trying to get stuff done on limited hardware.)

<comment><USER12> apparently they aren't though. People porting from PC to other platforms are still finding huge perf issues and have to refactor tons of code. I thought this gen (PS4/XboxOne) were going to be easy ports but from every team I've talked to they aren't

<comment><USER13> A lot of the criticisms I've seen for failed ports describe the exact type of behavior you would expect from an emulated system.

The ports you reference behave like game systems that used to occupy 100% or almost 100% of the resource capacity of a dedicated system failing to execute correctly when they're running in a non-dedicated environment.

They also had the advantage of targeting an exact set of hardware specifications in other ways and didn't have to scale down, up, or sideways (other ways of dividing labor) to the way that other platforms work.

In short, they weren't a game designed to scale to a theoretical platform that then ran on the platforms, they were fit precisely to a single platform and then weren't designed to be portable at all.

<comment><USER14> I think you got that backwards. Parent was talking about porting from PC to consoles.

</comment></comment></comment></comment><comment><USER15> _> When looking at games developed for one platform versus games developed for multiple platforms, we found a few advantages and disadvantages to choosing one over the other. First, 46% of developers that used multiple platforms listed art as something that went right, while only 33% of developers that used a single platform listed this. Additionally, 15% of the developers that used a single platform listed this as something that went wrong, while only 17% of developers that used multiple platforms listed art as having gone wrong. Therefore, the evidence shows us that games developed for multiple platforms may be more likely to have better game art._

I didn't quite understand the correlation between development for multiple platforms and quality of art. May be those who want to release for multiple platforms in general care about quality more?

<comment><USER16> I don't think this as anything to do with the quality of art. Dealing with assets is a pain, dealing with assets on multiple platforms is worse. I'm guessing those dealing with multiple platforms paid more attention to the process. Where those working with a single platform probably didn't think about the issues with at a much. Getting the art right, it's probably more important on multi platform.

<comment><USER15> Do you mean stuff like making shaders for multiple incompatible platforms or all art in general?

</comment></comment></comment><comment><USER12> > Example of poor project management: Leonard Paul of Moderngroove stated that > Modern Groove – The Ministry of Sound Edition suffered from poor project > management because they had no dedicated project manager. Most of those > responsibilities were given to the lead programmer, which resulted in an > over burdened team. Paul said "I believe hiring a good manager early in the > project and having clearer deadlines would have definitely helped the > project."

> Takeaway: In order to avoid conflicts during the development process, teams > need to have proper management.

This seems like the exact opposite of Naughty Dog's advice

[http://www.latimes.com/business/technology/la-fi-tn- naughty-...](http://www.latimes.com/business/technology/la-fi-tn-naughty-dog- uncharted-20160523-snap-htmlstory.html)

</comment><comment><USER17> >utilize prototypes as a method of proving features and concepts before committing them to your design. Third, don't be overly ambitious in your design. Be reasonable, and take into account your schedule and budget before adding something to your design. Building off of that, don't be overly optimistic with your scheduling. If you make an estimate that initially feels optimistic to you, don't give that estimate to your stakeholders. Revisit and reassess your design to form a better estimation.

</comment><comment><USER18> This is a strange paper to read because I think I have read most of the source material, something I certainly haven't felt before in a formal research paper! I was hoping that it would be more from the context of the publisher working with Microsoft PC/XBOX developers for new information though. Oh well, maybe next time.

</comment><comment><USER19> I wonder how long before this kind of analysis can be performed by AI just as efficiently and will be offered as a web service.

</comment><comment><USER20> This is excellent for the history and for the examples of what worked and what didn't. But the interpretations are a little dubious:

 _Takeaway: Game developers should create a well-defined concept before beginning development as opposed to an ad hoc method of game design._

This would exclude Dota, for example. Gamedev is a world of exceptions.

In general, it's best to playtest every idea you come up with and cut whatever isn't fun.

<comment><USER21> This advice is super generic.

<comment><USER8> Sometimes you start game development with a single mechanic and see where that takes you. Like "what if you could control gravity" and that might lead to VVVVVV or others. Or it might crash and burn. The advice is to encourage developers to flesh out ideas and vet them rigorously before starting development.

<comment><USER22> Sometimes it's complete serendipity. The Rocket League creators were originally making a deathmatch-with-cars mod for Unreal Tournament. Then someone dropped a giant ball into the map, and they realised 'car soccer' was a much stronger concept.

</comment></comment></comment></comment><comment><USER23> Uh? It's not software eng or art but too many lions competing for too little a pie. CRUDs' marginal return is much better again (or as always).</comment>