All Late Projects Are the Same (2011) [pdf] - <AUTHOR> http://www.systemsguild.com/pdfs/DeMarcoNov2011.pdf <comment><USER2> This is my favorite part:

 _If a project offered a value of 10 times its estimated cost, no one would care if the actual cost to get it done were double the estimate. On the other hand, if expected value were only 10 percent greater than expected cost, lateness would be a disaster. Yes it would be a disaster, but instead of obsessing over "What's the matter with those software folks who didn't deliver on the schedule we gave them?" we need to ask instead "Why did we ever kick off a project with such marginal expected value?"_

The message for us software developers is: try to work only on what I call _strategic_ projects: projects whose expected value is much greater than their cost could possibly be. That means they need to continue to pay benefits well into the future.

<comment><USER3> At a previous gig "strategic" was a euphemism for "we don't have a paying customer for this work" :-)

</comment></comment><comment><USER4> The first thing that comes to mind is Joel's Five Worlds

[http://www.joelonsoftware.com/articles/FiveWorlds.html](http://www.joelonsoftware.com/articles/FiveWorlds.html)

not all software dev is the same.

Coming from games, every project I've been on was an original title. That meant it was 100% up to the team to decide the scope of the game. Most teams (not all) guess how much they can get done and don't plan for cutting features. "We're going to have 12 characters, 30 levels, 10 power ups, 45 enemies, 600 puzzles, online multiplayer, and IAP for 500 items". And then they just start making it. They run into 2 big problems.

1) They don't know if the game will be fun until they're 50-60% in. If they guessed wrong they're off schedule

2) They're attached to the vision and unwilling to cut stuff.

My point though is this doesn't fit "we didn't start early enough" IMO. They knew how much time they had. No one was/is dicating too much work. They can cut the scope at any time. But they rarely do.

There are solutions. 1 is to prototype with a small team. Unfortunately most teams can't do that as they'd have 90% of their staff doing nothing. Larger companies can do this or maybe if they switched to the Hollywood model, no employees only contractors, they could do it.

2 is to design so things can be cut and to check the schedule and start cutting. This one is hard for most teams because they see the game they imagined in their minds and it's not the game with stuff cut. Players though don't know what that vision is and so won't missed the cut features because they aren't aware of them.

<comment><USER5> > My point though is this doesn't fit "we didn't start early enough" IMO. They > knew how much time they had. No one was/is dicating too much work. They can > cut the scope at any time. But they rarely do.

Doesn't knowing how much time you have, and then not cutting scope when you see your plans don't fit into that time, count as not starting early enough?

I also came from games, and I agree that scope is rarely cut, and often self- inflicted. I was also under Mr. Very Large Publisher, and it was more complicated than that- we had control over scope, but not over schedule. We had control over how much work to do and the quality, but not the budget.

The major problem in my experience was that the team was excited, motivated and willing to work hard, so optimistic about their ability to do great things inside the limited budget and schedule. Therefore, they tried to do too much. All other things being equal, more time would have helped, every single time. Which can be summarized as "we didn't start early enough." That doesn't mean that actually starting earlier would have solved the problem. Actually starting earlier might have led to even larger plans. So, IMO "we didn't start early enough." is not meant to be taken literally, it's a way of understanding the problem, not a way to solve the problem.

<comment><USER4> You can use "we didn't start early enough" in such a way that it loses all meaning.

If you know your deadline (most game devs do) and your 100% in control of your scope then it's up to you (the team) to pick a scope that fits your resources (people and time).

The original article talked about situations where people didn't know the scope or couldn't change it and often wasted lots of time not starting. I think those are fundamentally different situations.

<comment><USER5> You're absolutely right that games are in a generally different situation than a lot of other software projects; having control over scope means, in theory, you don't ever have to be late. And yet it happens routinely, possibly (I speculate) more often than not.

The article implied once that people wasted time waiting to start, in reason #1: "Nobody had the guts to kick off the project until the competition proved it doable and desirable."

But reason #2 is where games that have a deadline and scope control fit in. "If the project were started long enough before its due date to finish on time, all involved would have had to face up to the fact from the beginning that it was going to cost a lot more than anyone was willing to pay."

You picked a scope that fits your resources, and yet you manage to run well past the deadline. You started it thinking you'd get it done in time, and yet you managed to run late. _If_ you started earlier with the same deadline, scope, people, etc., you would finish on time. This might feel like a somewhat silly observation (which the author points out in advance) but it is nonetheless true for the situations you are describing. If you didn't finish on time, then you didn't start early enough. It's a slightly twisted way of saying your scope was bigger than you thought. It's not saying that actually starting earlier is the solution.

Reason #3 is entirely metaphorical. "No one knew that the project needed to be done until the window of opportunity was already closing." This is saying that you know your scope, and you have a deadline, but you didn't even think of the idea until it was too late. It is true to say that you didn't start early enough, but it is also true that you couldn't possibly have started it earlier, without a crystal ball.

The thrust here is not literal. I disagree that it loses any meaning at all, it just might not mean what you thought.

</comment></comment></comment></comment><comment><USER6> "1\. Nobody had the guts to kick off the project until the competition proved it doable and desirable; by then, the project was in catch-up mode and had to be finished lickety-split.

2.If the project were started long enough before its due date to finish on time, all involved would have had to face up to the fact from the beginning that it was going to cost a lot more than anyone was willing to pay."

In other words, projects are late because business justification is usually weak for software projects. When faced with "in order to ship this by 12monthsfromnow we need to hire 5 more software engineers today" management stalls.

<comment><USER7> Adding more manpower doesn't mean faster completion.

<comment> loup-vaillant At the beginning of a long project, it probably does. This is not like adding people to a late project.

Now if your project isn't easily separable in several relatively independent parts, you _are_ screwed.

</comment></comment></comment><comment><USER8> > "What's really wrong with us software folks is that we're continually > beating ourselves up for something that's somebody else's fault."

Amen to that.

</comment><comment><USER9> "All projects that finish late have this one thing in common: they started late"

<comment><USER10> Yea - I'm not really sure what the take away message here should be (although I found the article interesting).

I'm currently working at a late project right now that started early enough, but took too long to hire enough people to finish the project in time, and didn't use the time at the beginning of the project well enough to gather information about the current and future processes. Maybe that's an example of what the author means, but it certainly gets more complicated than "started too late" quite quickly...

<comment><USER11> The message is "late software" is really a failure of decision making at the highest level that waffle on necessary things until they become necessary but move too late on it. I've personally seen plenty of that, so works for me.

</comment></comment></comment><comment><USER12> Longer treatment of similar material by same author:

[http://www.goodreads.com/book/show/946076.Why_Does_Software_...](http://www.goodreads.com/book/show/946076.Why_Does_Software_Cost_So_Much_)

</comment><comment><USER13> Having worked in firmware, I have seen the issue with how things are partitioned between software and hardware. The hardware tends to get concrete requirements and everything else abstract or complex in dumped on the software. So you have to figure out algorithms which have not been thought through yet or deal with edge cases the hardware cannot deal with properly. These all add complexities to the software which we can't quantify early on. I believe if we specced out the software design to a level we are confident with the algorithms and data structures involved, you could get as predictable a schedule as hardware design.

</comment><comment><USER14> I think almost by definition anything that is late didn't start early enough.</comment>


iPhone 7 announced with water resistance, dual cameras, and no headphone jack - <AUTHOR> http://www.theverge.com/2016/9/7/12758236/apple-iphone-7-announced-features-price-release-date <comment><USER2> I know this will change within the hour, but I'm watching the apple keynote and they haven't said a thing about the iPhone yet...

<comment><USER3> Apple fudged and dropped it early on Twitter from what I've heard.

</comment></comment><comment><USER4> @Apple account accidentally leaked it on Twitter, it hasn't yet been announced in the keynote.</comment>


Pasukon – The easy JavaScript parser generator - <AUTHOR> https://pasukon.rocks/ <comment><USER2> I like this a lot!

A small critique: it might be useful to include a small explainer of why parsers / ADTs are useful - maybe in the context of decoding JSON.

(Learning about ADTs shifted the way that I think about data modelling and application development, so I am a huge fan.)

<comment><AUTHOR> Great point. I'll ellaborate on it in the readme and/or website :D Thanks for the feedback!

</comment></comment><comment><USER3> This looks cool but the "easy to learn" intro statement set a disappointing expectation. Building up some simple examples would _really_ help.

I mean this as constructive criticize, apologies if it feels too harsh.

<comment><AUTHOR> Yeah I really need to work on the docs. I want to write a "Getting Started" guide for someone with absolutely zero background on the topic and put it in the repo's Wiki.

Did you see the GitHub repo's README? I do ellaborate a bit over there. Do you think I should explain the basics in the website too? Or maybe just link to the wiki once I have that "Getting Started" guide up?

</comment></comment><comment><USER4> This looks like a nice simplification of yacc/bison and lex/flex. The `then` keyword seems superfluous though; what was the motivation for `then` as opposed to just juxtaposition?

<comment><AUTHOR> Thanks :) The reason it looks like that is because I need a way to distinguish binary calls from unary calls: `a b` would call the combinator `a` with the parser `b` as input. `a b c` will call the combinator b with a and c as input.

I certainly could have chosen another syntax, but I settled for that one. Because the user can define their own combinators I can't just hard-code all combinators into the syntax.

For example `*:A` is alias for `(many0 (token A))` but if the user defines his/her own unary combinator, I need a way to let him/her call it without changing the syntax.

</comment></comment><comment><USER5> Good stuff. I have 2 questions:

1\. Is there a Pasukon grammar for JavaScript/ES6?

2\. Is there a Pasukon grammar for Pasukon grammars?

<comment><AUTHOR> 1\. Nope, I still need to make some more example grammars :)

2\. Yup: [https://github.com/gosukiwi/Pasukon/blob/master/lib/grammar....](https://github.com/gosukiwi/Pasukon/blob/master/lib/grammar.pasukon)

<comment><USER5> I find (2.) rather interesting. A parser than can parse its own grammar definitions. A language that can define itself.

<comment><AUTHOR> Yeah it's pretty cool, if I say so myself :) It parses itself (using a pre- compiled AST) to parse other user-defined grammars.

Initially I started using PEGjs, and then swapped once it was ready.

</comment></comment></comment></comment><comment><USER6> Does the term "substraction" mean anything in particular? It's kind of funny to have the same typo four times on the main page... I guess it compiles either way. b^)

It looks like a great package. I have loved parser combinators forever.

<comment><AUTHOR> It's a typo I have, Spanish being my native language, I always get confused with that one :) Just pushed a new version with a fix

</comment></comment><comment><USER7> Similar tool: [https://github.com/DmitrySoshnikov/syntax](https://github.com/DmitrySoshnikov/syntax)

It can emit parser in many languages.

There are also a couple of udemy courses from the author, about parsing and language creation. They are quite good imho.

<comment><AUTHOR> Impressive!

</comment></comment><comment><USER8> Having written my own CommonMark parser, I often wonder if I should have used some kind of parser generator. Can I use this to parse CommonMark (in 100% compliant way)? – And, can I parse it such that I can recreate the original Markdown based on the resulting model?

<comment><AUTHOR> I think Pasukon should perform similarly to a hand-written recursive descent parser. If you already have something going, I don't see much benefit in porting it, besides ease of creating and maintaining the parser.

It can parse any context-free language, so it should have no problem parsing CommonMark.

</comment></comment><comment><USER9> Awesome! I've been wanting to add a proper parser to my Scheme interpreter for a long time now, and this might lower the barrier enough to motivate me to actually do it.

<comment><AUTHOR> Awesome, do it! Making languages is always fun :)

</comment></comment><comment><USER10> I am pretty happy with the parser from MS: [https://github.com/microsoft/ts- parsec#readme](https://github.com/microsoft/ts-parsec#readme) the API is pretty intuitive and it the package is reliable Can't imagine why I would want to switch to PASUKON(what an odd name by the way).

<comment><USER11> I believe the name is a play on

a) romanization of the Japanese pronunciation of PC ("personal computer" -> "PersCom" Japanese abbreviations of English tend to concatenate the initial syllable of every word, and "n" and "m" end up sounding very similar).

b) "parser combinator" would have the same abbreviation in Japanese

<comment><AUTHOR> Exactly that :) Glad someone got it! Heh.</comment></comment></comment>


Why the Scientist Stereotype Is Bad for Everyone, Especially Kids - <AUTHOR> http://www.wired.com/wiredscience/2012/06/opinion-scientist-stereotype/

<comment><USER2> I agree - the main problem is that basically no one has an idea what the day- to-day life of a scientist looks like. The outreach programs of Fermilab et.al. for school children are nice, but I'd be very interested to hear how to get in touch with working professionals to show them that

(a) scientists are not dangerous and (b) there is work done in the ivory towers that might have a positive impact in their field</comment>


Sales Guy Looking for Technical Co-Founder - <AUTHOR>

Just an idea I had for a sentiment value analytics program.
How do I go about finding a Technical Co-Founder for my idea?
I know this is a very genreal question but I hope to get some valuable insight.
Thank you! <comment><USER2> The other comments are solid, and this is one of those questions whose answer probably won't help much in solving your problem (due to its nature, not for a lack of good answers). However, I can offer you some time and insight if you want a technical person in the ML/NLP domain to bounce your ideas off of before you get too far into it. Contact information is on my profile, if you so wish.

<comment><AUTHOR> Thank you I will definitely be in contact. Thank you!

</comment></comment><comment><USER3> This also:

[http://founderdating.com/](http://founderdating.com/)

Whenever this gets asked, someone always mentions the YC concept of how to find a co-founder.

It should really be:

\- Someone you know \- Someone you've possibly worked with \- Someone you may even consider a friend

Check more on the PG/YC blogs for more info on this.

<comment><AUTHOR> I definitely agree with this model, and before i work with anyone I would definitely have to do some befriending. The only problem, none of my friends have a passion for business, or have the technical skills for it.

I really appreciate your input.

</comment></comment><comment><USER4> This might be what you're looking for..

[https://www.cofounderslab.com/](https://www.cofounderslab.com/)

<comment><AUTHOR> Thank you so much!

</comment></comment><comment><USER5> Instrument your passion for making your idea real. Write some code. Start with tests.

</comment><comment><USER6> What's your idea?

<comment><AUTHOR> In short form- an analytics software that analyzes what people are saying and the meaning. yes, i know there many variables and algorithms and it will never be 100% AND there is a way to monitize the data

<comment><USER7> Gathering what people are saying is relatively easy. Figuring out the meaning accurately is extremely difficult, and as you add longer text the difficulty grows.

Something like Siri wasn't developed in a void, the technology behind it took quite a long time, with many millions of dollars, and that is only aimed at "understanding" short sentences.

What you're asking for might cost a pretty penny to develop, it isn't just a "one guy in a basement for a long weekend" type of concept, it is a legitimately hard Computer Science problem with no easy outs.

If you were able to scan social media and extract WHAT people are talking about then it is definitely a monetizable idea. However from zero through to profit is a very long ride.

Maybe contact the NSA and try to negotiate re-using their in-house technology?

<comment><AUTHOR> I really appreciate the lengthy response, it really means a lot and it helps a lot getting insight from the tech world. It definitely is not an easy thing to do, but then what is right?

I have found some open source algorithms that claim to have a decent amount of accuracy, though decent isn't enough it may be a start. What is your opinion?

Going as far as contacting the NSA, I don't know how willing they are in sharing their technology. I know they do work with private corporations which benefit both parties, I guess it would be worth a shot. That would be an interesting approach.

How long do you think it would take to build something like this? Even if we could just get sentiment values ie. Positive / neutral / Negative comments, that would be a great start and I believe much simpler.

<comment><USER8> I do have good advice on how to find a tech co-founder, but this is the hard kind of idea for you to start as a non-tech. You would need a top level engineer on a niche domain. If he/she decide to do it, the tech challenge would be so much harder than the non-tech challenge, that he/she wouldn't need you at all as a co-founder.

I think you should pursue more a not so high tech dependant idea, like: Airbnb, Groupon, Yelp, this sort of thing.

This might apply to you: [http://xkcd.com/1425/](http://xkcd.com/1425/)

EDIT: I think the NSA part was just a joke.

</comment></comment></comment></comment></comment><comment><USER9> Where are you based?

<comment><AUTHOR> Canada, Ontario Toronto :)</comment></comment>


Google founders couldn't spell Googol? - <AUTHOR> http://en.wikipedia.org/wiki/Googol

<comment><USER2> Um no. They could and that is what they wanted to name Google _originally_

Off hand, I don't remember who it was, but their first investor misspelled the name as Google on the check and they just took that as the name.

</comment><comment><USER3> Bullshit, that domain was simply already taken. Googol.com was registered in 1995.

</comment><comment><USER4> The average person would have spelled Googol as "Google", so it was the natural choice.</comment>


Tale of Eternity (Pt.2): Private server finances, and first launch - <AUTHOR> http://shenglong.posterous.com/the-tale-of-eternity-part-2

<comment><AUTHOR> Someone posted a comment but seems to have redacted it. Anyway, I do have analytics, but not for this period. I didn't realize tracking people would be a good idea until later. Here's a map overlay... I'll compile more stats and post them some other time.

<http://i51.tinypic.com/rkycqt.png></comment>


YouTube self-defence - <AUTHOR> https://hakon.gylterud.net/tutorials/youtube.html <comment><AUTHOR> I wrote this tutorial after I recently started watching YouTube videos connected to my hobbies more regularly, and found that I needed to protect myself from all the clutter and distractions.

This could also be useful for making YouTube more child-safe. As per the discussion yesterday: [https://news.ycombinator.com/item?id=19446211](https://news.ycombinator.com/item?id=19446211)</comment>


A New Design for Cryptography's Black Box - <AUTHOR> https://www.quantamagazine.org/20150902-indistinguishability-obfuscation-cryptographys-black-box/ <comment><USER2> We've come a long way since the first IO result came out. Since then, we've gotten a couple more multilinear map candidates (though most are now broken), and some simpler constructions, but we're still really far from IO with a proof. This is primarily because of the underlying multilinear map that's being used. The Gentry et al result that proves IO secure in the generic multilinear model isn't that useful yet simply because there have been so many nongeneric attacks against mmap candidates, especially when they're used in IO. That is, at the moment there's no reason to believe that the generic multilinear model is even a good way to think about IO security.

What would be a really big result is finding IO that doesn't rely on multilinear maps.

</comment><comment><USER3> Is this about data encryption, or is it about hiding the inner workings of an executable binary (malware packing/copy protection)?

<comment><USER4> what's the difference? ;)

<comment><USER5> To expand on that: what the article is basically talking about is "DRM that actually works"—the ability to send someone some encrypted data embedded in a wrapper program. You can run the wrapper program and interrogate it all you like on its own terms—but other than satisfying the desire of the wrapper program's code-paths, there's no way to get decrypted data out.

If the data is dumb content like text, this amounts to regular DRM content encryption, except that there's no decryption key to be found in the wrapper program or anywhere else; the key is "baked into" the logic of the program in a non-recoverable way. (This would allow for things like "true" TPM chips, that can store your keys opaquely from forensic recovery.)

If, on the other hand, the data is itself a program for which the wrapper serves as an interpreter, this amounts to a mathematical basis for a real "Trusted Computing Base", enabling any manner of things, like simple distributed computation on untrusted hardware, or mathematically-strong anti- cheating protection for an MMO game, or satisfying cell carriers' desires for a protected "baseband processor" under their control without that needing to be instantiated as a physical chip.

Effectively, creating a wrapper VM (the "bootstrap program" in the article's terminology) would allow a processor to run a "binary" through the VM that is literally opaque to it; code that, even in its operation _as instructions on the CPU_ , the CPU is incapable of comprehending or interfering with (beyond simply terminating/interrupting the wrapper VM, or restricting its hardware access.) Not only would the interpreted program's code itself be opaque; the working state—the contents of the wrapper program's memory (and the processor's registers, and whatever else) would be opaque. The only place you could see such a program's intent realized would be in the IO it does—and that might be just encrypted network traffic sent to peers, too.

Such a software process, if given a full CPU hypervisor slot rather than having to make system calls to an OS, would be for the first time a "first- class citizen" on a computer, functioning more like[1] a flashable FPGA coprocessor connected to the CPU than a series of instructions that the CPU can edit to its whims. The CPU could _ignore_ such a coprocessor—choose to not interact with it or power it (not emulate it, in other words), or tell the IOMMU to remove the coprocessor's access to peripherals, etc. But the CPU couldn't reach inside the coprocessor to fiddle with it, even though it's a virtual coprocessor residing entirely within "the mind of" the CPU. [The CPU _could_ arbitrarily corrupt the memory the coprocessor was using for its state—but with good encryption, that would just immediately crash the wrapper VM with an assertion failure, rather than leaking any info.]

\---

[1] Note that this is _just_ an analogy from the CPU's perspective; we already have flashable coprocessors, but that doesn't help us any, because while the CPU can't poke into them, people can. Indistinguishability Obfuscation means that _we 're_ in the position the CPU is in; we can no more see into the VM or its state than the CPU can reach over and take apart a coprocessor.

<comment><USER4> There are "good" and "bad" uses for this technology.

Bad uses: Netflix will put their video stuff into this and now you will never jack content from their software.

Good uses: Your IM and email can live in this and no compromise of your host operating system can leak your information. Your computer can be hacked by every hacker on Earth simultaneously and your secrets are safe.

<comment><USER6> > Netflix will put their video stuff into this and now you will never jack > content from their software.

Yep. It is good I cannot record what is on the screen. Or in the video buffer.

</comment></comment></comment></comment></comment><comment><USER7> I remember pretty clearly reading a comment by the author of the paper about the 'unbreakable obfuscation' in which he said that the paper was greatly misrepresented in that it had made a proof in a specific problem domain that wasn't so applicable to real software.

I'm pretty sure it was posted on HN at some point. I don't remember the term IO being used, so it may have been a different kind of obfuscation. There were some allusions made to an unsolvable jigsaw puzzle.

</comment><comment><USER8> I'd like to note that IO does not give a guarantee of impossibility of extracting keys.

AFAIK, the definition of IO is: we have two programs that perform the same computation. After we apply IO to both programs, we cannot figure out which obfuscated program corresponds to a particular original program.

However, there is a flaw: programs encrypting data with different keys are performing _different_ computations.

So IO definition does _not claim_ that IO is able to hide the key.

<comment><USER9> >> So IO definition does not claim that IO is able to hide the key.

From what I've read, that doesn't even matter. The obfuscated program IS effectively the key. A copy of that obfuscated program is still a copy of the key. It's still not clear to me what the advantage is supposed to be.

<comment><USER10> The obfuscated program only uses its embedded key in ways it sees fit. You cannot sign arbitrary statements with the key.</comment></comment></comment>


Ask HN: In JavaScript, changing the value of an array item not advised? - <AUTHOR>

I'm reading through jQuery Fundamentals (http://www.rebeccamurphey.com/jqfundamentals/#javascript-basics.examples.change-array-value) and this tip in the JavaScript chapter surprised me: While it's possible to change the value of an array item as shown in Example 2.28, "Changing the value of an array item", it's generally not advised.
I wouldn't think just changing the value of an array element could cause a problem.  Is this truly inadvisable or was this written in error? <comment><USER2> It shouldn't be a problem with strings and numerics in arrays (the array contains copies of the values), but with arrays of functions/objects, the elements of the array are not copies, but the actual objects themselves. So if you change an object in the array, the object itself changes (and vice versa).

<comment><AUTHOR> Thank you.</comment></comment>


Oculus VR release a latency tester for the Rift - <AUTHOR> http://www.oculusvr.com/blog/latency-tester-pre-orders-now-open/

<comment><USER2> Not sure if the Oculus guys are reading here, but it would be interesting to get just a little bit more information on the technical side:

\- what kind of sensor is it and why does it have to be a color sensor, vs a much cheaper plain photodiode which as far as I know works equally well for this task i.e. thresholding

\- what is the samplerate used, just out of curiosity

\- where is the sensor placed with respect to the screen - for a normal 'scanning' LCD display the right bottom pixels of the screen are presented about a frame later, so this seriously affects measured latency

</comment><comment><USER3> I've used the oculus rift before, and I could only use it for about 5 minutes before I would become nauseous. My experience if vertigo differed depending on which game I was playing. Also the low resolution was a lot more distracting than I thought it would be. I hope they improve some of these things before launch otherwise I would fear that this may be another video game fad.

<comment><USER4> They've already sent a high-res prototype to some developers, and they have an even higher one in development. So that problem should be fixed quickly.

<comment><USER3> That will be great when they do. With the graphics expectations gamers have these days, low resolution graphics are pretty distracting.

</comment></comment></comment><comment><USER5> I was surprised when I tried the included demo (tuscan village or something). The latency was really bad. I thought they had a big focus on bringing down the latency though.

Could I have had a defective unit? Or does it matter what PC it's running on, etc? Is it normally really good?

<comment><USER6> The PC could certainly be a cause. Some video card drivers, for example, deal poorly with multiple displays or secondary displays, and add an extra frame or two of buffering.

Having a device like this certainly improves testing the PC side.</comment></comment>


Schiaparelli landing investigation makes progress - <AUTHOR> http://www.esa.int/Our_Activities/Space_Science/ExoMars/Schiaparelli_landing_investigation_makes_progress <comment><USER3> What is fantastic here is that the telemetry was received for nearly the entire descent, up to a few seconds before the crash.

The could reproduce the exact same scenario in simulation by replaying the telemetry as inputs to their firmware and see what happened in the software. This is tremendous advantage for debugging, it's like a wireshark replay of network packets for debugging an error in the TCP stack.

This orbiter/descender architecture is really paying off for future missions. If the orbiter hadn't been there, earth wouldn't have received any telemetry at all.. Let's hope that this "real world" telemetry is saved for simulating successive versions of the guidance software of future probes.

<comment><USER4> I've worked on sensor fusion systems and I fully agree. Replaying the sensor inputs and evaluating new estimated state is a really good way of debugging failures (because you can't just stop the system mid-air and evaluate internal state). It also helps with regression test suite and trying out new algorithms quickly.

<comment><USER5> We'd do the same for debugging networked games. Incredibly useful.

If you build it right you also get a replay feature for almost free.

</comment></comment></comment><comment><USER6> > the erroneous information generated an estimated altitude that was negative > – that is, below ground level.

Kind of sad to see that computers nowadays still lack what we call common sense. I mean the machine received negative altimeter data but if it had been able to just look around, it could have seen that this data was erroneous. Hell, considering the descent profile was carefully prepared, the computer should also have noticed that it was way too early to reach the ground and should have concluded that something was wrong with the altitude measure.

<comment><USER7> Well, common sense in a way would be enabled with assertions. But even if you know that you cannot be below ground, how would you recover? Your sensors clearly tell you that you're below ground, so what is the course of action? You have no way of knowing where you are, you just know that you don't know. It doesn't really change anything in the outcome. Sure, maybe you could go from altitude measurements to time measurements and most likely still leave a crater since it's not exact enough (there's a reason for the internal measurement instead of simpler ways, after all).

It would also require you to anticipate the malfunction in which case you already know that things like that could happen, in which case you'd likely determine how it could happen and how to avoid it and continue having useful data. Sounds to me like a better solution than trying to recover without data.

<comment><USER8> There are several ways. First off, ignore the erroneous data. Then, estimate the altitude by other methods. As you mentioned, one way would be to integrate time and velocity.

All inputs and outputs should be run through sanity checks, with a plan for when they go wrong.

Essentially, when you've got a system that can't be fixed, plan for failure of each and every component (in isolation).

<comment><USER9> Cars do this, you can kill many sensors and an engine will still limp even with misfires. I guess consulting Bosch isn't an option tho

<comment><USER10> It helps that car engines don't really _need_ sensors to function. Cars existed before electronic sensors. A mars lander actually needs to know when to deploy parachutes.

<comment><USER11> You'll find that fuel injectors don't work well without input from the MAF and O2 sensor(s).

<comment><USER8> True of modern EFI, but older mechanical fuel injectors did not have O2 sensors.

</comment></comment></comment></comment></comment></comment></comment><comment><USER4> While it may sound that the failure was caused by a rookie mistake- I'm sure the on-board systems were designed and developed by people who know what they are doing.

It must be hard for all the people involved in Schiaparelli development.

Sometimes I wonder whether majority of software errors are caused only by multiple layers of people having incorrect / unchallenged / unjustified (and mostly implicit) assumptions about something.

<comment><USER12> > I'm sure the on-board systems were designed and developed by people who know > what they are doing.

I have been working in safety-related domains, where each failure could involve the death of several hundred people. It wasn't pretty: everything was sub-contracted to death, and there were a minority of people who really knew what they were doing, a majority who was so-so and didn't give a fuck, and still a fair part who was notoriously incompetent. So, the minority of competent and concerned people cannot always make up for the others, and when themselves fail, there is almost no one to make up for them. Oh, and I was working in a company that was supposed to produce better quality than others. And it did :-(

So, for something like ESA where life is not even concerned, I can't imagine it is any better if stuff is outsourced, and since nowadays almost everything everywhere is outsourced, I assume ESA does it too.

> Sometimes I wonder whether majority of software errors are caused only by > multiple layers of people having incorrect / unchallenged / unjustified (and > mostly implicit) assumptions about something.

Yes and no. This causes a category of mistakes but on the other hand, it prevents another category. When someone knows the full system, he makes a whole lot of assumptions: it doesn't matter if this function cannot handle this or that case because it will never be called this way, it cannot happen, so I won't handle those cases and I won't even check them. And then, later, something changes in an upper function or system, the assumption is not valid any more, and boom.

When you have no idea what the system is, you just stick to the function definition and have it handle all the weird cases. You don't make assumptions. You can still make mistakes, though, some of them caused by the lack of global understanding of the system because the specification writer had those assumptions in mind and did not write them down in the specification of this function because they were obvious to him or they were already written in another part of the specification.

<comment><USER13> > And then, later, something changes in an upper function or system, the > assumption is not valid any more, and boom.

That's exactly how Ariane 5 crashed in 1996:

[https://en.wikipedia.org/wiki/Cluster_(spacecraft)#Launch_fa...](https://en.wikipedia.org/wiki/Cluster_\(spacecraft\)#Launch_failure)

"Specifically, the Ariane 5's greater horizontal acceleration caused the computers in both the back-up and primary platforms to crash and emit diagnostic data misinterpreted by the autopilot as spurious position and velocity data. Pre-flight tests had never been performed on the inertial platform under simulated Ariane 5 flight conditions so the error was not discovered before launch. During the investigation, a simulated Ariane 5 flight was conducted on another inertial platform. It failed in exactly the same way as the actual flight units."

<comment><USER14> I think it's also important to note that the inertial platform was developed for the Ariane 4 where it worked correctly.

The software was actually developed correctly, and functioned as intended. At least for it's intended use. Then it was tossed at a new use-case without any accounting for any differences in the new situation.

<comment><USER13> > The software was actually developed correctly

Not quite. If you read the details about the case you can find that it didn't have the handler for the overflow in the calculations(!) It's similar to this case now that both were developed with under the assumptions "can't happen," in the sense, developed to be _too brittle_ , for the inputs that were certainly possible to happen as soon as the trajectory (in the case of Ariane 5) or the duration of the spinning movement (this case now) doesn't match their initial test cases.

Still, the development, especially in this kind of projects, is always a balancing act to organize covering most of the cases that can go wrong. Murphy's law works against the whole organization. Given the amount of real problems, I'm still amazed that the Apollo 11 succeeded.

Or even that there weren't any really destructive "accidents" involving rockets with the nuclear warheads. Think about it, these are prone to the same problems any other computer-related projects are: the amount of the damage is effectively infinitely larger than the effort needed to start it.

[https://www.theguardian.com/world/2016/jan/07/nuclear- weapon...](https://www.theguardian.com/world/2016/jan/07/nuclear-weapons-risk- greater-than-in-cold-war-says-ex-pentagon-chief)

"These weapons are literally waiting for a short stream of computer signals to fire. They don't care where these signals come from."

"Their rocket engines are going ignite and their silo lids are going to blow off and they are going to lift off as soon as they have the equivalent of you or I putting in a couple of numbers and hitting enter three times."

[http://thebulletin.org/](http://thebulletin.org/)

"It is 3 minutes to midnight"

Also: "How Risky is Nuclear Optimism?"

[http://www-ee.stanford.edu/%7Ehellman/publications/75.pdf](http://www- ee.stanford.edu/%7Ehellman/publications/75.pdf)

And if you still think "but it works, the proof is that it hasn't exploded up to now", just consider this graph from Nassim Taleb:

[http://static3.businessinsider.com/image/5655f69c8430765e008...](http://static3.businessinsider.com/image/5655f69c8430765e008b57c8-1200-900/taleb- turkey.png)

<comment><USER14> > Not quite. If you read the details about the case you can find that it > didn't have the handler for the overflow in the calculations(!) It's similar > to this case now that both were developed with under the assumptions "can't > happen," in the sense, developed to be too brittle, for the inputs that were > certainly possible to happen as soon as the trajectory (in the case of > Ariane 5)

I'm not sure that's entirely fair. The software was intended for the Ariane 4 which wasn't intended to have as much horizontal acceleration as the 5. If the 4 had experienced such an acceleration it wasn't intended to be capable of recovering from it. That area of the code also explicitly had some protections provided by the language removed for the sake of efficiency. So it wasn't a total oversight that just happened to work out - there was a decision made based on the fact the rocket had already irrecoverably failed if the situation ever occurred.

While I agree it's somewhat distasteful not to cover all the bases in the most technically correct way all the time, I'm not sure how important it is to have an overflow handler fire in the inertial reference system just as the rocket self-destructs.

<comment><USER13> > That area of the code also explicitly had some protections provided by the > language removed for the sake of efficiency

As far as I know the efficiency wasn't the issue, just that the "model" was, as I've said, brittle. The overflow was to be handled with what we'd today call "an exception handler" and the selected solution was, instead of (reasonably) writing "keep the maximum value as the result" handler, to leave the processor effectively executing random code in the case the overflow occurs. And the "exception" occured. It's not that the overflow detection was turned off to save the cycles, or that some default handling was provided. It was that it was handled with "whatever" (execute random instructions)! by intentionally omitting the handlers.

<comment><USER14> I don't really see that as the main point. Perhaps I shouldn't have mentioned it at all.

I don't see the practical issue with a model being brittle in the face of imminent mission failure. The model breaking down shortly before you self- destruct the whole thing seems like a rather minor concern. It's entirely irrelevant at that point what the model is.

It turns into an issue when somebody throws the software into a new environment without looking at it or it's requirements and then doesn't do any testing with it. But that's not on the original developers. Their solution was entirely valid for their problem.

Even if they had done something like report the maximum value instead, the rest of the software for the Ariane 5 could well have been expecting it to do something else entirely which would still result in a serious problem.

It's an issue of inappropriately using software in a new situation. Without knowing and account for how it behaves, you can't just use it and expect everything to work perfectly the first time around. It doesn't matter how well the software accounts for various issues - at some point something won't have only a single correct answer and the software you are using will have to pick how to behave. If you aren't paying attention to that, it can/will come back to bite you.

<comment><USER13> > It doesn't matter how well the software accounts for various issues - <USER2> > some point something won't have only a single correct answer

It does, immensely. That's why we have floating point processing units instead of the fixed point. Think about it: even the single precision FP allows you to have "expected" responses between 10E-38 to 10E38. There are less stars in the observable universe. The double precision FP allows the ranges of inputs and outputs to be between 10E−308 and 10E308: there are only 10E80 atoms in the whole observable universe. Can the response which says how much the rocket is "aligned" be meaningful -- sure it can.

This piece of program catastrophically failed because some input was a just somewhat bigger than before.

Properly programmed components that are supposed to handle "continuous" inputs and provide "continuous" outputs (and that is the specific part we talk about) should not have "discontinuities" at the arbitrary points which are the accidents of some unimportant implementation decisions (leaving "operand error" exception for some input variables but protecting from it for others!).

I can understand that you don't understand this if you never worked in the area of the numerical computing or signal processing or something equivalently part of the "real life" responses, but I hope there are still enough professionals who know what I talk about.

Again from the report:

"The internal SRI software exception was caused during execution of a data conversion from 64-bit floating point to 16-bit signed integer value. The floating point number which was converted had a value greater than what could be represented by a 16-bit signed integer. This resulted in an Operand Error. _The data conversion instructions (in Ada code) were not protected from causing an Operand Error, although other conversions of comparable variables in the same place in the code were protected_.

The error occurred in a part of the software that only performs alignment of the strap-down inertial platform. This software module computes meaningful results only before lift-off. As soon as the launcher lifts off, this function serves no purpose."

<comment><USER14> > That's why we have floating point processing units instead of the fixed > point.

I'm not sure what that is supposed to mean. I was talking generally. Not every situation has a single appropriate value to represent it. I don't particularly care if this one example could have used a floating point or not.

> This piece of program catastrophically failed because some input was a just > somewhat bigger than before.

As far as the software was concerned the rocket had already catastrophically failed. It actually hadn't, because it was a different rocket than the software was designed for. It was "somewhat bigger" in the sense that it was large enough that the rocket the software was designed for would have been in an irrecoverable situation.

> Properly programmed components that are supposed to handle "continuous" > inputs and provide "continuous" outputs (and that is the specific part we > talk about) should not have "discontinuities" at the arbitrary points which > are the accidents of some unimportant implementation decisions (leaving > "operand error" exception for some input variables but protecting from it > for others!).

That's theoretically impossible. If you want to account for every possible value you're going to need an infinite amount of memory. There will be a cutoff somewhere, no matter what. Even if that cutoff is the maximum value of a double precision float - that's an arbitrary implementation limitation. You can't just say you can more than count the stars in the sky and that's clearly and obviously good enough for everything. It's not.

There will be a limit, somewhere. It will be an implementation-defined one. As long as the limit suits the requirements, it effectively doesn't matter. In this case, the limit was set such that if it was reached the mission had already catastrophically failed. That's all that can practically be asked for.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER15> So their model of the spacecraft behavior allowed for the possibility of the altitude instantly changing from 3.7km to a negative value. Seems like poor design.

<comment><USER16> "A large volume of data recovered from the Mars lander shows that the atmospheric entry and associated braking occurred exactly as expected"

In every communication and media event they keep putting a positive spin on what now turns out to be in essence a preventable design error...

<comment><USER17> They have to, unfortunately, because general public (including journalists) doesn't understand that mistakes _happen_ , and ESA being seriously underfunded as it is, cannot afford bad publicity.

Also, this was a test flight designed to gather data about the performance of the orbiter/lander platform, so it did perform its mission and according to the collected data, the platform behaves mostly as expected.

</comment></comment></comment><comment><USER18> What i'm wondering is why is it that when the Inertial Measurement Unit erroneous information was fed into the navigation system, and resulted in a negative estimated altitude, didn't the mars lander also have an accelerometer that could tell "hey, we're still accelerating, we're not stationary" and thus attempt some form of recovery of the navigation system.. perhaps reset the IMU or something? Reread data from the IMU after a certain delay?

I'm also quite confident that even before commencing entry, they had some idea from simulations as to how much the descent should take. Surely there's something wrong f you read negative altitude after just half the time required for landing.

<comment><USER19> What's the difference between an IMU and an accelerometer?

<comment><USER20> IMU means gyro, that is, it measures rotational rates. Accelerometers measure translational rates.

<comment><USER21> > IMU means gyro

In aerospace the phrase IMU typically refers to the _combination_ of accelerometers and gyroscopes.

</comment></comment></comment></comment><comment><USER22>
        When merged into the navigation system, the erroneous information generated an estimated altitude that was negative – that is, below ground level.

I wonder if this was a sensor fusion problem or a pedestrian integer overflow.

<comment><USER23> I would hope that it was a sensor fusion error.

IMU saturation is an expected condition, especially in this type of extreme real-world environment. It's absolutely normal to temporarily saturate IMU sensors on drones during hard landings for example and saturation values are generally clear in the documentation. A large part of the difficulty in position and attitude estimators is in rejecting glitching or erroneous data.

From the description (specifically that the duration of the saturation was an issue), it would sound like the position estimator (most likely a kalman filter) did not reject or distrust saturation values properly and converged on a bad solution. It appears that any strategy in place to (for example) reset the estimator state if it diverged from more reliable sensors (such as the radar altimeter), failed or were not included. It would also appear that the case where the IMU saturated during descent for over 1 second was not properly tested, as they were able to reproduce the issue in simulation.

This must be disappointing for the ESA but at least it was found now 'in beta' rather than in the relatively more important next lander.

<comment><USER21> Whilst incredibly disappointing for ESA, this is unfortunately not their first IMU-failure rodeo.

An Ariane 5 launch back in 1996 [0] suffered a catstrophic failure after the inertial reference units gave bad data, and the flight control computer accepted it as gospel.

It is sad that they might have lost another platform due to a lack of appropriate range / saturation checking, especially as there was a radar altimeter onboard telling them they weren't underground.

[0] [https://en.wikipedia.org/wiki/Cluster_(spacecraft)#Launch_fa...](https://en.wikipedia.org/wiki/Cluster_\(spacecraft\)#Launch_failure)

<comment><USER24> Ariane is built by Airbus, Schiaparelli was built by Thales Alenia, two completely independent companies. Not sure why there's supposed to be a correlation, other than both were contracted by ESA.

<comment><USER21> > both were contracted by ESA.

Yes I'm aware of how ESA works and who built what. But I would have hoped the common connection would have greater instilled the need for checking for these sorts of errors, if the Schiaparelli incident is as blutack theorised.

</comment></comment></comment></comment></comment><comment><USER25> Good work diagnosing the error. A reliable diagnostic is a great outcome of a bad situation. At least now, measures can be taken to eliminate this error from future scenarios.

</comment><comment><USER26> I'm still trying to understand whether this was a software or a hardware issue.. I mean, it's true that perhaps the software should have been better at rejecting the saturated data, but if an IMU persists in a saturated state for 1 second, that's a whole lot of time that the software just cannot know what is happening.

Granted, if it predicts an impossible configuration due to this, perhaps some different action should have been taken, like waiting for higher confidence and a more close-to-expected attitude estimate, but by then perhaps it would be too late. I mean, the thing is falling from the sky.. perhaps the "safe" thing was to deploy anyway. (Even if it didn't work.) I just don't know.

</comment><comment><USER27> So, spend more money on the sensor simulator's chaos monkey. Having worked on ESA commissioned ground systems emulation, I know this is easier said than done. Indeed getting _anything_ done is hard.

<comment><USER28> > Indeed getting anything done is hard.

Why? Too much Red tape?

<comment><USER27> Basically. Imagine how NASA splits up contracts for a large project, then put each of the companies in a different country.

</comment></comment></comment><comment><USER29> I don't understand why/how a feed of IMU data (gyro + probably accelerometers) could over-ride a Doppler altimeter that "functioned properly"... Any aircraft that would use inertial data to determine altitude would be in very deep trouble on Earth as well. I don't get it.

</comment><comment><USER30> To prevent an issue if IMU is saturated, couldn't they just add an and clause to conditional to prevent parachute release? altitude > 0 and time since entry > x then release parachute? Entry counter triggered once temperature reaches certain threshold or another sensor triggers entry.

<comment><USER31> Just by inspection that looks like that will have a ton of unwanted side effects... How do you define as "entry"? What if the triggering of the "entry" timer fails?

<comment><USER30> You can have multiple sensors start the timer.

</comment></comment></comment><comment><USER32> This feels like the kind of bug I catch early in implementation using quickcheck x_x Is property based testing used in the development of these kinds of systems?

<comment><USER33> Which would totally work if you have a super high resolution full system simulation. Hardware, software, mechanical and electrical interconnections all need to be simulated for issues like this to be found automatically.

</comment></comment><comment><USER34> Of course it is easy to be wise after the fact, but surely this is the sort of thing that should have been exhaustively tested (What happens if one sensor malfunctions and sends incorrect data)?

Massive cudo's for being public about the issue. It is not easy to talk about mistakes, especially those that appear to be silly in hindsight.

 _EDIT_ Interesting to see that I am getting downvoted multiple times on this. I don't really care about a few downvotes but would be interested to find out why.

</comment><comment><USER35> It isn't too clear from the description, but isn't _non-saturation_ of the inertial measurement responsible, rather than saturation? I'd expect that some computation did _not_ saturate its output, and the result overflowed (and hence became negative), which in turn was fed into altitude measurement.

</comment><comment><USER36> Why is rotation data being used to calculate altitude?

And why hasn't anybody asked that question yet? It's the first question I thought to myself.

It's a shame such an expensive piece of equipment and years of work and waiting around were obliterated by a bug that should have been caught.

</comment><comment><USER37> How could there not be a test case against negative attitude during descent?

</comment><comment><USER38> negative altitude ... so integer overflow it was ...

</comment><comment><USER39> if (altitude < 0) { return ERR_INVALID_ALTITUDE; }

<comment><USER40> Aircraft departing my local airfield commence at -4 metres according to GPS as it is actually below sea level.

<comment><USER41> And the intended landing site is at an altitude of -250m referenced to the Martian datum.

</comment></comment></comment><comment><USER42> First integer overflow to (literally) hit Mars!

</comment><comment><USER43> Maybe next time have a better system design so one instrument/sensor error/failure/loss doesn't lead to total loss of unit.

<comment><USER44> Maybe rather than a design issue this can be seen as a development process and engineering management issue.

I expect that a top level system model should reduce the risk of this sort of error.

Of course, it is hard to justify the cost of developing these models without being able to amortize that cost over multiple products / missions, so product line oriented engineering management and strong leadership from the sponsor has a big role to play as well.

<comment><USER43> Errors in INS are common. Saturation is common. What isn't common is a bad design that isn't failsafe or failop for a single failure.</comment></comment></comment>


Game Design Curriculum - <AUTHOR> https://www.riotgames.com/en/urf-academy/curriculum-guide <comment><USER2> Game design (whether computer or even board games) is a valuable topic to learn even if you never make a real game. If you make any kind of _thing_ that people use, learning a little about how to think like a game designer can help you make things that are more rewarding and pleasant to use.

One way to think of videogames is that they are _useless software_. Imagine a word processor that couldn't save or print files. Or a compiler that didn't generate executables that ran on any machine but your own. Why would anyone ever use such a pointless thing?

Well, games are in many ways exactly that. They don't really touch any part of the outside world or produce anything materially useful. They're self contained. So why do people sink time into them even though they can't get anything tangible in return? It's because the process of using the software itself—playing the game—is so _intrinsically_ enjoyable.

If you can learn a little bit of that and apply it to software that _does_ do something useful, you can end up with the kind of programs that build devoted fanbases.

<comment><USER3> Incredible how many posters are missing your point!

As a gamedev I 100% agree with your assertion that games are useless. An expression I use in game design is "fictional friction". Nothing in my game is real. None of the struggle is essential. Every mouse click or decision was a fiction I designed for players.

Nothing prevents me from giving players infitnite money, in fact there is a dev cheat menu which does just that. Instead all my effort revolves around crafting fake value for otherwise meaningless bits.

Thus games are the peak of software design: people put up with other software to pay for chances to play mine.

<comment><USER4> While we have the gamedevs here, and we're talking about game design, I would very much love and appreciate if you could upload a picture of your bookshelf and share it here.

I'm almost done with the Coursera CalArts game design specialisation, and they have cited a few resources, but I want to know what the industry recommend amongst themselves.

(That said, I've found the peer assessments on Coursera utterly lacking, some of the videos hard to understand and badly captioned, and will be starting a master's in game design in September...)

<comment><USER3> That's a fun question. Personally we do not have more than a couple dozen books in total at our house. This is more the product of my generation plus living in Japan where no one can afford the dead space for a book shelf.

For the subject matter I would say your best set of resources is GDC presentations. Our industry's great minds never write books, or even web articles for that matter. Best they give are the occasional GDC presentation. Even then Miyamoto has maybe given a couple hours of publicly accessible teaching material in his entire career.

If you are interested in Rendering that fields has a good number of useful books written. Design itself is more about "theories of games". Everyone has their own theories, mine might be more influenced by Sid Meier than anything.

My experience has been with a couple western designers, but more so stalwart japanese designers. On the Japanese side you'd be surprise how "personal" the design philosophies are. Westerners tend to be more rule driven, and thus if you want a systems driven understanding of design that needs to come from the west.

The training system in Japan is more apprenticeship driven. What the west would call designers first start as planners. Planner being a rather low level job with lots of manual grunt work. Then within a company the senior designers will couch the juniors on the subject of game design.

Thus there indeed does exist tomes on Nintendo's game design: but no one outside Nintendo has ever seen it and no one Ex-Nintendo can talk about it. Just look at how Ojiro ([https://twitter.com/moppin_](https://twitter.com/moppin_)) designer of Downwell went silent after joining Nintendo, then after leaving has remained silent.

Off the top of my head I know for sure Nintendo & Bandai use this "in-company learning" structure. Square Enix being a collection of fiefdoms tends more towards hiring a designer they like then putting them in charge of a team. Hence how you got Tokyo RPG Factory.

Note I'm approaching this with the understanding that a good game only has 1 lead designer. In Japan this is the game's director. It might be that in the west game's have a more group driven approach.

<comment><USER4> Thank you for highlighting the West's rules/systems-driven approach vs the mentoring approach in Japan.

I an indeed infinitely curious about the personal design philosophies of all the secretive greats. I guess there's something special inside all of us (game designers), and we just need to find it and nurture it. Books are good for starters, and up to a point.

PS good luck with Railgrade!

</comment></comment></comment></comment></comment><comment><USER5> (Riot's less than stellar history aside) I see these takes on game design a lot, and they're interesting to see because everyone takes their own stance on how to construct a fun game from distinct smaller concepts. Yahtzee's Dev Diary [0] talks a lot about loops (And I give it a watch recommendation), while at your average GDC talk you hear about the concept of pillars [1] and their role in shaping games. I'm interested to hear if anyone else has resources on other game design takes.

[0] [https://www.youtube.com/watch?v=CbArC2mvokQ](https://www.youtube.com/watch?v=CbArC2mvokQ)

[1] [https://www.youtube.com/watch?v=kzQDVtysXjA](https://www.youtube.com/watch?v=kzQDVtysXjA)

<comment><USER6> I've not played duskers (2). But from watching the video, sounds like he was super lucky. A happy accident, rather than an utter disaster.

One of those definite cases of survivorship bias.

<comment><USER7> I was privy to some of the development process of Duskers, and I would say that Tim worked pretty hard to create opportunities for happy accidents to befall him. There were several other prototypes that he was working on at the same time, but only Duskers crossed the finish line. This is not uncommon among game developers I've known; generally they're bursting with ideas and have many in-progress experiments in various stages at any time. It's a little like an incubator, where there's a diversity of ideas in order to have a better chance of having one be a hit.

There is survivorship bias, but it's kinda all happening within the one designer.

</comment></comment></comment><comment><USER8> Despite thinking Riot has excellent insight into competitive game development, I'm pretty cynical about their recent wave of sharing since it wreaks of image repair. It doesn't matter what there is to learn from Riot if the culture is poisonous, and I don't believe it's changed much since the last wave of allegations. It's a shame, since this looks pretty substantive, but I'd rather learn this elsewhere.

<comment><USER5> It's extremely difficult in Riot's case to separate the content they make from the culture of their company, and I agree with you that it's not really possible to do. I think that this resource is still an interesting collection of material and it's worth the time evaluating on its merits while keeping the perspective of the company's values in mind.

</comment></comment><comment><USER9> They put a PowerPoint into a PDF into a scrollable window too small to display a full slide. Ouch.

The course itself is interesting, in that it's not a programming course at all. It's about what makes a game fun. Download the PDF and look at page 6, which has the useful info.

<comment><USER10> There's a download link above the PDF view that allows you to view/download the PDF directly

</comment></comment><comment><USER11> Disclaimer: I am a DotA fan/nerd and am a huge fan of its design vs LoL's.

I applaud Riot for this, but I think they're specialty and reason for their huge growth is more to do with outreach, as jimbob45 noted. For a company with such a focus on competitive PvP games, I would argue they are not very good at balance at all. I would even go so far to say that their monetization scheme is detrimental to the balance of the game because of the incentive mismatch with creating a balanced game vs a profitable one. With that said, I think they have gotten a lot better over the years, and no one can take away from their success.

<comment><USER12> I agree with you, but I also wonder if balance is somewhat of an anti-pattern. By changing the balance of the game, they're influencing the meta builds and play styles, which keeps things interesting. So in a way, creating imbalances is a feature to keep the game fresh. All of this within reason of course.

<comment><USER11> I would agree with you IF you take traditional view of 'balance' in the way that LoL and other simpler games do. I would call that ideal a 'symmetric' balance, where they want each element in its category to be compared to all other elements in the same category. LoL definitely strives for this, as do many other games. Each champion is balanced against other champions of the same role. Items are relatively inflexible and augment the already present abilities of the champion. Each champion is essentially a complete package and each uses items to enhance what they already have. But when you have such rigid categories, power creep and best-in-slot champions will naturally appear, as whatever is best able to maximize their item's stats will be meta. My first competitive game was LoL back in its beta, and I was hooked, but what opened my eyes to what balance could be was DotA's approach.

DotA takes a systemic approach to balance. Each hero is a tool in the team's toolbox. Most heroes have very specific strengths and weaknesses, and their powerspikes are very pronounced. When you draft a team, you are specifically drafting a set of capabilities that you want to exploit against the enemy. This also leads to heroes having very hard counters. But I think the real beauty in DotA is it's item system, which allows you to patch holes in your team's composition and allow a huge amount of flexibility for each hero. I can keep going forever, as I really love its game design. But the most important thing that DotA does in terms of balance is its holistic approach that allows heroes to be individual components to a team, and have items to augment strengths or patch holes that the draft was not prepared for. (DotA's design and metagame sparked my interest in Complex Adaptive Systems, so I just nerd out about it any chance I get).

So to your point, I think the whole need to create imbalance is caused because of the way Riot balances their characters in the first place. I think it is fundamentally flawed from a design perspective, but it obviously makes them a ton of money. They really mastered monetization of multiplayer games. They also made their game very approachable for noobs, and that I have to commend them for.

</comment></comment></comment><comment><USER13> The thing that I take away from this is that this could be indicative of a shift away from the modern concept of university and towards a strengthening of the old master-apprentice models of learning.

</comment><comment><USER14> I was genuinely interested in this, however, after clicking around I couldn't find anything more than PDFs with empty areas for taking notes and vague curriculum module descriptions. Am I missing something? Perhaps there's missing content when viewing the page via mobile. Will revisit from my PC later.

<comment><USER5> The content here is a guide on how to teach the concepts outlined in the curriculum, and is not necessarily a self instruction course. See module one: [https://www.riotgames.com/en/urf-academy/fun-and- feeling](https://www.riotgames.com/en/urf-academy/fun-and-feeling)

<comment><USER14> Sure, but how do you teach these concepts without a source of truth? I can't find any reference material.

Edit: Just discovered the lesson plan download link under the headings, separate from the buttons at the bottom of the page. It helps, though am still confused about how anyone would use this material.

</comment></comment></comment><comment><USER15> The "8 kinds of fun" have been a huge help to me as a D&D DM. It's a great framework for running adventures that players will respond to and give everyone a chance to find their fun.

It also helps answer micro, nuts and bolts like what kinds of rewards your players will respond to. For example, a player really into challenge might only care about the stats on a magic sword they find, but a player more into narrative or exploration will dig a sword with a mysterious engraving on the hilt even if it isn't necessarily an upgrade. Real life people usually respond to multiple categories, but if you pay attention during the game you _will_ see this at work.

I got onto this via a post at theangrygm.com, who I really recommend. He's super opinionated, and the "angry GM" schtick gets old, but he knows what he's talking about.

</comment><comment><USER16> IMHO the biggest innovation Riot brought to the industry was player outreach. They were the first to heavily (30+ posts per day across multiple devs) interact with their player base. Even better, their patch notes were the first to have paragraphs explaining each change. Even _better_ , they stuck to a bi- weekly patch cycle religiously, unlike Blizzard, who had to be begged to put out a patch every six months.

To players who actually played SC2 and LoL circa 2011, it wasn't a surprise when LoL became the premiere e-sport and SC2 withered away.

<comment><USER17> I never picked up lol early, but I do vividly recall all the screeching wars between Hon (heroes of newerth) , Dota and lol people

The agreement was that lol won because it was free, tho it looked and played worse than either Dota or Hon, but the free aspect won riot the entire emerging markets sector

Me as a SC2 player through and through I just looked down on them overall, so feel free to correct me if I am wrong

</comment></comment><comment><USER18> Of all the non-FAANG gaming companies Riot is the one that most closely operates like a tech giant to me (with the exception of maybe Epic Games)

They have an incredibly robust M&A division, have mastered land + expand with their esports, are putting down roots in pretty much every major gaming category, and have modern monetization practices down to a tee.

Excluding the clearly egregious cultural issues, I would be fairly confident in betting on Riot in the long term

<comment><USER19> > _I would be fairly confident in betting on Riot in the long term_

Tencent already did that more than 10 years ago. I wonder how they pick who to invest into, because it seems that in the gaming space they have their hand in almost everything successful. And how come others aren't doing the same?

</comment></comment><comment><USER20> I must say that they're probably the most competent studio on the market

LoL is probably the only game that I play (maybe once a year I decide to spend up to 50h over some peroid on one title like GTA V or something) for years and it almost always feels "fresh" and very competitive.

<comment><USER21> LoL is also an amazing spectator sport, the format is so fun to watch. OverWatch/CS-GO/Valorant/Fortnite etc... the FPS aspect is a harder to follow especially for people with slower eye movement.

Lol on the other hand, its like watching Soccer/Golf/Football except the players also have magic/hi-tech abilities. The "depth" of the strategy is also sooo much more than an FPS. flex picks, lane swaps, lane freezing , meta shifts, (also mechanics but that spans across all esports).

Then theres also the music division, that collaborates with mega pop stars in korea/us/south america

<comment><USER22>
        The "depth" of the strategy is also sooo much more than an FPS

You only think that because you know more about LoL. As someone very into Overwatch, it's an absurdly complicated game where you only know about the complexity once you learn about it. After having learned a lot more about the game, I can look at high-level play and "see" a lot more of the things they're doing due to the level of complexity of the game, just like you can see that same stuff watching LoL.

At a high level, Overwatch is complicated because it effectively combines MOBA elements with FPS elements. It's not really a straight FPS, and the higher skill you get the more you have to pay attention to the MOBA elements in order to do well. As an example: Zarya is a tank whose entire ability kit (sans ultimate) is built around using an ability on herself and a teammate (on different cooldowns) to protect them from damage/negative effects for 2 seconds at a time. This ability can be destroyed if 200 damage is done to the player it's affecting, and Zarya gains increased damage if damage is done to this ability up to a cap. It sounds simple, but players who have mastered Zarya are ones who know _when_ to use these two abilities, which order, etc. which has a lot to do with the current state of the game. Bad Zarya players use both abilities instantly and simultaneously the second they see the opponent. It's all about timing abilities... which sounds a lot like playing a MOBA. But you also have to be good at aim and master the two firing modes she has, among the many, many other things to pay attention to during gameplay.

I don't whether LoL or Overwatch has more "depth", but they both have a lot of it.

<comment><USER23> Let me start a flamewar by informing you both that Dota 2 exists, and has much more complicated spells. And hence more depth. At least that's what people who have played both LoL and Dota 2 say.

<comment><USER24> Everything the GP has described is essentially only the mechanics part of a Dota 2 game. The strategic part of the game has way more depth than "use your abilities at the right time in the right order".

<comment><USER23> I agree to a large extent, but the last 2-3 years have not been kind to the Dota 2 strategic landscape. In the strategic rock-paper-scissors of teamfighting beats ganking beats splitpushing beats teamfighting, teamfighting has been overpowered by Valve.

It's not really possible to consistently win in pro Dota by ganking or splitpushing strategies. Only occasionally, can you do it, when you have a clear draft advantage the opponents don't see, and a clear plan that you can execute flawlessly. While, if both teams go for the teamfight strategy, then there is much more room for mistakes.

</comment></comment></comment></comment></comment></comment><comment><USER25> Their MMR system has changed over the years to maximize revenue at the expense of player pain. They force the above-average player to play 100s of games climb to rank where they "belong" per season, gradually increasing the amount of time required to be absolutely absurd. Combine that with their lack of care for the toxicity in their player base and you have one hot stinky mess, and I wouldn't put a high schooler anywhere near their core game design values. Direct reflection of their (bad) company culture.

In the micro, fun and well designed gameplay (Valorant too!). In the macro, their games are a terrible slog rooted in addiction science. All to maximize revenue from rolling skin and champion releases. Every single champion is released overpowered to add a temporary boost in play time and revenue. We can do better.

</comment><comment><USER26> Riot Games, isn't that the game company wholly owned by TenCent?

<comment><USER27> yeah I guess so, at least purchased most of its stock?

</comment></comment><comment><USER28> Is there any good game design curriculum you'd recommend? The webpage above is not suitable for self-paced learners...

</comment><comment><USER29> Bad joke sorry

<comment><USER30> People are downvoting you, but this is actually a ridiculously common thing in games.

The entire game industry is pretty bad in general compared to tech or otherwise: one huge concept is that you have to "love the game"; you're willing to take low pay, low benefits, deal with overrun unpaid overtime hours, etc. because you "love the game". Most game companies freely tell you about how disposable you are because they have hordes of players, many willing to work for free.

Riot has serious problems with this. The C-level that was repeatedly reported (for years) for grabbing testicles, dryhumping employees or farting on them is still in their role and hasn't been removed - they were just suspended for a month or two and reinstated. The gender-discrimination forced new legal counsel just a few months ago due to possible collusion.

<comment><USER31> How the hell does this fly? Why haven't employees sued the hell out of the company?

Why don't game engineers demand better? Start a company with decent work hours and comp and refuse to work elsewhere.

The games industry is sickly and broken.

<comment><USER30> From 2018 case (just google "riot farting" honestly): [https://html2-f.scribdassets.com/6454fy6neo6mwj3w/images/5-7...](https://html2-f.scribdassets.com/6454fy6neo6mwj3w/images/5-70337ee921.jpg)

> Why don't game engineers demand better? Start a company with decent work > hours and comp and refuse to work elsewhere.

Some do! I would imagine many indie game companies are not like this. There's kind of a split, where you have "strictly-business" companies - many Asian MMORPG publishers that have a subsidiary in the US are like this. There is no gaming culture, it's just a standard workday. You might get some plushies at the office or a poster or two.

The "cool" companies, like Riot and Blizzard, complete with campus statues of game characters and a culture to match, conference rooms named after game characters or items, are what gamers seem to want to look up to, though.

If you've played any MMORPG long enough, you'll see a lot of recurring themes, kids gleefully willing to "answer support tickets for free" and "I'll be a GM they don't even need to pay me" \-- this probably is part of the reason that produces toxic culture, along with the fact that if you strictly hire only gamers, you're already going to almost certainly have a higher toxicity demographic in the first place, depending on the game. If we're talking something like Harvest Moon or similar, of course there's going to be close to no toxic players.

Riot's primary product involves matchmaking you into 4 other random people that you have to cooperate with for anywhere from 20 minutes to an hour or more, where one person can lose the game for everyone by repeatedly dying to the enemy and making the enemy stronger. If you leave, you get banned or penalised for abandoning your team; this includes virtually unwinnable dragged-out hour long matches. You can kind of see how this playerbase would differ from, say, the usual singleplayer or co-op indie or puzzle game.

It's very difficult to boycott a game you've spent tens of thousands of hours on - a game that you're good at, that you have friends on, that you've spent heaps of cash shop currency on lootboxes for. Even if you desperately dislike the developer. This happens for many companies - look at all the loudmouthed people on reddit et al screaming about how no one should support EA or Blizzard, and then a month later go buy the new expansion pack for something and buy lootboxes and forget about it.

<comment><USER32> > _You can kind of see how this playerbase would differ from, say, the usual > singleplayer or co-op indie or puzzle game._

Oh yes. I must say, these days I hate playing multiplayer game modes with leaderboards attached.

Relevant to this thread: I used to play Heroes of the Storm (Blizzard's LoL clone) causally with friends. Then one day a colleague dragged me into his team, full of people dozens of levels above me (and otherwise experienced progamers). That was one stressful evening where I could tell everyone hated the noob that was me. My reflexes were OK, but my "meta" wasn't.

I never played a game of HotS with that colleague again.

I mean, seriously, I want to play games where one can excel in out-thinking the opponent. But HotS progaming feels more like an exercise in memorizing obscure game mechanics and a (constantly changing) Excel sheet full of character stats.

<comment><USER33> Well, the history is a bit different. LoL was created as a clone from Dota (Defense of the Ancients) mod for Warcraft 3. So, DOTA 2 is a continuation of that story and Blizzard basically recreated the mod into a standalone game and called it DOTA 2.

I do agree with the sentiment though. The community is definitively not n00b friendly and I gave up soon as well :S

<comment> purple-again Blizzard did not do that. Valve (the company behind Steam) did that. Much salt was sown around the Blizzard offices during the lawsuits that followed. Valve won.

Blizzard went on to release a remake of the game that spawned DOTA which was universally reviled, a large part of which was due to them killing incentives to build custom maps with their new IP rules designed to avoid another DOTA incident.

Armchair analysis with no expertise or special access to information, accuracy of statements may vary.

<comment><USER34> In case people are wondering, hes talking about Warcraft 3 Reforged, which is a remake of Warcraft 3. A real disaster of a release, I couldnt imagine anyone could have f __* up such a beloved game worse than Blizzard could.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER35> The problem I have with courses like this is it validates careers that lack integrity.

Game design 101 starts with learning to program or do graphics.

The same with software development. Unfortunately, we as an industry validated an entire class of people with jobs where they literally manage software (agile people) without building actual software for years.

It doesn't sit well with me, but life is not fair.

<comment><USER36> >Game design 101 starts with learning to program or do graphics.

No it doesn't, any more than architecture 101 starts with bricklaying, or novel writing 101 starts with learning typesetting and printing.

<comment><USER35> These are mostly false analogies.

I don't know enough about Architecture to point out the fundamentals that you should be making parallels to.

For novels, typesetting is an incredible false analogy. You need to learn how to communicate via writing on a technical level before you can write creatively (e.g stories).

Back to the shot at agile, I would not expect anyone that never dealt with deadlines or technical scoping to manage timelines for either of those things.

If someone doesn't even know how to write a game loop, they are going to move on to game design?

If you look at a movie, some of your bad screenplay writing can be fixed via casting. Imagine not having any hands on experience with the process, but alas, you move on to movie director because you took movie directing 101. It's insane, design is a holistic process that requires a lot of exposure.

<comment><USER36> >For novels, typesetting is an incredible false analogy. You need to learn how to communicate via writing on a technical level before you can write creatively (e.g stories).

Remember that the purpose of games is to communicate with the player, just as the purpose of a novel is to communicate with the reader. Novels do this through language, games do this through direction, design and assets, _not the code._ I think the analogy to typesetting is correct - writing the code that runs the game is a far lower level activity than writing the game itself.

>If someone doesn't even know how to write a game loop, they are going to move on to game design?

If they do, it's only because a company doesn't want to hire outside of house. But one doesn't need to know how to write a game loop to know how to design a game, any more than one needs to know how to design a game in order to write a game loop. They're completely separate disciplines.

>It's insane, design is a holistic process that requires a lot of exposure.

Not necessarily to programming. It can be useful, and sometimes with small studios or a single developer, it's unavoidable (but good design in those cases is usually the exception rather than the rule.)

<comment><USER35> If that's the case, then my friend, you live in a better world than I've come close to sniffing.</comment></comment></comment></comment></comment>


Samsung invests $500K in Linux Foundation to battle iOS - <AUTHOR> http://www.appleinsider.com/articles/12/06/05/samsung_invests_500m_in_linux_foundation_to_battle_ios.html

<comment><USER2> Is this a trend? First Valve is investing in Linux (by developing their platform for it) and now Samsung.

Whilst there has obviously been commercial investment in the past , I'm assuming the majority of it was for enterprise server stuff or embedded things. This is to build user facing stuff.

Perhaps companies are realizing that the only way to escape the nets of MS/Apple and their walled gardens is to invest in open technology.

<comment><USER3> Adding to this, I've heard Electronic Arts released two games on Ubuntu Market as well. Anyway, this growing interest targeting general user is overall good for Linux.

<comment><USER4> They are HTML 5 games that were already in the Chrome store [1]. It ended up being a big story of nothing.

[1] [http://www.ubuntuvibes.com/2012/05/new-ea-games-released- in-...](http://www.ubuntuvibes.com/2012/05/new-ea-games-released-in- ubuntu.html)

</comment></comment></comment><comment><USER5> This is not so much an investment, but a PR move to remind the world that Samsung continues to be involved in mobile Linux.

Samsung is investing much more substantial amounts of R&D money into Tizen, a Linux-based mobile OS with a HTML5 UI layer (co-developed with Intel). That's their real hedge against Apple and Google.

</comment><comment><USER6> $500k? When are they going to get serious?

<comment><USER7> $500k is almost the entire budget of the FreeBSD Foundation. I don't know what the Linux Foundation spends money on, but the FreeBSD Foundation does great stuff with its money.

And if you happen to have a "paltry" $500k sitting around, feel free to send it in FreeBSD's direction. :-)

<comment><USER8> That may be enough to fund the FreeBSD or even Linux Foundation's budget, but is it enough to make a difference in competing with iOS?

<comment><USER9> The kernel is just a small part of the software on a phone, of corse, and likely not the crucial differentiator either. Don't forget that Samsung probably uses Linux in a lot of other products too, e.g. TVs.

</comment></comment></comment></comment><comment><USER10> This is all about Tizen. Samsung, and its associates and partners in various markets around the world, knows that iOS can be competed with, rapidly, by a Linux-based alternative. It has been proven already with Android, and as we all well know .. there will be next generations.

Tizen could be a next-gen mobile/integrated platform, and it could well be that Samsung are the only ones who have anything to do with it, because there is one thing about a Linux-based approach: it is multi-variate. Tizen will be Samsungs Linux, but HTC might do something similar, and Creative Labs could even dust off Plaszma/ZiiOS and produce a shudder effect through the lower-end markets, too.

The Distro Wars won't be over for a while, I think, once this starts happening.

Edit: I want to add that I think that things like the Sharp/Samsung engineers' experience with AngstromOS and so on are sort of behind this.

<comment><USER11> They tried and failed before Android too.

These companies don't have what it takes to produce something as polished as Android. And even if they throw money at it and manage to do it somehow, they still do not have what it takes to build a developer community around it or a popular app store, like Google or Apple did.

Is anybody anywhere using the Samsung/Kies App Store for instance? It comes bundled on Galaxy S phones, I guess there are some people using it by mistake.

This effort can be easily underestimated, but even Microsoft is having problems doing it.

<comment><USER10> They have to get the hardware out there, first and foremost. Zii Labs (Creative Labs Plaszma OS partners) could have easily competed with iOS in its earlier days, if only they'd gotten the hardware out there. They lost because they didn't get things deployed; but factually the technology is there for a real ass-whooping, in the Mobile Linux-vs.-iOS department. They have to manage a good deployment, and with an investment like this it seems that they are gearing up to do so ..

</comment></comment></comment><comment><USER12> I think this article was more newsworthy when they accidentally quoted it as 500M instead of 500K (thats how I saw it on my Twitter feeds). The update to reflect the correct amount makes the title feel out of place.

<comment><USER13> Agreed. And it is interesting that they did not even do simple fact-checking before posting such a headline.

</comment></comment><comment><USER3> Samsung emerged as one the top vendors and gets revenue in order of billions due to Android. Hence it would be surprising if they _don't_ invest in Linux.

</comment><comment><USER14> But they still won't refund my unused Windows License even though French law says they have to.

</comment><comment><USER15> If Samsung worded it "We're donating $500k to support the great work the Linux Foundation does with FOSS" then I might give them a nod. But to say they are investing $500k (a paltry sum) to battle iOS is kind of sad. Why even say that, it sounds so petty.

So your only motivation is to attack Apple? It's not to simply support the legion of open source developers that have been giving you free software for years?

<comment><USER16> I _really_ doubt Samsung would have worded it this way.

Instead, you're reading commentary from Apple Insider, which is obviously a _completely_ unbiased source, that clearly doesn't believe everything revolves around Apple.

<comment><USER15> my mistake

</comment></comment></comment><comment><USER17> What does this investment actually buy them? And what does it go toward?

<comment><USER18> <http://www.linuxfoundation.org/programs>

<comment><USER19> Corporate membership: <http://www.linuxfoundation.org/about/join/corporate>

</comment></comment></comment><comment><USER20> I do not know if it is Firefox, NoScript, RequestPolicy or just me but the AppleInsider page looks like the default mobile formatted version of most websites.

</comment><comment><USER21> Samsung spends $500K in one hour on legal fees. Seems like a low amount to such a great foundation.

</comment><comment><USER22> Isn't this more like a donation?

</comment><comment><USER23> For a company the size of Samsung this is not a large amount, but it's certainly not a small amount either.

Articles like this make me feel like I'm back on slashdot sometimes. Am I alone?

</comment><comment><USER24> 500K is embarrassing. probably the samsung corporation is spending more money per year on toilet paper.

<comment><USER25> It's a _membership fee_ for the top tier of the Linux Foundation, not an investment.

</comment></comment><comment><USER26> Oh, wow $500K... Most people I know have houses worth more than that. Are Sumsung taking the piss out of Linux here? I get $500k if the entire FreeBSD Foundation budget but you'd think more money means they can pay to hire even more great developers. You don't battle iOS with $500K.</comment>


Apple's open secret: SproutCore is Cocoa for the Web - <AUTHOR> http://www.appleinsider.com/articles/08/06/16/apples_open_secret_sproutcore_is_cocoa_for_the_web.html <comment><USER2> Since <http://www.sproutcore.com> isn't too happy right now, folks may want to go straight to <http://code.google.com/p/sproutcore> instead.

</comment><comment><USER3> Is 280 North using this? Their "Objective-J" source code looks like a very similar idea, at least.

<comment><AUTHOR> They're not. Objective-J/Cappuccino is made by 280North.</comment></comment>


Freelancing questions - <AUTHOR> I have been working in the same company cor 5+ years, and I have not been doing a lot of cool stuff for the last years. I'm an experienced PHP developer , but I do have learned lots of stuff on my own though.
I am from Mexico, and the job market place is very difficult for developers so switching jobs it's hard, at least for me. For the last 5+ several years I've been under earning, and I have always been spending more than I earn because I negotiated very poorly my first salary (I was straight out of college).  I've tried to negotiate my salary several times, but the increments have been very small to make a real difference at the end of the month.
I need extra cash to stay a float, and I'm willing to do freelancing but: a) Most jobs require full time b) Most jobs are not long term at all and not stable, and some can even not pay you.
c) Many freelancing sites have hundreds of bidders (some automated), that it's very hard for me to land anything.
Do you have any tips or suggestions of what I can do to stay afloat? Any tips for freelancing? Or any part-time remote job site?
Any comment would be very helpful. <comment><USER2> I don't know the situation in Mexico at all so please take my humble opinion with a pinch of salt. About 3 years ago I was asked a similar question so I wrote a blog post about it:

[https://bjoernkw.com/2013/04/28/starting-a-software- consulti...](https://bjoernkw.com/2013/04/28/starting-a-software-consulting- business/)

I think most of that still applies - no matter if you're in Mexico or the EU (as in my case).

From my additional 3 years of experience since then I can't emphasise enough that networking is absolutely key! Talk to a lot of potential customers and peers (fellow developers, particularly those who already run their own consulting business). Go to meetups and industry events. Follow up with potential clients on a regular basis.

You might want to work with recruiters as well if you have to but having direct client connections in my opinion is the only long term way to run a sustainable consulting business.

I'd advise against using freelancing sites for the reasons you mentioned. It's usually a race to the bottom in terms of pricing because you're literally competing with the whole world.

<comment><AUTHOR> BjoernKW , thank you , this totally makes sense. I have not much direct contact with any (potential) clients, I'm going to try to work on that.

</comment></comment><comment><USER3> I completely understand your feelings as I'm in a similar situation these days. I'm from India and I've got 8.5 years of experience in consumer and SaaS startups + another 3 years as a freelancer prior to that. During 2005-2009, I used to get a lot of freelancing jobs, but as you mentioned, decent paying freelance work is hard to find these days.

I've been without a job for a week and don't want to take up another job right now. I just want to make $2.5K so that I can work on my own SaaS idea. I'm a PHP dev like you but I've also worked a lot on frontend (Angular, React/Redux, Backbone).

I've tried AngelList, Hackernews and pretty much all other sites mentioned on HN with very little success. If you are good at PHP/Laravel, you can probably try larajobs.com. Good Luck.

<comment><AUTHOR> Thanks keviv, I do have Laravel experience, but as you mention, no luck so far in landing something.

</comment></comment><comment><USER4> I've recently started freelancing and from my experience the job boards aren't great. I've had a few leads from LaraJobs but most of my work has come through networking and repeat business.

It's tough but work will come! Just keep going with it and talk to everyone about the fact that you're doing a bit of freelancing, you never know when someone will need some work done.

Also, a good portfolio website will obviously help. Shameless plug for mine at: [https://codefox.io](https://codefox.io) :P

<comment><AUTHOR> Thanks for your comment. Nice portfolio BTW</comment></comment>