A situation where security by obscurity might be the best solution - or am I wrong? Imagine the following situation. There is an application for a microcontroller-driven system. The profits are made from manufacturing and selling the device itself. However, it is expected, that the users can later download software-updates from a public website, and install it (using a USB-connection, for example).

Now, the problem is, if the updates were released as raw compiled binaries, anyone could just copy the hardware, and spare themselves the costs of a long software development.

Now, let the update be encrypted in some way, and the bootloader in the microcontroller (which is of course not part of the update, and will never be changed) The whole program sits in the internal flash which is protected against reading.

The problem is, there is so little space (measured in a few dozens of words or in best case up to maybe a hundred or so) in the bootloader that no cryptographically secure decryption algorithm can be implemented there. 

So, let's release an update which has valid but random and nonsensical instructions inserted at random but pre-determined places. The bootloader knows it, and will remove them. The idea is, that even if an attacker knew this method (but did not knew the positions where these instructions are inserted), it would be near impossible to get the real program, as you cannot know you got the right one until you tested it (and even then, some hidden bugs might be still lurking if the "cracking" was not perfect). 

The main objective in this scenario would be, to make it more time-consuming for the attacker to steal the software, than it would be to develop it on its own.

Is there an obvious flaw in this way of thinking?
In summary: the premise "there is so little space in the bootloader that no cryptographically secure decryption algorithm can be implemented there" is likely wrong; thus security-by-obscurity is not the way to go.

The method described in the question attempts to "prevent theft from the publicly released update"; but it fails to do that if an adversary manage to extract the bootloader and include it verbatim in a clone of the hardware: the clone and bootloader will happily load the software updates, no matter what clever and effective security-by-obscurity is performed.

Thus, something secret is needed in the bootloader, else this (or any) technique is doomed to allowing clones of the hardware to run the software updates. Once you assume that, even only 10 bytes of secret data, you no longer need security-by-obscurity: you can use that secret data as a key to a block cipher in e.g. CBC mode, and release the software updates encrypted with that, and perhaps signed integrity-protected with a MAC. Complete decryption with an appropriate block cipher, such as TEA (which is designed to be compact and reasonably efficient), can be implemented within less than 200 bytes on about any micro-controller.

Unfortunately, it is hard to keep something secret in a standard micro-controller: all data, including secret keys, tends to be readable using widely available tools, such as a JTAG port. And when that won't do, side-channel attacks will typically succeed. The premise "the internal flash which is protected against reading" might be true, but protected does not imply safe.

For this reason, in the integrity-protection part (if desired), it is best to use a public-key signature scheme, which does not need a secret (but will only marginally help towards the goal in the question: it will not prevent use of the software on unauthorized hardware, only prevent use of modified software on authorized hardware or exact clone thereof). An RSA signature with public exponent $e=3$ together with a proper padding scheme based on a hash can be implemented in say 800 bytes including 2048-bit public key (which will use less than 128 bytes; that's not a typo), and run at acceptable speed (run time is typically dominated by the hashing).



You are unlikely to get support for security-by-obscurity here: we officially believe in Kerckhoffs's principles. I would only condone security-by-obscurity as a second line of defense.