Does complicating a flawed algorithm make it secure? It's a basic question I guess, but I don't know the answer and I don't think that anyone asked it.

Suppose I built a system, it relies on MD5 for security, suddenly I read am article on how easy are MD5 to crack nowadays.

Not being able to implement other algorithms, I multiply the MD5 hash by Pi, and some Fibonacci sequence then divide it by 0.5 and multiplied it by a random string that I generated using a secure generator. I obtain a totally different string

Then I use a function from my program to reverse all that I have done and then pass the original MD5 Hash to be decrypted.

The example I gave is simple, you can over complicate it as much as you like. Suppose you're a hacker, not knowing what I did to the MD5 hash, would you be able to crack it?
There's a problem with boundaries here; how much "complication" is allowed? I could argue that SHA-2 is a complication of SHA-1 because they both use a Merkle-Damgård construction and have other similar elements. Then again, they are significantly different internally.

On the other hand the addition of a single bitwise rotation did make SHA-1 significantly more secure. Of course it has now proven to be not secure enough but that was after decades of good service.


  SHA-1 differs from SHA-0 only by a single bitwise rotation in the message schedule of its compression function; this was done, according to the NSA, to correct a flaw in the original algorithm which reduced its cryptographic security.


This also shows another problem with the question: what is secure enough? Obviously SHA-1 is not secure enough, but it is at least protected against some attacks on SHA-0. SHA-2 is much more secure than SHA-1, but it is still vulnerable against length extension attacks which SHA-3 is not.

In general you cannot just apply some random function to the inner workings or output of an insecure function. Cryptographic functions need proof (or at least a well reasoned strong indication if a prove cannot be constructed) to be secure. For instance, if the bitwise rotation was only performed on the output of SHA-0 then the result would not have been secure.

So unfortunately the answer is: it depends on the change being made and - in a practical sense -  the reasoning behind it.



With regard to your scheme: it only changes the end result of the hash, using operations that - according to your own text - should be reversible. Furthermore, you require the algorithm itself to be secure.

There are two major problems with this:


you lack a clear indication why this would make the MD5 hash more secure, trying a lot of mathematical constants and operators would likely break the scheme;
keeping the algorithm itself safe breaks the Kerckhoffs principle, you should not rely on keeping the algorithm safe.


Kerckhoffs principle and the reasons for making an algorithm public is clearly explained in this answer on Crypto.