Regarding logic behind Organic Chemistry
Sometimes it was a lucky accident, sometimes there was a lot of math and logic going in the designing experiments.  The organic chemistry of that era, though, was a lot more applied and empirical than the physics of today.  They usually weren't setting out to find the absolute laws-of-nature governing chemistry, they were just trying to make better dyes, medicines, rubber, etc, and stumbled upon new phenomena along the way.
Further Explanation
Almost all these guys worked in Germany in the late 1800s, when chemical industry was the hot new thing (think of biotechnology in today's world).  There was a lot of money to be made in making better dyes, medicines, rubber, etc, so people would hack/engineer the established methods for making that stuff, and see if the resulting products were any better.
For example:
Chemical A cures headaches really well, but causes nausea.  Chemical B gives a little relief from headaches, but doesn't have any side effects.  I bet I could make chemical C (which no one had ever made before), which has a structure that's sort of a combination of chemicals A and B, and maybe that will cure headaches with no side effects.
==&gt; {made the chemical and did some tests to find out the structure}.
What the heck!  This has a completely different structure than what I was expecting, it must have undergone a second reaction to rearrange itself.
==&gt; {publish paper},{get famous},{people 100 years later question the logic of your experiments} :)