Etymology of 奇 and 怪
「奇」 is not a person riding a horse - that is probably a learning mnemonic, meant to remind you of the word 「騎」. This character has received some attention in the question What is the glyph origin of '奇'?.
「怪」 (Zhengzhang OC: /*kruːds/, strange) is comprised of semantic 「心・忄」 (heart, mind) and phonetic 「圣」(/*kʰuːd/).

「圣」 (Mandarin Pinyin: kū, to dig) is comprised of semantic 「又」 (picture of a hand) and semantic 「土」 (picture of a lump of dirt).
「圣」 is rarely seen in the orthodox character system. However, you may see the identical shape 「圣」 in unorthodox systems representing completely different words or components, such as Simplified Chinese (where it is used as an abbreviation of 「聖」) and Japanese Shinjitai (where it is used as an abbreviation of 「巠」).
As usual, these abbreviations do not have anything to do with the language.

You shouldn't use 「奇」 by itself if you want to say strange; this is because 「奇」 forms the roots of several words which don't necessarily mean strange, but related yet different words like wonderful (奇妙), mysterious; magical (神奇), and rare (奇特).