Replacing outlet switch combination with GFCI outlet switch combination
First, that thing you did there, where you just tried random stuff, without really knowing what you were doing?  Don't ever do that with electrical. You would've stopped at the first combination that worked, and there are lots of combinations that work and will kill you. No more of that. If you are stuck/confused, stop until you learn.  

*In particular, the GFCI device came with a warning sticker across two of the terminals that said, in as many words, "Do not use. For wizards only."  Peeling that is where things went sideways. 

What you have now.

Switch+receptacles (that aren't GFCI) are made with a "common" tab that lets the recep hot and switch share a terminal. On yours, that tab is broken off, so the two devices are *totally independent**.  

Specifically, your switch black &amp; white is a switch loop, meaning hot power comes down the black and goes back on the white wire.  You didn't confuse them and I'm glad you didn't.  Just for reference, you can't put a smart switch here, because smart switches need neutral and you don't have a neutral you can use.  The receptacle neutral can't be used by the switch because it's fed from a different cable.  

Because this is a switch loop, forget everything on that application note. The white wire in that note is neutral, the white wire in your box is actually hot.  It should have been marked by wrapping it with black tape.  The application note is for a "power at the switch" arrangement, which isn't the style yours uses, and what's more, the app note is putting the lamps on GFCI, which is a silly thing to do unless it's in a water feature like a pool or shower. These sorts of details is why electric is best left to trained people who are used to seeing weird stuff like this. 

Wait. Why do you have a switch on a motion sensor?

Motion sensors are supposed to be switched by motion.  Not a switch.  The two aren't really compatible, and will cause poor performance from the motion sensor.  The switch will prevent the motion sensor from lighting when off, which may sound clever, but turning off the switch also makes the motion sensor forget what night and day look like from here.  If you measured light levels with an instrument, a shady location by day can be darker than a streetlit location by night. That confuses motion sensors the first time they power up, but they learn after a night or two. Turning off the switch wipes that memory. 

So your best recommendation is to bypass the switch altogether and fit a plain GFCI recep.  Then you can get 2 outlets instead of 1.  

Hooking up the GFCI

OK, the warning tape goes back on the LOAD terminals.  

Note the three wires (single cable + ground) going to the socket part of the existing switch.  Those go onto the GFCI's LINE terminals and the ground screw.   Black to brass, white to silver. 

That was easy, and we are done wiring the GFCI device, and we are done with screw terminals on the switch. 

Hooking up the light switch

Note the 2 wires that used to go to the light switch side of the device.  If you've decided to stop switching a motion sensor, just connect those two wires to each other and you are done.  You straighten out the shepherd's hooks, cut them slightly shorter, and feed them both into a "wire nut" and twist firmly. Proper technique and all that.  Figure out what that is if you don't know. 

If you really, really want that switch... OK, the GFCI+switch combo device has two wires coming out of it.  Each wire joins to one of the switch wires coming from the wall, again using wire nuts and 

If you decide you don't want that switch, don't use a GFCI device with a switch, use one without.  Still, wire-nut the two wall wires from the light to each other and don't use the LOAD screws.