SVM with different length features
Yes, you can do it.  Concatenate the two feature vectors to get a single feature vector of length 147; the first 144 components are from the HOG, and the last 3 components are from the RGB.  Then apply the SVM on the feature vector of length 147.  SVM doesn't care where the individual numbers came from.

Code is off-topic on this site.