Is there anywhere to complain about the behaviors of editors of journals except themselves?
It's probably just supply and demand.  Your paper isn't good enough to make it into the regular editions, but he is going to lower the bar a bit to fill out the special issue.  

I have seen this.  It's not nefarious.  In a way the regular/special difference is like the difference of journals with different importance, exclusivity.

In terms of what you should do:  I would lean to moving on to another journal.  And upgrade the paper based on the reviews.  Now, if the editor could tell you the paper was good to go (for the special), that is different.  But it sounds like this is not the case or else he would have just told you that at the time.  Being invited to enter another uncertain outcome of his would not interest me.  Yeah, you will have new competition at new journal, but at least you are moving on.  How I would prefer it.