Co-Author is asking me to remove abstract submission of paper
You ought not to have submitted anything naming the PhD student as a co-author without his clear consent.  So you do need to withdraw the submission, immediately, and you owe him an apology.

After that, you can all work out what to do next.  Maybe you can all come to some agreement, and maybe the professor can help to mediate.  There's a good chance that you may miss the deadline and have to submit to some other venue at a later date; you'll just have to live with that.

No one author can unilaterally decide author ordering, even if it's "obvious"; you all have to agree.

Keep in mind it's a two-way street; he also can't publish without your consent.