How do I delete apache from OS X Yosemite?
In this case you don't need to delete apache as you should not touch any executable under /usr/bin as Apple controls updating of this (i.e. could install it again). apt-get etc are not OS X or Unix commands but one packaging system for Linux. 

Any install of apache will put it in a different place to where Apple puts it.

To install Apache if you want to use a packaging system look at Macports, Homebrew or Fink. (if deb is the packaging you know then look at Fink) 

Otherwise the easiest installs are using MAMP, or use Apple's configuration in OS X Server

What you use depends on your knowledge and what you want to use it for and to use the correct tutorials i.e. not that that assume Linux