Single speed dirt jump specific frame for general riding
My short answer is no.  A true dirt jump frame is going to be horrible for short XC riding and even worse for extended 2 to 3 hour rides.  You won't be able to get the extension to pedal efficiently and will just end up wearing yourself out standing.  I have a dirt jump frame and after only a couple short commutes on it (4 miles) I decided it wasn't worth commuting on it.  A "hardcore hardtail" looks to be closer to what you should probably look at.  It should be far more comfortable for extended riding, but still handle the abuse of a dirt track.