Does an order by clause have a negative effect on performance?
An ORDER BY clause will always add a sort operation if there is no usable index to retrieve the data in order.  For larger sets of data the data being sorted may be written to disk adding additional processing.  For smaller result sets where indexes are being used, there may be no requirement for a sort step.

If sorting is required, the database will have to retrieve all the sort key data and sort it before presenting results.  This can delay the availability of the first rows.  If you only want the first few rows of a large set, this can generate a lot of extra work for the database.  

Than being said, I have rarely seen an ORDER BY clause a significant performance impact.  I recall one case where a developer attempted to improve performance by retrieving an unsorted result set and sorting it in the client, but it did not work well. 

I avoid the ORDER BY clause unless there is a requirement for data to appear in a given order.  If there is a requirement to present data in a given order, I use the ORDER BY clause.  If there is a question about performance, I test the query against a production database before testing alternatives.