For the longest time in places like Java's IRC channel, SO, and other places I've been told something along the lines of Worry about how the code looks and its readability/understandability now, and performance later if absolutely necessary. So for the longest time, I haven't really been OCD about performance for my small desktop or web apps, just removing the obviously inefficient.Most responses are What about scalability?. Thats a legitimate point, but if my app was only built to parse, say, files 10,000 lines long, should I make my code a mess for the small percentage of people that are going to shove in a 1,000,000 line file?My main question is when should I trade the easy but somewhat inefficient ways of doing tasks for big giant complicated beasts that do things extremely quickly but destroy any possible ways of upgrading and make the code excessively difficult and prone to rewriting anyway by the next developer? 
When should I care about performance?
usage;performance
Worry about performance when it becomes a problem.If you write a small app to process 10,000 line files and you get a 1,000,000 line file every 100th file, it probably doesn't matter that it takes longer to process that one file. However, if you are regularly getting files that are 5-10 times larger than initially and your application is taking too long to do its job, then you start profiling and optimizing.Now, I said too long to do its job. That is up to the user or sponsoring organization to decide. If I'm doing a task and it takes me 5 minutes to do something when it took me 3 without the software or with a different tool, I'd probably file a bug report or maintenance request to have that improved.If you are the user, how long you want your software to take to do its job is up to you - only you can decide if you want it done faster or if you are willing to wait longer to have more readable code.