What scientific evidence is there about the effectiveness of pre-flight walk-arounds?
First off you are required by the FAR's to preform a preflight check, 


  Sec. 91.7
  
  Civil aircraft airworthiness.
  
  (a) No person may operate a civil aircraft unless it is in an airworthy condition.
  
  (b) The pilot in command of a civil aircraft is responsible for
  determining whether that aircraft is in condition for safe flight. The
  pilot in command shall discontinue the flight when unairworthy
  mechanical, electrical, or structural conditions occur.


Don't forget the Preflight is also about making sure the aircraft is legally airworthy (VFR or IFR minimum equipment) as well as being physically airworthy. 

Here is a brief from the FAA on the matter. It has at least one example of an accident that occurred as a result of a bad preflight. According to this Flying Magazine Article 15 accidents have been the result of a bad preflight in the past 10 years (from the publish date of that article). Here is another example as well. Clearly it has lead to accidents in the past but you may have an issue finding any data on times that it has found things and potentially saved lives (as that is often not reported to anyone but the mechanic). 

Nasa conducted a study on checklist usage (preflight included) which is along the lines of what you may be looking for. 

Here is another FAA article on preflights and has some more specific stats on what you are looking for.  


  Consider this: NTSB aviation accident data show that in the last 10
  years, poor preflight inspections caused or contributed to 156 general
  aviation accidents and resulted in 41 fatalities.


You can compare that to the total number of other GA accidents for an approximation of how important it is in causing a crash. of When it comes to the FAA a great deal of analysis is carried out on accident data not necessarily preemptively. One of the other analysis issues here as mentioned in the comments is much of the reasons that planes get grounded does not get reported to any official authority (aside from the mechanic) so the data may not be public or easily accessible. 

If we include preflight checklists and walk arounds in the same category. Following the Northwest 255 accident that was the result of misunderstanding/misusing pre-flight checklists the NTSB did do a study on preflight (and all checklist) procedures. You can find the study and results report here which includes some stats. 


  This idea seems more plausible when I speak to more experienced pilots
  of the same type, and each of them warns me about checking one thing
  in particular that's not on the checklist, because they once saw an
  accident that was caused by a failure there - and crucially, each of
  them warns me of a different thing.


This may be more a case of correlation over causation. If you are involved in aviation long enough (especially the older generation that has flown when laws and planes were much different) you see lots of things. Some of the issues they talk about may have been covered by AD's since they encountered them (if you are flying a common airframe like a 172 or PA-28). 

As for not finding stuff this will come with how well you know your plane. As you start to fly the plane more and more (you should be training in one plane or at least only one type of plane). Eventually you will begin to know whats where, the POH can help with that, and you will be able to recognize if something is out of place. 

I can speak from personal experience that I have delayed many flights due to things I have found on the preflight. I had a cracked cowling bracket, countless low gear struts that needed air, many fuel calls in the rental planes, a fuel vent issue, and one breaks issue. 

Of course you will always have human factors present in manned flight.