Citing two authors with the same surname – is it worth noting that they are distinct people?
Check your style guide. Many surname-year styles, such as APA, have provisions for exactly this problem.


  But do you know how to proceed when a reference list includes publications by two or more different primary authors with the same surname? When this occurs, include the lead author’s initials in all text citations, even if the year of publication differs (see the sixth edition of the Publication Manual, p. 176). Including the initials helps the reader avoid confusion within the text and locate the entry in the reference list. 


http://blog.apastyle.org/apastyle/2014/01/when-to-use-author-initials-for-text-citations.html

In your case, this would lead to:


  We use the Smith corpus (F. Smith 2010), as prepared by Johnson et al. (2015).
  This corpus is partitioned into test, development and training subsets, and has minor clean-up from the original data collected by F. Smith (2010).
  
  It is also uses by B. Smith et al (2016), and Otter et al (2016). 


Note that since "Smith corpus" is a name of a corpus, rather than a citation, it can remain without the initial.

Also note that in APA style, all citations to any Smith papers have to have the initial. So even if this is the only time you mention B. Smith et al 2016, all of your references to F. Smith 2010 have to have the initial. This is a little annoying but nicely unambiguous.