.infer_objects() behavior is unintuitive when used with schemaless inputs
#### Code Sample, a copy-pastable example if possible

```python
import pandas as pd
from io import StringIO

df = pd.concat([
    pd.DataFrame({'a': ['a', 'b']}),
    pd.DataFrame({'a': [1, 'c']}),
    pd.DataFrame({'a': [2, 3]})
]).reset_index(drop=True)

#int64
print(df.loc[[2,4,5]].infer_objects().dtypes)

#object
df2 = pd.read_html(StringIO(df.to_html(index=False)))[0]
print(df2.loc[[2,4,5]].infer_objects().dtypes)

#object
df3 = pd.read_csv(StringIO(df.to_csv(None, index=False)))
print(df3.loc[[2,4,5]].infer_objects().dtypes)

#int64
df4 = pd.read_csv(StringIO(df.loc[[2,4,5]].to_csv(None, index=False)))
print(df4.dtypes)
```
#### Problem description

`.infer_objects()` attempts only a soft conversion of columns classified as objects. When you read from a file type that doesn't come with any schema information like HTML tables or CSVs, the type inference system will read numbers in a mixed format column as strings. As a result, even after you filter to just the numeric rows, `.infer_objects()` will leave them as `object` instead of `int64`/`float64`, whereas dumping/rereading them will convert them to numbers.

It would be nice if `.infer_objects()` did something more like rerunning the standard type inference that happens in `pd.read_csv()` or `pd.read_html()` on the DataFrame since the read/write feels hacky, and having to do a whole lot of `pd.to_numeric()` on a wide frame is cumbersome, particularly if it's just a subset of the columns.

#### Expected Output

```
a    int64
dtype: object
```
for all above.

#### Output of ``pd.show_versions()``

<details>

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.6.1.final.0
python-bits      : 64
OS               : Windows
OS-release       : 8.1
machine          : AMD64
processor        : Intel64 Family 6 Model 60 Stepping 3, GenuineIntel
byteorder        : little
LC_ALL           : None
LANG             : None
LOCALE           : None.None

pandas           : 0.25.1
numpy            : 1.17.1
pytz             : 2019.2
dateutil         : 2.8.0
pip              : 9.0.1
setuptools       : 41.0.0
Cython           : 0.25.2
pytest           : 3.0.7
hypothesis       : None
sphinx           : 1.5.6
blosc            : None
feather          : None
xlsxwriter       : 0.9.6
lxml.etree       : 3.7.3
html5lib         : 0.999
pymysql          : None
psycopg2         : 2.7.3 (dt dec pq3 ext lo64)
jinja2           : 2.9.6
IPython          : 5.3.0
pandas_datareader: None
bs4              : 4.6.0
bottleneck       : 1.2.1
fastparquet      : None
gcsfs            : None
lxml.etree       : 3.7.3
matplotlib       : 3.0.0
numexpr          : 2.6.2
odfpy            : None
openpyxl         : 2.4.7
pandas_gbq       : None
pyarrow          : 0.14.0
pytables         : None
s3fs             : None
scipy            : 1.1.0
sqlalchemy       : 1.1.9
tables           : 3.2.2
xarray           : None
xlrd             : 1.0.0
xlwt             : 1.2.0
xlsxwriter       : 0.9.6
</details>
