sparse int dataframe from dict crashes with default_fill_value=0
#### Code Sample, a copy-pastable example if possible

```python
pd.SparseDataFrame({1: {2: 3}, 3: {3: 4}}, dtype=np.int64, default_fill_value=0) 

```
#### Problem description

The command fails with `ValueError: Cannot convert non-finite values (NA or inf) to integer`. To me the problem seems to be that missing dict entries as always treated as `np.nan` instead of the fill value, however the nan values cannot be casted to ints. I haven't found any way to work around this yet that is feasible with larger data. E.g. if I use `pd.SparseDataFrame({1: {2: 3}, 3: {3: 4}}, dtype=np.int64)`, the fill value is `np.nan`, also if I use `pd.SparseDataFrame({1: {2: 3}, 3: {3: 4}}, default_fill_value=0)` (then just the dtype changes).

#### Expected Output

```python
    1  3
2   3  0
3   0  4
```

#### Output of ``pd.show_versions()``

<details>

INSTALLED VERSIONS
------------------
commit: None
python: 3.6.7.final.0
python-bits: 64
OS: Linux
OS-release: 3.10.0-693.el7.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.24.1
pytest: None
pip: 9.0.1
setuptools: 39.0.1
Cython: None
numpy: 1.16.2
scipy: None
pyarrow: None
xarray: None
IPython: 7.3.0
sphinx: None
patsy: None
dateutil: 2.8.0
pytz: 2018.9
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml.etree: None
bs4: None
html5lib: 0.999999999
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: None


</details>
