BUG: pd.to_datetime(infer_datetime_format=True) drops timezone 
- [X] I have checked that this issue has not already been reported.

- [X] I have confirmed this bug exists on the latest version of pandas.

- [ ] (optional) I have confirmed this bug exists on the master branch of pandas.

---

#### Code Sample, a copy-pastable example

```python
dates = ['2020-05-01T18:04:03Z', '2020-04-28T17:54:06Z', '2020-04-26T18:04:09Z']
print(pd.to_datetime(dates, infer_datetime_format=True)) #drops UTC
print(pd.to_datetime(dates, infer_datetime_format=False)) #works!

dates = ['2020-05-01T18:04:03.0Z', '2020-04-28T17:54:06.0Z', '2020-04-26T18:04:09.0Z']
print(pd.to_datetime(dates, infer_datetime_format=True)) #works!
```
Output:
```
DatetimeIndex(['2020-05-01 18:04:03', '2020-04-28 17:54:06',
               '2020-04-26 18:04:09'],
              dtype='datetime64[ns]', freq=None)
DatetimeIndex(['2020-05-01 18:04:03+00:00', '2020-04-28 17:54:06+00:00',
               '2020-04-26 18:04:09+00:00'],
              dtype='datetime64[ns, UTC]', freq=None)
DatetimeIndex(['2020-05-01 18:04:03+00:00', '2020-04-28 17:54:06+00:00',
               '2020-04-26 18:04:09+00:00'],
              dtype='datetime64[ns, UTC]', freq=None)
```

#### Problem description
Using infer_datetime_format=True drops the timezone info (UTC) if datestrings do not have decimal seconds as shown in the example above. '2020-05-01T18:04:03Z' is a valid ISO 8601 datetime (https://tools.ietf.org/html/rfc3339#section-5.6). 

#### Expected Output
```
dates = ['2020-05-01T18:04:03Z', '2020-04-28T17:54:06Z', '2020-04-26T18:04:09Z']
print(pd.to_datetime(dates, infer_datetime_format=True))
#DatetimeIndex(['2020-05-01 18:04:03+00:00', '2020-04-28 17:54:06+00:00',
#               '2020-04-26 18:04:09+00:00'],
#             dtype='datetime64[ns, UTC]', freq=None)
```




#### Output of ``pd.show_versions()``

<details>

INSTALLED VERSIONS
------------------
commit           : 2cb96529396d93b46abab7bbc73a208e708c642e
python           : 3.8.8.final.0
python-bits      : 64
OS               : Linux
OS-release       : 4.14.177-139.253.amzn2.x86_64
Version          : #1 SMP Wed Apr 29 09:56:20 UTC 2020
machine          : x86_64
processor        : x86_64
byteorder        : little
LC_ALL           : C.UTF-8
LANG             : C.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 1.2.4
numpy            : 1.20.2
pytz             : 2021.1
dateutil         : 2.7.5
pip              : 20.3.4
setuptools       : 49.6.0.post20210108
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : 1.10.2
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : 2.8.6 (dt dec pq3 ext lo64)
jinja2           : 2.11.3
IPython          : 7.22.0
pandas_datareader: None
bs4              : 4.9.3
bottleneck       : 1.3.2
fsspec           : 2021.04.0
fastparquet      : None
gcsfs            : 0.7.2
matplotlib       : 3.4.1
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : 3.0.0
pyxlsb           : None
s3fs             : 0.6.0
scipy            : 1.6.2
sqlalchemy       : 1.4.9
tables           : None
tabulate         : 0.8.9
xarray           : 0.17.0
xlrd             : None
xlwt             : None
numba            : 0.53.1

</details>
