TypeError when calculating min/max of period column using groupby
#### Code Sample, a copy-pastable example if possible

```python
import pandas as pd
periods = pd.period_range(start="2019-01", periods=4, freq="M")
groups = [1, 1, 2, 2]
df = pd.DataFrame({"periods": periods, "groups": groups})
result = df.groupby("groups")["periods"].min()
```
#### Problem description

The last line of the example throws a _TypeError: data type not understood_.

<details>
Traceback (most recent call last):
  File "test.py", line 6, in <module>
    result = df.groupby("groups")["periods"].min()
  File "...\site-packages\pandas\core\groupby\groupby.py", line 1378, in f
    return self._cython_agg_general(alias, alt=npfunc, **kwargs)
  File "...\site-packages\pandas\core\groupby\groupby.py", line 889, in _cython_agg_general
    result, agg_names = self.grouper.aggregate(
  File "...\site-packages\pandas\core\groupby\ops.py", line 580, in aggregate
    return self._cython_operation(
  File "...\site-packages\pandas\core\groupby\ops.py", line 573, in _cython_operation
    result = result.astype(orig_values.dtype)
TypeError: data type not understood
</details>

#### Expected Output

The `result` DataFrame should contain the earliest period (in this case 2019-01) for each grouping slice.

#### Output of ``pd.show_versions()``

<details>

INSTALLED VERSIONS                                                        
------------------                                                        
commit           : None                                                   
python           : 3.8.1.final.0                                          
python-bits      : 64                                                     
OS               : Windows                                                
OS-release       : 10                                                     
machine          : AMD64                                                  
processor        : Intel64 Family 6 Model 142 Stepping 10, GenuineIntel   
byteorder        : little                                                 
LC_ALL           : None                                                   
LANG             : en_US.UTF-8                                            
LOCALE           : English_United Kingdom.1252                            
                                                                          
pandas           : 1.0.0                                                  
numpy            : 1.18.1                                                 
pytz             : 2019.3                                                 
dateutil         : 2.8.1                                                  
pip              : 20.0.2                                                 
setuptools       : 45.1.0.post20200127                                    
Cython           : None                                                   
pytest           : None                                                   
hypothesis       : None                                                   
sphinx           : None                                                   
blosc            : None                                                   
feather          : None                                                   
xlsxwriter       : None                                                   
lxml.etree       : None                                                   
html5lib         : None                                                   
pymysql          : None                                                   
psycopg2         : None                                                   
jinja2           : None                                                   
IPython          : None                                                   
pandas_datareader: None                                                   
bs4              : None                                                   
bottleneck       : None                                                   
fastparquet      : None                                                   
gcsfs            : None                                                   
lxml.etree       : None                                                   
matplotlib       : None                                                   
numexpr          : None                                                   
odfpy            : None                                                   
openpyxl         : None                                                   
pandas_gbq       : None                                                   
pyarrow          : None                                                   
pytables         : None                                                   
pytest           : None                                                   
pyxlsb           : None                                                   
s3fs             : None                                                   
scipy            : None                                                   
sqlalchemy       : None                                                   
tables           : None                                                   
tabulate         : None                                                   
xarray           : None                                                   
xlrd             : None                                                   
xlwt             : None                                                   
xlsxwriter       : None                                                   
numba            : None                                                   

</details>
