BUG: Using pd.to_datetime function in groupby.apply function causes key value error
### 

- [X] I have checked that this issue has not already been reported.

- [X] I have confirmed this bug exists on the [latest version](https://pandas.pydata.org/docs/whatsnew/index.html) of pandas.

- [X] I have confirmed this bug exists on the master branch of pandas.


### Reproducible Example

```python
import pandas as pd

def test_apply(data):
    print(data)
    data['date'] = pd.to_datetime(data['date'])

dt1 = pd.DataFrame({
                    'key': ['aa', 'bb', 'cc', 'dd'],
                    'date': ['2020-01-01', '2020-01-01', '2020-02-01', '2020-02-01'],
                    'qty': [1.0, 1.0, 3.0, 3.0]})
dt1.groupby("key").apply(lambda x: test_apply(x))
```


### Issue Description

在“test_apply”中对输入参数data中类型为object的字段'date'使用pd.to_datetime()函数之后，在test_apply函数中循环打印lambda函数中的x，期望结果为每次打印key值不同，分别为“aa”,"bb","cc","dd",但实际结果为"aa","aa","aa","aa"。
After using the pd.to_datetime() function in the "test_apply" field "date" with the type of object in the input parameter data, print the x in the lambda function in the test_apply function in a loop. The expected result is that the key value is different each time it is printed, respectively "aa","bb","cc","dd", but the actual result is "aa","aa","aa","aa".
  key        date  qty
0  aa  2020-01-01  1.0
  key       date  qty
1  aa 2020-01-01  1.0
  key       date  qty
2  aa 2020-01-01  3.0
  key       date  qty
3  aa 2020-01-01  3.0

### Expected Behavior

  key        date  qty
0  aa  2020-01-01  1.0
  key       date  qty
1  bb 2020-01-01  1.0
  key       date  qty
2  cc 2020-01-01  3.0
  key       date  qty
3  dd 2020-01-01  3.0

### Installed Versions

<details>
python           : 3.6.5.final.0
pandas           : 1.1.5
</details>
