BUG: failed to aggregate
### 

- [X] I have checked that this issue has not already been reported.

- [X] I have confirmed this bug exists on the [latest version](https://pandas.pydata.org/docs/whatsnew/index.html) of pandas.

- [ ] I have confirmed this bug exists on the master branch of pandas.


### Reproducible Example

```python
import pandas as pd
df1 = pd.DataFrame(dict2 = {'index': {0: 7158892,1: 7158893,2: 7158894,3: 7158895,4: 7158896,5: 7158897,6: 7158898},
 'uid': {0: 1081156,1: 1081156,2: 1081156,3: 1081156,4: 1081156,5: 1081156,6: 1081156},
 'lat': {0: 37.524228,1: 37.546708,2: 37.433964,3: 38.245017,4: 37.546708,5: 37.500817,6: 37.433964},
'lng': {0: -121.94779,1: -122.069067,2: -121.886765,3: -121.455548,4: -122.069067,5: -121.981362,6: -121.886765}})

agg1 = df1.groupby(['uid']).agg(pd.Series.mode)
```


### Issue Description

ValueError: could not broadcast input array from shape (7,4) into shape (7,)

### Expected Behavior

expect something like this:
1081156,[37.433964 37.546708],[-121.886765 -122.069067]

### Installed Versions

<details>
INSTALLED VERSIONS
------------------
commit           : 945c9ed766a61c7d2c0a7cbb251b6edebf9cb7d5
python           : 3.9.7.final.0
python-bits      : 64
OS               : Linux
OS-release       : 5.4.0-81-generic
Version          : #91~18.04.1-Ubuntu SMP Fri Jul 23 13:36:29 UTC 2021
machine          : x86_64
processor        : x86_64
byteorder        : little
LC_ALL           : None
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 1.3.4
numpy            : 1.21.2
pytz             : 2021.1
dateutil         : 2.8.2
pip              : 21.2.4
setuptools       : 58.0.4
Cython           : None
pytest           : None
hypothesis       : None
show more (open the raw output data in a text editor) ...

tabulate         : None
xarray           : 0.19.0
xlrd             : None
xlwt             : None
numba            : None

</details>
