Addressing #40781 to raise ValueError as per discussion when a list with xlim or ylim is supplied, that contains elements in other format than integer or float type.
- [x] closes #40781 
- [x] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry
