read_csv() causes S3 ListObjectsV2 AccessDenied
#### Code Sample

```python
import pandas as pd
import s3fs

file_1 = f"s3://my-bucket/foo.csv"
file_2 = f"s3://my-bucket/bar.csv"
fs = s3fs.S3FileSystem()
if fs.exists(file_2):
    fs.rm(file_2)

# read file_1 which will be committed to the s3fs cache
df = pd.read_csv(file_1)
# copy existing file, path for file_2 is not in s3fs cache
fs.copy(file_1, file_2)
# s3fs.S3FileSystem.cachable = False    # <--------------- WORKAROUND
df_ = pd.read_csv(file_2)
```
#### Problem description

The above code snippet throws a `PermissionError: Access Denied` when it shouldn't. This happens even when I use admin rights for the AWS account to which the affected bucket belongs (`{"Effect": "Allow", "Action": "*", "Resource": "*"}`).

There may actually be two issues and the above workaround fixes only one, so that the second issue doesn't affect me in my specific case.

In the background `s3fs` is used to read files directly from S3. `s3fs` uses caching. This is why, when reading `file_2` a `FileNotFoundError` is initially raised (only `file_1` is in the cache after it's been read). This initial error can be fixed with the above workaround.

Without the workaround the initial `FileNotFoundError` is handled, seemingly, to specifically address the problem caused by the caching mechanism. The next error in the traceback is: `botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied`. (My assumption is that a list operation is used in an attempt to verify that the file does, in fact, not exist, instead of relying on the cache.)  
I'm not quite sure how this is possible or what exactly is going on here. There doesn't seem to be a `ListObjectsV2` action [documented by AWS](https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html). But even if there was, it should be covered when using admin rights.

1. Is there a parameter for `read_csv()` that I'm missing to avoid using the `s3fs` caching mechanism?
2. Is `ListObjectsV2` a special/new/old S3 action that I need to set permissions for separately from the admin rights?

Note: I submitted a very similar [issue with dask](https://github.com/dask/dask/issues/5134) which, in parts, uses pandas under the hood. Even though for the problem described here, fixing the Access Denied issue might suffice, it may be useful to have a parameter to disable `s3fs` caching so it can be used by dask also. I didn't run into the `AccessDenied` issue with dask, so it seems fixing only that wouldn't help with the dask issue. Just something to keep in mind.

#### Output of ``pd.show_versions()``

<details>

[paste the output of ``pd.show_versions()`` here below this line]
INSTALLED VERSIONS
------------------
commit           : None
python           : 3.7.2.final.0
python-bits      : 64
OS               : Darwin
OS-release       : 18.6.0
machine          : x86_64
processor        : i386
byteorder        : little
LC_ALL           : en_US.UTF-8
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 0.25.0
numpy            : 1.16.4
pytz             : 2019.1
dateutil         : 2.8.0
pip              : 19.1.1
setuptools       : 41.0.1
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : None
IPython          : None
pandas_datareader: None
bs4              : None
bottleneck       : None
fastparquet      : None
gcsfs            : None
lxml.etree       : None
matplotlib       : None
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pytables         : None
s3fs             : 0.3.0
scipy            : None
sqlalchemy       : None
tables           : None
xarray           : None
xlrd             : None
xlwt             : None
xlsxwriter       : None

</details>