# Select DataFrame rows between two dates

StackOverflow

I am creating a DataFrame from a csv as follows:

``````stock = pd.read_csv("data_in/" + filename + ".csv", skipinitialspace=True)
``````

The DataFrame has a date column. Is there a way to create a new DataFrame (or just overwrite the existing one) which only contains rows with date values that fall within a specified date range or between two specified date values?

There are two possible solutions:

• Use a boolean mask, then use `df.loc[mask]`
• Set the date column as a DatetimeIndex, then use `df[start_date : end_date]`

Ensure `df["date"]` is a Series with dtype `datetime64[ns]`:

``````df["date"] = pd.to_datetime(df["date"])
``````

Make a boolean mask. `start_date` and `end_date` can be `datetime.datetime`s, `np.datetime64`s, `pd.Timestamp`s, or even datetime strings:

``````#greater than the start date and smaller than the end date
mask = (df["date"] > start_date) & (df["date"] <= end_date)
``````

Select the sub-DataFrame:

``````df.loc[mask]
``````

or re-assign to `df`

``````df = df.loc[mask]
``````

For example,

``````import numpy as np
import pandas as pd

df = pd.DataFrame(np.random.random((200,3)))
df["date"] = pd.date_range("2000-1-1", periods=200, freq="D")
mask = (df["date"] > "2000-6-1") & (df["date"] <= "2000-6-10")
``````

yields

``````            0         1         2       date
153  0.208875  0.727656  0.037787 2000-06-02
154  0.750800  0.776498  0.237716 2000-06-03
155  0.812008  0.127338  0.397240 2000-06-04
156  0.639937  0.207359  0.533527 2000-06-05
157  0.416998  0.845658  0.872826 2000-06-06
158  0.440069  0.338690  0.847545 2000-06-07
159  0.202354  0.624833  0.740254 2000-06-08
160  0.465746  0.080888  0.155452 2000-06-09
161  0.858232  0.190321  0.432574 2000-06-10
``````

Using a DatetimeIndex:

If you are going to do a lot of selections by date, it may be quicker to set the `date` column as the index first. Then you can select rows by date using `df.loc[start_date:end_date]`.

``````import numpy as np
import pandas as pd

df = pd.DataFrame(np.random.random((200,3)))
df["date"] = pd.date_range("2000-1-1", periods=200, freq="D")
df = df.set_index(["date"])
print(df.loc["2000-6-1":"2000-6-10"])
``````

yields

``````                   0         1         2
date
2000-06-01  0.040457  0.326594  0.492136    # <- includes start_date
2000-06-02  0.279323  0.877446  0.464523
2000-06-03  0.328068  0.837669  0.608559
2000-06-04  0.107959  0.678297  0.517435
2000-06-05  0.131555  0.418380  0.025725
2000-06-06  0.999961  0.619517  0.206108
2000-06-07  0.129270  0.024533  0.154769
2000-06-08  0.441010  0.741781  0.470402
2000-06-09  0.682101  0.375660  0.009916
2000-06-10  0.754488  0.352293  0.339337
``````

While Python list indexing, e.g. `seq[start:end]` includes `start` but not `end`, in contrast, Pandas `df.loc[start_date : end_date]` includes both end-points in the result if they are in the index. Neither `start_date` nor `end_date` has to be in the index however.

Also note that `pd.read_csv` has a `parse_dates` parameter which you could use to parse the `date` column as `datetime64`s. Thus, if you use `parse_dates`, you would not need to use `df["date"] = pd.to_datetime(df["date"])`.

I feel the best option will be to use the direct checks rather than using loc function:

``````df = df[(df["date"] > "2000-6-1") & (df["date"] <= "2000-6-10")]
``````

It works for me.

Major issue with loc function with a slice is that the limits should be present in the actual values, if not this will result in KeyError.

You can also use `between`:
``````df[df.some_date.between(start_date, end_date)]
{"ft_lang_label":"__label__en","ft_lang_prob":0.6967454,"math_prob":0.95147866,"size":4059,"snap":"2021-43-2021-49","text_gpt3_token_len":1300,"char_repetition_ratio":0.13045622,"word_repetition_ratio":0.047377326,"special_character_ratio":0.42399606,"punctuation_ratio":0.16812228,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.9826203,"pos_list":[0,1,2,3,4],"im_url_duplicate_count":[null,null,null,null,null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2021-12-04T07:19:24Z\",\"WARC-Record-ID\":\"<urn:uuid:e56fee13-24fe-459a-9f49-915cf11d066c>\",\"Content-Length\":\"34194\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:4d28ac23-d807-4950-99be-e77c4ffef609>\",\"WARC-Concurrent-To\":\"<urn:uuid:8d674063-7c5a-4bb3-8a94-c54f12b8ea5e>\",\"WARC-IP-Address\":\"172.67.181.212\",\"WARC-Target-URI\":\"https://python.engineering/29370057-select-dataframe-rows-between-two-dates/\",\"WARC-Payload-Digest\":\"sha1:SWJF74QKGUNM6IEGLEBK6DLKSJ67LNZK\",\"WARC-Block-Digest\":\"sha1:XIM3B22PHAI5B2LW2KUMYLBDJ2P3SLEB\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2021/CC-MAIN-2021-49/CC-MAIN-2021-49_segments_1637964362952.24_warc_CC-MAIN-20211204063651-20211204093651-00235.warc.gz\"}"}