Title: while read loop ignoring last line in file
Tags: bash
Question: Reading in a file , ```members_08_14.csv``` which just contains a list of numbers, the while loop is reading each line. For each line, the number is matched against a regex to ensure that it's only numbers and exactly 11 characters long.

```while read card
do
    if [[ $card =~ ^[0-9]{11}$ ]]
    then
        echo "some sql statement with $card" &gt;&gt; temp.sql;
    else
        echo "Invalid card number in file: $card";
    fi
done <registered/members_08_14.csv
```

The interesting thing is, the else is not being executed if the regex does not match. I would expect that either the line would be written to ```temp.sql```, or a line would be printed to ```stdout``` saying the card number is invalid. 

The behaviour, however, is more along the lines of either only the ```true``` condition or only the ```false``` condition gets activated for the entire file. Why would this be?

Here's the contents of registered/members_08_14.csv:

```47678009583
47678009585
47678009587
47678009590
476780095905
```

The first lines are valid, the 5th line is invalid.

Output of ```cat -vte registered/members_08_14.csv```

```47678009583$
47678009585$
47678009587$
47678009590$
476780095905$
```

Comment: I have no idea, but it works great on my computer.

Comment: Show output of `cat -vte registered/members_08_14.csv` in your question.

Comment: When I run that code, I get `Invalid card number in file: 476780095905`

Comment: Maybe the last line of your file is missing a trailing newline? If that's the case, `read` will fail and the entire loop iteration won't run.

Comment: What does `set -x` give you for the iteration with the line in question?

Comment: I took the liberty of updating the summary to better reflect the actual problem, in the hope that this helps other folks in the same boat.

Comment: @CharlesDuffy I believe you were correct with a trailing newline. When I added the newline to the end of the file (and then removed it again) it seemed to work. The file did originally come from a Windows machine.

Comment: Well, it works perfectly fine on my bash 4.3-1 on Debian unstable...

Here is the accepted answer: If the last line of your file has no newline on the end, ```read``` will put its content into  ```card``` -- but will then exit with a nonzero value. Because ```read``` has exited with a nonzero value in this case, the ```while``` loop will exit without going on to the statement that runs the james89@example.org.

The easiest fix is to correct the file.

Another approach you can take is to ignore the exit status of ```read``` when it actually populates its destination (and, while at it, to put ```$'\r'``` into IFS, such that ```read``` will ignore the extra characters in DOS newlines):

```while card=; IFS=$' \t\r\n' read -r card || [[ $card ]]; do
    if [[ $card =~ ^[0-9]{11}$ ]]
    then
        echo "some sql statement with $card" &gt;&gt; temp.sql;
    else
        echo "Invalid card number in file: $card";
    fi
done <registered/members_08_14.csv
```

Comment for this answer: Thanks for posting the answer :)

Here is another answer: Perhaps your file is in DOS format that you also get to read carriage returns (```\r```) to the end of the variable. Try to run ```dos2unix file``` or ```sed -i 's|\r||' file```. Another way is to trim out that character after every input through this

```while IFS=$' \t\r\n' read -r card
```

Comment for this answer: @Hypino: But `cat -vte` isn't showing presence of `^M` in each line.
Comment for this answer: If the file were in DOS format, wouldn't _every_ line go through the `else` case?
Comment for this answer: I believe this was it. The file originally came from Windows, and when I added a new line to the end of the file (and then removed it again) it seemed to have worked.
Comment for this answer: @anubhava Sorry for the confusion, the `cat -vte` was from after I made a change as per @CharlesDuffy
Comment for this answer: @CharlesDuffy I'm just guessing. He was not specific which parts of the lines were getting the proper condition or not.

Here is another answer: To read all the lines, regardless of whether they are ended with a new line or not:
```cat &quot;somefile&quot; | { cat ; echo ; } | while read line; do echo $line; done
```
Source : My open source project https://sourceforge.net/projects/command-output-to-html-table/

Comment for this answer: You could of course use `{ cat "somefile"; echo; } | ...` and make the pipeline that much shorter. Or `cat "somefile" <(echo) | ...`

{'source': 'StackExchangeDataset', 'subset': 'stackoverflow.com', 'metadata': {'AnswerCount': 3, 'CommentCount': 8, 'ContentLicense': 'CC BY-SA 3.0', 'CreationDate': '2014:00:08/14/14', 'FavoriteCount': 1.0, 'PostTypeId': 1, 'Tags': '<bash>', 'Title': 'while read loop ignoring last line in file', 'ViewCount': 2048, 'id': 25318028, 'user_AboutMe': "<p>You'll always see people helping.</p>\n", 'user_AccountId': 1028197.0, 'user_DisplayName': 'Hypino', 'user_Reputation': 1200.0, 'user_Views': 91.0, 'user_id': 1037018.0}}