rdb.c: handle fclose error case differently to avoid double fclose (#7307)

When fclose would fail, the previous implementation would have attempted to do fclose again
this can in theory lead to segfault.

other changes:
check for non-zero return value as failure rather than a specific error code.
this doesn't fix a real bug, just a minor cleanup.