Java FileInputStream exercise
The cleanest way to solve this would probably be to check the argument before calling the super constructor. This allows the code to fail fast and prevents it from opening the file, rendering the close() call unnecessary:
public class TxtInputStream extends FileInputStream {
    public TextInputStream(String fileName) {
        // super call must be first statement so have to call validation method
        // as argument to super call
        super(checkFileName(fileName));
    }

    private static String checkFileName(String fileName) {
        if (!fileName.endsWith(&quot;.txt&quot;)) {
            throw new UnsupportedFileNameException();
        }
        return fileName;
    }
}

However, this is a pretty bad example for wrappers since the TxtInputStream does not add any functionality at all.
If you want to see some good examples have at look at which JDK classes implement InputStream:

BufferedInputStream: Buffers data, this can be useful when you are only consuming small amounts of data at a time, but reading such small amounts of data would be inefficient, e.g. for a network connection. Instead it reads larger chunks from the wrapped InputStream and when you then consume data it first serves it from the buffer.
CipherInputStream: Decrypts the data
InflaterInputStream: Uncompresses data

So assuming you read encrypted, compressed data over the network you could have:
InputStream networkInputStream = ...
BufferedInputStream bufferedInputStream = new BufferedInputStream(networkInputStream);
CipherInputStream cipherInputStream = new CipherInputStream(bufferedInputStream, cipher);
InflaterInputStream inflaterInputStream = new InflaterInputStream(cipherInputStream);

Then you can simply use inflaterInputStream like any other InputStream and do not even have to worry about all the previous processing steps which are taking place.