Oracle_Loader Quote in quoted field
The solution is to move the enclosed by clause into each line instead of overall for the table. For example:

create table ext_tb (
    a varchar2(255),
    b varchar2(255),
    height varchar2(255)
) organization external (
    type oracle_loader
    default directory my_data_dir
    access parameters (
        records delimited by newline
        fields terminated by '\t'
        -- DO NOT add the enclosed line here
        lrtrim
        missing field values are null
        (
            a char optionally enclosed by '"',
            b char optionally enclosed by '"',
            height
        )
    ) location('my_file.ttx')
)


Now, that pesky height field will be in the database as "5'6.35"" (three double-quotes in it). You can run a query on the column to remove the quotes.