BigQuery - load data with changing schema
I'm reading your question as "how to detect schema changes in in-coming data and automatically change out-going data to match?"

Obviously, you're going to need to examine the meta-data of the in-coming records. Since its JSON this should be easily handled by whatever technology you use to accept these records. Then you'll need a local store holding the meta data your application is already aware of. As JSON is a tree structure your local meta data store will model a tree. As records with new schema are detected this local store can be updated.

You will need a second local store to hold the structure of your output document and a third to map the input fields to output fields.

The next three parts cannot be automated. If there are transformations from input to output a human will have to write them. Where in the output document structure the new values should appear will also have to be hand-written, as will the mapping from input to output.

My feeling is this will be horribly inefficient. It may be possible, depending on your technology, to use reflection to alter the run-time definition of your output class to accommodate new columns. Good luck.