I am trying to learn/try out cloud composer/beam/dataflow on gcp.

I have written functions to do some basic cleaning of data in python, and used a DAG in cloud composer to run this function to download a file from a bucket, process it, and upload it to a bucket at a set frequency.

It was all bespoke written functionality. I am now trying to figure out how I use beam pipeline and data flow instead and use cloud composer to kick off the dataflow job.

The cleaning I am trying to do, is take a csv input of col1,col2,col3,col4,col5 and combine the middle 3 columns to output a csv of col1,combinedcol234,col5.

Questions I have are...

* How do I pull in my own functions within a beam pipeline to do this merge?
* Should I be pulling in my own functions or do beam have built in ways of doing this?
* How do I then trigger a pipeline from a dag?
* Does anyone have any example code on git hub?

I have been googling and trying to research but can't seem to find anything that helps me get my head around it enough.

Any help would be appreciated. Thank you.