airflow/dags/util.py
def generate_gcs_payload(workflow_id: str, filename: str = None,
                         url: str = None, gcs_bucket: str = None) -> dict:
    """Creates the payload object required for the GCS ingestion operator.

    workflow_id: ID of the datasource workflow. Should match ID defined in
                 DATA_SOURCES_DICT.
    filename: Name of gcs file to store the data in.
    url: URL where the data lives.
    gcs_bucket: GCS bucket to write to. Defaults to the GCS_LANDING_BUCKET env
                var."""
    message = get_required_attrs(workflow_id, gcs_bucket=gcs_bucket)
    if filename is not None:
        message['filename'] = filename
    if url is not None:
        message['url'] = url
    return {'message': message}
Creates the payload object required for the GCS ingestion operator.

    workflow_id: ID of the datasource workflow. Should match ID defined in
                 DATA_SOURCES_DICT.
    filename: Name of gcs file to store the data in.
    url: URL where the data lives.
    gcs_bucket: GCS bucket to write to. Defaults to the GCS_LANDING_BUCKET env
                var.
Creates the payload object required for the GCS ingestion operator.
workflow_id: ID of the datasource workflow. Should match ID defined in
DATA_SOURCES_DICT.
filename: Name of gcs file to store the data in.
url: URL where the data lives.
gcs_bucket: GCS bucket to write to. Defaults to the GCS_LANDING_BUCKET env
var.
Creates the payload object required for the GCS ingestion operator . workflow_id : ID of the datasource workflow . Should match ID defined in DATA_SOURCES_DICT . filename : Name of gcs file to store the data in . url : URL where the data lives . gcs_bucket : GCS bucket to write to . Defaults to the GCS_LANDING_BUCKET env var .
def generate_gcs_payload(workflow_id: str, filename: str = None,
                         url: str = None, gcs_bucket: str = None) -> dict:
    message = get_required_attrs(workflow_id, gcs_bucket=gcs_bucket)
    if filename is not None:
        message['filename'] = filename
    if url is not None:
        message['url'] = url
    return {'message': message}
def generate_gcs_payload ( workflow_id : str , filename : str = None , url : str = None , gcs_bucket : str = None ) -> dict : message = get_required_attrs ( workflow_id , gcs_bucket = gcs_bucket ) if filename is not None : message [ 'filename' ] = filename if url is not None : message [ 'url' ] = url return { 'message' : message }
Creates the payload object required for the GCS ingestion operator.
Creates the payload object required for the GCS ingestion operator .
"""Creates the payload object required for the GCS ingestion operator.

    workflow_id: ID of the datasource workflow. Should match ID defined in
                 DATA_SOURCES_DICT.
    filename: Name of gcs file to store the data in.
    url: URL where the data lives.
    gcs_bucket: GCS bucket to write to. Defaults to the GCS_LANDING_BUCKET env
                var."""
workflow_id
str
filename
str
url
str
gcs_bucket
str