AirflowからBigQueryのScheduledQueryを実行する

Airflowライブラリ一覧 によると、BigQueryDataTransferServiceStartTransferRunsOperatorを利用すればいいらしい。

 from airflow.providers.google.cloud.operators.bigquery_dts import BigQueryDataTransferServiceStartTransferRunsOperator
...
 # example
   gcp_bigquery_start_transfer = BigQueryDataTransferServiceStartTransferRunsOperator(
       task_id="gcp_bigquery_start_transfer",
       transfer_config_id=transfer_config_id,
       requested_run_time={"seconds": int(time.time() + 60)},
   )
...

https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/example_dags/example_bigquery_dts.py

自分用メモ

GcloudのCLIでスケジュール実行する

bq mk --transfer_run [--run_time=RUN_TIME | --start_time=START_TIME --end_time=END_TIME] CONFIG

参照:  ​https://cloud.google.com/bigquery/docs/reference/bq-cli-reference#synopsis_13