Airflow Time Zone Conversion Inside Jinja Template
At work, while writing an Airlfow DAG I needed to convert {{ execution_date }}
macro from UTC to IST inside Jinja template. Initially I thought {{ execution_date }}
gives a datatime object, so converting it inside Jinja template would be tricky as the it requires a tzinfo object.
For example,
from datetime import datetime
from dateutil.tz import gettz
utc = gettz("UTC")
utc_now = datetime.now(utc)
print(f"UTC: {utc_now}")
# UTC: 2020-06-17 16:04:04.952171+00:00
ist = gettz("Asia/Kolkata")
ist_now = utc_now.astimezone(ist)
print(f"IST: {ist_now}")
#IST: 2020-06-17 21:34:04.952171+05:30
Luckily, {{ execution_date }}
is not a datetime
object. Airflow uses Pendulum for datetime and it is an instance of Pendulum
. Pendulum
class has in_timezone(tz: str)
(or in_tz(tz: str)
) method that creates a new Pendulum
object in the specified time zone.
For example,
from pendulum import datetime
utc_now = datetime.now("UTC")
print(f"UTC: {utc_now}")
# UTC: 2020-06-17T16:09:29.844393+00:00
ist_now = utc_now.in_timezone("Asia/Kolkata")
print(f"IST: {ist_now}")
# IST: 2020-06-17T21:39:29.844393+05:30
So I just had to do {{ execution_date.in_timezone("Asia/Kolkata") }}
for the conversion.
An example DAG:
from datetime import datetime
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
dag = DAG(
dag_id="example",
schedule_interval=None,
start_date=datetime(year=1993, month=8, day=31)
)
with dag:
echo = BashOperator(
task_id="echo",
bash_command="""
echo UTC: {{ execution_date }}
echo IST: {{ execution_date.in_timezone('Asia/Kolkata') }}
"""
)
Log from Airflow:
...
[2020-06-17 22:01:11,793] {bash_operator.py:122} INFO - Output:
[2020-06-17 22:01:11,799] {bash_operator.py:126} INFO - UTC: 2020-06-17T16:30:37.072299+00:00
[2020-06-17 22:01:11,800] {bash_operator.py:126} INFO - IST: 2020-06-17T22:00:37.072299+05:30
[2020-06-17 22:01:11,800] {bash_operator.py:130} INFO - Command exited with return code 0
...
Similarly, we can also use it inside any custom operators.