fiz todo o passo a passo do curso, tudo funcionou
no final, copiei o código do arquivo "dados_climaticos.py", alterando apenas a data, o endereço e a chave da API
no entanto, na hora de executar o DAG, ele apenas cria a pasta e dá erro na hora de extrair os dados
se eu executo o arquivo "extrai_infos_clima.py" ele funciona normalmente
segue print
como resolver?
segue descrição do erro (log do airflow):
[2024-07-11, 20:46:08 UTC] {taskinstance.py:1569} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=dados_climaticos
AIRFLOW_CTX_TASK_ID=extrai_dados
AIRFLOW_CTX_EXECUTION_DATE=2024-06-24T00:00:00+00:00
AIRFLOW_CTX_TRY_NUMBER=1
AIRFLOW_CTX_DAG_RUN_ID=scheduled__2024-06-24T00:00:00+00:00
[2024-07-11, 20:46:09 UTC] {taskinstance.py:1889} ERROR - Task failed with exception
Traceback (most recent call last):
File "/home/gabriel/Documents/airflowalura/venv/lib/python3.9/site-packages/airflow/operators/python.py", line 171, in execute
return_value = self.execute_callable()
File "/home/gabriel/Documents/airflowalura/venv/lib/python3.9/site-packages/airflow/operators/python.py", line 189, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
File "/home/gabriel/Documents/airflowalura/dags/dados_climaticos.py", line 27, in extrai_dados
dados = pd.read_csv(URL)
File "/home/gabriel/Documents/airflowalura/venv/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1026, in read_csv
return _read(filepath_or_buffer, kwds)
File "/home/gabriel/Documents/airflowalura/venv/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 620, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "/home/gabriel/Documents/airflowalura/venv/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1620, in __init__
self._engine = self._make_engine(f, self.engine)
File "/home/gabriel/Documents/airflowalura/venv/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1880, in _make_engine
self.handles = get_handle(
File "/home/gabriel/Documents/airflowalura/venv/lib/python3.9/site-packages/pandas/io/common.py", line 728, in get_handle
ioargs = _get_filepath_or_buffer(
File "/home/gabriel/Documents/airflowalura/venv/lib/python3.9/site-packages/pandas/io/common.py", line 384, in _get_filepath_or_buffer
with urlopen(req_info) as req:
File "/home/gabriel/Documents/airflowalura/venv/lib/python3.9/site-packages/pandas/io/common.py", line 289, in urlopen
return urllib.request.urlopen(*args, **kwargs)
File "/usr/lib/python3.9/urllib/request.py", line 214, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.9/urllib/request.py", line 523, in open
response = meth(req, response)
File "/usr/lib/python3.9/urllib/request.py", line 632, in http_response
response = self.parent.error(
File "/usr/lib/python3.9/urllib/request.py", line 561, in error
return self._call_chain(*args)
File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain
result = func(*args)
File "/usr/lib/python3.9/urllib/request.py", line 641, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401:
[2024-07-11, 20:46:09 UTC] {taskinstance.py:1395} INFO - Marking task as FAILED. dag_id=dados_climaticos, task_id=extrai_dados, execution_date=20240624T000000, start_date=20240711T234608, end_date=20240711T234609
[2024-07-11, 20:46:09 UTC] {standard_task_runner.py:92} ERROR - Failed to execute job 21 for task extrai_dados (HTTP Error 401: ; 5685)
[2024-07-11, 20:46:09 UTC] {local_task_job.py:156} INFO - Task exited with return code 1
[2024-07-11, 20:46:09 UTC] {local_task_job.py:273} INFO - 0 downstream tasks scheduled from follow-on schedule check
[2024-07-11, 20:48:40 UTC] {taskinstance.py:1159} INFO - Dependencies all met for <TaskInstance: dados_climaticos.extrai_dados scheduled__2024-06-24T00:00:00+00:00 [queued]>
[2024-07-11, 20:48:40 UTC] {taskinstance.py:1159} INFO - Dependencies all met for <TaskInstance: dados_climaticos.extrai_dados scheduled__2024-06-24T00:00:00+00:00 [queued]>
[2024-07-11, 20:48:40 UTC] {taskinstance.py:1356} INFO -
