Solucionado (ver solução)
Solucionado
(ver solução)
1
resposta

Dag não aparece na interface do Airflow

Oi pessoal, eu estou tendo um problema que o Dag não está aparecendo na iterface do Airflow. Eu cirei a pasta "dags" no mesmo local onde está instalado o Airflow, mas mesmo assim nada. Algém teria alguma ideia de como resolver ?

Segue o código:

Insira aqui a descrição dessa imagem para ajudar na acessibilidadeLogs:

 scheduler | [2023-07-01 18:11:45,484] {scheduler_job.py:696} INFO - Starting the scheduler
 scheduler | [2023-07-01 18:11:45,484] {scheduler_job.py:701} INFO - Processing each file at most -1 times
 scheduler | [2023-07-01 18:11:45,486] {executor_loader.py:105} INFO - Loaded executor: SequentialExecutor
 scheduler | [2023-07-01 18:11:45,494] {manager.py:160} INFO - Launched DagFileProcessorManager with pid: 66970
 scheduler | [2023-07-01 18:11:45,495] {scheduler_job.py:1221} INFO - Resetting orphaned tasks for active dag runs
 webserver | [2023-07-01 18:11:45 -0300] [66967] [INFO] Starting gunicorn 20.1.0
 scheduler | [2023-07-01 18:11:45,965] {settings.py:55} INFO - Configured default timezone Timezone('UTC')
 scheduler | [2023-07-01 18:11:45,968] {manager.py:406} WARNING - Because we cannot use more than 1 thread (parsing_processes = 2) when using sqlite. So we set parallelism to 1.
 scheduler | [2023-07-01 18:11:46 -0300] [66969] [INFO] Starting gunicorn 20.1.0
 scheduler | [2023-07-01 18:11:46 -0300] [66969] [INFO] Listening at: http://0.0.0.0:8793 (66969)
 scheduler | [2023-07-01 18:11:46 -0300] [66969] [INFO] Using worker: sync
 scheduler | [2023-07-01 18:11:46 -0300] [66972] [INFO] Booting worker with pid: 66972
 scheduler | [2023-07-01 18:11:46 -0300] [66973] [INFO] Booting worker with pid: 66973
 webserver | [2023-07-01 18:11:46 -0300] [66967] [INFO] Listening at: http://0.0.0.0:8080 (66967)
 webserver | [2023-07-01 18:11:46 -0300] [66967] [INFO] Using worker: sync
 webserver | [2023-07-01 18:11:46 -0300] [66975] [INFO] Booting worker with pid: 66975
 webserver | [2023-07-01 18:11:46 -0300] [66976] [INFO] Booting worker with pid: 66976
 webserver | [2023-07-01 18:11:46 -0300] [66977] [INFO] Booting worker with pid: 66977
 webserver | [2023-07-01 18:11:46 -0300] [66978] [INFO] Booting worker with pid: 66978
 webserver | [2023-07-01 18:11:47,604] {manager.py:585} INFO - Removed Permission menu access on Permissions to role Admin
 webserver | [2023-07-01 18:11:47,617] {manager.py:543} INFO - Removed Permission View: menu_access on Permissions
 webserver | [2023-07-01 18:11:47,654] {manager.py:508} INFO - Created Permission View: menu access on Permissions
 webserver | [2023-07-01 18:11:47,654] {manager.py:511} ERROR - Creation of Permission View Error: (sqlite3.IntegrityError) UNIQUE constraint failed: ab_permission_view.permission_id, ab_permission_view.view_menu_id
 webserver | [SQL: INSERT INTO ab_permission_view (permission_id, view_menu_id) VALUES (?, ?)]
 webserver | [parameters: (5, 15)]
 webserver | (Background on this error at: http://sqlalche.me/e/14/gkpj)
 webserver | [2023-07-01 18:11:47,660] {manager.py:568} INFO - Added Permission menu access on Permissions to role Admin
 webserver | [2023-07-01 18:11:47,683] {manager.py:570} ERROR - Add Permission to Role Error: (sqlite3.IntegrityError) UNIQUE constraint failed: ab_permission_view_role.permission_view_id, ab_permission_view_role.role_id
 webserver | [SQL: INSERT INTO ab_permission_view_role (permission_view_id, role_id) VALUES (?, ?)]
 webserver | [parameters: (187, 1)]
 webserver | (Background on this error at: http://sqlalche.me/e/14/gkpj)
 webserver | [2023-07-01 18:11:47,699] {manager.py:585} INFO - Removed Permission menu access on Permissions to role Admin
 webserver | [2023-07-01 18:11:47,713] {manager.py:543} INFO - Removed Permission View: menu_access on Permissions
 webserver | [2023-07-01 18:11:47,723] {manager.py:508} INFO - Created Permission View: menu access on Permissions
 webserver | [2023-07-01 18:11:47,731] {manager.py:568} INFO - Added Permission menu access on Permissions to role Admin
 webserver | /Users/rodrigoalmeida/opt/anaconda3/lib/python3.9/site-packages/airflow/api_connexion/schemas/task_instance_schema.py:63 RemovedInMarshmallow4Warning: The 'default' argument to fields is deprecated. Use 'dump_default' instead.

Local da pasta: Insira aqui a descrição dessa imagem para ajudar na acessibilidade

1 resposta
solução!

Consegui resolver: bobeira no código, não tinha percebido que a parte final de ordem das tarefas não estava dentro do bloco da DAG

Segue código corrigido:

from airflow.models import DAG
from airflow.utils.dates import days_ago
from airflow.operators.empty import EmptyOperator
from airflow.operators.bash_operator import BashOperator

with DAG(
    'meu_primeiro_dag',
    start_date=days_ago(1),
    schedule_interval='@daily'
) as dag:

    tarefa_1 = EmptyOperator(task_id='tarefa_1')
    tarefa_2 = EmptyOperator(task_id='tarefa_2')
    tarefa_3 = EmptyOperator(task_id='tarefa_3')
    tarefa_4 = BashOperator(
        task_id = 'cria_pasta', 
        bash_command = 'mkdir -p "/Users/rodrigoalmeida/Documents/Projetos/Apache/pasta"'
    )
    ##Soluç
    )
    tarefa_1 >> [tarefa_2,tarefa_3]
    tarefa_3 >> tarefa_4

Quer mergulhar em tecnologia e aprendizagem?

Receba a newsletter que o nosso CEO escreve pessoalmente, com insights do mercado de trabalho, ciência e desenvolvimento de software