Galera,
Um abençoado deu crontab -e, logo, apagou todo meu crontab. Apos a criação ele nunca mais funcionou e para piorar não sei o que eu tinha feito antes para funcionar kkkkkk
Basicamento o meu cron executa um .sh que executa outros códigos. Lembre-se antes de dar dar o crontab -r o flow funcionava normalmente e o .sh roda normalmente quando executamos o mesmo, mas pelo cron não.
Executável do .sh
0 */6 * * * root /datascience/engineering/contractconditions/2hours/contractconditions_2hours_full_flow.sh
Código do contractconditions_2hours_full_flow.sh
#!/bin/bash
set -e
python_interpreter=/root/anaconda3/bin/python
bigquery=/snap/bin/bq
dog=/root/anaconda3/bin/dog
$dog metric post data.engineering.etl.flowstart.contractconditions.2hours 1
cd /datascience/engineering/contractconditions/2hours
$dog metric post data.engineering.etl.directorynavigate.contractconditions.2hours 1
echo "Realizando mongoexport"
sh ./contractconditions_2hours_export.sh
$dog metric post data.engineering.etl.mongoexport.contractconditions.2hours 1
echo "Realizando flatten e gerando schema"
$python_interpreter ./contractconditions_2hours_parse.py
$dog metric post data.engineering.etl.preprocess.contractconditions.2hours 1
echo "Subindo ao BigQuery"
$bigquery load --schema_update_option=ALLOW_FIELD_ADDITION --source_format=NEWLINE_DELIMITED_JSON --max_bad_records=3 --ignore_unknown_values --schema=./schemas/contractconditions_last2hours_treated_schema.json data-engineering-2:HugeData.ContractConditions_log ./treatedjson/contractconditions_last2hours_treated.json
$dog metric post data.engineering.etl.bigquery.upload.contractconditions.2hours 1
echo "Gerando consolidation query"
$python_interpreter ./contractconditions_consolidation_query.py
$dog metric post data.engineering.etl.bigquery.consolidationquery.generate.contractconditions.2hours 1
echo "Submittando consolidation query às tabelas live e log"
$bigquery query --use_legacy_sql=False --flagfile=./schemas/query.txt
$dog metric post data.engineering.etl.bigquery.consolidationquery.submit.contractconditions.2hours 1
$dog metric post data.engineering.etl.flowend.contractconditions.2hours 1