Criei a conexão no apache airflow e após isso criei meu gancho, ao rodar no terminal deu vários erros.
Criei a conexão no apache airflow e após isso criei meu gancho, ao rodar no terminal deu vários erros.
Eu também estou com problemas ao executar o script twitter_hook.py. Já criei a conexão com o twitter, mas não dá certo :(
Ola Anderson, Se puder envie uma mensagem com os erros que voce esta recebendo.
tamb'em d'a erro
warnings.warn(
[2021-09-27 08:08:39,488] {connection.py:344} ERROR - Invalid control character at: line 1 column 26 (char 25)
Traceback (most recent call last):
File "/home/edilson/Documentos/FMU/datapipeline/.env/lib/python3.8/site-packages/airflow/models/connection.py", line 342, in extra_dejson
obj = json.loads(self.extra)
File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.8/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Invalid control character at: line 1 column 26 (char 25)
[2021-09-27 08:08:39,488] {connection.py:345} ERROR - Failed parsing the json for conn_id twitter_default
[2021-09-27 08:08:39,489] {base_hook.py:89} INFO - Using connection to: id: twitter_default. Host: https://api.twitter.com, Port: None, Schema: None, Login: None, Password: None, extra: None
[2021-09-27 08:08:39,489] {connection.py:344} ERROR - Invalid control character at: line 1 column 26 (char 25)
Traceback (most recent call last):
File "/home/edilson/Documentos/FMU/datapipeline/.env/lib/python3.8/site-packages/airflow/models/connection.py", line 342, in extra_dejson
obj = json.loads(self.extra)
File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.8/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Invalid control character at: line 1 column 26 (char 25)
[2021-09-27 08:08:39,489] {connection.py:345} ERROR - Failed parsing the json for conn_id twitter_default
[2021-09-27 08:08:39,490] {twitter_hook.py:41} INFO - URL: https://api.twitter.com/2/tweets/search/recent?query=fmuoficial&tweet.fields=author_id,created_at,public_metrics,text
/home/edilson/Documentos/FMU/datapipeline/.env/lib/python3.8/site-packages/urllib3/connectionpool.py:981: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.twitter.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
warnings.warn(
[2021-09-27 08:08:39,792] {http_hook.py:150} ERROR - HTTP error: Unauthorized
[2021-09-27 08:08:39,792] {http_hook.py:151} ERROR - {"title":"Unauthorized","detail":"Unauthorized","type":"about:blank","status":401}
Traceback (most recent call last):
File "/home/edilson/Documentos/FMU/datapipeline/.env/lib/python3.8/site-packages/airflow/hooks/http_hook.py", line 148, in check_response
response.raise_for_status()
File "/home/edilson/Documentos/FMU/datapipeline/.env/lib/python3.8/site-packages/requests/models.py", line 941, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://api.twitter.com/2/tweets/search/recent?query=fmuoficial&tweet.fields=author_id,created_at,public_metrics,text
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "twitter_hook.py", line 64, in <module>
for pg in TwitterHook("fmuoficial").run():
File "twitter_hook.py", line 61, in run
yield from self.paginate(url, session)
File "twitter_hook.py", line 50, in paginate
data = self.connect_to_endpoint(full_url, session)
File "twitter_hook.py", line 42, in connect_to_endpoint
return self.run_and_check(session, prep, {}).json()
File "/home/edilson/Documentos/FMU/datapipeline/.env/lib/python3.8/site-packages/airflow/hooks/http_hook.py", line 181, in run_and_check
self.check_response(response)
File "/home/edilson/Documentos/FMU/datapipeline/.env/lib/python3.8/site-packages/airflow/hooks/http_hook.py", line 152, in check_response
raise AirflowException(str(response.status_code) + ":" + response.reason)
airflow.exceptions.AirflowException: 401:Unauthorized