O teste inicial recent_search.py funciona, porém quando executo o twitter_hook.py está acusando erro 401, conforme abaixo:
python twitter_hook.py
[2021-12-09 17:41:29,567] {connection.py:344} ERROR - Expecting ',' delimiter: line 1 column 17 (char 16)
Traceback (most recent call last):
File "/home/rodrigo/Documentos/alura/datapipeline/.env/lib/python3.9/site-packages/airflow/models/connection.py", line 342, in extra_dejson
obj = json.loads(self.extra)
File "/home/rodrigo/anaconda3/lib/python3.9/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/home/rodrigo/anaconda3/lib/python3.9/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/home/rodrigo/anaconda3/lib/python3.9/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 17 (char 16)
[2021-12-09 17:41:29,568] {connection.py:345} ERROR - Failed parsing the json for conn_id twitter_default
[2021-12-09 17:41:29,568] {base_hook.py:89} INFO - Using connection to: id: twitter_default. Host: https://api.twitter.com, Port: None, Schema: None, Login: None, Password: None, extra: None
[2021-12-09 17:41:29,568] {connection.py:344} ERROR - Expecting ',' delimiter: line 1 column 17 (char 16)
Traceback (most recent call last):
File "/home/rodrigo/Documentos/alura/datapipeline/.env/lib/python3.9/site-packages/airflow/models/connection.py", line 342, in extra_dejson
obj = json.loads(self.extra)
File "/home/rodrigo/anaconda3/lib/python3.9/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/home/rodrigo/anaconda3/lib/python3.9/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/home/rodrigo/anaconda3/lib/python3.9/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 17 (char 16)
[2021-12-09 17:41:29,568] {connection.py:345} ERROR - Failed parsing the json for conn_id twitter_default
[2021-12-09 17:41:29,569] {twitter_hook.py:36} INFO - URL: https://api.twitter.com/2/tweets/search/recent?query=AluraOnline&tweet.fields=author_id,conversation_id,created_at,id,in_reply_to_user_id,public_metrics,text&expansions=author_id&user.fields=id,name,username,created_at
/home/rodrigo/Documentos/alura/datapipeline/.env/lib/python3.9/site-packages/urllib3/connectionpool.py:981: InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.twitter.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
warnings.warn(
[2021-12-09 17:41:29,950] {http_hook.py:150} ERROR - HTTP error: Unauthorized
[2021-12-09 17:41:29,950] {http_hook.py:151} ERROR - {
"title": "Unauthorized",
"type": "about:blank",
"status": 401,
"detail": "Unauthorized"}
Traceback (most recent call last):
File "/home/rodrigo/Documentos/alura/datapipeline/.env/lib/python3.9/site-packages/airflow/hooks/http_hook.py", line 148, in check_response
response.raise_for_status()
File "/home/rodrigo/Documentos/alura/datapipeline/.env/lib/python3.9/site-packages/requests/models.py", line 941, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://api.twitter.com/2/tweets/search/recent?query=AluraOnline&tweet.fields=author_id,conversation_id,created_at,id,in_reply_to_user_id,public_metrics,text&expansions=author_id&user.fields=id,name,username,created_at
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/rodrigo/Documentos/alura/datapipeline/airflow/plugins/hooks/twitter_hook.py", line 59, in <module>
for pg in TwitterHook("AluraOnline").run():
File "/home/rodrigo/Documentos/alura/datapipeline/airflow/plugins/hooks/twitter_hook.py", line 56, in run
yield from self.paginate(url, session)
File "/home/rodrigo/Documentos/alura/datapipeline/airflow/plugins/hooks/twitter_hook.py", line 45, in paginate
data = self.connect_to_endpoint(full_url, session)
File "/home/rodrigo/Documentos/alura/datapipeline/airflow/plugins/hooks/twitter_hook.py", line 37, in connect_to_endpoint
return self.run_and_check(session, prep, {}).json()
File "/home/rodrigo/Documentos/alura/datapipeline/.env/lib/python3.9/site-packages/airflow/hooks/http_hook.py", line 181, in run_and_check
self.check_response(response)
File "/home/rodrigo/Documentos/alura/datapipeline/.env/lib/python3.9/site-packages/airflow/hooks/http_hook.py", line 152, in check_response
raise AirflowException(str(response.status_code) + ":" + response.reason)
airflow.exceptions.AirflowException: 401:Unauthorized
(.env) (base) rodrigo@Gaming-3-15IMH05:~/Documentos/alura