MicroPython urequests.post failures - http-post

I'm trying to receive notifications on my (Android) mobile device from an ESP8266 MCU running MicroPython. For this reason I subscribed to a couple of online services exposing some APIs for this task, Pushbullet, and Pushed, and I installed the matching apps on my device.
This is what I'm trying:
Pushbullet:
import json
import urequests
body = "Test Notification"
title = "Pushbullet"
data_sent = {"type": "note", "title": title, "body": body}
API_KEY = 'xxxxxxxxxxxxxxxxxxxxxxxxxxx'
pb_headers = {
'Authorization': 'Bearer ' + API_KEY,
'Content-Type': 'application/json'
}
r = urequests.post(
'https://api.pushbullet.com/v2/pushes',
data=json.dumps(data_sent),
headers=pb_headers
)
print(r)
Error:
ssl_handshake_status: -256
Traceback (most recent call last):
File "<stdin>", line 11, in <module>
File "urequests.py", line 104, in post
File "urequests.py", line 56, in request
OSError: [Errno 5] EIO
Pushed:
import json
import urequests
payload = {
"app_key": "xxxxxxxxxxxxxxxxxxxxxxxxxxx",
"app_secret": "xxxxxxxxxxxxxxxxxxxxxxxxxxx",
"target_type": "app",
"content": "Remote Mic MCU test from ESP8266"
}
r = urequests.post("https://api.pushed.co/1/push", data=payload)
print(r)
Error:
Traceback (most recent call last):
File "<stdin>", line 8, in <module>
File "urequests.py", line 104, in post
File "urequests.py", line 74, in request
TypeError: object with buffer protocol required
Searching for these errors, doesn't get me anywhere useful.
The exact same code snippets work OK on my Linux box (using requests instead of urequests), but I understand that urequests may have some limitations.
Do you have any hint on how to fix this?

The exception message suggests that you pass the type of data which urequests doesn't expect. From my knowledge of how HTTP POST works (see HTTP standard), I know that it accepts octet stream, which in Python would be represented by str or bytes type. Whereas you pass a dictionary.
`

problem is with urequests library, if you can switchback to usocket and close the socket before timeout then works fine. For urequests you need to have a board with spiram.

Related

StatusCode.UNIMPLEMENTED when making Vertex AI API call

I have a simple Python app that invokes a Vertex AI API that fails when it runs and I can't understand why. The application is as follows:
from google.cloud import aiplatform_v1
def sample_list_datasets():
client = aiplatform_v1.DatasetServiceClient()
request = aiplatform_v1.ListDatasetsRequest(
parent="projects/MYPROJECT/locations/us-central1",
)
page_result = client.list_datasets(request=request)
for response in page_result:
print(response)
sample_list_datasets()
when run, it fails with:
E0126 03:52:04.146970105 22462 hpack_parser.cc:1218] Error parsing metadata: error=invalid value key=content-type value=text/html; charset=UTF-8
Traceback (most recent call last):
File "/home/kolban/projects/vertex-ai/datasets/env/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 72, in error_remapped_callable
return callable_(*args, **kwargs)
File "/home/kolban/projects/vertex-ai/datasets/env/lib/python3.7/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/kolban/projects/vertex-ai/datasets/env/lib/python3.7/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNIMPLEMENTED
details = "Received http2 header with status: 404"
debug_error_string = "UNKNOWN:Error received from peer ipv4:108.177.120.95:443 {created_time:"2023-01-26T03:52:04.147076255+00:00", grpc_status:12, grpc_message:"Received http2 header with status: 404"}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "run.py", line 25, in <module>
sample_list_datasets()
File "run.py", line 19, in sample_list_datasets
page_result = client.list_datasets(request=request)
File "/home/kolban/projects/vertex-ai/datasets/env/lib/python3.7/site-packages/google/cloud/aiplatform_v1/services/dataset_service/client.py", line 1007, in list_datasets
metadata=metadata,
File "/home/kolban/projects/vertex-ai/datasets/env/lib/python3.7/site-packages/google/api_core/gapic_v1/method.py", line 113, in __call__
return wrapped_func(*args, **kwargs)
File "/home/kolban/projects/vertex-ai/datasets/env/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 74, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.MethodNotImplemented: 501 Received http2 header with status: 404
What might I be doing wrong?
Changing the code to the following caused it to work:
from google.cloud import aiplatform_v1
from google.api_core.client_options import ClientOptions
def sample_list_datasets():
service_base_path='aiplatform.googleapis.com'
region='us-central1'
client_options = ClientOptions(api_endpoint=f"{region}-{service_base_path}")
client = aiplatform_v1.DatasetServiceClient(client_options=client_options)
request = aiplatform_v1.ListDatasetsRequest(
parent="projects/MYPROJECT/locations/us-central1",
)
# Make the request
page_result = client.list_datasets(request=request)
# Handle the response
for response in page_result:
print(response)
sample_list_datasets()
The resolution was hinted at in the documentation for the API request found here. At that article there is a code sample and in the code sample there are some comments and in the comments the following is written:
It may require specifying regional endpoints when creating the service
client as shown in:
https://googleapis.dev/python/google-api-core/latest/client_options.html
And this was the core clue. When we make Vertex AI calls we must specify where the request is to be sent. We do this by setting the api_endpoint option to a URL of the form [REGION]-aiplatform.googleapis.com.

Bubble.io API Connector with Lambda + AWS Rekognition Problem

Hi I was trying to use AWS Rekognition -> Lambda -> AWS API -> Bubble.io
I want to pass a image base64 code to process the image recognition.
Here is the lambda code:
import json
import boto3
import base64
def lambda_handler(event, context):
def detect_labels():
# 1. create a client object, to connect to rekognition
client=boto3.client('rekognition')
image = base64.b64decode(event['face'])
# 4. call Rekognition, store result in 'response'
response = client.detect_labels(
Image={
'Bytes': image
},
MaxLabels=20,
)
#6. Return response from function
return response
# Call detect_labels
response = detect_labels()
# Return results to API gateway
return {
'statusCode': 200,
'body': json.dumps(response)
}
and if I test it within the lambda test with config:
{
"face": "<imgbase64codehere>"
}
It works fine and return success message.
However I set the same thing in Bubble.io
Setting img. It wont work.
I check the Cloudwatch, the error is:
[ERROR] KeyError: 'face'
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 30, in lambda_handler
response = detect_labels()
File "/var/task/lambda_function.py", line 14, in detect_labels
image = base64.b64decode(event['face'])
[ERROR] KeyError: 'face' Traceback (most recent call last): File "/var/task/lambda_function.py", line 30, in lambda_handler response = detect_labels() File "/var/task/lambda_function.py", line 14, in detect_labels image = base64.b64decode(event['face'])
I am sure there is not connection problem (if I dont use JSON it works fine)
So what do I do wrong in the setting?
Thanks for your help in advance
In your test you are sending an input of just a JSON request with face.
It looks like the implementation you've setup uses Amazon API Gateway. If you are doing that your Lambda function might receive the event in a different format depending on how you've implemented the API Gateway integration.
The Lambda developer guide has an example of the JSON format. You will probably want to access the body attribute.

why i keep having the same error in that code

i'm following a tutorial course i'm trying to make a web application in python 3.7 that see if there is any bad words in a text file
the text file contains the following text :
-- Houston, we have a problem. (Apollo 13)
-- Mama always said, life is like a box of chocolates. You never know what you are going to get. (Forrest Gump)
-- You cant handle the truth. (A Few Good Men)
-- I believe everything and I believe nothing. (A Shot in the Dark)
import urllib.request
def read_text():
quotes = open (r'C:\Users\M\Desktop\TEMP FILES\movie_quotes.txt')
content_of_file = quotes.read()
print(content_of_file)
quotes.close()
check_profanity(content_of_file)
def check_profanity(text):
connection = urllib.request.urlopen( 'http://www.wdylike.appspot.com/?q='+text)
output = connection.read()
print(output)
connection.close()
read_text()
and i have that error :
Traceback (most recent call last):
File "C:\Users\Mosa Abbas\Desktop\TEMP FILES\ipnd-starter-code-master\ipnd-starter-code-master\stage_3\lesson_3.3_classes\c_profanity_editor\check_profanity.py", line 32, in <module>
read_text()
File "C:\Users\Mosa Abbas\Desktop\TEMP FILES\ipnd-starter-code-master\ipnd-starter-code-master\stage_3\lesson_3.3_classes\c_profanity_editor\check_profanity.py", line 23, in read_text
check_profanity(content_of_file)
File "C:\Users\Mosa Abbas\Desktop\TEMP FILES\ipnd-starter-code-master\ipnd-starter-code-master\stage_3\lesson_3.3_classes\c_profanity_editor\check_profanity.py", line 27, in check_profanity
connection = urllib.request.urlopen( 'http://www.wdylike.appspot.com/?q='+text)
File "C:\Users\Mosa Abbas\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "C:\Users\Mosa Abbas\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 531, in open
response = meth(req, response)
File "C:\Users\Mosa Abbas\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Users\Mosa Abbas\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 569, in error
return self._call_chain(*args)
File "C:\Users\Mosa Abbas\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 503, in _call_chain
result = func(*args)
File "C:\Users\Mosa Abbas\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request
what is that error and what i should do to get rid of it
urllib.error.HTTPError: HTTP Error 400: Bad Request
i'v tried to add this line of code :
url = 'http://www.wdylike.appspot.com/?q='+urllib.parse.quote(text, safe = '/')
and then:
connection = urllib.request.urlopen(url)
and i got
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
i'm a beginner in python and programming web applications so please help :(
When sending a HTTP request through a GET method, it is necessary to escape the GET vars.
So you should probably escape your text before appending it to the URL.
Have a look to question #1695183
i found the solution
I think that happens be because i'm not encoding the string before appending it to the url.
in python3 we should do the following to 'text' before appending it to the url:
text_to_check = urllib.parse.quote_plus(text)
Python2 would be something like this
(urllib was broken into smaller components in python3):
text_to_check = urllib.quote_plus(text_to_check)
This means that, when appending a string with whitespace to the url it will appear as something like "Am+I+cursing%3F" instead of "Am I cursing?".
Full check_profanity() example:
def check_profanity(text_to_check):
text_to_check = urllib.parse.quote_plus(text_to_check)
connection = urlopen(
"http://www.wdylike.appspot.com/?q=" + text_to_check)
output = connection.read()
print(output)
connection.close()

How to integrate gspread with aiohttp

I am trying to use gspread, but I need the library to mesh well with another async library I am using.
After digging through the docs for gspread, I found this function that I can use:
class gspread.Client(auth, session=None)
An instance of this class communicates with Google API.
Parameters:
auth – An OAuth2 credential object. Credential objects are those created by the oauth2client library. https://github.com/google/oauth2client
session – (optional) A session object capable of making HTTP requests while persisting some parameters across requests. Defaults to requests.Session.
Which gives me an optional session parameter. How would I specify the session to use aiohttp?
I wrote a bit of test code, which compiles fine, but running the code crashes.
import aiohttp
import gspread
import random
from oauth2client.service_account import ServiceAccountCredentials
scope = ['https://spreadsheets.google.com/feeds',
'https://www.googleapis.com/auth/drive']
creds = ServiceAccountCredentials.from_json_keyfile_name('client_secret.json', scope)
c = gspread.authorize(creds)
client = gspread.Client(auth=c, session=aiohttp.ClientSession)
sheet = client.open_by_key('1Hkwo9gSpk3NjgPLPkG8kh0zBNw2nxsYWRw0cVdn0JA0')
ws = sheet.get_worksheet(0)
rcount = ws.row_count
msg = ws.cell(random.randint(1,rcount),1).value
print(msg)
The error message I get is below:
Traceback (most recent call last):
File "c:\Users\xxxxx\.vscode\extensions\ms-python.python-2018.9.0\pythonFiles\experimental\ptvsd_launcher.py", line 118, in <module>
vspd.debug(filename, port_num, '', '', run_as)
File "c:\Users\xxxxx\.vscode\extensions\ms-python.python-2018.9.0\pythonFiles\experimental\ptvsd\ptvsd\debugger.py", line 37, in debug
run(address, filename, *args, **kwargs)
File "c:\Users\xxxxx\.vscode\extensions\ms-python.python-2018.9.0\pythonFiles\experimental\ptvsd\ptvsd\_local.py", line 79, in run_file
run(argv, addr, **kwargs)
File "c:\Users\xxxxx\.vscode\extensions\ms-python.python-2018.9.0\pythonFiles\experimental\ptvsd\ptvsd\_local.py", line 140, in _run
_pydevd.main()
File "c:\Users\xxxxx\.vscode\extensions\ms-python.python-2018.9.0\pythonFiles\experimental\ptvsd\ptvsd\_vendored\pydevd\pydevd.py", line 1751, in main
debugger.connect(host, port)
File "c:\Users\xxxxx\.vscode\extensions\ms-python.python-2018.9.0\pythonFiles\experimental\ptvsd\ptvsd\_vendored\pydevd\pydevd.py", line 1107, in run
return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
File "c:\Users\xxxxx\.vscode\extensions\ms-python.python-2018.9.0\pythonFiles\experimental\ptvsd\ptvsd\_vendored\pydevd\pydevd.py", line 1114, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "c:\Users\xxxxx\.vscode\extensions\ms-python.python-2018.9.0\pythonFiles\experimental\ptvsd\ptvsd\_vendored\pydevd\_pydev_imps\_pydev_execfile.py", line 25, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "c:\Users\xxxxx\Desktop\Coding\Discord Bot\Testing\test.py", line 14, in <module>
ws = sheet.get_worksheet(0)
File "C:\Users\xxxxx\AppData\Local\Programs\Python\Python36\lib\site-packages\gspread\models.py", line 141, in get_worksheet
sheet_data = self.fetch_sheet_metadata()
File "C:\Users\xxxxx\AppData\Local\Programs\Python\Python36\lib\site-packages\gspread\models.py", line 123, in fetch_sheet_metadata
r = self.client.request('get', url, params=params)
File "C:\Users\xxxxx\AppData\Local\Programs\Python\Python36\lib\site-packages\gspread\client.py", line 73, in request
headers=headers
TypeError: get() missing 1 required positional argument: 'url'
PS C:\Users\xxxxx\Desktop\Coding\Discord Bot>
Any ideas?
It's impossible to use aiohttp with current version of gspread (3.0.1). the gspread library uses synchronous calls and aiohttp uses asynchronous calls.
Please, reconsider to use compatible library like requests or httplib2.
For anyone wondering, once I posted this question, I found that someone made an async wrapper for gspread. Check out the library here and show this guy your appreciation. I sure am!

How can I use scrapy to fetch the data through proxy, at the same time I do not use middlewares

I am using scrapy to fetch the data, I bought some proxies. I read the scrapy doc, I found that if I want to use proxy, I should write middlewares. Now my question is how can I use proxy without middlewares? I tried like this:
def start_requests(self):
name = my_name
password = password
proxy = my proxy
return[
FormRequest(url,formate={'account':my_name,'password':password},
meta={'proxy':proxy},
callback=self.after_login)
]
def after_login(self, response):
response.xpath(.....)
I got TypeError:argument of type 'NoneType' is not iterable
Here is the Traceback:
Traceback (most recent call last):
File "/home/think/sjs-project/sjs/venv/local/lib/python2.7/site-packages/twisted/internet/endpoints.py", line 542, in connect
timeout=self._timeout, bindAddress=self._bindAddress)
File "/home/think/sjs-project/sjs/venv/local/lib/python2.7/site-packages/twisted/internet/posixbase.py", line 482, in connectTCP
c = tcp.Connector(host, port, factory, timeout, bindAddress, self)
File "/home/think/sjs-project/sjs/venv/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 1165, in __init__
if abstract.isIPv6Address(host):
File "/home/think/sjs-project/sjs/venv/local/lib/python2.7/site-packages/twisted/internet/abstract.py", line 522, in isIPv6Address
if '%' in addr:
TypeError: argument of type 'NoneType' is not iterable
What am I doing wrong? Thanks very much.

Resources