paho-mqtt authentication error on pythonanywher - paho

I am trying to publish to an mqtt topic on beebotte.com using a simple publish.single on my linux machine it works fine but on python anywhere I get an authentication error. There are small differences in minor version numbers, can that be what is different?
This is the code I put into the python console:
import paho.mqtt.publish as publish
mqtt_host = "beebotte.com"
mqtt_topic = "climate/set/livingroom"
auth = {'username':"token:MY_SECRET_TOKEN"}
publish.single(mqtt_topic, "python sent", hostname=mqtt_host, auth = auth)
this is the error:
Python 2.7.6 (default, Oct 26 2016, 20:30:19)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import paho.mqtt.publish as publish
>>> mqtt_host = "beebotte.com"
>>> mqtt_topic = "climate/set/livingroom"
>>> auth = {'username':"MY_SECRET_TOKEN"}
>>> publish.single(mqtt_topic, "python sent", hostname=mqtt_host, auth = auth)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/<MYUSER>/python-email/local/lib/python2.7/site-packages/paho/mqtt/publish.py", line 216, in single
protocol, transport)
File "/home/<MYUSER>/python-email/local/lib/python2.7/site-packages/paho/mqtt/publish.py", line 152, in multiple
client.connect(hostname, port, keepalive)
File "/home/<MYUSER>/python-email/local/lib/python2.7/site-packages/paho/mqtt/client.py", line 768, in connect
return self.reconnect()
File "/home/<MYUSER>/python-email/local/lib/python2.7/site-packages/paho/mqtt/client.py", line 895, in reconnect
sock = socket.create_connection((self._host, self._port), source_address=(self._bind_address, 0))
File "/usr/lib/python2.7/socket.py", line 571, in create_connection
raise err
socket.error: [Errno 111] Connection refused
>>>
>>> exit()
and here it is working:
Python 2.7.13 (default, Nov 24 2017, 17:33:09)
[GCC 6.3.0 20170516] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import paho.mqtt.publish as publish
>>> mqtt_host = "beebotte.com"
>>> mqtt_topic = "climate/set/livingroom"
>>> auth = {'username':"MY_SECRET_TOKEN"}
>>> publish.single(mqtt_topic, "python sent", hostname=mqtt_host, auth = auth)
>>>
>>> exit()
edit (I'm not sure if this is true):
The key is in the actual format of token:token_KxDxlcmXgQBDfWRNC (not real) beebotte.com also accepts a so-called secret key in the format of 2A4Gfgv0puYFBEVbBQX24szALcyDvMRh If I use the secret key it works from pythonanywhere (sometimes). This is leading me to beleive it is some kind of formatting problem with the : in the token.
So the question now is how to format it to work properly.

I don't think mqtt will work from a free account on PythonAnywhere. Free accounts can only connect out through a proxy using http(s) to a specific whitelist of sites. If there's an http to mqtt bridge somewhere, you could possibly use that.

Related

netmiko key authentication failure

Seeing some issue with netmiko KEY authentication with N7K.
Python 3.8.10, netmiko 4.1.2.
Firstly tried on N9K without any issue, command can be sent after get the connection.
Python 3.8.10 (default, Jun 22 2022, 20:18:18)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from netmiko import ConnectHandler
>>> n9k = {"device_type": "cisco_nxos", "host": "10.1.1.10", "username": "admin", "use_keys": True,"key_file":"~/.ssh/id_rsa", "passphrase": "Cisco123"}
>>> target_con = ConnectHandler(**n9k)
Hit the issue when trying the connection to a N7K switch.
>>> n7k = {"device_type": "cisco_nxos", "host": "10.1.1.20", "username": "admin", "use_keys": True,"key_file":"~/.ssh/id_rsa", "passphrase": "Cisco123"}
>>> target_con = ConnectHandler(**n7k)
Traceback (most recent call last):
File "/home/admin/netmiko_test/lib/python3.8/site-packages/netmiko/base_connection.py", line 1046, in establish_connection
self.remote_conn_pre.connect(**ssh_connect_params)
File "/home/admin/netmiko_test/lib/python3.8/site-packages/paramiko/client.py", line 435, in connect
self._auth(
File "/home/admin/netmiko_test/lib/python3.8/site-packages/paramiko/client.py", line 771, in _auth
raise saved_exception
File "/home/admin/netmiko_test/lib/python3.8/site-packages/paramiko/client.py", line 747, in _auth
self._transport.auth_publickey(username, key)
File "/home/admin/netmiko_test/lib/python3.8/site-packages/paramiko/transport.py", line 1635, in auth_publickey
return self.auth_handler.wait_for_response(my_event)
File "/home/admin/netmiko_test/lib/python3.8/site-packages/paramiko/auth_handler.py", line 259, in wait_for_response
raise e
paramiko.ssh_exception.AuthenticationException: Authentication failed.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/admin/netmiko_test/lib/python3.8/site-packages/netmiko/ssh_dispatcher.py", line 365, in ConnectHandler
return ConnectionClass(*args, **kwargs)
File "/home/admin/netmiko_test/lib/python3.8/site-packages/netmiko/base_connection.py", line 439, in __init__
self._open()
File "/home/admin/netmiko_test/lib/python3.8/site-packages/netmiko/base_connection.py", line 444, in _open
self.establish_connection()
File "/home/admin/netmiko_test/lib/python3.8/site-packages/netmiko/base_connection.py", line 1083, in establish_connection
raise NetmikoAuthenticationException(msg)
netmiko.exceptions.NetmikoAuthenticationException: Authentication to device failed.
Common causes of this problem are:
1. Invalid username and password
2. Incorrect SSH-key file
3. Connecting to the wrong device
Device settings: cisco_nxos 10.1.1.20:22
Authentication failed.
>>>
The username and ssh key have been validated. All work well if using username/password instead.
Any advice would be appreciated.
Thanks!
Found the issue when studying another similar paramiko problem.
The N9K node I tested against with uses openssh 8.3, thus it is rsa-sha2-256.
The N7K node uses openssh5.9 which is ssh-sha1.
That makes a difference as netmiko seems don't like ssh-sha1 by default.
Adding disabled_algorithms = {'pubkeys': ['rsa-sha2-256', 'rsa-sha2-512']} to ConnectHandler fixed the issue.

netmiko connections from device list is only connecting to first ip in list

Im using
(base) C:\python
Python 3.8.8 (default, Apr 13 2021, 15:08:03) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.sing
Created on Wed Jun 16 10:15:59 2021
import logging
from netmiko import ConnectHandler
logging.basicConfig(filename='test2.log', level=logging.DEBUG)
logger = logging.getLogger("netmiko")
with open('host.txt', "r") as host:
for ip in host.read().splitlines():
cisco = {
'device_type': 'cisco_ios',
'ip': ip,
'username': 'user',
'password': 'password',
}
net_connect = ConnectHandler(**cisco)
print (' #### Connecting to ' + ip)
output = net_connect.find_prompt()
print(output)
net_connect.disconnect()
You need to create a for loop. Netmiko accepts one dictionary within the ConnectHandler class. So to make the code run "for each" device, you have to create a loop.
Also, in the for loop you created to read the IP addresses from hosts.txt, you keep overwriting the cisco dict every time in the loop. cisco = {} overwrites the previous value each time. New values should be appended to a list instead.
You can achieve this by doing:
from netmiko import ConnectHandler
import logging
logging.basicConfig(filename="test2.log", level=logging.DEBUG)
logger = logging.getLogger("netmiko")
with open(file="hosts.txt", mode="r") as hosts:
# A list comprehension
devices = [
{
"device_type": "cisco_ios",
"ip": ip,
"username": "cisco",
"password": "cisco",
}
for ip in hosts.read().splitlines()
]
print(devices) # <--- print value is below
# Connect to each device (one at a time)
for device in devices:
print(f'Connecting to {device["ip"]}') # Here you are still trying to connect
net_connect = ConnectHandler(**device)
print(f'Connected to {device["ip"]}') # Here you are already connected
prompt = net_connect.find_prompt()
net_connect.disconnect() # disconnect from the session
# Finally, print the prompt within the foor loop, but
# after you disconnect. You no longer need the connection to print.
print(prompt)
You can forget about net_connect.disconnect() by using with statement (Context Manager)
It's important to clear the vty line after you are done
for device in devices:
print(f'Connecting to {device["ip"]}') # Here you are still waiting to connect
with ConnectHandler(**device) as net_connect:
print(f'Connected to {device["ip"]}') # Here you are already logged-in
prompt = net_connect.find_prompt()
print(prompt)
If you print the devices list, you will get:
[{'device_type': 'cisco_ios',
'ip': '192.168.1.1', # From hosts.txt file
'password': 'cisco',
'username': 'cisco'},
{'device_type': 'cisco_ios',
'ip': '192.168.1.2', # From hosts.txt file
'password': 'cisco',
'username': 'cisco'}]

Using per-column compression codec in Parquet.write_table

I have pyarrow 2.0.0 installed. The docs for pyarrow.parquet.write_table state
compression (str or dict) – Specify the compression codec, either on a general basis or per-column. Valid values: {‘NONE’, ‘SNAPPY’, ‘GZIP’, ‘LZO’, ‘BROTLI’, ‘LZ4’, ‘ZSTD’}.
Works fine if compression is a string, but when I try using a dict for per-column specification, I get the following error. What am I doing wrong? I can use a similar dict for compression_level on a per-column basis without error.
(py3) C:\tmp\python>python
Python 3.8.5 (default, Sep 3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyarrow as pa
>>> import pyarrow.parquet as pq
>>> import pandas as pd
>>>
>>> df = pd.DataFrame([[1,2,3],[4,5,6]],columns=['foo','bar','baz'])
>>> t = pa.Table.from_pandas(df)
>>> pq.write_table(t,'test1.pq',compression=dict(foo='zstd',bar='snappy',baz='brotli'))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "c:\app\python\anaconda\3\envs\py3\lib\site-packages\pyarrow\parquet.py", line 1717, in write_table
with ParquetWriter(
File "c:\app\python\anaconda\3\envs\py3\lib\site-packages\pyarrow\parquet.py", line 554, in __init__
self.writer = _parquet.ParquetWriter(
File "pyarrow\_parquet.pyx", line 1390, in pyarrow._parquet.ParquetWriter.__cinit__
File "pyarrow\_parquet.pyx", line 1236, in pyarrow._parquet._create_writer_properties
File "stringsource", line 15, in string.from_py.__pyx_convert_string_from_py_std__in_string
TypeError: expected bytes, str found

MicroPython urequests.post failures

I'm trying to receive notifications on my (Android) mobile device from an ESP8266 MCU running MicroPython. For this reason I subscribed to a couple of online services exposing some APIs for this task, Pushbullet, and Pushed, and I installed the matching apps on my device.
This is what I'm trying:
Pushbullet:
import json
import urequests
body = "Test Notification"
title = "Pushbullet"
data_sent = {"type": "note", "title": title, "body": body}
API_KEY = 'xxxxxxxxxxxxxxxxxxxxxxxxxxx'
pb_headers = {
'Authorization': 'Bearer ' + API_KEY,
'Content-Type': 'application/json'
}
r = urequests.post(
'https://api.pushbullet.com/v2/pushes',
data=json.dumps(data_sent),
headers=pb_headers
)
print(r)
Error:
ssl_handshake_status: -256
Traceback (most recent call last):
File "<stdin>", line 11, in <module>
File "urequests.py", line 104, in post
File "urequests.py", line 56, in request
OSError: [Errno 5] EIO
Pushed:
import json
import urequests
payload = {
"app_key": "xxxxxxxxxxxxxxxxxxxxxxxxxxx",
"app_secret": "xxxxxxxxxxxxxxxxxxxxxxxxxxx",
"target_type": "app",
"content": "Remote Mic MCU test from ESP8266"
}
r = urequests.post("https://api.pushed.co/1/push", data=payload)
print(r)
Error:
Traceback (most recent call last):
File "<stdin>", line 8, in <module>
File "urequests.py", line 104, in post
File "urequests.py", line 74, in request
TypeError: object with buffer protocol required
Searching for these errors, doesn't get me anywhere useful.
The exact same code snippets work OK on my Linux box (using requests instead of urequests), but I understand that urequests may have some limitations.
Do you have any hint on how to fix this?
The exception message suggests that you pass the type of data which urequests doesn't expect. From my knowledge of how HTTP POST works (see HTTP standard), I know that it accepts octet stream, which in Python would be represented by str or bytes type. Whereas you pass a dictionary.
`
problem is with urequests library, if you can switchback to usocket and close the socket before timeout then works fine. For urequests you need to have a board with spiram.

Python EOF shortcut on Windows

Hello i worte this simple program for encripting stings.
It requires from user to enter a simple string after doing that user shoud hit EOF shortcut (CTRL+D or CTRL+Z) but nothing than entering ^D or ^Z happens. Everything is fine when I do this on Ubuntu (using CTRL+D), but on windows this problem occurs. (Windows 10, python 2.7) Is there any other way to enter EOF in userinput in console.
Program:
import string
table = string.maketrans("abcdefghijklmnopqrstuvwxyz", "nopqrstuvwxyzabcdefghijklm")
import fileinput
for line in fileinput.input():
line = line.rstrip()
print string.translate(line, table)
You can use sys.stdin.readline() that provides automatic EOL
Once you hit CTRL D , the program will quit gracefully.
Modified Code
import string
import atexit
import sys
#import fileinput
table = string.maketrans("abcdefghijklmnopqrstuvwxyz", "nopqrstuvwxyzabcdefghijklm")
while True:
line = sys.stdin.readline() #readline returns EOF by default after read
if line: #If data is read
line = line.rstrip() #read line and remove whitespace
print string.translate(line, table) #print translated string
else:
sys.exit(0) #exit on 'CTRL D'
Program Output
Python 2.7.9 (default, Dec 10 2014, 12:24:55) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> ================================ RESTART ================================
>>>
wefjdjbgngfpbgf
jrswqwotatscots
jgnfrejgrtnhkeritrevndsksdlv
wtaserwtegauxrevgeriaqfxfqyi
>>>

Resources