I'm trying to learn more about the Jupyter wire protocol. I want to collect examples of the messages sent on the IOPub socket.
SETUP:
I start a Jupyter console in one terminal then go find the connection file. In my case the contents are as follows:
{
"shell_port": 62690,
"iopub_port": 62691,
"stdin_port": 62692,
"control_port": 62693,
"hb_port": 62694,
"ip": "127.0.0.1",
"key": "9c6bbbfb-6ad699d44a15189c4f3d3371",
"transport": "tcp",
"signature_scheme": "hmac-sha256",
"kernel_name": ""
}
I create a simple python script as follows:
import zmq
iopub_port = "62691"
ip = "127.0.0.1"
transport = "tcp"
context = zmq.Context()
socket = context.socket(zmq.SUB)
socket.connect(f"{transport}://{ip}:{iopub_port}")
while True:
string = socket.recv()
print(string)
I open a second terminal and execute the script as follows (it blocks, as expected):
python3 script.py
And then I switch back to the first terminal (with the Jupyter console running) and start executing code.
ISSUE: Nothing prints on the second terminal.
EXPECTED: Some Jupyter IO messages, or at least some sort of error.
Uh, help? Is my code fine and this is probably an issue with my config? Or is my code somehow braindead?
From one of the owners of the Jupyter client repo:
ZMQ subscriber sockets need a subscription set before they'll receive
any messages. The subscription is a prefix of a valid message, and you
can set it to an empty bytes string to subscribe to all messages.
e.g. in my case I need to add
socket.setsockopt(zmq.SUBSCRIBE, b'')
before starting the while loop.
Do you know if it's possible to capture from IOPub if a process in Jupyter notebook is finished or not?
I'm looking here (http://jupyterlab.github.io/jupyterlab/services/modules/kernelmessage.html) but it is not very clear.
Related
I am using pexpect to connect to a remote server using ssh.
The following code works but I have to use time.sleep to make a delay.
Especially when I am sending a command to run a script on the remote server.
The script will take up to a minute to run and if I don't use a 60 seconds delay, then the script will end prematurely.
The same issue when I am using sftp to download a file. If the file is large, then it download partially.
Is there a way to control without using a delay?
#!/usr/bin/python3
import pexpect
import time
from subprocess import call
siteip = "131.235.111.111"
ssh_new_conn = 'Are you sure you want to continue connecting'
password = 'xxxxx'
child = pexpect.spawn('ssh admin#' + siteip)
time.sleep(1)
child.expect('admin#.* password:')
child.sendline('xxxxx')
time.sleep(2)
child.expect('admin#.*')
print('ssh to abcd - takes 60 seconds')
child.sendline('backuplog\r')
time.sleep(50)
child.sendline('pwd')
Many pexpect functions take an optional timeout= keyword, and the one you give in spawn() sets the default. Eg
child.expect('admin#',timeout=70)
You can use the value None to never timeout.
Using python3.6.8:
I'm attempting to script the initial configuration of network devices on boot. My script opens telnet connection on "ip_addr:port". Once connected, script stimulates the network device it's connecting to with "\n\n" (simulating two 'Enter' input from an admin).
connection = asyncio.open_connection(
ip_addr,
port,
)
try:
reader, writer = await asyncio.wait_for(connection, 5)
print(f"successfully connected to {ip_addr}:{port}")
writer.write(b'\n\n\n')
Some devices are already configured and I except " login: " to show up in the read buffer upon entering '\n'. However if the device is not configured yet, " login: " will not show up in the buffer. Therefore I thought I could use wait_for and timeout option to have this cancelled and move on with another reader.readuntil(...) expecting another output.
try:
await asyncio.wait_for(reader.readuntil(b' login: '), 3)
print(f"{ip_addr}:{port} alredy booted")
break
except (asyncio.TimeoutError, OSError):
print('Nope, moving forward')
await reader.readuntil(b'normal setup ?(yes/no)[n]: ')
However this raises a RuntimeError. Reading the documentation I excepted the task to be cancelled if the timeout is reached, so why can't it have another coroutine readuntil() ?
Using a Raspi/Debian - I have a script that parses the results from an iwlist scan and sends them via UDP to a Pure Data patch. This runs fine in gui mode, but now I'm trying to automate the whole process in another script with the following:
pd-extended -nogui /home/pi/patch.pd & /home/pi/libOSC/scan.sh && fg
But when I run this new script, the UDP appears to only send the info to Pure Data once, and then the scanning continues but Pd does not receive the packet. Any help with this would be appreciated.
What happens when you run /home/pi/libOSC/scan.sh? It sends the results only once? Then maybe you need to do it differently, like calling that script from within pd using the 'shell' or 'popen' objects for instance. Or you implement a polling command via UDP that will return the values.
how does your scan.sh script look like?
you probably want to make it something like:
pdhost=localhost
pdport=9999
do_scan() {
## some code here that does the scan and print's the result to stdout
}
doscan | while read line
do
echo "${line};" | pdsend ${pdhost} ${pdport}
done
rather than the following:
doscan | pdsend ${pdhost} ${pdport}
I am writing a piece of software which can fool Nmap into believing that GuildFTPd FTP Server is running on port 21. My python code so far looks like this:
import socket
s = socket.socket( )
s.bind(('', 21))
s.listen(1)
conn, addr = s.accept()
conn.send("220-GuildFTPd FTP Server (c) 1997-2002\r\n220-Version 0.999.14")
conn.close()
The Nmap regexp for matching this particular service is:
match ftp m|^220-GuildFTPd FTP Server \(c\) \d\d\d\d(-\d\d\d\d)?\r\n220-Version (\d[-.\w]+)\r\n| p/Guild ftpd/ v/$2/ o/Windows/
However when I scan the host which is running the script with Nmap the results are:
21/tcp open ftp?
How can this be? When I scan the real service with Nmap it identifies the service correctly.
First, you're missing a \r\n at the end of your fake response that the match line needs.
The other major issue is that your program only handles one connection, then closes. Nmap will first do a port scan, then send service fingerprinting probes. If you ran nmap as root (or Administrator on Windows), it would use a half-open TCP SYN scan, and your application wouldn't see the portscan as a connection, but otherwise it will accept the portscan, close the connection, and be unavailable for the service scan phase.
Here's a very basic adaptation of your script that can handle sequential connections (but not parallel), which is enough to fool Nmap:
import socket
s = socket.socket( )
s.bind(('', 21))
s.listen(1)
while True:
conn, addr = s.accept()
conn.send("220-GuildFTPd FTP Server (c) 1997-2002\r\n220-Version 0.999.14\r\n")
conn.close()
I am trying to send an email from R, using the sendmailR package. The code below works fine when I run it on my PC, and I recieve the email. However, when I run it with my macbook pro, it fails with the following error:
library(sendmailR)
from <- sprintf("<sendmailR#%s>", Sys.info()[4])
to <- "<myemail#gmail.com>"
subject <- "TEST"
sendmail(from, to, subject, body,
control=list(smtpServer="ASPMX.L.GOOGLE.COM"))
Error in socketConnection(host = server, port = port, blocking = TRUE) :
cannot open the connection
In addition: Warning message:
In socketConnection(host = server, port = port, blocking = TRUE) :
ASPMX.L.GOOGLE.COM:25 cannot be opened
Any ideas as to why this would work on a PC, but not a mac? I turned the firewall off on both machines.
Are you able to send email via the command-line?
So, first of all, fire up a Terminal and then
$ echo “Test 123” | mail -s “Test” user#domain.com
Look into /var/log/mail.log, or better use
$ tail -f /var/log/mail.log
in a different window while you send your email. If you see something like
... setting up TLS connection to smtp.gmail.com[xxx.xx.xxx.xxx]:587
... Trusted TLS connection established to smtp.gmail.com[xxx.xx.xxx.xxx]:587:\
TLSv1 with cipher RC4-MD5 (128/128 bits)
then you succeeded. Otherwise, it means you have to configure you mailing system. I use postfix with Gmail for two years now, and I never had have problem with it. Basically, you need to grab the Equifax certificates, Equifax_Secure_CA.pem from here: http://www.geotrust.com/resources/root-certificates/. (They were using Thawtee certificates before but they changed last year.) Then, assuming you used Gmail,
Create relay_password in /etc/postfix and put a single line like this (with your correct login and password):
smtp.gmail.com login#gmail.com:password
then in a Terminal,
$ sudo postmap /etc/postfix/relay_password
to update Postfix lookup table.
Add the certificates in /etc/postfix/certs, or any folder you like, then
$ sudo c_rehash /etc/postfix/certs/
(i.e., rehash the certificates with Openssl).
Edit /etc/postfix/main.cf so that it includes the following lines (adjust the paths if needed):
relayhost = smtp.gmail.com:587
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/relay_password
smtp_sasl_security_options = noanonymous
smtp_tls_security_level = may
smtp_tls_CApath = /etc/postfix/certs
smtp_tls_session_cache_database = btree:/etc/postfix/smtp_scache
smtp_tls_session_cache_timeout = 3600s
smtp_tls_loglevel = 1
tls_random_source = dev:/dev/urandom
Finally, just reload the Postfix process, with e.g.
$ sudo postfix reload
(a combination of start/stop works too).
You can choose a different port for the SMTP, e.g. 465.
It’s still possible to use SASL without TLS (the above steps are basically the same), but in both case the main problem is that your login informations are available in a plan text file... Also, should you want to use your MobileMe account, just replace the Gmail SMTP server with smtp.me.com.