HTTPS request restarts NodeMCU - https

NodeMCU Info:
> NodeMCU custom build by frightanic.com
> branch: dev
> SSL: true
> modules: crypto,file,gpio,http,net,node,sjson,tmr,uart,wifi,tls
> powered by Lua 5.1.4 on SDK 2.2.1
> commit: d5aca39f485bf46e6d77b7fb05eaf71b8ea51bee
> Build type: integer -- tried float, same error
> LFS : disabled
Build bin file
When I requested HTTPS request before adding the tls module it returned code -1 after I added tls module requesting HTTPS restarts the NodeMCU.
The code I used to send request.
print(node.heap()) -- 31824
http.request("https://www.google.com", "HEAD", "", "",
function(code, data)
if (code < 0) then
print("HTTP request failed")
else
print(code, data)
end
end)
and I tried the net module.
print(node.heap()) -- 31824
srv = tls.createConnection()
srv:on("receive", function(sck, c) print(c) end)
srv:on("connection", function(sck, c)
sck:send("GET / HTTP/1.1\r\nHost: google.com\r\nConnection: keep-alive\r\nAccept: */*\r\n\r\n")
end)
srv:connect(443,"google.com")
I got the same restart reason for both codes.
HTTP client: hostname=google.com
HTTP client: port=443
HTTP client: method=GET
HTTP client: path=/
HTTP client: DNS request
HTTP client: DNS pending
> HTTP client: DNS found google.com 172.217.18.46
client handshake start.
Fatal exception 0(IllegalInstructionCause):
epc1=0x4027ca20, epc2=0x00000000, epc3=0x00000000, excvaddr=0x402993e7, depc=0x00000000
þ
ets Jan 8 2013,rst cause:2, boot mode:(3,x) -- sometimes x is 0, 6 or 7
load 0x40100000, len 26104, room 16
tail 8
chksum 0xc0
load 0x3ffe8000, len 2332, room 0
tail 12
chksum 0x3c
ho 0 tail 12 room 4
load 0x3ffe891c, len 136, room 12
tail 12
chksum 0x56
csum 0x56
in the net moudle code the restart happens when srv:connect(443,"google.com") is called and shows one additional line RecvHold, By pespconn,find conn_msg fail

Assuming the date in your error is because of something you executed recently, I suggest that you set the time to a more recent one. If your time has drifted too much your OS cannot operate normally. Set time manuelly if NTP refuse your connection. How to set RTC rtctime module, but also look at sntp module or other relevant ones.

I overcome this problem by building the same SDK Firmware 2.2.1 with Docker with the same modules and same configuration, can't control cases and lines to debug in the Cloud Build version so cant know what exactly went wrong in the Cloud Build.
The Docker Firmware info
NodeMCU 2.2.0.0 built with Docker provided by frightanic.com
branch: master
commit: 4095c408e6a8cc9718cb06007b408d0aad15d9cd
SSL: true
Build type: integer
LFS: disabled
modules: crypto,file,gpio,http,net,node,sjson,tls,tmr,uart,wifi
build created on 2018-11-29 17:45
powered by Lua 5.1.4 on SDK 2.2.1(6ab97e9)

Related

Websocket connection ssl problem on startup (Windows)

We have a program called server.exe which starts a websocket server (ws, wss) on computer of client.
It`s main purpose accept connections from browser (127.0.0.1) and send some data to it. It uses openssl dlls (1.0.2.20).
Problem: After startup of Windows server.exe does not work. It does not accept secure connections.
Debug Log with errors:
10.12.2019_16:11:09:0861 <<< ID = 728, msg: SSL library error during handshake on fd = 728 error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher
10.12.2019_16:11:09:0876 <<< ID = 592, msg: SSL library error during handshake on fd = 592 error:1408A10B:SSL routines:ssl3_get_client_hello:wrong version number
10.12.2019_16:11:09:0876 <<< ID = 776, msg: SSL library error during handshake on fd = 776 error:1408A10B:SSL routines:ssl3_get_client_hello:wrong version number
But!! If we just restart server.exe - everything begin to work fine!
if we launch server.exe with .bat file (5 sec dealy) - everything is working!
Why? How can we solve problem?
Fixed.
server.exe file cannot find path to the dll.

SSLHandshakeException when trying to access ES instance from Docker

I am trying to access 6.x ES instance using High Level REST Client 6.7.2.
Access to this ES instance is provided to me via hostname (https://****.azureedge.net), username & password.
My Spring Boot application is getting data from the same ES without issues when it runs from my dev environment (IDE), but throws SSLHandshakeException as soon as I try run it from Docker container (from my development machine or K8s cluster in cloud).
Container is made with base image: FROM debian:stretch-slim & OpenJDK 11.0.2 with Spring Boot necessary modules.
I made some progress debugging with -Djavax.net.debug=all. It turns out that while running in docker image just few first steps of usual SSL handshaking happen:
Produced ClientHello handshake message
WRITE: TLS13 handshake, length = 2352
Raw write
Raw read (0000: 15 03 03 00 02 02 28 ......( )
READ: TLSv1.2 alert, length = 2
Received alert message (
"Alert": {
"level" : "fatal",
"description": "handshake_failure"
}
)
followed by SSLHandshakeException:
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:938)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:227)
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1764)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1749)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1708)
at org.elasticsearch.client.SecurityClient.getSslCertificates(SecurityClient.java:508)
....
Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
at java.base/sun.security.ssl.Alert.createSSLException(Unknown Source)
at java.base/sun.security.ssl.Alert.createSSLException(Unknown Source)
at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source)
at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Unknown Source)
at java.base/sun.security.ssl.TransportContext.dispatch(Unknown Source)
at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source)
at java.base/sun.security.ssl.SSLEngineImpl.decode(Unknown Source)
at java.base/sun.security.ssl.SSLEngineImpl.readRecord(Unknown Source)
at java.base/sun.security.ssl.SSLEngineImpl.unwrap(Unknown Source)
at java.base/sun.security.ssl.SSLEngineImpl.unwrap(Unknown Source)
at java.base/javax.net.ssl.SSLEngine.unwrap(Unknown Source)
at org.apache.http.nio.reactor.ssl.SSLIOSession.doUnwrap(SSLIOSession.java:271)
at org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:316)
at org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:509)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:120)
at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
When running from my local environment handshake looks uninterrupted:
Produced ClientHello handshake message
WRITE: TLS13 handshake, length = 460
Raw write
Raw read
READ: TLSv1.2 handshake, length = 155
Consuming ServerHello
ServerHello
Negotiated protocol version: TLSv1.3
Session initialized: Session(1560119025211|TLS_AES_256_GCM_SHA384)
WRITE: TLS13 change_cipher_spec, length = 1
Raw write
Raw read
READ: TLSv1.2 change_cipher_spec, length = 1
Consuming ChangeCipherSpec message
Raw read
READ: TLSv1.2 application_data, length = 27
...
Raw read
READ: TLSv1.2 application_data, length = 8469
Consuming server Certificate handshake message
... // here is the list of 3 certificates with "SHA256withRSA", "SHA256withRSA", "SHA1withRSA" signature algorithms
Found trusted certificate ⇢ SHA1withRSA
...
While running locally I noticed CN=Microsoft IT TLS CA 2, OU=Microsoft IT, O=Microsoft Corporation, L=Redmond, ST=Washington, C=US, as well as CN=Baltimore CyberTrust Root, OU=CyberTrust, O=Baltimore, C=IE as issuers, maybe this is important, but I guess it is expected considering ES host address (Azure).
At the end I wanted to emphasise that I didn't need to do anything special to make this work in my macOS Java 11.0.2 development environment.
I already tried following, but that didn't change anything:
Changing base Docker image from "slim" to non slim version
Using OpenJDK 11.0.1 or 11.0.2
Added cert from host to TrustStore that JVM is using in a runtime. (I checked in Docker container that there is indeed one more cert, but considering when handshake failure happens I imagine this is irrelevant)
tried to enforce App with: "-Dcom.sun.net.ssl.enableECC=false", "-Djdk.tls.client.protocols=TLSv1.3", "-Dhttps.protocols=TLSv1.3", didn't help
Interesting: curl from Docker image with BasicAuth "talks" with same URL without issues (handshake completes) & small query returns results. I guess that curl and JVM are using different sources of trusted CAs inside the Docker, different algorithms for handshaking etc..
Thanks in advance for any help
TLDR: enforcing TLSv1.2 for client in the app made handshaking complete from inside the docker
After a lot of try/fail attempts I made it work. Following things didn't make any difference:
using non "slim" debian base image insead of "slim"
using OpenJDK 11.0.2 instead of 11.0.1
adding host's certificate to JVM TrustedStore while building docker image so it is available when container starts.
enforcing com.sun.net.ssl.enableECC=false
enforcing TLSv1.3 for https.protocols and/or jdk.tls.client.protocols
enforcing TLSv1.2 for https.protocols
What fixed handshake with host, was enforcing TLSv1.2 for client by using -Djdk.tls.client.protocols=TLSv1.2 in Dockerfile, so app runs with this flag inside container. This allowed SSL handshake to complete as it should work anyways. For some reason actual negotiation about protocol version didn't work without enforcing lower version of protocol for client. Logs from local vs docker environment doesn't show any difference but this helped in docker.
What helped me to find out was:
setting of javax.net.debug=ssl:handshake or even more detailed javax.net.debug=all so I could see details of handshake attempts
confirming that "at least someone" can establish outbound communication from inside the docker by using curl to send same request as app is trying, which worked because curl somehow figured out how to proceed handshaking with host.
pure luck
Thanks everyone for support & ideas

Application using MQTT protocol from azure sdk, doesn´t work behind a corporative proxy

I´m newbie in this matter, and I don´t know why my application just work and runs in a open network, when is behind a proxy I have a return error.
I´m using a raspberry zero, with raspbian Stretch, using azure-iot-sdk-python and proxy squid
I already try this things:
The proxy allow HTTPS connection, and it has all PORT are available and without any restriction and the address *****. azure-devices.net is put inside a whitelist in
$ nano / etc / squid / whitelist
Beyond that I set the proxy in the operate system, raspbian Stretch, in the
$ nano / etc / environment
the follow configurations:
export http_proxy = "http://192.168.2.254:3128/"
export https_proxy = "https://192.168.2.254:3128/"
export no_proxy = "localhost, 127.0.0.1"
And also in
$ nano ~ / .bashrc
export http_proxy = http: //192.168.2.254:3128
export https_proxy = https: //192.168.2.254:3128
export no_proxy = localhost, 127.0.0.1
And,
$ nano /etc/apt/apt.conf.d/90proxy
Acquire :: http :: Proxy "http://192.168.2.254:3128/";
Acquire :: https :: Proxy "https://192.168.2.254:3128/";
from iothub_client import IoTHubClient, IoTHubTransportProvider, IoTHubMessage
import time
CONNECTION_STRING = "HostName=******.azure-devices.net;DeviceId=***;SharedAccessKey=*********"
PROTOCOL = IoTHubTransportProvider.MQTT
def send_confirmation_callback(message, result, user_context):
print("Confirmation received for message with result = %s" % (result))
if __name__ == '__main__':
client = IoTHubClient(CONNECTION_STRING, PROTOCOL)
message = IoTHubMessage("test message")
client.send_event_async(message, send_confirmation_callback, None)
print("Message transmitted to IoT Hub")
while True:
time.sleep(1)
Error: File: /usr/sdk/src/c/c-utility/adapters/socketio_berkeley.c Func: lookup_address_and_initiate_socket_connection Line: 282 Failure: getaddrinfo failure -3.
Error: File: /usr/sdk/src/c/c-utility/adapters/socketio_berkeley.c Func: socketio_open Line: 765 lookup_address_and_connect_socket failed
Error: File: /usr/sdk/src/c/c-utility/adapters/tlsio_openssl.c Func: on_underlying_io_open_complete Line: 760 Invalid tlsio_state. Expected state is TLSIO_STATE_OPENING_UNDERLYING_IO.
Error: File: /usr/sdk/src/c/c-utility/adapters/tlsio_openssl.c Func: tlsio_openssl_open Line: 1258 Failed opening the underlying I / O.
Error: File: /usr/sdk/src/c/umqtt/src/mqtt_client.c Func: mqtt_client_connect Line: 1000 Error: io_open failed
Error: File: /usr/sdk/src/c/iothub_client/src/iothubtransport_mqtt_common.c Func: SendMqttConnectMsg Line: 2122 failure connecting
You can not use a HTTP proxy with (native) MQTT, they are 2 totally separate protocols.
If you can use MQTT over WebSockets then you should be able to use a HTTP proxy as WebSockets are initially established by upgrading a HTTP connection.
If you have a SOCKS proxy available on your network, then you may be able to use that with native MQTT. The following question has hints on how to use a SOCKS proxy with Python. How can I use a SOCKS 4/5 proxy with urllib2?

Unresponsive socket after x time (puma - ruby)

I'm experiencing an unresponsive socket in with my Puma setup after random time. Up to this point I don't have a clue what's causing the issue. I was hoping somebody over here can help we with some answers or point me in the right direction. I'm having the following setup:
I'm using the official docker ruby-2.2.3-slim image together with the latest puma release 2.15.3, I've also installed Nginx as a reverse proxy. But I'm already sure Nginx isn't the problem over here because and I've tried to verify if the socket was working using this script. And the socket wasn't working, I got a timeout over there as well so I could ignore Nginx.
This is a testing environment so the server isn't experiencing any extreme load, I've also check memory consumption it has still several GB's of free space so that couldn't be the issue either.
What triggered me to look at the puma socket was the error message I got in my Nginx error logging:
upstream timed out (110: Connection timed out) while reading response header from upstream
Also I couldn't find anything in the logs of puma indicating what is going wrong, over here are my puma setup:
threads 0, 16
app_dir = ENV.fetch('APP_HOME')
environment ENV['RAILS_ENV']
daemonize
bind "unix://#{app_dir}/sockets/puma.sock"
stdout_redirect "#{app_dir}/log/puma.stdout.log", "#{app_dir}/log/puma.stderr.log", true
pidfile "#{app_dir}/pids/puma.pid"
state_path "#{app_dir}/pids/puma.state"
activate_control_app
on_worker_boot do
require 'active_record'
ActiveRecord::Base.connection.disconnect! rescue ActiveRecord::ConnectionNotEstablished
ActiveRecord::Base.establish_connection(YAML.load_file("#{app_dir}/config/database.yml")[ENV['RAILS_ENV']])
end
And this it the output in my puma state file:
---
pid: 43
config: !ruby/object:Puma::Configuration
cli_options:
conf:
options:
:min_threads: 0
:max_threads: 16
:quiet: false
:debug: false
:binds:
- unix:///APP/sockets/puma.sock
:workers: 1
:daemon: true
:mode: :http
:before_fork: []
:worker_timeout: 60
:worker_boot_timeout: 60
:worker_shutdown_timeout: 30
:environment: staging
:redirect_stdout: "/APP/log/puma.stdout.log"
:redirect_stderr: "/APP/log/puma.stderr.log"
:redirect_append: true
:pidfile: "/APP/pids/puma.pid"
:state: "/APP/pids/puma.state"
:control_url: unix:///tmp/puma-status-1449260516541-37
:config_file: config/puma.rb
:control_url_temp: "/tmp/puma-status-1449260516541-37"
:control_auth_token: cda8879717be7a645ea323d931b88d4b
:tag: APP
The application itself is a Rails app on the latest version 4.2.5, it's deployed on GCE (Google Container Engine).
If somebody could give me some pointer's on how to debug this any further would be very much appreciated. Because now I don't see any output anywhere which could help me any further.
EDIT
I replaced the unix socket with tcp connection to Puma with the same result, still hangs after x time
I'd start with:
How many requests get processed successfully per instance of puma?
Make sure you log the beginning and end of each request with the thread id of the thread executing it, what do you see?
Not knowing more about your application, I'd say it's likely the threads get stuck doing some long/blocking calls without timeouts or spinning on some computation until the whole thread pool gets depleted.
We'll see.
I finally found out why my application was behaving the way it was.
After trying to use a tcp connection and switching to Unicorn I start looking into other possible sources.
That's when I thought maybe my connection to Google Cloud SQL could be the problem. Once I read the faq of Cloud SQL, they mentioned that you have to tweak you Compute instances to ensure they keep open your DB connection. So I performed the next steps they recommend and that solved the problem for me, I added them just in case:
# Display the current tcp_keepalive_time value.
$ cat /proc/sys/net/ipv4/tcp_keepalive_time
# Set tcp_keepalive_time to 60 seconds and make it permanent across reboots.
$ echo 'net.ipv4.tcp_keepalive_time = 60' | sudo tee -a /etc/sysctl.conf
# Apply the change.
$ sudo /sbin/sysctl --load=/etc/sysctl.conf
# Display the tcp_keepalive_time value to verify the change was applied.
$ cat /proc/sys/net/ipv4/tcp_keepalive_time

Challenge response from CFEngine Server Failure while conencting cygwin to CFEngine

when i run from windows-7 with cygwin to connect CFEngine bersion 3.4.2
cf-agent -Bs 217.64.173.210
Challenge response from server 217.64.173.210/217.64.173.210 was incorrect!
I: Made in version 'not specified' of '/var/cfengine/inputs/update.cf' near line 47
!! Authentication dialogue with 217.64.173.210 failed
Challenge response from server 217.64.173.210/217.64.173.210 was incorrect!
I: Made in version 'not specified' of '/var/cfengine/inputs/update.cf' near line
and in /var/cfengine/inputs/update.cf on line 47 is
47 : perms => m("600"),
on cgwin in folder keys
/var/cfengine/ppkeys
localhost.pub
localhost.priv
root-MD5=b8825ba0a0e7017e34b15766d3b3ac58 (which is also at CFEngine Server Side shared ky)
on Cf-Engine Server Side
/var/cfengine/ppkeys/
localhost.priv
localhost.pub
root-MD5=b8825ba0a0e7017e34b15766d3b3ac58
With Regards
Sandeep
Did you also get the server to trust the client's key? like so:
cf-key -t root-MD5=b8825ba0a0e7017e34b15766d3b3ac58
(on the server)
Also, try restarting cf-serverd in verbose mode with the -v switch on the server, and watch what error messages you get on that end.

Resources