i'm using ESP32-S2 with sd card SPI.
I'm usind SD.h library
Wifi connection
and i need to dowload a BMP form url http://example.com/test.bmp ( or in future generic binary file ) and store in SD Card.
I'm able to connect but not to save file streaming coming from url
void downloadImage(){
// Connect to external web server
Serial.println("Checking if image available");
String imageName="bottone.bmp";
String fwVersionURL = FWURL + imageName;
client.begin(fwVersionURL);
// Get file, just to check if each reachable
int resp = client.GET();
Serial.print("Response: ");
Serial.println(resp);
// If file is reachable, start downloading
if(resp == 200){
// get length of document (is -1 when Server sends no Content-Length header)
totImageLength = client.getSize();
// transfer to local variable
int len = totImageLength;
Serial.printf("FW Size: %u\n",totImageLength);
// create buffer for read
uint8_t buff[totImageLength] = { 0 };
// get tcp stream
WiFiClient * stream = client.getStreamPtr();
// read all data from server
Serial.println("Updating firmware...");
while(client.connected() && (len > 0 || len == -1)) {
// get available data size
size_t size = stream->available();
if(size) {
// read up to 128 byte
int c = stream->readBytes(buff, ((size > sizeof(buff)) ? sizeof(buff) : size));
// pass to function
}
delay(1);
}
File remoteImg = SD.open("/"+imageName, FILE_WRITE);
remoteImg.write(buff, totImageLength);
remoteImg.close();
}else{
Serial.println("Cannot download file.");
}
client.end();
}
Related
I am trying to download my hex file of size 1500KB via UDS with CAPL test module,
p2 timer = 50ms
p2* timer = 5000ms
Here is snippet of my code for data transfer :
void TS_transferData()
{
byte transferData_serviceid = 0x36;
byte blockSequenceCounter = 0x1;
byte buffer[4093];
byte binarydata[4095];
long i,ret1,ret2,ret3,temp,timeout = 0,Counter = 0;
char filename[30] = "xxx.bin";
dword readaccess_handle;
diagrequest ECU_QUALIFIER.* request;
long valueleft;
readaccess_handle = OpenFileRead(filename,1);
if (readaccess_handle != 0 )
{
while( (valueleft = fileGetBinaryBlock(buffer,elcount(buffer),readaccess_handle))==4093 )
{
binarydata[0] = transferData_serviceid;
binarydata[1] = blockSequenceCounter;
for(i=0;i<elcount(buffer);i++)
{
binarydata[i+2] = buffer[i];
}
diagResize(request, elCount(binarydata));
DiagSetPrimitiveData(request,binarydata,elcount(binarydata));
DiagSendRequest(request);
write("length of binarydata %d ",elcount(binarydata));
// Wait until the request has been completely sent
ret1 = TestWaitForDiagRequestSent(request, 20000);
if(ret1 == 1) // Request sent
{
ret2=TestWaitForDiagResponse(request,50);
if(ret2==1) // Response received
{
ret3=DiagGetLastResponseCode(request); // Get the code of the response
if(ret3==-1) // Is it a positive response?
{
;
}
else
{
testStepFail(0, "4.0","Binary Datatransfer on server Failed");
break;
}
}
else if(ret2 == timeout)
{
testStepFail(0, "4.0","Binary Datatransfer on server Failed");
write("timeout occured while TestWaitForDiagResponse with block %d ",blockSequenceCounter);
}
}
else if(ret1 == timeout)
{
testStepFail(0, "4.0","Binary Datatransfer on server Failed");
write("timeout occured while TestWaitForDiagRequestSent %d ",blockSequenceCounter);
}
if(blockSequenceCounter == 255)
blockSequenceCounter = 0;
else
++blockSequenceCounter;
}
}
//handle the rest of the bytes to be transmitted
fileClose (readaccess_handle);
}
The software downloading is happening but it is taking a long.... time for download.
For TestWaitForDiagRequestSent() function any value for timeout less than 20000 is giving me timeout error.
Is there any other way I can reduce the software transfer time or where am I going wrong with calculation?
Is there any example I can refer to see How to transmit such a long data using CAPL ?
Sorry, I am a beginner to CAPL and UDS protocol.
I'm trying to post a (large) json file that's stored in the SPIFFS (LittleFS- of an arduino.
I first of all, fetch the JSON from a local IP address, which can change or is dynamic. Then, that json file gets stored in the internal flash of my ESP32 and sent over to our servers.
The goal is to have an ESP32 act as a bridge for local addresses.
For example:
Locally, there is a server running, that server is never accessible from outside of a network, but it is from within. We place an ESP32 in that network, to be able to read out data from the internal device and pass them through to our servers, for further processing.
The current code I have is the following;
HTTPClient http;
if (LittleFS.exists("/downloadedFile.json")) {
LittleFS.remove("/downloadedFile.json");
}
File f = LittleFS.open("/downloadedFile.json", "w", true);
if (f) {
String url;
// Build the URL...
if (username != "" && password != "") {
url = "http://" + username + ":" + password + "#" + ip + ":" + String(port) + "/data/file.json";
} else {
url = "http://" + ip + ":" + String(port) + "/data/file.json";
}
http.begin(url);
int httpCode = http.GET();
if (httpCode > 0) {
if (httpCode == HTTP_CODE_OK) {
http.writeToStream(&f);
}
} else {
// failed.
}
f.close();
} else {
// Unable to open file
}
http.end();
if (LittleFS.exists("/downloadedFile.json")) {
File f = LittleFS.open("/downloadedFile.json", "r");
if (f) {
WiFiClienSecure client;
client.setInsecure(); // test
if (!client.connect("my.somain.com", 443)) {
// failed to connect. (never gets in here)
return;
}
client.println("POST /path/to/my/endpoint HTTP/1.1");
client.println("Host: my.somain.com");
client.println("Content-Type: application/json");
client.println("Connection: keep-alive");
client.println("Content-Length: " + String(f.size()));
client.println();
// Build the body
int len = 0;
int total = 0;
uint8_t buf[1025] = {};
size_t _len = 256;
do {
len = f.read(buf, _len);
client.write(buf, len);
total += len;
} while (len > 0);
client.println(); // end line
client.stop();
f.close();
}
}
Downloading the file always works, except for sometimes random crashing the whole system. But that's the least of my concerns.
Locally I got it working to download and re-upload the file. Struggle is when I try to achieve the same result on the server (which uses https)
Only difference is this;
Server
Local
WiFiClienSecure client;
WiFiClient client;
client.setInsecure();
client.connect("my.somain.com", 443)
client.connect("IP", 80)
The main problem is that it doesn't work on the server and that it works really unstable. the file size (when I do f.size()) is 310831.
Any help and thoughts on getting this stable is appriciated!
When I tried using Ocelot as a WebSocket proxy, I could not get it working for wss. I was able to see it working for ws.
When we are trying to proxy for wss getting decrypt operation failed while reading the bytes at the server side socket. With plan ws I am able to get this working.
Ocelot config as follows, where wss proxying is specified:
{
"Routes": [
{
"DownstreamPathTemplate": "/ws",
"UpstreamPathTemplate": "/{anything}",
"DownstreamScheme": "wss",
"DownstreamHostAndPorts": [
{
"Host": "127.0.0.1",
"Port": 8080
}
]
}
]
}
Websocket Server code which listens on port 8080:
using System;
using System.IO;
using System.Net;
using System.Net.Security;
using System.Net.Sockets;
using System.Security.Authentication;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using System.Text.RegularExpressions;
class Server
{
public static void Main()
{
string ip = "127.0.0.1";
int port = 8080;
var server = new TcpListener(IPAddress.Parse(ip), port);
server.Start();
Console.WriteLine("Server has started on {0}:{1}, Waiting for a connection...", ip, port);
TcpClient client = server.AcceptTcpClient();
Console.WriteLine("A client connected.");
byte[] pfxData = File.ReadAllBytes(#"C:\Users\e409316\Desktop\test.pfx");
var cert = new X509Certificate2(pfxData, "Password1", X509KeyStorageFlags.UserKeySet | X509KeyStorageFlags.Exportable);
Stream sourceTcpStream = new SslStream(client.GetStream(), false);
(sourceTcpStream as SslStream).AuthenticateAsServer(
cert,
false,
SslProtocols.Tls12, true);
Stream stream = sourceTcpStream;//client.GetStream();
//Stream stream = client.GetStream();
// enter to an infinite cycle to be able to handle every change in stream
while (true)
{
//while (!stream.DataAvailable) ;
while (client.Available < 3) ; // match against "get"
byte[] bytes = new byte[client.Available];
stream.Read(bytes, 0, client.Available);
string s = Encoding.UTF8.GetString(bytes);
if (Regex.IsMatch(s, "^GET", RegexOptions.IgnoreCase))
{
Console.WriteLine("=====Handshaking from client=====\n{0}", s);
// 1. Obtain the value of the "Sec-WebSocket-Key" request header without any leading or trailing whitespace
// 2. Concatenate it with "258EAFA5-E914-47DA-95CA-C5AB0DC85B11" (a special GUID specified by RFC 6455)
// 3. Compute SHA-1 and Base64 hash of the new value
// 4. Write the hash back as the value of "Sec-WebSocket-Accept" response header in an HTTP response
string swk = Regex.Match(s, "Sec-WebSocket-Key: (.*)").Groups[1].Value.Trim();
string swka = swk + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11";
byte[] swkaSha1 = System.Security.Cryptography.SHA1.Create().ComputeHash(Encoding.UTF8.GetBytes(swka));
string swkaSha1Base64 = Convert.ToBase64String(swkaSha1);
// HTTP/1.1 defines the sequence CR LF as the end-of-line marker
byte[] response = Encoding.UTF8.GetBytes(
"HTTP/1.1 101 Switching Protocols\r\n" +
"Connection: Upgrade\r\n" +
"Upgrade: websocket\r\n" +
"Sec-WebSocket-Accept: " + swkaSha1Base64 + "\r\n\r\n");
stream.Write(response, 0, response.Length);
}
else
{
bool fin = (bytes[0] & 0b10000000) != 0,
mask = (bytes[1] & 0b10000000) != 0; // must be true, "All messages from the client to the server have this bit set"
int opcode = bytes[0] & 0b00001111, // expecting 1 - text message
msglen = bytes[1] - 128, // & 0111 1111
offset = 2;
if (msglen == 126)
{
// was ToUInt16(bytes, offset) but the result is incorrect
msglen = BitConverter.ToUInt16(new byte[] { bytes[3], bytes[2] }, 0);
offset = 4;
}
else if (msglen == 127)
{
Console.WriteLine("TODO: msglen == 127, needs qword to store msglen");
// i don't really know the byte order, please edit this
// msglen = BitConverter.ToUInt64(new byte[] { bytes[5], bytes[4], bytes[3], bytes[2], bytes[9], bytes[8], bytes[7], bytes[6] }, 0);
// offset = 10;
}
if (msglen == 0)
Console.WriteLine("msglen == 0");
else if (mask)
{
byte[] decoded = new byte[msglen];
byte[] masks = new byte[4] { bytes[offset], bytes[offset + 1], bytes[offset + 2], bytes[offset + 3] };
offset += 4;
for (int i = 0; i < msglen; ++i)
decoded[i] = (byte)(bytes[offset + i] ^ masks[i % 4]);
string text = Encoding.UTF8.GetString(decoded);
Console.WriteLine("{0}", text);
}
else
Console.WriteLine("mask bit not set");
Console.WriteLine();
}
}
}
}
Websocket Client code which tries to connect to ocelot endpoint (upstream endpoint on port 5000):
ClientWebSocket client = new ClientWebSocket();
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
client.ConnectAsync(new Uri("wss://127.0.0.1:5000/"), CancellationToken.None).Wait();
var buffer = new byte[]{1,2,3};
client.SendAsync(new ArraySegment<byte>(buffer), WebSocketMessageType.Text, true,CancellationToken.None);
Error: The decryption operation failed
Found it to be a certificate validity issue. Solved it by using a trusted certificate.
Saving base64String as image on FTP Server is saving it as corrupted file.
I am doing following things
converted base64String into byte[].
Initialized MemoryStream with byte converted in above step.
Opened stream from FTP
Write stream on ftp.
Below is the code
public bool WriteFromBase64ToFile(string base64, string path, string fileName)
{
bool result = false;
using (FtpClient ftp = new FtpClient())
{
// setting ftp properties with required values.
ftp.ReadTimeout = 999999999;
ftp.Host = host;
ftp.Credentials = new System.Net.NetworkCredential(username, password);
ftp.Port = Convert.ToInt32(port);
ftp.DataConnectionType = FtpDataConnectionType.AutoPassive;
ftp.Connect();
ftp.ConnectTimeout = 1000000;
// converting base64String into byte array.
byte[] file = Convert.FromBase64String(base64);
if (ftp.IsConnected)
{
int BUFFER_SIZE = file.Length; // 64KB buffer
byte[] buffer = new byte[file.Length];
// Initializing MemoryStream with byte converted from base64String.
MemoryStream ms = new MemoryStream(buffer);
using (Stream readStream = ms)
{
fileName = fileName.ReplacingSpecialCharacterswithEntities();
// Getting stream from ftp and then writing it on FTP server.
using (Stream writeStream = ftp.OpenWrite(path + "/" + fileName+".jpg", FtpDataType.Binary))
{
while (readStream.Position < readStream.Length)
{
buffer.Initialize();
// Reading stream
int bytesRead = readStream.Read(buffer, 0, BUFFER_SIZE);
// Writing stream
writeStream.Write(buffer, 0, bytesRead);
}
// flushing stream.
writeStream.Flush();
}
}
}
}
result = true;
return result;
}
I tried to make an example of simple socket server.
Build and run successfully. However it doesn't work well.
Client couldn't connect to this server.
How to solve this problem? I need your help, thanks.
import Foundation
let BUFF_SIZE = 1024
func initStruct<S>() -> S {
let struct_pointer = UnsafePointer<S>.alloc(1)
let struct_memory = struct_pointer.memory
struct_pointer.destroy()
return struct_memory
}
func sockaddr_cast(p: ConstUnsafePointer<sockaddr_in>) -> UnsafePointer<sockaddr> {
return UnsafePointer<sockaddr>(p)
}
func socklen_t_cast(p: UnsafePointer<Int>) -> UnsafePointer<socklen_t> {
return UnsafePointer<socklen_t>(p)
}
var server_socket: Int32
var client_socket: Int32
var server_addr_size: Int
var client_addr_size: Int
var server_addr: sockaddr_in = initStruct()
var client_addr: sockaddr_in = initStruct()
var buff_rcv: Array<CChar> = []
var buff_snd: String
server_socket = socket(PF_INET, SOCK_STREAM, 0);
if server_socket == -1
{
println("[Fail] Create Server Socket")
exit(1)
}
else
{
println("[Success] Created Server Socket")
}
server_addr_size = sizeof(server_addr.dynamicType)
memset(&server_addr, 0, UInt(server_addr_size));
server_addr.sin_family = sa_family_t(AF_INET)
server_addr.sin_port = 4000
server_addr.sin_addr.s_addr = UInt32(0x00000000) // INADDR_ANY = (u_int32_t)0x00000000 ----- <netinet/in.h>
let bind_server = bind(server_socket, sockaddr_cast(&server_addr), socklen_t(server_addr_size))
if bind_server == -1
{
println("[Fail] Bind Port");
exit(1);
}
else
{
println("[Success] Binded Port");
}
if listen(server_socket, 5) == -1
{
println("[Fail] Listen");
exit(1);
}
else
{
println("[Success] Listening : \(server_addr.sin_port) Port ...");
}
var n = 0
while n < 1
{
client_addr_size = sizeof(client_addr.dynamicType)
client_socket = accept(server_socket, sockaddr_cast(&client_addr), socklen_t_cast(&client_addr_size))
if client_socket == -1
{
println("[Fail] Accept Client Connection");
exit(1);
}
else
{
println("[Success] Accepted Client : \(inet_ntoa(client_addr.sin_addr)) : \(client_addr.sin_port)");
}
read(client_socket, &buff_rcv, UInt(BUFF_SIZE))
println("[Success] Received : \(buff_rcv)")
buff_snd = "\(strlen(buff_rcv)) : \(buff_rcv)"
write(client_socket, &buff_snd, strlen(buff_snd) + 1)
close(client_socket)
}
The port number in the socket address must be in big-endian byte order:
server_addr.sin_port = UInt16(4000).bigEndian
So your program actually listens on port 40975 (hex 0xA00F) and not
on port 4000 (hex 0x0FA0).
Another problem is here:
var buff_rcv: Array<CChar> = []
// ...
read(client_socket, &buff_rcv, UInt(BUFF_SIZE))
Your buffer is an empty array, but recv() expects a buffer of size BUFF_SIZE.
The behaviour is undefined. To get a buffer of the required size, use
var buff_rcv = [CChar](count:BUFF_SIZE, repeatedValue:0)
// ...
read(client_socket, &buff_rcv, UInt(buff_rcv.count))
Remark: Here you cast the address of an Int to the address of an socklen_t
and pass that to the accept() function:
client_socket = accept(server_socket, sockaddr_cast(&client_addr), socklen_t_cast(&client_addr_size))
That is not safe. If Int and socklen_t have different sizes then the behaviour
will be undefined. You should declare server_addr_size and client_addr_size
as socklen_t and remove the socklen_t_cast() function:
client_socket = accept(server_socket, sockaddr_cast(&client_addr), &client_addr_size)
As Martin R commented before, the write command shouldn't be using the swift string as that. Something like this will work properly:
write(client_socket, buff_snd.cStringUsingEncoding(NSUTF8StringEncoding)!, countElements(buff_snd) + 1)