Transmitting 1500KB (hex files )data over UDS using CAPL test module - capl

I am trying to download my hex file of size 1500KB via UDS with CAPL test module,
p2 timer = 50ms
p2* timer = 5000ms
Here is snippet of my code for data transfer :
void TS_transferData()
{
byte transferData_serviceid = 0x36;
byte blockSequenceCounter = 0x1;
byte buffer[4093];
byte binarydata[4095];
long i,ret1,ret2,ret3,temp,timeout = 0,Counter = 0;
char filename[30] = "xxx.bin";
dword readaccess_handle;
diagrequest ECU_QUALIFIER.* request;
long valueleft;
readaccess_handle = OpenFileRead(filename,1);
if (readaccess_handle != 0 )
{
while( (valueleft = fileGetBinaryBlock(buffer,elcount(buffer),readaccess_handle))==4093 )
{
binarydata[0] = transferData_serviceid;
binarydata[1] = blockSequenceCounter;
for(i=0;i<elcount(buffer);i++)
{
binarydata[i+2] = buffer[i];
}
diagResize(request, elCount(binarydata));
DiagSetPrimitiveData(request,binarydata,elcount(binarydata));
DiagSendRequest(request);
write("length of binarydata %d ",elcount(binarydata));
// Wait until the request has been completely sent
ret1 = TestWaitForDiagRequestSent(request, 20000);
if(ret1 == 1) // Request sent
{
ret2=TestWaitForDiagResponse(request,50);
if(ret2==1) // Response received
{
ret3=DiagGetLastResponseCode(request); // Get the code of the response
if(ret3==-1) // Is it a positive response?
{
;
}
else
{
testStepFail(0, "4.0","Binary Datatransfer on server Failed");
break;
}
}
else if(ret2 == timeout)
{
testStepFail(0, "4.0","Binary Datatransfer on server Failed");
write("timeout occured while TestWaitForDiagResponse with block %d ",blockSequenceCounter);
}
}
else if(ret1 == timeout)
{
testStepFail(0, "4.0","Binary Datatransfer on server Failed");
write("timeout occured while TestWaitForDiagRequestSent %d ",blockSequenceCounter);
}
if(blockSequenceCounter == 255)
blockSequenceCounter = 0;
else
++blockSequenceCounter;
}
}
//handle the rest of the bytes to be transmitted
fileClose (readaccess_handle);
}
The software downloading is happening but it is taking a long.... time for download.
For TestWaitForDiagRequestSent() function any value for timeout less than 20000 is giving me timeout error.
Is there any other way I can reduce the software transfer time or where am I going wrong with calculation?
Is there any example I can refer to see How to transmit such a long data using CAPL ?
Sorry, I am a beginner to CAPL and UDS protocol.

Related

ESP32-S2 download BMP file from url

i'm using ESP32-S2 with sd card SPI.
I'm usind SD.h library
Wifi connection
and i need to dowload a BMP form url http://example.com/test.bmp ( or in future generic binary file ) and store in SD Card.
I'm able to connect but not to save file streaming coming from url
void downloadImage(){
// Connect to external web server
Serial.println("Checking if image available");
String imageName="bottone.bmp";
String fwVersionURL = FWURL + imageName;
client.begin(fwVersionURL);
// Get file, just to check if each reachable
int resp = client.GET();
Serial.print("Response: ");
Serial.println(resp);
// If file is reachable, start downloading
if(resp == 200){
// get length of document (is -1 when Server sends no Content-Length header)
totImageLength = client.getSize();
// transfer to local variable
int len = totImageLength;
Serial.printf("FW Size: %u\n",totImageLength);
// create buffer for read
uint8_t buff[totImageLength] = { 0 };
// get tcp stream
WiFiClient * stream = client.getStreamPtr();
// read all data from server
Serial.println("Updating firmware...");
while(client.connected() && (len > 0 || len == -1)) {
// get available data size
size_t size = stream->available();
if(size) {
// read up to 128 byte
int c = stream->readBytes(buff, ((size > sizeof(buff)) ? sizeof(buff) : size));
// pass to function
}
delay(1);
}
File remoteImg = SD.open("/"+imageName, FILE_WRITE);
remoteImg.write(buff, totImageLength);
remoteImg.close();
}else{
Serial.println("Cannot download file.");
}
client.end();
}

gRPC slow serialization on large dataset

I know that google states that protobufs don't support large messages (i.e. greater than 1 MB), but I'm trying to stream a dataset using gRPC that's tens of megabytes, and it seems like some people say it's ok, or at least with some splitting...
However, when I try to send an array this way (repeated uint32), it takes like 20 seconds on the same local machine.
#proto
service PAS {
// analyze single file
rpc getPhotonRecords (PhotonRecordsRequest) returns (PhotonRecordsReply) {}
}
message PhotonRecordsRequest {
string fileName = 1;
}
message PhotonRecordsReply {
repeated uint32 PhotonRecords = 1;
}
where PhotonRecordsReply needs to be ~10 million uint32 in length...
Does anyone have an idea on how to speed this up? Or what technology would be more appropriate?
So I think I've implemented streaming based on comments and answers given, but it still takes the same amount of time:
#proto
service PAS {
// analyze single file
rpc getPhotonRecords (PhotonRecordsRequest) returns (stream PhotonRecordsReply) {}
}
class PAS_GRPC(pas_pb2_grpc.PASServicer):
def getPhotonRecords(self, request: pas_pb2.PhotonRecordsRequest, _context):
raw_data_bytes = flb_tools.read_data_bytes(request.fileName)
data = flb_tools.reshape_flb_data(raw_data_bytes)
index = 0
chunk_size = 1024
len_data = len(data)
while index < len_data:
# last chunk
if index + chunk_size > len_data:
yield pas_pb2.PhotonRecordsReply(PhotonRecords=data[index:])
# all other chunks
else:
yield pas_pb2.PhotonRecordsReply(PhotonRecords=data[index:index + chunk_size])
index += chunk_size
Min repro
Github example
If you changed it over to use streams that should help. It took less than 2 seconds to transfer for me. Note this was without ssl and on localhost. This code I threw together. I did run it and it worked. Not sure what might happen if the file is not a multiple of 4 bytes for example. Also the endian order of bytes read is the default for Java.
I made my 10 meg file like this.
dd if=/dev/random of=my_10mb_file bs=1024 count=10240
Here's the service definition. Only thing I added here was the stream to the response.
service PAS {
// analyze single file
rpc getPhotonRecords (PhotonRecordsRequest) returns (stream PhotonRecordsReply) {}
}
Here's the server implementation.
public class PhotonsServerImpl extends PASImplBase {
#Override
public void getPhotonRecords(PhotonRecordsRequest request, StreamObserver<PhotonRecordsReply> responseObserver) {
log.info("inside getPhotonRecords");
// open the file, I suggest using java.nio API for the fastest read times.
Path file = Paths.get(request.getFileName());
try (FileChannel fileChannel = FileChannel.open(file, StandardOpenOption.READ)) {
int blockSize = 1024 * 4;
ByteBuffer byteBuffer = ByteBuffer.allocate(blockSize);
boolean done = false;
while (!done) {
PhotonRecordsReply.Builder response = PhotonRecordsReply.newBuilder();
// read 1000 ints from the file.
byteBuffer.clear();
int read = fileChannel.read(byteBuffer);
if (read < blockSize) {
done = true;
}
// write to the response.
byteBuffer.flip();
for (int index = 0; index < read / 4; index++) {
response.addPhotonRecords(byteBuffer.getInt());
}
// send the response
responseObserver.onNext(response.build());
}
} catch (Exception e) {
log.error("", e);
responseObserver.onError(
Status.INTERNAL.withDescription(e.getMessage()).asRuntimeException());
}
responseObserver.onCompleted();
log.info("exit getPhotonRecords");
}
}
The client just logs the size of the array received.
public long getPhotonRecords(ManagedChannel channel) {
if (log.isInfoEnabled())
log.info("Enter - getPhotonRecords ");
PASGrpc.PASBlockingStub photonClient = PASGrpc.newBlockingStub(channel);
PhotonRecordsRequest request = PhotonRecordsRequest.newBuilder().setFileName("/udata/jdrummond/logs/my_10mb_file").build();
photonClient.getPhotonRecords(request).forEachRemaining(photonRecordsReply -> {
log.info("got this many photons: {}", photonRecordsReply.getPhotonRecordsCount());
});
return 0;
}

SetPerTcpConnectionEStats and GetPerTcpConnectionEStats is returning error code 1214 (invalid NETNAME)

I am trying to get stats similar to the ones shown in "Resource Monitor" in windows in a my c++ service. For that I have used the example shown at https://learn.microsoft.com/en-gb/windows/win32/api/iphlpapi/nf-iphlpapi-getpertcp6connectionestats?redirectedfrom=MSDN. But I am stuck because SetPerTcpConnectionEStats and GetPerTcpConnectionEStats is returning with error code 1214. The only difference btn. the code in the example at above mentioned link and mine is that I am not working on a particular local and remote port but on all the entries in the tcp table, but I don't think that should make any difference.
Can somebody help me out here?
I can reproduce this error if I work with all the entries. According to the sample you linked, in addition to local and remote port, GetTcpRow has a search parameter MIB_TCP_STATE_ESTAB. The state is the normal state for the data transfer phase of the TCP connection.
The following sample works for me.
DWORD RunEstatsTest(bool v6) //set as IPv4(FALSE)
{
PMIB_TCPTABLE tcpTable = NULL;
DWORD status, size = 0;
status = GetTcpTable(tcpTable, &size, TRUE);
if (status != ERROR_INSUFFICIENT_BUFFER) {
return status;
}
tcpTable = (PMIB_TCPTABLE)malloc(size);
if (tcpTable == NULL) {
return ERROR_OUTOFMEMORY;
}
status = GetTcpTable(tcpTable, &size, TRUE);
if (status != ERROR_SUCCESS) {
free(tcpTable);
return status;
}
for (int i = 0; i < tcpTable->dwNumEntries; i++) {
if (MIB_TCP_STATE_ESTAB == tcpTable->table[i].State)
{
ToggleAllEstats(&tcpTable->table[i], TRUE, v6);
GetAllEstats(&tcpTable->table[i], v6);
ToggleAllEstats(&tcpTable->table[i], FALSE, v6);
}
}
free(tcpTable);
return ERROR_SUCCESS;
}

SNMP: Response PDU Too Big

I need to test a driver code based on the snmp++ library.When I started this driver code, I printed out the following error log
"Snmp Trap Register Error :SNMP: Response PDU Too Big"
I'm using SNMP++v3.2.25.I found an SNMP simulator on the network and sent it to the SNMP driver.Still print out the error log
"Snmp Trap Register Error :SNMP: Response PDU Too Big" .
The SNMP simulator I used was SnmpTrapGen V1.1.I sent in the CMD command for
"SnmpTrapGen.Exe -r:172.20.31.126 -p:161 -v:3 -to: 1.3.6.1.4.1.2011.2.15.2.4.3.3.0.1"
int ScsSnmp::init_snmp()
{
Snmp::socket_startup();
int status;
// UdpAddress address(m_local_addr);
m_snmp = new Snmp(status/*,address*/);
if (( m_snmp == NULL) || ( status != SNMP_CLASS_SUCCESS))
{
printlog(LOGE_SNMP + m_link,"constructing Snmp Object failed ");
}
else
{
TargetCollection targets;
OidCollection trapids;
Oid trapoid = OidAlarmReportNotificationType;
Oid heartoid = OidHeartbeatNotificationType;
trapids += trapoid;
trapids += heartoid;
m_snmp->notify_set_listen_port(TRAP_LISTEN_PORT);
ScsSnmp* myself = this;
if ( status = m_snmp->notify_register( trapids, targets,my_trap_callback,myself) != SNMP_CLASS_SUCCESS)
{
printlog(LOGE_SNMP + m_link,"Snmp Trap Register Error : %s ",m_snmp->error_msg(status));
return -1;
}
m_snmp->start_poll_thread(1000); //1000ms
}
return 0;
}
void ScsSnmp::my_trap_callback (int reason, Snmp *session,Pdu &pdu, SnmpTarget &target, void *data)
{
ScsSnmp* scssnmp = (ScsSnmp*)data;
printlog(LOGE_SNMP + scssnmp->m_link,"start my_trap_callback");
if ( reason == SNMP_CLASS_NOTIFICATION)
{
Vb nextVb;
GenAddress addr;
target.get_address(addr);
IpAddress from(addr);
Oid notify_id,ent;
pdu.get_notify_id(notify_id);
pdu.get_notify_enterprise(ent);
if (notify_id == OidAlarmReportNotificationType)
{
memset(scssnmp->m_alarm_msg,0,128);
memset(scssnmp->m_alarm_info.station,0,64);
memset(scssnmp->m_alarm_info.subsystem,0,64);
memset(scssnmp->m_alarm_info.devicetype,0,64);
memset(scssnmp->m_alarm_info.device,0,64);
memset(scssnmp->m_alarm_info.alarm_msg,0,128);
for (int i = 0;i<pdu.get_vb_count();i++)
{
pdu.get_vb(nextVb, i);
scssnmp->process_alarm_vb(nextVb);
}
memset(scssnmp->m_alarm_buf,0,512);
memcpy(scssnmp->m_alarm_buf,&scssnmp->m_alarm_head,sizeof(alarm_head));
memcpy(scssnmp->m_alarm_buf+sizeof(alarm_head),&scssnmp->m_alarm_info,sizeof(alarm_event_info));
bool ret = scssnmp->m_ctrl_inf->addAlarm(scssnmp->m_alarm_buf,512);
if (ret)
{
printlog(LOGE_SNMP + scssnmp->m_link,"add an event alarm success !");
}
else
{
printlog(LOGE_SNMP + scssnmp->m_link,"add an event alarm failed !");
}
}
else if (notify_id == OidHeartbeatNotificationType)
{
printlog(LOGE_SNMP + scssnmp->m_link,"get a heartbeat !");
}
else
{
printlog(LOGE_SNMP + scssnmp->m_link,"Trap notify id is wrong,id=%s",notify_id.get_printable());
}
}
else
{
printlog(LOGE_SNMP + scssnmp->m_link,"Trap Receive Error = ",session->error_msg(reason));
}
printlog(LOGE_SNMP + scssnmp->m_link,"end my_trap_callback");
}
I want to send SNMP traps through the emulator.The SNMP driver then receives the data and prints it out.Simply put, you want to test whether the data interface works.But the actual receiving interface keeps printing out error log
"Snmp Trap Register Error :SNMP: Response PDU Too Big" .
We encountered this before but it's in a snmp get response. this always indicates the response pdu is larger than 65535 bytes. do a packet capture on port 162 will give a more clear verification. When we encounter this in snmp get request, we decreased the number of OIDs sent in each request to fix this problem.

Socket Server Example with Swift

I tried to make an example of simple socket server.
Build and run successfully. However it doesn't work well.
Client couldn't connect to this server.
How to solve this problem? I need your help, thanks.
import Foundation
let BUFF_SIZE = 1024
func initStruct<S>() -> S {
let struct_pointer = UnsafePointer<S>.alloc(1)
let struct_memory = struct_pointer.memory
struct_pointer.destroy()
return struct_memory
}
func sockaddr_cast(p: ConstUnsafePointer<sockaddr_in>) -> UnsafePointer<sockaddr> {
return UnsafePointer<sockaddr>(p)
}
func socklen_t_cast(p: UnsafePointer<Int>) -> UnsafePointer<socklen_t> {
return UnsafePointer<socklen_t>(p)
}
var server_socket: Int32
var client_socket: Int32
var server_addr_size: Int
var client_addr_size: Int
var server_addr: sockaddr_in = initStruct()
var client_addr: sockaddr_in = initStruct()
var buff_rcv: Array<CChar> = []
var buff_snd: String
server_socket = socket(PF_INET, SOCK_STREAM, 0);
if server_socket == -1
{
println("[Fail] Create Server Socket")
exit(1)
}
else
{
println("[Success] Created Server Socket")
}
server_addr_size = sizeof(server_addr.dynamicType)
memset(&server_addr, 0, UInt(server_addr_size));
server_addr.sin_family = sa_family_t(AF_INET)
server_addr.sin_port = 4000
server_addr.sin_addr.s_addr = UInt32(0x00000000) // INADDR_ANY = (u_int32_t)0x00000000 ----- <netinet/in.h>
let bind_server = bind(server_socket, sockaddr_cast(&server_addr), socklen_t(server_addr_size))
if bind_server == -1
{
println("[Fail] Bind Port");
exit(1);
}
else
{
println("[Success] Binded Port");
}
if listen(server_socket, 5) == -1
{
println("[Fail] Listen");
exit(1);
}
else
{
println("[Success] Listening : \(server_addr.sin_port) Port ...");
}
var n = 0
while n < 1
{
client_addr_size = sizeof(client_addr.dynamicType)
client_socket = accept(server_socket, sockaddr_cast(&client_addr), socklen_t_cast(&client_addr_size))
if client_socket == -1
{
println("[Fail] Accept Client Connection");
exit(1);
}
else
{
println("[Success] Accepted Client : \(inet_ntoa(client_addr.sin_addr)) : \(client_addr.sin_port)");
}
read(client_socket, &buff_rcv, UInt(BUFF_SIZE))
println("[Success] Received : \(buff_rcv)")
buff_snd = "\(strlen(buff_rcv)) : \(buff_rcv)"
write(client_socket, &buff_snd, strlen(buff_snd) + 1)
close(client_socket)
}
The port number in the socket address must be in big-endian byte order:
server_addr.sin_port = UInt16(4000).bigEndian
So your program actually listens on port 40975 (hex 0xA00F) and not
on port 4000 (hex 0x0FA0).
Another problem is here:
var buff_rcv: Array<CChar> = []
// ...
read(client_socket, &buff_rcv, UInt(BUFF_SIZE))
Your buffer is an empty array, but recv() expects a buffer of size BUFF_SIZE.
The behaviour is undefined. To get a buffer of the required size, use
var buff_rcv = [CChar](count:BUFF_SIZE, repeatedValue:0)
// ...
read(client_socket, &buff_rcv, UInt(buff_rcv.count))
Remark: Here you cast the address of an Int to the address of an socklen_t
and pass that to the accept() function:
client_socket = accept(server_socket, sockaddr_cast(&client_addr), socklen_t_cast(&client_addr_size))
That is not safe. If Int and socklen_t have different sizes then the behaviour
will be undefined. You should declare server_addr_size and client_addr_size
as socklen_t and remove the socklen_t_cast() function:
client_socket = accept(server_socket, sockaddr_cast(&client_addr), &client_addr_size)
As Martin R commented before, the write command shouldn't be using the swift string as that. Something like this will work properly:
write(client_socket, buff_snd.cStringUsingEncoding(NSUTF8StringEncoding)!, countElements(buff_snd) + 1)

Resources