I'm currently running batch predictions on Vertex AI with a custom FastAPI container and manualBatchTuningParameters set to {"batch_size": 2}. My JSONL file contains 646 predictions that mostly succeed except for the few that result in the following error:
('Post request fails. Cannot decode the prediction response
...<long and seemingly valid json>...
Error: Unterminated string starting at: line 1 column 97148 (char 97147)', 2)
Based on the common positioning (char 97147) of the character in the error, it seems like the response is being truncated before the stream is completely received by the batch "airflow worker." Given that TCP is a streaming protocol, I believe the batch interface is only receiving a portion of the buffers.
I've attempted to reproduce the error by deploying the same model as a vertex endpoint and requesting the same predictions that errored in batch mode.
Why am I occasionally getting this error?
Related
I have an issue in being unable to recive the URC message from the modem whenever it receives an SMS.
I know that it receives them since i can find and read them if I use AT+CMGL but, i don't receive any notification when the modem gets them. I played around with the URC related commands but I've been unable to get it to work (other URCs work fine).
The modem is a BG600L M3 from Quectel and following is the sequence of commands i'm sending ("AT" is always omitted and the first command is literally "AT\r", basically an empty one).
//general config
AT\r
CFUN=1,0
E1
+QCFG=\"urc/ri/other\",\"pulse\",8,1
H0
&F
V1
+CMEE=1
&D0
E1
+CREG=2
+CGREG=2
+CEREG=2
//sms config
+CPMS=\"ME\",\"ME\",\"ME\"
+QINDCFG=\"smsincoming\",1
+CMGF=1
+CSDH=0
+CSCS=\"GSM\"
+CNMI=2,2,0,2,0
//doing some deleting and reading
+CMGD=1,3
+CPMS?
//getting the gps fix
+QGPS=1
+QGPSCFG=\"gnssconfig\",3
+QGPSLOC=1
+QGPSEND
//resetting the gms connection
+CFUN=0
+CFUN=1,0
//setting up the gsm connection
+QICFG=\"dataformat\",0,0
+QICFG=\"viewmode\",0
+QICFG=\"recvind\",1
+QICFG=\"tcp/retranscfg\",3,600
+QISDE=0
+QCFG=\"band\",0xf,0x80085,0x80085,1
+QCFG=\"nwscanmode\",1,1
+QCFG=\"nwscanseq\",010101,1
+QCFG=\"iotopmode\",2,1
// checking if it's connected
+CREG?
+QNWINFO
+COPS?
//Getting the time
+CTZU=3
+CTZR=0
+QLTS
+CCLK?
You can set AT+CNMI=2,1,2,0,0 , that should do the trick.
According to specification ETSI TS 127 005 V11.0.0 (2012-10)
+CNMI: <mode>,<mt>,<bm>,<ds>,<bfr>
by keeping <mt> value to 1 we should get indication when message is stored in ME/TA
<mt>: integer type (the rules for storing received SMs depend on its
data coding scheme
0 No SMS-DELIVER indications are routed to the TE.
1 If SMS-DELIVER is stored into ME/TA, indication of the memory location is routed to the TE using unsolicited result code:
+CMTI: <mem>,<index>
I'm scripting for ISO Messages using JMeter to do Load Testing. I have used the ISO 8583 Sampler and connection configuration. All the data field by field is mapped correctly including the MTI (Field 0). But the issue is that the request is not being sent to the server from the JMeter. It just keep on giving the timeout issue and I'm getting no response against it. I've seen the request and validated it format from the implementer himself and he says that the format is correct. Can anyone help to trouble shoot this issue?
Check out Why am I getting timeouts? documentation section,
Why am I getting timeouts?
The three common reasons for response timeouts are:
The sampler does not receive any response.
The sampler does receive a response but fails to unpack it.
The sampler does receive a response and unpacks it but no request can be matched.
The debug log should contain Channel output similar to the following:
2022-02-22 12:34:56,789 DEBUG n.c.b.j.i.Q2: (channel/HOSTNAME:POST) [send] Out: 0800 000001
2022-02-22 12:34:56,987 DEBUG n.c.b.j.i.Q2: (channel/HOSTNAME:POST) [receive] In: 0810 000001
If only the first log line is present, no response was received (case 1 above). It is likely that the request is incorrectly formed and the remote system discarded it, so checking its logs/traces may be helpful. Double-check the Packager Configuration file! This defines how a request is packed (or response is unpacked) before (after) it goes over the wire.
If both lines are present, a response was in fact received (cases 2 and 3 above).
If the second log line instead contains an error like the following, then the response failed to unpack (case 2 above). Double-check the Packager Configuration file!
2022-02-22 12:34:56,987 ERROR n.c.b.j.i.Q2: (channel/HOSTNAME:POST) [receive] org.jpos.iso.SOMECLASSNAMEHERE: Problem unpacking field ...
If the second log line contains no error then it is likely that the plugin did not find a matching request (case 3 above). Double-check the Mux Settings! These define MTI values and message fields that are used for matching, and the default settings may not work.
In order to increase JMeter's logging verbosity for the ISO 8583 sampler add the next line to log4j2.xml file:
<Logger name="nz.co.breakpoint.jmeter.iso8583" level="debug" />
JMeter restart will be required to pick up the change.
Also make sure that you can reach the target server host/port from the machine where JMeter is running using a telnet client or equivalent.
I get a 410 when syncing messages:
{\"code\":\"SyncStateNotFound\",\"message\":\"The sync state
generation is not found; generation=1;[highest=4][4][2][3].\"}
This only occurs when syncing messages for select mailfolders on select accounts. It occurs when making a post-initial sync using the relevant delta token. I can recreate this via making
GET https://outlook.office365.com/api/v2.0/me/MailFolders('{folder_id}')/messages/?$deltaToken={delta_token}
In Microsoft's Outlook Sandbox.
Here are the literal steps it takes to reproduce deterministically:
1) Initial Message Sync:
GET https://outlook.office365.com/api/v2.0/me/MailFolders('{folder_id}')/messages
2) Sync with initial delta token:
GET https://outlook.office365.com/api/v2.0/me/MailFolders('{folder_id}')/messages/?$deltaToken={delta_token}
3) Sync with skip token until delta token:
GET https://outlook.office365.com/api/v2.0/me/MailFolders('{folder_id}')/messages/?$skipToken={skip_token}
4) ERROR OCCURS HERE: Mailfolder receives update, so I re-sync messages with delta token from (3). The call below throws a 410 and I can't sync messages.
GET https://outlook.office365.com/api/v2.0/me/MailFolders('{folder_id}')/messages/?$deltaToken={delta_token}
To reiterate: I've isolated this to just testing in the Outlook sandbox, and it still occurs. Testing as in making the GET call to sync (i.e., make perform (2)) using the deltaToken from (3) and its corresponding folderId as query parameters.
Dumb Mistake: Passed in initial delta token as opposed to current.
I am using the Microsoft Outlook REST API to synchronize messages in a folder using skipTokens with the Prefer: odata.track-changes header.
After 62 successful rounds of results, I get an error 500 ErrorInternalServerError with the message Unable to cast object of type 'LegacyPagingToken' to type 'Microsoft.Exchange.Services.OData.Model.SkipToken'
I have tried:
Retrying the same query (https://outlook.office.com/api/v2.0/me/MailFolders/Inbox/messages/?%24skipToken=1BWUA9eXs5dN89tPsr_FOvtzINQAA0Cwk5o), which results in the same error
Restarting the sync, which results in the same error at the same point
Adding a new message to the Inbox and restarting the sync, which results in the same error at the same point
Moving the messages from that part of the sync to another folder (in case the messages themselves were causing the problem), which results in the same error at the same point
Has anybody run into this error or have suggestions on what might cause it or workarounds?
It looks like the issue was on my end while parsing the skipToken from the #odata.nextLink response. The token in the original question is invalid - the actual skipToken passed back from the API had -AAAA on the end. After 63 queries, in which the skipToken increments, the Base64 encoded form started using characters the regexp I was using didn't find. Switching from a \w regexp to a proper URL parser solved the problem.
I am trying to get the number of label formats remaining in the print buffer of a Zebra printer. The printer is being accessed using the Zatar cloud service. To achieve this I am:
putting the printer in a paused state
calling the ^HS command
looking at the 5th field in string 1 of the response
According to the ZPL documentation this field is the
number of formats in receive buffer
However, this value does not appear to be correct. Each subsequent time we call it whilst the printer is paused the value in the field increases. No other jobs are being sent to the printer.
Here is same output of the response:
DEVICE_COMPLETED_SUCCESSFULLY - 030,0,1,0834,003,0,0,0,000,0,0,0
000,0,0,0,0,2,6,0,00000001,1,001
1234,0
Then this after a short interval:
DEVICE_COMPLETED_SUCCESSFULLY - 030,0,1,0834,026,0,0,0,000,0,0,0
000,0,0,0,0,2,6,0,00000001,1,001
1234,0
And so on:
DEVICE_COMPLETED_SUCCESSFULLY - 030,0,1,0834,028,0,0,0,000,0,0,0
000,0,0,0,0,2,6,0,00000001,1,001
1234,0
The initial response of 003 is correct. However I do not understand why it is then incrementing to 26 and then to 28.
Why is the response not providing the correct value for the formats remaining in the receive buffer?
The reason for the discrepancy in the number of formats in the receive buffer was due to how the printer was being accessed. The Zatar cloud service was used rather than any direct method such as USB.
The Zatar cloud service uses a device called an Edgebox to communicate with the printer. The Edgebox was periodically sending commands to the printer and it was these commands that were accumulating in the receive buffer.