I used the tcp protocol to deal the request the client, I found a phenomenon which is some of the content is missing while using the function of 'send'. the code is as fellow:
_stprintf(cData,"[%s]",send_back);
memset(send_back,0,sizeof(cData));
int send_count;
if((send_count=send(service_sock,cData,_tcslen(cData),0))!=SOCKET_ERROR){
fwrite(cData,sizeof(char),_tcslen(cData),hFile);
fflush(hFile);
g_log->print_log("%c%c%c%c",cData[0],cData[1],cData[2],cData[send_count-1]);
g_log->print_log("buffer len is :%d , send %d bytes",_tcslen(cData),send_count);
fclose(hFile);
memset(cData,0,sizeof(cData));
return true;
}
the send function is always successful, and the value of _tcslen(cData) is equal to send_count and the cData[send_count-1] is ']'.
But when I use the wireshark(a capture tool) to capture the packet which is send out by the socket, I found some content is always missing including the Character of ']'. the content is encapsulated by JSON protocol, so the ']' is important. the total size of every time send out is 8900 bytes. But when I change the request item one time (before is 100) to 50, there is nothing missed, the size of send back is about 4000 bytes.
I do not know why this happened.
from my log file, I am sure the array named 'cData' contain the total content, But why the the content from the packets captured by the wireshark is not complete?
Seeing that you're using TCP, it already looks wrong.
First off, TCP is stream protocol which is not suited for one time packets ( especially small ) but the benefits are far more greater than just use UDP instead.
Keep in mind that in case of TPC you are not in control you can only make sure that your requests are handled correctly, the actual communication is done by the Winsock library.
Always remember that the send functions len parameter is NOT a requirement it's a hint on how big is your buffer and how much you can send in one go, it may return less than you want to send, and this may depend on lot of factors how often it happens, lets say you use the loopback device it would probably never ever do this, meaning that send will actually send what you requested. In a real network it may send it on one go in about 90% or with even less probability.
You have to make sure you send as much as you want, i.e. check for the return value and call send again if it didn't send as much as you wanted and do the same on the other side with recv, call recv until you get as much data as you wanted. This method only works if you know exactly how much data you want to send over the network.
As for the loss off data, TCP, I would say almost always sends data, assuming that you checked the return value of send. If there is a network problem, like loss of data you would see the TCP retransmit packet.
For your way of sending data this is more suitable, this is to make sure you really send the amount of data you want :
xint xsend(SOCKET s,const char* buf,xint len)
{
xint lastSize;
xint result;
if (len==0 || s==(SOCKET)NULL || buf==(const char*)NULL)
return SOCKET_ERROR;
lastSize=0;
result=0;
do
{
result=send(s,buf+lastSize,len-lastSize,0);
if (result==0)
return 0;
if (result==SOCKET_ERROR)
return SOCKET_ERROR;
if (result==len)
return len;
if (result>len)
{
xlog(1,"xsend : socket sent too much data [ %i of %i ]",result,len);
return SOCKET_ERROR;
}
lastSize+=result;
if (lastSize>len)
{
xlog(1,"xsend : socket sent too much data ( overall ) [ %i of %i ]",result,len);
return SOCKET_ERROR;
}
if (lastSize==len)
return len;
}
while (1);
xlog(2,"failed to do xsend");
return SOCKET_ERROR;
}
This code is just a copy paste from one of my projects, xlog is simple logging function, you can figure it out.
Related
What I'm trying to accomplish is to implement reading a message from one of two sockets, wherever it arrives first. As far as I understand polling (zmq_poll) is the right thing to do (as demonstrated in mspoller in guide). Here I'll provide small pseudo-code snippet:
TimeSpan timeout = TimeSpan.FromMilliseconds(50);
using (var receiver1 = new ZSocket(ZContext.Current, ZSocketType.DEALER))
using (var receiver2 = new ZSocket(ZContext.Current, ZSocketType.PAIR))
{
receiver1.Bind("tcp://someaddress");
// Note that PAIR socket is inproc:
receiver2.Connect("inproc://otheraddress");
var poll = ZPollItem.CreateReceiver();
ZError error;
ZMessage msg;
while (true)
{
if (receiver1.PollIn(poll, out msg, out error, timeout))
{
// ...
}
if (receiver2.PollIn(poll, out msg, out error, timeout))
{
// ...
}
}
}
As you can see it is actually the same exact implementation as in mspoller in guide.
In my case receiver2 (PAIR socket) should receive a large number of messages. In fact I've created a test in which number of messages sent to it is always greater than the number of messages it is capable to receive (at least in demonstrated implementation).
I've run the test for 2 seconds, and I was very surprised with results:
Number of messages sent to receiver2: 180 (by "sent" I mean that they are handed out to another PAIR socket not shown in the previous snippet);
Number of messages received by receiver2: 21 ??? Only 21 messages in 2 seconds??? 10 messages per second???
Then I've tried to play with different timeout values and I've found out that it significantly influences the number of messages received. Duration (2 seconds) and number of messages sent (180) remain the same. The results are:
timeout value of 200 milliseconds - number of messages received drops to 10 (5 per second);
timeout value of 10 milliseconds - number of messages received rises to 120 (60 per second).
The results are telling me that polling simply does not work. If polling were working properly, as far as I understand the mechanism, timeout should not have any influence in this scenario. No matter if we set timeout to 1 hour or 5 milliseconds - since there are always messages to receive there's nothing to wait for, so the loop should work with the same speed.
My another big concern is the fact that even with very small timeout value receiver2 is not capable to receive all 180 messages. I'm struggling here to accomplish receiving rate of 100 messages per second, although I've selected ZeroMQ which should be very fast (benchmarks are mentioning numbers as 6 million messages per second).
So my question is obvious: am I doing something wrong here? Is there a better way to implement polling?
By browsing clrzmq4 code I've noticed that there's also possibility to call pollIn method on enumeration of sockets ZPollItems.cs, line 151, but I haven't found any example anywhere!
Can this be the right approach? Any documentation anywhere?
Thanks
I've found the problem / solution for this. Instead using PollIn method on each socket separately we should use PollIn method on array of sockets. Obviously the example from the guide is HUGELY MISLEADING. Here's the correct approach:
TimeSpan timeout = TimeSpan.FromMilliseconds(50);
using (var receiver1 = new ZSocket(ZContext.Current, ZSocketType.DEALER))
using (var receiver2 = new ZSocket(ZContext.Current, ZSocketType.PAIR))
{
receiver1.Bind("tcp://someaddress");
receiver2.Connect("inproc://otheraddress");
// We should "remember" the order of sockets within the array
// because order of messages in the received array will correspond to it.
ZSocket[] sockets = { receiver1, receiver2 };
// Note that we should use two ZPollItem instances:
ZPollItem[] pollItems = { ZPollItem.CreateReceiver(), ZPollItem.CreateReceiver() };
ZError error;
ZMessage[] msg;
while (true)
{
if (sockets.PollIn(pollItems, out msg, out error, timeout))
{
if (msg[0] != null)
{
// The first message gotten from receiver1
}
if (msg[1] != null)
{
// The second message gotten from receiver2
}
}
}
}
Now receiver2 reaches 15,000 received messages per second, no matter timeout value, and no matter number of messages received by receiver1.
UPDATE: Folks from clrzmq4 have acknowledged this issue, so probably the example will be corrected soon.
I am working on a Linux kernel module that needs to modify network packets and append an extra header. I already implemented the modification part, recomputed the check-sums and it worked nice. But I don't know how to safely append an extra header. If my input packet is something like:
ip-header / tcp-header / data
I would like to have an output packet like:
ip-header / tcp-header / my-header / data
For what I read, I think I need something like the following code. I wrote my specific questions on the code as comments. My general concern is if the code I am writing here is memory-safe or what should I do to have a memory-safe way to append the new header. Also, if I am doing something wrong or there is a better way to do it I will also appreciate the comment. I have tried to find examples but no luck so far. Here is the code:
static unsigned int my_iptables_target(struct sk_buff *skb, const struct xt_action_param *par) {
const struct xt_mytarget_info *info = par->targinfo;
/* Some code ... */
if (!skb_make_writable(skb, skb->len)) {
//Drop the packet
return NF_DROP;
}
struct newheader* myheader;
// Check if there is enough space and do something about it
if (skb_headroom(skb) < sizeof(struct newheader)) {
// So there is no enugh space.
/* I don't know well what to put here. I read that a function called pskb_expand_head might
* do the job. I do not understand very well how it works, or why it might fail (return value
* different from zero). Does this code work:
*/
if (pskb_expand_head(skb, sizeof(struct newheader) - skb_headroom(skb), 0, GPF_ATOMIC) != 0) {
// What does it mean if the code reaches this point?
return NF_DROP;
}
}
// At this point, there should be enough space
skb_push(skb, sizeof(struct newheader));
/* I also think that skb_push() creates space at the beggining, to open space between the header and
* the body I guess I must move the network/transport headers up. Perhaps something like this:
*/
memcpy(skb->data, skb->data + sizeof(struct newheader), size_of_all_headers - sizeof(struct newheader));
// Then set myheader address and fill data.
myheader = skb->data + size_of_all_headers;
//Then just set the new header, and recompute checksums.
return XT_CONTINUE;
}
I assumed that the variable size_of_all_headers contains the size in bytes of the network and transport headers. I also think that memcpy copies bytes in increasing order, so that call shouldn't be a problem. So does the above code works? It is all memory-safe? Are there better ways to do it? Are there examples (or can you provide one) that does something like this?
I used a code similar to the one in the question and so far it has worked very well for all the test I have done. To answer some of the specific questions, I used something like:
if (skb_headroom(skb) < sizeof(struct newheader)) {
printk("I got here!\n");
if (pskb_expand_head(skb, sizeof(struct newheader) - skb_headroom(skb), 0, GPF_ATOMIC) != 0) {
printk("And also here\n");
return NF_DROP;
}
}
But none of the print statements ever executed. I suppose that happens because the OS reserves enough space in memory such that there can be no problems given the limits of the IP header. But I think it is better to leave that if statement to grow the packet if necessary.
The other difference of the code that I tested and worked is that instead of moving all the other headers up to create a space for my header, I chose to move the body of the packet down.
I am using SIM900 GSM module connect to my AVR microcontroller.My compiler is Code Vision AVR.
I want to send a message and I don’t want to use delay.I use an usart rx interrupt. I want to receive '>' in rx interrupt after sending number.Then send my meessage.
But '>' charachter wasn’t received. I am checking this process with a 'a' variable. First of all 'a' variable is equal to zero and after 1 minute is equal to 2. But it is never equal to 3. As a result my message haven't been sent. I can't find my problem!
interrupt [USART_RXC] void usart_rx_isr(void)
{
char status,data;
status=UCSRA;
data=UDR;
if ((status & (FRAMING_ERROR | PARITY_ERROR | DATA_OVERRUN))==0)
{
rx_buffer[rx_wr_index++]=data;
if(a==2)
{
if(data=='>')
a=3;
}
#if RX_BUFFER_SIZE == 256
// special case for receiver buffer size=256
if (++rx_counter == 0)
{
#else
if (rx_wr_index == RX_BUFFER_SIZE) rx_wr_index=0;
if (++rx_counter == RX_BUFFER_SIZE)
{
rx_counter=0;
#endif
rx_buffer_overflow=1;
}
}
}
void main()
{
//...rest of code
printf("at+cmgf=1%1c",enter);
printf("at+cmgda=%1c%s%1c%1c",q,del,q,enter);
while (1)
{
//...rest of code
RTC();
if(minute!=1*count)
flag1=0;
if(minute==1*count && flag1!=1)
{
flag1=1;
count++;
a=1;
}
if(a==1)
{
printf("at+cmgs=%1c%s%1c%1c",q,tel,q,enter);
a=2;
}
if(a==3)
{
printf("Ba salam\rTemperature=%d\rHumidity=%d\rWind Direction=%s\rWind Speed=%d%1c%1c",temperature(),humidity(),direct1,anemometer(),cz,enter);
a=0;
}
}
}
You are not doing proper parsing of the responses from the modem.
First of all, never, ever, ever use time as a separator for command and response. You MUST read and parse everything received from the modem until you get a final response. Absolutely nothing else will work reliably. See this answer for some hints on how to fix.
And secondly, for AT+CMGS you MUST wait for a sequence of four characters, \r\n>, before sending data. There is also hints about this in the above linked answer.
You appear to be sending commands using lower case letters. The user manual specifies capital letters, that is, AT not at.
There is also something wrong with the minute-count code at the top of the loop. The flag1 will be set back to 0, which will eventually change a to 1 again, which will cause the message to be sent again. I don't think this is your immediate problem but it should be corrected.
We also don't know if the message is getting through to the SIM900. Maybe trying testing this by sending the AT commands over a terminal and checking the return values. (Your code doesn't check if the first two commands are acknowledged by the SIM900, for example.)
EDIT:
Looking more closely, I see that your code is not following the protocol set out in the manual for sending a text. This may be because the manual has a typographical error. It has two sections labelled "Sending a Text", but one of them is actually for reading a text.
You don't need to include the AT+CMGDA=”DEL ALL” part. This is in the section for reading a text. Also, you have misspelled ”DEL ALL” as "DElALL".
The bigger problem is that sending the two commands so close together is clobbering the return codes.
In the Majordomo pattern, a section of code in the worker looks like this
mdwrk session ("tcp://localhost:5555", sourceStr.c_str(), verbose);
zmsg *reply = 0;
while (1) {
zmsg *request = session.recv (reply);
if (request == 0) {
break; // Worker was interrupted
}
//reply = request; // Echo is complex... :-)
reply = new zmsg(sourceStr.c_str());
}
To my worker, the request from the client is an order to be sent to an exchange. I am trying to wrap my head around how, after I send the order to the exchange, and I get a message back like, Insert, Pending, New, etc, I can stuff the contents of the FIX response, into zmsg *reply.
The FIX message comes back asynchrously, so I won't be able to say
reply = FIXResponse;
How is this resolved?
I think the Majordomo protocol is meant to handle synchronous requests and not really appropriate here.
Just came across one of your other questions, and see that you have multiple sources for these replies. You could PUSH them all into a single stable PULL socket? (And then republish if appropriate. If the volume is low, you could even get away with durable subscribers for reliability.)
I have Protocol Buffer for logging data.
message Message {
required double val1 = 1;
optional int val2 = 2;
}
message BigObject {
repeated Message message = 1;
}
I receive messages one per second. They stored in memory with my BigObject and they used for some tasks. But at the same time i want to store that messages in file for backup in case application crash. Simple writing BigObject every time will be waste of time. And I trying to find way to write only added messages since last write to file. Is there a way for that?
Protobuf is an appendable format, and your layout is ideal for this. Just open your file positioned at the end, and start with a new (empty) BigObject. Add/serialize just the new Message instance, and write to the file (from the end onwards).
Now, if you parse your file from the beginning you will get a single BigObject with all the Message instances (old and new).
You could actually do this by logging each individual Message as it arrives, as long as you wrap it in a BigObject each time, i.e. in pseudo-code
loop {
msg = await NextMessage();
wrapper = new BigObject();
wrapper.Messages.Add(msg);
file = OpenFileAtEnd();
wrapper.WriteTo(file);
file.Close();
}