I am transfering some data remotely packet by packet.
Before sending each packet I need to have a sleep for some time (milliseconds). After transferring each file I have a feedback: fail or success.
Of course as smaller delay I have as smaller success rate will be however time for transferring will be less.
My goal is to adjust automatically current delay to make average SUCCESS RATE equal some constant (say 98%).
Intuitevly I assume:
After each success transfer I'll increase current delay
After each unsuccess transfer I'll decrease current delay
In time I'll modify current delay less (fade)
What algorithms would you suggest for efficient (from viewpoint of time to learn, memory) finding optimal parameter value?
You are essentially describing a network congestion solution. Look at http://en.wikipedia.org/wiki/Network_congestion_avoidance#Avoidance for much more information on the subject.
One algorithm that might suit you well is to decrease the time you wait after each successful transfer. After an unsuccessful transfer increase the time (either by a set amount or dynamically) and repeat indefinitely. I wish I could remember the specific name for this algorithm but at the moment it is escaping me.
Note if you are truly sending packages over a network and not just a play network "optimal" is not a constant as the network is always in a state of change.
Related
I am dealing with a CDN system that has a max request rate per minute (since all objects are about the same size, there is no bitrate limit)
Frankly, I do not yet know if it's #/clock minute, or calculated rate.
I have a single daemon that downloads items in threads-on-demand (as opposed to independent workers). This is the proper model for this system.
"They" suggested using an exponential backoff when the limit is hit, but that doesn't make any sense to me. The main use of exponential backoff is to resolve resource collision issues. I suppose if I had independent workers, this might make sense.
But for a single daemon system (again, the proper use model here), why is this better than either waiting for the next clock minute or just using a rate regulation mechanism?
Is there some Knuth "first fit equivalent to best fit" proof that shows this to be a good mechanism? It's certainly the easiest to implement!
I have an embedded system with WiFi (based on an 80MHz ESP8266, developed using Arduino IDE), and I would like it to keep reasonably accurate clock time (to one second), using the two tools at its disposal: the internet, and its own internal timers.
Challenges:
The processor clock will likely drift over time, ostensibly in a
predictable manner.
NTP uses UDP, so return packets with the time are not guaranteed to
return in order, or to return within any set interval, or to return
at all.
Latency of return NTP packets varies widely over time, from under
100ms to (potentially) several seconds.
Latency of DNS varies widely over time, from under 100ms up to
several seconds (I can't control the timeout). DNS is needed to look
up IP addresses for the NTP server pool(s).
The system takes various actions based on certain times and
intervals, so I don't want to over-control the time, setting it
forward and backward continually, causing actions to be needlessly
missed or duplicated (or, alternatively, to complicate the program
logic handling all of these conditions - an occasional miss or
duplicate is not mission critical).
Sometimes an NTP server will return the wrong time (e.g.,
pool.ntp.org occasionally returns 0 seconds since 1900, which is easy
to detect)
Sometimes a stray return packet with an old time will arrive just
ahead of the return packet from the current request.
Current Approach:
Keep a local device time incrementing using an ISR triggered every 0.1
second.
Periodically poll (currently every 6 minutes, but really doesn't have
to be this often) an NTP server (pool).
Try again if there's no response within a short interval (currently 1
second, which is shorter than typical but most requests return in
under 150ms).
Vary the NTP server (pool) on each try, to spread the load and to
average out response times and any service errors.
Extract the time to the nearest 0.1 second (and adjust for typical
receive latency).
If the NTP time is off from the local device time by more than a
second, update the local device time (in a critical section).
Timeout, retry, and re-initialize (where appropriate), for failed
network elements of the process. Abandon the request after most hope
is lost, and just try again next time.
Is there a better, or canonical, or best practices way to do this time synchronization? Are there other factors or approaches I'm missing?
For automatic updating time from the internet or GPS system will overkill a cheap arduino project, for you may require a Ethernet shield and write a TSR program that will regularly send updated time to the arduino. Instead you can purchase more accurate RTC clock ( some are getting more accurate these days) and manually update the clock from time to time using keyboard and LED display incorporated with the arduino. If at all your project is very time critical, it is better to you some cheap crystal based solution instead.
I have NTP client implementation (on Linux) to send/receive packets to (Stratum 1 or 2) NTP server and get the server time on the board. Also, I have another application running on Linux which gives me the GPS time.
Before I get the time information from NTP and GPS source, I will be setting the time manually(using date) on the board close to current GPS time(this info is taken from http://leapsecond.com/java/gpsclock.htm).
Keeping the system time on the board as the reference, I will take the difference of this reference time with NTP(say X) and GPS(Y). The difference between X and Y is coming to be 500+ ms. I was curious to know the time precision between NTP and GPS.
Is 500 ms an expected value?
I tried enabling hardware time-stamping on the NTP client,however it has not made any difference.
Using a GPS as a reference clock boils down to one thing: PPS (Pulse-Per-Second). Everything else is pretty jittery (unstable/unpredictable).
The PPS output is extremely accurate (nanoseconds).
PPS does not contain any information except when a second starts. That means we have to feed our clock the date and time from another source. NMEA (the actual data from the GPS) is fine as long as it’s good enough to determine the time to one second accuracy.
My guess is that your “GPS time” is the time (and date) from the GPS “data outlet”. This time can be off by 500ms (or even worse) and that’s normal. That’s why we do not use that time as an accurate reference clock.
You might want to read about time references. I think the GPS time system is not strictly identical to the UTC time returned by those time-server. The time measured by the atomic clocks have a leap second added periodically to get UTC time within +/1 second of the astronomical time which is not stable.
Is your NTP implementation able to correct network latency ? Try using a NTP server with a low-latency to you...
These factors may explain the difference you see.
I work for a company that develops psychological tests. One of these tests measures the reaction time of a candidate.
Anyone has an idea of the maximum delay between a key press and the time that this key event is available? What are the dependencies?
Is there a guaranteed maximum response time? I readed something about 5 - 25 ms.
What is the best way to handle keyevents to have a minimum delay?
Thanks in advance,
Kevin
Windows UI processing is very complex. It includes algorithms like priority promotion on keypress, but it will usually wait until the next scheduler ticklet (at worst, 30ms on desktop systems and 60ms on server systems) if another process asks for a full CPU cycle,
To overcome that, you would need a special keyboard driver that would provide event at the same latency, but also measure the accurate time. Accurate time measurement is possible on windows systems if dynamic CPU clock switching is disabled (Lookup entry QueryPerformanceCounter(), you would need to know how to invoke it from DDK), in which case, the keyboard event would still arrive with unpredictable latency, but the original bus event would be time-stamped correctly. Then you remain only with bus latencis which should be smaller than the standard deviation of your measurements. See also What happens from the moment we press a key on the keyboard, until it appears in your word document
This question is related with Microcontroller programming but anyone may suggest a good algorithm to handle this situation.
I have a one central console and set of remote sensors. The central console has a receiver and the each sensor has a transmitter operates on same frequency. So we can only implement Simplex communication.
Since the transmitters work on same frequency we cannot have 2 sensors sending data to central console at the same time.
Now I want to program the sensors to perform some "polling". The central console should get some idea about the existence of sensors (Whether the each sensor is responding or not)
I can imagine several ways.
Using a same interval between the poll messages for each sensor and start the sensors randomly. So they will not transmit at the same time.
Use of some round mechanism. Sensor 1 starts polling at 5 seconds the second at 10 seconds etc. More formal version of method 1.
The maximum data transfer rate is around 4800 bps so we need to consider that as well.
Can some one imagine a good way to resolve this with less usage of communication links. Note that we can use different poll intervals for each sensors if necessary.
I presume what you describe is that the sensors and the central unit are connected to a bus that can deliver only one message at a time.
A normal way to handle this is to have collision detection. This is e.g. how Ethernet operates as far as I know. You try to send a message; then attempt to detect collision. If you detect a collision, wait for a random amount (to break ties) and then re-transmit, of course with collision check again.
If you can't detect collisions, the different sensors could have polling intervals that are all distinct prime numbers. This would guarantee that every sensor would have dedicated slots for successful polling. Of course there would be still collisions, but they wouldn't need to be detected. Here example with primes 5, 7 and 11:
----|----|----|----|----|----|----|----| (5)
------|------|------|------|------|----- (7)
----------|----------|----------|-:----- (11)
* COLLISION
Notable it doesn't matter if the sensor starts "in phase" or "out of phase".
I think you need to look into a collision detection system (a la Ethernet). If you have time-based synchronization, you rely on the clocks on the console and sensors never drifting out of sync. This might be ok if they are connected to an external, reliable time reference, or if you go to the expense of having a battery backed RTC on each one (expensive).
Consider using all or part of an existing protocol, unless protocol design is an end in itself - apart from saving time you reduce the probability that your protocol will have a race condition that causes rare irreproducible bugs.
A lot of protocols for this situation have the sensors keeping quiet until the master specifically asks them for the current value. This makes it easy to avoid collisions, and it makes it easy for the master to request retransmissions if it thinks it has missed a packet, or if it is more interested in keeping up to date with one sensor than with others. This may even give you better performance than a system based on collision detection, especially if commands from the master are much shorter than sensor responses. If you end up with something like Alohanet (see http://en.wikipedia.org/wiki/ALOHAnet#The_ALOHA_protocol) you will find that the tradeoff between not transmitting very often and having too many collisions forces you to use less than 50% of the available bandwidth.
Is it possible to assign a unique address to each sensor?
In that case you can implement a Master/Slave protocol (like Modbus or similar), with all devices sharing the same communication link:
Master is the only device which can initiate communication. It can poll each sensor separately (one by one), by broadcasting its address to all slaves.
Only the slave device which was addressed will reply.
If there is no response after a certain period of time (timeout), device is not available and Master can poll the next device.
See also: List of automation protocols
I worked with some Zigbee systems a few years back. It only had two sensors so we just hard-coded them with different wait times and had them always respond to requests. But since Zigbee has systems However, we considered something along the lines of this:
Start out with an announcement from the console 'Hey everyone, let's make a network!'
Nodes all attempt to respond with something like 'I'm hardware address x, can I join?'
At first it's crazy, but with some random retry times, eventually the console responds to all nodes: 'Yes hardware address x, you can join. You are node #y and you will have a wait time of z milliseconds from the time you receive your request for data'
Then it should be easy. Every time the console asks for data the nodes respond in their turn. Assuming transmission of all of the data takes less time than the polling period you're set. It's best not to acknowledge the messages. If the console fails to respond, then very likely the node will try to retransmit just when another node is trying to send data, messing both of them up. Then it snowballs into complete failure...