I understand most of the POP3 protocol, but one thing that bothers me is how POP3 clients efficiently get a list of unretrieved message ids from the POP3 server. Many services like Yahoo and Gmail now offer gigs of space and most people (myself included), rarely if ever delete an email message.
I'm currently implementing a simple POP3 client in C#, though the question that I'm asking should be language agnostic.
On the client side I store a list of ALL the retrieved message ids that I've ever retrieved. There is no need for this client to ever delete messages, mark messages ad having been read etc. All it needs to do is get the newest email messages since the last time that it connected to the POP3 server.
Am I right in assuming that the algorithm goes something like this:
Retrieve ALL message IDs from the POP3 server. This list grows daily and can become megabytes in size easily.
Compare this list of message IDs with the message IDs I have already retrieved (this list being stored client side) and identify what mail messages I actually have to retrieve from the server.
Retrieve the mail messages one at a time from the server using the results from step 2 above.
Is there any way to make this more efficient?
There's no really efficient way to do this, because POP was never designed for leaving messages in the mailbox permanently - it's designed for the use-case where you fetch your mail once and delete it.
A better approach to the whole problem would be to use IMAP instead, which is designed for efficient storage.
Related
I have a websocket server and it keeps record of the users which are online. So if any change is made by a single user then it should reflect to all ther users which my server has maintained in an array.
Let's say a user changed name and total 100 users are online so server could easily loop through an array and send it to all connections.
Problem, what if I have like a good amount of user so there might be a case where my server stays in loop. So, to overcome this is there any other way to send data to all the users in websocket itself?
From the docs it is clear that message delivery is not guaranteed by django-channels. I want to implement a way in my chat app so that there is a guarantee that messages are delivered to the client.
I am using redis, if that matter. When I say messages I don't mean chat text messages. I haven't implemented that part and don't need that for now. I just have video call feature. What gets lost sometimes is when users join the room the room members list doesn't reach them. So I want to do something like:
Client connects.
Servers creates a record of the event in step 3 below.
Server sends the list of room members event to the client.
Client process and acknowledges the list.
Server on receiving the acknowledgment from the client marks the list sent event as SUCCESS or something.
My questions are.
Is this approach good?
And should I use a new model to store the message events so that server can know which messages to sent and which have been already sent successfully or is there anyway to do it in redis db?
I am new to this so I maybe overlooking something that you as experienced guys know of. Please guide me with your precious inputs.
Thanks
I have a PUB server. How can it tell what filters are subscribed to, so the server knows what data it has to create?The server doesn't need to create data once no SUB clients are interested in.
Say the set of possible filters is huge ( or infinite ), but subscribers at any given time are just subscribed to a few of them.
Example: Say SUB clients are only subscribed to a weather feed data for a few area codes in New York and Paris. Then the PUB server shouldn't have to create weather data for every other area code in every other city in the world, just to throw it all away again.
How do you find out all the subscribed to filters in a PUB server?
If there is no easy way, how do I solve this in another way?
I'll answer my own question here in case its of use to anyone else.
The requirements where:
The client should be able to ask the server what ids (topics) are available for subscription.
The client should chooses the id's it is interested in and tell the server about it.
The server should created data for all subscribed too id's and send that data to clients.
The client and server should not block/hang if either one goes away.
Implementation:
Step 1. Is two way traffic, and is done with REQ/REP sockets.
Step 2. Is one way traffic from one client to one server, and is done by PUSH/PULL sockets.
Step 3. Is one way traffic from one server to many clients, and is done by PUB/SUB sockets.
Step 4. The receives can block either the server or client if the other one is not there. Therefore I followed the "lazy pirate pattern" of checking if there is anything to receive in the queue, before I try and receive. (If there is nothing in the queue I'll check again on the next loop of the program etc).
Step 4+. Clients can die without unsubscribing, and the server wont know about it, It will continue to publish data for those ids. A solution is for the client to resends the subscription information (with a timestamp) every so often to the server. This works as a heartbeat for the ids the client has subscribed too. If the client dies without unsubscribing, the server notices that some subscription ids have not been refreshed in a while (the timestamp). The server removes those ids.
This solution seems to work fine. It was a lot of low level work though. It would be nice if zeromq was a bit higher level, and had some common and reliable architectures/frameworks ready to use out of the box.
Is there anyway to get the same "MessageId" you can get in Exchange EWS when using ActiveSync?
I thought this was an Exchange way to identify each message uniquely, but I can't seem to find a way to retrieve it using ActiveSync.
EDIT: I've got 2 applications, one that stores info using ActiveSync, and one that stores info using EWS, and I want them to be able to work separately on the same message.... To do this, I was hoping to use the EWS MessageId, which seems to be a GUID type identifier for each individual message. (Note: This doesn't appear to be the same Message-ID as is found in email headers).
Sadly, you're mostly out of luck.
ActiveSync is not an integration protocol, it's a mobile synchronization protocol designed for low-bandwidth communication devices like smart phones. A lot of capabilities in EWS will not exist in EAS.
Long-term message identification and correlation isn't as important for mobile devices. They simply get told what messages are in each folder, and allow the user to manipulate them. At any time the Exchange server may tell its EAS-connected clients to "re-sync" which causes them to forget the messages they have on the device and pull them cleanly from the server. That happens a lot with EAS, sometimes a couple of times an hour, depending on what is happening with that mailbox. For example, deleting a folder via Outlook causes a FolderSync to happen, and that forces connected devices to cleanly re-sync again.
Therefore EAS appears to have left behind the notion of GUIDs or other long term IDs for messages. Instead, the server will assign ephemeral IDs that are valid only until the next big resync is forced (which could happen at any time). You'll probably see Exchange give very simple IDs like 7:45 (which means message ID 45 within folder 7, IIRC). However after a resync that might have the number 7:32 (if the user deletes other messages in that folder) or something like 4:22 (if the message gets moved to another folder entirely).
Other EAS servers like Zimbra, Kerio or Notes Traveler might assign GUIDs, but from memory this is how Exchange behaves. Your only option might be to put a hidden correlation ID of your own into the body or subject of messages you're interested in. That will allow you to track the lifecycle of the items you're interested in, at the expense of some odd stuff being visible to users in their message contents.
#Brian is correct - There are no global unique identifiers for ActiveSync items that can be used to correlate with EWS (With some exceptions, for instance a meeting invite has a UID, as do Events which can be used with some hackery to retrieve an EWS ID for the related EWS calendar event) and there are no fields that aren't visible to the user that can be hijacked for adding your own data with which to correlate. This is most apparent in email, contacts, tasks, notes etc...
However if you are syncing both, it is possible to use the meta data in the objects to match. For instance, for contacts write a hashing algorithm that combines the data from the first name, last name, company name, etc... fields and produces a result. This can be run on the data from both sides and will have relatively little object collision for matching (and those that do collide will have exactly the same visible data to the user anyway so in most cases it won't matter that you didn't get an exact alignment)
Hypothetical: I want to send a single text message to all Verizon phones programmatically. I have multiple email gateways to use (obtained from the all-reliable wikipedia):
number#vtext.com
number#vzwpix.com
number#message.alltel.com
number#text.wireless.alltel.com
number#mms.alltel.net
I don't think that I'm guaranteed that any one of these will work and/or will still be in service (am I?) and I would not like to have to come back and change anything in the code at a later date.
Is there any way that I can make sure that I only send one text message to a given phone number when there are 5 possible gateways?
The only way way (that I know of) to ensure you only deliver one message to the recipient is to try each gateway sequentially until a message sends successfully, like some of the comments mentioned.
However, I've been sending a decent volume (>1500) of messages using #vtext.com lately and haven't noticed any bounces or downtime during sending. I have no way of knowing if every single message was delivered, but none of my test numbers have dropped a message yet. Most US carriers seem to have decent reliability on their gateways these days.
Just remember that SMS is still considered a best-effort service by most carriers. Even if you get your message to their servers successfully, there's no guarantee that the message will make it to its destination.