Is there anyway to get the same "MessageId" you can get in Exchange EWS when using ActiveSync?
I thought this was an Exchange way to identify each message uniquely, but I can't seem to find a way to retrieve it using ActiveSync.
EDIT: I've got 2 applications, one that stores info using ActiveSync, and one that stores info using EWS, and I want them to be able to work separately on the same message.... To do this, I was hoping to use the EWS MessageId, which seems to be a GUID type identifier for each individual message. (Note: This doesn't appear to be the same Message-ID as is found in email headers).
Sadly, you're mostly out of luck.
ActiveSync is not an integration protocol, it's a mobile synchronization protocol designed for low-bandwidth communication devices like smart phones. A lot of capabilities in EWS will not exist in EAS.
Long-term message identification and correlation isn't as important for mobile devices. They simply get told what messages are in each folder, and allow the user to manipulate them. At any time the Exchange server may tell its EAS-connected clients to "re-sync" which causes them to forget the messages they have on the device and pull them cleanly from the server. That happens a lot with EAS, sometimes a couple of times an hour, depending on what is happening with that mailbox. For example, deleting a folder via Outlook causes a FolderSync to happen, and that forces connected devices to cleanly re-sync again.
Therefore EAS appears to have left behind the notion of GUIDs or other long term IDs for messages. Instead, the server will assign ephemeral IDs that are valid only until the next big resync is forced (which could happen at any time). You'll probably see Exchange give very simple IDs like 7:45 (which means message ID 45 within folder 7, IIRC). However after a resync that might have the number 7:32 (if the user deletes other messages in that folder) or something like 4:22 (if the message gets moved to another folder entirely).
Other EAS servers like Zimbra, Kerio or Notes Traveler might assign GUIDs, but from memory this is how Exchange behaves. Your only option might be to put a hidden correlation ID of your own into the body or subject of messages you're interested in. That will allow you to track the lifecycle of the items you're interested in, at the expense of some odd stuff being visible to users in their message contents.
#Brian is correct - There are no global unique identifiers for ActiveSync items that can be used to correlate with EWS (With some exceptions, for instance a meeting invite has a UID, as do Events which can be used with some hackery to retrieve an EWS ID for the related EWS calendar event) and there are no fields that aren't visible to the user that can be hijacked for adding your own data with which to correlate. This is most apparent in email, contacts, tasks, notes etc...
However if you are syncing both, it is possible to use the meta data in the objects to match. For instance, for contacts write a hashing algorithm that combines the data from the first name, last name, company name, etc... fields and produces a result. This can be run on the data from both sides and will have relatively little object collision for matching (and those that do collide will have exactly the same visible data to the user anyway so in most cases it won't matter that you didn't get an exact alignment)
Related
A project I'm working on requires users to insert themselves into a list on a server. We expect a few hundred users over a weekend and while very unlikely, a collision could happen in which two users submit the list concurrently and one of them is lost. The server has no validation, it simply allows you to get and put data.
I was pointed in the direction of "optimistic locking" but I'm having trouble grasping when exactly the data should be validated and how it prevents this from happening. If one of the clients reads the data, adds itself and then checks again to ensure that the data is the same with the use of an index or timestamp, how does this prevent the other client from doing the same and then one overwriting the other?
I'm trying to understand the flow in the context of two clients getting data and putting data.
The point of optimistic locking is that the decision to accept or reject a write is taken on the server, and is protected against concurrency by a pessimistic transaction or some sort of hardware protection, such as compare-and-swap. So a client requests a write together with some sort of timestamp or version identifier, and the server only accepts the write if the timestamp is still accurate. If it isn't the client gets some sort of rejection code and will have to try again. If it is, the client gets told that its write succeeded.
This is not the only way to handle receiving data from multiple clients. One popular alternative is to use a reliable messaging system - for example the Java Messaging Service specifies an interface for such systems for which you can find open source implementations. Clients write into the messaging system and can go away as soon as their message is accepted. The server reads requests from the messaging system and acts on them. If the server or the network goes down it's no big deal: the messages will still be there to be read when they come back (typically they are written to disk and have the same level of protection as database data although if you look at a reliable message queue implementation you may find that it is not, in fact, built on top of a standard database table).
One example of a writeup of the details of optimistic locking is the HTTP server Etag specification e.g. https://en.wikipedia.org/wiki/HTTP_ETag
The Problem
We are currently architecting our new Notification Microservice but having trouble with how to handle aggregated emails. What we need to do is instead of sending one email every action performed (could be 20+ in a few minutes), we would send an email after an hour summarising all the actions that were completed.
What We Have So Far
We so far propose that we have this type of messaging pattern, where Client Service is any service in our cluster and Messagebot is our Notification Microservice.
Client Service sends a notification to Messagebot that it will need to send something in the future
Messagebot stores the details in its database
Messagebot periodically checks its database for what needs to be sent
Messagebot gets the required data from another service (could be Client Service) via API
Messagebot sends email using the data from #3 and an HTML template
The Debate
For the data that needs to be sent, we are less sure and it is what we need help with. So far we think this should be the structure of the JSON from Client Service to Notification Service (step #1):
{
template_id: SOME_TEMPLATE_ID,
user_id: SOME_USER_ID,
objectid: SOME_OBJECT_ID
}
or
{
template_id: SOME_TEMPLATE_ID,
user_id: SOME_USER_ID,
required_objects: { task_id: SOME_TASK_ID, document_id: SOME_DOCUMENT_ID }
}
Where task_id and document_id are just examples and it would change based on the template. It could just as easily be {product_id: SOME_PRODUCT_ID} for a different template.
Why The Debate
Our thoughts so far are that:
We only need template_id because the source of the data would be implied in the objects (like an ENV var). For example, the Task object would be at http://taskservice/:id. Otherwise, we can have problems with failing APIs or switching URLs in the future.
We should use userid instead of email and name because we prevent the issue of email/ name pairs not matching up over multiple messages
For the objects, we're still sceptical because it means that the client app service would need knowledge of the inner workings in Messagebot but a single objectid might not be very extensible. We could easily imagine many of our messages needing more than one object.
In Conclusion
Thank you for reading. The design of this service is important because it will be central to our entire organisation.
Which debated JSON structure is most appropriate in our situation? Also, knowing our requirements, what would be the proper setup for this type of service? (aka. Are we correct in our other assumptions?)
So your messagebot will
store notifications
get data from other services
compile emails from the data and
send the compiled emails
In my opinion, your messagebot were given too many tasks. If I were designing the system, I would like to keep the messagebot simpler. The servces should encapsulate the knowledge to compile the email, e.g. manage it's own template and so on. The services will push the compiled emails to a queue so the messagebot can pick up and send. The only logic in the messagebot is to pick up the emails from the queue and send. In this way, it doesn't matter how many more services you are going to have in the future, the messagebot will stay nice and simple.
Hypothetical: I want to send a single text message to all Verizon phones programmatically. I have multiple email gateways to use (obtained from the all-reliable wikipedia):
number#vtext.com
number#vzwpix.com
number#message.alltel.com
number#text.wireless.alltel.com
number#mms.alltel.net
I don't think that I'm guaranteed that any one of these will work and/or will still be in service (am I?) and I would not like to have to come back and change anything in the code at a later date.
Is there any way that I can make sure that I only send one text message to a given phone number when there are 5 possible gateways?
The only way way (that I know of) to ensure you only deliver one message to the recipient is to try each gateway sequentially until a message sends successfully, like some of the comments mentioned.
However, I've been sending a decent volume (>1500) of messages using #vtext.com lately and haven't noticed any bounces or downtime during sending. I have no way of knowing if every single message was delivered, but none of my test numbers have dropped a message yet. Most US carriers seem to have decent reliability on their gateways these days.
Just remember that SMS is still considered a best-effort service by most carriers. Even if you get your message to their servers successfully, there's no guarantee that the message will make it to its destination.
I am about to start building an SMS alerting function in my web application. The aim is to provide two services:
Host pays - e.g. to send an SMS alerting users to the cancellation of an event
User pays - e.g. to alert that an Email has been sent with, say, details of a new event (apparently a requirement from users when they are away from their EMail systems!)
I also figure that there will be other user-pays scenarios such as if they have secondary mobile phones that they want alerting as the Host will only want to pay for a single phone per user.
From my research:
I can use a 3rd party SMS Gateway provider. Cost is about GBP 0.05 per message. I can either send an EMail to 999999999#TheGatewayProvider.com or I can use an HTTP request with suitable parameters in the URL.
I can send an EMail to the user's network provider (I believe this is only available in the USA)
EDIT: There are variations on how the various providers will handle the From/Subject and Message, so presentation of actual message sent may be hard to predict.
I can set up my own Gateway (which I think is way beyond my ability, and may give grief at our data centre!)
It strikes me that:
EMail we send sometimes get delayed in my server's SMTP queue, let alone any downstream queues. Sending an email to the network provider's SMS gateway seems to often be treated as low priority.
Therefore HTTP to a 3rd party SMS Gateway provider should give me shortest latency (important for "This afternoons event is cancelled because of bad weather")
When I send SMS Text from my mobile occasionally they take days to arrive - I presume this is something we just have to live with?
Having said that we will also have low priority informational messages and sending these by cheapest route is attractive! so I am planning to allow users to enter an EMail address for such messages - the intention being that they will use the email address for their mobile phone company's SMS redirect service, or similar (i.e. the email address of a device, not an inbox).
I also wonder whether allowing the users to enter IM, Twitter or other such is likely to be well used and efficient in practice?
From the 3rd party gateways I have looked at it seems:
Some use higher grade networks than others, this may impact performance? (or is it just marketing hogwash?)
Some provide better feedback than others. I need to keep the arguments over billing - and exactly how many "credits" the client has used up - to a minimum; getting an answerback of "OK" / "Phone number does not exist" therefore strikes me as important. One provider I found creates a text log file daily that can be downloaded and which I could reconcile with my outgoing log.
I would appreciate your opinions and experiences on:
Users will enter their mobile numbers as they know them. Do I need to enforce +9912345... so that I get the country code too?
What happens if the mobile is foreign (I'm based in the UK) Does the recipient pay the international part? or does the gateway provider perhaps have local transmission services?
What do I need to do with non alpha characters? The UK Pound sign "£" and CR / LF spring to mind. If these are encoded might that cause a message right on the length limit to exceed it once encoded (such that I need to build this in to the validation of the message creation form). Are new lines CR+LF or will just CR do?
Do any gateways have simulations? such that I could test my application without incurring any actual costs of SMS texts.
I plan to log the mobile numbers of any failed attempts and flag the user's record so that the next time they login I can encourage them to correct the number.
Any other gotchas and suggestions you have would be much appreciated. Thanks.
The one big gotcha, that has happened to me, although this is not a programming question as such, its just an experience of it, that needs to be taken into account.
When someone sends you a message that is of an urgent meaning, and then the sender (owner of the handset) comes over to you and asked you did you get the message, and this is what irks me, a row can arise because you can claim to not have received it, thus making the sender somewhat feeling miffed/cross/angry that you have "ignored them" or "did not respond" when in fact it is down to the provider's network problems.
And funnily enough, after about an hour after having a row, your handset goes beep, and your message from the sender arrived!!!! This has happened to me several times! But what can I do without ever knowing what is happening on the sender's side of things (crisis, urgent query etc)
So be cautious if sending the message across from ethernet (internet) to the handset as ethernet is not exactly 100% stable (routers go down, dns disappearing, server outages etc) so it is worthwhile to keep this in mind. Like the question is how can you guarantee instant delivery of sms? That's a big question and reliability will have to come in, usually requiring extra effort to do so...
Handsets, most of them usually do, have what is known as a delivery report, some are switched on, some are switched off, that is a useful way of knowing if the recipient's handset is switched on or off, if it's on, you will get an instant delivery report indicating that it was delivered, likewise, if it's off, there will be a noticeable time delay in getting the delivery report, that is dependant on your providers' maximum time for delivery.
Sorry if this seems like a rambling...
Hope this gives you some food for thought,
Best regards,
Tom.
SMS is a telco service, not an Internet service. That comes with some different rules.
For starters, many endpoints are billed/billable, and have contracts with a single service provider. This will include all of your use cases.
Secondly, billing is a contract matter both on the sending and the receiving side. You simply cannot state as a sender that "Host pays", unless you restrict yourself to sending SMS to specific countries. USA is the most famous exception. "Receiver pays" is even worse. Due to SMS spam, telco's will usually allow this kind of traffic only when you have a contract with them.
Third-party SMS operators can deal with many of these problems. It's very easy for them to be more service-oriented than the average telco. They might even be able to deliver international SMS for you.
SMS tends to buffer in the network itself, not necessarily the email gateway. In individual cases, the difference is probably invisible to you. But you would still have delays even if you had a direct SS7 link to the telco.
Real telco's have test gateways, but the terms of use for those I can't give you. The idea though is definitely to be able to test you app at lower costs.
SMS uses its own alphabet, a rather nasty multi-septet encoding (7/14/21 bits!) The quoted 160 character limit comes from a 140 byte payload. This could also be coded as 70 UTF-16 characters.
I understand most of the POP3 protocol, but one thing that bothers me is how POP3 clients efficiently get a list of unretrieved message ids from the POP3 server. Many services like Yahoo and Gmail now offer gigs of space and most people (myself included), rarely if ever delete an email message.
I'm currently implementing a simple POP3 client in C#, though the question that I'm asking should be language agnostic.
On the client side I store a list of ALL the retrieved message ids that I've ever retrieved. There is no need for this client to ever delete messages, mark messages ad having been read etc. All it needs to do is get the newest email messages since the last time that it connected to the POP3 server.
Am I right in assuming that the algorithm goes something like this:
Retrieve ALL message IDs from the POP3 server. This list grows daily and can become megabytes in size easily.
Compare this list of message IDs with the message IDs I have already retrieved (this list being stored client side) and identify what mail messages I actually have to retrieve from the server.
Retrieve the mail messages one at a time from the server using the results from step 2 above.
Is there any way to make this more efficient?
There's no really efficient way to do this, because POP was never designed for leaving messages in the mailbox permanently - it's designed for the use-case where you fetch your mail once and delete it.
A better approach to the whole problem would be to use IMAP instead, which is designed for efficient storage.