I am about to start building an SMS alerting function in my web application. The aim is to provide two services:
Host pays - e.g. to send an SMS alerting users to the cancellation of an event
User pays - e.g. to alert that an Email has been sent with, say, details of a new event (apparently a requirement from users when they are away from their EMail systems!)
I also figure that there will be other user-pays scenarios such as if they have secondary mobile phones that they want alerting as the Host will only want to pay for a single phone per user.
From my research:
I can use a 3rd party SMS Gateway provider. Cost is about GBP 0.05 per message. I can either send an EMail to 999999999#TheGatewayProvider.com or I can use an HTTP request with suitable parameters in the URL.
I can send an EMail to the user's network provider (I believe this is only available in the USA)
EDIT: There are variations on how the various providers will handle the From/Subject and Message, so presentation of actual message sent may be hard to predict.
I can set up my own Gateway (which I think is way beyond my ability, and may give grief at our data centre!)
It strikes me that:
EMail we send sometimes get delayed in my server's SMTP queue, let alone any downstream queues. Sending an email to the network provider's SMS gateway seems to often be treated as low priority.
Therefore HTTP to a 3rd party SMS Gateway provider should give me shortest latency (important for "This afternoons event is cancelled because of bad weather")
When I send SMS Text from my mobile occasionally they take days to arrive - I presume this is something we just have to live with?
Having said that we will also have low priority informational messages and sending these by cheapest route is attractive! so I am planning to allow users to enter an EMail address for such messages - the intention being that they will use the email address for their mobile phone company's SMS redirect service, or similar (i.e. the email address of a device, not an inbox).
I also wonder whether allowing the users to enter IM, Twitter or other such is likely to be well used and efficient in practice?
From the 3rd party gateways I have looked at it seems:
Some use higher grade networks than others, this may impact performance? (or is it just marketing hogwash?)
Some provide better feedback than others. I need to keep the arguments over billing - and exactly how many "credits" the client has used up - to a minimum; getting an answerback of "OK" / "Phone number does not exist" therefore strikes me as important. One provider I found creates a text log file daily that can be downloaded and which I could reconcile with my outgoing log.
I would appreciate your opinions and experiences on:
Users will enter their mobile numbers as they know them. Do I need to enforce +9912345... so that I get the country code too?
What happens if the mobile is foreign (I'm based in the UK) Does the recipient pay the international part? or does the gateway provider perhaps have local transmission services?
What do I need to do with non alpha characters? The UK Pound sign "£" and CR / LF spring to mind. If these are encoded might that cause a message right on the length limit to exceed it once encoded (such that I need to build this in to the validation of the message creation form). Are new lines CR+LF or will just CR do?
Do any gateways have simulations? such that I could test my application without incurring any actual costs of SMS texts.
I plan to log the mobile numbers of any failed attempts and flag the user's record so that the next time they login I can encourage them to correct the number.
Any other gotchas and suggestions you have would be much appreciated. Thanks.
The one big gotcha, that has happened to me, although this is not a programming question as such, its just an experience of it, that needs to be taken into account.
When someone sends you a message that is of an urgent meaning, and then the sender (owner of the handset) comes over to you and asked you did you get the message, and this is what irks me, a row can arise because you can claim to not have received it, thus making the sender somewhat feeling miffed/cross/angry that you have "ignored them" or "did not respond" when in fact it is down to the provider's network problems.
And funnily enough, after about an hour after having a row, your handset goes beep, and your message from the sender arrived!!!! This has happened to me several times! But what can I do without ever knowing what is happening on the sender's side of things (crisis, urgent query etc)
So be cautious if sending the message across from ethernet (internet) to the handset as ethernet is not exactly 100% stable (routers go down, dns disappearing, server outages etc) so it is worthwhile to keep this in mind. Like the question is how can you guarantee instant delivery of sms? That's a big question and reliability will have to come in, usually requiring extra effort to do so...
Handsets, most of them usually do, have what is known as a delivery report, some are switched on, some are switched off, that is a useful way of knowing if the recipient's handset is switched on or off, if it's on, you will get an instant delivery report indicating that it was delivered, likewise, if it's off, there will be a noticeable time delay in getting the delivery report, that is dependant on your providers' maximum time for delivery.
Sorry if this seems like a rambling...
Hope this gives you some food for thought,
Best regards,
Tom.
SMS is a telco service, not an Internet service. That comes with some different rules.
For starters, many endpoints are billed/billable, and have contracts with a single service provider. This will include all of your use cases.
Secondly, billing is a contract matter both on the sending and the receiving side. You simply cannot state as a sender that "Host pays", unless you restrict yourself to sending SMS to specific countries. USA is the most famous exception. "Receiver pays" is even worse. Due to SMS spam, telco's will usually allow this kind of traffic only when you have a contract with them.
Third-party SMS operators can deal with many of these problems. It's very easy for them to be more service-oriented than the average telco. They might even be able to deliver international SMS for you.
SMS tends to buffer in the network itself, not necessarily the email gateway. In individual cases, the difference is probably invisible to you. But you would still have delays even if you had a direct SS7 link to the telco.
Real telco's have test gateways, but the terms of use for those I can't give you. The idea though is definitely to be able to test you app at lower costs.
SMS uses its own alphabet, a rather nasty multi-septet encoding (7/14/21 bits!) The quoted 160 character limit comes from a 140 byte payload. This could also be coded as 70 UTF-16 characters.
Related
We're having this weird problem with Twilio SMS messages hanging with status.
We have tried sending from different Twilio phone numbers to make sure it isn't a problem with that particular number being blocked and none of them go through.
Our system uses SMS messages in the standard form of two-step authentication with a code and a short message to the user inside the SMS body.
The carrier that the message sent is failing is Tune Talk (A Malaysian one).
The error in the Twilio Logs/Console I see is:
Status: Undelivered
Error: (Error: 30008) Unknown error. None
Message SID if it's in any way useful is: SM1024a2d519cf4f6bbfcbc838587cb2af
Any insight on why this is happening would be greatly welcomed.
We had also the same issue with phones that used to receive messages.
The problem is carrier blocking/filtering.
Every carrier uses different filters.
Some carriers block messages with 90% the same content, others use rate filtering (1 message per second or more) others use a combination. The blocking is not forever thought.
Twilio gives the following possible solutions:
Check that the phone you were sending to is turned on and can receive SMS
Ensure that the phone is not roaming off network. We cannot
guarantee message delivery on roaming phones.
Try sending to other phones who have the same mobile carrier.
If messages to other phones go through, the issue is likely device
related. Try rebooting the device or contact the mobile carrier for
help.
If you are sending SMS from an alphanumeric sender ID, see if using
a Twilio phone number works better. We’ve observed that certain
networks may block alpha sender IDs.
Try sending a shorter message to the phone, with simple content that
does not include any special characters. This would give our support
team an idea as to whether the failure is related to concatenation
or character encoding.
Twilio support can help investigate what went
wrong with our carriers. Please open a support request and include a
minimum of 3 or more message SIDs where a 30008 error was thrown.
Per our carriers' requirements, these SIDs can be no older than 48
hours at most.
Check -> Error 30008
Another solution is to use a 5digit code phone number.
Boris, error 30008 is certainly less descriptive than one would hope. In this case, it would be best to send that Message Sid along to support where we can dig a little deeper into the specifics.
Though it doesn't sound like it in this case, if there were a problem with your code, you could check out a production ready account verification tutorial here.
Came here because of exactly the same problem. I have someone who successfully received SMSes just 12 days ago, using the same Australian number, getting a 30008 for every attempt to send to them today. That's a really average-quality error message right there.
The user states that they ported the number from Telstra to Vodafone, but that was 3 months ago. I'm guessing that the forwarding is broken:
http://www.commsalliance.com.au/__data/assets/pdf_file/0013/2326/G565_2009.pdf
In particular:
1.4.4 Where internationally originated SMS is supported donor routing
must be supported wherever bilateral agreement exists for the
national leg, as international networks are not likely to access an
Australian mobile number portability database prior to routing the
message. However, certain limitations apply – see Appendix A.
Since Twilio aren't sending from an Australian number, they probably aren't looking through the number portability database. This would be my suspect cause for any failure to SMS route to a country with number portability.
I've had some of these errors when doing MMS's.
If I look at the detail in the Twilio console, there is additional detail and a secondary error message "12300 Invalid Content-Type. Attempt to retrieve MediaUrl returned an unsupported Content-Type."
I was putting images up on S3, but not setting the Content-Type of the image when I put it on S3.
In my case, the problem was that I was including a website URL in the message.
Once I removed it (replaced it with a simple name) it worked.
I'm not sure whether the issue was the URL itself (e.g. the network considers it spam) or the fact that it contained characters like /.
I set up a scheduling website at work which uses Twilio to send out sms messages with each person's next day assignment information as well as a link to the website where notes and everyone's next-day assignment is posted. Occasionally colleagues complain that they haven't received messages. Recently the problem has gotten much worse though is to be limited to people with AT&T and Sprint. People with Verizon and TMobile are get their messages without a problem.
A few more facts:
All of my code works well in testing and is consistent with Twilio's
instructions.
Twilio's logs list the messages as having been sent.
The people who's sms messages are not delivered tend to be at the end
of the alphabet
Anyone have any ideas what might be going on?
SMS messages the twilio logs show as sent which never arrive at the recipient's phone may be filtered by cell phone carriers as part of an effort to reduce spam sms messages. A website that sends out a batch of messages like the scheduling website you (I) describe may be particularly susceptible to such filtering since spam is also sent out in batches. The fact that recipients at the end of the alphabet tend to be blocked rather than those at the beginning might reflect an algorithm which flags earlier messages as suspicious and blocks subsequent ones that appear similar in that they contain the same website link.
In the past I was able to solve this problem by adding a 1 second delay between messages. That worked fine for my purposes. My site sent out the messages as a background job and with the 1 second delay about 25 sms messages get sent out over 2 minutes.
More recently AT&T and Sprint started blocking many more messages, though not all. At Jan 11, 15:04 PST, Twilio's status was aware of the problem and noted:
Identified - Messages with hyperlinks to goo.gl are being filtered on AT&T and Sprint. We are working with these carriers to address this filtering.
I was able to get around this problem by creating a shortened url using Bitly instead of Google's url shortening service.
Of note, Twilio is a great service but in the past their support folks were unaware that spam filtering could be blocking some of my sms messages. That's my prime motivation for posting about this issue here.
It seems that sites like mine can get caught in the crossfire between sms spammers and wireless carriers attempting to block them. Hopefully some day such sms spam filtering will get more sophisticated and stop blocking the messages sent by sites like mine.
I would love to hear if anyone with more sophisticated insight than mine has any comments.
Currently experiencing a really bad carrier filtering case through Twilio. Most messages sent to Verizon numbers are being blocked. According to Twilio support, Verizon recently implemented additional filtering methods to block A2P (Application-to-Peer) traffic.
Given this article: https://arstechnica.com/information-technology/2015/11/att-verizon-try-to-prevent-ban-on-text-message-blocking/ it seems like Verizon's more aggressive filtering might be related to the recent repeal of Net Neutrality, and their goal is to make more money by forcing businesses to get a short code.
If your using 10 digit phone numbers I Would also look at this
https://www.twilio.com/help/faq/short-codes/why-would-i-want-a-short-code-instead-of-sending-sms-from-a-regular-us-phone-number-or-phone-numbers
Long codes are meant for person-to-person communications, and can send only 1 message per second. For high-volume, application-driven messaging, Twilio recommends using a short code. Short codes can send SMS and MMS at 30 messages per second, and this high throughput is perfect for applications needing to send time-sensitive messages to many users at once. Furthermore, since carriers vet and approve all short codes for their intended use, they are not subject to carrier filtering or suspension for heavy traffic.
Is there anyway to get the same "MessageId" you can get in Exchange EWS when using ActiveSync?
I thought this was an Exchange way to identify each message uniquely, but I can't seem to find a way to retrieve it using ActiveSync.
EDIT: I've got 2 applications, one that stores info using ActiveSync, and one that stores info using EWS, and I want them to be able to work separately on the same message.... To do this, I was hoping to use the EWS MessageId, which seems to be a GUID type identifier for each individual message. (Note: This doesn't appear to be the same Message-ID as is found in email headers).
Sadly, you're mostly out of luck.
ActiveSync is not an integration protocol, it's a mobile synchronization protocol designed for low-bandwidth communication devices like smart phones. A lot of capabilities in EWS will not exist in EAS.
Long-term message identification and correlation isn't as important for mobile devices. They simply get told what messages are in each folder, and allow the user to manipulate them. At any time the Exchange server may tell its EAS-connected clients to "re-sync" which causes them to forget the messages they have on the device and pull them cleanly from the server. That happens a lot with EAS, sometimes a couple of times an hour, depending on what is happening with that mailbox. For example, deleting a folder via Outlook causes a FolderSync to happen, and that forces connected devices to cleanly re-sync again.
Therefore EAS appears to have left behind the notion of GUIDs or other long term IDs for messages. Instead, the server will assign ephemeral IDs that are valid only until the next big resync is forced (which could happen at any time). You'll probably see Exchange give very simple IDs like 7:45 (which means message ID 45 within folder 7, IIRC). However after a resync that might have the number 7:32 (if the user deletes other messages in that folder) or something like 4:22 (if the message gets moved to another folder entirely).
Other EAS servers like Zimbra, Kerio or Notes Traveler might assign GUIDs, but from memory this is how Exchange behaves. Your only option might be to put a hidden correlation ID of your own into the body or subject of messages you're interested in. That will allow you to track the lifecycle of the items you're interested in, at the expense of some odd stuff being visible to users in their message contents.
#Brian is correct - There are no global unique identifiers for ActiveSync items that can be used to correlate with EWS (With some exceptions, for instance a meeting invite has a UID, as do Events which can be used with some hackery to retrieve an EWS ID for the related EWS calendar event) and there are no fields that aren't visible to the user that can be hijacked for adding your own data with which to correlate. This is most apparent in email, contacts, tasks, notes etc...
However if you are syncing both, it is possible to use the meta data in the objects to match. For instance, for contacts write a hashing algorithm that combines the data from the first name, last name, company name, etc... fields and produces a result. This can be run on the data from both sides and will have relatively little object collision for matching (and those that do collide will have exactly the same visible data to the user anyway so in most cases it won't matter that you didn't get an exact alignment)
Hypothetical: I want to send a single text message to all Verizon phones programmatically. I have multiple email gateways to use (obtained from the all-reliable wikipedia):
number#vtext.com
number#vzwpix.com
number#message.alltel.com
number#text.wireless.alltel.com
number#mms.alltel.net
I don't think that I'm guaranteed that any one of these will work and/or will still be in service (am I?) and I would not like to have to come back and change anything in the code at a later date.
Is there any way that I can make sure that I only send one text message to a given phone number when there are 5 possible gateways?
The only way way (that I know of) to ensure you only deliver one message to the recipient is to try each gateway sequentially until a message sends successfully, like some of the comments mentioned.
However, I've been sending a decent volume (>1500) of messages using #vtext.com lately and haven't noticed any bounces or downtime during sending. I have no way of knowing if every single message was delivered, but none of my test numbers have dropped a message yet. Most US carriers seem to have decent reliability on their gateways these days.
Just remember that SMS is still considered a best-effort service by most carriers. Even if you get your message to their servers successfully, there's no guarantee that the message will make it to its destination.
I'm working on an application that sends and receives SMS messages to and from its users. (Don't worry - it's not spam - every user of the app expects to send and receive these messages).
One key aspect of the app... If a user sends a message to the app, the app then sends that message out to every person on that user's "team". So, the app will be sending a receiving a pretty significant number of messages (I'm hoping for a few thousand users, and 5,000-10,000 SMS messages per day).
I've been experimenting with a number of options:
SMS to email
Connecting a mobile phone to my server
Contracting an SMS gateway
Option 1 is great, since it's free, but it's unreliable (apparently mobile providers queue these messages after SMS messages they can charge for, so they're frequently received late or lost)
Option 2 is also cheap, but the mobile phone can't keep up with the number of messages I'll be sending. Also, the mobile phone provider will consider this volume of messages excessive.
Option 3 is perfect, except that SMS gateway providers charge PER MESSAGE (usually $0.02-$0.06 per), which creates an impossible scaling problem. (Reminds me of the old business adage... "Sure we're losing money on every transaction, but we'll make it up in volume...")
So, long story short - how on EARTH did Twitter pull this off? They've been doing a similar thing (allowing users to exchange SMS messages with the app) since the beginning. Even if they negotiated an INCREDIBLE discount (say, $0.001 per message), they'd be paying an ENORMOUS cost to send the hundreds of millions of messages they handle.
Does anyone have any idea how they did this?
After a bit of Googling, it looks like Twitter has simply been signing deals with cell phone companies in various countries. For example (Twitter blog post link spam incoming):
Australia: http://blog.twitter.com/2009/12/sms-tweets-for-telstra-australia.html
Indonesia: http://blog.twitter.com/2009/11/sms-for-axis-indonesia.html
New Zealand: http://blog.twitter.com/2009/05/hello-new-zealand.html
UK: http://blog.twitter.com/2009/03/full-sms-service-for-vodafone-uk.html
Some more details about SMS in general here.