We are working on a system were we will need to send push notifications to thousands of phones. We have set this up using Amazon SNS. We will be sending individual messages to each phone so we are sending direct to SNS Endpoint ARNs rather that Topic ARNs.
We are currently wondering about the performance of this system. I've not been able to find anything on the net talking about how many messages can be sent to SNS. For example, if I need to send 25,000 messages to 25,000 SNS end points, how fast can I send them? Seconds, minutes, hours?
I know there is a lot of things that can impact this, but I'm trying to find some indicative numbers so we can work out how simple or complex we might need to make the software doing the sending and hopefully avoid over-engineering our solutions.
Thanks for any info.
Okies. Don't worry. I've run a test where I wrapped a loop around the send function and sent myself 25,000 push notifications. Had to shut my phone down, but the end result was around 13 sends per second or approximately 30 minutes to send 25,000. Not bad given the variables of network latency etc and enough to indicate to use the sorts of architectures we will need to add to the software.
Related
I set up a scheduling website at work which uses Twilio to send out sms messages with each person's next day assignment information as well as a link to the website where notes and everyone's next-day assignment is posted. Occasionally colleagues complain that they haven't received messages. Recently the problem has gotten much worse though is to be limited to people with AT&T and Sprint. People with Verizon and TMobile are get their messages without a problem.
A few more facts:
All of my code works well in testing and is consistent with Twilio's
instructions.
Twilio's logs list the messages as having been sent.
The people who's sms messages are not delivered tend to be at the end
of the alphabet
Anyone have any ideas what might be going on?
SMS messages the twilio logs show as sent which never arrive at the recipient's phone may be filtered by cell phone carriers as part of an effort to reduce spam sms messages. A website that sends out a batch of messages like the scheduling website you (I) describe may be particularly susceptible to such filtering since spam is also sent out in batches. The fact that recipients at the end of the alphabet tend to be blocked rather than those at the beginning might reflect an algorithm which flags earlier messages as suspicious and blocks subsequent ones that appear similar in that they contain the same website link.
In the past I was able to solve this problem by adding a 1 second delay between messages. That worked fine for my purposes. My site sent out the messages as a background job and with the 1 second delay about 25 sms messages get sent out over 2 minutes.
More recently AT&T and Sprint started blocking many more messages, though not all. At Jan 11, 15:04 PST, Twilio's status was aware of the problem and noted:
Identified - Messages with hyperlinks to goo.gl are being filtered on AT&T and Sprint. We are working with these carriers to address this filtering.
I was able to get around this problem by creating a shortened url using Bitly instead of Google's url shortening service.
Of note, Twilio is a great service but in the past their support folks were unaware that spam filtering could be blocking some of my sms messages. That's my prime motivation for posting about this issue here.
It seems that sites like mine can get caught in the crossfire between sms spammers and wireless carriers attempting to block them. Hopefully some day such sms spam filtering will get more sophisticated and stop blocking the messages sent by sites like mine.
I would love to hear if anyone with more sophisticated insight than mine has any comments.
Currently experiencing a really bad carrier filtering case through Twilio. Most messages sent to Verizon numbers are being blocked. According to Twilio support, Verizon recently implemented additional filtering methods to block A2P (Application-to-Peer) traffic.
Given this article: https://arstechnica.com/information-technology/2015/11/att-verizon-try-to-prevent-ban-on-text-message-blocking/ it seems like Verizon's more aggressive filtering might be related to the recent repeal of Net Neutrality, and their goal is to make more money by forcing businesses to get a short code.
If your using 10 digit phone numbers I Would also look at this
https://www.twilio.com/help/faq/short-codes/why-would-i-want-a-short-code-instead-of-sending-sms-from-a-regular-us-phone-number-or-phone-numbers
Long codes are meant for person-to-person communications, and can send only 1 message per second. For high-volume, application-driven messaging, Twilio recommends using a short code. Short codes can send SMS and MMS at 30 messages per second, and this high throughput is perfect for applications needing to send time-sensitive messages to many users at once. Furthermore, since carriers vet and approve all short codes for their intended use, they are not subject to carrier filtering or suspension for heavy traffic.
We just developed a system that integrates azure queue with an azure cloud service to process batch items. One requirement we had was to have items be set in the future to process. So for example, we batch it now, but tell it not to start for 5 hours.
This is built right into azure queues AddMessage using initialVisibilityDelay, so we did not see this as being an issue. However, we just noticed when we add auto scale on our Cloud Service, it is going off the total items in queue. In our situation we added 100,000 queue items to be sent 5 days from now, however it is scaling assuming these 100,000 are ready to go right now.
So in our situation, we would basically have dozens of instances of our app running until these messages can even send, 5 days from now.
I feel like there is something simple we are missing here.
Any feedback would be very helpful.
Anthony
Have you considered using one queue for the waiting messages and another queue for the actual messages to be processed and scaling on that latter queue?
Hypothetical: I want to send a single text message to all Verizon phones programmatically. I have multiple email gateways to use (obtained from the all-reliable wikipedia):
number#vtext.com
number#vzwpix.com
number#message.alltel.com
number#text.wireless.alltel.com
number#mms.alltel.net
I don't think that I'm guaranteed that any one of these will work and/or will still be in service (am I?) and I would not like to have to come back and change anything in the code at a later date.
Is there any way that I can make sure that I only send one text message to a given phone number when there are 5 possible gateways?
The only way way (that I know of) to ensure you only deliver one message to the recipient is to try each gateway sequentially until a message sends successfully, like some of the comments mentioned.
However, I've been sending a decent volume (>1500) of messages using #vtext.com lately and haven't noticed any bounces or downtime during sending. I have no way of knowing if every single message was delivered, but none of my test numbers have dropped a message yet. Most US carriers seem to have decent reliability on their gateways these days.
Just remember that SMS is still considered a best-effort service by most carriers. Even if you get your message to their servers successfully, there's no guarantee that the message will make it to its destination.
I am planning to inegrate messaging middleware in my web application. Right now I am tesing different messaging middleware software like RabbitMQ,JMS, HornetQ, etc..
Examples provided with this softwares are working but its not giving as desired results.
So, I want to know that which are the factors which are responsible to improve peformance that one should keep in eyes?
Which are the areas, a developer should take care of to improve the performance of middleware messaging software?
I'm the project lead for HornetQ but I will try to give you a generic answer that could be applied to any message system you choose.
A common question that I see is people asking why a single producer / single consumer won't give you the expected performance.
When you send a message, and are asking confirmation right away, you need to wait:
The message transfer from client to server
The message being persisted on the disk
The server acknowledging receipt of the message by sending a callback to the client
Similarly when you are receiving a message, you ACK to the server:
The ACK is sent from client to server
The ACK is persisted
The server sends back a callback saying that the callback was achieved
And if you need confirmation for all your message-sends and mesage-acks you need to wait these steps as you have a hardware involved on persisting the disk and sending bits on the network.
Message Systems will try to scale up with many producers and many consumers. That is if many are producing they should all use the resources available at the server shared for all the consumers.
There are ways to speed up a single producer or single consumer:
One is by using transactions. So, you minimize the blocks and syncs you perform on disk while persisting at the server and roundtrips on the network. (This is actually the same on any database)
Another one, is by using Callbacks instead of blocking at the consumer. (JMS 2 is proposing a Callback similar to the ConfirmationHandler on HornetQ).
Also: most providers I know will have a performance section on their docs with requirements and suggestions for that specific product. You should look individually at each product
My application sends notifications to the customers in bulks. For example at 8am every day a back-end systems generates notifications for 50K customers, and those notifications should be delivered in a reasonable time.
During performance testing I've discovered that sending a single push request to C2DM server takes about 400 millis which is far too long. I was told that a production quota may provide better performance, but will it reduce to 10 millis?
Besides, I need C2DM performance marks before going to production because it may affect the implementation - sending the requests from multiple threads, using asynchronous http client etc.
Does anyone knows about the C2DM server benchmarks or any performance-related server implementation guidelines?
Thanks,
Artem
I use appengine, which makes the delivery of a newsletter to 1 million users a very painful task. Mostly because the c2dm API doesn't support multiple users delivery, which makes it necessary to create one http request per user.
The time to the c2dm server to respond will depend on your latency from the google servers. In my case (appengine) it's very small.
My advice to you is to create as many threads as possible, since the threads will be waiting for the IO over the network. If you every pass the quota, you can always ask permission for more traffic.