We have developed an SMPP server based on CloudHopper SMPP library. The system is in production processing a considerable volume of messages everyday.
Customers are allowed to maintain multiple TX binds for a higher throughput and redundancy. Recently we identified that a few of our customers send segmented messages (message parts) over multiple binds. So we receive parts of a single SMS over different binds from the same customer (same SystemId) and those are valid (in UDH, Having the same reference number, valid seq no etc).
We support concatenation on a bind level, but not across multiple binds.
Is it a standard/good practise to support concatenation for parts sent over multiple binds? Message reference number in the UDH header for concatenated messages should be unique for all binds created by an SMPP gateway? Any thoughts on this are welcome
UPDATE:
Regarding MessageRef no in UDH, we understood that there is no guarantee that it will be unique across multiple binds. (We see repeated MessageRef in concurrent binds from the same customer). Now we concatenate parts if they have the same MessageRef no and also the same destination number. We believe it is rare to receive two part sequences at the same time with equal MessageRef numbers, going towards the same mobile.
As long as a concatenated messages have valid UDH, I see not issue on this. SMSC further is usually transparent on this, altough it knows the multipart messages belong to single SMS message. From a practice side, customers usually have limitation on bind side just for 1 transmitter (on gigabit network should not be an issue), but this depends on number of messages/s they want to submit.
I usually have my own queue in between, so the submit response does not affect SMSC submit delay.
Related
Hello this is preety straightforward question, i've seen and read around that people write that the address can point to multiple actors. I'm wondering why? What would be the use case of such.
Let's say router actor can have one address but then, on message it can dispatch the messages to multiple actors but each of it's children still might have only one address.
Thank you
First, let's analyze your example: indeed, a message payload can be delivered to multiple actors, as you have described (i.e. via router), as well as via pub-sub pattern .
I think, pub-sub is better because of convenience as there is no need of additional actor (router) in-between. Another reason, is less coupling in the pub-sub case: to subscribe and publish the message, there is need to know the address only, meanwhile in the router case, the donwstream actors have to know also at least router address and "subscription protocol", (e.g. donwstream.send<subscribe_me, message_type>(router_address, donwstream.address) or the donwstream actors have to know router class instance and (in the worse case) they have to be children of the router (or to be owned by router) - e.g. (router.subscribe(downstream_actor)).
The last reason on the point which might matter and depends on the implementation, that is is not the same message is delivered: in the pub-sub model the original message is delivered for multiple actors, in the router case multiple clones of the original message are delivered.
Second, the multiple patterns with actor multi-addressing are possible. Here are a few examples:
the actor's behavior might be different, whether a message of the same type is delivered to one or other actor's address, e.g. if actor owns some finite resource, say memory, and when it is about out of the resource, it might answer with rejections on it's "main" address, and still provide the resource on "critical" address. Without multi-addressing, you have to either share the resource between 2 different actors (discouraged) or to have 2 different types of messages (better).
in rotor (disclaimer: I'm the author), one actor can subscribe to the other service-actor address for some silent-side effects, i.e. for auditing or logging messages which are coming for the service-actor.
the message supervising can be implemented because of multi-addressing, e.g. in rotor the request-response pattern is implemented the following way: the request is routed via supervisor, which spawns a timer and a new address(X), and sends a payload copy to the original destination; then the response arrives at (X), it is just forwarded to the requesting-actor and timer is cancelled; however if the timer triggers, the supervisor creates a new message with error code (timeout) and delivers it back to the requesting-actor.
something like NAT is possible, i.e. when 2 nodes are connected, and here is need to deliver a message from actorA from node1 to actorB node2, the message can be actually just delivered to the unique address of NAT-actor on node1 representing actorB, serialized it special way and send over network. The reverse procedure will happen on node2. In that case NAT-actor will have multiple addresses (like ports in real routers), and still will be transparent for actorA and actorB.
When sending a message, MassTransit wraps that payload with an envelope which has a field called destinationAddress. What purpose does this field have?
I found this because I have a number of C# microservices communicating with some node and java based services - so I've been using the minimum payload defined here:
http://masstransit-project.com/MassTransit/advanced/interoperability.html
I've had no problem integrating the two services together I was just wondering what the point was of having the destinationAddress as part of the message itself? Is it just a belts and braces kind of thing to make sure messages don't go on the wrong queue by mistake?
I would have thought that all of this information can be derived since it is literally just built up of a) the message bus host and b) the queue name used when actually sending the message?
Transports have a variety of ways to delivering messages. For instance, publishing a message to a topic would set the destination address to (URI of topic) but it may be delivered to a queue (via a subscription, forwarded by the transport) with a different address. In this case, the envelope has the original destinationAddress, whereas the queue would have a different address.
There are also cases where messages may be scheduled, redelivered, faulted, etc., and having that information helps in troubleshooting production systems in cases where the original destination may not be known otherwise.
So, yeah, in the simplest case it seems superfluous, however, it comes in useful down the road when trying to figure out why something doesn't work.
The JMS 2.0 specification says
The JMSMessageID header field contains a value that uniquely
identifies each message sent by a provider.
...and...
The exact scope of uniqueness is provider defined. It should at least
cover all messages for a specific installation of a provider where an
installation is some connected set of message routers.
The specification does not explicitly state that the JMSMessageID returned from the publish API call must match the one present in the message when it is consumed. The discussion in the spec about moving the JMSMessageID to the JMSCorrelationID when replying to a request implies that the two would be the same. If the message ID was changed between publication and consumption, this style of request/reply would fail.
Certainly in the unified domain model of JMS 1.1 and now 2.0, it would not make sense for the behavior of the JMSMessageID to change depending on whether the destination is a queue or a topic. Under the unified model, one would expect all destinations to act alike in this regard.
Also, if "provider" as used in the first paragraph refers to the thing that is sending messages, then a publication that fanned out to 10 identical messages, with identical JMSMessageID values, would meet the spec since uniqueness is measured at the sending side.
Unfortunately, the specification liberally switches between using the term "provider" to describe the thing sending messages versus using it to describe the vendor of the JMS transport. This is evident in the two quoted passages above. This ambiguity doesn't help matters any.
At least one implementation (IBM's MQ) takes the approach that a publication fanning out to 10 messages has created 10 unique, new messages, and therefore each of these has a unique JMSMessageID value. This is arguably consistent with the second quoted passage which requires uniqueness scoped to the provider, where "provider" appears to refer to the vendor implementation and not the thing sending messages.
It is my belief that when a published message fans out to multiple subscribers the correct behavior would be that the JMSMessageID would be preserved in each instance of the message so that replies can be correlated as expected. In other words, I believe IBM's implementation to be non-compliant. Since the specification is ambiguous on the matter, I'm looking for an authoritative source which either states outright or strongly implies the behavior as intended by the spec, one way or the other. Depending on the response, I'll either stand down, or else raise the issue with IBM as a compliance defect.
The term "provider" here is simply a reference to the specific messaging product being used, and covers both client-side and server-side components. To avoid confusion, I'll use the word JMS product vendor here.
The purpose of the JMS specification is to define a Java API implemented by that messaging product. It uses loose terms like "provider" because the JMS spec does not define how the product is architected and is trying to avoid suggesting how the implementation should be shared between client-side and server-side components, or even whether there is a server (or cluster of servers) at all. You'll notice the spec never (well, almost never) says "the server does this" or "the server does that".
The sentence about the "exact scope of uniqueness" is there to make it easy for the JMS product vendor to implement the code that generates JMSMessageID values. It's saying that the code that generates JMSMessageID values doesn't need to worry about ensuring that the values generated are unique across the entire universe. It's sufficient to ensure that they are unique to that particular product installation.
You say that "The specification does not explicitly state that the JMSMessageID returned from the publish API call must match the one present in the message when it is consumed."
I think this is stated in Section 4.4.11 "How message header values are set". This states that the JMSMessageID is set by the "JMS provider send method". The same section goes on to say that "Message header fields that are defined as being set by the 'JMS provider send method' will be available on the sending client as well as on the receiving client."
This means that after the call to send() or publish() has returned, the sending application can use the method getJMSMessageID() to find the message ID that was assigned to that message. When this message is received, the receiving application can use the same method, and get the same value.
Each message sent to a topic is delivered to every subscriber on that topic. These subscribers will receive a separate copy of the same message, with the same body, properties and headers, including JMSMessageID value.
Feel free to argue; the JMS spec is not free of ambiguities.
I think the issue here is less about when the JMSMessageID field is set on a published message, and more about what happens to that message when it is processed within the JMS provider.
As stated in T.Rob's and Nigel's posts, section 3.4.3 of the JMS 2.0 specification states:
"The JMSMessageID header field contains a value that uniquely
identifies each message sent by a provider."
and also:
"A JMSMessageID is a String value which should function as a unique
key for identifying messages in a historical repository. The exact
scope of uniqueness is provider defined. It should at least cover all
messages for a specific installation of a provider where an
installation is some connected set of message routers."
That is to say, two or more messages, even if they contain the same data, ought to have different JMSMessageID values if they constitute different messages within a repository.
The spec also states, in section 4.2.1 that,
"A topic can be thought of as a mini message broker that gathers and
distributes messages addressed to it. By relying on the topic as an
intermediary, message publishers are kept independent of subscribers
and vice versa."
This would imply that the intention of the spec is that, when a message is sent to a Topic, the Topic can do some work on the message, including creating multiple copies of the message (or, more specifically, creating multiple messages with the same data that are considered separate within the provider's repository.
Finally, section 4.2.2 states:
"A subscription will receive a copy of every message that is sent to
the topic after the subscription is created, ... Each copy of the
message is treated as a completely separate message. Work done on one
copy has no effect on any other; acknowledging one does not
acknowledge any other; one message may be delivered immediately, while
another waits for its consumer to process messages ahead of it."
Putting these passages together, the spec can be read as saying
When a message is sent to a Topic, that Topic can create a copy of the message for each current subscription.
The copies of the message created when sending to a Topic can be considered as completely separate messages.
Because separate JMS messages are uniquely identified by their JMSMessageID field, each separate subscription message should have a different JMSMessageID
To pick up Nigel's last sentence the JMS specification isn't free of ambiguities. This is very true and vendors and customers have previously worked around issues, and work in the expert group does take place to clarify these and provide guidance as well as make suggestions for improving the compliance tests. Based on the understanding outlined above, and the tests within the JMS 2.0 Compliance Test Suite that IBM MQ v8 passes, the IBM MQ v8 implementation is JMS2.0 compliant (and likewise earlier IBM MQ versions are JMS1.1 compliant; the JMS 1.1 specification has the same ambiguity).
The request-response paradigm is a common one, though with a pub-sub based distribution model the sending application does potentially have to cope with multiple responses not just the one that would be more likely with a point-point architecture. We acknowledge that there are messaging scenarios where the capability for a message id to have a different 'value of uniqueness' from the one currently implemented by IBM MQ would provide value to some IBM MQ customers
For the above reasons IBM strongly believes that its MQ JMS solution is compliant, so a PMR will not be accepted. However, we do acknowledge that there are a number of use cases where maintaining the message ID would be beneficial to you. For that reason we will make RFE 35062 an uncommitted candidate, which means it has the highest probability of being addressed and we promise that we're actively working to provide the solution that best fits the needs as quickly as possible. But to do this we'd appreciate additional feedback on the RFE with descriptions of what the actual problems our users are trying to solve here. For example is this for audit purposes, request-reply, message flows, etc, and what it is you need replicated? The more information we have, the more likely the solution is to satisfy the need.
The JMS 2.0 specification says
The JMSMessageID header field contains a value that uniquely
identifies each message sent by a provider.
...and...
The exact scope of uniqueness is provider defined. It should at least
cover all messages for a specific installation of a provider where an
installation is some connected set of message routers.
The specification does not explicitly state that the JMSMessageID returned from the publish API call must match the one present in the message when it is consumed. The discussion in the spec about moving the JMSMessageID to the JMSCorrelationID when replying to a request implies that the two would be the same. If the message ID was changed between publication and consumption, this style of request/reply would fail.
Certainly in the unified domain model of JMS 1.1 and now 2.0, it would not make sense for the behavior of the JMSMessageID to change depending on whether the destination is a queue or a topic. Under the unified model, one would expect all destinations to act alike in this regard.
Also, if "provider" as used in the first paragraph refers to the thing that is sending messages, then a publication that fanned out to 10 identical messages, with identical JMSMessageID values, would meet the spec since uniqueness is measured at the sending side.
Unfortunately, the specification liberally switches between using the term "provider" to describe the thing sending messages versus using it to describe the vendor of the JMS transport. This is evident in the two quoted passages above. This ambiguity doesn't help matters any.
At least one implementation (IBM's MQ) takes the approach that a publication fanning out to 10 messages has created 10 unique, new messages, and therefore each of these has a unique JMSMessageID value. This is arguably consistent with the second quoted passage which requires uniqueness scoped to the provider, where "provider" appears to refer to the vendor implementation and not the thing sending messages.
It is my belief that when a published message fans out to multiple subscribers the correct behavior would be that the JMSMessageID would be preserved in each instance of the message so that replies can be correlated as expected. In other words, I believe IBM's implementation to be non-compliant. Since the specification is ambiguous on the matter, I'm looking for an authoritative source which either states outright or strongly implies the behavior as intended by the spec, one way or the other. Depending on the response, I'll either stand down, or else raise the issue with IBM as a compliance defect.
The term "provider" here is simply a reference to the specific messaging product being used, and covers both client-side and server-side components. To avoid confusion, I'll use the word JMS product vendor here.
The purpose of the JMS specification is to define a Java API implemented by that messaging product. It uses loose terms like "provider" because the JMS spec does not define how the product is architected and is trying to avoid suggesting how the implementation should be shared between client-side and server-side components, or even whether there is a server (or cluster of servers) at all. You'll notice the spec never (well, almost never) says "the server does this" or "the server does that".
The sentence about the "exact scope of uniqueness" is there to make it easy for the JMS product vendor to implement the code that generates JMSMessageID values. It's saying that the code that generates JMSMessageID values doesn't need to worry about ensuring that the values generated are unique across the entire universe. It's sufficient to ensure that they are unique to that particular product installation.
You say that "The specification does not explicitly state that the JMSMessageID returned from the publish API call must match the one present in the message when it is consumed."
I think this is stated in Section 4.4.11 "How message header values are set". This states that the JMSMessageID is set by the "JMS provider send method". The same section goes on to say that "Message header fields that are defined as being set by the 'JMS provider send method' will be available on the sending client as well as on the receiving client."
This means that after the call to send() or publish() has returned, the sending application can use the method getJMSMessageID() to find the message ID that was assigned to that message. When this message is received, the receiving application can use the same method, and get the same value.
Each message sent to a topic is delivered to every subscriber on that topic. These subscribers will receive a separate copy of the same message, with the same body, properties and headers, including JMSMessageID value.
Feel free to argue; the JMS spec is not free of ambiguities.
I think the issue here is less about when the JMSMessageID field is set on a published message, and more about what happens to that message when it is processed within the JMS provider.
As stated in T.Rob's and Nigel's posts, section 3.4.3 of the JMS 2.0 specification states:
"The JMSMessageID header field contains a value that uniquely
identifies each message sent by a provider."
and also:
"A JMSMessageID is a String value which should function as a unique
key for identifying messages in a historical repository. The exact
scope of uniqueness is provider defined. It should at least cover all
messages for a specific installation of a provider where an
installation is some connected set of message routers."
That is to say, two or more messages, even if they contain the same data, ought to have different JMSMessageID values if they constitute different messages within a repository.
The spec also states, in section 4.2.1 that,
"A topic can be thought of as a mini message broker that gathers and
distributes messages addressed to it. By relying on the topic as an
intermediary, message publishers are kept independent of subscribers
and vice versa."
This would imply that the intention of the spec is that, when a message is sent to a Topic, the Topic can do some work on the message, including creating multiple copies of the message (or, more specifically, creating multiple messages with the same data that are considered separate within the provider's repository.
Finally, section 4.2.2 states:
"A subscription will receive a copy of every message that is sent to
the topic after the subscription is created, ... Each copy of the
message is treated as a completely separate message. Work done on one
copy has no effect on any other; acknowledging one does not
acknowledge any other; one message may be delivered immediately, while
another waits for its consumer to process messages ahead of it."
Putting these passages together, the spec can be read as saying
When a message is sent to a Topic, that Topic can create a copy of the message for each current subscription.
The copies of the message created when sending to a Topic can be considered as completely separate messages.
Because separate JMS messages are uniquely identified by their JMSMessageID field, each separate subscription message should have a different JMSMessageID
To pick up Nigel's last sentence the JMS specification isn't free of ambiguities. This is very true and vendors and customers have previously worked around issues, and work in the expert group does take place to clarify these and provide guidance as well as make suggestions for improving the compliance tests. Based on the understanding outlined above, and the tests within the JMS 2.0 Compliance Test Suite that IBM MQ v8 passes, the IBM MQ v8 implementation is JMS2.0 compliant (and likewise earlier IBM MQ versions are JMS1.1 compliant; the JMS 1.1 specification has the same ambiguity).
The request-response paradigm is a common one, though with a pub-sub based distribution model the sending application does potentially have to cope with multiple responses not just the one that would be more likely with a point-point architecture. We acknowledge that there are messaging scenarios where the capability for a message id to have a different 'value of uniqueness' from the one currently implemented by IBM MQ would provide value to some IBM MQ customers
For the above reasons IBM strongly believes that its MQ JMS solution is compliant, so a PMR will not be accepted. However, we do acknowledge that there are a number of use cases where maintaining the message ID would be beneficial to you. For that reason we will make RFE 35062 an uncommitted candidate, which means it has the highest probability of being addressed and we promise that we're actively working to provide the solution that best fits the needs as quickly as possible. But to do this we'd appreciate additional feedback on the RFE with descriptions of what the actual problems our users are trying to solve here. For example is this for audit purposes, request-reply, message flows, etc, and what it is you need replicated? The more information we have, the more likely the solution is to satisfy the need.
I have noticed that certain SMS messages that I receive from companies come with a 'sender name'. eg. Just today I received an SMS from a number I have never used before (not im my contacts), however the senders name showed up as 'Adobe'. I get this from other companies too. eg Facebook, Google & Banking.
Is it similar to how a email server works? (you tell the server who you 'are' before you send the message) Is this the case with a carrier's cell tower?
I guess I'm wondering what the service is called and how it works? (ie. can you send 'header info' with SMS messages or is the cell tower just spoofing the message's 'sender number' and replacing it with characters?)
(hopefully this is the right place to ask this question...)
The MAP protocol (the one used for sending SMS messages among others) allow specifying either a phone number or an alphanumeric number as the sender.
AFAIK this cannot be set from your phone where the sender number will be always your public phone number but SMS Centers can allow sending such messages on other interfaces like the ones used by banks and the companies mentioned by you (usually using the SMPP or UCP protocoll).
Please note that some Telcos do not allow this kind of sender address in messages originated elsewhere but sent to their customers (or they don't allow it for everybody). They use SMS spam filters/firewalls called Home Routers for this.
Mobile communication in GSM, UMTS and LTE is governed by 3GPP.
The TP-OA field in SMS-DELIVER TPDU in an incoming SMS typically contains the number of a sender.
The network fills the TP-OA field with usually an MSISDN.
Please see 3GPP TS 23.040 Figure C.10.
But in case of a company name, TP-OA can be made alphanumeric using the Type of Number Information Element as 7-bit default alphabet
I suggest you to read 3GPP TS 24.011 and 23.040 to get an idea of how SMSes work.
However, I must point out that since a sender does not send TP-OA, it can't be easily spoofed.