i have below configuration
Carrier_1 -> My_SMPP(Jasmin) -> Carrier_2
i have working incoming message from Carrier_1
and configured MOROUTER as below:
#Order Type Connector ID(s) Filter(s)
#0 DefaultRoute smpps(Carrier_2)
and MTROUTER as below:
#Order Type Rate Connector ID(s) Filter(s)
#0 DefaultRoute 0 (!) smppc(Carrier_2)
and message are not send to Carrier_2.
In log file deliversm-thrower.log i receive below information:
ERROR 9858 Throwing SMPP/DELIVER_SM [msgid:11257448-bf10-475e-abfe-18fbf9274577] to (simple 1/1)[cid:Carrier_2], <class 'jasmin.routing.throwers.SystemIdNotBound'>: Carrier_2.
on Jasmin and Carrier_2 side connection is BOUND.
Please point where i make mistake and where i need look to solve this issue.
Best Regards,
Kamil
Related
I am using https://github.com/zendesk/racecar (which uses https://github.com/appsignal/rdkafka-ruby under the hood, a Ruby wrapper for https://github.com/edenhill/librdkafka/) to consume a Kafka stream and every now and then I receive this error Local: Unknown partition (unknown_partition)
"/app/vendor/bundle/ruby/2.6.0/gems/rdkafka-0.8.1/lib/rdkafka/consumer.rb:339:in `store_offset'",
"/app/vendor/bundle/ruby/2.6.0/gems/racecar-2.3.0/lib/racecar/consumer_set.rb:51:in `store_offset'"
This error seem to be raised when calling rd_kafka_offset_store, responsible to store offset of a message to be used in the next commit of this consumer.
Inspecting logs with config.log_level = "debug" looks like the error above always is preceded by:
rdkafka: [thrd:sasl_ssl://a1-abc-d1er1.eu-east-2.aws.confluent.cloud:9092/2]: sasl_ssl://a1-abc-d1er1.eu-east-2.aws.confluent.cloud:9092/2: Disconnected (after 3600185ms in state UP)
(try 1/10): Error for topic subscription #<struct Racecar::Consumer::Subscription topic="my.topic", start_from_beginning=true, max_bytes_per_partition=1048576, additional_config={}>: Local: Broker transport failure (transport)
First time using Kafka here and I don't have an idea what might be.
We are testing the capacity of a Mail relay based on RHEL 7.6.
We are observing issues when sending an important number of msgs (e.g.: ~1000 msgs in 60 seconds).
While we have sent all the msgs and the recipient has received all the msgs, logs are missing in the /var/log/maillog_rfc5424.
We have the following message in the /var/log/messages:
rsyslogd: imjournal: XYZ messages lost due to rate-limiting
We adapted the /etc/rsyslog.conf with the following settings but without effect:
$SystemLogRateLimitInterval 0 # turn off rate limit
$SystemLogRateLimitBurst 0 # turn rate limit off
Any ideas ?
The error is from imjournal, but your configuration settings are for imuxsock.
According to the rsyslog configuration page you need to set
$imjournalRatelimitInterval 0
$imjournalRatelimitBurst 0
Note that for very high message rates you might like to change to imuxsock, as it says:
this module may be notably slower than when using imuxsock. The journal provides imuxsock with a copy of all “classical” syslog messages, however, it does not provide structured data. Only if that structured data is needed, imjournal must be used. Otherwise, imjournal may simply be replaced by imuxsock, and we highly suggest doing so.
I am running 3 instances of a service that I wrote using:
Scala 2.11.12
kafkaStreams 1.1.0
kafkaStreamsScala 0.2.1 (by lightbend)
The service uses Kafka streams with the following topology (high level):
InputTopic
Parse to known Type
Clear messages that the parsing failed on
split every single message to 6 new messages
on each message run: map.groupByKey.reduce(with local store).toStream.to
Everything works as expected but i can't get rid of a WARN message that keeps showing:
15:46:00.065 [kafka-producer-network-thread | my_service_name-1ca232ff-5a9c-407c-a3a0-9f198c6d1fa4-StreamThread-1-0_0-producer] [WARN ] [o.a.k.c.p.i.Sender] - [Producer clientId=my_service_name-1ca232ff-5a9c-407c-a3a0-9f198c6d1fa4-StreamThread-1-0_0-producer, transactionalId=my_service_name-0_0] Got error produce response with correlation id 28 on topic-partition my_service_name-state_store_1-repartition-1, retrying (2 attempts left). Error: UNKNOWN_PRODUCER_ID
As you can see, I get those errors from the INTERNAL topics that Kafka stream manage. Seems like some kind of retention period on the producer metadata in the internal topics / some kind of a producer id reset.
Couldn't find anything regarding this issue, only a description of the error itself from here:
ERROR CODE RETRIABLE DESCRIPTION
UNKNOWN_PRODUCER_ID 59 False This exception is raised by the broker if it could not locate the producer metadata associated with the producerId in question. This could happen if, for instance, the producer's records were deleted because their retention time had elapsed. Once the last records of the producer id are removed, the producer's metadata is removed from the broker, and future appends by the producer will return this exception.
Hope you can help,
Thanks
Edit:
It seems that the WARN message does not pop up on version 1.0.1 of kafka streams.
Spring boot properties for kafka producer:
spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.client-id=bam
#spring.kafka.producer.acks= # Number of acknowledgments the producer requires the leader to have received before considering a request complete.
spring.kafka.producer.batch-size=0
spring.kafka.producer.bootstrap-servers=localhost:9092
#spring.kafka.producer.buffer-memory= # Total bytes of memory the producer can use to buffer records waiting to be sent to the server.
spring.kafka.producer.client-id=bam-producer
spring.kafka.consumer.auto-offset-reset=earliest
#spring.kafka.producer.compression-type= # Compression type for all data generated by the producer.
spring.kafka.producer.key-serializer= org.apache.kafka.common.serialization.StringSerializer
#spring.kafka.producer.retries= # When greater than zero, enables retrying of failed sends.
spring.kafka.producer.value-serializer= org.apache.kafka.common.serialization.StringSerializer
#spring.kafka.properties.*= # Additional properties used to configure the client.
I am getting below exception when i am trying to send message to a kafka topic :
Caused by: org.springframework.kafka.core.KafkaProducerException: Failed to send; nested exception is org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for bam-0 due to 30004 ms has passed since last append
at org.springframework.kafka.core.KafkaTemplate$1.onCompletion(KafkaTemplate.java:255)
at org.apache.kafka.clients.producer.internals.RecordBatch.done(RecordBatch.java:109)
at org.apache.kafka.clients.producer.internals.RecordBatch.maybeExpire(RecordBatch.java:160)
at org.apache.kafka.clients.producer.internals.RecordAccumulator.abortExpiredBatches(RecordAccumulator.java:245)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:212)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:135)
... 1 more
Caused by: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for bam-0 due to 30004 ms has passed since last append
I am not able to figure our why i am getting this exception. Can some one please help ?
The producer is timing out trying to send messages. I notice you are using localhost in your bootstrap. Make sure a broker is available locally and listening on port 9092.
Issue resolved by setting advertised.listeners in server.properties to PLAINTEXT://<ExternalIP>:9092.
Note: Kafka is deployed over aws.
What's the topic of connection threshold events? How do I listen to connection count threshold events over the message bus, and how do I figure out what is the current connection count?
Connection threshold events can be published over the message bus to the following topics:
#LOG/WARNING/VPN/<router-name>/VPN_VPN_CONNECTIONS_HIGH/<vpn-name> when the connection count exceeds the high threshold.
#LOG/INFO/VPN/<router-name>/VPN_VPN_CONNECTIONS_HIGH_CLEAR/<vpn-name> when the connection count goes below the clear threshold.
If desired, you can apply wildcards to the topics. For example, #LOG/*/VPN/<router-name>/VPN_VPN_CONNECTIONS*/<vpn-name>.
Note that you will need to fill in <router-name> and <vpn-name> with appropriate values.
In order to have the connection count threshold events published over the message bus, you will need to do the following:
a. Configure the VPN to "Publish Message VPN Event Messages".
b. Your application needs to subscribe to the topic for connection threshold events.
In order to figure out the current connection count, you will need to send a SEMP over message bus query.
a. Enable SEMP over Message Bus Show Commands on the VPN.
b. Send a SEMP over Message Bus query. There's an SempGetOverMB sample in the API with detailed instructions to do this. You can also refer to the documentation for details.
<rpc semp-version="soltr/7_2">
<show>
<message-vpn>
<vpn-name>default</vpn-name>
</message-vpn>
</show>
</rpc>
c. Parse the XML based response.
<rpc-reply semp-version="soltr/7_2">
<rpc>
<show>
<message-vpn>
<vpn>
<name>default</name>
<connections-service-smf>3</connections-service-smf>
<connections-service-web>0</connections-service-web>
<connections-service-rest-incoming>0</connections-service-rest-incoming>
<connections-service-mqtt>0</connections-service-mqtt>
<connections-service-rest-outgoing>0</connections-service-rest-outgoing>
<max-connections>10</max-connections>
<max-connections-service-smf>9000</max-connections-service-smf>
<max-connections-service-web>9000</max-connections-service-web>
<max-connections-service-rest-incoming>9000</max-connections-service-rest-incoming>
<max-connections-service-mqtt>9000</max-connections-service-mqtt>
<max-connections-service-rest-outgoing>6000</max-connections-service-rest-outgoing>
... Removed non-relevant portions for clarity ...
</vpn>
</message-vpn>
</show>
</rpc>
<execute-result code="ok"/>
</rpc-reply>
Note that there is a system limit of 10 SEMP poll requests per second, and some topics should not be polled. Refer to the documentation for details.