In ActiveMQ, Can I anyway detect how much time a message took to deliver to subscriber from a publisher?
Or Can I see separate timings that message took go from Publisher to topic(JMS service) And from (JMS service)Topic to Subsriber?
When JMS message arrives at the consumer, you can look into the JMSTimestamp header property. According to the Javadoc:
The JMSTimestamp header field contains the time a message was handed
off to be sent.
Then you can simply subtract JMSTimestamp from current system time and calculate the time taken in flight by that JMS message.
There is a Maven plugin for AMQ performance testing, it might be what you are searching for.
Otherwise, you could come very far by doing some measurements with some load testing tool such as Apache jMeter and just see how many messages you can read/write / sec.
Notice that you will have rather big differences if you go persistent and/or transactional or not.
Update:
You could maybe do something with the log by configuring conf/log4j.properties and setting:
log4j.rootLogger=TRACE, logfile, console
Then in data/activemq.log you can get some information for a put message. It's not really obvious what the numbers represent, so you should make sure to define a test scenario. The following time stamps was from a message put to a queue (myqueue) by HermesJMS to ActiveMQ:
2012-06-06 16:31:19,979 | DEBUG | localhost adding consumer: ID:GOTL19946-53234-
1338993070006-0:2:3:1 for destination: queue://myqueue |org.apache.activemq.broker.region.AbstractRegion | ActiveMQ Transport: tcp:///127.0.0.1:53236
2012-06-06 16:31:19,989 | DEBUG | queue://myqueue add sub: QueueBrowserSubscription: consumer=ID:GOTL19946-53234-1338993070006-0:2:3:1, destinations=0, dispatched=0, delivered=0, pending=0, dequeues: 0, dispatched: 0, inflight: 0 | org.apache.activemq.broker.region.Queue | ActiveMQ Transport: tcp:///127.0.0.1:53236
2012-06-06 16:31:19,989 | DEBUG | myqueue toPageIn: 1, Inflight: 0, pagedInMessages.size 0, enqueueCount: 1, dequeueCount: 0 | org.apache.activemq.broker.region.Queue | Queue:myqueue
2012-06-06 16:31:19,989 | TRACE | QueueStorePrefetch513024249 - fillBatch | org.apache.activemq.broker.region.cursors.AbstractStoreCursor | Queue:myqueue
2012-06-06 16:31:19,989 | DEBUG | dispatch to browser: QueueBrowserSubscription: consumer=ID:GOTL19946-53234-1338993070006-0:2:3:1, destinations=1, dispatched=0, delivered=0, pending=0, already dispatched/paged count: 1 | org.apache.activemq.broker.region.Queue | Queue:myqueue
2012-06-06 16:31:19,999 | TRACE | ID:GOTL19946-53234-1338993070006-0:2:3:1 dispatched: ID:GOTL19946-53234-1338993070006-0:2:2:1:1 - queue://myqueue, dispatched: 1, inflight: 1 | org.apache.activemq.broker.region.PrefetchSubscription | ActiveMQ Connection Dispatcher: /127.0.0.1:53236
2012-06-06 16:31:19,999 | TRACE | ack:MessageAck {commandId = 13, responseRequired = false, ackType = 2, consumerId = ID:GOTL19946-53234-1338993070006-0:2:3:1, firstMessageId = ID:GOTL19946-53234-1338993070006-0:2:2:1:1, lastMessageId = ID:GOTL19946-53234- 1338993070006-0:2:2:1:1, destination = queue://myqueue, transactionId = TX:ID:GOTL19946- 53234-1338993070006-0:2:2, messageCount = 1, poisonCause = null} | org.apache.activemq.broker.region.PrefetchSubscription | ActiveMQ Transport: tcp:///127.0.0.1:53236
2012-06-06 16:31:19,999 | DEBUG | commit: TX:ID:GOTL19946-53234-1338993070006-0:2:2 syncCount: 1 | org.apache.activemq.transaction.LocalTransaction | ActiveMQ Transport: tcp:///127.0.0.1:53236
2012-06-06 16:31:19,999 | DEBUG | myqueue toPageIn: 0, Inflight: 1, pagedInMessages.size 1, enqueueCount: 1, dequeueCount: 0 | org.apache.activemq.broker.region.Queue | Queue:myqueue
2012-06-06 16:31:20,049 | DEBUG | localhost removing consumer: ID:GOTL19946-53234-1338993070006-0:2:3:1 for destination: queue://myqueue | org.apache.activemq.broker.region.AbstractRegion | ActiveMQ Transport: tcp:///127.0.0.1:53236
2012-06-06 16:31:20,049 | DEBUG | queue://myqueue remove sub: QueueBrowserSubscription: consumer=ID:GOTL19946-53234-1338993070006-0:2:3:1, destinations=1, dispatched=0, delivered=1, pending=0, lastDeliveredSeqId: 67, dequeues: 0, dispatched: 1, inflight: 0 | org.apache.activemq.broker.region.Queue | ActiveMQ Transport: tcp:///127.0.0.1:53236
2012-06-06 16:31:20,049 | DEBUG | myqueue toPageIn: 0, Inflight: 0, pagedInMessages.size 1, enqueueCount: 1, dequeueCount: 0 | org.apache.activemq.broker.region.Queue | Queue:myqueue
Related
I'm getting a deploy-time permissions error related to a Secret referenced with Secret.from_secret_attributes.
# Both param 2 and secret_complete_arn need to populate from env vars.
secret = Secret.from_secret_attributes(self, "AmazonMSK_AdminSecret", secret_complete_arn="somearn" )
# MSK, the one, the only.
# Defaults to the sandbox_ideas2IT MKS instance Sampath set up while prototyping.
mks = os.getenv('MKS_ARN', 'somearn')
# Lambda layer.
lambdaLayer = lambda_.LayerVersion(self, 'lambda-layer',
code = lambda_.AssetCode('utils/lambda-deployment-packages/lambda-layer.zip'),
compatible_runtimes = [lambda_.Runtime.PYTHON_3_7],
)
# Source for the lambda.
with open("src/Blah/BlahIngest.py", encoding="utf8") as fp:
Blah_code = fp.read()
# Config for it.
lambdaFn = lambda_.Function(
self, "QFDS",
code=lambda_.InlineCode(Blah_code),
# role=QFDSrole,
handler="lambda_handler",
timeout=Duration.seconds(300),
runtime=lambda_.Runtime.PYTHON_3_7,
layers=[lambdaLayer],
)
# Set up the event (managed Kafka).
lambdaFn.add_event_source(ManagedKafkaEventSource(
cluster_arn=mks,
topic="foreign.blah.availabilty.notify",
secret=secret,
batch_size=100, # default
starting_position=lambda_.StartingPosition.TRIM_HORIZON
))
This fails to deploy with errors similar to:
Resource handler returned message: "Invalid request provided: Cannot access secret manager value arn:aws:secretsmanager:arn_stuff. Please ensure the role can perform the 'secretsmanager:GetSecretValue' action on your broker in IAM.
However, I am unsure what role would be needed for this. CDK seems to be creating several roles all on its own to perform tasks and to set up a role for the stack itself. I've attempted manually creating a role and trying to assign permissions to that, but I get no end of weird error messages with that, and I suspect that it's some misunderstanding on my part. If the secret is being included into the lambda event source by CDK, is it the CDK role itself that needs permission to access this secret?
Using fedonev's answer seems to get farther than I used to, but I still receive similar errors:
Do you wish to deploy these changes (y/n)? y
ZZZ: deploying... [1/1]
[10:14:11] Retrieved account ID 4### from disk cache
[10:14:11] Assuming role 'some_arn'.
[10:14:11] Waiting for stack CDKToolkit to finish creating or updating...
[0%] start: Publishing 55ce025aa43e61714e9a9eed823a61b7008f6994a8aee67cd1bed9689e2342ad:current_account-current_region
[0%] start: Publishing 8cb1c33ae6d9c870bb9fdd5c36a8d8fa9655d03004f85a79994a33c41690efce:current_account-current_region
[10:14:12] Retrieved account ID 4### from disk cache
[10:14:12] Assuming role 'some_arn'.
[10:14:12] Retrieved account ID 4### from disk cache
[10:14:12] Assuming role 'some_arn'.
[10:14:12] [0%] check: Check s3://cdk-hnb659fds-assets-445841279194-us-east-1/8cb1c33ae6d9c870bb9fdd5c36a8d8fa9655d03004f85a79994a33c41690efce.json
[10:14:12] [0%] check: Check s3://cdk-hnb659fds-assets-445841279194-us-east-1/55ce025aa43e61714e9a9eed823a61b7008f6994a8aee67cd1bed9689e2342ad.zip
[10:14:13] [0%] found: Found s3://cdk-hnb659fds-assets-445841279194-us-east-1/55ce025aa43e61714e9a9eed823a61b7008f6994a8aee67cd1bed9689e2342ad.zip
[50%] success: Published 55ce025aa43e61714e9a9eed823a61b7008f6994a8aee67cd1bed9689e2342ad:current_account-current_region
[10:14:13] [50%] found: Found s3://cdk-hnb659fds-assets-445841279194-us-east-1/8cb1c33ae6d9c870bb9fdd5c36a8d8fa9655d03004f85a79994a33c41690efce.json
[100%] success: Published 8cb1c33ae6d9c870bb9fdd5c36a8d8fa9655d03004f85a79994a33c41690efce:current_account-current_region
[10:14:13] Found existing stack ZZZ that had previously failed creation. Deleting it before attempting to re-create it.
[10:14:14] Waiting for stack ZZZ to finish creating or updating...
[10:14:14] Stack ZZZ has an ongoing operation in progress and is not stable (DELETE_IN_PROGRESS (User Initiated))
[10:14:20] Call failed: describeStacks({"StackName":"ZZZ"}) => Stack with id ZZZ does not exist (code=ValidationError)
[10:14:20] Stack ZZZ does not exist
[10:14:20] ZZZ: checking if we can skip deploy
[10:14:20] ZZZ: no existing stack
[10:14:20] ZZZ: deploying...
[10:14:20] Attempting to create ChangeSet with name cdk-deploy-change-set to create stack ZZZ
ZZZ: creating CloudFormation changeset...
[10:14:21] Initiated creation of changeset: some_arn; waiting for it to finish creating...
[10:14:21] Waiting for changeset cdk-deploy-change-set on stack ZZZ to finish creating...
[10:14:21] Changeset cdk-deploy-change-set on stack ZZZ is still creating
[10:14:26] Initiating execution of changeset some_arn on stack ZZZ
[10:14:27] Execution of changeset some_arn on stack ZZZ has started; waiting for the update to complete...
[10:14:27] Waiting for stack ZZZ to finish creating or updating...
[10:14:27] Stack ZZZ has an ongoing operation in progress and is not stable (CREATE_IN_PROGRESS (User Initiated))
ZZZ | 0/7 | 10:14:20 AM | REVIEW_IN_PROGRESS | AWS::CloudFormation::Stack | ZZZ User Initiated
ZZZ | 0/7 | 10:14:27 AM | CREATE_IN_PROGRESS | AWS::CloudFormation::Stack | ZZZ User Initiated
ZZZ | 0/7 | 10:14:32 AM | CREATE_IN_PROGRESS | AWS::Lambda::LayerVersion | lambda-layer (lambdalayerAEB0E7A7)
ZZZ | 0/7 | 10:14:32 AM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata)
[10:14:33] Stack ZZZ has an ongoing operation in progress and is not stable (CREATE_IN_PROGRESS)
ZZZ | 0/7 | 10:14:33 AM | CREATE_IN_PROGRESS | AWS::IAM::Role | QFDS/ServiceRole (QFDSServiceRole9EF8ABD9)
ZZZ | 0/7 | 10:14:33 AM | CREATE_IN_PROGRESS | AWS::IAM::Role | QFDS/ServiceRole (QFDSServiceRole9EF8ABD9) Resource creation Initiated
ZZZ | 0/7 | 10:14:35 AM | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) Resource creation Initiated
ZZZ | 1/7 | 10:14:35 AM | CREATE_COMPLETE | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata)
[10:14:38] Stack ZZZ has an ongoing operation in progress and is not stable (CREATE_IN_PROGRESS)
ZZZ | 1/7 | 10:14:40 AM | CREATE_IN_PROGRESS | AWS::Lambda::LayerVersion | lambda-layer (lambdalayerAEB0E7A7) Resource creation Initiated
ZZZ | 2/7 | 10:14:40 AM | CREATE_COMPLETE | AWS::Lambda::LayerVersion | lambda-layer (lambdalayerAEB0E7A7)
[10:14:44] Stack ZZZ has an ongoing operation in progress and is not stable (CREATE_IN_PROGRESS)
ZZZ | 3/7 | 10:14:47 AM | CREATE_COMPLETE | AWS::IAM::Role | QFDS/ServiceRole (QFDSServiceRole9EF8ABD9)
[10:14:49] Stack ZZZ has an ongoing operation in progress and is not stable (CREATE_IN_PROGRESS)
ZZZ | 3/7 | 10:14:50 AM | CREATE_IN_PROGRESS | AWS::IAM::Policy | QFDS/ServiceRole/DefaultPolicy (QFDSServiceRoleDefaultPolicyDDA95578)
ZZZ | 3/7 | 10:14:51 AM | CREATE_IN_PROGRESS | AWS::IAM::Policy | QFDS/ServiceRole/DefaultPolicy (QFDSServiceRoleDefaultPolicyDDA95578) Resource creation Initiated
[10:14:54] Stack ZZZ has an ongoing operation in progress and is not stable (CREATE_IN_PROGRESS)
[10:15:00] Stack ZZZ has an ongoing operation in progress and is not stable (CREATE_IN_PROGRESS)
ZZZ | 4/7 | 10:15:04 AM | CREATE_COMPLETE | AWS::IAM::Policy | QFDS/ServiceRole/DefaultPolicy (QFDSServiceRoleDefaultPolicyDDA95578)
[10:15:05] Stack ZZZ has an ongoing operation in progress and is not stable (CREATE_IN_PROGRESS)
ZZZ | 4/7 | 10:15:08 AM | CREATE_IN_PROGRESS | AWS::Lambda::Function | QFDS (QFDSA3E8E14E)
[10:15:10] Stack ZZZ has an ongoing operation in progress and is not stable (CREATE_IN_PROGRESS)
ZZZ | 4/7 | 10:15:15 AM | CREATE_IN_PROGRESS | AWS::Lambda::Function | QFDS (QFDSA3E8E14E) Resource creation Initiated
[10:15:16] Stack ZZZ has an ongoing operation in progress and is not stable (CREATE_IN_PROGRESS)
[10:15:21] Stack ZZZ has an ongoing operation in progress and is not stable (CREATE_IN_PROGRESS)
ZZZ | 5/7 | 10:15:21 AM | CREATE_COMPLETE | AWS::Lambda::Function | QFDS (QFDSA3E8E14E)
ZZZ | 5/7 | 10:15:23 AM | CREATE_IN_PROGRESS | AWS::Lambda::EventSourceMapping | QFDS/KafkaEventSource:ZZZQFDSD3CBC52Eforeign.engrain.availabilty.notify (QFDSKafkaEventSourceZZZQFDSD3CBC52EforeignengrainavailabiltynotifyEFE73996)
ZZZ | 5/7 | 10:15:26 AM | CREATE_FAILED | AWS::Lambda::EventSourceMapping | QFDS/KafkaEventSource:ZZZQFDSD3CBC52Eforeign.engrain.availabilty.notify (QFDSKafkaEventSourceZZZQFDSD3CBC52EforeignengrainavailabiltynotifyEFE73996) Resource handler returned message: "Invalid request provided: Cannot access secret manager value some_arn. Please ensure the role can perform the 'secretsmanager:GetSecretValue' action on your broker in IAM. (Service: Lambda, Status Code: 400, Request ID: 54dd811e-ede9-4de2-aace-839344aab72f)" (RequestToken: 7c072ae3-b566-3b19-f0b0-d6ff51960f29, HandlerErrorCode: InvalidRequest)
[10:15:27] Stack ZZZ has an ongoing operation in progress and is not stable (CREATE_IN_PROGRESS)
ZZZ | 5/7 | 10:15:27 AM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | ZZZ The following resource(s) failed to create: [QFDSKafkaEventSourceZZZQFDSD3CBC52EforeignengrainavailabiltynotifyEFE73996]. Rollback requested by user.
[10:15:32] Stack ZZZ has an ongoing operation in progress and is not stable (ROLLBACK_IN_PROGRESS)
ZZZ | 5/7 | 10:15:32 AM | DELETE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata)
ZZZ | 4/7 | 10:15:33 AM | DELETE_COMPLETE | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata)
[10:15:37] Stack ZZZ has an ongoing operation in progress and is not stable (ROLLBACK_IN_PROGRESS)
This github issue mentions the same error in the same context. It suggests the problem is due to missing Lambda permissions on the (customer-managed) KMS key used to encrypt the MKS cluster secret.
If so, the solution -- shamelessly stolen from the linked issue -- is something like:
# get a reference to the MSK cluster key
cluster_key = aws_kms.Key.from_key_arn(this, 'ClusterKey', CLUSTER_ENCRYPTION_KEY_ARN)
# add the key reference to the secret reference
secret = Secret.from_secret_attributes(self, "AmazonMSK_AdminSecret",
secret_complete_arn="somearn",
encryption_key=cluster_key
)
# give the Lambda permission to decrypt the Cluster Key
cluster_key.grant_decrypt(lambdaFn.role)
We find during a TLS Handshake from time to time a delay in the flow. This takes about from 5 seconds to over 60 seconds.
The Application use JAVA 8 and TLS 1.2
We activate the TLS Debug and find some unexpeceted delay during the handshake for example
Analysing SSL handshake on kagu I found some handshakes with time gaps in between.
One example is:
2022/11/25 08:13:56 | INFO | jvm 1 | %% Initialized: [Session-229, SSL_NULL_WITH_NULL_NULL]
2022/11/25 08:13:56 | INFO | jvm 1 | %% Negotiating: [Session-229, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384]
2022/11/25 08:14:21 | INFO | jvm 1 | %% Cached server session: [Session-229, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384]
Comparing the log of session-229 and session-230 (ignoring the differences in timestamp, random data, etc) all data (certificates etc) are equal. Usually the SSL handshake takes less than a second.
In this case there is a gap of 25 seconds. After the gap the handshake continues and is successful.
Before the gap:
2022/11/25 08:13:56 | INFO | jvm 1 | cCredit HTTP client-interface validator task-3, WRITE: TLSv1.2 Handshake, length = 3694
After the gap:
2022/11/25 08:14:20 | INFO | jvm 1 | cCredit HTTP client-interface validator I/O-1, READ: TLSv1.2 Handshake, length = 2565
Have someone an idea what happens.
Thanks for any response
I am unable to shutdown my activemq gracefully after enabling jmx. Please help and tell me what am I doing wrong. Here is what I am trying to do.
start activemq:-
[mwapp#JMNGD1BAO150V02 ~]$ /app/apache-activemq-5.14.0/bin/activemq start xbean:/app/apache-activemq-5.14.0/conf/activemq-security.xml
INFO: Loading '/app/apache-activemq-5.14.0//bin/env'
INFO: Using java '/usr/java/jre1.7.0_79//bin/java'
INFO: Starting - inspect logfiles specified in logging.properties and log4j.properties to get details
INFO: pidfile created : '/app/apache-activemq-5.14.0//data/activemq.pid' (pid '16917')
activemq.log:- To me it's looking fine
2017-10-12 13:48:18,936 | INFO | Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1#2142b533: startup date [Thu Oct 12 13:48:18 IST 2017]; root of context hierarchy | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
2017-10-12 13:48:20,008 | INFO | Loading properties file from URL [file:/app/apache-activemq-5.14.0//conf/credentials-enc.properties] | org.jasypt.spring31.properties.EncryptablePropertyPlaceholderConfigurer | main
2017-10-12 13:48:20,975 | INFO | Loaded the Bouncy Castle security provider. | org.apache.activemq.broker.BrokerService | main
2017-10-12 13:48:21,283 | INFO | Using Persistence Adapter: KahaDBPersistenceAdapter[/jms_nas/kahadb] | org.apache.activemq.broker.BrokerService | main
2017-10-12 13:48:21,308 | INFO | JMX consoles can connect to service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi | org.apache.activemq.broker.jmx.ManagementContext | JMX connector
2017-10-12 13:48:21,594 | INFO | KahaDB is version 6 | org.apache.activemq.store.kahadb.MessageDatabase | main
2017-10-12 13:48:21,653 | INFO | Recovering from the journal #1105:27118028 | org.apache.activemq.store.kahadb.MessageDatabase | main
2017-10-12 13:48:21,657 | INFO | Recovery replayed 58 operations from the journal in 0.046 seconds. | org.apache.activemq.store.kahadb.MessageDatabase | main
2017-10-12 13:48:21,719 | INFO | PListStore:[/app/apache-activemq-5.14.0/data/localhost/tmp_storage] started | org.apache.activemq.store.kahadb.plist.PListStoreImpl | main
2017-10-12 13:48:21,903 | INFO | Apache ActiveMQ 5.14.0 (localhost, ID:JMNGD1BAO150V02-59661-1507796301746-0:1) is starting | org.apache.activemq.broker.BrokerService | main
2017-10-12 13:48:22,786 | INFO | Listening for connections at: ssl://JMNGD1BAO150V02:61616?needClientAuth=true&maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2017-10-12 13:48:22,787 | INFO | Connector ssl started | org.apache.activemq.broker.TransportConnector | main
2017-10-12 13:48:22,787 | INFO | Apache ActiveMQ 5.14.0 (localhost, ID:JMNGD1BAO150V02-59661-1507796301746-0:1) started | org.apache.activemq.broker.BrokerService | main
2017-10-12 13:48:22,787 | INFO | For help or more information please see: http://activemq.apache.org | org.apache.activemq.broker.BrokerService | main
2017-10-12 13:48:22,788 | WARN | Store limit is 102400 mb (current store usage is 1397 mb). The data directory: /jms_nas/kahadb only has 91534 mb of usable space. - resetting to maximum available disk space: 91534 mb | org.apache.activemq.broker.BrokerService | main
2017-10-12 13:48:23,646 | INFO | No Spring WebApplicationInitializer types detected on classpath | /admin | main
2017-10-12 13:48:23,755 | INFO | ActiveMQ WebConsole available at http://localhost:8161/ | org.apache.activemq.web.WebConsoleStarter | main
2017-10-12 13:48:23,755 | INFO | ActiveMQ Jolokia REST API available at http://localhost:8161/api/jolokia/ | org.apache.activemq.web.WebConsoleStarter | main
2017-10-12 13:48:23,799 | INFO | Initializing Spring FrameworkServlet 'dispatcher' | /admin | main
2017-10-12 13:48:24,068 | INFO | No Spring WebApplicationInitializer types detected on classpath | /api | main
2017-10-12 13:48:24,185 | INFO | jolokia-agent: Using policy access restrictor classpath:/jolokia-access.xml | /api | main
stop activemq:-
[mwapp#JMNGD1BAO150V02 ~]$ /app/apache-activemq-5.14.0/bin/activemq stop xbean:/app/apache-activemq-5.14.0/conf/activemq-security.xml
INFO: Loading '/app/apache-activemq-5.14.0//bin/env'
INFO: Using java '/usr/java/jre1.7.0_79//bin/java'
INFO: Waiting at least 30 seconds for regular process termination of pid '16917' :
Java Runtime: Oracle Corporation 1.7.0_79 /usr/java/jre1.7.0_79
Heap sizes: current=63488k free=61608k max=932352k
JVM args: -Xms64M -Xmx1G -Djava.util.logging.config.file=logging.properties -Djava.security.auth.login.config=/app/apache-activemq-5.14.0//conf/login.config -Dactivemq.classpath=/app/apache-activemq-5.14.0//conf:/app/apache-activemq-5.14.0//../lib/: -Dactivemq.home=/app/apache-activemq-5.14.0/ -Dactivemq.base=/app/apache-activemq-5.14.0/ -Dactivemq.conf=/app/apache-activemq-5.14.0//conf -Dactivemq.data=/app/apache-activemq-5.14.0//data
Extensions classpath:
[/app/apache-activemq-5.14.0/lib,/app/apache-activemq-5.14.0/lib/camel,/app/apache-activemq-5.14.0/lib/optional,/app/apache-activemq-5.14.0/lib/web,/app/apache-activemq-5.14.0/lib/extra]
ACTIVEMQ_HOME: /app/apache-activemq-5.14.0
ACTIVEMQ_BASE: /app/apache-activemq-5.14.0
ACTIVEMQ_CONF: /app/apache-activemq-5.14.0/conf
ACTIVEMQ_DATA: /app/apache-activemq-5.14.0/data
Connecting to JMX URL: service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi
ERROR: java.lang.NullPointerException
java.lang.NullPointerException
at org.apache.activemq.console.command.AbstractCommand.handleException(AbstractCommand.java:167)
at org.apache.activemq.console.command.AbstractJmxCommand.execute(AbstractJmxCommand.java:390)
at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:154)
at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)
at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
at org.apache.activemq.console.Main.main(Main.java:115)
...............................
INFO: Regular shutdown not successful, sending SIGKILL to process
INFO: sending SIGKILL to pid '16917'
As you can see, the system is forcefully shutting down using pid which is not expected. I am currently using apache-activemq-5.14.0 and the configuration file will look something like below. I am not sure why activemq gave two separate file to enable JMX i.e. env and activemq-security.xml. Or the env file has some different role to play. I read the documentation, and I got more confuse when they mention from V.5.12.0 onwards they are supporting OCSP. Do I need to enable that too?
${ACTIVEMQ_HOME}/bin/env
#!/bin/sh
# Active MQ installation dirs
# ACTIVEMQ_HOME="<Installationdir>/"
# ACTIVEMQ_BASE="$ACTIVEMQ_HOME"
# ACTIVEMQ_CONF="$ACTIVEMQ_BASE/conf"
# ACTIVEMQ_DATA="$ACTIVEMQ_BASE/data"
# ACTIVEMQ_TMP="$ACTIVEMQ_BASE/tmp"
ACTIVEMQ_OPTS_MEMORY="-Xms64M -Xmx1G"
if [ -z "$ACTIVEMQ_OPTS" ] ; then
ACTIVEMQ_OPTS="$ACTIVEMQ_OPTS_MEMORY -Djava.util.logging.config.file=logging.properties -Djava.security.auth.login.config=$ACTIVEMQ_CONF/login.config"
fi
#ACTIVEMQ_OPTS="$ACTIVEMQ_OPTS -Dorg.apache.activemq.audit=true"
# ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.port=1099"
# ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.password.file=${ACTIVEMQ_CONF}/jmx.password"
# ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.access.file=${ACTIVEMQ_CONF}/jmx.access"
# ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.ssl=true"
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote"
#ACTIVEMQ_SUNJMX_CONTROL="--jmxurl service:jmx:rmi:///jndi/rmi://127.0.0.1:1099/jmxrmi --jmxuser controlRole --jmxpassword abcd1234"
ACTIVEMQ_SUNJMX_CONTROL=""
if [ -z "$ACTIVEMQ_QUEUEMANAGERURL" ]; then
ACTIVEMQ_QUEUEMANAGERURL="--amqurl tcp://localhost:61616"
fi
if [ -z "$ACTIVEMQ_SSL_OPTS" ] ; then
#ACTIVEMQ_SSL_OPTS="-Djava.security.properties=$ACTIVEMQ_CONF/java.security"
ACTIVEMQ_SSL_OPTS=""
fi
#ACTIVEMQ_DEBUG_OPTS="-Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005"
if [ -z "$ACTIVEMQ_KILL_MAXSECONDS" ]; then
ACTIVEMQ_KILL_MAXSECONDS=30
fi
ACTIVEMQ_USER=""
# ACTIVEMQ_PIDFILE="$ACTIVEMQ_DATA/activemq.pid"
JAVA_HOME="/usr/java/jre1.7.0_79/"
${ACTIVEMQ_HOME}/conf/activemq-security.xml
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.data}">
----------------------------------------------
<managementContext>
<!--managementContext createConnector="true" connectorPort="1099"/-->
<managementContext createConnector="true">
<property xmlns="http://www.springframework.org/schema/beans" name="environment">
<map xmlns="http://www.springframework.org/schema/beans">
<entry xmlns="http://www.springframework.org/schema/beans" key="jmx.remote.x.password.file" value="${activemq.base}/conf/jmx.password"/>
<entry xmlns="http://www.springframework.org/schema/beans" key="jmx.remote.x.access.file" value="${activemq.base}/conf/jmx.access"/>
</map>
</property>
</managementContext>
</managementContext>
----------------------------------------------
<shutdownHooks>
<bean xmlns="http://www.springframework.org/schema/beans" class="org.apache.activemq.hooks.SpringContextHook"/>
</shutdownHooks>
</broker>
In org.springframework.http.converter.AbstractHttpMessageConverter.write() is a call to
writeInternal(t, outputMessage);
outputMessage.getBody().flush();
in the Implementation in SerializingHttpMessageConverter.writeInternal() is a call to
objectStream.flush();
objectStream.close();
I guess this is the Reaseon for my Error in GZIPResponseStream.flush():
public void flush() throws IOException {
if(this.closedFlag) {
throw new IOException("Cannot flush a closed output stream");
Producing the Message
INFO | jvm 1 | 2016/04/09 14:05:06.673 | SEVERE: Servlet.service() for servlet dispatcherServlet threw exception
INFO | jvm 1 | 2016/04/09 14:05:06.673 | java.io.IOException: Cannot flush a closed output stream
INFO | jvm 1 | 2016/04/09 14:05:06.673 | at de.hybris.platform.util.GZIPResponseStream.flush(GZIPResponseStream.java:70)
INFO | jvm 1 | 2016/04/09 14:05:06.673 | at org.springframework.http.converter.AbstractHttpMessageConverter.write(AbstractHttpMessageConverter.java:182)
INFO | jvm 1 | 2016/04/09 14:05:06.673 | at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter$ServletHandlerMethodInvoker.writeWithMessageConverters(AnnotationMethodHandlerAdapter.java:975)
is there an Error in the Framework, or do I miss someting. Is there another reaseon for this error? (AND PROB. A FIX?)
I'm trying to use vagrant-openstack-provider to manage Bluemix VMs.
All is looking good, except for an error message at the end: Floating ip pool not found.
2015-09-27 11:17 | DEBUG | request => method : POST
2015-09-27 11:17 | DEBUG | request => url : https://api2-dal09.open.ibmcloud.com:8774/v2/.../os-floating-ips
2015-09-27 11:17 | DEBUG | request => headers : {"X-Auth-Token"=>"...", :accept=>:json, :content_type=>:json}
2015-09-27 11:17 | DEBUG | request => body : {"pool":"private"}
2015-09-27 11:17 | DEBUG | response => code : 404
2015-09-27 11:17 | DEBUG | response => headers : {:content_length=>"73", :content_type=>"application/json; charset=UTF-8", :x_compute_request_id=>"...", :date=>"Sun, 27 Sep 2015 10:17:30 GMT"}
2015-09-27 11:17 | DEBUG | response => body : {"itemNotFound": {"message": "Floating ip pool not found.", "code": 404}}
2015-09-27 11:17 | WARN | Error allocating ip in pool private : Floating ip pool not found.
2015-09-27 11:17 | WARN | Impossible to allocate a new IP
ERROR warden: Error occurred: Floating ip pool not found.
Is it possible to create an IP pool in the horizon console? If so, how do I do this? I couldn't find any documentation online.
I specified 'private' in the Vagrantfile:
os.floating_ip_pool = 'private'
I should have been using 'Public-Network' instead:
os.floating_ip_pool = 'Public-Network'
I didn't realise at the time, but you can find the floating_ip_pool with:
snowch$ vagrant openstack floatingip-list
+-------------------+
| Floating IP pools |
+-------------------+
| Public-Network |
+-------------------+