I'm starting my activemq container like so:
docker run -p 61616:61616 -p 8161:8161 --name='activemq' -d \
-e 'ACTIVEMQ_LOGLEVEL=DEBUG' \
-e 'ACTIVEMQ_ADMIN_USER=bot' \
-e 'ACTIVEMQ_ADMIN_PASSWORD=blahblah' \
-e 'ACTIVEMQ_OWNER_LOGIN=bot' \
-e 'ACTIVEMQ_OWNER_PASSWORD=blahblah' \
-e 'ACTIVEMQ_JMX_LOGIN=bot' \
-e 'ACTIVEMQ_JMX_PASSWORD=blahblah' \
-v /data/activemq:/data/activemq \
-v /var/log/activemq:/var/log/activemq \
webcenter/activemq:latest
The application.yml for my app has the following:
spring:
activemq:
broker-url: ${ACTIVEMQ_PORT_61616_TCP}
user: bot
password: blahblah
and I'm starting my app container like so:
docker run --name='myapp' \
-w /app/ -v /home/ubuntu/myapp/logs:/app/logs \
-v /home/ubuntu/myapp/config:/app/config \
-v /tmp:/tmp -p 4980:4980 \
--link activemq:activemq \
-d myapp
Note that I'm linking my app's container with the activemq container.
Finally, when I attempt to send a msg to activeMQ from my app (via a Camel route - but I don't think that's relevant), I see this in the logs:
javax.jms.JMSSecurityException: User name [bot] or password is invalid.
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:52)
at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1417)
at org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1522)
at org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:527)
at org.springframework.jms.listener.AbstractJmsListeningContainer.refreshSharedConnection(AbstractJmsListeningContainer.java:400)
at org.springframework.jms.listener.DefaultMessageListenerContainer.refreshConnectionUntilSuccessful(DefaultMessageListenerContainer.java:907)
at org.springframework.jms.listener.DefaultMessageListenerContainer.recoverAfterListenerSetupFailure(DefaultMessageListenerContainer.java:882)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1053)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.SecurityException: User name [bot] or password is invalid.
at org.apache.activemq.security.JaasAuthenticationBroker.addConnection(JaasAuthenticationBroker.java:80)
at org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:92)
at org.apache.activemq.broker.MutableBrokerFilter.addConnection(MutableBrokerFilter.java:97)
at org.apache.activemq.broker.TransportConnection.processAddConnection(TransportConnection.java:764)
at org.apache.activemq.broker.jmx.ManagedTransportConnection.processAddConnection(ManagedTransportConnection.java:79)
at org.apache.activemq.command.ConnectionInfo.visit(ConnectionInfo.java:139)
at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:294)
at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:148)
at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:270)
at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:214)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
... 1 common frames omitted
Caused by: javax.security.auth.login.LoginException: No LoginModules configured for activemq-domain
at javax.security.auth.login.LoginContext.init(LoginContext.java:264)
at javax.security.auth.login.LoginContext.<init>(LoginContext.java:417)
at org.apache.activemq.security.JaasAuthenticationBroker.addConnection(JaasAuthenticationBroker.java:72)
... 14 common frames omitted
Needless to say, it all 'just works' when running these components locally (i.e. not dockerized) on my dev machine. Just doesn't work inside docker containers on an EC2 instance.
Any and all help appreciated!
Thanks.
I have eventually got it to work by customising the docker image as follows:
Clone the git repo: git clone https://github.com/disaster37/activemq.git
Fix the URL passed to curl in this file: assets/setup/install
(the current url in that file no longer works)
Edit this file: assets/config/activemq.xml and replace the <jaasAuthenticationPlugin> and <authorizationPlugin> elements with this:
<simpleAuthenticationPlugin anonymousAccessAllowed="false">
<users>
<authenticationUser username="system" password="blubblub" groups="users,admins"/>
<authenticationUser username="bot" password="blahblah" groups="users"/>
</users>
</simpleAuthenticationPlugin>
Build my own docker image: docker build --tag="zackattack/activemq"
Run it: docker logs -f $(docker run --name='activemq' -d zackattack/activemq)
It's not pretty - but it works!
Judging by what I can glean from the debug logs. First, the debug log shows that 'bot' is getting across (the name), so, there does appear to be a connection between myapp and activemq. So the linkage is working. The clue for me is :
Caused by: javax.security.auth.login.LoginException: No LoginModules configured for activemq-domain
I don't know what a login module is, but, I would venture a guess that you need to tell activemq what to do with the name/password it receives? Looking at the description on the docker hub for activemq, there is an example:
-e 'ACTIVEMQ_READ_LOGIN=consumer_login' -e 'ACTIVEMQ_READ_PASSWORD=consumer_password'\
and
-e 'ACTIVEMQ_STATIC_TOPICS=topic1;topic2;topic3' -e 'ACTIVEMQ_STATIC_QUEUES=queue1;queue2;queue3'
Is it possible that either of these settings is needed to configure an activemq-domain login module? Or is that done in a configuration file somewhere? From a docker perspective I think everything is in order. I think there is a configuration issue with activemq. When this works in your regular environment, how do you start activemq (what are the command line arguments, environment variables, and configuration files)?
Related
I've set up continuous deployment to Cloud Run from GitHub for my Spring Boot project, and while it's successfully building in Cloud Build, when I go over to Cloud Run, I get the following error under Creating Revision:
The user-provided container failed to start and listen on the port defined provided by the PORT=8080 environment variable.
When I go over to the Logs, I see the following errors:
2022-09-23 09:42:47.881 BST
Error: Unable to access jarfile /app/target/educity-manager-0.0.1-SNAPSHOT.jar
{
insertId: "632d7187000d739d29eb84ad"
labels: {5}
logName: "projects/educity-manager/logs/run.googleapis.com%2Fstderr"
receiveTimestamp: "2022-09-23T08:42:47.883252595Z"
resource: {2}
textPayload: "Error: Unable to access jarfile /app/target/educity-manager-0.0.1-SNAPSHOT.jar"
timestamp: "2022-09-23T08:42:47.881565Z"
}
2022-09-23 09:43:48.800 BST
run.googleapis.com
…ager/revisions/educity-manager-00011-fod
Ready condition status changed to False for Revision educity-manager-00011-fod with message: Deploying Revision.
{
insertId: "w6ptr6d20ve"
logName: "projects/educity-manager/logs/cloudaudit.googleapis.com%2Fsystem_event"
protoPayload: {
#type: "type.googleapis.com/google.cloud.audit.AuditLog"
resourceName: "namespaces/educity-manager/revisions/educity-manager-00011-fod"
response: {6}
serviceName: "run.googleapis.com"
status: {2}}
receiveTimestamp: "2022-09-23T08:43:49.631015104Z"
resource: {2}
severity: "ERROR"
timestamp: "2022-09-23T08:43:48.800371Z"
}
Dockerfile is as follows (and looking at the build log all of the commands in it completed successfully):
FROM openjdk:17-jdk-alpine
RUN addgroup -S spring && adduser -S spring -G spring
USER spring:spring
COPY . /app
ENTRYPOINT [ "java","-jar","/app/target/educity-manager-0.0.1-SNAPSHOT.jar" ]
I've read that Cloud Run defaults to exposing Port 8080, but just to be on the safe side I've put server.port=${PORT:8080} in my application.properties file (but it seems to make no difference one way or the other).
I have run into similar issues in the past. Usually, I am able to resolve this issue by:
specifying the port in the application itself (as you indicated in your post), and
exposing the required port in my dockerfile eg. EXPOSE 8080
Oh my good god I have done it. After two full days of digging, I realised that because I was doing it through github, my .gitignore file was excluding the /target folder containing the jar file, so Cloud Build never got the jar file mentioned in the Dockerfile.
I am going to have a cry and then go to the pub.
I am currently doing this tutorial. And on the same machine it worked as expected: The nodes are connecting and are creating and finalizing blocks. But now I want to do the same over the internet. So I have a server (Ubuntu 16.04 xenial) with open port 30333 on which I am running this command:
./target/release/node-template \
--base-path /tmp/alice \
--chain ./customSpecRaw.json \
--alice \
--rpc-methods Unsafe \
--port 30333 \
--ws-port 9945 \
--rpc-port 9933 \
--node-key 0000000000000000000000000000000000000000000000000000000000000001 \
--telemetry-url 'wss://telemetry.polkadot.io/submit/ 0' \
--validator \
--name Node01
And my PC (Manjaro 20.2.1 Nibia) with no open ports on which I am running this command:
./target/release/node-template
--base-path /tmp/bob
--chain ./customSpecRaw.json
--bob
--port 30334
--ws-port 9946
--rpc-port 9934
--telemetry-url 'wss://telemetry.polkadot.io/submit/ 0'
--validator
--rpc-methods Unsafe
--name Node02
--bootnodes /ip4/<SERVER IP>/tcp/30333/p2p/<BOOTNODE P2P ID>
In the terminal I see network traffic on both nodes so networking should not be the problem. But there are 0 peers on both nodes and there are no blocks created/finalized. But I am getting two errors on the bootnodes terminal printed repeatedly:
Error while dialing /dns/telemetry.polkadot.io/tcp/443/x-parity-wss/%2Fsubmit%2F: Custom { kind: Other, error: Timeout }
and
Parsing mdns packet failed: LabelIsNotAscii
Both errors are already output before I try to connect to the bootnode from my PC.
Both nodes are compiled from the same code and are using the same custom chain spec file generated on the server.
So my questions are:
What do the errors/warnings mean?
How to fix them in order to get the expected results?
If the errors/warnings are not causing the problem what else could it be?
I did reclone and recompile both nodes and somehow it's working now. I did not change anything in the command except the --no-mdns flag.
The introduction
I am currently trying to build a docker image with all of my node project dependencies, so I can use it to run the tests on Bitbucket Pipelines.
The reason I decided to create an image was due to the fact I want to be in control of what version of the dependencies I have, and to control when to upgrade them accordingly.
The implementation
After having built the image using the following dockerfile:
FROM selenium/standalone-chrome-debug
LABEL name="nodejs-chrome-java"
USER root
# Install Java 8
RUN set -x \
&& apt-get update \
&& apt-get install -y \
ca-certificates-java \
openjdk-8-jre-headless \
openjdk-8-jre \
openjdk-8-jdk-headless \
openjdk-8-jdk \
&& apt-get clean
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
RUN export JAVA_HOME
# Install node 10 and npm
RUN set -x \
&& curl -sL https://deb.nodesource.com/setup_14.x | bash - \
&& apt-get update \
&& apt-get install -y nodejs \
&& npm install -g npm#latest \
&& apt-get clean
# Make node available
RUN set -x \
&& touch ~/.bashrc \
&& echo 'alias nodejs=node' > ~/.bashrc
# Install PhantomJS
RUN set -x \
&& apt-get update \
&& apt-get install -y \
phantomjs \
&& apt-get clean
# Set PhantomJS to run headless
ENV QT_QPA_PLATFORM offscreen
RUN mkdir /logs
RUN touch /logs/selenium.log
I executed the docker image using the following command:
docker run -it --entrypoint /bin/bash -v /my/project:/project -w /project <DOCKER_IMAGE_ID>
And I realised that in order to have selenium running, I would have to run the following command:
/opt/bin/start-selenium-standalone.sh
Which yields the following output:
22:14:13.034 INFO [GridLauncherV3.parse] - Selenium server version: 3.141.59, revision: e82be7d358
22:14:13.304 INFO [GridLauncherV3.lambda$buildLaunchers$3] - Launching a standalone Selenium Server on port 4444
2020-06-29 22:14:13.466:INFO::main: Logging initialized #1128ms to org.seleniumhq.jetty9.util.log.StdErrLog
22:14:14.228 INFO [WebDriverServlet.<init>] - Initialising WebDriverServlet
22:14:14.547 INFO [SeleniumServer.boot] - Selenium Server is up and running on port 4444
However, because I do not want the command line within the container to get stuck, I tried the following instead:
/opt/bin/start-selenium-standalone.sh > /logs/selenium.log 2>&1 &
Which indeed outputs the same content as stated before, into the log file I defined (/logs/selenium.log) when creating the docker image. So, so far so good 👍
And if I then run my tests using the npm test command, all tests pass successfully. 🎉
Given this outcome, and because when I use this image within the Bitbucket Pipelines I wouldn't be able to run this command manually, I decided to include the line that starts the standalone selenium server in the background, on my bash script that gets executed when I call npm test, like so:
#!/bin/bash
printf "Starting Selenium Server"
/opt/bin/start-selenium-standalone.sh > /logs/selenium.log 2>&1 &
PROCESS_ID=$!
retry=0
maxRetries=10
until [ ${retry} -ge ${maxRetries} ]
do
grep "Selenium Server is up and running on port 4444" /logs/selenium.log > /dev/null \
&& echo \
&& break;
retry=$[${retry}+1]
printf . ;
sleep 1
done
if [ ${retry} -ge ${maxRetries} ]; then
echo "Failed after ${maxRetries} attempts!"
exit 1
fi
printf "Running UI tests...\n"
CONFIG_FILE_PATH='../../config_test.json' ./node_modules/.bin/nightwatch
The problem
When the command gets included into the bash script to be executed from there, it seems like the command cannot be executed on the same way as when it's done so manually. And I get the following on the log file:
22:29:12.814 INFO [GridLauncherV3.parse] - Selenium server version: 3.141.59, revision: e82be7d358
22:29:13.093 INFO [GridLauncherV3.lambda$buildLaunchers$3] - Launching a standalone Selenium Server on port 4444
2020-06-30 22:29:13.253:INFO::main: Logging initialized #1602ms to org.seleniumhq.jetty9.util.log.StdErrLog
22:29:14.058 INFO [WebDriverServlet.<init>] - Initialising WebDriverServlet
22:29:14.381 INFO [SeleniumServer.boot] - Selenium Server is up and running on port 4444
22:29:21.121 INFO [ActiveSessionFactory.apply] - Capabilities are: {
"acceptSslCerts": true,
"browserName": "chrome",
"chromeOptions": {
"w3c": false,
"args": [
"headless",
"no-sandbox"
]
},
"javascriptEnabled": true,
"name": "Route Subdomain Dashboard Test"
}
22:29:21.128 INFO [ActiveSessionFactory.lambda$apply$11] - Matched factory org.openqa.selenium.grid.session.remote.ServicedSession$Factory (provider: org.openqa.selenium.chrome.ChromeDriverService)
/my/project/node_modules/chromedriver/lib/chromedriver/chromedriver: 1: /my/project/node_modules/chromedriver/lib/chromedriver/chromedriver: Syntax error: ")" unexpected
22:29:41.214 ERROR [OsProcess.checkForError] - org.apache.commons.exec.ExecuteException: Process exited with an error: 2 (Exit value: 2)
And the following output from my running tests:
⠴ Connecting to 127.0.0.1 on port 4444...
Response 500 POST /wd/hub/session (20425ms)
{
value: {
error: [
"Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'",
"System info: host: '75136e9ec116', ip: '172.17.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '4.19.76-linuxkit', java.version: '1.8.0_252'",
'Driver info: driver.version: unknown'
],
message: 'Timed out waiting for driver server to start.'
},
status: 13
⚠ Error connecting to 127.0.0.1 on port 4444.
_________________________________________________
TEST FAILURE: 1 error during execution; 0 tests failed, 0 passed (24.402s)
✖ route-subdomain-dashboard-test
An error occurred while retrieving a new session: "Timed out waiting for driver server to start."
Error: An error occurred while retrieving a new session: "Timed out waiting for driver server to start."
at endReadableNT (_stream_readable.js:1224:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
Error: An error occurred while retrieving a new session: "Timed out waiting for driver server to start."
at endReadableNT (_stream_readable.js:1224:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
The question
Is there anything I am missing that would allow the command to be successfully executed through the bash script? Why am I seeing such disparate results?
The appreciation
Sorry for the long post, but I think I needed to provide as much context as possible, since it seems a very tricky problem that I've been struggling for quite some time now.
Many thanks in advance 🙏
Updates
02/10/2020 - I realised today that once inside the docker image, if I execute the npm test command in order to run the tests, I have the issue described under the section named The problem. However, if I run the bash script that I created and that npm test command is calling under the hood, I have the tests successfully executed. Could it be the way that npm executes the scripts?
When I run an image in Kubernetes with kubectl run, environment variables are injected into the container.
My problem is that the values are wrong. I do not have anything running at 10.0.0.1. I believe the correct value there would be 10.1.0.1. This misconfiguration causes, as far as I know, among other things, the error from kube-dns reproduced below.
I would like to ask how are these variables injected into the container, preferably for a link into the code which takes care of this (I could not find anything). Also, some hints where the value 10.0.0.1 could be coming from.
pod variables:
$ kubectl run -i --image=busybox --restart=Never -t busybox
If you don't see a command prompt, try pressing enter.
/ # env
KUBERNETES_SERVICE_PORT=443
KUBERNETES_PORT=tcp://10.0.0.1:443
HOSTNAME=busybox
SHLVL=1
HOME=/root
TERM=xterm
KUBERNETES_PORT_443_TCP_ADDR=10.0.0.1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_PORT_443_TCP=tcp://10.0.0.1:443
KUBERNETES_SERVICE_HOST=10.0.0.1
PWD=/
kube-dns error:
$ kubectl --namespace kube-system logs kube-dns-2190035132-gxf80 kubedns
[...]
E0119 10:04:05.271499 55 reflector.go:199] k8s.io/dns/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Service: Get https://10.0.0.1:443/api/v1/services?resourceVersion=0: dial tcp 10.0.0.1:443: i/o timeout
I0119 10:04:05.771477 55 dns.go:174] Waiting for services and endpoints to be initialized from apiserver...
The closest thing to 10.0.0.1 that I have in my config is --service-cluster-ip-range=10.0.0.0/24 parameter I am giving to kube-apiserver.
I have the IP 10.0.0.1 in my etcd, in
# ETCDCTL_API=3 etcdctl get "" --from-key
[...]
/registry/services/specs/default/kubernetes
k8s
v1Service
kubernetes▒default"*$b198bc22-fcff-11e7-83a9-185e0fec8ce528B
Z
component apiserverZ
provider
kuberneteszC
▒
httpsTCP▒▒(10.0.0.1" ClusterIPClientIPBRZ`▒
▒"
/registry/services/specs/kube-system/kubernetes-dashboard
k8s
v1Service
kubernetes-dashboard▒
kube-system"*$b9f0daef-fcff-11e7-83a9-185e0fec8ce528B
ԾZ,
addonmanager.kubernetes.io/mode ReconcileZ
ppkubernetes-dashboardZ*
kubernetes.io/minikube-addons dashboardZ3
&kubernetes.io/minikube-addons-endpoint dashboardb
0kubectl.kubernetes.io/last-applied-configuration{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"addonmanager.kubernetes.io/mode":"Reconcile","app":"kubernetes-dashboard","kubernetes.io/minikube-addons":"dashboard","kubernetes.io/minikube-addons-endpoint":"dashboard"},"name":"kubernetes-dashboard","namespace":"kube-system"},"spec":{"ports":[{"nodePort":30000,"port":80,"targetPort":9090}],"selector":{"app":"kubernetes-dashboard"},"type":"NodePort"}}
z_
TCP▒PG▒(
ppkubernetes-dashboard▒ 10.0.0.82NodePort:NoneBRZCluster`▒
▒"
https://github.com/kubernetes/kubernetes/blob/v1.9.0/pkg/kubelet/envvars/envvars.go#L45-L48 which I found via git grep SERVICE_PORT
It's possible that if your kubernetes.default.svc.cluster.local is pointing to the wrong IP, then kubectl --namespace=kube-system edit svc kubernetes and changing the ClusterIP would sort that out; I don't have a cluster in front of me to test it, though
This worked in GFV2:
$AS_HOME/bin/asadmin \
--host $AS_ADMIN_HOST \
--user $AS_ADMIN_USER \
--port $AS_ADMIN_PORT \
create-jms-resource \
--restype javax.jms.QueueConnectionFactory \
--description XA\ Queue\ Connection\ Factory \
--property Name=myXAQueueConnectionFactory:SupportsXA=true \
jms/myXAQueueConnectionFactory
But the SupportsXA=true no longer works. Maybe I can't find it in the GFV3 manuals, nor can I find it via our friend Google: how to specify XA transactionality using asadmin to configure the factory? Anybody out there know how?
--property ...:transaction-support=XATransaction:...
This seems to be what I needed. Works. Did not find by search of documentation or by Google. Deduced it by looking at the domain.xml file and taking educated guess at syntax.
Am now trying to figure out what property name/value pair sets the connection-validation property the way I want it.
Question has morphed to: what's the full asadmin syntax and property setting for GFV3 connection factories.