We are unable to deploy our sample go web program to Bluemix. Server is trying to start and finally crashing without any error. Please let me know if any files needs modification.
View Our Code In GitHub
Build Log:-
Preparing to start the job...
Downloading build artifacts...
Target: https://api.ng.bluemix.net
Using manifest file /home/pipeline/4095d817-6887-4f7e-b317-c58c97c5afd7/manifest.yml
Creating app JoinSplit-20170529103009108 in org Subhadeep / space dev as subhadeep.bose#in.ibm.com...
OK
Creating route joinsplit-20170529103009108-anecdotal-menorah.mybluemix.net...
OK
Binding joinsplit-20170529103009108-anecdotal-menorah.mybluemix.net to JoinSplit-20170529103009108...
OK
Uploading JoinSplit-20170529103009108...
Uploading app files from: /home/pipeline/4095d817-6887-4f7e-b317-c58c97c5afd7
Uploading 165.2K, 34 files
Done uploading
OK
Starting app JoinSplit-20170529103009108 in org Subhadeep / space dev as subhadeep.bose#in.ibm.com...
Creating container
Successfully created container
Downloaded app package (437.8K)
-----> Download go 1.8.3
-----> Checking Godeps/Godeps.json file
-----> Installing godep v79
Download [https://buildpacks.cloudfoundry.org/dependencies/godep/godep-v79-linux-x64-9e37ce0f.tgz]
-----> Installing glide v0.12.3
Download [https://buildpacks.cloudfoundry.org/dependencies/glide/glide-v0.12.3-linux-x64-aa256363.tgz]
-----> Installing go 1.8.3
Download [https://buildpacks.cloudfoundry.org/dependencies/go/go1.8.3.linux-amd64-32ec5ac6.tar.gz]
-----> Running go build finalize
[31;1m**WARNING**[0m vendor/ directory does not exist.
[31;1m**WARNING**[0m Installing package '.' (default)
-----> Running: go install -tags cloudfoundry -buildmode pie .
Exit status 0
Staging complete
Uploading droplet, build artifacts cache...
Uploading build artifacts cache...
Uploading droplet...
Uploaded build artifacts cache (194B)
Uploaded droplet (2.7M)
Uploading complete
Successfully destroyed container
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 crashed
FAILED
Error restarting application: Start unsuccessful
TIP: use 'cf logs JoinSplit-20170529103009108 --recent' for more information
Finished: FAILED
Your code is listening on the hardcoded port "8081", not on the one provided by the Bluemix/Cloud Foundry environment. Try something like the following in your Main function:
err := http.ListenAndServe(":"+os.Getenv("PORT"), nil)
May be this link can help you out there are some Q&A are available on using go with Bluemix
https://developer.ibm.com/answers/questions/13591/about-using-go-with-bluemix.html
Related
I got this error when trying to mess around with Spring Boot. I create a new project with the initializr. Got JDK and Maven installed. I run the command ./mvnw spring-boot:run and get
./mvnw: line 280: /Users/alesh/projects/spring-boot-demo/.mvn/wrapper/maven-wrapper.properties: No such file or directory
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Failed to create the file
Warning: /Users/alesh/projects/spring-boot-demo/.mvn/wrapper/maven-wrapper.jar:
Warning: No such file or directory
1 58727 1 1033 0 0 4152 0 0:00:14 --:--:-- 0:00:14 4251
curl: (23) Failure writing output to destination
Error: Could not find or load main class org.apache.maven.wrapper.MavenWrapperMain
Caused by: java.lang.ClassNotFoundException: org.apache.maven.wrapper.MavenWrapperMain
mvn spring-boot:run
Running this worked for me. Looks like nobody else on the internet has had this problem, at least nobody for whom the solution wasn't obvious. If I'm missing anything please let me know!
I am using a containerized Spring boot application in Kubernetes. But the application automatically exits and restarts with exit code 143 and error message "Error".
I am not sure how to identify the reason for this error.
My first idea was that Kubernetes stopped the container due to too high resource usage, as described here, but I can't see the corresponding kubelet logs.
Is there any way to identify the cause/origin of the SIGTERM? Maybe from spring-boot itself, or from the JVM?
Exit Code 143
It denotes that the process was terminated by an external signal.
The number 143 is a sum of two numbers: 128+x, # where x is the signal number sent to the process that caused it to terminate.
In the example, x equals 15, which is the number of the SIGTERM signal, meaning the process was killed forcibly.
Hope this helps better.
I've just run into this exact same problem. I was able to track down the origin of the Exit Code 143 by looking at the logs on the Kubernetes nodes (note, the logs on the node not the pod). (I use Lens as an easy way to get a node shell but there are other ways)
Then if you look in /var/log/messages for terminated you'll see something like this:
Feb 2 11:52:27 np-26992252-3 kubelet[23125]: I0202 11:52:27.541751 23125 kubelet.go:2214] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="default/app-compute-deployment-56ccffd87f-8s78v"
Feb 2 11:52:27 np-26992252-3 kubelet[23125]: I0202 11:52:27.541920 23125 kubelet.go:2214] "SyncLoop (probe)" probe="readiness" status="" pod="default/app-compute-deployment-56ccffd87f-8s78v"
Feb 2 11:52:27 np-26992252-3 kubelet[23125]: I0202 11:52:27.543274 23125 kuberuntime_manager.go:707] "Message for Container of pod" containerName="app" containerStatusID={Type:containerd ID:c3426d6b07fe3bd60bcbe675bab73b6b4b3619ef4639e1c23bca82692633765e} pod="default/app-comp
ute-deployment-56ccffd87f-8s78v" containerMessage="Container app failed liveness probe, will be restarted"
Feb 2 11:52:27 np-26992252-3 kubelet[23125]: I0202 11:52:27.543374 23125 kuberuntime_container.go:723] "Killing container with a grace period" pod="default/app-compute-deployment-56ccffd87f-8s78v" podUID=89fdc1a2-3a3b-4d57-8a4d-ab115e52dc85 containerName="app" containerID="con
tainerd://c3426d6b07fe3bd60bcbe675bab73b6b4b3619ef4639e1c23bca82692633765e" gracePeriod=30
Feb 2 11:52:27 np-26992252-3 containerd[22741]: time="2023-02-02T11:52:27.543834687Z" level=info msg="StopContainer for \"c3426d6b07fe3bd60bcbe675bab73b6b4b3619ef4639e1c23bca82692633765e\" with timeout 30 (s)"
Feb 2 11:52:27 np-26992252-3 containerd[22741]: time="2023-02-02T11:52:27.544593294Z" level=info msg="Stop container \"c3426d6b07fe3bd60bcbe675bab73b6b4b3619ef4639e1c23bca82692633765e\" with signal terminated"
The bit to look out for is containerMessage="Container app failed liveness probe, will be restarted"
I'm running Apache Openwhisk on k3s, installed using helm.
Below is the invoker logs, taken several hours after a fresh install, with several functions set to run periodically. This message appears every few seconds after the problem starts.
[2020-03-17T13:27:12.691Z] [ERROR] [#tid_sid_invokerHealth] [ContainerPool]
Rescheduling Run message, too many message in the pool, freePoolSize: 0 containers and 0 MB,
busyPoolSize: 8 containers and 4096 MB, maxContainersMemory 4096 MB, userNamespace: whisk.system,
action: ExecutableWhiskAction/whisk.system/invokerHealthTestAction0#0.0.1, needed memory: 128 MB,
waiting messages: 24
Here are the running pods. Notice all the function pods have an age of 11+ hours.
NAME READY STATUS RESTARTS AGE
openwhisk-gen-certs-n965b 0/1 Completed 0 14h
openwhisk-init-couchdb-4s9rh 0/1 Completed 0 14h
openwhisk-install-packages-pnvmq 0/1 Completed 0 14h
openwhisk-apigateway-78c64dd7c9-2gsw6 1/1 Running 2 14h
openwhisk-couchdb-844c6df68f-qrxq6 1/1 Running 2 14h
openwhisk-wskadmin 1/1 Running 2 14h
openwhisk-redis-77494b8d44-gkmlt 1/1 Running 2 14h
openwhisk-zookeeper-0 1/1 Running 2 14h
openwhisk-kafka-0 1/1 Running 2 14h
openwhisk-controller-0 1/1 Running 2 14h
openwhisk-nginx-5f795dd747-c228s 1/1 Running 4 14h
openwhisk-cloudantprovider-69fd94b6f6-x88f4 1/1 Running 2 14h
openwhisk-kafkaprovider-544fbfdcc7-kn29p 1/1 Running 2 14h
openwhisk-alarmprovider-58c5454cc8-q4wbw 1/1 Running 2 14h
openwhisk-invoker-0 1/1 Running 2 14h
wskopenwhisk-invoker-00-1-prewarm-nodejs10 1/1 Running 0 14h
wskopenwhisk-invoker-00-6-prewarm-nodejs10 1/1 Running 0 13h
wskopenwhisk-invoker-00-15-whisksystem-checkuserload 1/1 Running 0 13h
wskopenwhisk-invoker-00-31-whisksystem-guacscaleup 1/1 Running 0 12h
wskopenwhisk-invoker-00-30-whisksystem-guacscaledown 1/1 Running 0 12h
wskopenwhisk-invoker-00-37-whisksystem-functionelastalertcheckd 1/1 Running 0 11h
wskopenwhisk-invoker-00-39-whisksystem-checkuserload 1/1 Running 0 11h
wskopenwhisk-invoker-00-40-whisksystem-functionelastalertcheckd 1/1 Running 0 11h
wskopenwhisk-invoker-00-42-whisksystem-guacscaleup 1/1 Running 0 11h
wskopenwhisk-invoker-00-43-whisksystem-functionelastalertcheckd 1/1 Running 0 11h
Shouldn't Openwhisk be killing these pods after they reach the timeout? The functions all have a timeout of either 3 or 5 minutes, but Openwhisk doesn't seem to enforce this.
One other thing I noticed was "timeout" being set to "false" on the activations.
$ wsk activation get ...
{
"annotations": [
...
{
"key": "timeout",
"value": false
},
...
}
The timeout annotation is specific to an particular activation. If the value is true it means that particular activation of the corresponding function exceeded its set maximum duration which is a range of values from 100 ms to 5 minutes by default (per the docs) unless changed for the system deployment as a whole.
The pods are used to execute the functions - they will stick around for some duration while idle to facilitate future warm starts. The openwhisk invoker will terminate these warm pods eventually after an idle timeout, or when resource are required to run other pods.
Ok I fixed this by changing the invoker container factory implementation to docker. I'm not sure why the kubernetes implementation fails to kill pods (and release memory), but we are using docker as the container runtime for k3s.
To set this, change invoker.containerFactory.impl to docker in the helm chart values:
https://github.com/apache/openwhisk-deploy-kube/blob/master/helm/openwhisk/values.yaml#L261
I also increased the invoker memory (invoker.jvmHeapMB) to 1024:
https://github.com/apache/openwhisk-deploy-kube/blob/master/helm/openwhisk/values.yaml#L257
Here is a link that explains the container factory setting:
https://github.com/apache/openwhisk-deploy-kube/blob/master/docs/configurationChoices.md#invoker-container-factory
We are using New Relic infrastructure agent from last 2 yrs but after 13th Nov 2019 suddenly its not working. Then I update the version of newrelic to 5.2.3.131. But the problem is not resolve. The problem is i’m unable to restart new relic infra agent.
I used below commands…
echo “license_key: ${NEW_RELIC_LICENSE_KEY}” | sudo tee /etc/newrelic-infra.yml
sudo curl -o /etc/yum.repos.d/newrelic-infra.repo https://download.newrelic.com/infrastructure_agent/linux/yum/el/6/x86_64/newrelic-infra.repo
sudo yum -q makecache -y --disablerepo=’*’ --enablerepo=‘newrelic-infra’
sudo yum install newrelic-infra -y
sudo initctl restart newrelic-infra
Application hosted aws elastic beanstalk.
I’m getting initctl: Unknown instance.
Deatails error are below…
INFO [7168] - [Application update pem.pem-staging.f6e105eb760.20191117-164558#668/AppDeployStage0/EbExtensionPostBuild/Infra-EmbeddedPostBuild/postbuild_1_PEM/Command 04-configure_new_relic] : Activity execution failed, because: license_key: XXXXXXXXXXXXXXXX
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0
100 239 100 239 0 0 2091 0 --:–:-- --:–:-- --:–:-- 2096
Loaded plugins: priorities, update-motd, upgrade-helper
Package newrelic-infra-1.7.1-1.x86_64 already installed and latest version
Nothing to do
initctl: Unknown instance:
(ElasticBeanstalk::ExternalInvocationError)
in my case it looks:
.ebextensions/new_relic.yml
packages:
yum:
newrelic-sysmond: []
rpm:
newrelic: http://yum.newrelic.com/pub/newrelic/el5/x86_64/newrelic-repo-5-3.noarch.rpm
commands:
"01":
command: nrsysmond-config --set license_key=XXXXXXXXXXXXXXXXXXXX
"02":
command: echo hostname=$SERVER_URL >> /etc/newrelic/nrsysmond.cfg
"03":
command: /etc/init.d/newrelic-sysmond start
You might not need '02' command
Problem is new relic end. I have solved this temporarily by using below commands...
cat /etc/newrelic-infra.yml
ps aux | grep newrelic-infra
Just kill all processes returned by the ps command above with kill -9 pid1 pid2...
Then start the service with sudo initctl start newrelic-infra
It's now working fine.
I started learn Kubernetes, so I follow this guide https://kubernetes.io/docs/setup/turnkey/gce/
I got an error after I run "cluster/kube-up.sh"
Waiting up to 300 seconds for cluster initialization.
This will continually check to see if the API for kubernetes is reachable.
This may time out if there was some uncaught error during start up.
.........................................................................................................................................Cluster failed to initialize within 300 seconds.
Last output from querying API server follows:
% Total % Received % Xferd Average Speed Time Time Time current
Dload Upload Total Spent Left speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to "[kubernetes-master (external IP)]"
So I tried to search about this error. And then found one solution. It said go to the "cluster/gce/config-default.sh"
secret: $(dd if=/dev/urandom iflag=fullblock bs=32 count=1 2>/dev/null | base64 | tr -d '\r\n')
And then change 'dd' to 'gdd'. So I modified config-default.sh, But it doesn't work.
I don't know how to fix it. Is there are any solution about this error ?̊̈ Also in the /var/logs there are no logs about Kubernetes. Where logs are saved ?̊̈
My Mac version: mas OS Mojave
version: 10.14.3
Looks like GDD is only available in MacOS, about the Kubernetes logs you can have a look at this other article. If you would have a Google Kubernetes Engine cluster it would be easier to check the log by looking at Stackdriver.