I just downloaded a new docker image. When I try to run it I get this log on my console
Setting Active Processor Count to 4
Calculating JVM memory based on 381456K available memory
unable to calculate memory configuration
fixed memory regions require 654597K which is greater than 381456K available for allocation: -XX:MaxDirectMemorySize=10M, -XX:MaxMetaspaceSize=142597K, -XX:ReservedCodeCacheSize=240M, -Xss1M * 250 threads
Please, how can I fix this?
I am assuming that you have multiple services and you are going to start them at a time. The issue is related to memory which docker and spring boot uses.
Try this:
environment:
- JAVA_TOOL_OPTIONS=-Xmx128000K
deploy:
resources:
limits:
memory: 800m
You have to provide memory which I mentioned in the .yaml file syntax.
While at the time of startup each service takes lot of memory, so there is no memory remaining for rest of the services and because of that other services starts failing with the memory related message.
Related
I have a task app in the PCF. Whenever I run the task or schedule job.Memory allocation for task/job execution is not the same as for the app. It always allocated memory 512MB (default in my case). But app memory allocated for 2GB. Below is my manifest.yml
applications:
- name: hello-world
instances: 0
memory: 2G
I can allocate memory for task from the CLI like below. But I don't know about PCF scheduler job :
cf run-task hello-world ".java-buildpack/open_jdk_jre/bin/java org.springframework.boot.loader.JarLauncher" -m 2GB
What about the Production environment where I can't use CLI.
Is there any way I can allocate memory for the task and PCF scheduler job from manifest.yml.
The PCF Scheduler does not, at the time I write this, support setting the memory limit for a scheduled task. It will always use the default memory limit set by your platform operations team. This is a popular feature request though and on the roadmap.
I am trying to run 2 tasks on the same EC2 container. The EC2 container is running on a t2.large type EC2 instance.
One of the tasks (which is a daemon) starts fine and is RUNNING.
The other task which is an application, does not start and I see the following errors in the Events tab.
service test-service was unable to place a task because no container instance met all of its requirements. The closest matching container-instance xxxxxx has insufficient CPU units available. For more information, see the Troubleshooting section.
service test-service was unable to place a task because no container instance met all of its requirements. The closest matching container-instance xxxxxx has insufficient memory available. For more information, see the Troubleshooting section.
I looked at the CPU and memory section for the container instance and the values are -
Registered Available
CPU 1024 1014
Memory 985 729
My task definitions for the task that does not run has the following CPU and Memory values -
"memory": 512,
"cpu": 10
The daemon that successfully runs on the same EC2 container instance also has the same values for memory and CPU.
I read thru the AWS docs here at https://aws.amazon.com/premiumsupport/knowledge-center/ecs-container-instance-requirement-error/ and tried to reduce the CPU and memory requirements for the test-service task definition but nothing helped. I also changed the instance type to something bigger but that did not help either.
Can someone please help me with what I should do CPU and memory for both the tasks (daemon and application) so they can run on the same EC2 container instance ?
Note: I cannot add another container to the ECS cluster.
The task definition sets CPU limit to 10 units which is probably insufficient in your case. ECS can manage CPU resources dynamically when you set up the value to 0. However it is not possible in case of memory.
I am trying to run a load test for a application. For this i am using JMeter (v4 & v5) on linux Red hat 7.5 Vm with 16GB Ram and 8vCPU power. Goal is to reach 20k Users connected via ยต-service.
However during the test runs i get the following errors on the console:
Uncaught Exception java.lang.OutOfMemoryError: unable to create new native thread.
Here is my jvm jmeter configuration :
cat bin/jmeter | grep HEAP
HEAP (Optional) Java runtime options for memory management
: "${HEAP:="-Xms1g -Xmx4g -XX:MaxMetaspaceSize=256m"}"
Any ideas?
I tried changing the heap size in jmeter, but that didn't seem to help at all.
unable to create new native thread is not something you can work around by increasing JVM Heap, you're going above maximum number of threads threshold which is defined on OS level.
You will need to amend nproc value via ulimit command or by modifying /etc/security/limits.conf file to look like:
your_user soft nproc 1024
your_user hard nproc 32768
Reference: Unable to create new native thread
If you will be still receiving this error after raising maximum number of processes on OS level - most probably you will have to go for Distributed Testing
I'm running multiple microservices (Spring cloud + docker) in small/medium machines on AWS and recently I found that these machines are often exhausted and need rebooting.
I'm investigating the causes of this loss of power, thinking of possible memory leaks or misconfigurations on the instance/container.
I tried to limit the amount of memory these containers can use by doing:
docker run -m 500M --memory-swap 500M -d my-service:latest
At this point my service (standard spring cloud service with one single endpoint that writes stuff to a Redis DB, using spring-data-redis) didn't even start.
Increased the memory to 760M and it worked, but monitoring it with docker I see the minimum is:
CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
cd5f64aa371e 0.18% 606.9 MiB / 762.9 MiB 79.55% 102.4 MB / 99 MB 1.012 MB / 4.153 MB 60
I added some parameters to limit the JVM memory heap but it doesn't seem to reduce it very much:
_JAVA_OPTIONS: "-Xms8m -Xss256k -Xmx512m"
I'm running
Spring Cloud Brixton.M5
Spring Boot 1.3.2
Java 8 (Oracle JVM)
Docker
Spring data Redis 1.7.1
Is there a reason why such simple service uses so much memory to run? Are there any features I should disable to improve that?
We've investigated a number of things in a similar setup in terms of the JVM itself. A quick way to save some memory if using Java 8 is to use the following options:
-Xms256m -Xmx512m -XX:-TieredCompilation -Xss256k -XX:+UseG1GC -XX:+UseStringDeduplication
The G1GC is well documented, the UseStringDeduplication reduces heap usage by de-duplicating the storage of Strings in the heap (we found about 20% in a JSON/XML web service type environment), and the TieredCompilation makes a big difference in the use of CodeCache (from 70Mb down to 10Mb), as well as about 10% less Metaspace at the expense of about 10% startup time.
According to Spring's Installing Spring Boot applications page you can customize the application startup script by either environment variable or configuration file with the JAVA_OPTS variable.
For example: JAVA_OPTS=-Xmx64m
Weblogic 10.3 gives out of memory
Followings thing I have done
Increased the -Xms512m
Increased the -Xmx1024m
Increased the max perm size in setdomainenv.bat
Is there any other way to resolve this issue I have a 2 GB system?
It is a production machine and the size of the log is around 4 GB .When analysed the log I found many connection refused error
You'll need to profile your application to find the memory leak. It could be open database connections or other resources not being handled properly
Just increasing the Xms and Xmx wont work beyond a point
Take a Heap Dump into an HPROF file and run this using Eclipse Memory Analyzer Tool or VisualVM
or monitor this using JConsole