I have a 3 node Distributed Hbase cluster of version 2.0.2 and its working properly. After I Installed apache ranger From Same Ambari & I just Enabled the Hbase plugin and then I restarted the HBase Immediately Both HBase masters are getting down with Below logs.
ERROR [Thread-16] master.HMaster: ***** ABORTING master ,16000,1585061451214: The coprocessor org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor threw java.lang.UnsatisfiedLinkError: /run/hbase/.cache/JNA/temp/jna2781046120401699711.tmp: /run/hbase/.cache/JNA/temp/jna2781046120401699711.tmp: failed to map segment from shared object *****
ERROR [Thread-16] master.HMaster: Failed to become active master
java.lang.NullPointerException
at org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1405)
at org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1310)
at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:930)
at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2234)
at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:570)
at java.lang.Thread.run(Thread.java:745)
If I Disable the Plugin and everything is working fine
We were experiencing a similar issue with a near identical stack trace to yours. Like you, everything worked only if the plugin was disabled.
For us it turns out that the reason was that the /run mount had a noexec flag, which prevented tmp files from being executed within it. The solution was to remount /run on the Hbase master nodes using:
sudo mount -o remount,exec /run
After this we restarted the HBase services and everything was working again.
Note that this modified mount will not be saved after a restart of the machine. For it to persist you may need to use something like fstab
I am currently trying to setup Elasticsearch for a project. I have installed Elasticsearch 7.4.1 and I have also installed Java, that is openjdk 11.0.4.
But when I try to start Elasticsearch using the command
sudo systemctl start elasticsearch
I get the error below
Job for elasticsearch.service failed because the control process exited with error code.
See "systemctl status elasticsearch.service" and "journalctl -xe" for details.
And when I try to run the command
systemctl status elasticsearch.service
I get the error message
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vend
Active: failed (Result: exit-code) since Fri 2019-11-01 06:09:54 UTC; 12s ago
Docs: http://www.elastic.co
Process: 5960 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DI
Main PID: 5960 (code=exited, status=1/FAILURE)
I have removed/purged Elasticsearch from my machine and re-installed several times, but it doesn't seem to fix the issue.
I have tried to modify the default network.host and host.port settings in /etc/default/elasticsearch to network.host: 0.0.0.0 and http.port: 9200 to fix the issue, but no luck yet.
Here's how I solved
Firstly, Open /etc/elasticsearch/elasticsearch.yml in your nano editor using the command below:
sudo nano /etc/elasticsearch/elasticsearch.yml
Your network settings should be:
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 127.0.0.1
#
# Set a custom port for HTTP:
#
http.port: 9200
In order for Elasticsearch to allow connections from localhost, and to also listen on port 9200.
Next, run the code below to determine the cause of the error:
journalctl -xe
Error 1
There is insufficient memory for the Java Runtime Environment to continue
Solution
As a JVM application, the Elasticsearch main server process only utilizes memory devoted to the JVM. The required memory may depend on the JVM used (32- or 64-bit). The memory used by JVM usually consists of:
heap space (configured via -Xms and -Xmx)
metaspace (limited by the amount of available native memory)
internal JVM (usually tens of Mb)
OS-dependent memory features like memory-mapped files.
Elasticsearch mostly depends on the heap memory, and this setting manually by passing the -Xms and -Xmx(heap space) option to the JVM running the Elasticsearch server.
Solution
Open /etc/elasticsearch/jvm.options in your nano editor using the command below:
sudo nano /etc/elasticsearch/jvm.options
First, un-comment the value of Xmx and Xms
Next, modify the value of -Xms and -Xmx to no more than 50% of your physical RAM. The value for these settings depends on the amount of RAM available on your server and Elasticsearch requires memory for purposes other than the JVM heap and it is important to leave space for this.
Minimum requirements: If your physical RAM is <= 1 GB
Then, your settings should be:
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms128m
-Xmx128m
OR
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms256m
-Xmx256m
Medium requirements: If your physical RAM is >= 2 GB but <= 4 GB
Then, your settings should be:
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms512m
-Xmx512m
OR
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms750m
-Xmx750m
Large requirements: If your physical RAM is >= 4 GB but <= 8 GB
Then, your settings should be:
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms1024m
-Xmx1024m
OR
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms2048m
-Xmx2048m
Note: If your physical RAM is >= 8 GB you can decide how much heap space you want to allocate to Elasticsearch. You can allocate -Xms2048m and -Xmx2048m OR -Xms4g and -Xmx4g or even higher for better performance based on your available resources.
Error 2
Initial heap size not equal to the maximum heap size
Solution
Ensure the value of -Xms and Xmx are equal. That is, say, you are using the minimum requirements since your physical RAM is <= 1 GB, instead of this:
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms128m
-Xmx256m
it should be this:
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms128m
-Xmx128m
OR this:
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms256m
-Xmx256m
Error 3
the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
Solution
Open /etc/elasticsearch/elasticsearch.yml in your nano editor using the command below:
sudo nano /etc/elasticsearch/elasticsearch.yml
Your discovery settings should be:
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: []
Once all the errors are fixed run the command below to start and confirm the status of Elasticsearch:
sudo systemctl start elasticsearch
sudo systemctl status elasticsearch
That's all.
Had the same problem with a small virtual machine. The above configurations were already set. The only thing that helped was to increase the start timeout. The standard systemd timeout was just not enough.
As a precaution, I set the timeout to 5 minutes as follows.
sudo nano /usr/lib/systemd/system/elasticsearch.service
Added under [Service] section in elasticsearch.service file.
TimeoutStartSec=300
Activate change to service.
sudo /bin/systemctl enable elasticsearch.service
Start service again.
service elasticsearch start
A basic solution to this problem is to just uninstall Elasticsearch and Kibana and again re-install them and your problem will be solved.
For uninstalling Elasticsearch:
sudo apt-get remove --purge elasticsearch
The message was:
dpkg: warning: while removing elasticsearch, directory '/var/lib/elasticsearch' not empty so not removed
dpkg: warning: while removing elasticsearch, directory '/etc/elasticsearch' not empty so not removed
Removed those directories as well:
sudo rm -rf /etc/elasticsearch
sudo rm -rf /var/lib/elasticsearch
Then install it again:
sudo apt-get install elasticsearch=7.10.1
sudo systemctl start elasticsearch
curl http://localhost:9200/
For uninstalling Kibana:
sudo apt-get remove --purge kibana
Removed those directories as well:
sudo rm -rf /etc/kibana
sudo rm -rf /var/lib/kibana
Then install it again:
sudo apt-get install kibana=7.10.1
sudo systemctl start kibana
For opening Kibana on browser:
http://localhost:5601
If you installed using package management, check if the owner of /etc/elasticsearch directory is elasticsearch.
sudo chown -R elasticsearch:elasticsearch /etc/elasticsearch/
First verify that this is the same problem with command:
journalctl -xe
If you see error like this java.lang.NoClassDefFoundError: Could not initialize class
then do this:
My solution I got from here https://github.com/elastic/elasticsearch/issues/57018
sudo nano /etc/sysconfig/elasticsearch
Add this at the end or beggining of the file
# Elasticsearch temp directory
ES_TMPDIR=/var/log/elasticsearch
Try a system restart or just logging-out
I had the same issue as OP on a fresh install of ES. Before going down a rabbit hole of logs and Google searches, I simply tried logging-out of my OS (Ubuntu 20.04) and logged back in. Opened a fresh terminal and elasticsearch was able to start successfully.
For reference, I used:
sudo service elasticsearch restart
and checked it with:
sudo service elasticsearch status
In my case, java was missing from my server
When I have reconfigured my new server I did not check java.
After install java, it starts working.
It may help someone
Please first check java is pre-installed or not... because it is a pre-requirement of elasticsearch.
# systemctl status elasticsearch.service
# which java
# locate java
# java --version
# sudo apt install openjdk-11-jre-headless
# java --version
# sudo systemctl stop elasticsearch
# sudo systemctl start elasticsearch
Thank you.
Steps to install elasticsearch 7.15.2
Follow this digital ocean article
If you see this error
Job for elasticsearch.service failed because the control process exited with error code.
See "systemctl status elasticsearch.service" and "journalctl -xe" for
details.
Open sudo nano /etc/elasticsearch/elasticsearch.yml
Un comment these
network.host: 127.0.0.1
http.port: 9200
Open sudo nano /etc/elasticsearch/jvm.options
Un comment these
-Xms4g
-Xmx4g
Open sudo nano /etc/elasticsearch/elasticsearch.yml
Update this
discovery.seed_hosts: []
At last run these
sudo systemctl start elasticsearch
sudo systemctl status elasticsearch
To check if its working or not run this command
curl -X GET 'http://localhost:9200'
I am using ubuntu 20.04 and in my case, the issue was with the installation part. I followed automatic installation steps in official documentation.
After searching for a while, I tried the manual approach described in the same documentation, work like a magic for me.
For some people this might be the case as it was for me so this might help someone.I am noob in writing such things so bear with me.
So got this error
Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: org.elasticsearch.cli.UserException: unable to create temporary keystore a>
Likely root cause: java.nio.file.AccessDeniedException: /etc/elasticsearch/elasticsearch.keystore.tmp
elasticsearch.service: Failed with result 'exit-code'.
elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
for ubuntu 20.04,
workaround for this was to run these two commands:
sudo chmod g+w /etc/elasticsearch
Above command changes file permissions (allowing) for creating keystroke manually.And below command create that manually.
sudo -u elasticsearch -s /usr/share/elasticsearch/bin/elasticsearch-keystore create
I also faced the same problem
I checked the elasticsearch service status
sudo systemctl start elasticsearch
Next, run the code below to determine the cause of the error:
journalctl -xe
I see a lots of line about Performance analyzer
Dec 20 21:07:37 my performance-analyzer-agent-cli[13112]: INFO: Single batch : No bind variables have been provided with a single statement batch execution. This may be due to accidental API misuse
Dec 20 21:07:37 my performance-analyzer-agent-cli[13112]: Dec 20, 2019 9:07:37 PM org.jooq.tools.JooqLogger info
Dec 20 21:07:37 my performance-analyzer-agent-cli[13112]: INFO: Single batch : No bind variables have been provided with a single statement batch execution. This may be due to accidental API misuse
Dec 20 21:07:42 my performance-analyzer-agent-cli[13112]: Dec 20, 2019 9:07:42 PM org.jooq.tools.JooqLogger info
Dec 20 21:07:42 my performance-analyzer-agent-cli[13112]: INFO: Single batch : No bind variables have been provided with a single statement batch execution. This may be due to accidental API misuse
Dec 20 21:07:42 my performance-analyzer-agent-cli[13112]: Dec 20, 2019 9:07:42 PM org.jooq.tools.JooqLogger info
Dec 20 21:07:42 my performance-analyzer-agent-cli[13112]: INFO: Single batch : No bind variables have been provided with a single statement batch execution. This may be due to accidental API misuse
Dec 20 21:07:47 my performance-analyzer-agent-cli[13112]: Dec 20, 2019 9:07:47 PM org.jooq.tools.JooqLogger info
Dec 20 21:07:47 my performance-analyzer-agent-cli[13112]: INFO: Single batch : No bind variables have been provided with a single statement batch execution. This may be due to accidental API misuse
Dec 20 21:07:47 my performance-analyzer-agent-cli[13112]: Dec 20, 2019 9:07:47 PM org.jooq.tools.JooqLogger info
Dec 20 21:07:47 my performance-analyzer-agent-cli[13112]: INFO: Single batch : No bind variables have been provided with a single statement batch execution. This may be due to accidental API misuse
Dec 20 21:07:52 my performance-analyzer-agent-cli[13112]: Dec 20, 2019 9:07:52 PM org.jooq.tools.JooqLogger info
Dec 20 21:07:52 my performance-analyzer-agent-cli[13112]: INFO: Single batch : No bind variables have been provided with a single statement batch execution. This may be due to accidental API misuse
Dec 20 21:07:52 my performance-analyzer-agent-cli[13112]: Dec 20, 2019 9:07:52 PM org.jooq.tools.JooqLogger info
Dec 20 21:07:52 my performance-analyzer-agent-cli[13112]: INFO: Single batch : No bind variables have been provided with a single statement batch execution. This may be due to accidental API misuse
Dec 20 21:07:57 my performance-analyzer-agent-cli[13112]: Dec 20, 2019 9:07:57 PM org.jooq.tools.JooqLogger info
Dec 20 21:07:57 my performance-analyzer-agent-cli[13112]: INFO: Single batch : No bind variables have been provided with a single statement batch execution. This may be due to accidental API misuse
Dec 20 21:07:57 my performance-analyzer-agent-cli[13112]: Dec 20, 2019 9:07:57 PM org.jooq.tools.JooqLogger info
Dec 20 21:07:57 my performance-analyzer-agent-cli[13112]: INFO: Single batch : No bind variables have been provided with a single statement batch execution. This may be due to accidental API misuse
Dec 20 21:08:02 my performance-analyzer-agent-cli[13112]: Dec 20, 2019 9:08:02 PM org.jooq.tools.JooqLogger info
Dec 20 21:08:02 my performance-analyzer-agent-cli[13112]: INFO: Single batch : No bind variables have been provided with a single statement batch execution. This may be due to accidental API misuse
Dec 20 21:08:02 my performance-analyzer-agent-cli[13112]: Dec 20, 2019 9:08:02 PM org.jooq.tools.JooqLogger info
Dec 20 21:08:02 my performance-analyzer-agent-cli[13112]: INFO: Single batch : No bind variables have been provided with a single statement batch execution. This may be due to accidental API misuse
Dec 20 21:08:07 my performance-analyzer-agent-cli[13112]: Dec 20, 2019 9:08:07 PM org.jooq.tools.JooqLogger info
Dec 20 21:08:07 my performance-analyzer-agent-cli[13112]: INFO: Single batch : No bind variables have been provided with a single statement batch execution. This may be due to accidental API misuse
Dec 20 21:08:07 my performance-analyzer-agent-cli[13112]: Dec 20, 2019 9:08:07 PM org.jooq.tools.JooqLogger info
Dec 20 21:08:07 my performance-analyzer-agent-cli[13112]: INFO: Single batch : No bind variables have been provided with a single statement batch execution. This may be due to accidental API misuse
This makes problem to analyze the issue
So First I tried to stop it so I found a link
in /usr/lib/systemd/system/opendistro-performance-analyzer.service
under [Service] add StandardOutput=null
after that reload systemd via ‘/bin/systemctl daemon-reload’ for it to take affect
For more Detail follow the below link
https://discuss.opendistrocommunity.dev/t/performance-analyzer-agent-cli-spamming-syslog-in-od-1-3-0/2040/4
Now the picture was clear easily I found the issue that There were duplicate properties in the elasticsearch.yml file that I forgot to comment on. I commented out the duplicate property and restart the elasticsearch service.
sudo systemctl start elasticsearch
sudo systemctl status elasticsearch
That's all.
I hope It will help.
I had to also disable security in /etc/elasticsearch/elasticsearch.yml
xpack.security.enabled: false
When I restarted my cluster, ambari didn't start because of a db check failed config:
sudo service ambari-server restart --skip-database-check
Using python /usr/bin/python
Restarting ambari-server
Waiting for server stop...
Ambari Server stopped
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari Server is starting with the database consistency check skipped. Do not make any changes to your cluster topology or perform a cluster upgrade until you correct the database consistency issues. See "/var/log/ambari-server/ambari-server-check-database.log" for more details on the consistency issues.
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start.....................
DB configs consistency check failed. Run "ambari-server start --skip-database-check" to skip. You may try --auto-fix-database flag to attempt to fix issues automatically. If you use this "--skip-database-check" option, do not make any changes to your cluster topology or perform a cluster upgrade until you correct the database consistency issues. See /var/log/ambari-server/ambari-server-check-database.log for more details on the consistency issues.
ERROR: Exiting with exit code -1.
REASON: Ambari Server java process has stopped. Please check the logs for more information.
I looked in the logs in "/var/log/ambari-server/ambari-server-check-database.log", and I saw:
2017-08-23 08:16:13,445 INFO - Checking Topology tables
2017-08-23 08:16:13,447 ERROR - Your topology request hierarchy is not complete for each row in topology_request should exist at least one raw in topology_logical_request, topology_host_request, topology_host_task, topology_logical_task.
I tried both options --auto-fix-database and --skip-database-check, it didn't work.
It seems that postgresql didn't start correctly, and even if in the log of Ambari there was no mention of postgresql not started or not available, but it was weird that ambari couldn't access to the topology configuration stored in it.
sudo service postgresql restart
Stopping postgresql service: [ OK ]
Starting postgresql service: [ OK ]
It did the trick:
sudo service ambari-server restart
Using python /usr/bin/python
Restarting ambari-server
Ambari Server is not running
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start.........
Server started listening on 8080
I am having some trouble with running hadoop jobs in both pseudo cluster and in cluster mode under ubuntu 16.04.
While running a vanila hadoop/hdfs installation - my hadoop user gets
logged out and all of the processes that are run by this user are closed.
I don't see anything indicating in logs (/var/log/systemd, journalctl or
dmesg) that explains why the user gets logged out.
Seems like I am not the only who has problems with this or similar issue:
https://stackoverflow.com/questions/38288162/in-ubuntu-16-04-running-hadoop-jar-laptop-gets-rebooted
Note: creating special hadoop user hadn't actually solved the problem in my case - but limited the logouts to the dedicated user.
https://askubuntu.com/questions/784591/ubuntu-16-04-kills-session-when-resource-usage-is-extremely-high
Is it possible that some problem around the UserGroupInformation class
(that can under some circumstances cause a logout), with maybe some changes in systemd in ubuntu 16.04 can cause this behavior?
The last lines of hadoop log that I get before the logout:
...
16/07/13 16:45:37 DEBUG ipc.ProtobufRpcEngine: Call: getJobReport took 4ms
16/07/13 16:45:37 DEBUG security.UserGroupInformation: PrivilegedAction
as:hduser (auth:SIMPLE)
from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:320)
16/07/13 16:45:37 DEBUG ipc.Client: IPC Client (1360814716) connection to
laptop/127.0.1.1:37339 from hduser sending #375
16/07/13 16:45:37 DEBUG ipc.Client: IPC Client (1360814716) connection to
laptop/127.0.1.1:37339 from hduser got value #375
16/07/13 16:45:37 DEBUG ipc.ProtobufRpcEngine: Call: getJobReport took 2ms
Terminated
hduser#laptop:~$ 16/07/13 16:45:37 DEBUG ipc.Client: stopping client from
cache: org.apache.hadoop.ipc.Client#4e7ab839
exit
journalctl:
Jul 12 16:06:44 laptop systemd-logind[978]: Removed session 7.
Jul 12 16:06:44 laptop systemd-logind[978]: Removed session 6.
Jul 12 16:06:44 laptop systemd-logind[978]: Removed session 5.
Jul 12 16:06:44 laptop systemd-logind[978]: Removed session 8.
syslog:
Jul 12 16:06:43 laptop systemd[4172]: Stopped target Default.
Jul 12 16:06:43 laptop systemd[4172]: Reached target Shutdown.
Jul 12 16:06:44 laptop systemd[4172]: Starting Exit the Session...
Jul 12 16:06:44 laptop systemd[4172]: Stopped target Basic System.
Jul 12 16:06:44 laptop systemd[4172]: Stopped target Sockets.
Jul 12 16:06:44 laptop systemd[4172]: Stopped target Paths.
Jul 12 16:06:44 laptop systemd[4172]: Stopped target Timers.
Jul 12 16:06:44 laptop systemd[4172]: Received SIGRTMIN+24 from PID
10101 (kill).
Jul 12 16:06:44 laptop systemd[1]: Stopped User Manager for UID 1001.
Jul 12 16:06:44 laptop systemd[1]: Removed slice User Slice of hduser.
I also had the problem. It took me time, but I found the solution here: https://unix.stackexchange.com/questions/293069/all-services-of-a-user-are-killed-when-running-multiple-services-under-this-user
Basically, some hadoop processes just stop, because why not. But systemd seems to kill all user's process when he see a service's process dying.
The fix is to add
[login]
KillUserProcesses=no
to /etc/systemd/logind.confand reboot.
I had multiple ubuntu's version to debug the problem, and the fix seems to works only on ubuntu 16.04.
I had the same issue. I was using Apache APEX which is hadoop native. While killing any APEX application my system used to log me out.
Solution : Replace the kill file (present in /bin/kill) of Ubuntu 16 with kill file of Ubuntu 14.
Everything works smoothly like before upgrade for me.
I had the same problem too. Finally, I found /bin/kill in ubuntu16.04 has bug in killing process group can solve this problem.
If pid is less than -1, then sig is sent to every process in the process group whose ID is -pid
Because of the bug in procps-ng-3.3.10, kill the process group whose ID starts with 1, invoked by bin/yarn application -kill AppID, will cause the user logouts.
The problem is solved after replacing /bin/kill with the new kill compiled from procps-ng-3.3.12.
tar xJf procps-ng-3.3.12.tar.xz
cd procps-ng-3.3.12
./configure
sudo cp .lib/kill /bin/kill
sudo chown root:root /bin/kill
sudo cp proc/.libs/libprocps.so.6.0.0 /lib/x86_64-linux/gnu/
sudo chown root:root /lib/x86_64-linux-gnu/libprocps.so.6.0.0
I am running a 16 GB Macbook pro with El Capitan OS. I installed the cloudera docker image using
docker pull cloudera/quickstart:latest
docker run --privileged=true --hostname=quickstart.cloudera -t -i 9f3ab06c7554 /usr/bin/docker-quickstart
the image boots fine, and I can see most services starting up
Started Hadoop historyserver: [ OK ]
starting nodemanager, logging to /var/log/hadoop-yarn/yarn-yarn-nodemanager-quickstart.cloudera.out
Started Hadoop nodemanager: [ OK ]
starting resourcemanager, logging to /var/log/hadoop-yarn/yarn-yarn-resourcemanager-quickstart.cloudera.out
Started Hadoop resourcemanager: [ OK ]
starting master, logging to /var/log/hbase/hbase-hbase-master-quickstart.cloudera.out
Started HBase master daemon (hbase-master): [ OK ]
starting rest, logging to /var/log/hbase/hbase-hbase-rest-quickstart.cloudera.out
Started HBase rest daemon (hbase-rest): [ OK ]
starting thrift, logging to /var/log/hbase/hbase-hbase-thrift-quickstart.cloudera.out
Started HBase thrift daemon (hbase-thrift): [ OK ]
Starting Hive Metastore (hive-metastore): [ OK ]
Started Hive Server2 (hive-server2): [ OK ]
Starting Sqoop Server: [ OK ]
Sqoop home directory: /usr/lib/sqoop2
Some failures as well
Failure to start Spark history-server (spark-history-server[FAILED]n value: 1
Starting Hadoop HBase regionserver daemon: starting regionserver, logging to /var/log/hbase/hbase-hbase-regionserver-quickstart.cloudera.out
hbase-regionserver.
Starting hue: [FAILED]
But once the bootup is complete, if I try to run anything it fails
for example trying to run spark-shell
[root#quickstart /]# spark-shell
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000b0000000, 357892096, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 357892096 bytes for committing reserved memory.
# An error report file with more information is saved as:
# //hs_err_pid3113.log
or trying to run hive shell
[root#quickstart /]# hive
Unable to determine Hadoop version information.
'hadoop version' returned:
Hadoop 2.6.0-cdh5.5.0 Subversion http://github.com/cloudera/hadoop -r fd21232cef7b8c1f536965897ce20f50b83ee7b2 Compiled by jenkins on 2015-11-09T20:37Z Compiled with protoc 2.5.0 From source with checksum 98e07176d1787150a6a9c087627562c This command was run using /usr/jars/hadoop-common-2.6.0-cdh5.5.0.jar
[root#quickstart /]#
My question is what can I do so that I can run the spark-shell and the hive shell successfully?
Since you are running Docker on a Mac, Docker runs under VirtualBox, not directly with the Mac's memory. (Same thing would happen in Windows).
You probably wouldn't get these errors on a Linux host since Docker isn't virtualized there.
The Cloudera quickstart vm recommends 8Gb of memory to run all the services and the docker vm is only 512Mb, I think.
The solution would be to stop the docker-machine instance, open VirtualBox, and increase the memory size of the "default" VM to the necessary amount.