I am using JRE - 1.8 and databricks-connect version 9.1.15 with python version - Python 3.7.9
openjdk version "1.8.0_41"
OpenJDK Runtime Environment (build 1.8.0_41-b04)
Python 3.7.9 (tags/v3.7.9:13c94747c7, Aug 17 2020, 16:30:00)
I see that command - databricks-connect always returns below error even when I invoke commands-
databricks-connect configure or databricks-connect test
"Access is denied."
I do not receive prompt for configuring cluster details.
Did I miss anything in configuration?
Note- databricks cli is working fine with above setup
As per official documentation, The minor version of your client Python installation must be the same as the minor Python version of your Databricks cluster. The table shows the Python version installed with each Databricks Runtime.
Related
I have
OS Red Hat Enterprise Linux Server release 7.4 (Maipo)
Ambari Version 2.5.1.0
HDP 2.6
Tried to start Zeppelin Server returned error:
raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/hdp/current/zeppelin-server/bin/zeppelin-daemon.sh start >> /var/log/zeppelin/zeppelin-setup.log' returned 1.
after step
2018-02-28 22:09:17,946 - Execute['/usr/hdp/current/zeppelin-server/bin/zeppelin-daemon.sh start >> /var/log/zeppelin/zeppelin-setup.log'] {'user': 'zeppelin'}
In log found exception:
ZEPPELIN_CLASSPATH: ::/usr/hdp/current/zeppelin-server/lib/interpreter/*:/usr/hdp/current/zeppelin-server/lib/*:/usr/hdp/current/zeppelin-server/*::/usr/hdp/current/zeppelin-server/conf
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
Error: Could not find or load main class org.apache.zeppelin.server.ZeppelinServer
How solve this problem?
Checked folder
/usr/hdp/current/zeppelin-server
erased folders /conf
reinstall zeppelin:
yum erase zeppelin
after more installed
yum install zeppelin
Tried more start Zeppelin - OK
When I try and use elastic beanstalk from the command line (ie eb create project) I got the following error.
Select a platform version.
1) Docker 17.03.2-ce
2) Docker 1.12.6
3) Docker 1.11.2
4) Docker 1.9.1
5) Docker 1.7.1
6) Docker 1.6.2
7) Docker 1.5.0
(default is 1): 1
Note: Elastic Beanstalk now supports AWS CodeCommit; a fully-managed source control service. To learn more, see Docs: https://aws.amazon.com/codecommit/
Do you wish to continue with CodeCommit? (y/N) (default is n): n
Do you want to set up SSH for your instances?
(Y/n): y
Select a keypair.
1) patientplatypus
2) phoenixvuenewsly
3) [ Create new KeyPair ]
(default is 2): 1
patientplatypus:~/Documents/newsly_project:11:56:37$eb create newsly_project_production
ERROR: Platform Docker 17.03.2-ce does not appear to be valid
patientplatypus:~/Documents/newsly_project:11:57:44$docker --version
Docker version 17.09.0-ce, build afdb6d4
I thought the above seemed understandable enough, at first. I have version 17.09.0-ce of Docker, but I need version 17.03.2-ce. It would seem reasonable that AWS would be a few versions behind the latest version of Docker if nothing else than to double check system integration and security.
However, when it came time to find version 17.03.2-ce I found it almost impossible. https://github.com/moby/moby/releases points me to 17.03.02-ce as a direct link on the downloads page, which again downloads the wrong version. I tried downloading the link to the static binaries, however that was very confusing. This page https://docs.docker.com/engine/installation/linux/docker-ce/binaries/#next-steps says that in order to get the static binaries to compile I should run the command sudo cp docker/docker /usr/local/bin/, however nowhere in that static binary folder is there a folder labeled docker.
Meanwhile this page https://docs.docker.com/docker-for-mac/release-notes/#docker-community-edition-17090-ce-mac32-2017-10-02-stable, goes from release Docker Community Edition 17.03.1-ce-mac12, 2017-05-12 (stable) to release Docker Community Edition 17.06.0-ce-mac18, 2017-06-28 (stable). Not only does this skip the one release I need, but it seems to have versions that are ahead of the most recent release at https://github.com/moby/moby/releases.
I'm very very confused. It should not be as difficult as this to get the currently stable version of Docker that runs with AWS. Can someone let me know how to resolve this issue?
EDIT:
I spun up my Linux Mint 18 (Ubuntu 16.04) and installed docker to try and get this to work. As it happens linux ALSO installs version Docker version 17.09.0-ce, however this one works with the 1) Docker 17.03.2-ce elastic beanstalk option. All the same commands on a linux environment running the same version of docker now run on eb cli. Therefore I am retitling this post as an EB CLI bug for mac. My question now is, where do I submit a bug report to AWS? I have searched for a github repository for them, but have not found it, and I think that most of their reporting through aws.amazon.com is for enterprise level customers.
Thanks!
AWS Elastic Beanstalk - Cannot Clone With Latest Platform or eb upgrade from Ruby 2.1 to Ruby 2.2
I've been smashing my head on this one. Back in May, AWS announced that their Ruby Elastic Environments now offer Ruby-2.2 (e.g. ruby-2.2-(passenger-standalone) or ruby-2.2-(puma)). I can't upgrade my existing ruby-2.1 environments to ruby-2.2. It appears I have to recreate them completely...that seems silly? Anyone else out there experiencing this? Am I just missing something simple?
Extra Information
I've been gleefully using ruby-2.1-(passenger-standalone) for several months in my staging and production environments. Now I'd like to upgrade them to the latest ruby-2.2 platform. The AWS documentation says this is pretty trivial, in fact some of their documentation even appear to state that you can use an eb clone <env-name> --update. That flag doesn't exist in the EB CLI 3.4.5 that I'm using :( Additionally, the web console has a Clone with Latest Platform option menu item, yet it is disabled.
It appears that all I can do to get to the latest Ruby-2.2 instances is to create a brand new environment from scratch. That's tremendously annoying.
So here's what I've tried...
$ eb status
Environment details for: staging
Application name: xyz
Region: us-west-2
Deployed Version: fbe7
Environment ID: someId
Platform: 64bit Amazon Linux 2015.03 v1.4.3 running Ruby 2.1 (Passenger Standalone)
Tier: WebServer-Standard
CNAME: bla.bla.bla.elasticbeanstalk.com
Updated: 2015-06-26 22:48:59.689000+00:00
Status: Ready
Health: Green
Great, confirms I'm using Ruby 2.1. So let's try to upgrade:
$ eb upgrade
Environment already on most recent platform version.
Okay, that's nice, so what about the platform itself:
$ eb platform select
It appears you are using Ruby. Is this correct?
(y/n): y
Select a platform version.
1) Ruby 2.2 (Puma)
2) Ruby 2.2 (Passenger Standalone)
3) Ruby 2.1 (Puma)
4) Ruby 2.1 (Passenger Standalone)
5) Ruby 2.0 (Puma)
6) Ruby 2.0 (Passenger Standalone)
7) Ruby 1.9.3
(default is 1): 2
So now, here's my platform settings:
$ eb platform show
Current default platform: Ruby 2.2 (Passenger Standalone)
New environments will be running: 64bit Amazon Linux 2015.03 v1.4.3 running Ruby 2.2 (Passenger Standalone)
Platform info for environment "staging":
Current: 64bit Amazon Linux 2015.03 v1.4.3 running Ruby 2.1 (Passenger Standalone)
Latest: 64bit Amazon Linux 2015.03 v1.4.3 running Ruby 2.1 (Passenger Standalone)
So New environments will take advantage of the Ruby-2.2 editions. That's great, but do I seriously have to retire my already configured Ruby-2.1 editions in lieu of a newly created Ruby-2.1 edition? Come on?
I found a solution for this, by using the aws cli (NOT the eb cli):
aws elasticbeanstalk update-environment --solution-stack-name "64bit Amazon Linux 2016.03 v2.1.0 running Ruby 2.3 (Puma)" --environment-name "dev-bg-123456" --region "us-east-1"
The aws cli can be installed with homebrew:
brew install awscli
My solution to this challenge was to:
Launch a brand new environment.
Create a Web Application or Worker.
Choose the environment you want, likely the latest greatest. In my case Passenger with Ruby-2.2.
I choose to deploy the Sample Application so that the new environment succeeds.
Choose the remaining options for your new environment and let it launch.
Upon successful launch, I set up all my environment variable; e.g. all my database, smtp, RAILS_ENV/RACK_ENV, etc. Then I go to my previously deployed applications, and simply deploy my latest quality version to the brand new environment.
One would agree that this is a lot more work than a simple eb upgrade. Lame sauce.
I found a different approach to this problem.
The way I made it work is similar to the comment above but I managed to update the existing beanstalk environment and not deploy code to a completely new one.
Steps I did:
Launched a completely new environment on beanstalk running the ruby version that I need, 2.3 (in my case I needed an update from Ruby 2.2 to Ruby 2.3)
SSH-ed into the new instance and installed all dependencies that my application needs (packages, dev tools etc.) - you can also do these in the .ebextensions directory
Created a custom AMI from the instance running ruby 2.3 with everything installed
Used the same command that Ralph posted but with some tweeks:
aws elasticbeanstalk update-environment --region "REGION" --environment-name "ENV_NAME" --solution-stack-name "64bit Amazon Linux 2018.03 v2.8.1 running Ruby 2.3 (Puma)" --option-settings Namespace=aws:autoscaling:launchconfiguration,OptionName=ImageId,Value="ami-
xxxxxxxx"
The AMI option-settings was key for me, as this helped me the AMI with ruby 2.3.7 installed to be used for launching the new instance on the existing environment.
Previously just hitting Ralph's suggested command failed on every try.
Error message from Ambari Server setup
Transaction check error:
file /etc/sudoers.d from install of ambari-agent-2.0.0-151.x86_64 conflicts with file from package sudo-1.8.6p7-13.el7.x86_64
Systems
Red Hat Enterprise Linux (RHEL) 7.1 (HVM)
m3.larget
Action
Running Ambari 2.0.0 server setup
Ambari Repo:
http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.0.0/ambari.repo
RHEL 7.x is not supported by Ambari 2.0.0. See OS requirements here:
http://docs.hortonworks.com/HDPDocuments/Ambari-2.0.0.0/bk_Installing_HDP_AMB/content/_operating_systems_requirements.html
I'm using RHEL 6.5 and proceeded as per the latest documentation.
I getting an error(the first time as well as in successive attempts) regarding the jdk installation during the ambari server set-up.
ambari-server setup
Using python /usr/bin/python2.6
Setup ambari-server
Checking SELinux...
SELinux status is 'enabled'
SELinux mode is 'permissive'
WARNING: SELinux is set to 'permissive' mode and temporarily disabled.
OK to continue [y/n] (y)? y
Ambari-server daemon is configured to run under user 'root'. Change this setting [y/n] (n)? n
Adjusting ambari-server permissions and ownership...
Checking firewall...
Checking JDK...
[1] - Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[2] - Oracle JDK 1.6 + Java Cryptography Extension (JCE) Policy Files 6
[3] - Custom JDK
==============================================================================
Enter choice (1):1
JDK already exists, using /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz
Installing JDK to /usr/jdk64
Installation of JDK has failed: 'Fatal exception: Installation of JDK returned exit code 2, exit code 2'
JDK found at /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz. Would you like to re-download the JDK [y/n] (y)? y
jdk-7u67-linux-x64.tar.gz... 100% (135.8 MB of 135.8 MB)
Successfully re-downloaded JDK distribution to /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz
Installing JDK to /usr/jdk64
Installation of JDK was failed: 'Fatal exception: Installation of JDK returned exit code 2, exit code 2'
ERROR: Exiting with exit code 1.
REASON: Downloading or installing JDK failed: 'Fatal exception: Unable to install JDK. Please remove JDK, file found at /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz and re-run Ambari Server setup, exit code 1'. Exiting.
What am I missing ?
I had the same issue so I went went with the "yum install java-1.7.0-openjdk" option.
1.) Install openjdk: yum install java-1.7.0-openjdk
2.) Tell it to use the openjdk: ambari-server setup -j /usr/lib/jvm/jre-1.7.0-openjdk.x86_64
Good Luck!