I am unable to connect to tableau using the cloudera hive driver using kerberos authentication, even after configuring the kerb5.conf with appropriate info.
Issue is with the encryption property used in the /etc/krb5.conf file. I have removed the following lines before made it work.
default_tgs_enctypes = rc4-hmac default_tkt_enctypes = rc4-hmac permitted_enctypes = rc4-hmac
the issue is with the encryption property used in the /etc/krb5.conf file. I have removed the following lines before made it work.
default_tgs_enctypes = rc4-hmac
default_tkt_enctypes = rc4-hmac
permitted_enctypes = rc4-hmac
the following is the complete (sudo) contents of /etc/krb5.conf file.
[libdefaults]
default_realm = NA.CORP.xxx.com
dns_lookup_kdc = true
dns_lookup_realm = false
ticket_lifetime = 86400
renew_lifetime = 604800
forwardable = true
udp_preference_limit = 1
kdc_timeout = 3000
[realms]
NA.xxx.COM = {
kdc = xxx.com
admin_server = xxx1.com
}
default_cc_type = FILE
default_ccache_name = FILE:/tmp/krb5cc_501
[domain_realm]
Related
Hi I am new to Hive and kerberos.
I have some hive jobs which run more then life time of ticket. how can I configure hive so that when I start hive shell if ticket is not cached it automatically request for ticket. After acquiring ticket lets suppose if ticket expire is the middle then automatically acquire new one and also I may have simultaneous job running by same user so may be one cached ticket can be used by many jobs.
Any Solutions or direction to look upon will be highly appreciated.
Thanks in Advance.
I am looking for a solution in which hive cli or shell can automatically acquire or renew Kerberos credentials.
What you need to look into is Java Authentication and Authorization Service (JAAS)
It's how to enable java to use kerberos without adding anything to your code. Specifically here you might want to look at how beeline uses kerberos config as an example.
Create setEnv.sh file and save it inside "bin" folder. Paste below
content inside it:
export HADOOP_HOME=/home/user/beeline/hadoop-2.5.1
export HIVE_HOME=/home/user/beeline/apache-hive-1.2.1-bin
export JAVA_HOME=/home/user/beeline/jre
PATH=$PATH:$HIVE_HOME/bin:$JAVA_HOME/bin
export HADOOP_OPTS="$HADOOP_OPTS -Dsun.security.krb5.debug=true -Djava.security.krb5.conf=/home/user/beeline/conf/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=false -Djava.security.auth.login.config=/home/user/beeline/conf/jaas.conf"
jaas.conf File:
Create and save jaas.conf file under conf folder
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=false
useTicketCache=true;
};
krb5.conf File:
Create and save krb5.conf File under conf folder. Modify this file as
per your environment.
[logging]
default = FILE:~/krb5libs.log
kdc = FILE:~/krb5kdc.log
admin_server = FILE:~/kadmind.log
kdc_rotate = {"period"=>"1d", "versions"=>200}
admin_server_rotate = {"period"=>"1d", "versions"=>201}
[libdefaults]
default_realm = DOMAIN.COM
dns_lookup_realm = false
dns_lookup_kdc = false
forwardable = true
renew_lifetime = 30d
ticket_lifetime = 30d
renewable = yes
service = yes
kdc_timeout = 5000
default_tgs_enctypes = aes256-cts-hmac-sha1-96 aes128-cts arcfour-hmac-md5 des-cbc-crc des-cbc-md5 des-hmac-sha1
default_tkt_enctypes = aes256-cts-hmac-sha1-96 aes128-cts arcfour-hmac-md5 des-cbc-crc des-cbc-md5 des-hmac-sha1
allow_weak_crypto = yes
udp_preference_limit = 1
[realms]
DOMAIN.COM = {
kdc = kdcserver.domain.com:88
default_domain = domain.com
}
[domain_realm]
.domain.com = DOMAIN.COM
domain.com = DOMAIN.COM
[appdefaults]
pam = {
debug = false
forwardable = true
renew_lifetime = 36000
ticket_lifetime = 36000
krb4_convert = false
}
It should be noted that the above config doesn't use a renewable kerberos ticket but that's just and example and you can make it renewable.
I'm currently setting up Kerberos for an Ambari Hortonworks environment. For a number of reasons, I'm unable to use a distinct domain name as the realm name for this install. This is strange because - from what I read - the realm name is just set to the domain name by convention. In theory it can be any ASCII string.
For this Ambari environment I'm essentially trying to set up Kerberos where
[libdefaults]
default_realm = FOOBAR
In fact, my current krb5.conf looks something like this:
[libdefaults]
renew_lifetime = 7d
forwardable = true
default_realm = FOOBAR
ticket_lifetime = 24h
dns_lookup_realm = false
dns_lookup_kdc = false
rdns = false
default_ccache_name = /tmp/krb5cc_%{uid}
#default_tgs_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5
#default_tkt_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5
[domain_realm]
#Not sure how to use this mapping property in this case
FOOBAR = FOOBAR
.FOOBAR = FOOBAR
[logging]
default = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
kdc = FILE:/var/log/krb5kdc.log
[realms]
FOOBAR = {
admin_server = {admin ip adress}
kdc = {kdc ip address}
}
/etc/hosts
{kdc ip address} FOOBAR kdc
One ought to be able to short-circuit the DNS check with the hosts file. But I can't seem to get Kerberos working this way. All the documentation I found so far online describes the nice, safe setup following the DNS convention.
Can anyone point to a tutorial, or describe the necessary steps to make Kerberos work without a domain name?
Given the lack of helpful response I'll just share what I end up using (works but might not be optimal)
[libdefaults]
renew_lifetime = 7d
forwardable = true
default_realm = FOOBAR
ticket_lifetime = 24h
dns_lookup_realm = false
dns_lookup_kdc = false
default_ccache_name = /tmp/krb5cc_%{uid}
#default_tgs_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5
#default_tkt_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5
[domain_realm]
FOOBAR = FOOBAR
.FOOBAR = FOOBAR
[logging]
default = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
kdc = FILE:/var/log/krb5kdc.log
[realms]
FOOBAR = {
admin_server = {admin_server ip}
kdc = {kdc_server ip}
}
In addition, be sure to add the ip addresses and hostnames for all machines in the cluster to /etc/hosts files.
I am new to Hadoop and I made a Hadoop cluster with 3 centos machine in my VMware, and I also kerberosing the cluster, it works fine in the VMware, I can reach the URL by FireFox in CenotOS machine
However, when I try to reach the page outside the VMware(in my windows machine) it always shows like this
I can ping each other by IP or hostname(I have set the hosts file)
I have got the ticket from KDC in my windows machine by MIT Kerberos, like this and when I type klist in my windows cmd, it showed the ticket.
I have set the firefox as suggested(as in centos I can reach the
page.)
what else should i set?
help please!
the ticket i got
this is my krb5.ini and krb5.conf in my windows and centos machine
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
dns_lookup_realm = false
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
renewable = true
rdns = false
default_realm = HADOOP.COM
[realms]
HADOOP.COM = {
kdc = master:88
admin_server = master:749
}
[domain_realm]
master = HADOOP.COM
slave1 = HADOOP.COM
slave2 = HADOOP.COM
I'm using ChefDK 0.3.5 on Windows 7 64bit. I'm having a bit of a problem trying to use Chef and WinRM to execute the
chef-client command on a remote server. The command I'm issuing is below:
C:\U\user1> knife winrm "fqdn:testserver.*" "netstat" -x 'domain\user1' -P 'password'
WARNING: Switching to Negotiate authentication, Basic does not support Domain Authentication
ERROR: RuntimeError: Error: Unencrypted communication not supported. Please check winrm configuration winrm/config/service AllowUnencrypted flag.
The result I got doesn't let me do what I want. So I googled it, and saw
something about not verifying SSL, or using :verify_peer, which did nothing
as well. So I executed the WinRM configuration commands again, in Command
Prompt, because PowerShell tells me their wrong.
C:\Users\user1>C:\WIndows\System32\cmd.exe /c winrm set winrm/config/winrs #{MaxMemoryPerShellMB="300"}
Winrs
AllowRemoteShellAccess = true
IdleTimeout = 7200000
MaxConcurrentUsers = 10
MaxShellRunTime = 2147483647
MaxProcessesPerShell = 25
MaxMemoryPerShellMB = 300
MaxShellsPerUser = 30
C:\Users\user1>C:\WIndows\System32\cmd.exe /c winrm set winrm/config #{MaxTimeoutms="1800000"}
Config
MaxEnvelopeSizekb = 500
MaxTimeoutms = 1800000
MaxBatchItems = 32000
MaxProviderRequests = 4294967295
Client
NetworkDelayms = 5000
URLPrefix = wsman
AllowUnencrypted = false
Auth
Basic = true
Digest = true
Kerberos = true
Negotiate = true
Certificate = true
CredSSP = false
DefaultPorts
HTTP = 5985
HTTPS = 5986
TrustedHosts
Service
RootSDDL = O:NSG:BAD:P(A;;GA;;;BA)(A;;GR;;;IU)S:P(AU;FA;GA;;;WD)(AU;SA;G
XGW;;;WD)
MaxConcurrentOperations = 4294967295
MaxConcurrentOperationsPerUser = 1500
EnumerationTimeoutms = 240000
MaxConnections = 300
MaxPacketRetrievalTimeSeconds = 120
AllowUnencrypted = true
Auth
Basic = false
Kerberos = true
Negotiate = true
Certificate = false
CredSSP = false
CbtHardeningLevel = Relaxed
DefaultPorts
HTTP = 5985
HTTPS = 5986
IPv4Filter = *
IPv6Filter = *
EnableCompatibilityHttpListener = false
EnableCompatibilityHttpsListener = false
CertificateThumbprint
AllowRemoteAccess = true
Winrs
AllowRemoteShellAccess = true
IdleTimeout = 7200000
MaxConcurrentUsers = 10
MaxShellRunTime = 2147483647
MaxProcessesPerShell = 25
MaxMemoryPerShellMB = 300
MaxShellsPerUser = 30
C:\Users\user1>C:\WIndows\System32\cmd.exe /c winrm set winrm/config/service#{AllowUnencrypted="true"}
Service
RootSDDL = O:NSG:BAD:P(A;;GA;;;BA)(A;;GR;;;IU)S:P(AU;FA;GA;;;WD)(AU;SA;GXGW;
;;WD)
MaxConcurrentOperations = 4294967295
MaxConcurrentOperationsPerUser = 1500
EnumerationTimeoutms = 240000
MaxConnections = 300
MaxPacketRetrievalTimeSeconds = 120
AllowUnencrypted = true
Auth
Basic = false
Kerberos = true
Negotiate = true
Certificate = false
CredSSP = false
CbtHardeningLevel = Relaxed
DefaultPorts
HTTP = 5985
HTTPS = 5986
IPv4Filter = *
IPv6Filter = *
EnableCompatibilityHttpListener = false
EnableCompatibilityHttpsListener = false
CertificateThumbprint
AllowRemoteAccess = true
After executing these commands, I try again, and ChefDK looks at me like I'm
stupid. I'm not sure why Chef is trying to use basic auth then I give it
domain credentials. This also worked previously, but I had to get a new
computer and my notes, and installations were lost. Are there any ideas on
what I could be missing? If I'm missing any information let me know and I'll update my question.
The client command tells you:
Unencrypted communication not supported. Please check winrm configuration winrm/config/service AllowUnencrypted flag.
Your configuration, as printed out from the result of the timeout command, tells you:
Config
MaxEnvelopeSizekb = 500
MaxTimeoutms = 1800000
MaxBatchItems = 32000
MaxProviderRequests = 4294967295
Client
NetworkDelayms = 5000
URLPrefix = wsman
AllowUnencrypted = false
You need to set the AllowUnencrypted flag to true, by running the command:
winrm set winrm/config/service #{AllowUnencrypted="true"}
The latest github versions of knife-windows add commands for creating ssl certs and tightening up your connection, which you will hopefully want to do when you're ready.
Note: I'm using Active Directory authentication - there's no way around this.
I'm trying to use the following script:
require 'winrm'
IP=ARGV[0]
USER=ARGV[1]
endpoint = "http://#{IP}:5985/wsman"
winrm = WinRM::WinRMWebService.new endpoint, :plaintext, user: USER, pass: 'secret', basic_auth_only: true
winrm.set_timeout 1800
installer_file = "c:\\Users\\#{USER}\\Downloads\\chef_client.msi"
installer_cmd = "msiexec /qn /i #{installer_file} ADDLOCAL=\"ChefServiceFeature,ChefClientFeature\""
puts 'Installating Chef'
puts installer_cmd
install_output = winrm.cmd installer_cmd
sleep 300
puts 'Configuring Chef'
winrm.cmd %Q(mv c:\\Users\\#{USER}\\Downloads\\client.rb c:\\chef\\client.rb)
winrm.cmd %Q(mv c:\\Users\\#{USER}\\Downloads\\chef-validator.pem c:\\chef\\validation.pem)
winrm.cmd 'net start chef-client || net start chef-client'
When I do: ruby chef_windows_boostrap.rb test168.domain.com ctote, I get the following output:
c:\opscode\chef>ruby chef_windows_boostrap.rb test168.domain.com ctote
Installating Chef
msiexec /qn /i c:\Users\ctote\Downloads\chef_client.msi ADDLOCAL="ChefServiceFeature,ChefClientFeature"
C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/winrm-1.1.3/lib/winrm/http/transport.rb:48:in `send_request': Bad HTTP response returned from server (401). (WinRM::WinRMHTTPTransportError)
from C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/winrm-1.1.3/lib/winrm/winrm_service.rb:368:in `send_message'
from C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/winrm-1.1.3/lib/winrm/winrm_service.rb:113:in `open_shell'
from C:/opscode/chef/embedded/lib/ruby/gems/1.9.1/gems/winrm-1.1.3/lib/winrm/winrm_service.rb:239:in `run_cmd'
from chef_windows_boostrap.rb:14:in `<main>'
below is my winrm config settings:
c:\opscode\chef>winrm get winrm/config
Config
MaxEnvelopeSizekb = 150
MaxTimeoutms = 1800000
MaxBatchItems = 32000
MaxProviderRequests = 4294967295
Client
NetworkDelayms = 5000
URLPrefix = wsman
AllowUnencrypted = true
Auth
Basic = true
Digest = true
Kerberos = true
Negotiate = true
Certificate = true
CredSSP = false
DefaultPorts
HTTP = 5985
HTTPS = 5986
TrustedHosts
Service
RootSDDL = O:NSG:BAD:P(A;;GA;;;BA)S:P(AU;FA;GA;;;WD)(AU;SA;GWGX;;;WD)
MaxConcurrentOperations = 4294967295
MaxConcurrentOperationsPerUser = 15
EnumerationTimeoutms = 60000
MaxConnections = 25
MaxPacketRetrievalTimeSeconds = 120
AllowUnencrypted = true
Auth
Basic = true
Kerberos = true
Negotiate = true
Certificate = false
CredSSP = false
CbtHardeningLevel = Relaxed
DefaultPorts
HTTP = 5985
HTTPS = 5986
IPv4Filter = *
IPv6Filter = *
EnableCompatibilityHttpListener = false
EnableCompatibilityHttpsListener = false
CertificateThumbprint
Winrs
AllowRemoteShellAccess = true [Source="GPO"]
IdleTimeout = 180000
MaxConcurrentUsers = 5
MaxShellRunTime = 2147483647
MaxProcessesPerShell = 15
MaxMemoryPerShellMB = 300
MaxShellsPerUser = 5
At this point, I'm pretty lost on what to try - anyone have some suggestions? Is there a way I can wireshark this to help debug or something?
The following scripts has to be run on the windows server.
winrm set winrm/config/client/auth #{Basic="true"}
winrm set winrm/config/service/auth #{Basic="true"}
winrm set winrm/config/service #{AllowUnencrypted="true"}
Source: http://talk.manageiq.org/t/steps-to-join-hyper-v-vmm-to-manageiq/151