How to configure Hive Cli to automatically get the kerberos ticket and renew/request new if expires by it own - hadoop

Question

Hi I am new to Hive and kerberos.
I have some hive jobs which run more then life time of ticket. how can I configure hive so that when I start hive shell if ticket is not cached it automatically request for ticket. After acquiring ticket lets suppose if ticket expire is the middle then automatically acquire new one and also I may have simultaneous job running by same user so may be one cached ticket can be used by many jobs.
Any Solutions or direction to look upon will be highly appreciated.
Thanks in Advance.
I am looking for a solution in which hive cli or shell can automatically acquire or renew Kerberos credentials.

Answer 1

What you need to look into is Java Authentication and Authorization Service (JAAS)
It's how to enable java to use kerberos without adding anything to your code. Specifically here you might want to look at how beeline uses kerberos config as an example.
Create setEnv.sh file and save it inside "bin" folder. Paste below
content inside it:
export HADOOP_HOME=/home/user/beeline/hadoop-2.5.1
export HIVE_HOME=/home/user/beeline/apache-hive-1.2.1-bin
export JAVA_HOME=/home/user/beeline/jre
PATH=$PATH:$HIVE_HOME/bin:$JAVA_HOME/bin
export HADOOP_OPTS="$HADOOP_OPTS -Dsun.security.krb5.debug=true -Djava.security.krb5.conf=/home/user/beeline/conf/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=false -Djava.security.auth.login.config=/home/user/beeline/conf/jaas.conf"
jaas.conf File:
Create and save jaas.conf file under conf folder
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=false
useTicketCache=true;
};
krb5.conf File:
Create and save krb5.conf File under conf folder. Modify this file as
per your environment.
[logging]
default = FILE:~/krb5libs.log
kdc = FILE:~/krb5kdc.log
admin_server = FILE:~/kadmind.log
kdc_rotate = {"period"=>"1d", "versions"=>200}
admin_server_rotate = {"period"=>"1d", "versions"=>201}
[libdefaults]
default_realm = DOMAIN.COM
dns_lookup_realm = false
dns_lookup_kdc = false
forwardable = true
renew_lifetime = 30d
ticket_lifetime = 30d
renewable = yes
service = yes
kdc_timeout = 5000
default_tgs_enctypes = aes256-cts-hmac-sha1-96 aes128-cts arcfour-hmac-md5 des-cbc-crc des-cbc-md5 des-hmac-sha1
default_tkt_enctypes = aes256-cts-hmac-sha1-96 aes128-cts arcfour-hmac-md5 des-cbc-crc des-cbc-md5 des-hmac-sha1
allow_weak_crypto = yes
udp_preference_limit = 1
[realms]
DOMAIN.COM = {
kdc = kdcserver.domain.com:88
default_domain = domain.com
}
[domain_realm]
.domain.com = DOMAIN.COM
domain.com = DOMAIN.COM
[appdefaults]
pam = {
debug = false
forwardable = true
renew_lifetime = 36000
ticket_lifetime = 36000
krb4_convert = false
}
It should be noted that the above config doesn't use a renewable kerberos ticket but that's just and example and you can make it renewable.