Web interface login Apache Hadoop Cluster with Kerberos - hadoop

Question

I've a Docker stack with an Apache Hadoop (version 3.3.4) cluster, composed by one namenode and two datanodes, and a container with both Kerberos admin server and Kerberos kdc.
I'm trying to configure Kerberos authentication on the Apache Hadoop cluster.
The namenode and the datanodes connect correctly to the Kerberos container and to each other using the Kerberos prncipals. However, authentication to the web interface of the namenode doesn't work and I get the following error:
The details of my configuration are the following.
I have four container in my Docker stack:
The namenode with hostname "namenodehostname.host.com" and alias "namenode";
First datanode with hostname "datanode1hostname.host.com" and alias "datanode1";
Second datanode with hostname "datanode2hostname.host.com" and alias "datanode2";
Kerberos server and kdc with hostname "krb5.host.com" and alias "kerberos".
The namenode and the datanodes starts with a custom user "hadoop" created in the Dockerfile.
All four containers have the following /etc/hosts file:
namenode  namenodehostname.host.com
datanode1 datanode1hostname.host.com
datanode2 datanode2hostname.host.com
kerberos krb5.host.com
The file krb5.conf (in the namenode, in the datanodes and in the Kerberos container) is:
[libdefaults]
default_realm = TESTREALM 
# The following krb5.conf variables are only for MIT Kerberos.
kdc_timesync = 1
ccache_type = 4
forwardable = true
proxiable = true # The following encryption type specification will be used by MIT Kerberos
# if uncommented.  In general, the defaults in the MIT Kerberos code are
# correct and overriding these specifications only serves to disable new
# encryption types as they are added, creating interoperability problems.
#
# The only time when you might need to uncomment these lines and change
# the enctypes is if you have local software that will break on ticket
# caches containing ticket encryption types it doesn't know about (such as
# old versions of Sun Java). #       default_tgs_enctypes = des3-hmac-sha1
#       default_tkt_enctypes = des3-hmac-sha1
#       permitted_enctypes = des3-hmac-sha1 # The following libdefaults parameters are only for Heimdal Kerberos. fcc-mit-ticketflags = true 
[realms]
TESTREALM = {
kdc = krb5.host.com
admin_server = krb5.host.com
} 
[domain_realm]
.host.com = TESTREALM
host.com = TESTREALM
The file kdc.conf (in the Kerberos container) is:
[kdcdefaults]
    kdc_ports = 750,88 
[realms]
    TESTREALM = {
        database_name = /etc/krb5kdc/data/database/principal
        admin_keytab = FILE:/etc/krb5kdc/data/keytabs/kadm5.keytab
        acl_file = /etc/krb5kdc/kadm5.acl
        key_stash_file = /etc/krb5kdc/data/stashfile/stash
        kdc_ports = 750,88
        max_life = 10h 0m 0s
        max_renewable_life = 7d 0h 0m 0s
        master_key_type = des3-hmac-sha1
        #supported_enctypes = aes256-cts:normal aes128-cts:normal
        default_principal_flags = +preauth
    }
I created five principals in Kerberos container:
root/admin
nn/namenodehostname.host.com: used by the namenode
HTTP/namenodehostname.host.com: used by the namenode
dn/datanode1hostname.host.com: used by first datanode
dn/datanode2hostname.host.com: used by second datanode
All this principals except for the root/admin, are mapped on the user "hadoop" in the namenode and in the datanodes (see property hadoop.security.auth_to_local in the del core-site.xml file).
I also created a keytab file for each principal ending in *.host.com.
The namenode is configured with the following files:
core-site.xml
<configuration>
<property><name>fs.defaultFS</name><value>hdfs://namenodehostname.host.com:9000</value></property>
<property><name>hadoop.security.authentication</name><value>kerberos</value></property>
<property><name>hadoop.security.authorization</name><value>true</value></property>
<property><name>hadoop.rpc.protection</name><value>authentication</value></property>
<property><name>hadoop.security.auth_to_local</name><value>
RULE:[1:$1](nn/namenodehostname.host.com#TESTREALM)s/^.*$/hadoop/
RULE:[1:$1](dn/datanode1hostname.host.com#TESTREALM)s/^.*$/hadoop/
RULE:[1:$1](dn/datanode2hostname.host.com#TESTREALM)s/^.*$/hadoop/
RULE:[1:$1](http/namenodehostname.host.com#TESTREALM)s/^.*$/hadoop/
DEFAULT</value></property>
<property><name>hadoop.http.filter.initializers</name><value>org.apache.hadoop.security.AuthenticationFilterInitializer</value></property>
<property><name>hadoop.http.authentication.token.validity</name><value>3600</value></property>
<property><name>hadoop.http.authentication.cookie.domain</name><value>host.com</value></property>
<property><name>hadoop.http.authentication.cookie.persistent</name><value>false</value></property>
<property><name>hadoop.http.authentication.simple.anonymous.allowed</name><value>false</value></property>
<property><name>hadoop.http.authentication.kerberos.principal</name><value>http/namenodehostname.host.com#TESTREALM</value></property>
<property><name>hadoop.http.authentication.kerberos.keytab</name><value>/etc/security/keytab/spnego.service.keytab</value></property>
</configuration>
hdfs-site.xml
<configuration>
<property><name>dfs.namenode.name.dir</name><value>file:///home/hadoop/hadoopdata/hdfs/namenode</value></property>
<property><name>dfs.namenode.edits.dir</name><value>file:///home/hadoop/hadooplogs/hdfs/edits</value></property>
<property><name>dfs.replication</name><value>1</value></property>
<property><name>dfs.datanode.http.address</name><value>0.0.0.0:8108</value></property>
<property><name>dfs.datanode.https.address</name><value>0.0.0.0:8109</value></property>
<property><name>dfs.webhdfs.enabled</name><value>true</value></property>
<property><name>dfs.client.use.datanode.hostname</name><value>true</value></property>
<property><name>dfs.datanode.use.datanode.hostname</name><value>true</value></property>
<property><name>dfs.namenode.acls.enabled</name><value>false</value></property>
<property><name>dfs.namenode.posix.acl.inheritance.enabled</name><value>true</value></property>
<property><name>dfs.permissions.enabled</name><value>false</value></property>
<property><name>dfs.namenode.datanode.registration.ip-hostname-check</name><value>false</value></property>
<property><name>dfs.namenode.rpc-bind-host</name><value>0.0.0.0</value></property>
<property><name>dfs.namenode.servicerpc-bind-host</name><value>0.0.0.0</value></property>
<property><name>dfs.namenode.http-bind-host</name><value>0.0.0.0</value></property>
<property><name>dfs.namenode.https-bind-host</name><value>0.0.0.0</value></property>
<property><name>dfs.block.access.token.enable</name><value>true</value></property>
<property><name>dfs.namenode.keytab.file</name><value>/etc/security/keytab/nn.service.keytab</value></property>
<property><name>dfs.namenode.kerberos.principal</name><value>nn/namenodehostname.host.com#TESTREALM</value></property>
<property><name>dfs.namenode.kerberos.internal.spnego.principal</name><value>HTTP/namenodehostname.host.com#TESTREALM</value></property>
<property><name>dfs.web.authentication.kerberos.keytab</name><value>/etc/security/keytab/spnego.service.keytab</value></property>
<property><name>dfs.web.authentication.kerberos.principal</name><value>HTTP/namenodehostname.host.com#TESTREALM</value></property>
<property><name>dfs.http.policy</name><value>HTTPS_ONLY</value></property>
</configuration>
The first datanode is configured with the following files:
core-site.xml
<configuration>
<property><name>fs.defaultFS</name><value>hdfs://namenodehostname.host.com:9000</value></property>
<property><name>hadoop.security.authentication</name><value>kerberos</value></property>
<property><name>hadoop.security.authorization</name><value>true</value></property>
<property><name>hadoop.rpc.protection</name><value>authentication</value></property>
<property><name>hadoop.security.auth_to_local</name><value>
RULE:[1:$1](nn/namenodehostname.host.com#TESTREALM)s/^.*$/hadoop/
RULE:[1:$1](dn/datanode1hostname.host.com#TESTREALM)s/^.*$/hadoop/
RULE:[1:$1](dn/datanode2hostname.host.com#TESTREALM)s/^.*$/hadoop/
RULE:[1:$1](http/namenodehostname.host.com#TESTREALM)s/^.*$/hadoop/
DEFAULT</value></property>
<property><name>hadoop.http.filter.initializers</name><value>org.apache.hadoop.security.AuthenticationFilterInitializer</value></property>
<property><name>hadoop.http.authentication.token.validity</name><value>3600</value></property>
<property><name>hadoop.http.authentication.cookie.domain</name><value>host.com</value></property>
<property><name>hadoop.http.authentication.cookie.persistent</name><value>false</value></property>
<property><name>hadoop.http.authentication.simple.anonymous.allowed</name><value>false</value></property>
<property><name>hadoop.http.authentication.kerberos.principal</name><value>http/namenodehostname.host.com#TESTREALM</value></property>
<property><name>hadoop.http.authentication.kerberos.keytab</name><value>/etc/security/keytab/spnego.service.keytab</value></property>
</configuration>
The second datanode has the same core-site.xml file of the first datanode and the following hdfs site.xml file.
<configuration>
<property><name>dfs.datanode.data.dir</name><value>file:///home/hadoop/hadoopdata/hdfs/datanode</value></property>
<property><name>dfs.datanode.failed.volumes.tolerated</name><value>0</value></property>
<property><name>dfs.datanode.address</name><value>0.0.0.0:8100</value></property>
<property><name>dfs.datanode.http.address</name><value>0.0.0.0:8108</value></property>
<property><name>dfs.webhdfs.enabled</name><value>true</value></property>
<property><name>dfs.client.use.datanode.hostname</name><value>true</value></property>
<property><name>dfs.datanode.use.datanode.hostname</name><value>true</value></property>
<property><name>dfs.permissions.enabled</name><value>false</value></property>
<property><name>dfs.namenode.datanode.registration.ip-hostname-check</name><value>false</value></property>
<property><name>dfs.namenode.rpc-bind-host</name><value>0.0.0.0</value></property>
<property><name>dfs.namenode.servicerpc-bind-host</name><value>0.0.0.0</value></property>
<property><name>dfs.namenode.http-bind-host</name><value>0.0.0.0</value></property>
<property><name>dfs.namenode.https-bind-host</name><value>0.0.0.0</value></property>
<property><name>dfs.datanode.hostname</name><value>datanode2hostname.host.com</value></property>
<property><name>dfs.block.access.token.enable</name><value>true</value></property>
<property><name>dfs.datanode.data.dir.perm</name><value>700</value></property>
<property><name>dfs.datanode.keytab.file</name><value>/etc/security/keytab/dn.service.keytab</value></property>
<property><name>dfs.datanode.kerberos.principal</name><value>dn/datanode2hostname.host.com#TESTREALM</value></property>
<property><name>dfs.encrypt.data.transfer</name><value>false</value></property>
<property><name>dfs.datanode.https.address</name><value>0.0.0.0:8109</value></property>
<property><name>dfs.data.transfer.protection</name><value>authentication</value></property>
<property><name>dfs.http.policy</name><value>HTTPS_ONLY</value></property>
</configuration>
The namenode and datanodes have the following
ssl-server.xml file.
<configuration>
<property><name>ssl.server.keystore.location</name><value>/home/hadoop/keystore.jks</value></property>
<property><name>ssl.server.keystore.password</name><value>password123.</value></property>
<property><name>ssl.server.keystore.type</name><value>JKS</value></property>
<property><name>ssl.server.truststore.location</name><value>/home/hadoop/truststore.jks</value></property>
<property><name>ssl.server.truststore.password</name><value>password123.</value></property>
<property><name>ssl.server.truststore.type</name><value>JKS</value></property>
</configuration>
Is there anything else I need to do to get login on the namenode web interface?