I am having a Micro-Service Platform having multiple Micro-Services connected to each other, Platform uses Kerberos for authentication of Micro-Services. In One of Micro-Service Node hadoop is installed which uses separate KDC for Hadoop cluster authentication.
Lets say platform domain is "idm.com" and hadoop domain is "hadoop.com".
Resource Manager is running on one node. I have configure HTTP principal for spnego in core-site.xml using "hadoop.http.authentication.kerberos.principal" property to "HTTP/master.hadoop.com#HADOOP.COM" and nodes Hostname is "hadoopmaster.idm.com".
I do Kinit and acquire root user ticket from TGS. When I tried to do curl using "curl -k -v --negotiate -u : https://master.hadoop.com:8090/cluster" It shows GSS Exception: No valid credentials provided.
If I see klist it shows two ticket one krbtgt and second "HTTP/hadoopmaster.idm.com#HADOOP.COM"(I have added this principal in kdc database). First krbtgt i got using kinit and second HTTP one i Got it automatically after doing curl before curl the ticket was not there. Krb client acquired another for using HTTP service.
After some debugging I noticed the problem/behaviour is I got ticket for HTTP/hadoopmaster.idm.com#HADOOP.COM where I have configure hadoop to use HTTP/master.hadoop.com#HADOOP.COM. If we configure hadoop to use "HTTP/hadoopmaster.idm.com#HADOOP.COM" then ui is accessible.
I have added both FQDNs to /etc/hosts file.
It seems when I do curl using any of the FQDNs I got the HTTP ticket of the first entry in /etc/hosts file.
For example if
...
10.7.0.5 hadoopmaster.idm.com
10.7.0.5 master.hadoop.com
...
now if i do curl i will get HTTP/hadoopmaster.idm.com#HADOOP.COM in klist.
and if /etc/hosts looks like this
...
10.7.0.5 master.hadoop.com
10.7.0.5 hadoopmaster.idm.com
...
Now if i do curl i will get HTTP/master.hadoop.com in klist
In both the cases if i configure the hadoop property to the same i got using curl then UI will be accessible and other wise it will shows 403 GSSException which i guess means curl used spnego but didn't get valid credentials.
And if it matches with the hadoop's configured principal then it will work.
It looks like Hostname is causing problem is there any way to map this hostname or is there any kerberos config which can map this or any property which will give me exact ticket with exact hostname i have specified in curl despite of hadoop configurations.
I am trying to setup my dart http server to run only with https. So I gather I need to use HttpServer.bindSecure but I'm not clear from the description what needs to be passed in as certificateName and whether requestClientCertificate being true makes it more or less secure, or has no impact on security what so ever. The small sample code at the top of the HttpServer page passes in certificateName: 'localhost_cert' but before that it does something with a database, but doesn't seem to use it in anyway. Can anyone explain in more detail what these values are and what they need to be in order to make them secure?
The requestClientCertificate parameter of bindSecure is used to specify a client certificate. Client certificates are used by servers to identify and authorize clients, which appears not to be the objective of this question. It should be noted that there is a known issue with using client certificates in Dart on IE9 and Windows 7.
The certificateName parameter is used to specify the nickname of a certificate that exists in your certificate database. You specify the certificate nickname using the -n <nickname> option when importing a certificate to your database using certutil.
Use the following steps to:
Install the NSS utility (including certutil),
Create a new certificate database in directory <dir> with a password <password>, and
Import your self-signed or purchased certificate identified by nickname <host> such that it can be used to create an HTTPS server using the following sample code. Though the nickname can be chosen arbitrarily, we use the host name in this example. These steps have been confirmed working in Ubuntu 14.04 and Dart SDK 1.6 through (currently last stable version) 1.8.3.
Install the NSS utility
sudo apt-get install libnss3-tools
cd to the directory that will contain your certificate database
cd <dir>
Create a password file to use with the certificate database:
echo "<password>" > pwdfile
Create the certificate database
certutil -N -d 'sql:./' -f pwdfile
Either:
Generate a self-signed certificate:
certutil -S -s "cn=<host>" -n "self signed for dart" -x -t "C,C,C" -m 1000 -v 120 -d "sql:./" -k rsa -g 2048 -f pwdfile
where <host> is the host ("common name") for which to generate a certificate, for example "localhost"
Or, purchase a certificate by first creating a signing request for a real domain <host>, for example "myhost.com":
certutil -R -s "CN=<host>, O=None, L=San Diego, ST=California, C=US" -a -g 2048 -o <host>.csr -d "sql:./"
Then specify the content of file <host>.csr when prompted for a CSR upon purchasing a certificate from a signing authority.
Copy the purchased certificate to a file named <host>.crt
Import the certificate to the database
certutil -A -n <host> -t "p,p,p" -i <host>.crt -d "sql:./"
If necessary to use an intermediate certificate, it can be imported as such:
certutil -A -n my_intermediate_certificate -t "p,p,p" -i intermediate.crt -d "sql:./"
where "intermediate.crt" is the intermediate certificate file downloaded from the signing authority.
Verify that the certificates exist in the database
certutil -L -n <host> -d "sql:./"
certutil -L -n my_intermediate_certificate -d "sql:./"
To use this certificate and create an HTTPS server, do the following:
// Initialize secure socket to use certificate database (note: replace `<dir>`
// with the absolute path to the certificate database directory, and `<password>`
// with the value chosen above)
SecureSocket.initialize(database: "<dir>", password: "<password>");
// Bind secure HTTP server to specified host and port (typically 443)
HttpServer.bindSecure("<host>", 443, certificateName: "<host>")
.then((HttpServer httpServer) {
// Listen for incoming requests
httpServer.listen((HttpRequest httpRequest) {
// TODO: process request
});
})
.catchError((error) {
// TODO: handle error
});
Update
I don't have enough reputation points to respond to the comments, so here are additional details that may help answer the questions: Client certificates are not used to encrypt client-server communication and are not needed in the common scenario of establishing secure communication between a web browser and a webserver via HTTPS. The steps outlined above show how to create an HTTPS server in Dart using bindSecure.
this is my krb5.ini file.
[libdefaults]
default_realm = TEST.EXAMPLE.COM
.....
[realms]
TEST.EXAMPLE.COM = {
kdc = test.example.com:88
master_kdc = test.example.com:88
default_domain = example.com
}
[domain_realm]
.example.com = EXAMPLE.COM
example.com = EXAMPLE.COM
I created a user USERA on the domain, created a credential using kinit and have my jaas.config file defined as
example_config {
com.ibm.security.auth.module.Krb5LoginModule required
useDefaultCcache=false
principal="userA#TEST.EXAMPLE.COM"
useCcache="file:///c:/Users/userA/krb5cc_userA"
debug=true;
};
everything seems ok as it gets authenticated.
[JGSS_DBG_CRED] Retrieving Kerberos creds from cache for principal=userA#TEST.EXAMPLE.COM
[JGSS_DBG_CRED] Non-interactive login; no callbacks necessary.
[JGSS_DBG_CRED] Done retrieving Kerberos creds from cache
[JGSS_DBG_CRED] Login successful
[JGSS_DBG_CRED] userA#TEST.EXAMPLE.COM added to Subject
[JGSS_DBG_CRED] Kerberos ticket for userA#TEST.EXAMPLE.COM added to Subject
[JGSS_DBG_CRED] No keys to add to Subject for userA#TEST.EXAMPLE.COM
however now, I am trying to use a service principal name , instead of user principal. I used ktpass to create a keytab file
ktpass -out "c:\mytab.keytab" -princ "Installation1/test.example.com#TEST.EXAMPLE.COM" -mapUser "TEST\userA" -mapOp set -pass password -crypto DES-CBC-MD5 -pType KRB5_NT_PRINCIPAL +DesOnly
and then use setspn to check
C:\>setspn -l userA
Registered ServicePrincipalNames for CN=userA,CN=Users,DC=test,DC=example,DC=com:
Installation1/test.example.com
i change my config to
example_config{
com.ibm.security.auth.module.Krb5LoginModule required
credsType = both
principal="userA/test.example.com"
useDefaultCcache = true
useDefaultKeytab = false
useKeytab = "file:///c:/mytab.keytab"
debug=true;
};
now when i run my app, it gives
[JGSS_DBG_CRED] Retrieving Kerberos creds from keytab for principal=Installation1/test.example.com
[JGSS_DBG_CRED] Service name=Installation1/test.example.com#TEST.EXAMPLE.COM
[JGSS_DBG_CRED] Check for Default keytab :
[JGSS_DBG_CRED] No Kerberos creds in keytab for principal Installation1/test.example.com
[JGSS_DBG_CRED] No service key in keytab; login failed
What did i do wrong in the config file? or is there something i miss in the ktpass command? Installation1 is the service name (of Websphere MQ) when I go to check Control Panel -> Services. I want to autheticate userA to use Websphere MQ service. (like if i want to let user use HTTP I would put principal as HTTP/.....TEST.EXAMPLE.COM
thanks
There is nothing in MQ that performs authentication. MQ only performs authorization (permission lookup) via its OAM module.
For a default install of MQ (on any platform including z/OS), there is no effective security in MQ. You can configure MQ to use MQ SSL between a client application and the queue manager (or between 2 queue managers).
The other choice is to purchase a 3rd party MQ security solution like MQAUSX which handles authentication to various targets.
I have an access to a preconfigured Windows XP 32-bit workstation.
It's under firewall and to get outside it uses http proxy server.
To get proxy settings I need host, port, username and password.
http: // username : password # server : port
I'm able to find the proxy url and port (from PAC - automatic config file),
but I don't know how to get username/password. Is there a way to read it somehow? At least username? I might get the admin rights.
Do you know any tool that can help?
I only found these MS docs about some methods to get proxy config connection, but I don't know how to determinate what's the provided credentials:
WinHttpGetIEProxyConfigForCurrentUser function
WINHTTP_CURRENT_USER_IE_PROXY_CONFIG structure
WinHttpGetProxyForUrl function
I've also found this lib, but it's rather for parsing PAC:
http://code.google.com/p/pacparser/
In windows you cannot get password.
To get connect throw a proxy you have to use SSPI, check libcurl, you can get a connected socket throw proxy, build it with enable-sspi and openssl.
Iterating on Fernando Sanchez's answer and Robert's comment, you need to authenticate via SSPI. In my case I've connected using NTLM using this link
With curl using ntlm :
curl.exe --proxy <proxy_name>:<proxy_port> --proxy-ntlm -U : https://www.google.com
Also to partially answer your question the -U : means from curl man page :
If you use a Windows SSPI-enabled curl binary and do either Negotiate
or NTLM authentication then you can tell curl to select the user name
and password from your environment by specifying a single colon with
this option: "-U :".
You can get the proxy name and port from the windows registry (regedit). Quote from Robert's comment:
[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet
Settings]
It can be present either in a manual way using REG_SZ ProxyServer or using an autoconfiguration script using REG_SZ AutoConfigURL = https://<configuration_url>
If the curl command above fails with Failure when receiving data from the peer maybe you have a curl release without NTLM, SSPI or OpenSSL. Try with the latest curl release
Cntlm is an NTLM / NTLM Session Response / NTLMv2 authenticating HTTP proxy intended to help you break free from the chains of Microsoft proprietary world.
I have my proxy URL in the following format:
http://user:passwords#my_proxy_server.com:80
And I have to provide this information to cntlm. Its config file cntlm.ini has following structure and parameters:
Username
Domain
Password
Proxy
I am not sure, how to break up my original proxy property to fill these four options?
Update your user, domain, and proxy information in cntlm.ini, then test your proxy with this command (run in your Cntlm installation folder):
cntlm -c cntlm.ini -I -M http://google.ro
It will ask for your password, and hopefully print your required authentication information, which must be saved in your cntlm.ini
Sample cntlm.ini:
Username user
Domain domain
# provide actual value if autodetection fails
# Workstation pc-name
Proxy my_proxy_server.com:80
NoProxy 127.0.0.*, 192.168.*
Listen 127.0.0.1:54321
Listen 192.168.1.42:8080
Gateway no
SOCKS5Proxy 5000
# provide socks auth info if you want it
# SOCKS5User socks-user:socks-password
# printed authentication info from the previous step
Auth NTLMv2
PassNTLMv2 98D6986BCFA9886E41698C1686B58A09
Note: on linux the config file is cntlm.conf
The solution takes two steps!
First, complete the user, domain, and proxy fields in cntlm.ini. The username and domain should probably be whatever you use to log in to Windows at your office, eg.
Username employee1730
Domain corporate
Proxy proxy.infosys.corp:8080
Then test cntlm with a command such as
cntlm.exe -c cntlm.ini -I -M http://www.bbc.co.uk
It will ask for your password (again whatever you use to log in to Windows_). Hopefully it will print 'http 200 ok' somewhere, and print your some cryptic tokens authentication information. Now add these to cntlm.ini, eg:
Auth NTLM
PassNT A2A7104B1CE00000000000000007E1E1
PassLM C66000000000000000000000008060C8
Finally, set the http_proxy environment variable in Windows (assuming you didn't change with the Listen field which by default is set to 3128) to the following
http://localhost:3128
Without any configuration, you can simply issue the following command (modifying myusername and mydomain with your own information):
cntlm -u myusername -d mydomain -H
or
cntlm -u myusername#mydomain -H
It will ask you the password of myusername and will give you the following output:
PassLM 1AD35398BE6565DDB5C4EF70C0593492
PassNT 77B9081511704EE852F94227CF48A793
PassNTLMv2 A8FC9092D566461E6BEA971931EF1AEC # Only for user 'myusername', domain 'mydomain'
Then create the file cntlm.ini (or cntlm.conf on Linux using default path) with the following content (replacing your myusername, mydomain and A8FC9092D566461E6BEA971931EF1AEC with your information and the result of the previous command):
Username myusername
Domain mydomain
Proxy my_proxy_server.com:80
NoProxy 127.0.0.*, 192.168.*
Listen 127.0.0.1:5865
Gateway yes
SOCKS5Proxy 5866
Auth NTLMv2
PassNTLMv2 A8FC9092D566461E6BEA971931EF1AEC
Then you will have a local open proxy on local port 5865 and another one understanding SOCKS5 protocol at local port 5866.
Here is a guide on how to use cntlm
What is cntlm?
cntlm is an NTLM/NTLMv2 authenticating HTTP proxy
It takes the address of your proxy and opens a listening socket, forwarding each request to the parent proxy
Why cntlm?
Using cntlm we make it possible to run tools like choro, pip3, apt-get from a command line
pip3 install requests
choco install git
The main advantage of cntlm is password protection.
With cntlm you can use password hashes.
So NO PLAINTEXT PASSWORD in %HTTP_PROXY% and %HTTPS_PROXY% environment variables
Install cntlm
You can get the latest cntlm release from sourceforge
Note! Username and domain
My username is zezulinsky
My domain is local
When I run commands I use zezulinsky#local
Place your username when you run commands
Generate password hash
Run a command
cntlm -u zezulinsky#local -H
Enter your password:
Password:
As a result you are getting hashed password:
PassLM AB7D42F42QQQQ407552C4BCA4AEBFB11
PassNT PE78D847E35FA7FA59710D1231AAAF99
PassNTLMv2 46738B2E607F9093296AA4C319C3A259
Verify your generated hash is valid
Run a command
cntlm -u zezulinsky#local -M http://google.com
Enter your password
Password:
The result output
Config profile 1/4... OK (HTTP code: 301)
----------------------------[ Profile 0 ]------
Auth NTLMv2
PassNTLMv2 46738B2E607F9093296AA4C319C3A259
------------------------------------------------
Note! check that PassNTLMv2 hash is the same
The resulting hash is the same for both commands
PassNTLMv2 46738B2E607F9093296AA4C319C3A259
Change configuration file
Place generated hashes into the cntlm.ini configuration file
C:\Program Files (x86)\Cntlm\cntlm.ini
Here is how your cntlm.ini should look like
Username zezulinsky
Domain local
PassLM AB7D42F42QQQQ407552C4BCA4AEBFB11
PassNT PE78D847E35FA7FA59710D1231AAAF99
PassNTLMv2 46738B2E607F9093296AA4C319C3A259
Proxy PROXYSERVER:8080
NoProxy localhost, 127.0.0.*
Listen 3128
Note! newline at the end of cntlm.ini
It is important to add a newline at the end of the cntlm.ini configuration file
Set your environment variables
HTTPS_PROXY=http://localhost:3128
HTTP_PROXY=http://localhost:3128
Check that your cntlm works
Stop all the processes named cntlm.exe with process explorer
Run the command
cntlm -u zezulinsky#local -H
The output looks like
cygwin warning:
MS-DOS style path detected: C:\Program Files (x86)\Cntlm\cntlm.ini
Preferred POSIX equivalent is: /Cntlm/cntlm.ini
CYGWIN environment variable option "nodosfilewarning" turns off this warning.
Consult the user's guide for more details about POSIX paths:
http://cygwin.com/cygwin-ug-net/using.html#using-pathnames
section: local, Username = 'zezulinsky'
section: local, Domain = 'local'
section: local, PassLM = 'AB7D42F42QQQQ407552C4BCA4AEBFB11'
section: local, PassNT = 'PE78D847E35FA7FA59710D1231AAAF99'
section: local, PassNTLMv2 = '46738B2E607F9093296AA4C319C3A259'
section: local, Proxy = 'PROXYSERVER:8080'
section: local, NoProxy = 'localhost, 10.*, 127.0.0.*
section: local, Listen = '3128'
Default config file opened successfully
cntlm: Proxy listening on 127.0.0.1:3128
Adding no-proxy for: 'localhost'
Adding no-proxy for: '10.*'
Adding no-proxy for: '127.0.0.*'
cntlm: Workstation name used: MYWORKSTATION
cntlm: Using following NTLM hashes: NTLMv2(1) NT(0) LM(0)
cntlm: PID 1234: Cntlm ready, staying in the foreground
Open a new cmd and run a command:
pip3 install requests
You should have requests python package installed
Restart your machine
Congrats, now you have cntlm installed and configured
Just to add , if you are performing a "pip" operation , you might need to add and additional "--proxy=localhost:port_number"
e.g pip install --proxy=localhost:3128 matplotlib
Visit this link to see full details.
Once you generated the file, and changed your password, you can run as below,
cntlm -H
Username will be the same. it will ask for password, give it, then copy the PassNTLMv2, edit the cntlm.ini, then just run the following
cntlm -v
Thank you Sasha Zezulinsky.
In windows:
I used
SET HTTPS_PROXY=http://localhost:3128
When it was set to
SET HTTPS_PROXY=http://127.0.0.1:3128, it never worked.
Below posts are also very helpful.
How to use pip on windows behind an authenticating proxy
For me just using cntlm -H wasn't generating the right hash, but it does with the command below providing the user name.
If you need to generate a new password hash for cntlm, because you have change it or you've been forced to update it, you can just type the below command and update your cntlm.conf configuration file with the output:
$ cntlm -u test -H
Password:
PassLM D2AABAF8828482D5552C4BCA4AEBFB11
PassNT 83AC305A1582F064C469755F04AE5C0A
PassNTLMv2 4B80D9370D353EE006D714E39715A5CB # Only for user 'test', domain ''