I have a Spring-Boot application that use Spring Cloud Config and I'm trying to get the application's configuration file from Bitbucket. I was able to get the configuration file some time ago but now I'm getting an error when I try to access by config-server url.
application.yml:
server:
port: 8888
spring:
application:
name: config-server
cloud:
config:
server:
git:
password: ##########
username: ##########
uri: https://USERNAME#bitbucket.org/repositorios-closeup/cup-configuration-files
searchPaths: '{application}'
When I try to access the url the application is showing an error - NOT AUTHORIZED:
org.eclipse.jgit.api.errors.TransportException: https://USERNAME#bitbucket.org/repositorios-closeup/cup-configuration-files: not authorized
at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:254) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
at org.eclipse.jgit.api.CloneCommand.fetch(CloneCommand.java:306) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
Does anybody know what is happening? I've already check all credentials and url on bitbucket.
I had to generate a Personal Access Token on my Github user settings. And use it, instead of the real password.
I know you mentioned Bitbucket, but I've got the same issue with Github instead. And that's how I solved the "Not authorized" message.
I want to add my contribution to this question. I hope it might help someone. #henriqueor answer was what helped me. As I've enabled SSH I needed to generate a Personal Access Token with all scopes granted. (As I'm a Junior Developer, I would need more time to investigate what scopes are necessary to avoid this error.)
Selected scopes: admin:enterprise, admin:gpg_key, admin:org, admin:org_hook, admin:public_key, admin:repo_hook, admin:ssh_signing_key, delete:packages, delete_repo, gist, notifications, project, repo, user, workflow, write:discussion, write:packages
I added the following command line. It worked for me.
spring.cloud.config.server.git.ignore-local-ssh-settings=true
This setting ignores the local ssh config.
I needed to change the authentication to use ssh instead of https.
I generated the ssh using this command:
sh-keygen -m PEM -t rsa -b 4096 -C 'bitbucket_username'
Imported the public key to bitbucket using this tutorial:
https://support.atlassian.com/bitbucket-cloud/docs/set-up-an-ssh-key/
And changed my application.yml to use the ssh-private-key:
cloud:
config:
server:
git:
uri: git#bitbucket.org:repositorios/configuration-files.git
searchPaths: '{application}'
ignore-local-ssh-settings: true
private-key: |
-----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----
default-label: master
For BitBucket i had to create an App Password and put it in the spring.cloud.config.server.git.password property
BitBucket create App Passwords
Related
I am banging my head around why spring cloud config is using ssh even when i try to provide https uri in the application.yaml. Here's my quick config file:
spring:
cloud:
config:
server:
git:
uri: https://github.com/myOrg/myRepo
skipSslValidation: true
username: ${GITHUB_USERNAME}
password: ${GITHUB_PASSWORD}
server:
port: 8888
ssl:
enabled: false
I would assume spring cloud config to use https to do this connection and clone the repo. Can someone help me understand why still it uses ssh and fails on auth(which is obvious as I don't provide any private key):
org.springframework.cloud.config.server.environment.NoSuchRepositoryException: Cannot clone or checkout repository: https://github.com/myOrg/myRepo
at org.springframework.cloud.config.server.environment...
Caused by: org.eclipse.jgit.api.errors.TransportException: git#github.com:myOrg/myRepo: Auth fail
....
Caused by: com.jcraft.jsch.JSchException: Auth fail
at com.jcraft.jsch.Session.connect(Session.java:519) ~[jsch-0.1.55.jar:na]
at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:115) ~[org.eclipse.jgit.ssh.jsch-5.12.0.202106070339-r.jar:5.12.0.202106070339-r]
... 62 common frames omitted
My suspicion is that if https doesn't work, it tries via ssh. That's why it's using ssh. Has anyone faced issues like this?
Any help appreciated!
Answering my own question. Apparently, the fallback was due to the ~/.git folder in which ssh access was recommended. Deleting it solved the problem.
I implemented Spring Cloud config server. How can I prevent the config server bootstrap.yml file from storing the GIT user name and password as clear text?
Vault: https://github.com/hashicorp/vault
Use Vault: https://cloud.spring.io/spring-cloud-config/reference/html/#_vault
Set up Spring Vault:
https://docs.spring.io/spring-vault/docs/2.2.2.RELEASE/reference/html/
https://spring.io/projects/spring-vault
In your Spring Cloud config server, file bootstrap.yml
spring:
cloud:
config:
token: YourVaultToken
ok So this is working fine for me, the issue was my config server's bootstrap.yml need to connect to GIT repository as backend and GIT repo is secured with username and password but I can not pass the username and password in bootstrap.yml file.
To solve this:
Pass the credential as environmental variable and store these environment variable in terraform or any other secure location.
I tried to use the method for using private key (that has passphrase and is added to ssh-agent from file) (according to this stack post):
spring:
cloud:
config:
server:
git:
uri: git#github.com-forApp:myorg/myrepo.git
search-paths: '{application}'
clone-on-start: true
private_key_file: ~/.ssh/id_rsa
but I keep getting
org.eclipse.jgit.api.errors.TransportException:
git#github.com:myorg/myrepo.git: USERAUTH fail
Do I have to do it exactly as doc says with pasting the key into config file or can one just point to the key file somehow?
EDIT
Actually it turns out that the private_key_file is not needed at all or ignored by Spring. But you need the ~/.ssh/config section pointing to private key to use:
Host github.com-forApp # used in spring uri
HostName github.com
User git
IdentityFile ~/.ssh/gitHubKey
I was able to replicate your behavior and resolved it with following. Let me know your thoughts.
USERAUTH fail is happening because you are not providing the passphrase for the RSA private key.(password for Basic Auth and passphrase for ssh private key)
spring:
cloud:
config:
server:
git:
uri: git#github.com:myorg/myrepo.git
search-paths: '{application}'
clone-on-start: true
passphrase: myprivatekeypassword
By default ~/.ssh/id_rsa is sent during GIT SSH Authentication(Test with command ssh -vT git#github.com. You don't need to specify it in configuration. Also, I am not sure whether private_key_file works or not, since I don't see any official documentation for it.
If you have different named RSA file under .ssh then I would advise to create config file under ~/.ssh/config with github host details and identify file.
Here is one example.
Host github.com
IdentityFile ~/.ssh/mygitid_rsa
Check this stack answer for more details which desired the configuration providing private key file path within config.
After going the number of SO threads and blogs and Spring cloud config documentation still, I couldn't find on how I can connect to remote GITLAB repository as I'm getting below error while starting the spring-cloud-config server.
Caused by: com.jcraft.jsch.JSchException: Auth fail
spring:
cloud:
config:
server:
git:
uri: git#private_gitlab_repo:project
search-paths: '{application}'
skip-ssl-validation: true
strict-host-key-checking: false
known-hosts-file: C:\Users\myname\.ssh\known_hosts
spring-boot :2.1.2.RELEASE
spring-cloud.version: Greenwich.RELEASE
OS: Windows-7
With the command prompt, I could able to interact with the GITLAB repository. I do have the SSH key generated and added the public key in GITLAB settings. Also, I do not have the option to use username and password to connect to GITLAB.
Any pointers on where I'm missing the configuration or steps?
Found that this is my IntelliJ idea IDE issue and when I try running the same project in command prompt it worked without any issues.
I have a spring cloud config server running with spring bus. I want to make the calls to that server secure:
When a client is asking for configurations.
When calling /monitor - used by the webhook.
What is the best practice to do that? basic? encryption?
Can someone provide a working example?
Thanks!
You can secure it by adding encrypting and decrypting properties
You need to provide jks for securely encrypting and decrypting them
Spring cloud config server supports symmetric and asymmetric keys
To configure a symmetric key, you need to set encrypt.key to a secret String (or use the ENCRYPT_KEY environment variable to keep it out of plain-text configuration files).
For asymmetric you need to provide in bootsrap.yml such properties:
server:
port: 8888
spring:
cloud:
config:
server:
git:
uri: your git url or your local repository on file system
username: username for git or bitbucket if needed
password: password
clone-on-start: true this property will clone all repo localy on starttup
force-pull: true
application:
name: config-server
encrypt:
key-store:
location: jks location
password: letmein
alias: mytestkey
secret: changeme
For generating jks you need to execute this command
keytool -genkeypair -alias mytestkey -keyalg RSA \
-dname "CN=Web Server,OU=Unit,O=Organization,L=City,S=State,C=US" \
-keypass changeme -keystore server.jks -storepass letmein
Actually java by default has a limitation on certain key length parameters.
Its 128 bit by default.
To use key more key length you just need replace existing local_policy.jar and US_export_policy.jar in <java-home>/lib/security
Here is link for download :
https://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html
And also you can encrypt and decrypt your properties by such endpoints :
curl config_server_host:port/encrypt-d your data to be encrypted
curl config_server_host:port/decrypt -d your data to be decrypted // this will automatically use this endpoint to decrypt values
//Both are http post requests
To use encryption by config server you need to provide such prefix in your configuration for your application which will get configs from config server:
'{cipher}your_encrypted_data'
Also, you can control access to secrets in the config by the using of Spring Cloud Vault.
This solution simpler than encrypt all communication between your application and config server, but maybe this is not what you want.
I hope it helps.