I have issue with the cert manager which prevents me from synchronizing the robot with the cloud cluster.
It seems that it is due to a certificate error some way I can solve it.
the logs of cert manager:
1 controller.go:200] cert-manager/controller/orders "msg"="re-queuing item due to error processing" "error"="error creating new order: acme: urn:ietf:params:acme:error:rateLimited: Error creating new order :: too many certificates already issued for exact set of domains: www.endpoints.guminator.cloud.goog: see https://letsencrypt.org/docs/rate-limits/" "key"="default/cloud-robotics-2181866887"
This is hitting the "Duplicate Certificate limit of 5 per week" mentioned at https://letsencrypt.org/docs/rate-limits/. https://tools.letsdebug.net/cert-search?m=domain&q=www.endpoints.guminator.cloud.goog&d=168 shows when the certificates were created.
For a short-term resolution: I'd hope it will start working again in the next days, since the first certificates were issued on the 12th.
For a long-term resolution: Have you done anything that would cause this, for example deleting and recreating the cluster multiple times? If you need to do this regularly, you have two options:
Back up the certificate
https://cert-manager.readthedocs.io/en/release-0.9/tasks/backup-restore-crds.html suggests the following approach for backing up and restoring the configuration:
kubectl get -o yaml \
issuer,clusterissuer,certificates,orders,challenges > cert-manager-backup.yaml
kubectl get -o yaml secret letsencrypt-prod > cert-manager-secret-backup.yaml
Then, to restore after recreating the cluster:
kubectl apply -f cert-manager-secret-backup.yaml
Caveat: I have not tested this approach.
Use the Staging environment
https://letsencrypt.org/docs/staging-environment/ states that the staging environment allows 30,000 duplicate certs per week. You'd need to:
edit src/app_charts/base/cloud/cert-manager-issuers.yaml to refer to https://acme-staging-v02.api.letsencrypt.org/directory
configure the clients to trust the staging CA: https://letsencrypt.org/certs/fakeleintermediatex1.pem
Related
I’m trying to upload my certs to Bitrise by using the codesigndoc tool, which I’m running with the following command
bash -l -c "$(curl -sfL https://raw.githubusercontent.com/bitrise-tools/codesigndoc/master/_scripts/install_wrap-xcode.sh) --scheme='my Scheme' --certs-only"
The issue is that it’s not detecting any distribution certificates
The Xcode archive used development certificate: iPhone Developer: Me (XXX) [1234].
Please select a distribution certificate:
Please select from the list:
(type in the option's number, then hit Enter) [1] :
It errors out if I try and continue, with an invalid option response (if I just try to put in 1 or 0).
I already archived and exported the ipa manually, which I heard is sometimes needed.
This seems to be an issue in codesigndoc - tracking it at: https://discuss.bitrise.io/t/no-codesigndoc-distribution-certificatea/4355
Will report the progress there.
I was able to circumvent the problem by removing the scheme & certs-only flags.
Not sure why, but it fixed the problem.
I have a working setup where Let's Encrypt certificates are generated with certbot. I wonder how you effectively test whether the renewal will work in production.
The certificates last for 90 days. Is there a way to reduce the lifespan to, for instance, 10 minutes, to see if the renewal works? (Using the staging system for that is fine.)
If you have an alternative approach how to make sure that your renewal code works (without having to wait for 90 days), it would also be appreciated.
You use the --dry-run option. E.g.:
$ sudo certbot renew --dry-run
From certbot -h:
certbot [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ...
...
--dry-run Test "renew" or "certonly" without saving any certificates
to disk
This ensures that the certbot can validate your domain with your current configuration.
If you really want to save the certificates to disk and see if your system is using the new cert, then you can also use the --force-renewal option. In that case, you should visit your website and check that the active certificate is the new one. If it isn't, you likely need to adjust your cronjob to restart your web server. E.g.:
certbot renew && service apache24 restart
You can use "certbot renew --force-renewal"
https://certbot.eff.org/docs/using.html#configuration-file
--force-renewal, --renew-by-default
If a certificate already exists for the requested
domains, renew it now, regardless of whether it is
near expiry. (Often --keep-until-expiring is more
appropriate). Also implies --expand. (default: False)
All of a sudden, with no change to the Xcode build server, I started getting an error, even across different bots, branches, and apps (all are Mac apps). I didn't change any software or configuration on the build server (that I know of), or even sign into it. I get the error once during analysis:
Check dependencies
Code Sign error: Multiple matching codesigning identities found: Multiple codesigning identities (i.e. certificate and private key pairs) matching “Mac Developer: OS X Server (N________L)” were found.
And again during Testing (though my tests pass – the one above is what fails the build):
Testing failed:
Multiple matching codesigning identities found: Multiple codesigning identities (i.e. certificate and private key pairs) matching “Mac Developer: OS X Server (N________L)” were found.
I have tried the following, triggering a build after each step, with no change in the message:
Triggered another build
Updated to the latest Xcode on the build server (6.3.2)
Searched through Keychain Access for any "OS X Server" certificates or keys, finding none
Removed all Developer Teams from Server.app. Note: this acted weird. I had to remove it twice before the setting read "No Teams", but now it's stayed that way
I signed in to add my team again
Removed it again, which still had to be done twice before it "stuck"
Restarted the machine, multiple times
Removed all provisioning profiles from /Library/Developer/XcodeServer/ProvisioningProfiles
Signed in with my developer account again after removing the provisioning profiles. They have been recreated, but the error remains
I turned off "Perform analyze action" in the bot, which did get rid of the message, but turning it back on, or turning on "Perform archive action", reintroduce it, so that's not an option (also, I want static analysis performed, which is why it's turned on in the first place).
Assuming for the moment that Xcode Server's list of signing identities is corrupted, how would I go about resetting them, outside of Server.app (since I already removed the Development Team there)? I found a couple of directories that seem like candidates for some troubleshooting and/or backup restoration, but I'm hesitant to mess around without knowing for sure what they're for.
/Library/Developer/XcodeServer/Certificates
/Library/Developer/XcodeServer/Keychains
I noticed that my provisioning profiles all expire on May 20, 2016. This means my old ones expired two days ago (5/20/2015), which was between my last successful (5/19) and first failed (5/21) builds.
So now, knowing the source of these failures, the problem now becomes: how do I clear out the old provisioning profiles (or certificates, etc.) that are causing the duplicates?
The server is up-to-date on Yosemite (10.10.3), Server (4.1), and Xcode (6.3.2).
Update
I'm trying to list codesigning identities, but can't get even a single "Mac Developer: OS X Server" to show up, let alone multiples. These are the commands I've tried, which only list the non-Server identities in the keychain:
security find-identity -p codesigning
sudo -u _xcsbuildd security find-identity -p codesigning
sudo -u _xcsd security find-identity -p codesigning
It looks like a bug in Server did in fact introduce a duplicate signing identity. I reported it as rdar://21080937, if you'd like to dupe it.
In order to fix it, I had to learn about how Xcode Server stores signing identities (thanks entirely to an extremely helpful answer to an unrelated question).
Xcode Server stores Developer Program certificates in
/Library/Developer/XcodeServer/Keychains/Portal.keychain
Following the steps below, I was able to open the keychain, find the duplicate, remove it, and put it back. These steps are adapted from the answer linked above.
Sign out of the Developer Portal from Server.app by removing the developer team you're having issues with
Copy the Portal keychain to your desktop:
> sudo cp /Library/Developer/XcodeServer/Keychains/Portal.keychain ~/Desktop/
Password: your-administrator-password
> sudo chown `whoami`:staff ~/Desktop/Portal.keychain
Set the Portal keychain password to “123”
> security set-keychain-password -o "`sudo cat /Library/Developer/XcodeServer/SharedSecrets/PortalKeychainSharedSecret`" ~/Desktop/Portal.keychain
New Password: 123
Retype New Password: 123
Open the Keychain in Keychain Access:
> open -b com.apple.keychainaccess ~/Desktop/Portal.keychain
Unlock the “Portal” keychain using password “123”
Find the duplicate keys with the name listed in the error
Remove one (I picked the one modified less recently, but it may not matter)
Lock the “Portal” keychain, quit “Keychain Access”
Reset the Portal keychain password:
> security set-keychain-password -p "`sudo cat /Library/Developer/XcodeServer/SharedSecrets/PortalKeychainSharedSecret`" ~/Desktop/Portal.keychain
Password: your-administrator-password (optional step)
Old Password: 123
It may or may not ask you for your administrator password again, pay attention to the prompt.
Backup the original keychain
> sudo cp /Library/Developer/XcodeServer/Keychains/Portal.keychain /Library/Developer/XcodeServer/Keychains/Portal.keychain.old
Copy the Portal keychain back
> sudo cp ~/Desktop/Portal.keychain /Library/Developer/XcodeServer/Keychains/
Fix the new keychain's permissions
> sudo chown _xcsbuildd:_xcs /Library/Developer/XcodeServer/Keychains/Portal.keychain
Since the system caches open keychains, restart the server
Add your developer team back, in Server.app
Run a build, verifying everything looks good. If so, remove the backup copies of the keychain
> rm ~/Desktop/Portal.keychain
> sudo rm /Library/Developer/XcodeServer/Keychains/Portal.keychain.old
My eventual workaround for the issue below was to convince our IT guys not to man-in-the-middle the dockerhub registry. I was not able to get anything else to work, alas.
I am running into a problem with my initial attempt to get Docker running on my Mac at work, which is running 10.8.5. It appears that my company's certificate-rewriting proxy seems to be getting in the way of fetching images:
orflongpmacx8:docker pohl_longsine$ docker run hello-world
Unable to find image 'hello-world:latest' locally
Pulling repository hello-world
FATA[0001] Get https://index.docker.io/v1/repositories/library/hello-world/images: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "bcauth")
(Indeed, when I log onto the guest wireless – which does not have the meddlesome proxy – I can get past this step. However, I need to figure out how to make this work through the proxy since using the guest wireless is untenable as a long-term solution.)
My issue, on the surface, appears to be very much like the one answered in this question. However, the accepted answer in that question does not work for me, since the root_unix.go file they discuss does not get invoked on a Mac. (From browsing around, I would guess that root_cgo_darwin.go and/or root_darwin.go would be involved instead.)
That doesn't really tell me how, operationally, I need to do the equivalent work of installing some sort of trusted certificate. I managed to get my hands on a *.cer file that I believe to be the one that I need, but I'm at a loss as to what to do with it.
I'm hoping that someone can point me in the right direction.
Edit: I thought that maybe I needed to to something akin to what this page suggests, to add the certificate. Alas, my attempt at following those instructions failed in the following way:
orflongpmacx8:docker pohl_longsine$ sudo security add-trusted-cert -d -r trustRoot -k "/Library/Keychains/System.keychain" "~/Desktop/Certs/redacted.cer"
Password:
***Error reading file ~/Desktop/Certs/redacted.cer***
Error reading file ~/Desktop/Certs/redacted.cer
Edit 2: I may have come one step closer to solving this. I should have known better to use a path with a tilde inside quotation marks. If I use an absolute path instead, I can successfully run the above command to add certs.
Alas, this did not alleviate the ultimate symptom:
FATA[0001] Get https://index.docker.io/v1/repositories/library/hello-world/images: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "bcauth")
According to the boot2docker README
Insecure Registry
As of Docker version 1.3.1, if your registry doesn't support HTTPS, you must add it as an insecure registry.
$ boot2docker init
$ boot2docker up
$ boot2docker ssh
$ echo 'EXTRA_ARGS="--insecure-registry <YOUR INSECURE HOST>"' | sudo tee -a /var/lib/boot2docker/profile
$ sudo /etc/init.d/docker restart
then you should be able to do a docker push/pull.
The source of http://golang.org/src/crypto/x509/root_darwin.go shows that the command:
cmd := exec.Command("/usr/bin/security", "find-certificate", "-a", "-p", "/System/Library/Keychains/SystemRootCertificates.keychain")
is used to find the certificate.
Try adding the .cer file into the OSX certificate key-chain.
If you use the docker-machine
edit $USER/.docker/machine/machines/default/config.json
"EngineOptions": {
"InsecureRegistry": [
"XXX.XXX.virtual"
],
}
I am having a problem with my server and so far couldn't find any solution for this. When I try to add a server from a server manager (windows server 2012) I can see only the kerberos security error. Both servers are in the same domain(i have tried from several servers from domain and got the same error).
The strange thing is when I unjoin the problematic server from domain and rejoin it with another name it works normally. But the problem is to make it work with existing name. Anyhelp will be highly appreciated
thanks in advance.
Late reply, but I've just encountered the same error and hope this solution proves useful to others.
Situation: I had to wipe and reinstall a virtual server on which I'd previously had to set some Service Principal Names, and some SPNs for a service account. Turns out the SPNs were still there for the old server/account and I had to remove them.
I recommend checking for and removing rogue SPNs to resolve this. Use the following commands in an elevated command prompt:
setspn -l <servername/username>
In my case I had problems with MBAM, the Bitlocker admin tool, so for example I used:
setspn -l mbam01
Which gave me the output (changed names to protect the innocent):
Registered ServicePrincipalNames for CN=MBAM01,OU=Member Servers,DC=corp,DC=domainname,DC=com:
termserv/mbam01.corp.domainname.com
termserv/mbam01
http/mbam01.corp.domainname.com
http/mbam01
HOST/MBAM01
HOST/mbam01.corp.domainname.com
This will list the SPNs associated with the server or user account. Then you remove the errant SPNs with this command:
setspn -d <listed service> <servername/username>
In my case it turned out the mbamapppool user had http/mbam01 and http/mbam01.corp.domainname.com associated with it, causing Server Manager to fail to poll the server. I removed the http/ refs from the user and then added them to the server with the following commands:
setspn -d http/mbam01 corp\mbamapppooluser
setspn -d http/mbam.corp.domainname.com corp\mbamapppooluser
setspn -s http/mbam01 mbam01
setspn -s http/mbam01.corp.domainname.com mbam01
I then refreshed Server Manager and it polled the server successfully, and the Kerberos Security Error had gone.