Etcd cluster creation failure - cluster-computing

I'm trying to build an etcd cluster on three Ubuntu 20.04 VMs. I installed the apt etcd version:
etcd Version: 3.2.26
Git SHA: Not provided (use ./build instead of go build)
Go Version: go1.13.7
Go OS/Arch: linux/amd64
etcdctl version: 3.2.26
API version: 2
Here is the /etc/default/etcd contents (this changes in the IPs and NAME on the second and third cluster member):
ETCD_NAME=etcd1
ETCD_LISTEN_PEER_URLS="https://10.0.50.41:2380"
ETCD_LISTEN_CLIENT_URLS="https://10.0.50.41:2379,https://127.0.0.1:2379"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-clstr"
ETCD_INITIAL_CLUSTER="etcd1=https://10.0.50.41:2380,etcd2=https://10.0.50.42:2380,etcd3=https://10.0.50.43:2380"
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://10.0.50.41:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://10.0.50.41:2379"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_AUTO_TLS=true
ETCD_PEER_AUTO_TLS=true
and the #etcdctl cluster-health output:
cluster may be unhealthy: failed to list members
Error: unexpected status code 400
Is there a way to properly create a working cluster?
Thank you very much

Related

Build error on circleci (spin up enviroment )

I am facing a problem with the Build error on circle-ci.
this is my repo
Error
Build-agent version 1.0.41563-0e4d6629 (2020-10-22T11:30:36+0000)
Docker Engine Version: 18.09.6
Kernel Version: Linux 2e2f534dcc94 4.15.0-1077-aws #81-Ubuntu SMP Wed Jun 24 16:48:15 UTC 2020 x86_64 Linux
Starting container circleci/node:6.14.8
Warning: No authentication provided, this pull may be subject to Docker Hub download rate limits.
image cache not found on this host, downloading circleci/node:6.14.8
Error response from daemon: manifest for circleci/node:6.14.8 not found
6.14.8 is a really historic version. Are you sure you didn't mean 14.6.8, 14.8.6, etc.?
In order to see what versions CircleCI supports, see pre-built CircleCI Docker images.
You can also check official node versions you can pull at https://hub.docker.com/_/node
Off-topic: you're not authenticating your pulls. Docker is applying pull rate limit as of Nov 1st, so it would be a good idea to authenticate before the pull, see Docker auth on CircleCI for a tip.

Build or Run docker image error : CreateComputeSystem - Insufficient system resources exist to complete the requested service

i got this error on windows server 2019 standard running docker and no idea how to solve it. Running windows container with process isolation. Memory is always max 40% (32Gb), CPU always below 20% and disks lot of space (250Gb free).
Running an official container like : docker run -d --rm -p 8080:80 --name test mcr.microsoft.com/dotnet/core/samples:aspnetapp
Got this error after extracting phase and start of container :
C:\Program Files\Docker\docker.exe: Error response from daemon: CreateComputeSystem ba698710f7bff4d46dce2ee65453b9da143195770b93cf82f8bb3818fca70174: Insufficient system resources exist to complete the requested service. (extra info: {"SystemType":"Container","Name":"ba698710f7bff4d46dce2ee65453b9da143195770b93cf82f8bb3818fca70174","Owner":"docker","VolumePath":"\\\\?\\Volume{61d19b2c-50e1-4dd5-9257-9c67998f1123}","IgnoreFlushesDuringBoot":true,"LayerFolderPath":"C:\\ProgramData\\docker\\windowsfilter\\ba698710f7bff4d46dce2ee65453b9da143195770b93cf82f8bb3818fca70174","Layers":[{"ID":"05435bc5-50b1-5044-9210-12d7ab180470","Path":"C:\\ProgramData\\docker\\windowsfilter\\84fc4324bf3d2de92b5f9c0a80e238bc3b3f1fdbe0fea13b7a35146069c21663"},{"ID":"3f1c6cd1-b66d-5d8d-aa2e-d11e489b74fe","Path":"C:\\ProgramData\\docker\\windowsfilter\\11d9c87c3490cf087488aaf21a3c88b45c68273ea270c6f850261583715cf241"},{"ID":"655918d6-84bb-5633-8bb2-981f3a29e3c6","Path":"C:\\ProgramData\\docker\\windowsfilter\\1f11a5340a1c13ef658355b9e30f7794ee398fb863105d6e18b48bd8a4687f79"},{"ID":"41893d4a-0704-50df-8f4e-41cad4480d3e","Path":"C:\\ProgramData\\docker\\windowsfilter\\36296cba828c68663184ddd5e75f5084ce59b755fcc9025845fe516782b2e243"},{"ID":"dc4a1317-f05d-5470-847d-d1e4bd029e6d","Path":"C:\\ProgramData\\docker\\windowsfilter\\8293bbc58e2c9c7da089187afd6cbf0e22b1bcb27cadc3e569542e324733ef82"},{"ID":"6f247d11-4160-56dc-9dbe-b40a4b67d151","Path":"C:\\ProgramData\\docker\\windowsfilter\\d9ba1e3f6cffcc5f69e2d2265a7539a0e4aff9b4f8f86e3edc3ff5b4026a584b"},{"ID":"5f54a86d-b6e0-5f1c-977d-dc7bba046d7e","Path":"C:\\ProgramData\\docker\\windowsfilter\\9beb006814b54bda1f7afa7a0f2870a4a970ef624ab6d2bffec4e27f2b749d6e"},{"ID":"03a39db9-584f-5d5b-8bbc-95883aa2af8b","Path":"C:\\ProgramData\\docker\\windowsfilter\\38ad1eb3b36d16eb77e6264daca47fad96080cb9596aa6aeb8c6d269ffbead89"},{"ID":"bffe53fb-2619-5d30-ae46-da6f759a87ed","Path":"C:\\ProgramData\\docker\\windowsfilter\\d4b5eaee9002c7b73dd7a72880c9caa3a39b7ef5c304b19e7c0a732c01b2a4bb"},{"ID":"aaec8c93-d215-59d6-99e2-001f38fa3ee3","Path":"C:\\ProgramData\\docker\\windowsfilter\\34ccac695ab1683cc21309e94c64110476d45b07b69b6956da9e795c31f6b815"}],"HostName":"ba698710f7bf","HvPartition":false,"EndpointList":["9070F572-5CDD-4CCE-8C7E-218F0BA2F221"],"AllowUnqualifiedDNSQuery":true}).
Server: Docker Engine - Enterprise
Engine:
Version: 18.09.11
API version: 1.39 (minimum version 1.24)
Go version: go1.12.12
Git commit: 6112046bc9
Built: 11/13/2019 07:49:53
OS/Arch: windows/amd64
Experimental: false
Go to registry,
Computer\HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\NDIS\IfTypes\24
and set ifUsedNetLuidIndices to 01
and then restart the computer.

Docker pull command fails when launched in Jenkins

I am writing a Jenkins pipeline aimed to pull a docker image from a private registry. At the moment I have an issue in launching the docker pull command, since it fails when launched from Jenkins.
Currently, the Jenkins pipeline works as follow:
takes a set of parameters from user (i.e.: image name and tag)
launches a separate bash script performing the pull operation.
The bash script does two operations:
logs in the registry through the following: echo $pwd | docker login -u$username --password-stdin $registryUrl
pulls the image through the command: docker pull $image:$tag
The login command succeeds, while the pull command replies with:
Error response from daemon: access denied:
no access to Image Load, on collection swarm
At first sight I think that's a matter of user privileges, but running the bash script outside the Jenkins context (as the owner of the jenkins process) it works.
Am I missing any configuration? As an alternative, I tried to implement the Jenkins pipeline using the Docker Jenkins Plugin API, but it fails too.
Final Note
The owner of the Jenkins process is different from the user logging in Docker private registry. May this affect the behavior?
Operating System:
macOS 10.14.6
Docker Version details:
Client: Docker Engine - Community
Version: 19.03.4
API version: 1.40
Go version: go1.12.10
Git commit: 9013bf5
Built: Thu Oct 17 23:44:48 2019
OS/Arch: darwin/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.4
API version: 1.40 (minimum version 1.12)
Go version: go1.12.10
Git commit: 9013bf5
Built: Thu Oct 17 23:50:38 2019
OS/Arch: linux/amd64
Pipelines have their own commands and tricks to login to dockerhub. For example,
docker.withRegistry('https://registry.hub.docker.com', 'dockerhub-creds') {
docker.image('myteam/secretimage:latest').inside() {
checkout scm
sh './configure && make && make test'
}
}
Here, dockerhub-creds are credentials you should create in Jenkins itself.
This technique is better than echoing password because:
Password will be hidden in build's stdout and from other users who don't have access to credentials
Password is magically mirrored across users, build nodes, docker containers etc and you will not face situation that "execution context is different, nothing works"

Cassandra Detected unreadable sstables(data not caches)

ERROR [main] 2017-08-04 13:24:21,949 CassandraDaemon.java:638 - Detected unreadable sstables /opt/cassandra/data/some_key_space/ep_lc_events-adc44160dbe611e6953689bcd3ed73aa/mc-547-big-Summary.db, and many others...
That has happened after I upgraded Cassandra to 3 version and after a while downgraded it to 2nd version.
When I run this command: sudo service cassandra status
I have got such message:
could not access pidfile for Cassandra
In /var/log/cassandra/system.log I have logs which I wrote at the beginning.
PS: let me pay your attention that everything is happening on EC2 Amazon instance.
Well, I have just upgraded back to 3rd version, used cassandra-unloader to export all data, then downgraded back to 2nd version and used cassandra-loader to import all data. But if you were lucky and had backups and snapshots it would not be an obstacle for you.
PS. Afterwards, I had to run this command nodetool resetlocalschema to reset local schema and resynchronize.
PPS. This you can find how to do that.
https://github.com/brianmhess/cassandra-loader
I also got the same error, but it was due to switching between cassandra 4.0.0 and version 3.11 and back again while using docker.
Update the version to the right one for the ssltable, or delete the data volume:
docker-compose logs cassandra
docker volumes ls
docker ps
docker-compose down
docker volume rm testapp_cassandra
docker volume ls
docker-compose up

Is the latest version of docker from Amazon on ec2 broken?

As of last night all our new docker deployments started failing because the latest version of docker (docker-1.3.2-1.0.amzn1.x86_64) in the amazon repo fails to start up.
Steps to reproduce are:
## Launch instance with default amazon AMI
yum install docker-1.3.2-1.0.amzn1.x86_64
service docker restart
### Get the following error in /var/log/docker
2014/11/26 05:14:16 docker daemon: 1.3.2 c78088f/1.3.2; execdriver: native; graphdriver:
[8f6d7cfb] +job serveapi(unix:///var/run/docker.sock)
[info] Listening for HTTP on unix (/var/run/docker.sock)
docker: relocation error: docker: symbol dm_task_get_info_with_deferred_remove,
version Base not defined in file libdevmapper.so.1.02 with link time reference
If I downgrade back to docker-1.3.1-1.0.amzn1.x86_64 everything seems to be fine.
Is the AWS package actually broken, or is it just our setup?
Is there a work around other than downgrading?
Yes, it is broken for me too.
Downgrading has been the solution yet.
The same error was by me on a centos VM provisioned at my workplace - a yum update resolved it.
I suspect a build was broken but went out, and has been fixed subsequently.

Resources