I'm trying to use GitLab Pipelines to auto-build an AWS image with Packer, then auto-run tests with the given AMI string from the console output.
The Gitlab process looks like this:
Gitlab pipeline build stage: Buil an AWS image with Packer, which also installs an Apache service from provisioners.
Gitlab pipeline Test stage: Use the given AMI string from the console to launch a new AWS instance, then run some commands to verify if the Apache service up and run.
Gitlab pipeline Deployment stage
So far, from the build stage, everything works as expected, I can see the AMI string from the console, but I have no clue about the Test stage.
Console:
==> Wait completed after 4 minutes 33 seconds
==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:
us-east-1: ami-0ea59f553d87cff53
--> amazon-ebs: AMIs were created:
us-east-1: ami-0ea59f553d87cff53
Here's my question:
How to get the AMI string and use it in the next stage(Test stage)?
Can I still use Packer for testing purposes in the Test stage, by starting up an instance with the given AMI, and running commands from provisioners? Or is there any other better way to do the test?
Note, I only want to test the built image from a new instance.
Any feedback is appreciated.
Related
I am deploying my Spring Boot application as a compiled jar file running in a docker container deployed to gcp, and deploys it through gcloud cli in my pipeline:
gcloud beta run deploy $SERVICE_NAME --image $IMAGE_NAME --region europe-north1 --project
Which will work and give me the correct response when the application succeeds to start. However, when there's an error and the application fails to start:
Cloud Run error: The user-provided container failed to start and listen on the port defined provided by the PORT=8080 environment variable.
The next time the pipeline runs (with the errors fixed), the gcloud beta run deploy command fails and gives the same error as seen above. While the actual application runs without issues in Cloud Run. How can I solve this?
Currently I have to check Cloud Run manually as I cannot trust my pipeline, and I have to run it twice to make it succeed. Any help will be appreciated. Let me know if you want any extra information.
I have got an assignment. The assignment is "Write a shell script to install and configure docker swarm(one master/leader and one node) and automate the process using Jenkins." I am new to this technology and finding it difficult to proceed. Can anyone help me in explaining step-by-step process of how to proceed?
#Rajnish Kumar Singh, Have you tried to check resources online? I understand you are very new to this technology, but googling some key words like
what is docker swarm
what is jenkins , etc would definitely helps
Having said that, Basically you need to do below set of steps to complete your assignment
Pre-requisites
2 or more - Ubuntu 20.04 Server
(You can use any linux distros like ubuntu, Redhat etc, But make sure your install and execute commands change accordingly.
Here we need two nodes mainly to configure the master and worker node cluster)
Eg :
manager --- 132.92.41.4
worker --- 132.92.41.5
You can create these nodes in any of public cloud providers like AWS EC2 instances or GCP VMs etc
Next, You need to do below set of steps
Configure Hosts
Install Docker-ce
Docker Swarm Initialization
You can refer this article for more info https://www.howtoforge.com/tutorial/ubuntu-docker-swarm-cluster/
This completes first part of your assignment.
Next, You can create one small shell script and include all those install and configuration commands in that script. Basically shell script is collection of set of linux commands. Instead of running each commands separately , you will run script alone and all set up will be done for you.
You can create small script using touch command
touch docker-swarm-install.sh
Specify proper privileges to script to make it executable
chmod +x docker-swarm-install.sh
Next include all your install + configure commands, which you have used earlier to do docker swarm set up in scripts (You can refer above shared link)
Now, when your script is ready, you can configure this script in jenkins job and whenever jenkins job is run, script will get execute and docker swarm cluster will be created
You need a jenkins server. Jenkins is open source software, you can install it in any of public cloud instance (Aws EC2)
Reference : https://devopsarticle.com/how-to-install-jenkins-on-aws-ec2-ubuntu-20-04/
Next once installation is completed. You need to configure job in jenkins
Reference : https://www.toolsqa.com/jenkins/jenkins-build-jobs/
Add your 'docker-swarm-install.sh' as build step in created job
Reference : https://faun.pub/jenkins-jobs-hands-on-for-the-different-use-cases-devops-b153efb483c7
If all set up is successful and now when you run your jenkins job, your docker swarm cluster must be get created.
We are constantly getting an error while starting our Beam Golang SDK pipeline (driver program) from a docker image which works when started from local / VM instance. We are using Dataflow runner for our pipeline and Kubernetes to deploy.
LOCAL SETUP:
We have GOOGLE_APPLICATION_CREDENTIALS variable set with service account for our GCP cluster. When running the job from local, job gets submitted to dataflow and completes successfully.
DOCKER SETUP:
Build image used is FROM golang:1.14-alpine. When we pack the same program with Dockerfile and try to run, it fails with error
User program exited: fork/exec /bin/worker: no such file or directory
On checking Stackdriver logs for more details, we see this:
Error syncing pod 00014c7112b5049966a4242e323b7850 ("dataflow-go-job-1-1611314272307727-
01220317-27at-harness-jv3l_default(00014c7112b5049966a4242e323b7850)"),
skipping: failed to "StartContainer" for "sdk" with CrashLoopBackOff:
"back-off 2m40s restarting failed container=sdk pod=dataflow-go-job-1-
1611314272307727-01220317-27at-harness-jv3l_default(00014c7112b5049966a4242e323b7850)"
Found reference to this error in Dataflow common errors doc, but it is too generic to figure out whats failing. After multiple retries, we were able to eliminate any permission / access related issues from pods. Not sure what else could be the problem here.
After multiple attempts, we decided to start the job manually from a new Debian 10 based VM instance and it worked. This brought to our notice that we are using alpine based golang image in Docker which may not have all the required dependencies installed to start the job.
On golang docker hub, we found a golang:1.14-buster where buster is codename for Debian 10. Using that for docker build helped us solve the issue. Self answering here to help anyone else facing the same issues.
I have used cloud bees Jenkins account to setup my build to run on a Jenkins Node machine which is an AWS system(Windows Server 2016). I have setup SSH(port 22) on AWS system using CYGWIN, configured and launched agent on this machine using following launch method:
Now my tests are running on the Jenkins Node (AWS Machine) but I am not able to see the GUI for it. Please let me know what setting I need to enable to make it visible? also let me know if more details are required.
I need to install GitLab on a server running Windows 7, but I'm blocked at this line. The documentation doesn't really helping me. The following is from my command prompt:
C:\GitLab-Runner>gitlab-runner.exe register
Please enter gitlab-ci coordinator URL (e.g. https://gitlab.com/):
https://gitlab.com
Please enter the gitlab-ci token for this runner:
Where can I find this token?
You're attempting to install the GitLab Runner which is used to run your jobs and send the results to a GitLab which is a server. As you're talking about GitLab running at a server you have to install that and not the Runner.
But it is not supported to install GitLab on Windows, see here in the GitLab forum. They recommend to use Linux in a virtual machine for that if you want it on Windows.
In all seriousness this is something that will probably never be supported.
Nevertheless to get the needed project registration token follow these steps described here. Also there is a discussion about it on GitHub.
To create a specific Runner without having admin rights to the GitLab instance, visit the project you want to make the Runner work for in GitLab:
Go to Settings ➔ Pipelines to obtain the token
Register the Runner
Further the process of registering the GitLab Runner which is actually what you're doing is described here.