I want to run a section of the script only when the build is off a push and not a PR. But on build it doesn't go. How can I only run the section when the build is off a push?
if [ "$TRAVIS_PULL_REQUEST" == "false" ]; then
setup_git
commit_website_files
publish_gh_pages
fi
Just use secure environment variables to determine if PR or push. Secure environment variables aren't available in push
if [ "${ghToken:-false}" != "false" ]; then
setup_git
commit_website_files
publish_gh_pages
fi
Related
My thought is that, in order to automatically fetch updates only if upstream has pushed some commits, I should check its hashes.
2 ways, I could do is that to check the logs or use rev-list something like below:
# this fetches top 30 commits from forked repo
git rev-list -n 30 main
# this also does the same but not limited to 30
git log --format='%H'
I'm finding it difficult to compare the hashes as I'm failing to fetch more than 1 commit from forked repo.
Explained my approach below:
git remote add upstream https://github.com/upstream/upstream.git
git fetch upstream --tags
upstream_commit="$(git rev-parse upstream/main)"
echo "Upstream commit: ${upstream_commit}"
# This should store 30 recent commit hashes as an array AFAIK
forked_commits=($(git rev-list -n 30 main))
# another I tried to put forloop is this:
# for forked_commit in "$forked_commits[#]"; do
for forked_commit in $(git log --format='%H'); do
# but prints only the top most element and exits the loop
echo "$forked_commit"
if [ "$upstream_commit" == "$forked_commit" ]; then
has_new_commits=false
else
# since I've added my commits on top, if condition fails and this always returns true
has_new_commits=true
fi
done
if [ "${has_new_commits}" ]; then
git checkout main
git rebase upstream/main
git push -f origin main
echo "Rebase successful!"
else
echo "No commits to be synced!"
fi
Ok, I did many mistake that were minor.
Array stuff didn't work. I was supposed to put them directly within the forloop.
Next mistake I never validated them properly. i.e., even if the hash was found, it never exited.
Using [ $has_new_commits ]; then is wrong. I should be using [ $has_new_commits == "true"]; then instead.
Corrected code:
git remote add upstream https://github.com/upstream_repo/upstream.git
git fetch upstream --tags
upstream_commit="$(git rev-parse upstream/main)"
for forked_commit in $(git rev-list -n 20 main); do
if [ $upstream_commit != $forked_commit ]; then
has_new_commits=true
continue
else
has_new_commits=false
break
fi
done
if [ $has_new_commits == "true" ]; then
echo "New commits found!"
else
echo "No commits to be synced!"
fi
I'm trying to create an output to use later in the job.
However, for some reason, the BRANCH env variable which I'm getting to be the GITHUB_REF_NAME is an empty string, which according to the docs, should be the branch.
Also using the variable directly produces the same result.
- name: Set Terraform Environment Variable
id: set_tf_env
env:
BRANCH: ${{env.GITHUB_REF_NAME}}
run: |
if [ "$BRANCH" == "dev" ]; then
run: echo "::set-output name=TF_ENV::dev"
elif [ "$BRANCH" == "prod" ]; then
run: echo "::set-output name=TF_ENV::prod"
else
echo "Branch has no environment"
exit 1
fi
So after a bit of more research and thanks to the comments, I discovered the reason why it wasn't working.
It was because I was triggering a GitHub action in a Pull Request, something I failed to mention.
So what I ended up using was:
github.event.pull_request.head.ref
A branch job in which the branch property of the trigger property is using a variable will always fail with reason: downstream pipeline can not be created.
Steps to reproduce
Set up a downstream pipeline with a trigger property as you would normally.
Add a branch property to the trigger property. Write the name of an existing branch on the downstream repository, like master/main or the name of a feature branch.
Run the pipeline and observe that the downstream pipeline is successfully created.
Now change the branch property to use a variable instead, like branch: $CI_TARGET_BRANCH.
Manually run the CI pipeline with that, setting variable through the GitLab GUI.
The job will instantly fail with reason: downstream pipeline can not be created.
Code example
The goal is to create a GitLab CI config that runs the pipeline of a specified downstream branch. The bug occurs when attempting to do it with a variable.
This works, creating a downstream pipeline like normal. But the branch name is hardcoded:
stages:
- deploy
deploy:
variables:
environment: dev
stage: deploy
trigger:
project: group/project
branch: foo
strategy: depend
This does not work; although TARGET_BRANCH is set successfully, the job fails because the downstream pipeline can not be created:
stages:
- removeme
- deploy
before_script:
- if [ -z "$TARGET_BRANCH" ]; then TARGET_BRANCH="main"; fi
- echo $TARGET_BRANCH
test_variable:
stage: removeme
script:
- echo $TARGET_BRANCH
deploy:
variables:
environment: dev
stage: deploy
trigger:
project: group/project
branch: $TARGET_BRANCH
strategy: depend
If you know what I'm doing wrong, or you have something that does work with variable expansion of the branch property, please share it (along with your GitLab version). Alternate solutions are also welcome, but this one seems like it should work.
GitLab Version on which bug occurs
Self-hosted GitLab Community Edition 12.10.7
What is the current bug behavior?
The job always fails for reason: downstream pipeline can not be created.
What is the expected correct behavior?
The branch property should be set to the value of the variable and the downstream pipeline should be created as normal, just as if you simply hardcoded/typed the name of the branch.
More details
The ability to use variable expansion in the trigger branch property was added in v12.4, and it's explicitly mentioned in the docs.
I searched for other .gitlab-ci.yml / GitLab config files. Every single one that attempted to use variable expansion in the branch property had it commented out, saying it was bugged for an unknown reason (example.
I haven't been able to find a repository in which someone claimed to have a working variable expansion for the branch property of the trigger property.
Unfortunately, the alternate solutions are either (a) hardcoding every downstream branch name into the GitLab CI config of the upstream project, or (b) not being able to test changes to the downstream GitLab CI config without first committing them to master/main, or having to use only/except.
TL;DR: How to use the value of a variable for the branch property of a bridge job? My current solution makes it so the job fails and the downstream pipeline isn't created.
this is a 'works as designed', and gitlab will improve in upcoming releases.
trigger job will pretty weak b/c it is not a full job that runs on a runner. Therefore most of the trigger configuration needs to be hardcoded.
I use direct API calls to trigger downstream jobs passing the CI_JOB_TOKEN which links the upstream job to downstream as the trigger does
API calls give you full control
curl -X POST \
-s \
-F token=${CI_JOB_TOKEN} \
-F "ref=${REF_NAME}" \
-F "variables[STAGE]=${STAGE}" \
"${CI_SERVER_URL}/api/v4/projects/${CI_PROJECT_ID}/trigger/pipeline"
now this will not wait and monitor for when the job is done so you will need to code for that if you need to wait for the downstream job to finish,
Moreover, CI_JOB_TOKEN cannot be used to get the status of the downstream job, so you will another token for that.
- |
DOWNSTREAM_RESULTS=$( curl --silent -X POST \
-F token=${CI_JOB_TOKEN} \
-F "ref=${DOWNSTREAM_PROJECT_REF}" \
-F "variables[STAGE]=${STAGE}" \
-F "variables[SLS_PACKAGE_PATH]=.serverless-${STAGE}" \
-F "variables[INVOKE_SLS_TESTS]=false" \
-F "variables[UPSTREAM_PROJECT_REF]=${CI_COMMIT_REF_NAME}" \
-F "variables[INSTALL_SLS_PLUGINS]=${INSTALL_SLS_PLUGINS}" \
-F "variables[PROJECT_ID]=${CI_PROJECT_ID}" \
-F "variables[PROJECT_JOB_NAME]=${PROJECT_JOB_NAME}" \
-F "variables[PROJECT_JOB_ID]=${PROJECT_JOB_ID}" \
"${CI_SERVER_URL}/api/v4/projects/${DOWNSTREAM_PROJECT_ID}/trigger/pipeline" )
echo ${DOWNSTREAM_RESULTS} | jq .
DOWNSTREAM_PIPELINE_ID=$( echo ${DOWNSTREAM_RESULTS} | jq -r .id )
echo "Monitoring Downstream pipeline ${DOWNSTREAM_PIPELINE_ID} status..."
DOWNSTREAM_STATUS='running'
COUNT=0
PIPELINE_API_URL="${CI_SERVER_URL}/api/v4/projects/${DOWNSTREAM_PROJECT_ID}/pipelines/${DOWNSTREAM_PIPELINE_ID}"
echo "Pipeline api endpoint => ${PIPELINE_API_URL}"
while [ ${DOWNSTREAM_STATUS} == "running" ]
do
if [ $COUNT -eq 0 ]
then
echo "Starting loop"
fi
if [ ${COUNT} -ge 350 ]
then
echo 'TIMEOUT!'
DOWNSTREAM_STATUS="TIMEOUT"
break
elif [ $(( ${COUNT} % 60 )) -eq 0 ]
then
echo "Downstream pipeline status => ${DOWNSTREAM_STATUS}"
echo "Count => ${COUNT}"
sleep 10
else
sleep 10
fi
DOWNSTREAM_CALL=$( curl --silent --header "PRIVATE-TOKEN: ${GITLAB_TOKEN}" ${PIPELINE_API_URL} )
if [ $COUNT -eq 0 ]
then
echo ${DOWNSTREAM_CALL} | jq .
fi
DOWNSTREAM_STATUS=$( echo ${DOWNSTREAM_CALL} | jq -r .status )
COUNT=$(( ${COUNT} + 1 ))
done
#pipeline status is running, failed, success, manual
echo "PIPELINE STATUS => ${DOWNSTREAM_STATUS}"
if [ ${DOWNSTREAM_STATUS} != "success" ]
then
exit 2
fi
This question already has answers here:
Dockerfile if else condition with external arguments
(14 answers)
Closed 3 years ago.
Never wrote any shell script before but after extensively googling it I came up with the following code for my docker file. But don't understand why is it doesn't work.
###stage 2####################
FROM nginx:alpine
##########Calculate the environment type #########
ARG BUILD_TYPE
####echo of build build_type does gives me output of Development when passed argument is Development.
RUN if [ "$BUILD_TYPE" = "Development" ]; then BUILD_TYPE='dev'; fi
RUN if [ "$BUILD_TYPE" = "Production" ]; then BUILD_TYPE='prod'; fi
RUN echo "UI BUILD_TYPE=$BUILD_TYPE---------"
##########Calculate the environment type #########
The above echo always comes as Development.
UPDATE
Now I built a sample in a separate docker file to isolate the issue. After this I realised that the assignment is not happening though the condition matched.
Here is the new sample docker file code.
FROM nginx:alpine
ARG BUILD_TYPE
ARG ENV_TYPE
RUN if [ "$BUILD_TYPE" = "Development" ]; then ENV_TYPE='dev'; echo "matched dev"; fi
RUN if [ "$BUILD_TYPE" = "Production" ]; then ENV_TYPE="prod"; echo "matched prod"; fi
RUN echo "UI BUILD_TYPE=$BUILD_TYPE ENV_TYPE = $ENV_TYPE---------"
The output is
matched dev
UI BUILD_TYPE=Development ENV_TYPE = ---------
I see ENV_TYPE is empty.
Each RUN command in a Dockerfile is executed in a separate shell session, so when you set BUILD_TYPE, you are setting an environment variable for that session only, which overrides the build-argument. You are not overwriting the build-argument for the entire docker build.
You can see this by the fact that if you change your if statements to:
RUN if [ "$BUILD_TYPE" = "Development" ]; then BUILD_TYPE='dev'; fi; echo $BUILD_ENV
RUN if [ "$BUILD_TYPE" = "Production" ]; then BUILD_TYPE='prod'; fi; echo $BUILD_ENV
The env var is correctly set, and echoed at the end of the line but your final echo will still return the build argument.
If you instead put these statements in a shell script and run that instead, it works just fine:
build.sh:
####echo of build build_type does gives me output of Development when passed argument is Development.
if [ "$BUILD_TYPE" = "Development" ]; then BUILD_TYPE='dev'; fi
if [ "$BUILD_TYPE" = "Production" ]; then BUILD_TYPE='prod'; fi
echo "UI BUILD_TYPE=$BUILD_TYPE---------"
##########Calculate the environment type #########
Dockerfile:
###stage 2####################
FROM nginx:alpine
##########Calculate the environment type #########
ARG BUILD_TYPE
COPY build.sh .
RUN ./build.sh
Output:
docker build --build-arg BUILD_TYPE=Production .
Sending build context to Docker daemon 166.9kB
Step 1/4 : FROM nginx:alpine
---> 36189e6707f4
Step 2/4 : ARG BUILD_TYPE
---> Running in cab2e8749e7e
Removing intermediate container cab2e8749e7e
---> ea9ec7779909
Step 3/4 : COPY build.sh .
---> 336989bf6389
Step 4/4 : RUN ./build.sh
---> Running in ecd09ee58780
UI BUILD_TYPE=prod---------
Removing intermediate container ecd09ee58780
---> ed9ca30af483
Successfully built ed9ca30af483
Using Fake 5.0 on TeamCity. Prior to 5.0 if there was a compilation error the error would be visible in the build failure summary. However now moving to 5.0 if there is an error the details in the summary are the generic output from Fake.
In order to diagnose you have to then dig through the logs to find the compilation error.
This may not be specific to TeamCity because the same outputs are reported from the console.
Wondering if there is a configuration that I am missing either in the way that fake is being run or how the tasks are configured that needs to be set to allow the actual error to propagate up.
Running build script from TeamCity using bash:
%env.BashPath% build.sh run build.fsx
Bash script as per the getting started examples:
#!/usr/bin/env bash
set -eu
set -o pipefail
# liberated from https://stackoverflow.com/a/18443300/433393
realpath() {
OURPWD=$PWD
cd "$(dirname "$1")"
LINK=$(readlink "$(basename "$1")")
while [ "$LINK" ]; do
cd "$(dirname "$LINK")"
LINK=$(readlink "$(basename "$1")")
done
REALPATH="$PWD/$(basename "$1")"
cd "$OURPWD"
echo "$REALPATH"
}
TOOL_PATH=$(realpath .fake)
FAKE="$TOOL_PATH"/fake
if ! [ -e "$FAKE" ]
then
dotnet tool install fake-cli --tool-path $TOOL_PATH --version 5.*
fi
"$FAKE" "$#"
Running the MSBuild task:
Target.create "Build" (fun _ ->
solutionFile
|> MSBuild.build (fun p ->
{ p with
ToolsVersion = Some "15.0"
Verbosity = Some(Quiet)
Targets = ["Build"]
Properties = ["Optimize", "True"
"DebugSymbols", "True"
"Configuration", "Release"
"RunCodeAnalysis", "True"
"CodeAnalysisGenerateSuccessFile", "False"]
}))