Loop each file and check file size - bash

I am using Github Action to run pylint, I have a list of modified files and need to check if they are blank or not, if they are blank then skip them, how do I do this?
Part of Action:
- name: Get modified files
id: files
uses: umani/changed-files#v3.3.0
with:
repo-token: ${{ github.token }}
pattern: '^src.*\.(py)$'
result-encoding: 'string'
- name: Check files size
id: file_check
run: |
echo "file_size="$(printf "%s" "${{ steps.files.outputs.files_updated }} ${{ steps.files.outputs.files_created }}" | wc -m) >> $GITHUB_ENV
echo 'MESSAGE=No valid file found, skipped' >> $GITHUB_ENV
- name: Lint with pylint
if: env.file_size > 1
working-directory: ./
run: |
pip install pylint
OUTPUT=$(pylint ${{ steps.files.outputs.files_updated }} ${{ steps.files.outputs.files_created }} --exit-zero --jobs=0 --rcfile=.pylintrc)
LAST_LINE=$(tail -1 <<< "$OUTPUT")
SCORE=$(sed -n '$s/[^0-9-]*\([-0-9.]*\).*/\1/p' <<< "$OUTPUT")
OUTPUT=$(echo "$OUTPUT" | sed -e '20{$!N;s/\n.*/\n\n\n... results too long, run pylint locally to get full result/' -e 'q}')
OUTPUT+=". Pylint finished with score: $SCORE"
echo "SCORE=$SCORE" >> $GITHUB_ENV
echo 'MESSAGE<<EOF' >> $GITHUB_ENV
echo "$OUTPUT" >> $GITHUB_ENV
echo 'EOF' >> $GITHUB_ENV
So what I need to do is loop the variables ${{ steps.files.outputs.files_updated }} ${{ steps.files.outputs.files_created }}" which are currently encoded as comma separated strings. And check if that file is blank or not, then create a new list with non-blank files, how do I do this?

You can try something like this
IFS=, read -a fa <<<"${{ steps.files.outputs.files_updated }}"
declare -a fe fne
for f in "${fa[#]}"
do
if [[ -s "$f" ]]
then
fne+=( "$f" )
else
fe+=( "$f" )
fi
done
# fne: non-empty files
# fe: empty files

Related

Execute bash script via ansible playbook

I'm looking to run the following shell script via ansible playbook.
#!/bin/bash
oci compute instance list --lifecycle-state RUNNING --region ca-toronto-1 --compartment-id < compartment OCID> --all | grep display-name -A 0 > hostnames.txt
for line in `cat hostnames.txt`
do
#echo $line
if [[ $line == *","* ]]; then
# hostname=$(echo ${line//"display-name"/} | tr -d '",: ')
hostname=$(echo "$line" | tr -d '",')
echo "$hostname"
ssh -tt "$hostname" "sudo puppet agent -tv && sleep 10"
# break
fi
done
This works just like how i want when i run the shell script but I get a blank output when i run via ansible.
Playbook:
---
- name: puppet agent trigger
gather_facts: false
become_user: true
hosts: all
tasks:
- name: trigger puppet agent
shell: |
oci compute instance list --lifecycle-state RUNNING --region ca-toronto-1 --compartment-id <compartment OCID> --all | grep display-name -A 0 > hostnames.txt
for line in `cat hostnames.txt`
do
if [[ $line == *","* ]]; then
hostname=$(echo "$line" | tr -d '",')
echo "$hostname"
ssh -tt "$hostname" "sudo puppet agent -tv && sleep 10"
fi
done
register: shell_output
- debug:
msg: "{{ shell_output.stdout }}"
Please point me as to what im missing.
According the description of your use case it seems to be recommended to transfer the whole logic of the script into Ansible itself. To do so and in respect to the comment regarding add_host_module, you could use an approach like
- name: Create an instance list
shell:
cmd: oc get nodes --no-headers | cut -d " " -f 1 | tr '\n' "," | rev | cut -c 2- | rev
register: instance_list
changed_when: false
To generate the instance_list I've used an OpenShift cluster as example, because I don't have something like oci compute instance list ... accessible.
- name: Add all hosts from instance_list to the script group
add_host:
name: '{{ item }}'
groups: script
loop: "{{ instance_list }}"
From there you could proceed further to execute your command puppet agent -tv && sleep 10.

Gitlab CI/CD create and use custom user functions

I tried create like this function in my gitlab config file:
deploy:
stage: dev
services:
- docker:dind
script:
- myFunction () { api_pl_tmp=$(curl -s --header "PRIVATE-TOKEN: $TOKEN_VAR" "https://git.example.ru/api/v4/projects/1/pipelines/latest" | jq .) }
- while myFunction; do
- if [ $(echo $api_pl_tmp | jq -r .status) = "success" ]
- then
- export PROJECT_CURRENT=$($api_pl_tmp | jq -r '{id:.id,sha:.sha[0:8]}' | base64)
- break
- fi
- if [ $(echo $api_pl_tmp | jq -r .status) = "failed" ]
- then
- echo "Error: Frontend can't be deployed!"
- exit 1
- fi
- if [ $(echo $api_pl_tmp | jq -r .status) = "running" ]
- then
- echo "Wait 5 sec... Frontend deploying!"
- sleep 5
- else
- echo Unknow status $(echo $api_pl_tmp | jq -r .status)
- exit 1
- fi
- done
But it's doesn't work and gitlab return me error with message:
This GitLab CI configuration is invalid: jobs:deploy-to-dev:script
config should be a string or a nested array of strings up to 10 levels
deep
How I can fix this problem or maybe I have an error in my custom function?
- separates commands with commands in between. Put your commands as one command, not multiple. Remember they are joined with spaces.
The problem with your script is the colon - see https://gitlab.com/gitlab-org/gitlab-foss/-/issues/30097 .
deploy:
stage: dev
services:
- docker:dind
script:
- "colon=:"
- myFunction () {
api_pl_tmp=$(curl -s --header "PRIVATE-TOKEN$colon $TOKEN_VAR" \
"https$colon//git.example.ru/api/v4/projects/1/pipelines/latest" | jq .);
}
- while myFunction; do
if [ $(echo $api_pl_tmp | jq -r .status) = "success" ]; then
export PROJECT_CURRENT=$($api_pl_tmp | jq -r '{id:.id,sha:.sha[0:8]}' | base64);
break;
fi;
if [ $(echo $api_pl_tmp | jq -r .status) = "failed" ]; then
echo "Error$colon Frontend cant be deployed";
exit 1;
fi;
if [ $(echo $api_pl_tmp | jq -r .status) = "running" ]; then
echo "Wait 5 sec... Frontend deploying!";
sleep 5;
else
echo Unknow status $(echo $api_pl_tmp | jq -r .status);
exit 1;
fi;
done
Also jq .) } is missing a ;. Check your scripts in your own shell one at a time first. Check your scripts with https://shellcheck.net .
Also $($api_pl_tmp is missing an echo and there are a lot of problem with quoting. Use consistent indentation and try to write readable code to minimize typos.

How to pass value to ansible command module

I am new to ansible and trying to write some basic playbook. I need to run a command via command module, but that command needs use a value derived from other command this is what I have:
+++++++++++++++++++++++++++++++++++++++++++
---
- name: Post SentinelOne Agent
hosts: all
tasks:
- name: user sentinel exist
command: /usr/bin/grep sentinelone /etc/passwd
register: user_exist
ignore_errors: yes
- name: Create sentinelone user using existing id, only if the user does not exist
command: "useradd -d /opt/sentinelone/home/ -u `ls -ld /opt/sentinelone/ | awk '{print $3}'`** -f 1 -l -g sentinelone -s /sbin/nologin sentinelone"
when: user_exist.rc == 1
- name: Check again to confirm user exist for copy purpose
command: /usr/bin/grep sentinelone /etc/passwd
register: does_user_exist
ignore_errors: yes
- name: Whitelist sentinelone user ... Copy line from /etc/passwd
command: echo "`cat /etc/passwd | grep -i sentinelone`" >> /etc/crypt/passwd_local
when: does_user_exist.rc == 0
- name: Whitelist sentinelone user ... Copy line from /etc/shadow
command: echo "`cat /etc/shadow | grep -i sentinelone`" >> /etc/crypt/shadow_local
when: does_user_exist.rc == 0
+++++++++++++++++++++++++++++
get error:
for
command: "useradd -d /opt/sentinelone/home/ -u `ls -ld /opt/sentinelone/ | awk '{print $3}'`** -f 1 -l -g sentinelone -s /sbin/nologin sentinelone"
STDERR:
useradd: invalid user ID '`ls'
MSG:
non-zero return code

Ansible: escape quotes and listing users password expiry information

How to escape in shell module quotation?
I've tried as follows:
- name: UList
shell: "cut -f 1 -d: /etc/passwd | sudo xargs -n 1 -I {} bash -c \" echo {} ; chage -l {}\""
and
- name: UList
shell: "cut -f 1 -d: /etc/passwd | sudo xargs -n 1 -I {} bash -c \' echo {} ; chage -l {}\'"
Where is a mistake?
This playbook worked for me, hope this will be helpful for you too. You can use single quotes' for a command if you have an issue escaping " quote
Either do it this way
'cut -f 1 -d: /etc/passwd | sudo xargs -n 1 -I {} bash -c " echo {} ; chage -l {}"'
Or
"cut -f 1 -d: /etc/passwd | sudo xargs -n 1 -I {} bash -c ' echo {} ; chage -l {}'"
Both are working i have tested it.
---
- name: Set my hosts variable
hosts: localhost
tasks:
- name: UList
shell: 'cut -f 1 -d: /etc/passwd | sudo xargs -n 1 -I {} bash -c " echo {} ; chage -l {}"'
register: result
- name: debug
debug:
msg: "{{result}}"
Or
For the output you are expecting for that you can use
awk -F':' '{ system("echo " $1 " && chage -l " $1) }' /etc/passwd
Command explanation
---
- name: Set my hosts variable
hosts: localhost
tasks:
- name: UList
shell: "awk -F':' '{ system(\"echo \" $1 \" && chage -l \" $1) }' /etc/passwd"
register: result
- name: debug
debug:
msg: "{{result}}"

Setting GitLab environmental variable via a bash command

So. I'm using GitLab and in my .gitlab-ci.yml file I have the following:
variables:
CONTAINER_NAME: $CI_BUILD_REF_NAME | tr / -
The $CI_BUILD_REF_NAME basically reads the branch name, in my case development/karl. I'm trying to set the variable name CONTAINER_NAME to development-karl though.
It doesn't seem to be working as when I echo the variable, the following shows:
$ echo $CONTAINER_NAME
development/karl | tr / -
I was expecting:
$ echo $CONTAINER_NAME
development-karl
I've also tried:
variables:
CONTAINER_NAME: $( echo "$CI_BUILD_REF_NAME" | tr / - )
CONTAINER_NAME_2: $( tr / - <<<"$CI_BUILD_REF_NAME" )
CONTAINER_NAME_3: "${CI_BUILD_REF_NAME////-}"
//.. more script stuff
script:
- echo $CONTAINER_NAME
- echo $CONTAINER_NAME_2
- echo $CONTAINER_NAME_3
And got:
Successfully extracted cache
$ echo $CONTAINER_NAME
( echo "development/karl" | tr / - )
$ echo $CONTAINER_NAME_2
( tr / - <<<"development/karl" )
$ echo $CONTAINER_NAME_3
$ npm install

Resources