Trying to read values from a file inside gitlab-ci.yml file:
#!/usr/bin/bash
- IN_FILE="base_dir/apps_namespaces.txt"
- echo "IN_FILE = " > $IN_FILE
- while IFS= read -r line || [ -n "$line" ]; do
- echo "Name read from file - " > $line
- done < $IN_FILE
But this script fails with error - "/usr/bin/bash: line 131: read: line: invalid number"
Whereas same logic works in a plain script file, having following code:
#!/bin/bash
while IFS= read -r line || [[ -n "$line" ]];
do
echo "Text read from file: $line"
done < "$1"
when ran this script using command : ./scrpt apps_namespaces.txt
it rans and print each line.
What can be done if needs a long list to be read inside a gitlab-ci yml ?
Check if you can emulate this gitlab-ci.yml loop example, which:
does not include the shebang (#!/bin/bash)
does not have a - in front of each line
script:
[...]
- while read line; do
echo $line;
./sireg-linux exec --loader-sitemap-sitemap \"$line\" >> ./output/${line##*/}_out.txt;
done < sitemap-index
Of course, the alternative is to put the logic in a script, called from the pipeline, as documented in gitlab-org/gitlab-runner issue 2672
This is basic Unix, you just need your script to be executable in your git repo
chmod a+x scripts/test.sh
git add scripts/test.sh
git commit "fixing permissions"
git push
don't put /usr/bin/env for sh nor bash: both are 100% guaranteed to be present as #! /bin/sh or /bin/bash if bash is installed, or at least a much guaranteed to be there than /usr/bin/env to exists (and using env sh is not very good security)
A better approach which I followed:
put the logic to read file contents inside a script file. And refer this script inside gitlab-ci.yml worked.
It is cleaner and manageable.
Related
so im trying to create a bash script that runs on MAC command line to a remote server and uses some mv commands to move some files around but i also need it to open up a file and add a line to the top of the file and save it in the middle of the script heres what i have so far:
(this is adjusting permissions so i can edit the file)
chef-client -r xxxxxxredactedxxxxxxredacted
cd /xxx/postgresql/xx/main/
Sudo chmod -R 775 filenamehere
sudo chown -R postgres:postgres filenamehere
read -p 'Enter the IP: ' ip
echo "Enter the file name :"
read -r filename
echo "Type this below: host all all "$ip"/24 trust : "
read -r line
cd /etc/postgresql/12/main/
printf '1i\n%s\n.\nwq\n' "$line" | ed "$filename". <-- **this is the problem line**
^ this command gives me permission denied because of access, for some reason i can edit it with vim but not this command
its worth noting these commands arent ran through my pc so my ability to move files is somewhat limited, its ran through SSM ing into an IP of a test enviroment through my command line
Normally I manually VIM into the file and add a line to the top
Don't know if you're using echo to output the prompts because you didn't know about the -p read option or you wanted the new lines.
You could use command grouping to add a line at the top of your file.
read -p "Have you copied a file to the data shipper? (yes/no)"
if [ "$REPLY" == "yes" ]; then
read -p "Enter a variable: " VARIABLE
read -p "Enter a file name: " FILE
cd /var/xxxredacted////
cd /tmp/
sudo mv "$FILE" /varxxxredactedxxxxxxxx/drop
cd /var/redactedxxxxredactedxx/drop
sudo unzip "$FILE"
fi
read -p "Enter the file name:\n" FILENAME
read -p "Enter the line to be added:\n" LINE
{ echo $LINE; cat "$FILENAME"; } > "${FILENAME}.new"
mv "$FILENAME"{.new,}
sed could be used too, if the line had to go to a specific line :
# If \n is omnited, $LINE will just be inserted on
# line 1 instead of appending a new line
sed -i "${LINENB}s/$LINE\n/" $FILENAME
enter image description here
hello, this is a file I created in google shell. I want to plus three for each line such as the first line will be 699 from 696. I tried to use "while read line; do c="line" c=c+3 "line"; done
You can use the following script:
#!/bin/bash
while read -r line || [[ -n "$line" ]]; do
echo $(($line+3));
done < "$1"
After creating it you need to give execution permission to your user:
chmod u+x my_script.sh
before calling it in the following way:
./my_script.sh file.in > file.out
the result will be saved in the file.out.
Good luck!
TODO
Error management when you do have a line that is not a number, or contains other characters.
Suppose there's a script called 'test.sh':
#!/bin/bash
while read line; do
APP=/apps echo "$line"
done < ./lines
And the 'lines':
cd $APP && pwd
If I bash test.sh, it prints out 'cd $APP && pwd'.
But when I type APP=/apps echo "cd $APP && pwd" in the terminal, it prints out 'cd /apps && pwd'.
Is it possible using echo to extract variables which are reading from a regular file?
Depending on the contents of the file, you may want to use eval:
#!/bin/bash
APP=/apps
while read line; do
eval "echo \"$line\"" # WARNING: dangerous
done < ./lines
However, eval is extremely dangerous. Although the quoting here will work for simple cases, it is quite easy to execute arbitrary commands by manipulating the input.
You should use eval to evaluate string line read from file
If you know the variable(s) you want to substitute, just substitute them.
sed 's%\$APP\>%/apps%g' ./lines
I have a script called autoinstall:
#!/usr/bin/sh
echo "Installasi membutuhkan free space minimal 2MB, pastikan ada punya cukup space di router anda"
read -p "Anda yakin ingin melanjutkan installasi?(y/n) " -n 1 -r
echo ""
if [[ $REPLY = ^[Yy]$ ]]
then
cd /
cd /tmp/
tar -xvf OpenWrt_Angel_Beats_Edition_v1.3.3.tar -C /
chmod -R 744 /root/crt
chmod 744 /www/wget/wget_download.sh
chmod 744 /usr/bin/gsm
chmod 744 /usr/bin/profile
opkg update && opkg install elinks
cp /etc/rc.local /etc/rc.local.backup
cat > /etc/rc.local << END
#!bin/sh
# /etc/rc.local: Local system initialization script.
#
# Put any local startup commands in here. Also, if you have
# anything that needs to be run at shutdown time you can
# make an /etc/rc.d/rc.local_shutdown script and put those
# commands in there.
sh /www/wget/wget_download.sh > /dev/null 2>&1 &
exit 0
END
killall sh /www/wget/wget_download.sh
sh /www/wget/wget_download.sh > /dev/null 2>&1 &
echo "File backup /etc/rc.local.backup telah dibuat, gunakan file ini untuk mengembalikan konfigurasi rc.local anda yang dulu jika diperlukan"
echo "Installasi selesai. Jangan lupa di akun openvpn yang digunakan (/root/crt/xxx.ovpn) tambahkan baris ini:
script-security 2
up client-connect.sh"
else
echo ""
echo "Installasi dibatalkan"
fi
Every command that I put in the first line always gets the error above (line 1:xxx not found) and I'm sure I've typed in the correct command, even echo gives the error like that, how do I solve this?
There can be two problems here:
The file doesn't exist. Usually, for sh, the path is /bin/sh, so it should be #!/bin/sh
You're editing the file on Windows. Windows uses CR+LF as line ending. Unix (and Linux) uses just LF. So for Linux, the command reads "execute /bin/sh<CR> and sh<CR> doesn't exist.
Solution: When editing the file, make sure you use Unix line endings.
The file might have been edited with an editor that insert a Unicode BOM (Byte Order Mark).
Have a look to the first line contents with:
od -c autoinstall | head -1
or
hd -n 16 autoinstall
If you see unexpected characters before #!/usr/bin/sh, you might try one of the methods described here Using awk to remove the Byte-order mark to remove the BOM.
Is it possible to save last entered value of a variable by the user in the bash script itself so that I reuse value the next time while executing again?.
Eg:
#!/bin/bash
if [ -d "/opt/test" ]; then
echo "Enter path:"
read path
p=$path
else
.....
........
fi
The above script is just a sample example I wanted to give(which may be wrong), is it possible if I want to save the value of p permanently in the script itself to so that I use it somewhere later in the script even when the script is re-executed?.
EDIT:
I am already using sed to overwrite the lines in the script while executing, this method works but this is not at all good practice as said. Replacing the lines in the same file as said in the below answer is much better than what I am using like the one below:
...
....
PATH=""; #This is line no 7
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )";
name="$(basename "$(test -L "$0" && readlink "$0" || echo "$0")")";
...
if [ condition ]
fi
path=$path
sed -i '7s|.*|PATH='$path';|' $DIR/$name;
Someting like this should do the asked stuff :
#!/bin/bash
ENTERED_PATH=""
if [ "$ENTERED_PATH" = "" ]; then
echo "Enter path"
read path
ENTERED_PATH=$path
sed -i 's/ENTERED_PATH=""/ENTERED_PATH='$path'/g' $0
fi
This script will ask user a path only if not previously ENTERED_PATH were defined, and store it directly into the current file with the sed line.
Maybe a safer way to do this, would be to write a config file somewhere with the data you want to save and source it . data.saved at the begining of your script.
In the script itself? Yes with sed but it's not advisable.
#!/bin/bash
test='0'
echo "test currently is: $test";
test=`expr $test + 1`
echo "changing test to: $test"
sed -i "s/test='[0-9]*'/test='$test'/" $0
Preferable method:
Try saving the value in a seperate file you can easily do a
myvar=`cat varfile.txt`
And whatever was in the file is not in your variable.
I would suggest using the /tmp/ dir to store the file in.
Another option would be to save the value as an extended attribute attached to the script file. This has many of the same problems as editing the script's contents (permissions issues, weird for multiple users, etc) plus a few of its own (not supported on all filesystems...), but IMHO it's not quite as ugly as rewriting the script itself (a config file really is a better option).
I don't use Linux, but I think the relevant commands would be something like this:
path="$(getfattr --only-values -n "user.saved_path" "${BASH_SOURCE[0]}")"
if [[ -z "$path" ]]; then
read -p "Enter path:" path
setfattr -n "user.saved_path" -v "$path" "${BASH_SOURCE[0]}"
fi