I tried with below script. But, not working for cut the first field of each line and to be executed for "chmod".
#!/bin/bash
if [ -z "$1" ]; then
echo -e "Usage: $(basename $0) FILE\n"
exit 1
fi
if [ ! -e "$1" ]; then
echo -e "$1: File doesn't exist.\n"
exit 1
fi
while read -r line; do
awk '{print $1}'
[ -n "$line" ] && chown root "$line" && echo -e "$line Ownership changed"
done < "$1"
If field separator is space, try this:
while read -r line; do
FILE_TO_CHANGE=$(echo $line | awk '{print $1}')
[ -n "$line" ] && chown root "$FILE_TO_CHANGE" && echo -e "$line Ownership changed"
done < "$1"
awk read $line and print first token on standard output, the result is saved in FILE_TO_CHANGE variable and then it is used to run chown.
Another way could be:
awk '{print $1}' $1 | while read line; do
chown root "$line" && echo -e "$line Ownership changed"
done
awk read your file and print the first field of each line, in this case, while loop read awk output line by line and run chown on field.
You could extract the first word on each line with awk and pipe to xargs, invoking chown only as few times as possible:
awk '{print $1}' "$1" | xargs chown root
Related
I have a Property file in Jenkins lets call it Something.txt
and Something.txt contains
A_B_C
D_E_F
and i have used below shell to read the file and Execute my Automation
file="/var/lib/jenkins/components.txt"
if [ -f "$file" ]
then
echo "$file found."
Websites="$(awk -F '_' '{print $1}' $file | paste -d, -s)"
Profiles="$(awk -F '_' '{print $2}' $file | paste -d, -s)"
Component="$(awk -F '_' '{print $3}' $file | paste -d, -s)"
for i in $(echo $Websites | sed "s/,/ /g"); do
for j in $(echo $Profiles | sed "s/,/ /g"); do
for k in $(echo $Component| sed "s/,/ /g"); do
mvn clean verify -D "cucumber.options=--tags #"${j} -D surefire.suiteXmlFiles=./XMLScripts/${i}.${k}.testng.xml ||true
done
done
done
but what is happening is My Job is running as
A-B-C & A-B-F & D-B-C & B-E-F
but the expected result is A-B-C & D-E-F how to achieve this?
Don't read lines with for
#!/usr/bin/env bash
file="/var/lib/jenkins/components.txt"
if [[ -f "$file" ]]; then
while IFS=_ read -r website profile component; do
printf '%s %s %s\n' "$website" "$profile" "$component"
done < "$file"
fi
In your case you can do
#!/usr/bin/env bash
file="/var/lib/jenkins/components.txt"
if [[ -f "$file" ]]; then
while IFS=_ read -r website profile component; do
echo mvn clean verify -D cucumber.options=--tags #"$website" -D "surefire.suiteXmlFiles=./XMLScripts/$profile.$component.testng.xml" ||true
done < "$file"
fi
Remove the echo if you're satisfied with the result.
i have a script which copy files from one s3 bucket to local server, do some stuff and upload it to another s3 bucket.
in the original bucket i have few folders, one of them called "OTHER"
i dot want my script to work on this folder
i tried to define a loop to check if the path string does not contains the string "OTHER" only then to continue to other commands but for some reason it is not working.
what am i doing wrong ?
#!/bin/bash
shopt -s extglob
gcs3='s3://gc-reporting-pud-production/splunk_printer_log_files/'
gcs3ls=$((aws s3 ls 's3://gc-reporting-pud-production/splunk_printer_log_files/' --recursive) | sed 's/^.*\(splunk_printer.*\)/\1/g'| tr -s ' ' | tr ' ' '_')
ssyss3=s3://ssyssplunk
tokenFile=/splunkData/GCLogs/tokenFile.txt
nextToken=$((aws s3api list-objects-v2 --bucket "gc-reporting-pud-production" --prefix splunk_printer_log_files/ --max-items 5) |grep -o 'NEXTTOKEN.*' |awk -F " " '{print $2}')
newToken=$( tail -n 1 /splunkData/GCLogs/tokenFile.txt )
waterMark=$(aws s3api list-objects-v2 --bucket "gc-reporting-pud-production" --prefix splunk_printer_log_files/ --max-items 5 --starting-token
$newToken|sed 's/^.*\(splunk_printer.*zip\).*$/\1/'|sed '1d'|sed '$d')
while true; do
for j in $waterMark ; do
echo $j
if [ "$j" != *"OTHER"* ]; then
gcRegion=$(echo $j | awk -F'/' '{print $2}')
echo "gcRegion:"$gcRegion
if [ "$gcRegion" != "OTHER" ]; then
gcTech=$(echo $j | awk -F'/' '{print $3}')
echo "GCTech:"$gcTech
gcPrinterFamily=$(echo $j | awk -F'/' '{print $4}')
echo "gcPrinterFamily:" $gcPrinterFamily
gcPrinterType=$(echo $j | awk -F'/' '{print $5}')
echo "gcPrinterType:" $gcPrinterType
gcPrinterName=$(echo $j| awk -F'/' '{print $6}')
echo "gcPrinterName:" $gcPrinterName
gcFileName=$(echo $j| awk -F'/' '{print $7}'| awk -F'.zip' '{print $1}')
echo "gcFileName:" $gcFileName
cd /splunkData/GCLogs
dir="/splunkData/GCLogs/$gcRegion/$gcTech/$gcPrinterFamily/$gcPrinterType/$gcPrinterName"
echo "dir:"$dir
mkdir -p $dir
aws s3 sync $gcs3$gcRegion/$gcTech/$gcPrinterFamily/$gcPrinterType/$gcPrinterName/ $dir
find $dir -name '*.zip' -exec sh -c 'unzip -o -d "${0%.*}" "$0"' '{}' ';'
aws s3 cp $dir $ssyss3/$gcRegion/$gcTech/$gcPrinterFamily/$gcPrinterType/$gcPrinterName/ --recursive --exclude "*.zip"
newToken=$( tail -n 1 /splunkData/GCLogs/tokenFile.txt )
nextToken=$(aws s3api list-objects-v2 --bucket "gc-reporting-pud-production" --prefix splunk_printer_log_files/ --max-items 5 --starting-token $newToken |grep -o 'NEXTTOKEN.*' |awk -F " " '{print $2}')
waterMark=$(aws s3api list-objects-v2 --bucket "gc-reporting-pud-production" --prefix splunk_printer_log_files/ --max-items 5 --starting-token $newToken|sed 's/^.*\(splunk_printer.*zip\).*$/\1/'|sed '1d'|sed '$d')
echo "$nextToken" > "$tokenFile"
fi
fi
done
done
You need to use the double-bracket conditional command to turn == and != into pattern matching operators:
if [[ "$j" != *"OTHER"* ]]; then
# ^^ ^^
Or use case
case "$j" in
*OTHER*) ... ;;
*) echo "this is like an `else` block" ;;
esac
Paste your code into https://www.shellcheck.net/ for other things to fix.
I think glenn jackman was on the right path. Try this:
if [[ "$j" != *OTHER* ]]; then
The [[ ]] is required for pattern string matching (and you have to remove the " ). The case statement is also a good idea. You can abandon the shell test altogether and use grep as follows:
if
grep -q '.*OTHER.*' <<< "$j" 2>/dev/null
then
...
fi
Here's a check of the [[ ]]:
$ echo $j
abOTHERc
$ [[ "$j" == *OTHER* ]]
$ echo $?
0
As per BenjaminW., the quotes around $j in [[ ]] are unnecessary. However, the quotes around *OTHER* do make a big difference. See below:
$ j="OTHER THINGS"
$ [[ $j == "*OTHER*" ]] ; echo "$j" matches '"*OTHER*"': $?
OTHER THINGS matches "*OTHER*": 1
$ [[ $j == *OTHER* ]] ; echo "$j" matches '*OTHER*': $?
OTHER THINGS matches *OTHER*: 0
I have a file with filename and path to the file
I want to delete the the rows which have files that do not exist anymore
file.txt (For now all existing files):
file1;~/Documents/test/123
file2;~/Documents/test/456
file3;~/Test
file4;~/Files/678
Now if I delete any of the given files(file 2 AND file4 fore example) and run my script I want it to test if the file in the given row exists and remove the row if it does not
file.txt(after removing file2, file4):
file1;~/Documents/test/123
file3;~/Test
What I got so far(Not working at all):
-Does not want to run at all
#!/bin/sh
backup=`cat file.txt`
rm -f file.txt
touch file.txt
while read -r line
do
dir=`echo "$line" | awk -F';' '{print $2}'`
file=`echo "$line" | awk -F';' '{print $1}'`
if [ -f "$dir"/"$file" ];then
echo "$line" >> file.txt
fi
done << "$backup"
Here's one way:
tmp=$(mktemp)
while IFS=';' read -r file rest; do
[ -f "$file" ] && printf '%s;%s\n' "$file" "$rest"
done < file.txt > "$tmp" && mv "$tmp" file.txt
or if you don't want a temp file for some reason:
tmp=()
while IFS=';' read -r file rest; do
[ -f "$file" ] && tmp+=( "$file;$rest" )
done < file.txt &&
printf '%s\n' "${tmp[#]}" > file.txt
Both are untested but should be very close if not exactly correct.
If I understand, this should do it.
touch file.txt file2.txt
for i in `cat file.txt`; do
fp=`echo $i|cut -d ';' -f2`
if [ -e $fp ];then
echo "$i" >> file2.txt
fi
done
mv file2.txt file.txt
As part of some batch "bash" program, how can I automatically remove inactive ossec agents in cases of autoscaling groups where instances are created/deleted constantly?
Here is a quick script you can run to remove 'Disconnected' and 'Never connected' agents
for OUTPUT in $(/var/ossec/bin/agent_control -l | grep -E 'Disconnected|Never' | tr ':' ',' | cut -d "," -f 2 )
do
/var/ossec/bin/manage_agents -r $OUTPUT
done
#This is to be run on ossec server, path for ossec is /var/ossec/
file=agents.txt
/var/ossec/bin/agent_control -l > $file
#Wipe working tmp files
rm remove.txt
rm removed.txt
echo -n "" > remove.txt
echo -n "" > removed.txt
#Find Disconnected agents
while IFS= read -r line
do
ids=$(echo $line | awk '{print $2}')
status=$(echo $line | awk '{print $NF}')
if [ "$status" == "Disconnected" ]; then
echo $ids >> remove.txt
fi
done < "$file"
#Find Never connected agents
while IFS= read -r line
do
ids=$(echo $line | awk '{print $2}')
status=$(echo $line | awk '{ if (NF > 1) print $(NF-1),$NF ; else print $NF; }')
if [ "$status" == "Never connected" ]; then
echo $ids >> remove.txt
fi
done < "$file"
#Remove commas
sed 's/.$//' remove.txt > removed.txt
#Remove agents with IDs in removed.txt file
file2=removed.txt
while IFS= read -r line
do
/var/ossec/bin/manage_agents -r "$line"
done < $file2
#Restart OSSEC service
/var/ossec/bin/ossec-control restart
#End
I'm trying to put together a bash script that will search a bunch of files and if it finds a particular string in a file, it will add a new line on the line after that string and then move on to the next file.
#! /bin/bash
echo "Creating variables"
SEARCHDIR=testfile
LINENUM=1
find $SEARCHDIR* -type f -name *.xml | while read i; do
echo "Checking $i"
ISBE=`cat $i | grep STRING_TO_SEARCH_FOR`
if [[ $ISBE =~ "STRING_TO_SEARCH_FOR" ]] ; then
echo "found $i"
cat $i | while read LINE; do
((LINENUM=LINENUM+1))
if [[ $LINE == "<STRING_TO_SEARCH_FOR>" ]] ; then
echo "editing $i"
awk -v "n=$LINENUM" -v "s=new line to insert" '(NR==n) { print s } 1' $i
fi
done
fi
LINENUM=1
done
the bit I'm having trouble with is
awk -v "n=$LINENUM" -v "s=new line to insert" '(NR==n) { print s } 1' $i
if I just use $i at the end, it will output the content to the screen, if I use $i > $i then it will just erase the file and if I use $i >> $i it will get stuck in a loop until the disk fills up.
any suggestions?
Unfortunately awk dosen't have an in-place replacement option, similar to sed's -i, so you can create a temp file and then remove it:
awk '{commands}' file > tmpfile && mv tmpfile file
or if you have GNU awk 4.1.0 or newer, the -i inplace is added, so you can do:
awk -i inplace '{commands}' file
to modify the original
#cat $i | while read LINE; do
# ((LINENUM=LINENUM+1))
# if [[ $LINE == "<STRING_TO_SEARCH_FOR>" ]] ; then
# echo "editing $i"
# awk -v "n=$LINENUM" -v "s=new line to insert" '(NR==n) { print s } 1' $i
# fi
# done
# replaced by
sed -i 's/STRING_TO_SEARCH_FOR/&\n/g' ${i}
or use awk in place of sed
also
# ISBE=`cat $i | grep STRING_TO_SEARCH_FOR`
# if [[ $ISBE =~ "STRING_TO_SEARCH_FOR" ]] ; then
#by
if [ $( grep -c 'STRING_TO_SEARCH_FOR' ${i} ) -gt 0 ]; then
# if file are huge, if not directly used sed on it, it will be faster (but no echo about finding the file)
If you can, maybe use a temporary file?
~$ awk ... $i > tmpfile
~$ mv tmpfile $i
Or simply awk ... $i > tmpfile && mv tmpfile $i
Note that, you can use mktemp to create this temporary file.
Otherwise, with sed you can insert a line right after a match:
~$ cat f
auie
nrst
abcd
efgh
1234
~$ sed '/abcd/{a\
new_line
}' f
auie
nrst
abcd
new_line
efgh
1234
The command search if the line matches /abcd/, if so, it will append (a\) the line new_line.
And since sed as the -i to replace inline, you can do:
if [[ $ISBE =~ "STRING_TO_SEARCH_FOR" ]] ; then
echo "found $i"
echo "editing $i"
sed -i "/STRING_TO_SEARCH_FOR/{a
\new line to insert
}" $i
fi