I'm having trouble in a script and i want to know if it's possible to store the path of the matching result of a grep ?
I'm on RHEL 7, the script is a check of the rsyslog.conf file which complete or add the correct value to a parameter (CIS rhel7 benchmark, part 4.2.1.3).
Full script so far :
#!/bin/bash
if grep "^\$FileCreateMode" /etc/rsyslog.conf /etc/rsyslog.d/*.conf
then
read -p "Is $FileCreateMode superior or equal to 0640 ? [y/n]" rep
if [ $rep == "y" ]
then
echo "No action needed"
else
read -p "Enter the new $FileCreateMode value (0640 recommanded)" rep2
sed -i "/^\$FileCreateMode/ $rep2"
echo "$FileCreateMode new value is now $rep2"
fi
else
echo "$FileCreateMode doesn't exist in rsyslog conf files"
read -p "What's the path of the file to modify ?(Press [ENTER] for default /etc/rsyslog.conf)" path
if [ $path -z ]
then
echo "$FileCreateMode 0640" >> /etc/rsyslog.conf
else
echo "$FileCreateMode 0640" >> $path
fi
fi
So my problem is on the sed at the 11th line.
Am i able to get the right path if my grep on 3rd line matched into a variable to reuse it on the 11th.
And i'm struggling with the same sed because i want him to replace the value after $FileCreateMode but it keep changing the $FileCreateMode string.
i've tried this syntax too but i still don't get the result i want
sed -i -e "s,^\($FileCreateMode[ ]*\).*,\1 0640 ,g" /etc/rsyslog.conf
Thanks in advance for any help you can bring, and have a good day :)
Edit :
As requested i'm simplifying here.
I want to grep $FileCreateMode in /etc/rsyslog.conf and /etc/rsyslog.d/*.conf and i'm trying to get the destination file (could be rsyslog.conf but it can be testpotato.conf in rsyslog.d) into a variable (like $var) to be able to use the path in my sed on the 11th line like
sed -i "/^\$FileCreateMode/ 0640" $var
And for the sed problem when i execute this command i would like to have something like
old : $FileCreateMode 0777
sed -i "/^\$FileCreateMode/ 0640" $var
new : $FileCreateMode 0640
But instead i get
old : $FileCreateMode 0777
sed -i "/^\$FileCreateMode/ 0640" $var
new : 0640 ($FileCreateMode is deleted)
hope i'm more understable, thanks again and feel free to ask for more details
Use $() to assign the results of grep into a variable, and then use a for loop to process files one by one:
# Assign grep results to FILES
FILES=$(grep -l '^$FileCreateMode' /etc/rsyslog.conf /etc/rsyslog.d/*.conf)
# Check if FILES variable is not empty
if [[ -n ${FILES} ]]; then
# Loop through all the files
for file in ${FILES}; do
# ...
sed -iE "s/^(\\\$FileCreateMode\s+)[[:digit:]]+/\1${rep2}/" ${file}
# ...
done
else
# OP's logic for when $FileCreateMode doesn't exist in any of the files
sed fix:
Notice that I've also updated your sed expression (above). You were very close, but you had to double escape the dollar sign: once for using it inside "", and once so that it isn't interpreted as END_OF_LINE in the regex.
If your grep supports -H, you could just do:
while grep -H "^\$FileCreateMode" /etc/rsyslog.conf /etc/rsyslog.d/*.conf \
| IFS=: read path line; do
# Here, $path is the path to the file that matches
# and $line is the line that matched.
done
If you prefer, you can use if ...; then instead of while ...; do Be aware that the variables will lose their value after the subshell terminates.
Related
I'm starting to learn how to write shell scripts. I have them all placed in a folder 'personal-scripts' in my home directory. They are starting to add up though. To solve this, I am attempting to create a script that loops over the directory and gives me a brief sentence about what each script does.
For now I can only output the script location and names via:
scriptsinfo
#!/bin/bash
for filename in ~/personal-scripts/*
do
echo $filename
done
Since I don't want to go back and update this file manually, I want to place the about sentence inside each other script either as a comment or string variable that can be evaluated.
How can I read the contents of each other script in the folder and output their string/info about what they do in this script?
You can do that using head command, which prints the first n lines of a file.
test.sh
# this is about line
# date is 14-9-2017
script data
..
..
~# head -n 2 test.sh
# this is about line
# date is 14-9-2017
If you add the description on each second line of your script, (after #!/bin/bash then let use sed -n "2p" $filenamein your script. You can also add a separator between each script:
#!/bin/bash
for filename in ~/personal-scripts/*
do
echo "---------------------"
echo $filename
echo "---------------------"
sed -n "2p" $filename
done
The alternative is to put the description anywhere, in a line starting by e.g # about: then you can grep it:
#!/bin/bash
for filename in ~/personal-scripts/*
do
echo "---------------------"
echo $filename
echo "---------------------"
grep "# about:" $filename | sed 's/# about://'
done
The | sed 's/# about://' is there to keep the description only.
If you have a comment inside all your scripts with a specific pattern such as
#info : This script runs daily
Then you can simply grep for the pattern and append to the name of the script in each line.
for filename in ~/personal-scripts/*
do
echo "$i : $(grep '#info' $i)"
done
During the process of setting up my Raspberries, I want to prevent root logins via ssh. As always, it's a "scriptlet" (called by a runner).
My research told me that, even in times of systemd, /etc/ssh/sshd_config is the file to modify. So far, so good. In my humble understanding, what needs to be done is this: read the config file line by line, match for "PermitRootLogin yes" (whitespace match); if no match, write the line to another file, if yes, replace it with "PermitRootLogin no", and write to the other file, and finally replace the original configuration file with the new file, and restart sshd via the systemd stuff.
In Perl, I'd read the whole file, replace() the line, and write stuff back to another file. But as the young Buddhist said: "There is no Perl!"
Bash(2) only, please.
Script:
$ sed -i 's/PermitRootLogin yes/PermitRootLogin no/g' /etc/ssh/sshd_config
Then:
$ service ssh reload
You can do something like that
#!/bin/bash
if [[ "${UID}" -ne 0 ]]; then
echo " You need to run this script as root"
exit 1
fi
# To directly modify sshd_config.
sed -i 's/#\?\(Port\s*\).*$/\1 2231/' /etc/ssh/sshd_config
sed -i 's/#\?\(PermitRootLogin\s*\).*$/\1 no/' /etc/ssh/sshd_config
sed -i 's/#\?\(PubkeyAuthentication\s*\).*$/\1 yes/' /etc/ssh/sshd_config
sed -i 's/#\?\(PermitEmptyPasswords\s*\).*$/\1 no/' /etc/ssh/sshd_config
sed -i 's/#\?\(PasswordAuthentication\s*\).*$/\1 no/' /etc/ssh/sshd_config
# Check the exit status of the last command
if [[ "${?}" -ne 0 ]]; then
echo "The sshd_config file was not modified successfully"
exit 1
fi
/etc/init.d/ssh restart
exit 0
Configuring SSH programmatically
I did some investigation on this and I think JNevevill's answer is the most solid.
I have some example script that illustrated the two ways, SED and AWK.
Preparation
Showing the test data only once for better readability in the examples. It has been initialized for each approach the same way. It will define some rules in a test file and a dictionary of new rules that should replace the rules in the file or be written to the file.
echo 'LoginGraceTime 120' > ./data.txt
echo '#PermitRootLogin yes' >> ./data.txt
echo 'PermitRootLogin no' >> ./data.txt
echo 'PasswordAuthentication yes' >> ./data.txt
declare -A rules=(
["LoginGraceTime"]="1m"
["PermitRootLogin"]="no"
["PasswordAuthentication"]="no"
["AllowUsers"]="blue"
)
SED
SED will replace the rules it finds. If a line is commented, it will remove the #
and replace the value. This approach is less solid as it can lead to duplicate rules. I.E. A rule exists commented and uncommented. Also, if the rule wasn't there, it will not be written at all.
for rule in "${!rules[#]}"; do
regex="s/#\?\(${rule}\s*\).*$/\1 ${rules[${rule}]}/"
sed "${regex}" ./data.txt > temp.txt;
mv -f temp.txt ./data.txt
done
Result:
LoginGraceTime 1m
PermitRootLogin no
PermitRootLogin no
PasswordAuthentication no
AWK
AWK is more solid in this situation. It yields better control.
It will read line by line and replace the value if it exists without changing commented rules. If a rule wasn't found at the end of the file, it will append this rule to the file. This is much more solid than the SED approach. We can be sure there will be not duplicate rule and all rules are defined.
for rule in "${!rules[#]}"; do
awk -v key="${rule}" -v val="${rules[${rule}]}" \
'$1==key {foundLine=1; print key " " val} $1!=key{print $0} END{if(foundLine!=1) print key " " val}' \
./data.txt > sshd_config.tmp && mv sshd_config.tmp ./data.txt
done
Result:
LoginGraceTime 1m
#PermitRootLogin yes
PermitRootLogin no
PasswordAuthentication no
AllowUsers blue
Conclusion
AWK is clearly the better choice. It is more safe and can handle rules that are not in the file.
You could use awk for this:
awk '$1=="PermitRootLogin"{foundLine=1; print "PermitRootLogin no"} $1!="PermitRootLogin"{print $0} END{if(foundLine!=1) print "PermitRootLogin no"}' sshd_config > sshd_config.tmp && mv sshd_config.tmp sshd_config
That goes through each line in the file, if the first element (seperated by awk's default delims) is "PermitRootLogin" then change it to "no" and capture that you found it. If the line doesn't contain "PermitRootLogin" then just write it out as-is. If you didn't find "PermitRootLogin" after all the lines are processed (END) then write that line.
#Remove cause had to edit 6 chars....
If you have GNU sed 4.2.2 you can use the following trick:
sed -z 's/PermitRootLogin yes\|$/PermitRootLogin no/' file
-z will read lines delimited by NUL (\0) so basically enabling slurp mode in sed
-i will do in place edit on your file, so remember to make a backup before running with it
The substitution is pretty straight forward:
Replaces PermitRootLogin yes with PermitRootLogin no and if not found append PermitRootLogin no to the end.
You could add word boundaries around the search: \<PermitRootLogin yes\>
Please note that this is omit a trailing newline if PermitRootLogin yes is not matched, since PermitRootLogin no will be inserted after the last newline.
The accepted sed answer is good for most situations, and the one you should probably use if you're concerned with other people modifying it. Using sed -i does have mild portability issues however, and changing more than one configuration value requires writing the file multiple times. For an elegant (if mildly esoteric) solution, you can use ed:
ed /etc/ssh/sshd_config << EOF
%s/^PermitRootLogin.*/PermitRootLogin no
%s/^ChallengeResponseAuthentication.*/ChallengeResponseAuthentication no
wq
EOF
Explanation of each line:
Run ed on the file and provide a set of commands to standard input with a heredoc
Run a global substitute (%s), on the file, replacing lines that match the ^PermitRootLogin.* regex with PermitRootLogin no
Same as 2, but with ChallengeResponseAuthentication
Write the file and quit
You'll probably raise a few eyebrows using ed but it's portable, easy to extend, and only writes to disk once.
#cyrus:
My current script looks like this:
#!/bin/bash
touch /tmp/sshd_config
while read -r line || [[ -n "$line" ]]; do
if [ "$line" = "PermitRootLogin yes" ]; then
match=1
echo "PermitRootLogin no" >> /tmp/sshd_config
else
echo "$line" >> /tmp/sshd_config
fi
done < /etc/ssh/sshd_config
if [ "$match" != "1" ]; then
echo "PermitRootLogin no" >> /tmp/sshd_config
fi
It works, but it looks poor. I'd prefer a more \s+ 'ish style to catch "PermitRootLogin(manySpaces)yes" and "PermitRootLogin(tabTabTab)yes" lines. An approach using 'grep' would definitely be nicer.
To anybody else who has answered so far and mentioned sed and awk: There are three caveats in your proposals.
1: I, as I am not the maintainer of the distro, cannot guarantee that one or both will be installed.
2: I don't know if people who want to modify my scripts are sed and/or awk cracks.
3: I can guarantee that, except of the fact that sed and awk are magnificent, awesome, elegant tools to deal with such stuff, I have absolutely no knowledge when it comes to sed and awk. And yes, this is a humiliating gap.
=========
Post scriptum...
After playing around a bit and finding one ugly caveat in the original script, here is my current version:
#!/bin/bash
INFILE=/etc/ssh/sshd_config
TMPFILE=/var/tmp/sshd_config
touch $TMPFILE
while read -r line || [[ -n "$line" ]]; do
if [ `echo "$line" | grep -c -P "^\s*PermitRootLogin\s+"` = "1" ]; then
match=1
echo "PermitRootLogin no" >> $TMPFILE
else
echo "$line" >> $TMPFILE
fi
done < $INFILE
if [ "$match" != "1" ]; then
echo "" >> $TMPFILE
echo "# Do not permit root to log in directly" >> $TMPFILE
echo "PermitRootLogin no" >> $TMPFILE
fi
cp -f $TMPFILE $INFILE
sync
The difference to the old version is, at first sight, the change from the simple comparison to grep, but the pcre is indeed neccessary. If a future distro comes with "PermitRootLogin no", the former version will add an at least unneccessary entry to the config. Another nifty thing is that the config file, at another line, contains "PermitRootLogin yes" within a comment. A simple grep -c -P "PermitRootLogin\s+yes" would match there (again).
The new version still looks clumsy and ugly, but it works :)
This can be further simplified as a few liner script where you pass variables to change and it can change on the fly for you. Check all sorts of syntax and make sure end results are all perfectly done.
yes and no can be converted into a switch as well
To User Scripts Do
replace_string.sh sshd_config.old PasswordAuthentication PermitRootLogin
My Script Looks Like this
#! /bin/bash
sshd_config_file=$1
search_replace()
{
grep -i "^${search_string} yes$" ${sshd_config_file} | grep -v "^#"|grep -v "${search_string} no" || sed -i "" "s|^#*${search_string}.*$|${search_string} yes|" ${sshd_config_file}
}
#for search_string in $#; do
for search_string in $(eval echo ${*:2}); do
search_replace
done
I have written a script to change file ownerships based on an input list read in. My script works fine on directories without space in their name. However it fails to change files on directories with space in their name. I also would like to capture the output from the chown command to a file. Could anyone help ?
here is my script in ksh:
#!/usr/bin/ksh
newowner=eg27395
dirname=/home/sas/sastest/
logfile=chowner.log
date > $dir$logfile
command="chown $newowner:$newowner"
for fname in list
do
in="$dirname/$fname"
if [[ -e $in ]]
then
while read line
do
tmp=$(print "$line"|awk '{if (substr($2,1,1) == "/" ) print $2; if (substr($0,1,1) == "/" ) print '})
if [[ -e $tmp ]]
then
eval $command \"$tmp\"
fi
done < $in
else
echo "input file $fname is not present. Check file location in the script."
fi
done
a couple of other errors:
date > $dir$logfile -- no $dir variable defined
to safely read from a file: while IFS= read -r line
But to answer your main concern, don't try to build up the command so dynamically: don't bother with the $command variable, don't use eval, and quote the variable.
chmod "$newowner:$newowner" "$tmp"
The eval is stripping the quotes on this line
command="chown $newowner:$newowner"
In order to get the line to work with spaces you will need to provide backslashed quotes
command="chown \"$newowner:$newowner\""
This way the command that eval actually runs is
chown "$newowner:$newowner"
Also, you probably need quotes around this variable setting, although you'll need to tweak the syntax
tmp="$(print "$line"|awk '{if (substr($2,1,1) == "/" ) print $2; if (substr($0,1,1) == "/" ) print '})"
To capture the output you can add 2>&1 > file.out where file.out is the name of the file ... in order to get it working with eval as you are using it you will need to backslash any special characters much in the same way you need to backslash the double quotes
Your example code suggests that list is a "meta" file: A list of files that each has a list of files to be changed. When you only have one file you can remove the while loop.
When list is a variable with filenames you need echo "${list}"| while ....
It is not completely clear why you sometimes want to start with the third field. It seems that sometimes you have 2 words before the filename and want them to be ignored. Cutting the string on spaces becomes a problem when your filenames have spaces as well. The solution is look for a space followed by a slash: that space is not part of a filename and everything up to that space can be deleted.
newowner=eg27395
# The slash on the end is not really part of the dir name, doesn't matter for most commands
dirname=/home/sas/sastest
logfile=chowner.log
# Add braces, quotes and change dir into dirname
date > "${dirname}/${logfile}"
# Line with command not needed
# Is list an inputfile? It is streamed using "< list" at the end of while []; do .. done
while IFS= read -r fname; do
in="${dirname}/${fname}"
# Quotes are important
if [[ -e "$in" ]]; then
# get the filenames with a sed construction, and give it to chmod with xargs
# The sed construction is made for the situation that in a line with a space followed by a slash
# the filename starts with the slash
# sed is with # to avoid escaping the slashes
# Do not redirect the output here but after the loop.
sed 's#.* /#/#' "${in}" | xargs chmod ${newowner}:${newowner}
else
echo "input file ${fname} is not present. Check file location in the script."
fi
done < list >> "${dirname}/${logfile}"
I'm trying to write a small script that either takes input from a file or from user, then it gets rid of any blank lines from it.
I'm trying to make it so that if there is no file name specified it will prompt the user for input. Also is the best way to output the manual input to a file then run the code or to store it in a variable?
So far I have this but when I run it with a file it give 1 line of error before returning the output I want. The error says ./deblank: line 1: [blank_lines.txt: command not found
if [$# -eq "$NO_ARGS"]; then
cat > temporary.txt; sed '/^$/d' <temporary.txt
else
sed '/^$/d' <$#
fi
Where am I going wrong?
You need spaces around [ and ]. In bash, [ is a command and you need spaces around it for bash to interpret it so.
You can also check for the presence of arguments by using (( ... )). So your script could be rewritten as:
if ((!$#)); then
cat > temporary.txt; sed '/^$/d' <temporary.txt
else
sed '/^$/d' "$#"
fi
If you want to use only the first argument, then you need to say $1 (and not $#).
Try using this
if [ $# -eq 0 ]; then
cat > temporary.txt; sed '/^$/d' <temporary.txt
else
cat $# | sed '/^$/d'
fi
A space is needed between [ and $# and your usage of $# is not good. $# represents all arguments and -eq is used to compare numeric values.
There are multiple problems here:
You need to leave a space between the square brackets [ ] and the variables.
When using a string type, you cannot use -eq, use == instead.
When using a string comparison you need to use double square brackets.
So the code should look like:
if [[ "$#" == "$NO_ARGS" ]]; then
cat > temporary.txt; sed '/^$/d' <temporary.txt
else
sed '/^$/d' <$#
fi
Or else use $# instead.
Instead of forcing user input to a file, I'd force the given file to stdin:
#!/bin/bash
if [[ $1 && -r $1 ]]; then
# it's a file
exec 0<"$1"
elif ! tty -s; then
: # input is piped from stdin
else
# get input from user
echo "No file specified, please enter your input, ctrl-D to end"
fi
# now, let sed read from stdin
sed '/^$/d'
The output is blank fr the below script. What is it missing? I am trying to grep a string
#!/bin/ksh
file=$abc_def_APP_13.4.5.2
if grep -q abc_def_APP $file; then
echo "File Found"
else
echo "File not Found"
fi
In bash, use the <<< redirection from a string (a 'Here string'):
if grep -q abc_def_APP <<< $file
In other shells, you may need to use:
if echo $file | grep -q abc_def_APP
I put my then on the next line; if you want your then on the same line, then add ; then after what I wrote.
Note that this assignment:
file=$abc_def_APP_13.4.5.2
is pretty odd; it takes the value of an environment variable ${abc_def_APP_13} and adds .4.5.2 to the end (it must be an env var since we can see the start of the script). You probably intended to write:
file=abc_def_APP_13.4.5.2
In general, you should enclose references to variables holding file names in double quotes to avoid problems with spaces etc in the file names. It is not critical here, but good practices are good practices:
if grep -q abc_def_APP <<< "$file"
if echo "$file" | grep -q abc_def_APP
Yuck! Use the shell's string matching
if [[ "$file" == *abc_def_APP* ]]; then ...