I'm trying to parse vsftpd logs to do some extra processing on the successfully uploaded files.
username will be the user so I create the home dir
filename is the file name in the log: it gives a wonky result i.e. "/foo.txt" but that doesn't matter
#!/bin/sh
sudo tail -F /var/log/vsftpd.log | while read line; do
if sudo echo "$line" | grep -q 'OK UPLOAD:'; then
username=$(echo "$line" | cut -d" " -f8 | sed 's/\[\(.*\)\]/\1/')
filename=$(echo "$line" | cut -d, -f2 | sed 's/^[ \t]*//')
home="/home/vsftpd/$username"
if sudo ls "$home$filename" &> /dev/null; then
# do something with $filename
echo "some text"
fi
fi
done
When a file is uploaded I expect the text "some text". I never get that instead I can see it reports:
ls: cannot access /home/vsftpd/user1"/foo.txt": No such file or directory
Although I can run the command in the shell:
$ sudo ls /home/vsftpd/user1"/foo.txt"
/home/vsftpd/user1/foo.txt
I'm guessing permissions related but I've got it running as sudo and I've given the directories full access. Any ideas?
Your problem is that you have an extra set of quotes around the file name component that you need to strip. The file name in the vsftpd logs (just verified this for myself) is surrounded with quotes, and unlike with username you're not removing those quotes.
This means that $filename ends up being set to, literally, "/foo.txt" including the quotes. When you construct the file name for ls with "$home$filename", the variables are interpolated, but the shell isn't then going to strip off another level of quotes. The quotes stay in the final file name, and the directory /home/vsftpd/user1" with the trailing quote doesn't exist.
This works when you enter the command from the shell because you aren't quoting the file name, so the shell does another round of quote interpolation and removes the double quotes.
If sudo works from the shell, it's possible that sudo has the NOEXEC flag set, which prevents it from executing scripts. You can read more about NOEXEC here.
Related
How do you use a command line argument as a file path and check for file existence in Bash?
I have the simple Bash script test.sh:
#!/bin/bash
set -e
echo "arg1=$1"
if [ ! -f "$1" ]
then
echo "File $1 does not exist."
exit 1
fi
echo "File exists!"
and in the same directory, I have a data folder containing stuff.txt.
If I run ./test.sh data/stuff.txt I see the expected output:
arg1=data/stuff.txt
"File exists!"
However, if I call this script from a second script test2.sh, in the same directory, like:
#!/bin/bash
fn="data/stuff.txt"
./test.sh $fn
I get the mangled output:
arg1=data/stuff.txt
does not exist
Why does the call work when I run it manually from a terminal, but not when I run it through another Bash script, even though both are receiving the same file path? What am I doing wrong?
Edit: The filename does not have spaces. Both scripts are executable. I'm running this on Ubuntu 18.04.
The filename was getting an extra whitespace character added to it as a result of how I was retrieving it in my second script. I didn't note this in my question, but I was retrieving the filename from folder list over SSH, like:
fn=$(ssh -t "cd /project/; ls -t data | head -n1" | head -n1)
Essentially, I wanted to get the filename of the most recent file in a directory on a remote server. Apparently, head includes the trailing newline character. I fixed it by changing it to:
fn=$(ssh -t "cd /project/; ls -t data | head -n1" | head -n1 | tr -d '\n' | tr -d '\r')
Thanks to #bigdataolddriver for hinting at the problem likely being an extra character.
I have a bash script which needs to go through files in a directory in an iOS device and remove files one by one.
To list files from command line I use the following command:
ios-deploy --id UUID --bundle_id BUNDLE -l | grep Documents
and to go one by one on each file I use the following for loop in my script
for line in $(ios-deploy --id UUID --bundle_id BUNDLE -l | grep Documents); do
echo "${line}"
done
Now the problem is that there are files which names have spaces in them, and in such cases the for loop treats them as 2 separate lines.
How can I escape that whitespace in for loop definition so that I get one line per each file?
This might solve your issue:
while IFS= read -r -d $'\n'
do
echo "${REPLY}"
done < <(ios-deploy --id UUID --bundle_id BUNDLE -l | grep Documents)
Edit per Charles Duffy recommendation:
while IFS= read -r line
do
echo "${line}"
done < <(ios-deploy --id UUID --bundle_id BUNDLE -l | grep Documents)
Question
How to expand a shell variable, single quote it, then pass it to sudo?
Attempt
This does not work.
key="some string including special characters"
file='/home/ansible/.ssh/authorized_keys'
sudo -i -u ansible grep -q -E "'"${key}"'" $file
It is not working.
sudo -i -u ansible cat /home/ansible/.ssh/authorized_keys
hoge
h oge
hoge
key='h oge'
sudo -i -u ansible grep -E "'"$key"'" $file
grep: oge': No such file or directory
I never thought I'd say this, but Use Fewer Quotes (trademark pending). This should work fine:
sudo -i -u ansible grep -q -E "$key" "$file"
Basically, you only have to provide syntactic quotes for the literal code (unless you are doing something eval).
If you have characters in key which have a special meaning in a regular expression that is an entirely different problem. In that case use grep -q -F "$key" "$file" to search for a literal string rather than an extended regular expression.
I wrote a basic script which changes the directory to a specific path and shows the list of folders, but my script shows the list of files of the current folder where my script lies instead of which I specify in script.
Here is my script:
#!/bin/bash
v1="$(ls -l | awk '/^-/{ print $NF }' | rev | cut -d "_" -f2 | rev)"
v2=/home/PS212-28695/logs/
cd $v2 && echo $v1
Does any one knows what I am doing wrong?
Your current script makes no sense really. v1 variable is NOT a command to execute as you expect, but due to $() syntax it is in fact output of ls -t at the moment of assignment and that's why you have files from current directory there as this is your working directory at that particular moment. So you should rather be doing ordinary
ls -t /home/PS212-28695/logs/
EDIT
it runs but what if i need to store the ls -t output to variable
Then this is same syntax you already had, but with proper arguments:
v1=$(ls -t /home/PS212-28695/logs/)
echo ${v1}
If for any reason you want to cd then you have to do that prior setting v1 for the same reason I explained above.
I am deeply confused by Bash's Heredoc construct behaviour.
Here is what I am doing:
#!/bin/bash
user="some_user"
server="some_server"
address="$user"#"$server"
printf -v user_q '%q' "$user"
function run {
ssh "$address" /bin/bash "$#"
}
run << SSHCONNECTION1
sudo dpkg-query -W -f='${Status}' nano 2>/dev/null | grep -c "ok installed" > /home/$user_q/check.txt
softwareInstalled=$(cat /home/$user_q/check.txt)
SSHCONNECTION1
What I get is
cat: /home/some_user/check.txt: No such file or directory
This is very bizarre, because the file exists if I was to connect using SSH and check the following path.
What am I doing wrong? File is not executable, just a text file.
Thank you.
If you want the cat to run remotely, rather than locally during the heredoc's evaluation, escape the $ in the $(...):
softwareInstalled=\$(cat /home/$user_q/check.txt)
Of course, this only has meaning if some other part of your remote script then refers to "$softwareInstalled" (or, since it's in an unquoted heredoc, "\$softwareInstalled").