I am trying to pipe the output of fswatch to several commands in a shell script with following technique:
$ fswatch -0 [opts] [paths] | xargs -0 -n 1 -I {} [command]
Instead of [command] I put the shell script path. Here is my command line:
fswatch -0 -Ie ".*\.*$" -i ".*\.mp4$" ~/Desktop/encoding\ tests | xargs -0 -n 1 -I {} ~/Desktop/s3cmd.sh
The script is following:
#!/bin/sh
terminal-notifier -message "s3cmd Upload {}" ;
s3cmd sync --acl-public -m video/mp4 --add-header=Cache-Control:public,max-age=2052000 {} s3://saltanat-test/ &&
terminal-notifier -message "s3cmd Upload of {} done"
Sorry I am not experienced with shell scripting.
How can I pipe the the fswatch output into the script?
Thank you.
#!/bin/bash
fswatch -0 -x --event Created --event Updated --event Renamed -Ie '.*\.*$' -i '.*\.mp4$' ~/Desktop/encoding\ tests \
| xargs -0 -n 1 ~/Desktop/s3cmd-3.sh
This works. Thanks.
For anyone in the future with this question :
fswatch command:
fswatch -0 /path/to/watch | xargs -0 -n 1 -I {} ~/yourfile.sh {}
.sh file :
#!/bin/bash
echo $1
Now $1 will be equal to the output of fswatch, which is the full path to the file that was changed.
Related
I am trying to use xargs to run ffmpeg on every file in a folder and output the names of files that have vorbis audio.
I have come up with the following that mostly works, but it fails on files that have single quotes in the filename.
find . -maxdepth 1 | onlyvideos | tr '\n' '\0' | xargs -n1 -0 -I '{}' bash -c "if ( ffmpeg -i '{}' 2>&1 | grep Stream | grep -q vorbis ) ; then echo '{}' ; fi;"
onlyvideos is a script I have written that only prints the names of files with video extensions to stdout.
How can I get xargs to run bash -c on filenames that might include either or both single and double quotes?
All this could be simplified a lot, probably, but if you want to stick with your onlyvideos script you could try:
find . -maxdepth 1 | onlyvideos | tr '\n' '\0' |
xargs -n1 -0 bash -c 'if ( ffmpeg -i "$1" 2>&1 | grep Stream | grep -q vorbis ) ; then echo "$1" ; fi;' bash
Explanation: bash -c 'cmd' arg0 arg1 executes cmd with position parameters set to $0=arg0 (bash in our case) and $1=arg1 (the file name in our case, passed by xargs). So, using "$1" in your command script 'cmd' should work as you expect.
A while style
find . -maxdepth 1 | onlyvideos | while read f; do if ( ffmpeg -i "$f" 2>&1 | grep Stream | grep -q vorbis ) ; then echo "$f" ; fi; done
I have a file named files.txt with all files which I want download.
files.txt
http://file/to/download/IC_0000.tpl
http://file/to/download/IC_0001.tpl
If I use
cat files.txt | egrep -v "(^#.*|^$)" | xargs -n 1 wget
all files are downloaded.
But I dont know how to use If files.txt contains only files without http
files.txt
IC_0000.tpl
IC_0001.tpl
I have "wget" only with this paramter:
Usage: wget [-c|--continue] [-s|--spider] [-q|--quiet] [-O|--output-document FILE]
[--header 'header: value'] [-Y|--proxy on/off] [-P DIR]
[--no-check-certificate] [-U|--user-agent AGENT] [-T SEC] URL...
Can you help me, please.
Many thanks.
Simply try wget -i files.txt (see http://www.gnu.org/software/wget/manual/wget.html#Logging-and-Input-File-Options)
If you don't have the host in the file, try:
for i in `cat files.txt`; do wget "${HOST}/${i}"; done
Just leave it here...
for MacOSX
file_name=newMP3List.txt && cur_path=$(pwd) && split -l 50 $file_name PART && find . -name "PART*" -print0 | xargs -0 -I f osascript -e "tell application \"Terminal\" to do script \"cd $cur_path && cat f | while read CMD; do curl -O \\\"\$CMD\\\"; done; rm f;\""
for linux
cat newMP3List.txt | while read CMD; do curl -O $CMD; done;
need to replace newMP3List.txt with your filename
I want to merge output of three logwatch outputs and pipe result through sendmail.
Example:
#!/bin/sh
LOG_DIR="/var/log/remote-hosts"
MAIL_TO="me#email.com"
sh -c "logwatch && find ${LOG_DIR} -type d -name \"ip*\" -print0 | xargs -0 -I{} sh -c 'logwatch --logdir {} --hostname $(basename {})'" |
sed '1!b;s/^/To: '${MAIL_TO}'\nSubject: Logwatch report\n\n/' | sendmail -t
first logwatch is executed on /var/log folder
and then I would like to traverse /var/log/remote-hosts subfolders (ip-10-0-0-38 and ip-10-0-0-39 ) with find and also do logwatch on them.
The merged output will be sent throught sentmail. However I would like to replace hostname with basename of /var/log/remote-hosts subfolder so instead of /var/log/remote-hosts/ip-10-0-0-38 I will have ip-10-0-0-38 only.
But unfortunatelly I don't how to do the basename part correctly. Any help? Thanks in advance.
Don't use sh -c for grouping statements, use (...):
(logwatch && find ${LOG_DIR} -type d -name "ip*" -print0 | xargs -0 -I{} sh -c 'logwatch --logdir {} --hostname $(basename {})') |
sed '1!b;s/^/To: '${MAIL_TO}'\nSubject: Logwatch report\n\n/' | sendmail -t
This scripts will sort the files by date then move the first 2500 files to another directory.
When I run below scripts, system prompt out Argument list too long msg. Anyone can help me enhance the scripts ? Thanks
NUM_OF_FILES=2500
FROM_DIRECTORY=/apps/data01/RAID/RC/MD/IN_MSC/ERC/in
DESTINATION_DIRECTORY=/apps/data01/RAID/RC/MD/IN_MSC/ERC/in_load
if [ ! -d $DESTINATION_DIRECTORY ]
then
echo "unused_file directory does not exist!"
mkdir $DESTINATION_DIRECTORY
echo "$DESTINATION_DIRECTORY directory created!"
else
echo "$DESTINATION_DIRECTORY exist!"
fi
echo "Moving $NUM_OF_FILES oldest files to $DESTINATION_DIRECTORY directory"
ls -tr $FROM_DIRECTORY/MSCERC*.Z|head -$NUM_OF_FILES |
xargs -i sh -c "mv {} $DESTINATION_DIRECTORY"
You didn't say, but I assume this is where the problem occurs:
ls -tr $FROM_DIRECTORY/MSCERC*.Z|head -2500 | \
xargs -i sh -c "mv {} $DESTINATION_DIRECTORY"
(You can verify it by adding "set -x" to the top of your script.)
The problem is that the kernel has a fixed maximum size of the total length of the command line given to a new process, and your exceeding that in the ls command. You can work around it by not using globbing and instead using grep:
ls -tr $FROM_DIRECTORY/ | grep '/MSCERC\*\.Z$' |head -2500 | \
xargs -i sh -c "mv {} $DESTINATION_DIRECTORY"
(grep uses regular expressions instead of globs, so the pattern looks a little bit different.)
Change
ls -tr $FROM_DIRECTORY/MSCERC*.Z|head -2500 | \
xargs -i sh -c "mv {} $DESTINATION_DIRECTORY"
do something like the following:
find "$FROM_DIRECTORY" -maxdepth 1 -type f -name 'MSCERC*.Z' -printf '%p\t%T#\n' | sort -k2,2 -r | cut -f1 | head -$NUM_OF_FILES | xargs mv -t "$DESTINATION_DIRECTORY"
This uses find to create a list of files with modification timestamps, sorts by the timestamp, then removes the unneeded field before passing the output to head and xargs
EDIT
Another variant, should work with non GNU utils
find "$FROM_DIRECTORY" -type f -name 'MSCERC*.Z' -printf '%p\t%T#' |sort -k 2,2 -r | cut -f1 | head -$NUM_OF_FILES | xargs -i mv \{\} "$DESTINATION_DIRECTORY"
First of create a backup list of the files to be treated. Then read the backup file line-by-line and heal it. For example
#!/bin/bash
NUM_OF_FILES=2500
FROM_DIRECTORY=/apps/data01/RAID/RC/MD/IN_MSC/ERC/in
DESTINATION_DIRECTORY=/apps/data01/RAID/RC/MD/IN_MSC/ERC/in_load
if [ ! -d $DESTINATION_DIRECTORY ]
then
echo "unused_file directory does not exist!"
mkdir $DESTINATION_DIRECTORY
echo "$DESTINATION_DIRECTORY directory created!"
else
echo "$DESTINATION_DIRECTORY exist!"
fi
echo "Moving $NUM_OF_FILES oldest files to $DESTINATION_DIRECTORY directory"
ls -tr $FROM_DIRECTORY/MSCERC*.Z|head -2500 > list
exec 3<list
while read file <&3
do
mv $file $DESTINATION_DIRECTORY
done
A quick way to fix this would be to change to $FROM_DIRECTORY, so that you can refer the files using (shorter) relative paths.
cd $FROM_DIRECTORY &&
ls -tr MSCERC*.Z|head -2500 |xargs -i sh -c "mv {} $DESTINATION_DIRECTORY"
This is also not entirely fool-proof, if you have too many files that match.
I'm trying to rename all files in current directory such that upper case name is converted to lower. I'm trying to do it like this:
ls -1|gawk '{print "`mv "$0" "tolower($0)"`"}'|xargs -i -t eval {}
I have two files in the directory, Y and YY
-t added for debugging, and output is:
eval `mv Y y`
xargs: eval: No such file or directory
if I execute the eval on its own, it works and moves Y to y.
I know there are other ways to achieve this, but I'd like to get this working if I can!
Cheers
eval is a shell builtin command, not a standalone executable. Thus, xargs cannot run it directly. You probably want:
ls -1 | gawk '{print "`mv "$0" "tolower($0)"`"}' | xargs -i -t sh -c "{}"
Although you're looking at an xargs solution, another method to perform the same thing can be done with tr (assuming sh/bash/ksh syntax):
for i in *; do mv $i `echo $i | tr '[A-Z]' '[a-z]'`; done
If your files are created by creative users, you will see files like:
My brother's 12" records
The solutions so far do not work on that kind of files. If you have GNU Parallel installed this will work (even on the files with creative names):
ls | parallel 'mv {} "$(echo {} | tr "[:upper:]" "[:lower:]")"'
Watch the intro video to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ
You can use eval with xargs like the one below.
Note: I only tested this in bash shell
ls -1| gawk '{print "mv "$0" /tmp/"toupper($0)""}'| xargs -I {} sh -c "eval {}"
or
ls -1| gawk '{print "mv "$0" /tmp/"toupper($0)""}'| xargs -I random_var_name sh -c "eval random_var_name"
I generally use this approach when I want to avoid one-liner for loop.
e.g.
for file in $(find /some/path | grep "pattern");do somecmd $file; done
The same can be written like below
find /some/path | grep "pattern"| xargs -I {} sh -c "somecmd {}"