Let's say I want to write a script that does ls with a prefix for each filename. I tried this
ls | xargs -n 1 echo prefix_
But the result was
prefix_ first_file
prefix_ second_file
...
How can I remove the space between the prefix and the filename? I.e. how to I make xargs put the variable after the command, without space? (Or in any other place for that matter)
The solution: -I
-I lets you name your argument and put it anywhere you like. E.g.
ls | xargs -n 1 -I {} echo prefix_{}
(replace {} with any string)
Related
My requirement is
localfolder=green.txt,yellow.txt,blue.txt
remotefolder=green_202105050333.txt,yellow_202105050333.txt,blue_202105050333.txt
I want to compare both folders as
IF[[localfolder==remotefolder]] ex: green.txt= green_202105050333.txt(here condition will look always first characters of each file or eliminating the data&time)
then
display the results as "matched"
kindly help me to get the logic here please.
Thanks in advance!!
You can do the next, create a file for each directory(folder), for example if you use ls :
# in local directory
ls localfolder > localFolder.txt
# in some remote directory
ls remoteFolder > remoteFolder.txt
We can suppose that files are the next:
localFolder.txt:
green.txt
yellow.txt
blue.txt
remoteFolder.txt:
green_202105050333.txt
yellow_202105050333.txt
blue_202105050333.txt
gray_202105050333.txt
Now put the files in the same directory and execute the next:
sed -r 's/\.[a-z]+//g' localFolder.txt | xargs -I{} grep {} remoteFolder.txt | sed -r 's/_[0-9]+//g'
The output will be:
green.txt
yellow.txt
blue.txt
Now if you want do it in script, then create a file for example myScript.sh and the content must be:
#!/bin/bash
sed -r 's/\.[a-z]+//g' $1 | xargs -I{} grep {} $2 | sed -r 's/_[0-9]+//g'
Give execution permissions to the script:
chmod +x myScript.sh
And executed like the next:
./diff.sh localFolder.txt remoteFolder.txt
You will have the same result.
This script only works for your case and this is an explain what do each part
# Remove .txt or any extensions for each line in localFolder
sed -r 's/\.[a-z]+//g' localFolder.txt
# The ouput of each line is received by grep and search in remoteFolder
| xargs -I{} grep {} remoteFolder.txt
# Remove the datetime
| sed -r 's/_[0-9]+//g'
I want to look through 100K+ text files from a directory and copy to another directory only the ones which contain at least one word from a list.
I tried doing an if statement with grep and cp but I have no idea how to make it to work this way.
for filename in *.txt
do
grep -o -i "cultiv" "protec" "agricult" $filename|wc -w
if [ wc -gt 0 ]
then cp $filename ~/desktop/filepath
fi
done
Obviously this does not work but I have no idea how to store the wc result and then compare it to 0 and only act on those files.
Use the -l option to have grep print all the filenames that match the pattern. Then use xargs to pass these as arguments to cp.
grep -l -E -i 'cultiv|protec|agricult' *.txt | xargs cp -t ~/desktop/filepath --
The -t option is a GNU cp extension, it allows you to put the destination directory first so that it will work with xargs.
If you're using a version without that option, you need to use the -J option to xargs to substitute in the middle of the command.
grep -l -E -i 'cultiv|protec|agricult' *.txt | xargs -J {} cp -- {} ~/desktop/filepath
Saying that I have two files t1 and t2, they have the same content: abc.
Now I want to delete all files, who contains the string abc.
So I tried to execute the command: grep -rl abc . | rm but it doesn't work.
Then I add xargs: grep -rl abc . | xargs rm and it works.
I can't understand clearly what xargs did.
grep puts the output as stdout. But rm cannot process data from stdin (the pipe links both).
You want instead, that the output of grep is put as argument of rm. So xargs command "convert" stdin into arguments of xargs first argument, and it call the command (the first argument).
As alternative, you could do
rm `grep -rl abc .`
or
rm $(grep -rl abc .)
But xargs handles well also the case where there are too many arguments for a single call of the command. The above command will give you shell error (argument string too long).
rm doesn't read from standard input (except when prompting, like with -i) but takes its arguments on the command line. That's what xargs does for you: read things from standard input and give them to rm as arguments.
Example with echo:
$ (echo a; echo b; date) | xargs echo
a b tor 12 apr 2018 14:18:50 CEST
I'm having trouble with trying to achieve this bash command:
Concatenate all the text files in the current directory that have at least one occurrence of the word BOB (in any case) within the text of the file.
Is it correct for me to do this use the cat command then use grep to find the occurences of the word BOB?
cat grep -i [BOB] *.txt > catFile.txt
To handle filenames with whitespace characters correctly:
grep --null -l -i "BOB" *.txt | xargs -0 cat > catFile.txt
Your issue was the need to pass grep's file names to cat as an inline function:
cat $(grep --null -l -i "BOB" *.txt ) > catFile.txt
$(.....) handles the inline execution
-l returns only filenames of the things that matched
You could use find with -exec:
find -maxdepth 1 -name '*.txt' -exec grep -qi 'bob' {} \; \
-exec cat {} + > catFile.txt
-maxdepth 1 makes sure you don't search any deeper than the current directory
-name '*.txt' says to look at all files ending with .txt – for the case that there is also a directory ending in .txt, you could add -type f to only look at files
-exec grep -qi 'bob' {} \; runs grep for each .txt file found. If bob is in the file, the exit status is zero and the next directive is executed. -q makes sure the grep is silent.
-exec cat {} + runs cat on all the files that contain bob
You need to remove the square brackets...
grep -il "BOB" *
You can also use the following command that you must run from the directory containing your BOB files.
grep -il BOB *.in | xargs cat > BOB_concat.out
-i is an option used to set grep in case insensitive mode
-l will be used to output only the filename containing the pattern provided as argument to grep
*.in is used to find all the input files in the dir (should be adapted to your folder content)
then you pipe the result of the command to xargs in order to build the arguments that cat will use to produce your file concatenation.
HYPOTHESIS:
Your folder does only contain files without strange characters in their name (e.g. space)
I have search string in one variable ($AUD_DATE) and replace string in another variable ($YEST_DATE). I need to search file name in a folder using $AUD_DATE and then replace it with $YEST_DATE.
I tried using this link to do it but its not working with variables.
Find and replace filename recursively in a directory
shrivn1 $ AUD_DATE=140101
shrivn1 $ YEST_DATE=140124
shrivn1 $ ls *$AUD_DATE*
NULRL.PREM.DATA.CLRSFIFG.140101.dat NULRL.PREM.DATA.CLRTVEH.140101.dat
shrivn1 $ ls *$AUD_DATE*.dat | awk '{a=$1; gsub("$AUD_DATE","$YEST_DATE");printf "mv \"%s\" \"%s\"\n", a, $1}'
mv "NULRL.PREM.DATA.CLRSFIFG.140101.dat" "NULRL.PREM.DATA.CLRSFIFG.140101.dat"
mv "NULRL.PREM.DATA.CLRTVEH.140101.dat" "NULRL.PREM.DATA.CLRTVEH.140101.dat"
Actual output I need is
mv "NULRL.PREM.DATA.CLRSFIFG.140101.dat" "NULRL.PREM.DATA.CLRSFIFG.140124.dat"
mv "NULRL.PREM.DATA.CLRTVEH.140101.dat" "NULRL.PREM.DATA.CLRTVEH.140124.dat"
Thanks in advance
Approach 1
I generally create mv commands using sed and then pipe the output to sh. This approach allows me to see the commands that will be executed beforehand.
For example:
$ AUD_DATE=140101
$ YEST_DATE=140124
$ ls -1tr | grep "${AUD_DATE}" | sed "s/\(.*\)/mv \1 \1_${YEST_DATE}"
Once you are happpy with the output of the previous command;repeat it and pipe it's output to sh, like so:
$ ls -1tr | grep "${AUD_DATE}" | sed "s/\(.*\)/mv \1 \1_${YEST_DATE}" | sh
Approach 2
You could use xargs command.
ls -1tr | grep ${AUD_DATE}" | xargs -I target_file mv target_file target_file${YEST_DATE}