I have Makefile in which I get a list of files from a certain directory. I need to choose only those, in the names of which there are numbers (like 123.txt). What is the best way to do it?
Thanks.
You can use a glob or wildcard, for example:
all.txt: text/*.txt
cat $^ >> $#
Usage:
$ cd -- "$(mktemp --directory)"
$ mkdir text
$ echo foo > text/1.txt
$ echo bar > text/2.txt
$ make
cat text/1.txt text/2.txt >> all.txt
NUMBER_FILENAMES := $(shell echo " $(LIST) " | sed 's/ [^ 0-9]* / /g')
Related
I wrote a bash script trying to generate one directory named after each file inside the directory from which I run the script.
Original directory= /home/agalvez/data//sims/phylip_format
sim1.phylip
sim2.phylip
Directories to create = sim1 sim2
The contents of these new directories should be a copy of the original file that names the new directory and an extra file called "input". This file should contain the name of the .phylip file as well as the following:
"Name of original file"
U
5
Y
/home/agalvez/data/sims/trees/tree_nodenames.txt
After that I want to run the following command (sequentially) in all these new directories:
phylip dollop < input > screenout
My approach is the following one but it is not working:
!/bin/bash
for f in *.phylip;
mkdir /home/agalvez/data/sims/dollop/$f;
cp $f /home/agalvez/data/sims/dollop/$f;
cd /home/agalvez/data/sims/dollop/$f;
echo "$f" | cat > input;
echo "U" | cat >> input;
echo "5" | cat >> input;
echo "Y" | cat >> input;
echo "/home/agalvez/data/sims/trees/tree_nodenames.txt" | cat >> input;
phylip dollop < input > screenout;
;done
Edit: The error messge looks like this:
line 4: syntax error near unexpected token `mkdir'
line 4: ` mkdir /home/agalvez/data/sims/dollop/$f;'
FINAL SOLUTION:
#!/bin/bash
for f in *.phylip;
do
mkdir /home/agalvez/data/sims/dollop/$f;
cp /home/agalvez/data/sims/phylip_format/$f /home/agalvez/data/sims/dollop/$f;
cd /home/agalvez/data/sims/dollop/$f;
echo "$f" | cat > input;
echo "U" | cat >> input;
echo "5" | cat >> input;
echo "Y" | cat >> input;
echo "/home/agalvez/data/sims/trees/tree_nodenames.txt" | cat >> input;
phylip dollop < input > screenout;
done
The immediate problem is that you are lacking a do at the beginning of the loop body; but you'll want to refactor this code to avoid hardcoding the directory structure etc.
The first line needs to start with literally the two characters # and ! in order to be a valid shebang.
Notice also When to wrap quotes around a shell variable?
The printf could be replaced with a here document; I like the compactness of printf here.
#!/bin/bash
for f in *.phylip; do
mkdir -p dollop/"$f"
cp "$f" dollop/"$f"
cd dollop/"$f"
printf "%s\n" "$f" "U" "5" "Y" \
"/home/agalvez/data/sims/trees/tree_nodenames.txt" |
phylip dollop > screenout
done
Going forward, try http://shellcheck.net/ for diagnosing many common beginner problems in shell scripts.
Assuming you have a directory named pingping in your ${HOME} folder with files 1.txt, 2.txt, 3.txt. You can accomplish that like this. Modify this code to suit your needs.
#! /bin/bash
working_directory="${HOME}/pingping/"
cd $working_directory
for f in *.txt
do
mkdir "${f%%.*}"
if [ -f "${f%%.*}.txt" ]
then
if [ -d "${f%%.*}" ]
then
cp ${f%%.*}.txt ${f%%.*}
echo "Done copying"
#phylip dollop < input > screenout
#echo "Succesfully ran the command
fi
else
echo "not found"
fi
done
By running this:
GITIGNORE_CONTENTS = $(shell while read -r line; do printf "$$line "; done < "$(.gitignore)")
all:
echo $(GITIGNORE_CONTENTS)
The contents of the variable GITIGNORE_CONTENTS are the expanded version of the ignore patterns on my .gitignore file, i.e., myfile.txt instead of *.txt
I am using this solution because none of the solution on the question Create a variable in a makefile by reading contents of another file work.
This solution works inside a make rule to print the file names, but not put them inside a variable:
all:
while read -r line; do \
printf "$$line "; \
done < ".gitignore"
echo $(GITIGNORE_CONTENTS)
This is an issue of shell's wildcard expansion, not of make's. Consider this,
GITIGNORE_CONTENTS:=$(shell cat .gitignore)
.PHONY: all
all:
echo "$(GITIGNORE_CONTENTS)"
# or alternatively:
#/bin/echo $(GITIGNORE_CONTENTS)
By running this:
GITIGNORE_CONTENTS = $(shell while read -r line; do printf "$$line "; done < ".gitignore")
all:
echo "$(GITIGNORE_CONTENTS)"
With the following .gitignore file:
*.txt
*.var
# Comment
*.xml
*.html
It correctly outputs:
echo "*.txt *.var # Comment *.xml *.html "
*.txt *.var # Comment *.xml *.html
I would like to change the file name from Sub****_Ses1 to HU_TT_12_****_UU;
(**** numbered from 0001 to 1600)
I did the below
#!/bin/sh
#Change file name
Subj_id=/Users/dave/biomark/dat
cd Subj_id
for abcd in Sub****_Ses1; do
mv Sub$a$b$c$d_Ses1 HU_TT_12_$a$b$c$d_UU;
done
for and wildcards don't work like this. Use cut to extract the number.
$ touch Sub000{1,2,3,4}_Ses1
$ for f in Sub????_Ses1
do
abcd=$(echo $f | cut -b4-7)
mv $f HU_TT_12_${abcd}_UU
done
$ ls HU_TT_12_000*
HU_TT_12_0001_UU HU_TT_12_0002_UU HU_TT_12_0003_UU HU_TT_12_0004_UU
You can use sed and mv
#!/bin/bash
set -x
Subj_id=/Users/dave/biomark/dat
cd $Subj_id
for i in Sub*_Ses1 ; do
#echo $i|sed -r 's/^.*\([[:digit:]]{4}\).*/HU_TT_12_\1_UU/'
mv $i $(echo $i|sed -rn 's/^.*([[:digit:]]{4}).*/HU_TT_12_\1_UU/ p')
done
Sample data in a stackoverflow.csv file I have:
foo
foo
foo
foo
bar
bar
bar
bar
baz
baz
baz
baz
I know with sed -n /foo/p stackoverflow.csv > foo.csv
I'll get all records matching foo directed to that file, but I don't want to specify the matching pattern on the cli, I'd rather put it in a script and have all records (foo, bar and baz) sent to their own file.
Basically this in a script:
sed -n /foo/p stackoverflow.csv > foo.csv
sed -n /bar/p stackoverflow.csv > bar.csv
sed -n /baz/p stackoverflow.csv > baz.csv
Like this:
#!/bin/sh
sleep 2
sed -n /foo/p > foo.csv
sed -n /bar/p > bar.csv
sed -n /baz/p > baz.csv
This creates the files but they're all empty.
Also, if the script were to have just one print statement, it works.
Any input?
You missed the input filename
#!/bin/sh
sleep 2
sed -n /foo/p stackoverflow.csv > foo.csv
sed -n /bar/p stackoverflow.csv > bar.csv
sed -n /baz/p stackoverflow.csv > baz.csv
You can provide input file as an argument to your script, in that case change the script to read an argument and pass it to the sed command.
Run the script like this: ./script.sh input_filename, where you can specify different input files as argument.
#!/bin/sh
file=${1}
sed -n /foo/p ${file} > foo.csv
sed -n /bar/p ${file} > bar.csv
sed -n /baz/p ${file} > baz.csv
Here's a solution with grep in a loop where you don't need to provideeach term in advance. Assuming a bash shell:
for i in $(sort stackoverflow.csv | uniq); do grep $i testfile > $i; done
The w command in sed allows you to write to a named file.
sed -n -e '/foo/wfoo.csv' -e '/bar/wbar.csv' \
-e '/baz/wbaz.csv' stackoverflow.csv
Not all sed dialects accept multiple -e arguments; a multi-line string with the commands separated by newlines may be the most portable solution.
sed -n '/foo/wfoo.csv
/bar/wbar.csv
/baz/wbaz.csv' stackoverflow.csv
This examines the input file just once, and writes out the matching lines as it proceeds through the file.
To create a script which accepts one or more file names as arguments, replace the hard-coded file name with "$#".
#!/bin/sh
sed -n ... "$#"
im looking for a bash script to count the occurences of a word in a given directory and it's subdirectory's files with this pattern:
^str1{n}str2{m}$
for example:
str1= yo
str2= uf
n= 3
m= 4
the match would be "yoyoyoufufufuf"
but i'm having trouble with grep
that's what i have tried
for file in $(find $dir)
do
if [ -f $file ]; then
echo "<$file>:<`grep '\<\$str1\{$n\}\$str2\{$m\}\>'' $file | wc -l >" >> a.txt
fi
done
should i use find?
#Barmar's comment is useful.
If I understand your question, I think this single grep command should do what you're looking for:
grep -r -c "^\($str1\)\{$n\}\($str2\)\{$m\}$" "$dir"
Note the combination of -r and -c causes grep to output zero-counts for non-matching files. You can pipe to grep -v ":0$" to suppress this output if you require:
$ dir=.
$ str1=yo
$ str2=uf
$ n=3
$ m=4
$ cat youf
yoyoyoufufufuf
$ grep -r -c "^\($str1\)\{$n\}\($str2\)\{$m\}$" "$dir"
./noyouf:0
./youf:1
./dir/youf:1
$ grep -r -c "^\($str1\)\{$n\}\($str2\)\{$m\}$" "$dir" | grep -v ":0$"
./youf:1
./dir/youf:1
$
Note also $str1 and $str2 need to be put in parentheses so that the {m} and {n} apply to everything within the parentheses and not just the last character.
Note the escaping of the () and {} as we require double-quotes ", so that the variables are expanded into the grep regular expression.