I have a template file that I use to submit PBS jobs to a server. I need to make n number of copies of this template. Ultimately I would like to enter the following command or something similar:
copy n pbs_template
I would like the newly made duplicate files to be named:
pbs_template_1
pbs_template_2
pbs_template_3
pbs_template_4
pbs_template_n....
The following is what i have so far...
function copy() {
INPUT=pbs_template
number=$1
shift
for n in $(seq $number); do
cp "$INPUT" "$INPUT"
done
}
Obviously I need to specify the name of my output (otherwise I get the following error cp: pbs_template and pbs_template are identical (not copied)), but how do I make them number sequentially n times?
Try something like this?
function duplicate () {
for i in $(seq 1 $2)
do
cp $1 "$1_$i"
echo "Made copy '$1_$i'"
done
}
You call it like so duplicate foo.txt 10. It will create 10 copies of foo.txt each with a numerical suffix. In this invocation, $1 will be foo.txt and $2 will be 10.
just change your statement to this,
cp -p "$INPUT" "$INPUT"_${n}
here i have used -p switch in order to preserve attributes of file.
If you don't want then just ignore -p switch.
Hope that helps.
Related
I am new in sh and I am trying to scan output files and take some rows starts with "enthalpy new" to a new file. In advance I created a file named step.txt in the directory. There is not a certain number of files so I tried to do it like that:
for (( i=1; i<=$# ; i++))
do
grep "enthalpy new" $i >> step.txt
done
And I wrote this command to the terminal:
$bash hw1.sh sample2.out sample1.out
Then, I took these errors:
grep: 1: No such a file or directory
grep: 2: No such a file or directory
I am expecting to have step.txt file having 28 lines which 12 of them coming from sample1.out and 16 of them coming from sample2.out. Inside of step.txt will look like:
enthalpy new = 80 Ry
enthalpy new = 76 Ry
....
....
enthalpy new = 90 Ry
Is there anyone to tell my error and to help me fixing the code?
At present $I is referencing the iterations of the loop and so 1,2,3 .... These files cannot be found by grep and hence the error.
There are two approaches to ocercome this. $# contains the parameters passed to the script and so you could try:
grep "enthalpy new" "$#" >> step.txt
Alternatively, if you want to loop through each parameter/file try:
for fil in "$#"
do
grep "enthalpy new" "$fil" >> step.txt
done
To input filenames or any other variable you have to use $1 $2 $3 e.g. as the input for the script. It would work more flexible if you would drop them in a specific directory (let's say ./output) and call the script without variables in the parent directory - then it would be more flexible in terms of how many files you drop in there, without incriminating the variables and capturing input for code injection - the code should look like this:
for i in $(find ./output -name '*.out')
do
grep "enthalpy new" $i >> step.txt
done
I have a script that extracts filenames from an input file and is supposed to read each line (filename) and unzip the specified file, saving the unzipped content as individual files. However, I can't get my counter to work and just get all the unzipped files in one large file.
Input file contains a list:
ens/484/59/traj.pdb 0.001353
ens/263/39/traj.pdb 0.004178
ens/400/35/traj.pdb 0.004191
I'm using the regex /.*?/.*?/ to extract the file that I'd like to unzip and name each output{1..40}.pdb -- instead I get one output file: output1.pdb which contains all the contents of the 40 unzipped files.
My question is: how do I correct my counter in order to achieve the desired naming scheme?
#!/bin/bash
file="/home/input.txt"
grep -Po '/.*?/.*?/' $file > filenames.txt
i=$((i+1))
structures='filenames.txt'
while IFS= read line
do
gunzip -c 'ens'$line'traj.pdb.gz' >> 'output'$i'.pdb'
done <"$structures"
rm "$structures"
file="/home/input.txt"
grep -Po '/.*?/.*?/' $file > filenames.txt
structures='filenames.txt'
i=1
while IFS= read "line"
do
gunzip -c 'ens'$line'traj.pdb.gz' >> 'output'$i'.pdb'
i=$(expr $i + 1)
done <$structures
rm $structures
couple of logical mistakes, the counter has to be fined as one out of the while loop and the counter +1 should be inside the loop, also for the counter to work you have to use expr, in this case i made the counter start from 1, so the first entry will get this value. Also on the parameter for the while loop i dont really understand what you are doing, if it works as you have it then cool or else use a test statement after while and before the parameters.
I have a folder with, say, ten data files I01.txt, ..., I10.txt.. Each file, when executed using the command /a.out, gives me five output files, namely f1.txt, f2.txt, ... f5.txt.
I have written a simple bash program to execute all the files and save the output printed on the screen to a variable file using the command
./ cosima_peaks_444_temp_muuttuva -args > $counter-res.txt.
Using this, I am able to save the on screen output to the file. But the five files f1 to f5 are altered to store results of the last file run, in this case I10, and the results of the first nine files are lost.
So I want to save the output of each I*.txt file (f1 ... f5) to a a different file such that, when the program executes I01.txt, using ./a.out it stores the output of the files
f1>var1-f1.txt , f2>var1-f2.txt... f5 > var1-f5.txt
and then repeats the same for I02 (f1>var2-f1.txt ...).
#!/bin/bash
# echo "for looping over all the .txt files"
echo -e "Enter the name of the file or q to quit "
read dir
if [[ $dir = q ]]
then
exit
fi
filename="$dir*.txt"
counter=0
if [[ $dir == I ]]
then
for f in $filename ; do
echo "output of $filename"
((counter ++))
./cosima_peaks_444_temp_muuttuva $f -m202.75 -c1 -ng0.5 -a0.0 -b1.0 -e1.0 -lg > $counter-res.txt
echo "counter $counter"
done
fi
If I understand you want to pass files l01.txt, l02.txt, ... to a.out and save the output for each execution of a.out to a separate file like f01.txt, f02.txt, ..., then you could use a short script that reads each file named l*.txt in the directory and passes the value to a.out redirecting the output to a file fN.txt (were N is the same number in the lN.txt filename.) This presumes you are passing each filename to a.out and that a.out is not reading the entire directory automatically.
for i in l*.txt; do
num=$(sed 's/^l\(.*\)[.]txt/\1/' <<<"$i")
./a.out "$i" > "f${num}.txt"
done
(note: that is 's/(lowercase L) ... /\one..')
note: if you do not want the same N from the filename (with its leading '0'), then you can trim the leading '0' from the N value for the output filename.
(you can use a counter as you have shown in your edited post, but you have no guarantee in sort order of the filenames used by the loop unless you explicitly sort them)
note:, this presumes NO spaces or embedded newline or other odd characters in the filename. If your lN.txt names can have odd characters or spaces, then feeding a while loop with find can avoid the odd character issues.
With f1 - f5 Created Each Run
You know the format for the output file name, so you can test for the existence of an existing file name and set a prefix or suffix to provide unique names. For example, if your first pass creates filenames 'pass1-f01.txt', 'pass1-f02.txt', then you can check for that pattern (in several ways) and increment your 'passN' prefix as required:
for f in "$filename"; do
num=$(sed 's/l*\(.*\)[.]txt/\1/' <<<"$f")
count=$(sed 's/^0*//' <<<"$num")
while [ -f "pass${count}-f${num}.txt" ]; do
((count++))
done
./a.out "$f" > "pass${count}-f${num}.txt"
done
Give that a try and let me know if that isn't closer to what you need.
(note: the use of the herestring (<<<) is bash-only, if you need a portable solution, pipe the output of echo "$var" to sed, e.g. count=$(echo "$num" | sed 's/^0*//') )
I replaced your cosima_peaks_444_temp_muuttuva with a function myprog.
The OP asked for more explanation, so I put in a lot of comment:
# This function makes 5 output files for testing the construction
function myprog {
# Fill the test output file f1.txt with the input filename and a datestamp
echo "Output run $1 on $(date)" > f1.txt
# The original prog makes 5 output files, so I copy the new testfile 4 times
cp f1.txt f2.txt
cp f1.txt f3.txt
cp f1.txt f4.txt
cp f1.txt f5.txt
}
# Use the number in the inputfile for making a unique filename and move the output
function move_output {
# The parameter ${1} is filled with something like I03.txt
# You can get the number with a sed action, but it is more efficient to use
# bash functions, even in 2 steps.
# First step: Cut off from the end as much as possiple (%%) starting with a dot.
Inumber=${1%%.*}
# Step 2: Remove the I from the Inumber (that is filled with something like "I03").
number=${Inumber#I}
# Move all outputfiles from last run
for outputfile in f*txt; do
# Put the number in front of the original name
mv "${outputfile}" "${number}_${outputfile}"
done
}
# Start the main processing. You will perform the same logic for all input files,
# so make a loop for all files. I guess all input files start with an "I",
# followed by 2 characters (a number), and .txt. No need to use ls for listing those.
for input in I??.txt; do
# Call the dummy prog above with the name of the first input file as a parameter
myprog "${input}"
# Now finally the show starts.
# Call the function for moving the 5 outputfiles to another name.
move_output "${input}"
done
I guess you have the source code of this a.out binary. If so, I would modify it so that it outputs to several fds instead of several files. Then you can solve this very cleanly using redirects:
./a.out 3> fileX.1 4> fileX.2 5> fileX.3
and so on for every file you want to output. Writing to a file or to a (redirected) fd is equivalent in most programs (notable exception: memory mapped I/O, but that is not commonly used for such scripts - look for mmap calls)
Note that this is not very esoteric, but a very well known technique that is regularly used to separate output (stdout, fd=1) from errors (stderr, fd=2).
This question already has answers here:
Concatenating Files And Insert New Line In Between Files
(8 answers)
Closed 7 years ago.
How can we copy all the contents of all the files in a given directory into a file so that there are two empty lines between contents of each files?
Need not to mention, I am new to bash scripting, and I know this is not an extra complicated code!
Any help will be greatly appreciated.
Related links are following:
* How do I compare the contents of all the files in a directory against another directory?
* Append contents of one file into another
* BASH: Copy all files and directories into another directory in the same parent directory
After reading comments, my initial attempt is this:
cat * > newfile.txt
But this does not create two empty lines between contents of each new files.
Try this.
awk 'FNR==1 && NR>1 { printf("\n\n") }1' * >newfile.txt
The variable FNR is the line number within the current file and NR is the line number overall.
One way:
(
files=(*)
cat "${files[0]}"
for (( i = 1; i < "${#files[#]}" ; ++i )) ; do
echo
echo
cat "${files[i]}"
done
) > newfile.txt
Example of file organization:
I have a directory ~/Pictures/Temp
If I wanted to move PNG's from that directory to another directory I would first want to set a variable for my file names:
# This could be other file types as well
file=$(find ~/Pictures/Temp/*.png)
Of course there are many ways to view this check out:
$ man find
$ man ls
Then I would want to set a directory variable (especially if this directory is going to be something like a date
dir=$(some defining command here perhaps an awk of an ls -lt)
# Then we want to check for that directories existence and make it if
# it doesn't exist
[[ -e $dir ]] || mkdir "$dir"
# [[ -d $dir ]] will work here as well
You could write a for loop:
# t is time with the sleep this will be in seconds
# super useful with an every minute crontab
for ((t=1; t<59; t++));
do
# see above
file=$(blah blah)
# do nothing if there is no file to move
[[ -z $file ]] || mv "$file" "$dir/$file"
sleep 1
done
Please Google this if any of it seems unclear here are some useful links:
https://www.gnu.org/software/gawk/manual/html_node/For-Statement.html
http://www.gnu.org/software/gawk/manual/gawk.html
Best link on this page is below:
http://mywiki.wooledge.org/BashFAQ/031
Edit:
Anyhow where I was going with that whole answer is that you could easily write a script that will organize certain files on your system for 60 sec and write a crontab to automatically do your organizing for you:
crontab -e
Here is an example
$ crontab -l
* * * * * ~/Applications/Startup/Desktop-Cleanup.sh
# where ~/Applications/Startup/Desktop-Cleanup.sh is a custom application that I wrote
How do I merge ls with wc -l to get the name of a file, modification time and number of rows in a file?
thanks!
There are a number of ways you can approach this from the shell or your programming language of choice, but there's really no "right" way to do this, since you need to both stat and read each file in order to form your custom output. You can do this without pipelines inside a basic for-loop by using command substitution:
custom_ls () {
for file in "$#"; do
echo "$file, $(date -r "$file" '+%T'), $(wc -l < "$file")"
done
}
This will generate output like this:
$ custom_ls .git*
.gitconfig, 14:02:56, 44
.gitignore, 17:07:13, 21
There are certainly other ways to do it, but command substitution allows the intent of the format string to remain short and clear, without complex pipelines or temporary variables.