I'm trying to make directories using a variable in the directory name from a variables in an array, but it doesn't seem to concatenate the streams like I thought they would. I've tried a couple different ways.
I'm trying to get three directories named:
test_6_bash_with_directory, test_8_bash_with_directory, test_10_bash_with_directory
variable=`seq 6 2 10`
for i in "${variable[#]}"
do
:
directory_name="./test_${i}_bash_with_directory"
mkdir $directory_name
echo $i
done
This gives me three directories test_6, 8 and 10_bash_with_directory. Replacing ${i} with ${variable} has the same result.
I'd also tried having the mkdir call in the same line as the directory concatenation:
mkdir "./test_${i}_bash_with_directory"
and I got one directory called test_6?8?10_bash_with_directory
So, how do I write this correctly? Thank you for replies!
Your problem is that variable is a string, not an array. You want:
variable=(`seq 6 2 10`)
for i in "${variable[#]}"
do
directory_name="./test_${i}_bash_with_directory"
mkdir $directory_name
echo $i
done
Note the (...) around your sequence of values.
Related
I have 40 csv files that I need to edit. 20 have matching format and the names only differ by one character, e.g., docA.csv, docB.csv, etc. The other 20 also match and are named pair_docA.csv, pair_docB.csv, etc.
I have the code written to edit and combine docA.csv and pair_docA.csv, but I'm struggling writing a loop that calls both the above files, edits them, and combines them under the name combinedA.csv, then goes on the the next pair.
Can anyone help my rudimentary bash scripting? Here's what I have thus far. I've tried in a single for loop, and now I'm trying in 2 (probably 3) for loops. I'd prefer to keep it in a single loop.
set -x
DIR=/path/to/file/location
for file in `ls $DIR/doc?.csv`
do
#code to edit the doc*.csv files ie $file
done
for pairdoc in `ls $DIR/pair_doc?.csv`
do
#code to edit the piar_doc*.csv files ie $pairdoc
done
#still need to combine the files. I have the join written for a single iteration,
#but how do I loop the code to save each join as a different file corresponding
#to combined*.csv
Something along these lines:
#!/bin/bash
dir=/path/to/file/location
cd "$dir" || exit
for file in doc?.csv; do
pair=pair_$file
# "${file#doc}" deletes the prefix "doc"
combined=combined_${file#doc}
cat "$file" "$pair" >> "$combined"
done
ls, on principle, shouldn't be used in a shell script in order to iterate over the files. It is intended to be used interactively and nearly never needed within a script. Also, all-capitalized variable names shouldn't be used as ordinary variables, since they may collide with internal shell variables or environment variables.
Below is a version without changing the directory.
#!/bin/bash
dir=/path/to/file/location
for file in "$dir/"doc?.csv; do
basename=${file#"$dir/"}
pair=$dir/pair_$basename
combined=$dir/combined_${basename#doc}
cat "$file" "$pair" >> "$combined"
done
This might work for you (GNU parallel):
parallel cat {1} {2} \> join_{1}_{2} ::: doc{A..T}.csv :::+ pair_doc{A..T}.csv
Change the cat commands to your chosen commands where {1} represents the docX.csv files and {2} represents the pair_docX.csv file.
N.B. X represents the letters A thru T
I have a question concerning why this doesn't work. Probably, it's a simple answer, but I just can't seem to figure it out.
I want to move a couple of files I have. They all have the same filename (let's say file1) but they are all in different directories (lets say /tmp/dir1,dir2 and dir3). If I were to move these individually I could do something along the lines of:
mv /tmp/dir1/file1 /tmp
That works. However, I have multiple directories and they're all going to end up in the same spot....AND I don't want to overwrite. So, I tried something like this:
mv /tmp/{dir1,dir2,dir3}/file1 /tmp/file1.{a,b,c}
When I try this I get:
/tmp/file1.c is not a directory
Just to clarify...this also works:
mv /tmp/dir1/file1 /tmp/file1.c
Pretty sure this has to do with brace expansion but not certain why.
Thanks
Just do echo to understand how the shell expands:
$ echo mv /tmp/{dir1,dir2,dir3}/file1 /tmp/file1.{a,b,c}
mv /tmp/dir1/file1 /tmp/dir2/file1 /tmp/dir3/file1 /tmp/file1.a /tmp/file1.b /tmp/file1.c
Now you can see that your command is not what you want, because in a mv command, the destination (directory or file) is the last argument.
That's unfortunately now how the shell expansion works.
You'll have to probably use an associative array.
!/bin/bash
declare -A MAP=( [dir1]=a [dir2]=b [dir3]=c )
for ext in "${!MAP[#]}"; do
echo mv "/tmp/$ext/file1" "/tmp/file1.${MAP[$ext]}"
done
You get the following output when you run it:
mv /tmp/dir2/file1 /tmp/file1.b
mv /tmp/dir3/file1 /tmp/file1.c
mv /tmp/dir1/file1 /tmp/file1.a
Like with many other languages key ordering is not guaranteed.
${!MAP[#]} returns an array of all the keys, while ${MAP[#]} returns the an array of all the values.
Your syntax of /tmp/{dir1,dir2,dir3}/file1 expands to /tmp/dir1/file /tmp/dir2/file /tmp/dir3/file. This is similar to the way the * expansion works. The shell does not execute your command with each possible combination, it simply executes the command but expands your one value to as many as are required.
Perhaps instead of a/b/c you could differentiate them with the actual number of the dir they came from?
$: for d in 1 2 3
do echo mv /tmp/dir$d/file1 /tmp/file1.$d
done
mv /tmp/dir1/file1 /tmp/file1.1
mv /tmp/dir2/file1 /tmp/file1.2
mv /tmp/dir3/file1 /tmp/file1.3
When happy with it, take out the echo.
A relevant point - brace expansion is not a wildcard. It has nothing to do with what's on disk. It just creates strings.
So, if you create a bunch of files named with single letters or digits, echo ? will wildcard and list them all, but only the ones actually present. If there are files for vowels but not consonants, only the vowels will show. But -
if you say echo {foo,bar,nope} it will output foo bar nope regardless of whether or not any or all of those exist as files or directories, etc.
I have a template file that I use to submit PBS jobs to a server. I need to make n number of copies of this template. Ultimately I would like to enter the following command or something similar:
copy n pbs_template
I would like the newly made duplicate files to be named:
pbs_template_1
pbs_template_2
pbs_template_3
pbs_template_4
pbs_template_n....
The following is what i have so far...
function copy() {
INPUT=pbs_template
number=$1
shift
for n in $(seq $number); do
cp "$INPUT" "$INPUT"
done
}
Obviously I need to specify the name of my output (otherwise I get the following error cp: pbs_template and pbs_template are identical (not copied)), but how do I make them number sequentially n times?
Try something like this?
function duplicate () {
for i in $(seq 1 $2)
do
cp $1 "$1_$i"
echo "Made copy '$1_$i'"
done
}
You call it like so duplicate foo.txt 10. It will create 10 copies of foo.txt each with a numerical suffix. In this invocation, $1 will be foo.txt and $2 will be 10.
just change your statement to this,
cp -p "$INPUT" "$INPUT"_${n}
here i have used -p switch in order to preserve attributes of file.
If you don't want then just ignore -p switch.
Hope that helps.
I created an array in bash by arr=($(seq 0.75 0.01 1)) . Problem is I want to create multiple folders with names like this, "folder0.75 , folder0.76,... folder1" . I tried mkdir folder${arr} but it does not give me what I want.
for i in ${arr[#]}; do
mkdir "folder${i}"
done
or with bash's Parameter Expansion:
mkdir "${arr[#]/#/folder}"
Use the following code:
for item in $(cmd); do
mkdir "$item";
done
I have the following script:
#!/bin/sh
FILES=../folder/files/*
echo "Total files in array : ${#FILES[#]}"
for f in $FILES
do
echo "Processing file $f"
done
In ../folder/files/ I have 3 files and the for loops through them properly. However, the number of files in array is incorrect. It returns 18 instead of 3.
I am sure there are not any other files in the folder.
Concretely for my purpose, the output is:
$ sh run_benchmark.sh
Total files in array : 18
Running benchmark for file ../benchmark/cfg/2_150.cfg
Running benchmark for file ../benchmark/cfg/2_300.cfg
Running benchmark for file ../benchmark/cfg/2_500.cfg
What is wrong?
Thank you!
Unless for some special reason you need arrays, don't bother yourself with arrays, simply do:
for f in ../path/*
do
echo "==$f=="
done
Using #EtanReisner comments:
#!/bin/bash
FILES=(../benchmark/cfg/*)
n=0
for f in "${FILES[#]}"
do
let n++
echo "Processing file($n from ${#FILES[#]}) ==$f=="
done
This line FILES=../folder/files/* is not expanding the glob.
You aren't expanding the glob until this line for f in $FILES.
I was going to say that when you write echo "Total files in array : ${#FILES[#]}" you are asking the shell for the length of the string ../folder/files/* but that seems to not be the case. I don't know exactly what the shell is doing here (trying to use FILES as an array and failing but getting one element (the value of FILES?) and reporting 1?).
Anyway, if you want the glob expansion in an array then you need to use an array.
FILES=(../folder/files/*)
And then you can use "${#FILES[#]}" to get the length and for f in "${FILES[#]}" to iterate over the values of the array.
Still, in case someone wants to use arrays for some reason:
list=( $path/* )
that assumes no funny files with spaces, tabs or newlines in the names.