issue with bash : 2 variables instead of one - bash

Question

I've written this piece of code.
The aim is the following :
for each files in the temp list, it should take the first occurence of the list, put it into a variable called $name1 and then the second occurence of the list into a second variable called $name2. The variables are file names. With the 2 variables, I do a join.
for files in $(cat temp.lst); do
if [ $(cat temp.lst | wc -l) == 1 ]
then
name=$(head -1 temp.lst)
join -t\; -j 1 file_minus1.txt "$name" | sed 's/;;/;/g' > file1.txt
else
name1=$(head -1 temp.lst)
name2=$(head -2 temp.lst)
echo "var1 "$name1 "var2 "$name2
sed '1,2d' temp.lst > tmpfile.txt
mv tmpfile.txt temp.lst
join -t\; -j 1 "$name1" "$name2" | sed 's/;;/;/g' > file_minus1.txt
fi
;done
Theoretically, it should work but here it is not working, alas.
The echo line I've put in my code is giving me 3 variables instead of 2
var1 ei_etea17_m.tsv var2 ei_etea17_m.tsv ei_eteu25_m.tsv
Worse, the join is not functionning the way I thought, giving me this error code instead
join: ei_etea17_m.tsv
ei_eteu25_m.tsv: No such file or directory
Please find a sample of my temp.lst
ei_eteu27_m.tsv
ei_eteu28_m.tsv
ei_isbr_m.tsv
ei_isbu_m.tsv
ei_isin_m.tsv
Any suggestions are welcomed.
Best.

Answer 1

To extract 2 lines of a file in a loop, try this:
paste - - < temp.lst |
while read name1 name2; do
if [[ -z $name2 ]]; then
name2=$name1
name1=file_minus1.txt
output=file1.txt
else
output=file_minus1.txt
fi
join -t\; "$name1" "$name2" | sed 's/;;/;/g' > $output
done
Notes
the paste command takes 2 consecutive lines from the file and joins them into a single line (separated by tab)
demo: seq 7 | paste - -
read can assign to multiple variables: the line will be split on whitespace (default) and assigned to the named variables.
in the loop body, I basically follow your logic

Answer 2

To perform an n-way join, use recursion :)
recursive_join () {
# Zero files: do nothing (special case)
# One file: output it
# Multiple files: join the first with the result of joining the rest
file1=$1
shift || return
[ "$#" -eq 0 ] && cat "$file1" ||
recursive_join "$#" | join -t\; -j1 "$file1" -
}
recursive_join ei_eteu27_m.tsv ei_eteu28_m.tsv ei_isbr_m.tsv ei_isbu_m.tsv ei_isin_m.tsv
Adapting this to use a file listing the input files, rather than using command-line arguments, is a little tricker. As long as none of the input file names contain whitespace or other special characters, you could simply use
recursive_join $(cat temp.lst)
Or, if you want to avail yourself of bash features, you could use an array:
while read; do files+=("$REPLY"); done < temp.lst
recursive_join "${files[#]}"
or in bash 4:
readarray files < temp.list
recursive_join "${files[#]}"
However, if you want to stick with standard shell scripting only, it would be better to modify the recursive function to read the input file names from standard input. This makes the function a little uglier, since in order to detect if there is only one file left on standard input, we have to try to read a second one, and put it back on standard input if we succeed.
recursive_join () {
IFS= read -r file1 || return
IFS= read -r file2 &&
{ echo "$file2"; cat; } | recursive_join | join -t\; -j1 "$file1" - ||
cat "$file1"
}
recursive_join < temp.lst
Creating a function that can take either command-line arguments or read a list from standard input is left as an exercise for the reader.

Answer 3

Variable name1 is getting the first line.
Variable name2 is getting the first two lines.
If you want name2 to have only the second line you could try something like:
name2=$(sed -n '2p')
Also sed -i will remove the need for tmpfile.txt.

Answer 4

Ok Gents or Ladies.
I found out the Why.
head -1 temp.lst is only given the file name without the extension.
I need to find a way to include the extension. Doable.