How to automate saving plots with Shell scripting - bash

I have 10 .txt files that contain my data as in output1.txt, output2.txt, output3.txt and so on. I plot this data with the following commands:
#!/bin/bash
gnuplot
set o "output1.png"
p "output1.txt" u 1:2 with lines
set o "output2.png"
p "output2.txt" u 1:2 with lines
set o "output3.png"
p "output3.txt" u 1:2 with lines
and so on.
Obviously when I have like 100 data files, this approach is not very useful. How can I write a shell script that basically automates saving my data into files with a for loop in a script? Is it even possible or should I turn somewhere else like a python script?

I guess you require something like this:
for each in output*.txt
do
BASE_NAME=$(basename $each .txt)
echo ${BASE_NAME}
gnuplot <<- EOF
set o "${BASE_NAME}.png"
p "$each" u 1:2 with lines
EOF
done

Completely untested:
#!/bin/bash
for fn in $(ls output*.txt); do
# Extract file "ID"
fid=${fn:6:(-4)}
# Build output png
out=output${fid}.png
# Echo the commands
echo set o "${out}"
echo p "${fn}" u 1:2 with lines
done
Which you'd either pipe directly into gnuplot:
./abovescript.sh | gnuplot
Or redirect into a script file, then call using gnuplot
./abovescript.sh > myscript
gnuplot myscript
Updated per Mark Setchell's suggestions:
for fn in output*.txt; do
# Extract the file "base"
fid=${fn%%.txt}
# Build output png
out=${fid}.png
# Echo the commands
echo set o \"${out}\"
echo p \"${fn}\" u 1:2 with lines
done | gnuplot
This also assumes that your "input" files are named exactly outputX.txt where X doesn't contain any whitespace.
Added by Mark Setchell... just for fun, the plots look kind of more professional if you label them a little bit, by adding the following:
...
out=...
# Echo the commands
echo set title \"${out} - plotted with gnuplot\"
echo set ylabel \"y axis\"
echo set xlabel \"x axis\"

Related

"invalid expression" running gnuplot in bash script

I am running this program SCRIPT.sh,
#!/bin/bash
FFT_FILE="$(find . -type f -iname "HR*.fft")"
FFT_FILE1=$FFT_FILE".GNUPLOT"
od -f -v -w8 $FFT_FILE > $FFT_FILE1
gnuplot -persist <<-EOFMarker
reset
set term postscript enh
set out 'FFT_1.ps'
set xlabel 'Frequency (Hz)'
set key box
filename="$FFT_FILE1"
stats filename nooutput
max_row = STATS_records
plot [(1000.0/4880.0):(1000.0/59.0)] filename u ($0/(max_row*81.92e-6)):(sqrt(($2*$2)+($3*$3))) w l lw 3 title "FFT"
reset
EOFMarker
but the output comes as
gnuplot> plot [(1000.0/4880.0):(1000.0/59.0)] filename u (./SCRIPT.sh/(max_row*81.92e-6)):(sqrt((*)+(*))) w l lw 3 title "FFT"
^
line 0: invalid expression
May I request for detailed explanation and help regarding this situation?
The "HR*.fft" file contains the results of Fast Fourier Transform from PRESTO in binary.

shell script of gnuplot: about various input files in a single graph for compare the lines

I am trying to plot a graph and save it as file using gnuplot. I have various text files. The number of text files is not sure. Each text file contains two columns. The text file is about linear absorption spectrum. The first column is the frequency and the second is the intensity. I want to compare the difference between the spectra by plot them in a single graph for all the files. I am using this script to do that.
#!/bin/bash
# plot linear absorption spectrum (las)
# Usage:
# ./lasplot.sh a.ods b.ods c.ods ..
j=1
for i in $*
do
echo "Normalization $i"
maxle=`cat $i|tr -s "," " "|awk 'BEGIN{max=0} {if($2>max) max=$2}END{print max}'`
echo " The max value is $maxle"
max[$j]=$maxle
filename[$j]=$i
((j=$j + 1))
done
num=$#
echo ${filename[*]}
echo $num
read -p "Please select set data range (1) or auto set(2): " setrange
case $setrange in
1)
read -p "set min x range: " minx
read -p "set max x range: " maxx
;;
esac
# call gnuplot to plot the 1D map
gnuplot << EOF
set datafile separator ","
set title 'Linear Absorption Spectra'
set xlabel '{/Symbol w} (cm^{-1})'
set ylabel 'Intensity (a.u.)'
#set data range
if (${setrange}==1){
set xrange [$minx:$maxx]
set yrange [0:1.1]
} else {
#set xrange [0:50]
#set yrange [0:50]
}
set size 1,1
set key right top
plot "${filename[1]}" using 1:(column(2)/${max[1]}) with lines linewidth 2 ,\
"${filename[2]}" using 1:(column(2)/${max[2]}) with points pointtype 21
pause mouse keypress "Type a letter"
set term pdfcairo font "arial,12"
set term pdfcairo enh
set term pdfcairo size 6,5
set output "las.pdf"
plot "${filename[1]}" using 1:(column(2)/${max[1]}) with lines linewidth 2 ,\
"${filename[2]}" using 1:(column(2)/${max[2]}) with points pointtype 21
EOF
My script can only plot two text files? How can change it to plot multiple input files?
you could prepare the plot command in your script before invoking gnuplot, store it into a variable and then reuse it in the gnuplot input (also avoiding thus repetition):
plotCmd="plot"
for((i=1;i<=${num};i++))
do
if [ $i -eq 1 ]; then
join=" "
else
join=", "
fi
plotCmd="${plotCmd}${join}\"${filename[$i]}\" using 1:2 with lines"
done
and then:
gnuplot << EOF
... some other stuff ...
${plotCmd}
EOF

Speed up gnuplot column iteration on input piped through a shell command

I have a script that converts raw data for interactive use with gnuplot. The scipt takes arguments that let me apply some filtering on the data. In principle, it could be emulated by this script:
import time, sys
print('Generating data...', file=sys.stderr)
time.sleep(1)
print('x', '100*x', 'x**3', '2**x')
for x in range(*map(int, sys.argv[1:3])):
print(x, 100*x, x**3, 2**x)
I plot the series of data in columns by piping the shell command to gnuplot and iterating over columns:
gnuplot> plot for [i=2:4] '< python3 pipe_data.py 1 11' u 1:i w l t columnhead(i)
Generating data...
Generating data...
Generating data...
gnuplot>
Currently, when I notice that the script takes too long to run it multiple times, I execute it outside of gnuplot and save the output to a file. However, it is cumbersome having to do it whenever I want to change the arguments to the script.
I would like gnuplot to execute '< python3 pipe_data.py' only once, so that only one Generating data... is printed on the screen. Is this possible?
Ideally, gnuplot would cache contents of special filenames starting with a <. This way it would be possible to tweak the appearance of the plot, without regenerating the data, e.g.:
gnuplot> plot for [i=2:4] '< python3 pipe_data.py 1 11' u 1:i w l t columnhead(i)
Generating data...
gnuplot> plot for [i=2:4] '< python3 pipe_data.py 1 11' u 1:i w lp t columnhead(i)
gnuplot> plot for [i=2:4] '< python3 pipe_data.py 5 12' u 1:i w lp t columnhead(i)
Generating data...
gnuplot> plot for [i=2:4] '< python3 pipe_data.py 1 11' u 1:i w p t columnhead(i)
gnuplot>
This could become problematic when the raw data changes, gnuplot would have no way of knowing this. But I still hope there is some way to achieve this effect. If not with just gnuplot, then maybe with some external tools?
For the record, I use gnuplot v4.6.
The following command should do what you want. I am generating a ona-data-point file that takes two parameters, i and j. The file is generated automatically when you call plot data(i,j) and then it's reused every subsequent time. Change my sleep 5; echo %i %i by your command. You'll also need to change the formats if you're not using integers.
data(i,j) = system(sprintf('name="temp_%i_%i"; if [ ! -s $name ]; then sleep 5; echo %i %i > $name; fi; echo $name', i, j, i, j))
Example usage:
# You'll notice a 5-second pause here while the command is running:
plot data(1,1)
# Now it will run at once because the file already exists:
plot data(1,1)
I came up with a bash script that solves all the issues for me:
when iterating over columns, the script for crunching data is run only once
when tweaking plot display, there's no need to wait for the script to run anymore
I can pass other arguments to the script and have the results cached
I can come back to previous arguments without having to wait
it works with any script, regardless of what and how many arguments it may take
if any file (script or data) changes, it will be picked up and re-run
it is completely transparent, there's no additional set-up nor tuning needed
I don't have to worry about cleaning up, because the system will do it for me
I named it $ (for cash/cache), chmod u+x'd it, and placed it within PATH:
#!/bin/bash
# hash all arguments
KEY="$#"
# hash last modified dates of any files
for arg in "$#"
do
if [ -f $arg ]
then
KEY+=`date -r "$arg" +\ %s`
fi
done
# use the hash as a name for temporary file
FILE="/tmp/command_cache.`echo -n "$KEY" | md5sum | cut -c -10`"
# use cached file or execute the command and cache it
if [ -f $FILE ]
then
cat $FILE
else
$# | tee $FILE
fi
Now I can take advantage of it using <$ instead of <:
> plot for [i=2:4] '<$ python3 pipe_data.py 1 11' u 1:i w l t columnhead(i)
Generating data...
> plot for [i=2:4] '<$ python3 pipe_data.py 1 11' u 1:i w lp t columnhead(i)
> plot for [i=2:4] '<$ python3 pipe_data.py 5 12' u 1:i w lp t columnhead(i)
Generating data...
> plot for [i=2:4] '<$ python3 pipe_data.py 1 11' u 1:i w p t columnhead(i)
>

Using R with in command bash terminal

I have a set of files *.txt in a specific directory. I have written an .r file code called SampleStatus.r which contains a unique function that reads, proceeses data and writes the results to an output file.
The function is like:
format_windpro(import_file="in.txt", export_file="out.txt")
I would like to use bash commands to read and compute every file in one command using my R file.
Use Rscript. Example code:
for f in ${INPUT_DIR}/*.txt; do \
base=$(basename $f) \
Rscript SampleStatus.R $f ${OUTPUT_DIR}/$base \
done
While in your SampleStatus.R you handle command line arguments like this:
#!/usr/bin/env Rscript
# ...
argv <- commandArgs(T)
# error checking...
import_file <- argv[1]
export_file <- argv[2]
# your function call
format_windpro(import_file, export_file)

Bash user prompt while reading file

I am trying to create a user prompt while reading a file a line by line in Bash. The idea is for me to plot various files one-by-one using Gnuplot. Here is what I have:
#!/bin/bash
echo "Enter filename that contains the filenames:"
read fname
xr="[1e8:1e20]"
yr="[1:1e13]"
while read line
do
echo -e "reset\nset log\nset xrange$xr\nset yrange$yr\nset xlabel \"Frequency [Hz]\"\nset ylabel \"F_{/Symbol n} [Jy Hz]\"\nset key top left\nplot \"$line.dat\" u 3:(\$3*\$4)*1e26 w l ti \"$line^o\" \n"> plt.gp
gnuplot plt.gp
done < $fname
I would like to enter a user input/"continue?" type thing before "gnuplot plt.gp" command, because at the moment it just plots everything rapidly and then exits. The standard read -p command does not work here. I read somewhere I may need to use file descriptor exec 5 command, but I don't understand. Thanks.
#!/bin/bash
read -p 'Enter filename that contains the filenames: ' fname
xr="[1e8:1e20]"
yr="[1:1e13]"
while read line
do
echo -e "reset\nset log\nset xrange$xr\nset yrange$yr\nset xlabel \"Frequency [Hz]\"\nset ylabel \"F_{/Symbol n} [Jy Hz]\"\nset key top left\nplot \"$line.dat\" u 3:(\$3*\$4)*1e26 w l ti \"$line^o\" \n"> plt.gp
gnuplot plt.gp
read -p 'Do you want to continue? [Y/n]: ' want_to_continue </dev/tty
case "${want_to_continue}" in
Y|y)
continue
;;
*)
echo "OK. Bye!"
break
;;
esac
done < ${fname}

Resources