I have created a shell script (.run) that accepts the prefix for the names of the pictures as a parameter, and then calls gnuplot. However, for some reason, the picture is not saved. The code is:
#!/bin/sh
molecule=$1
echo "Plotting DFT-ADF PY results for $molecule"
echo "Tranmission plot (negatory SO)"
gnuplot -p << EOF
#!/usr/bin/gnuplot
set terminal epslatex size 5,3 color colortext
set output '$molecule_trans.tex'
plot cos(x) w l title 'cos(x)', sin(x) w l title 'sin(x)'
EOF
For my bachelor thesis I have to make several plots that are the same. Additionally, the computational cluster uses a qeueing system. For the purpose of being true to this system, I have created several shell scripts that automatically do stuff. In particular, about 45 simulations are called by the shell scripts, followed by a shell script that enters each simulations' directory and uses python files to evaluate the data into [.dat] files. Next, it should use a gnuplot file to make the graph. I use EPSLaTeX to make my figures, because it is so much nicer. However, in the current implementation this required me to manually edit the various latex files to rename the pictures.
In case you'll need more variables and do not want a $1, $2... mess:
You must use brackets around the variable name
set output '${molecule}_trans.tex'
because the underscore is a valid character for variable names, and bash looks for the variable $molecule_trans, see http://www.gnu.org/software/bash/manual/bashref.html#Shell-Parameter-Expansion.
Related
I need to plot multiple files in one plot with a bash script. I can manage this by stating the location of the files manually in plot function ie
plot "$outputdir/10/values.csv" using 1:2 with lines title "10", \
"$outputdir/20/values.csv" using 1:2 with lines title "20", \
"$outputdir/30/values.csv" using 1:2 with lines title "30"
This will print all three values files in one plot. But I have a dynamic array which changes depending on how many values it is suppose to get. So lets say the array arr looks like this instead
arr=(10 20 30 40)
#Sometimes arr has more values ex; arr=(10 20 30 40 50 60 70 80)
#arr corresponds to folders that will be created
#And will contain a values.csv file that I want to plot
Then the 40/values.csv will not be printed if I don't manually change the plot code
My current attempt to fix this is a for loop in gnuplot
#The array arr is dynamically generated in another function
#Varies in size as example above
ArrLength=${#arr[#]} #Get the length of array
plot for [p=0:$ArrLength] "$outputdir/${arr[$p]}/values.csv" using 1:2 with lines title "${arr[$p]}"
I don't get a error when plotting but in plot only one value is plotted and thats the first value in array ie it will only plot $outputdir/10/values.csv.
I tried setting p=0:4 to plot the first five files and still it only plotted the first file only. What is wrong with the for loop above?
Best regards
You seem to be mixing bash and gnuplot in a strange way. Using a bash script to try to generate a gnuplot script on the fly with inserted variables is a quick way to confuse yourself. It also makes it difficult to read and debug. It is easy to forget what bash is evaluating and what gnuplot is evaluating.
When I look at your first line
ArrLength=${#arr[#]} #Get the length of array
I can see that this is bash code because gnuplot would interpret a comment beginning with the first #. (This is also bash's syntax for arrays, not gnuplot's.) The dollar sign $ has a different meaning in gnuplot. Rather than mark a variable identifier, $ is a column number operator ($2 is column 2, $i is column i, etc.). So look at the line
plot for [p=0:$ArrLength] "$outputdir/${arr[$p]}/values.csv" using 1:2 with lines title "${arr[$p]}"
This is clearly a line of bash syntax, apparently inside a string trying to write a line of gnuplot. Bash will evaluate the variables $ArrLength, $outputdir, and ${arr[$p]}, and replace them with some string of their values. Also keep in mind that p is a variable in gnuplot, not a variable in bash. Bash will evaluate $p to something (an empty string if it has not been defined). You can't expect the gnuplot variable p to be used as the index in the bash evaluation of ${arr[$p]}, and then somehow result in a different string for each iteration of gnuplot's loop.
In short, what you have written is not gnuplot syntax, and it is really not a minimal and complete bash script either. It is not clear exactly how you intended bash and gnuplot to fit together like this, but it seems you have joined them too tightly.
My suggestion is to write a bash script and write a gnuplot script (as separate files). Gnuplot has its own flow control, iteration loops, and variable evaluation. You can write a self-contained gnuplot script for the general case of everything you need it to do, and then give it specifics on the command line from your bash script.
For example, it seems that your subdirectories are all multiples of 10, and always starting with 10. The only variable aspect is how many there are (what the last one is). Let's say this last value was somehow stored in a gnuplot variable last. Also, suppose we also somehow have the base output directory in outputdir:
(Inside the gnuplot script, named plot.gp):
plot for [p=10:last:10] sprintf("%s/%d/values.csv", outputdir, p) with lines title sprintf("%d", p)
The for [p=10:last:10] means to iterate from 10 through last (inclusive), adding 10 at each iteration. The first sprintf() function (like C) builds a string with the outputdir and p (both are variables in gnuplot). The using 1:2 is not necessary as the first two columns are the default columns to use with lines, but you can include them if you want to be explicit. The second sprintf() builds a title string from the iteration variable p.
Now, this assumes that outputdir and last have meaningful values. You can assign these values from your bash script when you invoke gnuplot on the command line:
(Inside the bash script, invoke the gnuplot script)
gnuplot -e "last=40" -e "outputdir=\"$outputdir\"" plot.gp
The -e option tells gnuplot to evaluate the given string before running the script in the file plot.gp. In this example, the gnuplot variable last will have the value 40 and the gnuplot variable outputdir will have whatever value bash evaluates $outputdir to be. Notice the escaped double quotes inside double quotes. The outer double quotes are to allow bash to evaluate variables inside the string ($outputdir needs to be evaluated by bash). The inner (escaped) quotes are to delimit the string within the gnuplot code. For example, if bash evaluates $outputdir to data, then gnuplot would see outputdir="data" which is a valid gnuplot assignment of a string to the variable outputdir. You could, if you want, combine these two -e options into one:
gnuplot -e "last=40;outputdir=\"$outputdir\"" plot.gp
You will likely want to use the value for last from your array in bash, rather than hard coding it like this. So in practice it may look more like
gnuplot -e "last=${arr[${#arr[#]}-1]};outputdir=\"$outputdir\"" plot.gp
Or, if you have bash 4.3 or later, you should be able to use a negative index:
gnuplot -e "last=${arr[-1]};outputdir=\"$outputdir\"" plot.gp
Notice that there are no escaped quotes around the use of the array variable. It is expected that it will evaluate to an integer (40, 90, etc.) and we want to assign last to an integer, not a string like outputdir.
If this one string seems complex, try thinking about the entire script like this. It would be easy to get confused as to what bash is doing and what gnuplot is doing.
In summary, write a bash script, and a separate gnuplot script. Gnuplot is capable of handling a general case. From bash, just give it some specifics on the fly, don't try to generate the entire script on the fly. It really does make things simpler.
I created an empty directory on zsh and added a file
called hello.rb by doing the following:
echo 'Hello, world.' >hello.rb
If I want to make changes in this file using the terminal
what's the proper way of doing it without opening the file
itself using let's say TextEditor?
I want to be able to make changes in the file hello.rb strictly
by using my zsh terminal, is this at all possible?
Zsh is not a terminal but a shell. The terminal is the window in which the shell executes. The shell is the text program prompting you commands and executing them.
If you want to edit the file within the terminal, then using vim, nano, emacs -nw or any other text-mode text editor will do it. They are not Zsh commands, but external commands that you can call from Zsh or from any other shell.
If you want to edit the file within Zsh, then use zed. You will need to run once (in ~/.zshrc)
autoload zed
and then you can edit hello.rb using:
zed hello.rb
(exit and save with Control-j)
You have already created and edited the file.
To edit it again, you can use the >> to append.
For example
echo "\nAnd you too!\n" >> hello.rb
This would edit the file by concatenating the additional string.
Edit, of course, by your use and definition of 'changing' a file, this is the simplest way to do so using the shell.
In a normal way, though you probably want to use a terminal editor.
Zed is a great answer, but to be even more stripped down - for a level of editing that even a script can do - zsh can hand all 256 characters/byte-values (including null) in variables. This means you can edit line by line or chunk by chunk almost any kind of file data directly from the command-line. This is approximately what zed/vared does. If you have a current version with all the standard modules included, it is a great benefit to have zsh/mapfile or zsh/system loaded so that you can capture any of the characters that are left out by command-expansion (zed uses $(<$file) to read a file to memory). Here is an example of a way you could use this variable manipulation method:
% typeset -T Buffer buffer $'\n'
% typeset -T Edit edit $'\n'
It is most common to use newline to divide a text file one wishes to edit.
This handy feature will make zsh give you full access to one line or a range of lines at a time without unintentionally messing with the data.
% zmodload zsh/mapfile
% Buffer=$mapfile[path/to/file]
Here, I use the handy mapfile module because I can load the contents of a file byte-for-byte. Alternately you can use % Buffer="$(<path/to/file)", like zed does, but you will always have the trailing newlines removed and other word splitting is possible with a typo or environment variation, so the simplicity of the module's method is best. When finished, you save the changes by simply assigning the $Buffer value back to the $mapfile[file] or use a more classic command like printf '%s' $Buffer >path/to/file (this is exact string writing, byte-for-byte, so any newlines or formatting you added back will be written).
You transfer the lines between Buffer and Edit using the mapped arrays as follows, however, remember that in its simplest form assigning one array to another drops elements that are completely empty (one \n \n two \n three becomes one \n two \n three). You can suppress this empty-element removal by quoting the input array and adding an '#' symbol to its index "$buffer[#]", if using the whole array; and adding the '#' symbol to the flags if using a range of the array "${(#)buffer[2,50]}". Preserving empty lines can be a bit troublesome for typing, but these multiple arrays should only be used in a script or function, since you can just edit one line at a time from the command line with buffer[54]="echo This is a newly written line."
% edit=($buffer[50,70])
...
% buffer[50,70]=($edit)
This is standard Zsh syntax, that means in the ... area you can edit and manipulate the $edit array of lines or the $Edit scalar block of text all you want, including adding more lines or taking some away. When you add the lines back into $buffer it will replace the specified block of lines (50-70) with the new lines, automatically expanding or reducing the other array elements to accommodate the reintegrated lines. -- Because of the dynamic array accommodations, you can also just insert whatever you need as a new line like this buffer[40]=("new string as new line" "$buffer[40]"). This inserts it before the index given, while swapping the order of the elements ("$buffer[40]" "new string as new line") inserts the new line after the index given. Either will adjust all following elements, including totally empty elements, to their current index plus one.
If you wanted to re-write the zed function to use this method in some complex way like: newzed /path/to/file [start-line] [end-line], that would be great and handy too.
Before I leave, I wanted to mention that using vared directly, once you have these commands typed on the interactive terminal, you may find it frustrating that you can't use "Enter" for inserting or appending new lines. I found that with my terminal and Zsh version using ESC-ENTER worked well, but I don't know about older versions (Mac usually comes stocked with a not-most-recent version, if my memory is right). If that doesn't work, you may have to do some documentation digging to learn how to set up your ZLE (Zsh Line Editor, a component of Zsh) or acquire a newer version of Zsh. Also, some other shells, when indexing a scalar variable may count by the byte because in ascii and C a byte is the same as a character, but Zsh supports UTF8 and will index a scalar string by the UTF8 character unless you turn off the shell option multibyte (on by default). This will help with manipulating each line if you need to use the old byte-character indexing. Also, if you have a version of Zsh that for whatever was not compiled with zsh/mapfile or zsh/system, then you can achieve a similar effect using number of options to the read builtin, like <path/to/file |read -u 0 -k $[5 * 2**20] -r -s Buffer ||(($#Buffer)). As you can see here, you have to make the read length big enough to accommodate the file's size or it will leave off part of the file, and the read return code will nearly always be an error because of not being able to read the full length of the string. We fix this with ||(($#Buffer)), but this builtin was simply not meant to handle large scale byte manipulation efficiently, so what you see is what you can get from it.
When I try to write a shell script such as
gnuplot
cd '/home/cagirici/test'
plot "case1.test" with linespoints
and run it in sudo, the script stops after running gnuplot.
In each .test file, I have 12 columns. I won't plot them all at once but choose what to plot. For instance, if I choose to plot the success percentage, it should plot 1:2 and 1:8. But if I choose error percentage, it should plot 1:4 and 1:10 (it is basically i:(i+6).)
I need to write a scripts that takes two inputs: i) file name ii) what to plot.
Notice that I also want to choose the line type, point type and output shell. (if there is a configuration to set specific line and point types as default values, please tell me this as well)
Can I write this script in Ubuntu? Or should I use a different method.
My script would look something like
case "$2" in
("success percentage") plot "$1" using 1:2, plot "$1" using 1:8
("error percentage") plot "$1" using 1:4, plot "$1" using 1:10
You can make this work by creating a Gnuplot script with variables and then passing values to these variables at run time from command line or your bash script.
Lets call the Gnuplot script Myscript.gp:
set terminal pngcairo
set output 'graph.png'
plot file u x:y w linespoints
Here file and x:y can be varried when you invoke Gnuplot from your bash script. A possible call to Myscript.gp is as follows:
gnuplot -e "x=1;y=2;file = \"mydata.dat\"" Myscript.gp
In the snippet of your bash script that you provide this may look like:
case "$2" in
("success percentage") gnuplot -e "x=1; y=2; file = \"$1\"" Myscript.gp
Below I'm also providing a dummy mydata.dat file which you can use to try things.
1.54 23.66 43.66
1.75 26.25 46.25
1.92 30.20 40.20
2.08 34.46 44.46
2.44 42.08 42.08
2.78 46.81 46.81
3.03 51.10 41.10
3.70 52.99 42.99
4.17 56.15 46.15
4.76 59.34 49.34
If you are calculating the column numbers (x and y) in your bash script then you can also use variables inside the plot command.
gnuplot -e "x=$x;y=$y;file = \"mydata.dat\"" Myscript.gp
Below I show the output once using:
gnuplot -e "x=1;y=2;file = \"mydata.dat\"" Myscript.gp
and once
gnuplot -e "x=1;y=3;file = \"mydata.dat\"" Myscript.gp
I had faced similar problem 2 years back, then my senior team members decided to write a tool for plotting graphs using a simple csv and plotting specifications and tool will plot graphs for you,
Initially it was designed for sar data on linux so named RSAR but in general it is a csv plotter.)
if required you can contact me nachiket.kate90#gmail.com
I'm dealing with a pipeline of predominantly shell and Perl files, all of which pass parameters (paths) to the next. I decided it would be better to use a single file to store all the paths and just call that for every file. The issue is I am using awk to grab the files at the beginning of each file, and it's turning out to be a lot of repetition.
My question is: I do not know if there is a way to store key-value pairs in a file so shell can natively do something with the key and return the value? It needs to access an external file, because the pipeline uses many scripts and a map in a specific file would result in parameters being passed everywhere. Is there some little quirk I do not know of that performs a map function on an external file?
You can make a file of env var assignments and source that file as need, ie.
$ cat myEnvFile
path1=/x/y/z
path2=/w/xy
path3=/r/s/t
otherOpt1="-x"
Inside your script you can source with either . myEnvFile or the more versbose version of the same feature sourc myEnvFile (assuming bash shell) , i.e.
$cat myScript
#!/bin/bash
. /path/to/myEnvFile
# main logic below
....
# references to defined var
if [[ -d $path2 ]] ; then
cd $path2
else
echo "no pa4h2=$path2 found, can't continue" 1>&1
exit 1
fi
Based on how you've described your problem this should work well, and provide a-one-stop-shop for all of your variable settings.
IHTH
In bash, there's mapfile, but that reads the lines of a file into a numerically-indexed array. To read a whitespace-separated file into an associative array, I would
declare -A map
while read key value; do
map[$key]=$value
done < filename
However this sounds like an XY problem. Can you give us an example (in code) of what you're actually doing? When I see long piplines of grep|awk|sed, there's usually a way to simplify. For example, is passing data by parameters better than passing via stdout|stdin?
In other words, I'm questioning your statement "I decided it would be better..."
I know this question has been asked, but I can't find more than one solution, and it does not work for me. Essentially, I'm looking for a bash script that will take a file list that looks like this:
image1.jpg
image2.jpg
image3.jpg
And then make a copy of each one, but number it sequentially backwards. So, the sequence would have three new files created, being:
image4.jpg
image5.jpg
image6.jpg
And yet, image4.jpg would have been an untouched copy of image3.jpg, and image5.jpg an untouched copy of image2.jpg, and so on. I have already tried the solution outlined in this stackoverflow question with no luck. I am admittedly not very far down the bash scripting path, and if I take the chunk of code in the first listed answer and make a script, I always get "2: Syntax error: "(" unexpected" over and over. I've tried changing the syntax with the ( around a bit, but no success ever. So, either I am doing something wrong or there's a better script around.
Sorry for not posting this earlier, but the code I'm using is:
image=( image*.jpg )
MAX=${#image[*]}
for i in ${image[*]}
do
num=${i:5:3} # grab the digits
compliment=$(printf '%03d' $(echo $MAX-$num | bc))
ln $i copy_of_image$compliment.jpg
done
And I'm taking this code and pasting it into a file with nano, and adding !#/bin/bash as the first line, then chmod +x script and executing in bash via sh script. Of course, in my test runs, I'm using files appropriately titled image1.jpg - but I was also wondering about a way to apply this script to a directory of jpegs, not necessarily titled image(integer).jpg - in my file keeping structure, most of these are a single word, followed by a number, then .jpg, and it would be nice to not have to rewrite the script for each use.
Perhaps something like this. It will work well for something like script image*.jpg where the wildcard matches a set of files which match a regular pattern with monotonously increasing numbers of the same length, and less ideally with a less regular subset of the files in the current directory. It simply assumes that the last file's digit index plus one through the total number of file names is the range of digits to loop over.
#!/bin/sh
# Extract number from final file name
eval lastidx=\$$#
tmp=${lastidx#*[!0-9][0-9]}
lastidx=${lastidx#${lastidx%[0-9]$tmp}}
tmp=${lastidx%[0-9][!0-9]*}
lastidx=${lastidx%${lastidx#$tmp[0-9]}}
num=$(expr $lastidx + $#)
width=${#lastidx}
for f; do
pref=${f%%[0-9]*}
suff=${f##*[0-9]}
# Maybe show a warning if pref, suff, or width changed since the previous file
printf "cp '$f' '$pref%0${width}i$suff'\\n" $num
num=$(expr $num - 1)
done |
sh
This is sh-compatible; the expr stuff and the substring extraction up front is ugly but Bourne-compatible. If you are fine with the built-in arithmetic and string manipulation constructs of Bash, converting to that form should be trivial.
(To be explicit, ${var%foo} returns the value of $var with foo trimmed off the end, and ${var#foo} does similar trimming from the beginning of the value. Regular shell wildcard matching operators are available in the expression for what to trim. ${#var} returns the length of the value of $var.)
Maybe your real test data runs from 001 to 300, but here you have image1 2 3, and therefore you extract one, not three digits from the filename. num=${i:5:1}
Integer arithmetic can be done in the bash without calling bc
${#image[#]} is more robust than ${#image[*]}, but shouldn't be a difference here.
I didn't consult a dictionary, but isn't compliment something for your girl friend? The opposite is complement, isn't it? :)
the other command made links - to make copies, call cp.
Code:
#!/bin/bash
image=( image*.jpg )
MAX=${#image[#]}
for i in ${image[#]}
do
num=${i:5:1}
complement=$((2*$MAX-$num+1))
cp $i image$complement.jpg
done
Most important: If it is bash, call it with bash. Best: do a shebang (as you did), make it executable and call it by ./name . Calling it with sh name will force the wrong interpreter. If you don't make it executable, call it bash name.