Problem in generating file names
I have around 4000 .txt files each containing three columns of data. I want to read all the 3 columns from a single file one at a time and then plot three values which correspond to x,y,z values on a contour plot.
These files are created at various time step. So a plot from one file will be a level curve and plots from all of them will give me a contour plot.
But the problem I want to do something which I can do in bash like this:
for n in `seq -f "%09g" 30001 200 830001`; do
./someFile$n.whateverFileFormat
done
How can I do this in matlab so that if I have let's say:
t-000030001.txt
1 2 3
......
......
......
t-0000320001.txt
2 4 5
. . .
. . .
. . .
and so on to
t-0008300001.txt
3 5 6
. . .
. . .
and on it goes.
I want to load all these files one at a time store the values in a infx3 array plot them on a contour plot and do this again and again for all the files so that I can have all of them on a single plot.
P.S. I need to reproduce something equivalent to that bash script mentioned above so as to load files appropriately then only I will be read from them
One way to get the list of file names is this:
fnames = arrayfun(#(num)sprintf('t-%09g.txt', num), 30001:200:830001, 'Uniformoutput', 0);
Let's have a closer look: 30001:200:830001 generates an array, starting at 30001, incrementing by 200, ending at 830001. sprintf generates a formatted string, and arrayfun applies the anonymous function passed as its first argument to each element of the array in its second argument (the sequence). The output is a cell array containing the file names.
EDIT
The solution above is equivalent to the following code:
ind = 30001:200:830001;
fnames = cell(numel(ind), 1);
for i = 1:numel(ind)
fnames{i} = sprintf('t-%09g.txt',ind(i));
end
This stores all the values in the a cell array.
Writing #(num)sprintf('t-%09g.txt', num) creates an anonymous function. The looping happens in arrayfun.
Related
I would like to plot multiple curve on the same graph using a for loop. Each data file (named stat_coupe) is located in a different folder (fwal055wal055/rep16/ and fwal055wal055_c2/rep20/). fwal055wal055 and fwal055wal055_c2 correspond to names of simulation. First, I need to get a previous result, a single number (Utau), in other files (named file_fwal055wal055 and file_fwal055wal055_c2). This is successfully done thanks to the command awk. The result depend on the file: Utaufwal055wal055=10.5 and Utaufwal055wal055_c2=12.2.
Then I need to divid the 1st column of the file stat_coupe corresponding to the path fwal055wal055/rep16/ by the value of Utaufwal055wal055 and do the same thing for the file stat_coupe corresponding to the path fwal055wal055_c2/rep20/ with the value of Utaufwal055wal055_c2. Moreover, each plot should have a specific format which depend on the type of simulation run (fwal055wal055 or fwal055wal055_c2).
The presented problem is reduced to 2 simulations fwal055wal055 and fwal055wal055_c2 and 1 plot but I have about 20 simulations and 15 various graphs to plot that is why I would like to use the for loop.
To summary at each iteration I have:
a specific format,
a specific path,
a specific value of Utau
I want to indicate the wright format, path and value of Utau at each iteration of the for loop. The solution I propose below successfully permits to obtain the value of Utau for each simulation but the code #path_.i and #format_.i does not work.
#!/bin/bash
for elem in fwal055wal055 fwal055wal055_c2;
do
Utau[${elem}]=$(awk 'FNR==5{print $1}' file_$elem)
done
gnuplot -persist <<-EOFMarker
format_fwal055wal055='pt 1 ps 1.0 lc 0 title "WALE"'
format_fwal055wal055_c2='pt 2 ps 1.0 lc 0 title "WALE c2"'
path_fwal055wal055='"fwal055wal055/rep16/stat_coupe"'
path_fwal055wal055_c2='"fwal055wal055_c2/rep20/stat_coupe"'
list="fwal055wal055 fwal055wal055_c2"
plot for [i in list] #path_.i u 1:(\$2/${Utau[${i}]}) #format_.i
EOFMarker
I would like to obtain something equivalent to:
plot #path_fwal055wal055 u 1:(\$2/${Utau[${i}]}) #format_fwal055wal055,\
#path_fwal055wal055_c2 u 1:(\$2/${Utau[${i}]}) #format_fwal055wal055_c2
Can someone help me to solve this issue ?
Thank you very much,
Martin
Check help sprintf, help words and help word.
I would create two strings with the same number of items and then combine them with sprintf(). From gnuplot 5.2 on you could also do it with arrays.
# Version 1
PATHS = '"fwal055wal055/rep16/stat_coupe" "fwal055wal055_c2/rep20/stat_coupe"'
FILES = "fwal055wal055 fwal055wal055_c2"
plot for [i=1:words(FILES)] sprintf("%s_%s",word(PATHS,i),word(FILES,i)) u 1:2
or you could define a function for your filenames to keep the plot command short and readable.
# Version 2
PATHS = '"rep16/stat_coupe" "rep20/stat_coupe"'
FILES = "fwal055wal055 fwal055wal055_c2"
myFilename(i) = sprintf("%s/%s_%s",word(FILES,i),word(PATHS,i),word(FILES,i))
plot for [i=1:words(FILES)] myFilename(i) u 1:2
Addition (after some clarifications...)
If I understand your question now correctly, the following code should do the job.
For the extraction of the UTAUS you do a separate loop before plotting and store the extracted values in a string. During plotting you get these values back via word(UTAUS,i). Since you do the mathematical operation column(2)/word(UTAUS,i), gnuplot will interpret them as number. Check help words, help word, help sprintf, help every.
Code:
### extract and normalize in a loop with individual files and directories
reset session
FILES = 'fwal055wal055 fwal055wal055_c2'
DIRS = 'rep16 rep20'
TITLES = '"WALE" "WALE c2"' # if you have spaces you need to put it into double quotes
UTAUS = ''
# define functions for better readability
myExtractionFile(i) = sprintf("file_%s",word(FILES,i))
myDataFile(i) = sprintf("%s/%s/stat_coupe",word(FILES,i),word(DIRS,i))
myTitle(i) = word(TITLES,i)
# define point or line appearance. Add more if you have more files
set style line 1 pt 1 ps 1.0 lc 0
set style line 2 pt 2 ps 1.0 lc 1
# extract the UTAUs
do for [i=1:words(FILES)] {
set table $Dummy
plot myExtractionFile(i) u (utau=$1) every ::4::4 w table # extract value row 5, column 1 (not counting header lines)
unset table
UTAUS = UTAUS.sprintf(" %g",utau) # append the extracted value as string
}
plot for [i=1:words(FILES)] myDataFile(i) u 1:(column(2)/word(UTAUS,i)) ls i title myTitle(i)
### end of code
I'm writing a program in IDL to read DICOM images, then store them in a big matrix and finally save them in .dat file. The DICOMs are under the name IM0,IM1,IM2,..IM21777. I wrote the code below but I am getting an error. I am using IDL version 6.4.
files = file_search('E:\SE7\IM*)
n_files = n_elements(files)
full_data = fltarr(256,256,n_files)
for i=0L, n_files-1 do begin
full_data[*,*,i] = read_dicom('E:\SE7\IM')
endfor
path = 'E:\'
open, 1, path + "full_data.dat'
writeu, 1, full_data
close, 1
I am not sure how to loop over the DICOM name i.e. IM0, IM1,IM2 etc
After I store them in the big matrix (i.e. full_data =[256,256,2178]) I would like to make the 3D matrix 4D. Is that possible? I would like to make it have the dimensions [256, 256, 22, 99] i.e. 2178/99.
I'm not sure what error you are getting, but you are missing a quotation mark in the first line. It should be:
files = file_search('E:\SE7\IM*')
To loop over the DICOM name, you can string concatenate the loop index using + and STRTRIM() as follows:
for i=0L, n_files-1 do begin
full_data[*,*,i] = read_dicom('E:\SE7\IM'+STRTRIM(i,2))
endfor
Finally, to turn your (256,256,2178) matrix into a (256,256,22,99) matrix, use REBIN:
final_data = REBIN(full_data, 256, 256, 20, 99)
Depending on the way you want the dimensions arranged, you may need additional operations. This post is a great primer on how to manipulate arrays and their dimensionality: Coyote Dimensional Juggling Tutorial.
I am using IML/SAS in SAS Enterprise Guide for the first time, and want to do the following:
Read some datasets into IML matrices
Average the matrices
Turn the resulting IML matrix back into a SAS data set
My input data sets look something like the following (this is dummy data - the actual sets are larger). The format of the input data sets is also the format I want from the output data sets.
data_set0: d_1 d_2 d_3
1 2 3
4 5 6
7 8 9
I proceed as follows:
proc iml;
/* set the names of the migration matrix columns */
varNames = {"d_1","d_2","d_3"};
/* 1. transform input data set into matrix
USE data_set_0;
READ all var _ALL_ into data_set0_matrix[colname=varNames];
CLOSE data_set_0;
USE data_set_1;
READ all var _ALL_ into data_set1_matrix[colname=varNames];
CLOSE data_set_1;
USE data_set_2;
READ all var _ALL_ into data_set2_matrix[colname=varNames];
CLOSE data_set_2;
USE data_set_3;
READ all var _ALL_ into data_set3_matrix[colname=varNames];
CLOSE data_set_3;
/* 2. find the average matrix */
matrix_sum = (data_set0_matrix + data_set1_matrix +
data_set2_matrix + data_set3_matrix)/4;
/* 3. turn the resulting IML matrix back into a SAS data set */
create output_data from matrix_sum[colname=varNames];
append from matrix_sum;
close output_data;
quit;
I've been trying loads of stuff, but nothing seems to work for me. The error I currently get reads:
ERROR: Matrix matrix_sum has not been set to a value
What am I doing wrong? Thanks up front for the help.
The above code works. In the full version of this code (this is simplified for readability) I had misnamed one of my variables.
I'll leave the question up in case somebody else wants to use SAS / IML to find an average matrix.
I have a number of files (having 10 columns each) with following order:
file_001.txt, file_002.txt, file_003_txt,
file_021.txt, file_023.txt, file_023.txt,
file_041.txt, file_042.txt, file_043.txt,
file_061.txt, file_062.txt, file_063.txt,
file_081.txt, file_082.txt, file_083.txt,
I would like to plot each file with different line. e.g. using 1:2, using 1:3, using 1:5, using 1:8. I can not able to make a loop to call different columns. My following script is not working for k field
plot for [k=2, 3, 5, 8] for [j=0:8:2] for [i=1:3] 'file_0'.j.i.'.txt' u 1:k;
Use for [k in "2 3 5 8"] if you have a list rather than a range.
If j can be > 9, you should set up a function
fname(j,i) = sprintf("name%02.f%.f",j,i)
to get proper file names.
Format string "%02.f" means float (f), no digits after the comma (.), minimum two postions (2), fill empty space with zeroes.
print fname(2,3)
name023
print fname(13,3)
name133
print fname(113,3)
name1133
These are libc format strings, they are not documented inside the gnuplot docs, but there are many sources in the web.
I have the following code, which plots 4 lines:
plot for [i=1:4] \
path_to_file using 1:(column(i)) , \
I also want to plot 8 horizontal lines on this graph, the values of which come from mydata.txt.
I have seen, from the answer to Gnuplot: How to load and display single numeric value from data file, that I can use the stats command to access the constant values I am interested in. I think I can access the cell (row, col) as follows:
stats 'mydata.txt' every ::row::row using col nooutput
value = int(STATS_min)
But their location is a function of i. So, inside the plot command, I want to add something like:
for [i=1:4] \
stats 'mydata.txt' every ::(1+i*10)::(1+i*10) using 1 nooutput
mean = int(STATS_min)
stats 'mydata.txt' every ::(1+i*10)::(1+i*10) using 2 nooutput
SE = int(STATS_min)
upper = mean + 2 * SE
lower = mean - 2 * SE
and then plot upper and lower, as horizontal lines on the graph, above.
I think I can plot them separately by typing plot upper, lower but how do I plot them on the graph, above, for all i?
Thank you.
You can create an array and store the values in it, then using an index that refers to the value's position in the array you can access it inside a loop.
You can create the array as follows:
array=""
do for [i=1:4] {
val = i / 9.
array = sprintf("%s %g",array,val)
}
where I have stored 4 values: 1/9, 2/9, 3/9 and 4/9. In your case you would run stats and store your upper and/or lower variables. You can check what the array looks like in this way:
gnuplot> print array
0.111111 0.222222 0.333333 0.444444
For plotting, you can access the different elements in the array using word(array,i), where i refers to the position. Since the array is a string, you need to convert it to float, which can be done multiplying by 1.:
plot for [i=1:4] 1.*word(array,i)
If you have values stored in a data file, you can process it with awk or even with gnuplot:
array = ""
plot for [i=1:4] "data" every ::i::i u (array=sprintf("%s %g",array,$1), 1/0), \
for [i=1:4] 1.*word(array,i)
The first plot instance creates the array from the first column data entries without plotting the points (the 1/0 option tells gnuplot to ignore them, so expect warning messages) and the second plot instance uses the values stored in array as variables (hence as horizontal lines in this case). Note that every takes 0 as the first entry, so [i=1:4] runs from the second through to the fifth lines of the file.