sed: no input files - bash

I have a script where I read in columns of data from an "input file", then use those to change some variables in another file.
Here is my script, titled FA_grid_changer.sh
#!/bin/bash
# before runnning, please ensure your inlist has the following parameters: RSP_mass, RSP_Teff, RSP_L, RSP_X, RSP_Z,
# log_directory, photo_directory, RSP_alfa, photo_interval, profile_interval, history_interval, terminal_interval,
# max_num_profile_models, RSP_max_num_periods
export OMP_NUM_THREADS=8
# get current directory
dir=$PWD
data_file="$dir"/input.dat
mesa_inlist="$dir"/inlist_rsp_RR_Lyrae
# set STR to 1 for saving history and profile files BEFORE the model has reached full-amplitude pulsations
# set STR to 2 for saving history and profile files AFTER the model has reached full-amplitude pulsations
STR=1
max_num_periods=4000
max_num_periods_2=$(($max_num_periods + 4))
if [[ "$STR" = 1 ]]
then
echo Configuring inlist to save settings before full-amplitude pulsations...
sed -i \
-e "s/^\([[:blank:]]*photo_interval\).*/\1 = 1000/i" \
-e "s/^\([[:blank:]]*profile_interval\).*/\1 = 1000/i" \
-e "s/^\([[:blank:]]*history_interval\).*/\1 = 1000/i" \
-e "s/^\([[:blank:]]*terminal_interval\).*/\1 = 4000/i" \
-e "s/^\([[:blank:]]*max_num_profile_models\).*/\1 = 100/i" \
-e "s/^\([[:blank:]]*RSP_max_num_periods\).*/\1 = $max_num_periods/i" \
"$mesa_inlist"
sleep 1.5
while read -ra fields; do
echo Changing model parameters...
sleep 0.5
sed -i \
-e "s/^\([[:blank:]]*RSP_mass\).*/\1 = ${fields[3]}/i" \
-e "s/^\([[:blank:]]*RSP_Teff\).*/\1 = ${fields[5]}/i" \
-e "s/^\([[:blank:]]*RSP_L\).*/\1 = ${fields[4]}/i" \
-e "s/^\([[:blank:]]*RSP_X\).*/\1 = ${fields[2]}/i" \
-e "s/^\([[:blank:]]*RSP_Z\).*/\1 = ${fields[1]}/i" \
-e "s/^\([[:blank:]]*log_directory\).*/\1 = 'LOGS\/LOGS_${fields[0]}'/i" \
-e "s/^\([[:blank:]]*photo_directory\).*/\1 = 'photos\/photos_${fields[0]}'/i" \
#-e "s/^\([[:blank:]]*RSP_alfa\).*/\1 = ${fields[6]}/i" \
"$mesa_inlist"
echo Compiling MESA...
sleep 0.5
./mk
echo Running MESA...
sleep 0.5
./rn
done < "$data_file"
elif [[ "$STR" = 2 ]]
then
echo Configuring inlist to save settings after full-amplitude pulsations...
sed -i \
-e "s/^\([[:blank:]]*photo_interval\).*/\1 = 1000/i" \
-e "s/^\([[:blank:]]*profile_interval\).*/\1 = 1/i" \
-e "s/^\([[:blank:]]*history_interval\).*/\1 = 1/i" \
-e "s/^\([[:blank:]]*terminal_interval\).*/\1 = 4000/i" \
-e "s/^\([[:blank:]]*max_num_profile_models\).*/\1 = -1/i" \
-e "s/^\([[:blank:]]*RSP_max_num_periods\).*/\1 = $max_num_periods_2/i" \
"$mesa_inlist"
sleep 1.5
else
echo STR\: Not an acceptable option. Please \set STR to 1 or 2.
fi
Here is my input file input.dat, which I am keeping in the same directory.
RRL1 0.001 0.749 0.61036165206037 48.3930329458203 6795.10018238295
RRL2 0.001 0.749 0.60627327846453 59.8125833648911 6793.11236483182
RRL3 0.001 0.749 0.606272337551755 56.5141274900899 7059.10474568471
And here is the file I am trying to change inlist_rsp_RR_Lyrae
! long-period RR Lyrae star: M=0.65Ms, L=60Ls, Teff=6500K, X=0.75, Z=0.0004
! original test case contributed by Radek Smolec.
&star_job
!pgstar_flag = .true.
/ ! end of star_job namelist
&controls
! check for retries and backups as part of test_suite
!max_number_retries = -1
!max_number_backups = 0
!max_model_number = 15000
! RSP controls
! x_integer_ctrl(1) = 10 ! which period to check
x_ctrl(1) = 0.71262d0 ! expected period (in days)
RSP_mass = 0.606272337551755
RSP_Teff = 7059.10474568471
!RSP_L = 59.3141274900899
RSP_L = 56.5141274900899
RSP_X = 0.749
RSP_Z = 0.001
RSP_max_num_periods = 4000
RSP_nmodes = 10 ! number of modes LINA will calculate for initial model
RSP_nz = 150 ! number of zones of model
log_directory = 'LOGS/LOGS_M081_second'
photo_directory = 'photos/photos_M081_second'
! output controls
!num_trace_history_values = 2
trace_history_value_name(1) = 'rel_E_err'
trace_history_value_name(2) = 'log_rel_run_E_err'
! for cases in which you have a run that has reached steady pulses
! and you want to look at plots of just a few periods to see the details,
! i suggest the following method. interrupt the run soon after
! it makes a photo. remove or delete LOGS/history.data to force
! creation of a new one when restart. edit the inlist to set
! history_interval to 1 to get maximum time resolution.
! restart the run and let it go for as many periods as desired.
do_history_file = .true.
photo_interval = 1000
profile_interval = 1000
history_interval = 1000
terminal_interval = 4000
max_num_profile_models = 100
!photo_interval = 1
!profile_interval = 1
!history_interval = 1
!terminal_interval = 1
! FOR DEBUGGING
!RSP_report_undercorrections = .true.
!report_hydro_solver_progress = .true. ! set true to see info about newton iterations
!report_ierr = .true. ! if true, produce terminal output when have some internal error
!stop_for_bad_nums = .true.
!trace_evolve = .true.
! hydro debugging
!hydro_check_everything = .true.
!hydro_inspectB_flag = .true.
!hydro_sizequ_flag = .true.
! for making logs, uncomment hydro_dump_call_number plus the following
! to make residual logs, uncomment hydro_sizequ_flag
! to make correction logs, uncomment hydro_inspectB_flag
! to make jacobian logs, uncomment hydro_numerical_jacobian, hydro_save_numjac_plot_data
! to do dfridr test, uncomment hydro_get_a_numerical_partial, hydro_test_partials_k,
! hydro_numerical_jacobian, hydro_save_numjac_plot_data, hydro_dump_iter_number
!hydro_get_a_numerical_partial = 1d-7
!hydro_test_partials_k = 190
!hydro_numerical_jacobian = .true.
!hydro_save_numjac_plot_data = .true.
!hydro_dump_call_number = 1
!hydro_dump_iter_number = 6
!hydro_epsder_struct = 1d-6
!hydro_epsder_chem = 1d-6
!hydro_save_photo = .true. ! Saves a photo when hydro_call_number = hydro_dump_call_number -1
!fill_arrays_with_NaNs = .true.
!report_why_dt_limits = .true.
!report_all_dt_limits = .true.
!report_hydro_dt_info = .true.
!report_dX_nuc_drop_dt_limits = .true.
!report_bad_negative_xa = .true.
!show_mesh_changes = .true.
!mesh_dump_call_number = 95
!trace_newton_bcyclic_solve_input = .true. ! input is "B" j k iter B(j,k)
!trace_newton_bcyclic_solve_output = .true. ! output is "X" j k iter X(j,k)
!trace_newton_bcyclic_matrix_input = .true.
!trace_newton_bcyclic_matrix_output = .true.
!trace_newton_bcyclic_steplo = 1 ! 1st model number to trace
!trace_newton_bcyclic_stephi = 1 ! last model number to trace
!trace_newton_bcyclic_iterlo = 2 ! 1st newton iter to trace
!trace_newton_bcyclic_iterhi = 2 ! last newton iter to trace
!trace_newton_bcyclic_nzlo = 1 ! 1st cell to trace
!trace_newton_bcyclic_nzhi = 10000 ! last cell to trace; if < 0, then use nz as nzhi
!trace_newton_bcyclic_jlo = 1 ! 1st var to trace
!trace_newton_bcyclic_jhi = 100 ! last var to trace; if < 0, then use nvar as jhi
!trace_k = 0
/ ! end of controls namelist
&pgstar
!pause = .true.
pgstar_interval = 6
Grid2_win_flag = .true.
Grid2_title = '4.165 M\d\(2281)\u Z=0.007 Classical Cepheid'
History_Panels1_xaxis_name='star_age_day'
History_Panels1_max_width = 365 ! only used if > 0. causes xmin to move with xmax.
! Grid2_file_flag = .true.
file_digits = 7
Grid2_file_dir = 'png'
Grid2_file_prefix = 'grid'
Grid2_file_interval = 5 ! output when mod(model_number,Grid2_file_interval)==0
!Profile_Panels1_show_grid = .true.
Profile_Panels1_xaxis_name = 'logtau'
Profile_Panels1_xaxis_reversed = .true.
Profile_Panels1_xmin = -101D0
Profile_Panels1_xmax = -101D0
Profile_Panels1_dymin(4) = 0.02
Profile_Panels1_yaxis_name(2) = 'avg_charge_He'
/ ! end of pgstar namelist
Yet, when I try to run the shell script in bash, inlist_rsp_RR_Lyrae remains unchanged. Instead, this occurs:
Configuring inlist to save settings before full-amplitude pulsations...
Changing model parameters...
sed: no input files
/home/nick/mesa-r11701/star/test_suite/rsp_RR_Lyrae_grid/inlist_rsp_RR_Lyrae: line 1: long-period: command not found
/home/nick/mesa-r11701/star/test_suite/rsp_RR_Lyrae_grid/inlist_rsp_RR_Lyrae: line 3: original: command not found
/home/nick/mesa-r11701/star/test_suite/rsp_RR_Lyrae_grid/inlist_rsp_RR_Lyrae: line 5: syntax error near unexpected token `&'
/home/nick/mesa-r11701/star/test_suite/rsp_RR_Lyrae_grid/inlist_rsp_RR_Lyrae: line 5: `&star_job'
Compiling MESA...
As you can see, it is mistaking the first few lines of comments in inlist_rsp_RR_Lyrae as commands. Is there something I am missing here?

The problem is
sed -i \
...
-e "s/^\([[:blank:]]*photo_directory\).*/\1 = 'photos\/photos_${fields[0]}'/i" \
#-e "s/^\([[:blank:]]*RSP_alfa\).*/\1 = ${fields[6]}/i" \
"$mesa_inlist"
Here the comment before $mesa_inlist also comments out the \ at the end of the line. Therefore the command ends at that comment and sed -i complains that there is no file given. Afterwards bash tries to execute "$mesa_inlist" (that is inlist_rsp_RR_Lyrae) as a command, which fails.
To fix the problem, simply remove the comment line.
Bye the way: With this function you can drastically improve the readability of your script.
sedi() {
mapfile -t args < <(printf -- '-e\ns/^\([[:blank:]]*%s\).*/\\1 = %s/i\n' "$#")
sed -i "${args[#]}" "$mesa_inlist"
}
Now you can replace ...
sed -i \
-e "s/^\([[:blank:]]*photo_interval\).*/\1 = 1000/i" \
-e "s/^\([[:blank:]]*profile_interval\).*/\1 = 1/i" \
-e "s/^\([[:blank:]]*history_interval\).*/\1 = 1/i" \
-e "s/^\([[:blank:]]*terminal_interval\).*/\1 = 4000/i" \
-e "s/^\([[:blank:]]*max_num_profile_models\).*/\1 = -1/i" \
-e "s/^\([[:blank:]]*RSP_max_num_periods\).*/\1 = $max_num_periods_2/i" \
"$mesa_inlist"
... by ...
sedi \
photo_interval 1000 \
profile_interval 1 \
history_interval 1 \
terminal_interval 4000 \
max_num_profile_models -1 \
RSP_max_num_periods "$max_num_periods_2"
... and so on for the other sed commands.

Related

cat * mixes up the order of files

Right now, I am working on a "Text Editor" made with Bash. Everything was going perfectly until I tested it. When I opened the file the script created, everything was jumbled up. I eventually figured out it had something to do with the cat BASHTE/* >> $file I had put in. I still have no idea why this happens. My crappy original code is below:
#!/bin/bash
# ripoff vim
clear
echo "###############################################################################"
echo "# BASHTE TEXT EDITOR - \\\ = interupt :q = quit :w = write #"
echo "# :wq = Write and quit :q! = quit and discard :dd = Delete Previous line #"
echo "###############################################################################"
echo ""
read -p "Enter file name: " file
touch .$file
mkdir BASHTE
clear
echo "###############################################################################"
echo "# BASHTE TEXT EDITOR - \\\ = interupt :q = quit :w = write #"
echo "# :wq = Write and quit :q! = quit and discard :dd = Delete Previous line #"
echo "###############################################################################"
while true
do
read -p "$lines >" store
if [ "$store" = "\\:q" ]
then
break
elif [ "$store" = "\\:w" ]
then
cat BASHTE/* >> $file
elif [ "$store" = "\\:wq" ]
then
cat BASHTE/* >> $file
rm -rf .$file
break
elif [ "$store" = "\\:q!" ]
then
rm -rf BASHTE
rm -rf $file
break
elif [ "$store" = "\\:dd" ]
then
LinesMinusOne=$(expr $lines - 1)
rm -rf BASHTE/$LinesMinusOne.txt
else
echo $store >> BASHTE/$lines.txt
# counts the number of times the while loop is run
((lines++))
fi
done
This is what I got after I typed in the alphabet:
b
j
k
l
m
n
o
p
q
r
s
c
t
u
v
w
x
y
z
d
e
f
g
h
I
This was what I inputted
a
v
c
d
e
f
g
h
I
j
k
l
m
n
o
p
q
r
s
t
u
v
w
x
y
z
\\:wq
Any help would be great, Thanks
BASHTE/* is in lexical order, so every line starting with 1 will come before every line starting with 2 and so on. That means the order of your input lines is:
1 a
10 j
11 k
12 l
...and so on...
To make the lines sort well with the * operator, you'll need to name them with leading zeros, for example:
# ...
echo $store >> BASHTE/$(printf %020d $lines).txt
# ...
I chose the %020d format because it should store any number of lines applicable for a 64-bit system, since 2 ** 64 = 18446744073709551616, which is 20 digits long.

extracting vdate form nc header with bash without opening file

I have to change the title of a couple of hundred files by adding the vdate from its header to its title.
If vdate = 19971222, then I want the name of that nc file to become rerun4_spindown_19971222.nc
I know I can find the vdate by ncdump -h filename (see example header below).
ncdump -h rerun4_1997_spindown_09191414_co2
netcdf rerun4_1997_spindown_09191414_co2 {
dimensions:
lon = 768 ;
lat = 384 ;
nhgl = 192 ;
nlevp1 = 96 ;
spc = 32896 ;
// global attributes:
:file_type = "Restart history file" ;
:source_type = "IEEE" ;
:history = "" ;
:user = " Linda" ;
:created = " Date - 20190919 Time - 134447" ;
:label_1 = " Atmospheric model " ;
:label_2 = " Library 23-Feb-2012" ;
:label_3 = " Lin & Rood ADVECTION is default" ;
:label_4 = " Modified physics" ;
:label_5 = " Modified radiation" ;
:label_6 = " Date - 20190919 Time - 134447" ;
:label_7 = " Linda " ;
:label_8 = " Linux " ;
:fdate = 19950110 ;
:ftime = 0 ;
:vdate = 19971222 ;
:vtime = 235800 ;
:nstep = 776158 ;
:timestep = 120. ;
However, then I have to manually open all the files and manually change the title of the file... of hundreds of files. I would prefer making a bash that can automatically do that.
I am sure there must be a more intelligent way to extract the vdate from the nc header, could you guys help me out?
Thank you!
In theory, something like that should work:
#! /bin/sh
for file in rerun4_*_spindown_* ; do
vdate=$(ncdump -h $file | awk '$1 == ":vdate" { print $3 }')
new_name="rerun4_spindown_$vdate.nc"
mv "$file" "$new_name"
done
I do not have access to netCDF files - more testing is needed.

Neural-Style output flag

I'm messing around with jcjohnson/neural-style. I have it up and running just fine. However, I cannot for the life of me make the -output_image or -init flags work. My gut says it's some very simple misunderstanding on my part.
-output_image is driving me especially nuts because it should just be a string to use as the filename, the script even appends the file formate in function that takes -output_image as an argument. I've tried single, double, and no quotes, as well as changing the order of the flags. My gut says it's something trivially simple because nobody is complaining about this on-line anywhere,but I'm a little in over my head because I'm not especially proficient with either bash, torch, or lua.
I'm using this simple script I hacked together that works just fine if I leave off both of the troublesome flags
#!/bin/bash
echo "#### Starting Job B ####"
th neural_style.lua \
-gpu -1 \
-print_iter 1 \
-save_iter 10 \
-num_iterations 3000 \
-style_image a_style.jpg \
-content_image a_content.jpg \
-style_scale 0.15 \
-style_weight 25 \
-content_weight 1e3 \
-output_image a \
-init a_460.png \
The above throws "Invalid argument: "
if it helps, in the script 'neural_style.lua'
the flags are defined as
cmd:option('-init', 'random', 'random|image')
...
cmd:option('-output_image', 'out.png')
and used as
local function maybe_save(t)
local should_save = params.save_iter > 0 and t % params.save_iter == 0
should_save = should_save or t == params.num_iterations
if should_save then
local disp = deprocess(img:double())
disp = image.minmax{tensor=disp, min=0, max=1}
local filename = build_filename(params.output_image, t)
if t == params.num_iterations then
filename = params.output_image -- ## LOOK HERE ##
end
-- Maybe perform postprocessing for color-independent style transfer
if params.original_colors == 1 then
disp = original_colors(content_image, disp)
end
image.save(filename, disp) -- ## LOOK HERE ##
end
end
as well as
if params.init == 'random' then
img = torch.randn(content_image:size()):float():mul(0.001)
elseif params.init == 'image' then
img = content_image_caffe:clone():float()
else
error('Invalid init type') -- ## NOT THE ERROR THROWN #
end
if params.gpu >= 0 then

How to print a character using Shell script

Below is my script that acquire the MAC of the machine and store within a config file.
My problem is that within each line have the character " " and it dont print it inside the file, how can I write the file using " "
MAC=$(ifconfig eth0 | grep -o -E '([[:xdigit:]]{1,2}:){5}[[:xdigit:]]{1,2}')
echo "
view.sslVerificationMode = *3*
view.autoConnectDesktop = *TRUE*
view.autoConnectBroker = *TRUE*
view.kioskLogin = *TRUE*
view.nonInteractive = *TRUE*
view.fullScreen = *TRUE*
view.nomenubar = *TRUE*
view.defaultBroker = *viewcs*
view.defaultUser = *CM-${MAC//:/_}*
" > /etc/vmware/view-mandatory-config;
sed -i 's/*/"/g' /etc/vmware/view-mandatory-config
with cat and a here-doc you can use multi-line input without escaping charaters
MAC=$(ifconfig eth0 | grep -o -E '([[:xdigit:]]{1,2}:){5}[[:xdigit:]]{1,2}')
cat << EOT > /etc/vmware/view-mandatory-config
view.sslVerificationMode = "3"
view.autoConnectDesktop = "TRUE"
view.autoConnectBroker = "TRUE"
view.kioskLogin = "TRUE"
view.nonInteractive = "TRUE"
view.fullScreen = "TRUE"
view.nomenubar = "TRUE"
view.defaultBroker = "viewcs"
view.defaultUser = "CM-${MAC//:/_}"
EOT
cat will relay the stdin input generated by the here document to stdout, thus enabling redirection to a file
Simply escape double-quotes within the outer double-quotes by putting a backslash in front of them:
MAC=$(ifconfig eth0 | grep -o -E '([[:xdigit:]]{1,2}:){5}[[:xdigit:]]{1,2}')
echo "
view.sslVerificationMode = \"3\"
view.autoConnectDesktop = \"TRUE\"
view.autoConnectBroker = \"TRUE\"
view.kioskLogin = \"TRUE\"
view.nonInteractive = \"TRUE\"
view.fullScreen = \"TRUE\"
view.nomenubar = \"TRUE\"
view.defaultBroker = \"viewcs\"
view.defaultUser = \"CM-${MAC//:/_}\"
" > /etc/vmware/view-mandatory-config

How do I determine what shell a script was meant to use?

I have this non-functioning script, before I can attempt to fix it I need to know what kind of shell this was supposed to run in.
#
set pwrs = ( 0 0 0 0 0 0 0 0 )
# pwrs[1]=1
# next=2
while ( $next < 9 )
# last = $next - 1
# pwrs[$next] = $pwrs[$last] * 2
# next = $next + 1
end
#count = 1
while ( $count <= 8 )
echo $pwrs[$count]
# count = $count + 1
end
I was able to get this to produce appropriate output using csh. I would expect tcsh to work as well.
Turns out I do have a csh available. I pasted the code into the cmdline.
There was one error. All math calculations that begin with the # char must be separated from their following variable name, hence
#count = 1
Needed to be fixed as
# count = 1
$ set pwrs = ( 0 0 0 0 0 0 0 0 )
$ # pwrs[1]=1
$ # next=2
$ while ( $next < 9 )
while? # last = $next - 1
while? # pwrs[$next] = $pwrs[$last] * 2
while? # next = $next + 1
while? end
$ # count = 1
$ while ( $count <= 8 )
while? echo $pwrs[$count]
while? # count = $count + 1
while? end
output
1
2
4
8
16
32
64
128
[oracle#localhost ~]$
See Grymoire Unix - Csh for explanation of the csh #, and $pwrs[$next] (array notation). Your specific case is not addressed, but you should be able to work it out. If you have other questions after you build small test cases that don't work, post them with sample inputs, expected outputs, and your current code and output.
Also, don't spend any more time on csh, while not as bad as some portray it, you will ultimately discover that there are problems you can't solve in csh. All of it's nice cmd-line features are included in bash, so you might want to consider using it.Finally, see Grymoire csh top 10 for detailed reasons why you want to convert this code to a newer shell.
IHTH

Resources