There are variables that I need to change and run sequentially within certain codes.For example, the ionex_list variable looks like this and i need to change this in a loop. I have 3 variable files like this and 9 pieces of data in a row (ionex_list,receiver_ids,data_record).
ionex_list data_record
jplg0910.19i.Z ISTA00TUR_R_20190910000_01D_30S_MO.crx.gz
jplg0920.19i.Z ISTA00TUR_R_20190920000_01D_30S_MO.crx.gz
... ...
jplg0980.19i.Z ISTA00TUR_R_20190980000_01D_30S_MO.crx.gz
jplg0990.19i.Z ISTA00TUR_R_20190990000_01D_30S_MO.crx.gz
For example, I need to show ISTA00TUR_R_20190910000_01D_30S_MO.crx.gz as -dataFile in rnxEditGde.py, but I get the error "File: "/y2019/d091" does not exist" and I cannot show the path. these .gz files are a few directories under from the directory I'm working in. Since I couldn't pass the rnxEditGde.py part, I can't see if the codes below work properly.
dir=`pwd`
function pl {
sed -n "$1p" $2
}
for j in {091..099}; do
ids=$(pl $j receiver_ids)
ionex=$(pl $j ionex_list)
dataRecordFile=$(pl $j data_record)
gd2e.py -mkTreeS Trees
sed -i "s/jplg.*/**$ionex**/g" $dir/Trees/ppp_0.tree
rnxEditGde.py -dataFile "/y2019/d${j}/**$dataRecordFile**" -o dataRecordFile.Orig.gz
gd2e.py \
-treeSequenceDir Trees \
-drEditedFile** $dataRecordFile** \
-antexFile goa-var/etc/antennaCalsGNSS/igs14_2038.atx \
-GNSS https://sideshow.jpl.nasa.gov/pub/JPL_GNSS_Products/MultiGNSS_Rapid_Test\
-runType PPP \
-recList **$ids** \
-staDb /goa-var/sta_info/sta_db_qlflinn \
>gd2e.log 2>gd2e.err
done
Related
I have some -001.mkv files splitted in this way
the rings + of power-001.mkv
the rings + of power-002.mkv
..
nightmare.return-001.mkv
nightmare.return-002.mkv
..
I need to join -001.mkv files to obtain files like this
the rings + of power.mkv
nightmare.return.mkv
I thought of such a code, but doesn't work
for file in "./source/*001.mkv;" \
do \
echo mkvmerge --join \
--clusters-in-meta-seek -o "./joined/$(basename "$file")" "$file"; \
done
source is the source folder where -001.mkv files are located
joined is the target folder
Can you try this?
for f in ./source/*-001.mkv
do
p=${f%-001.mkv}
echo mkvmerge --clusters-in-meta-seek \
-o ./joined/"${p##*/}".mkv "$p"-*.mkv
done
Given your example, the for f in ./source/*-001.mkv should loop though nightmare.return-001.mkv and the rings + of power-001.mkv.
Let's assume that we're in the first iteration and $f expands to nightmare.return-001.mkv
${f%-001.mkv} right-side-strips -001.mkv from $f; that would give ./source/nightmare.return. We store this prefix in the variable p.
${p##*/} left-side-strips all characters up to the last / from $p; that would give nightmare.return.
So ... -o ./joined/"${p##*/}".mkv "$p"-*.mkv would be equivalent to ... -o ./joined/nightmare.return.mkv ./source/nightmare.return-*.mkv
PS: Once you checked that the displayed commands correspond to what you expect then you can remove the echo
So I have this function, I want this function to run everything that It contains in itself at the same time. So far it isn't working, and according to other sources, this is how you do it. The function itself works if its not in parallel.
#!/bin/bash
foo () {
cd ${HOME}/sh/path/to/script/execute
for f in *.sh; do #goes to "execute" directory and executes all
#scripts the current directory "execute" basically run-parts without cron
cd ~/sh/path/to/script
while IFS= read -r l1 #Line 1 in master.txt
IFS= read -r l2 #Line 2 in master.txt
IFS= read -r l3 #Line 3 in master.txt
do
cd /dev/shm/arb
echo ${l1} > arg.txt & echo ${l2} > arg2.txt & echo ${l3} > arg3.txt
cd ${HOME}/sh/path/to/script/execute
bash -H ${f} #executes all scripts inside "execute" folder
cd ~/sh/path/to/script/here
./here.sh &
cd ~/sh/path/to/script &
done <master.txt
done
}
export -f foo
parallel ::: foo
Results in
#No result at all....., just buffers. htop doesn't acknowledge any
#processes, and when this runs its pretty taxing on the cores.
master.txt content
In case this is relevant:
apple_fruit
apple_veggie
veggie_fruit
#apple changes
pear_fruit
pear_veggie
veggie_fruit
#pear changes
cucumber_fruit
...
I'm very new to using parallel, and don't know how it works in advanced(and basic) situations so would the loops interfere? And if it does interfere, is there a workaround?
The result is probably going to be something like:
inner() {
script="$1"
parallel -N3 "'$script' {}; here.sh {}" :::: master.txt
}
export -f inner
parallel inner ::: ${HOME}/sh/path/to/script/execute/*.sh
This will call each of the scripts in ${HOME}/sh/path/to/script/execute/ (and here.sh) with 3 arguments from master.txt like this:
${HOME}/sh/path/to/script/execute/script1.sh apple_fruit apple_veggie veggie_fruit
You need to change the scripts so that:
They get the arguments from the command line (not from arg.txt, arg2.txt, arg3.txt).
They send their output to stdout
I have written a program (Cifti_subject_fmri) which compares whether file name matches in two folders and essentially executes a set of instructions
#!/bin/bash -- fix_mni_paths
source activate ciftify_v1.0.0
export SUBJECTS_DIR=/scratch/m/mchakrav/dev/functional_data
export HCP_DATA=/scratch/m/mchakrav/dev/tCDS_ciftify
## make the $SUBJECTS_DIR if it does not already exist
mkdir -p ${HCP_DATA}
SUBJECTS=`cd $SUBJECTS_DIR; ls -1d *` ## list of my subjects
HCP=`cd $HCP_DATA; ls -1d *` ## List of HCP Subjects
cd $HCP_DATA
## submit the files to the queue
for i in $SUBJECTS;do
for j in $HCP ; do
if [[ $i == $j ]];then
parallel "echo ciftify_subject_fmri $i/filtered_func_data.nii.gz $j fMRI " ::: $SUBJECTS |qbatch --walltime '05:00:00' --ppj 8 -c 4 -j 4 -N ciftify_subject_fmri -
fi
done
done
When i run this code in the cluster i am getting an error which says
./Cifti_subject_fmri: [[AS1: command not found
The query ciftify_subject_fmri is part of toolbox ciftify, for it to execute it requires following instructions
ciftify_subject_fmri <func.nii.gz> <Subject> <NameOffMRI>
I have 33 subjects [AS1 -AS33] each with its own func.nii.gz files located SUBJECTS directory,the results need to be populated in HCP directory, fMRI is name of file format .
Could some one kindly let me know why i am getting an error in loop
Sorry, I'm from Brazil and my english is not fluent.
I wanna concatenate 20 files using a shellscript through cat command. However when I run it from a file, all content of files are showed on the screen.
When I run it directly from terminal, works perfectly.
That's my code above:
#!/usr/bin/ksh
set -x -a
. /PROD/INCLUDE/include.prod
DATE=`date +'%Y%m%d%H%M%S'`
FINAL_NAME=$1
# check if all paremeters are passed
if [ -z $FINAL_NAME ]; then
echo "Please pass the final name as parameter"
exit 1
fi
# concatenate files
cat $DIRFILE/AI6LM760_AI6_CF2_SLOTP01* $DIRFILE/AI6LM761_AI6_CF2_SLOTP02* $DIRFILE/AI6LM763_AI6_CF2_SLOTP04* \
$DIRFILE/AI6LM764_AI6_CF2_SLOTP05* $DIRFILE/AI6LM765_AI6_CF2_SLOTP06* $DIRFILE/AI6LM766_AI6_CF2_SLOTP07* \
$DIRFILE/AI6LM767_AI6_CF2_SLOTP08* $DIRFILE/AI6LM768_AI6_CF2_SLOTP09* $DIRFILE/AI6LM769_AI6_CF2_SLOTP10* \
$DIRFILE/AI6LM770_AI6_CF2_SLOTP11* $DIRFILE/AI6LM771_AI6_CF2_SLOTP12* $DIRFILE/AI6LM772_AI6_CF2_SLOTP13* \
$DIRFILE/AI6LM773_AI6_CF2_SLOTP14* $DIRFILE/AI6LM774_AI6_CF2_SLOTP15* $DIRFILE/AI6LM775_AI6_CF2_SLOTP16* \
$DIRFILE/AI6LM776_AI6_CF2_SLOTP17* $DIRFILE/AI6LM777_AI6_CF2_SLOTP18* $DIRFILE/AI6LM778_AI6_CF2_SLOTP19* \
$DIRFILE/AI6LM779_AI6_CF2_SLOTP20* > CF2_FINAL_TEMP
mv $DIRFILE/CF2_FINAL_TEMP $DIRFILE/$FINAL_NAME
I solved the problem putting the cat block inside a function, and redirecting stdout to the final file.
Ex:
concatenate()
I have a makefile where I want to read file name from input and then make other names based on it`s name. I have tried the following code
mlext = .ml
testext = test.ml
nativeext = test.native
test:
#read -p "Enter file name: " file; \
echo $$file$(mlext); \
echo $$file$(testext); \
echo $$file$(nativeext)
for example:
If i type: foo
then I want to get foo.ml, footest.ml, footest.native
however, I can only get foo.ml. For the rest two i only get .ml and .native
How can i fix this?
First, let us see what is the exact recipe given to the shell by removing the # in your Makefile:
read -p "Enter file name: " file; \
echo $file.ml; \
echo $filetest.ml; \
echo $filetest.native;
The issue is thus that the content of $(testext) gets appended to $$file, creating the shell variable $filetest, which (very probably) does not exist, resulting in an empty string in the end. This does not occur with $(mlext), as the initial dot cannot be part of a variable name.
To overcome this, use $${file} instead of $$file in your Makefile rule.