I am trying to store the output of total count to a variable and then print the variable.
for i in `cat /home/retailer.sh`;do file = "$(grep $i /home/DD.inventory.loaded_master_list.txt|sort -u|wc -l)";echo $file;done
/home/retailer.sh
HAG
GAT
/home/DD.inventory.loaded_master_list.txt
/dailydata_hershey/prd/work/all/2109/delivery/POSDATA_FINAL.GAT.20200201.dat
/dailydata_hershey/prd/work/all/2109/delivery/POSDATA_FINAL.HAG.20200201.dat
Below error i get
=: cannot open = (No such file or directory)
file = "$(...)" - you have to use file="$(...)" (no spaces).
Otherwise it will be interpreted as a command. (e.g. file with an argument of =)
For future reference, shellcheck.net is a great resource for catching these kinds of things. Just copy-paste your script into the box and it will tell you all of the errors, how to fix them, and a handful of other warnings about potential bugs.
Related
I need to rename the files inside the folder that has a space in it eg(Deco/main library/file1.txt )
code:
while IFS="," read orig new pat
do
mv -v $pat$new $pat$orig
done < new.csv
csv file:
newname,file1.txt,Deco/main\\\ library/
error:
mv: invalid option -- '\'
Welcome to Stackoverflow!
First: Use quotes around the use of variables. That means except in very rare occasions, you always should use "$foo" instead of $foo because if you are using the latter, the shell is supposed (and will) interpret spaces in the variables as word delimiters which you rarely want. Especially in your case you do not want it.
Second: Your CSV file seems to contain backslashes to quote the spaces. And some additional step seems to have added another level of quotation so than now you end up with three backslashes and a space for each original space. If this really is the case (please double check if what you wrote in your question is correct, otherwise my answer doesn't fit), you need to unquote this before you can use it.
There are security issues involved in using eval, so do not use it lightly (this disclaimer is necessary whenever proposing to use eval), but if you have trust in the input you are handling to not contain any nastinesses, then you can do this using this code:
while IFS="," read orig new pat
do
eval eval mv -v "$pat$new" "$pat$orig"
done < new.csv
Using this, two levels of quotation are evaluated (that's what eval does) before the mv command is executed.
I strongly suggest to do a dry run by adding echo before the mv first. Then instead of executing your commands they are merely printed first.
I have a folder with several files named : something_1001.txt; something_1002.txt; something_1003.txt; etc.
Inside the files there is some text. Of course each file has a different text but the structure is always the same: some lines identified with the string ">TEXT", which are the ones I am interested in.
So my goal is :
for each file in the folder, read the file's name and extract the number between "_" and ".txt"
modify all the lines in this particular file that contain the string ">TEXT" in order to make it ">{NUMBER}_TEXT"
For example : file "something_1001.txt"; change all the lines containing ">TEXT" by ">1001_TEXT"; move on to file "something_1002.txt" change all the lines containing ">TEXT" by ">1002_TEXT"; etc.
Here is the code I wrote so far :
for i in /folder/*.txt
NAME=`echo $i | grep -oP '(?<=something_/).*(?=\.txt)'`
do
sed -i -e 's/>TEXT/>${NAME}_TEXT/g' /folder/something_${NAME}.txt
done
I created a small bash script to run the code but it's not working. There seems to be syntax errors and a loop error, but I can't figure out where.
Any help would be most welcome !
There are two problems here. One is that your loop syntax is wrong; the other is that you are using single quotes around the sed script, which prevents the shell from interpolating your variable.
The grep can be avoided, anyway; the shell has good built-in facilities for extracting the base name of a file.
for i in /folder/*.txt
do
base=${i#/folder/something_}
sed -i -e "s/>TEXT/>${base%.txt}_TEXT/" "$i"
done
The shell's ${var#prefix} and ${var%suffix} variable manipulation facility produces the value of $var with the prefix and suffix trimmed off, respectively.
As an aside, avoid uppercase variable names, because those are reserved for system use, and take care to double-quote any variable whose contents may include shell metacharacters.
Two questions about problems I'm having writing up a BASH script that uses variables. I cannot for the life of me figure this out and it is KILLING me.
1) I have the following code.
pdir=/media/The_Enforcer/ICA_Doug/Participants/RS1
cd ${pdir}
for subject in * ; do
subdir=${pdir}/${subject} ;
cd ${subdir} ;
subj= echo ${subject} | head -c-9
3dAFNItoNIFTI -prefix ICA/cleanRS_NII_${subj} RSFC_LFF_rall_${subj}+orig ;
cd ${pdir} ;
done
${subject} is a subject ID which is ########.results. For example: 1R101U1A.results. Basically my code cd's me into that directory in which is a file called RSFC_LFF_rall_1R101U1A+orig which I want to process via the code line that starts with 3dAFNItoNIFTI. Obviously I can't use ${subject} variable in that code line because it would attempt to find the file RSFC_LFF_rall_1R101U1A.results+orig which does not exist. So to fix this I made a new variable called ${subj} which, via the echo pipeline, basically cuts off the last 9 letters of ${subject} which, in effect, removes the .results. When I do this and echo ${subj} it gives me 1R101U1A which is exactly what I want.
However, the line of code that starts with 3dAFNItoNIFTI errors with the following:
FATAL ERROR: Can't open dataset 'RSFC_LFF_rall_+orig'
I have tried declaring the ${subj} variable like, five different ways (including using head, tail, cut, and colons) and I still get this error.
What am I doing wrong?
2) In attempting to define ${subj} in numerous ways I also tried this method:
${subj}= ${subject:0:8}
When I did this, the final bracket refused to close - i.e. the closing bracket did not turn the color of the opening bracket and when I attempted to run the script I got an error at that line saying 'command not found.' I checked my syntax against the examples I was following and it looks fine? Am I missing something here?
Try this:
cd "/media/The_Enforcer/ICA_Doug/Participants/RS1"
for subject in * ; do
cd $subject
subj=${subject%.results}
3dAFNItoNIFTI -prefix ICA/cleanRS_NII_${subj} RSFC_LFF_rall_${subj}+orig
cd ..
done
${subject%.results} removes .results from the end of the string.
See Shell-Parameter-Expansion.
I am trying to pass a file into a program for data processing with bash and I am wondering if I have the correct syntax
/home/mumps/CS3150/Script/HW1/textfiles/CardioAndPulmonary.txt | /home/mumps/Medline2012/getDocs.mps > /home/mumps/CS3150/Scripts/HW1/textfiles/Titles.txt
The text files I am sending in are all valid and correctly formatted, but am just getting back a file error from the getDocs.mps (I should note that getDocs does work properly because it was something that my teacher passed out along with the debian vdi and other people aren't having a issue with it.)
getDocs does however call a text file that is located in Medline2012 as well which is where the error is coming from I believe.
Or just use bash redirection throughout without cat.
/home/mumps/Medline2012/getDocs.mps < /home/mumps/CS3150/Script/HW1/textfiles/CardioAndPulmonary.txt > /home/mumps/CS3150/Scripts/HW1/textfiles/Titles.txt
/home/mumps/Medline2012/getDocs.mps < /home/mumps/CS3150/Script/HW1/textfiles/CardioAndPulmonary.txt > /home/mumps/CS3150/Scripts/HW1/textfiles/Titles.txt
or
~/Medline2012/getDocs.mps < ~/CS3150/Script/HW1/textfiles/CardioAndPulmonary.txt > ~/CS3150/Scripts/HW1/textfiles/Titles.txt
or even
< ~/CS3150/Script/HW1/textfiles/CardioAndPulmonary.txt ~/Medline2012/getDocs.mps > ~/CS3150/Scripts/HW1/textfiles/Titles.txt
You either need to cat your .txt file, to pass the contents of it to the script via the pipe,
cat /home/mumps/CS3150/Script/HW1/textfiles/CardioAndPulmonary.txt | /home/mumps/Medline2012/getDocs.mps > output
or, depending on what's in the script, it might need to go as a command line parameter, i.e.
/home/mumps/Medline2012/getDocs.mps /home/mumps/CS3150/Script/HW1/textfiles/CardioAndPulmonary.txt > output
You are trying to execute your data file and feed the results to your script.
try
cat /home/mumps/CS3150/Script/HW1/textfiles/CardioAndPulmonary.txt | /home/mumps/Medline2012/getDocs.mps > /home/mumps/CS3150/Scripts/HW1/textfiles/Titles.txt
If you are still having trouble do a cd to the Medline2012 before you execute getDocs.mps. The reason is because when you access the getDoc.mps it calls to open the osu.medline database. This will cause a "file error" because the call in getDoc.mps does not include the path to osu.medline.
EDIT: A lot of people are telling you that you need to "cat" which is wrong. getDoc.mps has its own printing. If it didn't it wouldn't be printing "file error" for you. I also saw that you said that it is breaking after the loop. Did you test to make sure it isn't at the opening of the file. You can check by adding and indicating word in between the quotes in the first printing of "file error". You could change it to something like "file error 1." I realize you probably know that I just like to be thorough.
I have to ls command to get the details of certain types of files. The file name has a specific format. The first two words followed by the date on which the file was generated
e.g.:
Report_execution_032916.pdf
Report_execution_033016.pdf
Word summary can also come in place of report.
e.g.:
Summary_execution_032916.pdf
Hence in my shell script I put these line of codes
DATE=`date +%m%d%y`
Model=Report
file=`ls ${Model}_execution_*${DATE}_*.pdf`
But the value of Model always gets resolved to 'REPORT' and hence I get:
ls: cannot access REPORT_execution_*032916_*.pdf: No such file or directory
I am stuck at how the resolution of Model is happening here.
I can't reproduce the exact code here. Hence I have changed some variable names. Initially I had used the variable name type instead of Model. But Model is the on which I use in my actual code
You've changed your script to use Model=Report and ${Model} and you've said you have typeset -u Model in your script. The -u option to the typeset command (instead of declare — they're synonyms) means "convert the strings assigned to all upper-case".
-u When the variable is assigned a value, all lower-case characters are converted to upper-case. The lower-case attribute is disabled.
That explains the upper-case REPORT in the variable expansion. You can demonstrate by writing:
Model=Report
echo "Model=[${Model}]"
It would echo Model=[REPORT] because of the typeset -u Model.
Don't use the -u option if you don't want it.
You should probably fix your glob expression too:
file=$(ls ${Model}_execution_*${DATE}*.pdf)
Using $(…) instead of backticks is generally a good idea.
And, as a general point, learn how to Debug a Bash Script and always provide an MCVE (How to create a Minimal, Complete, and Verifiable Example?) so that we can see what your problem is more easily.
Some things to look at:
type is usually a reserved word, though it won't break your script, I suggest you to change that variable name to something else.
You are missing an $ before {DATE}, and you have an extra _ after it. If the date is the last part of the name, then there's no point in having an * at the end either. The file definition should be:
file=`ls ${type}_execution_*${DATE}.pdf`
Try debugging your code by parts: instead of doing an ls, do an echo of each variable, see what comes out, and trace the problem back to its origin.
As #DevSolar pointed out you may have problems parsing the output of ls.
As a workaround
ls | grep `date +%m%d%y` | grep "_execution_" | grep -E 'Report|Summary'
filters the ls output afterwards.
touch 'Summary_execution_032916.pdf'
DATE=`date +%m%d%y`
Model=Summary
file=`ls ${Model}_execution_*${DATE}*.pdf`
worked just fine on
GNU bash, version 4.3.11(1)-release (x86_64-pc-linux-gnu)
Part of question:
But the value of Model always gets resolved to 'REPORT' and hence I get:
This is due to the fact that in your script you have exported Model=Report
Part of question:
ls: cannot access REPORT_execution_*032916_*.pdf: No such file or directory
No such file our directory issue is due to the additional "_" and additional "*"s that you have put in your 3rd line.
Remove it and the error will be gone. Though, Model will still resolve to Report
Original 3rd line :
file=`ls ${Model}_execution_*${DATE}_*.pdf`
Change it to
file=`ls ${Model}_execution_${DATE}.pdf`
Above change will resolve the could not found issue.
Part of question
I am stuck at how the resolution of Model is happening here.
I am not sure what you are trying to achieve, but if you are trying to populate the file parameter with file name with anything_exection_someDate.pdf, then you can write your script as
DATE=`date +%m%d%y`
file=`ls *_execution_${DATE}.pdf`
If you echo the value of file you will get
Report_execution_032916.pdf Summary_execution_032916.pdf
as the answer
There were some other scripts which were invoked before the control reaches the line of codes which I mentioned in the question. In one such script there is a code
typeset -u Model
This sets the value of the variable model always to uppercase which was the reason this error was thrown
ls: cannot access REPORT_execution_032916_.pdf: No such file or directory
I am sorry that
i couldn't provide a minimal,complete and verifiable code