Modify existing code for creating menu names in an interactive shell script - bash

When i give the command awk NR==7 ABC.mod it gives me the title ('Add') I need which is present on the 7th line of the file Currently I am just able to read the title but I am not aware of how to append it to the output. Can somebody help me organize this so I can modify the code to get the expected menu output with minimal disruption (I hope) to the existing script?

Assuming you can pull out the "Add", "Delete" ... and other "titles" from the 7th line of each *.mod file, then you need to modify your script where it looks at the file a1.out somewhere before the line which seems to create the menu, namely: tr ' ' '\n' < ~/a1.out > ~/b.dat.
I say "assuming" because, even though you mention awk NR==7, I don't see where you are using it in the script. In any case, if you can get the "title" from the 7th line of a given *.mod file, then you can get the menu "name" from the file name (which seems to be the way you are constructing your menu) like this:
awk '{ln=length(ARGV[1]); if(NR==7) print substr(ARGV[1],0,ln-4)"..."$0}' ABC.mod
outputs:
ABC...Add
There's may be a shorter, easier way to do this using sed, but you mentioned awk.
For me at least, there's not really enough information to go on to help you much further. If you update your question someone may be able to give more concrete advice.
EDIT:
I'll post my work here in the hope that you find it useful. It is not a complete solution. I have to say, this is a strangely written application - with shell code and variables hardwired to temporary data to locations strewn about the file system. It's a bit hairy to try and set up a local version to try it out. Hopefully by experimenting and making modifications to the cod e you will learn more about how your application works and about shell programming in general. Extra advice: record your changes; sketch out how/where your application reads and writes its data; use comments in the source code to help you and others remember how the code works; make backups; use source control.
My assumptions:
pradee.sh looks like this (why does the file has a .sh extension - it seems more like a it defines some constants for your script)
% cat pradee.sh
HBKTM
ABC
HBKTM
CBC
HBKTM
DBC
HBKTM
IBC
HBKTM
MBCE
HBKTM
UBC
HBKTM
VBCM
Here's how I created my "test environment":
% for file in `grep -v HBKTM pradee.sh`; do touch $file.mod ; done
% ls
ABC.mod CBC.mod DBC.mod IBC.mod MBCE.mod UBC.mod VBCM.mod pradee.sh
% echo -e "_ctrl.jsp \n\n\n\n\n" > *.mod # mod files have required text+6 lines
% echo -e "_ctrl.jsp \n\n\n\n\n" > HBKTM.mod # this file seems special ?
% sed -i'' -e "7i\\[Ctrl-V Ctrl-J]
Add" ABC.mod
OR since the files now have 6 lines ... echo the menu title onto the last line:
% echo "Delete" >> DBC.mod
% echo "Insert" >> IBC.mod
... [continue inserting titles like "Add" "Delete" etc to the other *.mod files]
After that I think I have data files that mimic your set up. You tell me. Now, if I make a few small changes to your script (so the file locations don't remove overwrite my own files) and add the awk command I mentioned previously, here is what I end up with:
# menu_create.sh
# See http://stackoverflow.com/questions/17297671
rm -f *.dat
clear
cont="y"
while [ "$cont" = "y" ] # "$" is need for POSIX
do
echo -e "\n\nPlease Enter ONS Name : "
read ons
currpath=.
up=$(echo $ons|tr '[:lower:]' '[:upper:]')
#echo "\n ONS menu name \n"
#echo $up
if [ -f $up.mod ]; then
#in=$(grep -ri $up pradee.sh) # changed to following
# - how could this have worked ?
in=$(grep -v $up pradee.sh)
if [ -n "$in" ]; then
onsname=$(grep -ri "_ctrl.jsp" $up.mod)
#echo "onsname : $onsname"
if [ -n "$onsname" ]; then
echo -e "\n ONS menu name : $up "
echo $in > a1.dat
#echo "written to a1.dat\n"
#cat ~/a1.dat
#tr ' ' '\n' < ~/a1.dat > ~/a.dat
#cat ~/a.dat
sed "s/$up//g" a1.dat >a1.out
for i in `cat a1.dat`;
do
awk '{ln=length(ARGV[1]);if(NR==7) print substr(ARGV[1],0,ln-4)"..."$0}' $i.mod >> menu.dat ;
done
echo -e "\n FINUX Names \n"
#tr ' ' '\n' < a1.out > b.dat
tr ' ' '\n' < menu.dat > b.dat
cat b.dat
else
echo -e "ONS Name Not Valid !"
fi
else
echo -e "FINUX menu Name not found in our Repository"
fi
else
echo -e "\n Please Enter valid ONS name !!"
fi
echo -e "\n\n Press "y" to continue, Any other key to exit"
read cont
done
It gives me this output:
Please Enter ONS Name :
hbktm
ONS menu name : HBKTM
FINUX Names
ABC...Add
CBC...Cancel
DBC...Delete
IBC...Insert
MBCE...Modify
UBC...Undelete
VBCM...Verify
Press y to continue, Any other key to exit
q
I hope my response to your question helps you learn more about how to modify your application.

Related

Making a script in debian which would create a new file from names file with different order of the names

Existing names in the "names" file is in form of lastname1,firstname1 ; lastname2,firstname2.
In the new file it should be like down below.
Create a script that outputs a list of existing users (from the "names" file) in the form:
firstname1.lastname1
firstname2.lastname2
etc.
And saves a file called "cat list"
This kind of command line should be a solution for you :
awk -F '\.' '{print $2","$1}' source_file >> "cat list"
First awk revers the order of the field and put the char ',' under
">>" Second step redirect full output to a file called "cat list" as requested
I don't think I have the most efficient solution here but it works and outputs the different stages of translation to help illustrate the process:
#!/bin/sh
echo "lastname1,firstname1 ; lastname2,firstname2" >testfile
echo "original file:"
cat testfile
echo "\n"
# first replace semi-colon with newline
tr ';' '\n' <testfile >testfile_n
echo "after first translation:"
cat testfile_n
echo "\n"
# also remove extra spaces
tr -d '[:blank:]' <testfile_n >testfile_n_s
echo "after second translation:"
cat testfile_n_s
echo "\n"
# now swap name order using sed and use periods instead of commas
sed -E 's/([a-zA-Z0-9]*),([a-zA-Z0-9]*)/\2\.\1/g' testfile_n_s >"cat list"
echo "after third iteration:"
cat "cat list"
echo "\n"
The script above will save a file called 'cat list' and output something similar to:
original file:
lastname1,firstname1 ; lastname2,firstname2
after first translation:
lastname1,firstname1
lastname2,firstname2
after second translation:
lastname1,firstname1
lastname2,firstname2
after third iteration:
firstname1.lastname1
firstname2.lastname2

Writing a script for large text file manipulation (iterative substitution of duplicated lines), weird bugs and very slow.

I am trying to write a script which takes a directory containing text files (384 of them) and modifies duplicate lines that have a specific format in order to make them not duplicates.
In particular, I have files in which some lines begin with the '#' character and contain the substring 0:0. A subset of these lines are duplicated one or more times. For those that are duplicated, I'd like to replace 0:0 with i:0 where i starts at 1 and is incremented.
So far I've written a bash script that finds duplicated lines beginning with '#', writes them to a file, then reads them back and uses sed in a while loop to search and replace the first occurrence of the line to be replaced. This is it below:
#!/bin/bash
fdir=$1"*"
#for each fastq file
for f in $fdir
do
(
#find duplicated read names and write to file $f.txt
sort $f | uniq -d | grep ^# > "$f".txt
#loop over each duplicated readname
while read in; do
rname=$in
i=1
#while this readname still exists in the file increment and replace
while grep -q "$rname" $f; do
replace=${rname/0:0/$i:0}
sed -i.bu "0,/$rname/s/$rname/$replace/" "$f"
let "i+=1"
done
done < "$f".txt
rm "$f".txt
rm "$f".bu
done
echo "done" >> progress.txt
)&
background=( $(jobs -p) )
if (( ${#background[#]} ==40)); then
wait -n
fi
done
The problem with it is that its impractically slow. I ran it on a 48 core computer for over 3 days and it hardly got through 30 files. It also seemed to have removed about 10 files and I'm not sure why.
My question is where are the bugs coming from and how can I do this more efficiently? I'm open to using other programming languages or changing my approach.
EDIT
Strangely the loop works fine on one file. Basically I ran
sort $f | uniq -d | grep ^# > "$f".txt
while read in; do
rname=$in
i=1
while grep -q "$rname" $f; do
replace=${rname/0:0/$i:0}
sed -i.bu "0,/$rname/s/$rname/$replace/" "$f"
let "i+=1"
done
done < "$f".txt
To give you an idea of what the files look like below are a few lines from one of them. The thing is that even though it works for the one file, it's slow. Like multiple hours for one file of 7.5 M. I'm wondering if there's a more practical approach.
With regard to the file deletions and other bugs I have no idea what was happening Maybe it was running into memory collisions or something when they were run in parallel?
Sample input:
#D00269:138:HJG2TADXX:2:1101:0:0 1:N:0:CCTAGAAT+ATTCCTCT
GATAAGGACGGCTGGTCCCTGTGGTACTCAGAGTATCGCTTCCCTGAAGA
+
CCCFFFFFHHFHHIIJJJJIIIJJIJIJIJJIIBFHIHIIJJJJJJIJIG
#D00269:138:HJG2TADXX:2:1101:0:0 1:N:0:CCTAGAAT+ATTCCTCT
CAAGTCGAACGGTAACAGGAAGAAGCTTGCTTCTTTGCTGACGAGTGGCG
Sample output:
#D00269:138:HJG2TADXX:2:1101:1:0 1:N:0:CCTAGAAT+ATTCCTCT
GATAAGGACGGCTGGTCCCTGTGGTACTCAGAGTATCGCTTCCCTGAAGA
+
CCCFFFFFHHFHHIIJJJJIIIJJIJIJIJJIIBFHIHIIJJJJJJIJIG
#D00269:138:HJG2TADXX:2:1101:2:0 1:N:0:CCTAGAAT+ATTCCTCT
CAAGTCGAACGGTAACAGGAAGAAGCTTGCTTCTTTGCTGACGAGTGGCG
Here's some code that produces the required output from your sample input.
Again, it is assumed that your input file is sorted by the first value (up to the first space character).
time awk '{
#dbg if (dbg) print "#dbg:prev=" prev
if (/^#/ && prev!=$1) {fixNum=0 ;if (dbg) print "prev!=$1=" prev "!=" $1}
if (/^#/ && (prev==$1 || NR==1) ) {
prev=$1
n=split($1,tmpArr,":") ; n++
#dbg if (dbg) print "tmpArr[6]="tmpArr[6] "\tfixNum="fixNum
fixNum++;tmpArr[6]=fixNum;
# magic to rebuild $1 here
for (i=1;i<n;i++) {
tmpFix ? tmpFix=tmpFix":"tmpArr[i]"" : tmpFix=tmpArr[i]
}
$1=tmpFix ; $0=$0
print $0
}
else { tmpFix=""; print $0 }
}' file > fixedFile
output
#D00269:138:HJG2TADXX:2:1101:1:0 1:N:0:CCTAGAAT+ATTCCTCT
GATAAGGACGGCTGGTCCCTGTGGTACTCAGAGTATCGCTTCCCTGAAGA
+
CCCFFFFFHHFHHIIJJJJIIIJJIJIJIJJIIBFHIHIIJJJJJJIJIG
#D00269:138:HJG2TADXX:2:1101:2:0 1:N:0:CCTAGAAT+ATTCCTCT
CAAGTCGAACGGTAACAGGAAGAAGCTTGCTTCTTTGCTGACGAGTGGCG
I've left a few of the #dbg:... statements in place (but they are now commented out) to show how you can run a small set of data as you have provided, and watch the values of variables change.
Assuming a non-csh, you should be able to copy/paste the code block into a terminal window cmd-line and replace file > fixFile at the end with your real file name and a new name for the fixed file. Recall that awk 'program' file > file (actually, any ...file>file) will truncate the existing file and then try to write, SO you can lose all the data of a file trying to use the same name.
There are probably some syntax improvements that will reduce the size of this code, and there might be 1 or 2 things that could be done that will make the code faster, but this should run very quickly. If not, please post the result of time command that should appear at the end of the run, i.e.
real 0m0.18s
user 0m0.03s
sys 0m0.06s
IHTH
#!/bin/bash
i=4
sort $1 | uniq -d | grep ^# > dups.txt
while read in; do
if [ $((i%4))=0 ] && grep -q "$in" dups.txt; then
x="$in"
x=${x/"0:0 "/$i":0 "}
echo "$x" >> $1"fixed.txt"
else
echo "$in" >> $1"fixed.txt"
fi
let "i+=1"
done < $1

Searching in bash shell

I have a text file.
Info in text file is
Book1:Author1:10.50:50:5
Book2:Author2:4.50:30:10
First one is book name, second is author name, third is the price, fourth is the quantity and fifth is the quantity sold.
Currently I have this set of codes
function search_book
{
read -p $'Title: ' Title
read -p $'Author: ' Author
if grep -Fq "${Title}:${Author}" BookDB.txt
then
record=grep -c "${Title}:${Author}" BookDB.txt
echo "Found '" $record "' record(s)"
else
echo "Book not found"
fi
}
for $record, I am trying the count the number of lines that is found. Did I do the right thing for it because when I run this code, it just shows error command -c.
When i did this
echo "Found"
grep -c "${Title}" BookDB.txt
echo "record(s)"
It worked, but the output is
Found
1
record(s)
I would like them to be together
Can I also add grep -i to grep -Fq in order to make all into small letters for better searching?
Lets say if I want to search Book1 and Author1, if I enter 'ok' for title and 'uth' for author, is there any % command to add to the title to search in the middle of the title and author?
The expected output is also expected to be..
Found 1 record(s)
Book1,Author1,$10.50,50,5.
Is there any where I can change the : delimiter to ,?
And also adding $ to the 3rd column which is the rice?
Please help..
Changing record=grep -c "${Title}:${Author}" BookDB.txt to record=$(grep -c "${Title}:${Author}" BookDB.txt) will fix the error. record=$(cmd) means assigning the output of command cmd to the variable record. Without that, shell will interpret record=grep -c ... as a command -c prepended by a environment variable setting(record=grep).
BTW, since your DB format is column-oriented text data, awk should be a better tool. Sample code:
function search_book
{
read -p $'Title: ' Title
read -p $'Author: ' Author
awk -F: '{if ($1 == "'"$Title"'" && $2 ~ "'"$Author"'") {count+=1; output=output "\n" $0} }
END {
if (count > 0) {print "found", count, "record(s)\n", output}
else {print "Book not found";}}' BookDB.txt
}
As you can see, using awk makes it easier to change delimiter(e.g. awk -F, for comma delimiter), and also makes the program more robust(e.g. it restricts the matching string to the first two fields). If you only need fuzzy match instead of exact match, you could change == to ~ in condition.
The "unnamed command -c" error can be avoided by enclosing the right part of the assignment in backticks or "$()", e.g.:
record=`grep -ic "${Title}:${Author}" BookDB.txt`
record=$(grep -ic "${Title}:${Author}" BookDB.txt)
Also, this snippet shows that -i is perfectly fine. However, please note that both grep commands should use the same list of flags (-F is missing in the 2nd one) - except for -q, of course.
Anyway, performing grep twice is probably not the best way to go. What about...
record=`grep -ic "${Title}:${Author}" BookDB.txt 2>/dev/null`
if [ ! -z "$record" ]; then ...
... or something like that?
By the way: If you omit -F you allow the user to operate with regular expressions. This would not only provide wildcards but also the possibility for more complex patterns. You could also apply an option to your script that decides whether to use -F or not..
Last but not least: To modify the lines, in order to change the delimiter or manipulate the columns at all, you could look into the manual pages or awk(1) or cut(1), at least. Although I believe that a more sophisticated language is more suitable here, e.g. perl(1) or python(1), especially when the script is to be extended with more features.
to add to the answer(s) above (this started as a comment, but it grew...) :
the $() form is preferred:
- it allows nesting,
- and it simplifies a lot the use of " and ' (each "level" of nesting see them at their level, so to speak). Tough to do with as using nested quotes and single-quotes becomes a nightmare of` and \\... depending on the "level of subshell" they are to be interpreted in...
ex: (trying to only grep once)
export results="$(grep -i "${Title}:${Author}" BookDB.txt)" ;
export nbresults=$(echo "${results}" | wc -l) ;
printf "Found %8s record(s)\n" "nbresults" ;
echo "$nbresults" ;
or, if too many results to fit in variable:
export tmpresults="/tmp/results.$$"
grep -i "${Title}:${Author}" BookDB.txt > "${tmpresults}"
export nbresults=$(wc -l "${tmpresults}") ;
printf "Found %8s record(s)\n" "nbresults" ;
cat "${tmpresults}" ;
rm -f "${tmpresults}" ;
Note: I use " a lot (except on the wc -l line) to illustrate it could be needed sometimes (not in all the cases above!) to keep spaces, newlines, etc. (And I purposely drop it for nbresults so that it only contain the number of lines, not the preceding spaces).

replace first line of files by another made by changing path from unix to windows

I am trying to do a bash script that:
loop over some files : OK
check if the first line matches this pattern (#!f:\test\python.exe) : OK
create a new path by changing the unix style to windows style : KO
Precisely,
From: \c\tata\development\tools\virtualenvs\test2\Scripts\python.exe
I want to get: c:\tata\development\tools\virtualenvs\test2\Scripts\python.exe
insert the new line by appending #! and the new path : KO
Follow is my script but I'm really stuck!
for f in $WORKON_HOME/$env_name/$VIRTUALENVWRAPPER_ENV_BIN_DIR/*.py
do
echo "----"
echo file=$f >&2
FIRSTLINE=`head -n 1 $f`
echo firstline=$FIRSTLINE >&2
unix_path=$WORKON_HOME/$env_name/$VIRTUALENVWRAPPER_ENV_BIN_DIR/python.exe
new_path=`echo $unix_path | awk '{gsub("/","\\\")}1'`
echo new_path=$new_path >&2
# I need to change the new_path by removing the first \ and adding : after the first letter => \c -> c:
new_line="#!"$new_path
echo new_line=$new_line >&2
case "$FIRSTLINE" in
\#!*python.exe* )
# Rewrite first line
sed -i '1s,.*,'"$new_line"',' $f
esac
done
Output:
file=/c/tata/development/tools/virtualenvs/test2/Scripts/pip-script.py
firstline=#!f:\test\python.exe
new_path=\c\tata\development\tools\virtualenvs\test2\Scripts\python.exe
new_line=#!\c\tata\development\tools\virtualenvs\test2\Scripts\python.exe
Line that is written in the file: (some weird characters are written I do not know why...)
#!tatadevelopment oolsirtualenvs est2Scriptspython.exe
Line I am expecting:
#!c:\tata\development\tools\virtualenvs\test2\Scripts\python.exe
sed is interpreting the backslashes and characters following them as escapes, so you're getting, e.g. tab. You need to escape the backslashes.
sed -i "1s,.*,${new_line//\\/\\\\}," "$f"

How do I edit the output of a bash script before executing it?

For example look at the following line of bash-code
eval `echo "ls *.jpg"`
It lists all jpgs in the current directory. Now I want it to just print the line to the prompt so I can edit it before executing. (Like key-up does for example)
How do I do that?
The reason for this question comes from a much more usefull alias:
alias ac="history 2 | sed -n '1 s/[ 0-9]*//p' >> ~/.commands; sort -fu ~/.commands > ~/.commandsTmp; mv ~/.commandsTmp ~/.commands"
alias sc='oldIFS=$IFS; IFS=$'\n'; text=(); while read line ; do text=( ${text[#]-} "${line}") ; done < ~/.commands; PS3="Choose command by number: " ; eval `select selection in ${text[#]}; do echo "$selection"; break; done`; IFS=$oldIFS'
alias rc='awk '"'"'{print NR,$0}'"'"' ~/.commands; read -p "Remove number: " number; sed "${number} d" ~/.commands > ~/.commandsTmp; mv ~/.commandsTmp ~/.commands'
Where ac adds or remembers the last typed command, sc shows the available commands and executes them and rc deletes or forgets a command. (You need to touch ~/.commands before it works)
It would be even more usefull if I could edit the output of sc before executing it.
history -s whatever you want
will append "whatever you want" to your bash history. Then a simple up arrow (or !! followed by enter if you have shopt histreedit enabled --- I think that's the option I'm thinking of, not 100% sure), will give you "whatever you want" on the command line, ready to be edited.
Some comments on your aliases:
Simplified quoting:
alias rc='awk "{print NR,\$0}" ~/.commands ...'
No need for tail and you can combine calls to sed:
alias ac="history 2 | sed -n '1 s/[ 0-9]*//p'..."
Simplified eval and no need for $IFS:
alias sc='text=(); while read line ; do text+=("${line}") ; done < ~/.commands; PS3="Choose command by number: " ; select selection in "${text[#]}"; do eval "$selection"; break; done'
#OP, you should really put those commands into subroutines, and when you want to use them, source it. (taken from dennis's answers)
rc(){
awk "{print NR,\$0}" ~/.commands ...
}
ac(){
history 2 | sed -n '1 s/[ 0-9]*//p'...
}
sc(){
text=()
while read line
do
text+=("${line}")
done < ~/.commands
PS3="Choose command by number: "
select selection in "${text[#]}"
do
eval "$selection"
break
done
}
then save it as "library.sh" or something and when you want to use it
$ source /path/to/library.sh
Or
$ . /path/to/library.sh
Maybe you could use preexec.bash?
http://www.twistedmatrix.com/users/glyph/preexec.bash.txt
(On a related note, you can edit the current command line by using ctrl-x-e as well!)
cheers,
tavod

Resources