I have written a shell script for automating some tasks that I run from the terminal as -
v#ubuntu:$ ./automate.sh from:a1 to:a2 msg:'edited'
How can I (if at all) customize the script so as to enter each argument in a custom format on a separate line and execute it by pressing some other key to execute the shell script? So, I would do -
v#ubuntu:$ ./automate.sh
from : a1
to : a2
msg : 'next change'
... and then hit say Ctrl+Enter or F5 to execute this particular script?
NOTE : I know there is a hacky work around by simply typing ./automate.sh \ and hitting Enter after the trailing backslash to get a new line, but I was hoping to find a more elegant way to do this from within the script itself.
Also, I've purposely changed each argument to include whitespaces and the msg argument to include a string with spaces. So if anyone can point me in the right direction as to how to accomplish that as an added bonus, I'll be really grateful :)
If you know the number of arguments it is easy. Basics first.
#!/bin/bash
if [ $# == 0 ]
then
read v1 # gets onto new line. reads the whole line until ENTER
read v2 # same
read v3 # same
fi
# Parse $v1, $v2, $v3 as needed and run your script
echo ""
echo "Got |$v1|, |$v2|, |$v3|"
When you type automate.sh and hit enter the script is started, having received no arguments. With no arguments ($# == 0) the first read is executed, which prints a new line, waits, and gets the line typed in (once enter is hit) into $v1. The control goes back to the script, the next read gets the next typed line ... after the last one it drops out of if-else and continues. Parse your variables and run the script.
Session:
> automate.sh Enter
typed line Enter
more items Enter
yet more Enter
Got |typed line| |more items| |yet more|
>
You don't need Control-Enter or F5, it continues after 3 (three) lines.
This also allows you to provide both behaviors. Add an else, which will be executed if there are some arguments. You can then use the script by either supplying arguments on the first line (invocation you have so far), or in this new way.
If you need an unspecified number of arguments this approach will need more work.
Read words in input line into variables
If read is followed by variable names, like read v1 v2, then it reads each word into a variable, and the last variable gets everything that may have remained on the line. So replace read lines with
read k1 p1 s1
read k2 p2 s2
read k3 p3 s3
Now $k1 contains the first word (from), and $k2 and $k3 have the first words on their lines; then $p1 (etc) have the second word (:), and $s1 (etc) have everything else to the end of their lines (a1, a2, 'next change'). So you don't need those single quotes. All this is simple to modify if you want the script to print something on each line before input.
Based on the clarification in the comment, it is indeed desirable to not have to enter the whole strings, as one might think. This is "simple to modify"
read -p 'from :' s1
read -p 'to :' s2
read -p 'msg :' s3
Now the user only needs to enter the part after :, captured in $s variables. All else is the same.
See, for example: The section on user input in the Bash Guide; Their Advanced Guide (special variables); For a far more involved user interaction, this post. And, of course, man read.
It will be hard to bind Control-Enter in bash. If you are ok to change it for Control-D then everything might look like:
#!/usr/local/bin/bash
read -p 'From: ' from
read -p 'To: ' to
read -p 'Msg: ' msg
read keystroke
if [ "$keystroke" == "^D" ]; then
echo "$from $to $msg"
# do something else
fi
Related
this is my first stackoverflow question, regarding bash scripting. I am a beginner in this language, so be kind with me.
I am trying to write a comparison script. I tried to store all the outputs into variables, but only the last one is stored.
Example code:
me:1234567
you:2345678
us:3456789
My code:
#!bin/bash
while read -r forName forNumber
do
aName="$forName"
echo "$aName"
aNumber="$forNumber"
echo "$aNumber"
done < "exampleCodeFile.txt"
echo "$aNumber"
For the first time, everything will be printed out fine. However, the second echo will only print out "3456789", but not all the numbers again. Same with $aName. This is a problem because i have another file, which i stored a bunch of numbers to compare $aNumber with, using the same method listed above, called $aMatcher, consisting:
aMatcher:
1234567
2345678
3456789
So if i tried to run a comparison:
if [ "$aNumber" == "$aMatcher" ]; then
echo "match found!"
fi
Expected output (with bash -x "scriptname"):
'['1234567 == 1234567']'
echo "match found!"
Actual output (with bash -x "scriptname"):
'['3456789 == 3456789']'
echo "match found!"
Of course my end product would wish to list out all the matches, but i wish to solve my current issue before attempting anything else. Thanks!
When you run your following code
aNumber="$forNumber"
You are over-writing the variable $aNumber for every line of the file exampleCodeFile.txt rather than appending.
If you really want the values to be appended, change the above line to
aNumber="$aNumber $forNumber"
And while matching with $aMatcher, you again have to use a for/while loop to iterate through every value in $aNumber and $aMatcher.
I'm new on bash and I'm trying to write a bash script that will save user's multilines inputs (a text with newlines, somes lines of code, etc.). I need to allow newline (when you press "Enter"), multiline paste (when you paste few lines "Ctrl+V") and set a new key, instead of "Enter", to validate, send the input and continue to the next step of the script.
I tried with read but you can not do multiline.
echo "Enter content :"
read content
I found an example with readarray here (How to delete a character in bash read -d multiline input?) that allow to press "Enter" for newline but each words separate by space are separate in the array. I would like to have only the lines separated.
echo "Enter package names, one per line: hit Ctrl-D on a blank line to stop"
readarray -t pkgs
Do you have any ideas ? Or there is maybe a completely different way to do it ? Thank you for your help.
You can set IFS to newline so that only newlines will separate items in the array.
IFS=$'\n' readarray lines
The first line read will be ${lines[0]}, the second ${lines[1]}, etc. ${#lines[#]} tells you how many lines, and the last one will be ${lines[${#lines[#]}-1]}.
To loop over the array, you should use "${lines[#]}", not ${lines[*]}; the latter will take you right back to looping over individual words.
I would like to write a script to execute the steps outlined below. If someone can provide simple examples on how to modify files and search through folders using a script (not necessarily solving my problem below), I will greatly appreciate it.
submit job MyJob in currentDirectory using myJobShellFile.sh to a queue
upon completion of MyJob, goto to currentDirectory/myJobDataFolder.
In myJobDataFolder, there are folders
myJobData.0000 myJobData.0001 myJobData.0002 myJobData.0003
I want to find the maximum number maxIteration of all the listed folders. Here it would be maxIteration=0003.\
In file myJobShellFile.sh, at the last line says
mpiexec ./main input myJobDataFolder
I want to append this line to
'mpiexec ./main input myJobDataFolder 0003'
I want to submit MyJob to the que while maxIteration < 10
Upon completion of MyJob, find the new maxIteration and change this number in myJobShellFile.sh and goto step 4.
I think people write python scripts typically to do this stuff, but am having a hard time finding out how. I probably don't know the correct terminology for this procedure. I am also aware that the script will vary slightly depending on the queing system, but any help will be greatly appreciated.
Quite a few aspects of your question are unclear, such as the meaning of “submit job MyJob in currentDirectory using myJobShellFile.sh to a que”, “append this line to
'mpiexec ./main input myJobDataFolder 0003'”, how you detect when a job is done, relevant parts of myJobShellFile.sh, and some other details. If you can list the specific shell commands you use in each iteration of job submission, then you can post a better question, with a bash tag instead of python.
In the following script, I put a ### at the end of any line where I am guessing what you are talking about. Lines ending with ### may be irrelevant to whatever you actually do, or may be pseudocode. Anyway, the general idea is that the script is supposed to do the things you listed in your items 1 to 5. This script assumes that you have modified myJobShellFile.sh to say
mpiexec ./main input $1 $2
instead of
mpiexec ./main input
because it is simpler to use parameters to modify what you tell mpiexec than it is to keep modifying a shell script. Also, it seems to me you would want to increment maxIter before submitting next job, instead of after. If so, remove the # from the t=$((1$maxIter+1)); maxIter=${t#1} line. Note, see the “Parameter Expansion” section of man bash re expansion of the ${var#txt} form, and the “Arithmetic Expansion” section re $((expression)) form. The 1$maxIter and similar forms are used to change text like 0018 (which is not a valid bash number because 8 is not an octal digit) to 10018.
#!/bin/sh
./myJobShellFile.sh MyJob ###
maxIter=0
while true; do
waitforjobcompletion ###
cd ./myJobDataFolder
maxFile= $(ls myJobData* | tail -1)
maxIter= ${maxFile#myJobData.} #Get max extension
# If you want to increment maxIter, uncomment next line
# t=$((1$maxIter+1)); maxIter=${t#1}
cd ..
if [[ 1$maxIter -lt 11000 ]] ; then
./myJobShellFile.sh MyJobDataFolder $maxIter
else
break
fi
done
Notes: (1) To test with smaller runs than 1000 submissions, replace 11000 by 10000+n; for example, to do 123 runs, replace it with 10123. (2) In writing the above script, I assumed that not-previously-known numbers of output files appear in the output directory from time to time. If instead exactly one output file appears per run, and you just want to do one run per value for the values 0000, 0001, 0002, 0999, 1000, then use a script like the following. (For testing with a smaller number than 1000, replace 1000 with (eg) 0020. The leading zeroes in these numbers tell bash to fill the generated numbers with leading zeroes.)
#!/bin/sh
for iter in {0000..1000}; do
./myJobShellFile.sh MyJobDataFolder $iter
waitforjobcompletion ###
done
(3) If the system has a command that sleeps while it waits for a job to complete on the supercomputing resource, it is reasonable to use that command in place of waitforjobcompletion in the above scripts. Otherwise, if the system has a command jobisrunning that returns true if a job is still running, replace waitforjobcompletion with something like the following:
while jobisrunning ; do sleep 15; done
This will run the jobisrunning command; if it returns true, the shell will sleep for 15 seconds and then retest. Here is an example that illustrates waiting for a file to appear and then for it to go away:
while [ ! -f abc ]; do sleep 3; echo no abc; done
while ls abc >/dev/null 2>&1; do sleep 3; echo an abc; done
The second line's test could be [ -f abc ] instead; I showed a longer example to illustrate how to suppress output and error messages by routing them to /dev/null. (4) To reverse the sense of a while statement's test, replace the word while with until. For example, while [ ! -f abc ]; ... is equivalent to until [ -f abc ]; ....
I am trying to read part of a file and stop and a particular line, using bash. I am not very familiar with bash, but I've been reading the manual and various references, and I don't understand why something like the following does not work (but instead produces a syntax error):
while { read -u 4 line } && (test "$line" != "$header_line")
do
echo in loop, line=$line
done
I think I could write a loop that tests a "done" variable, and then do my real tests inside the loop and set "done" appropriately, but I am curious as to 1) why the above does not work, and 2) is there some small correction that would make it work? I still fairly confused about when to use [, (, {, or ((, so perhaps some other combination would work, though I have tried several.
(Note: The "read -u 4 line" works fine when I call it above the loop. I have opened a file on file descriptor 4.)
I think what you want is more like this:
while read -u 4 line && test "$line" != "$header_line"
do
...
done
Braces (the {} characters) are used to separate variables from other parts of a string when whitespace cannot be used. For example, echo "${var}x" will print the value of the variable var followed by an x, but echo "$varx" will print the value of the variable varx.
Brackets (the [] characters) are used as a shortcut for the test program. [ is another name for test, but when test detects that it was called with [ it required a ] as its last argument. The point is clarity.
Parenthesis (the () characters) are used in a number of different situations. They generally start subshells, although not always (I'm not really certain in case #3 here):
Retrieving a single exit code from a series of processes, or a single output stream from a sequence of commands. For example, (echo "Hi" ; echo "Bye") | sed -e "s/Hi/Hello/" will print two lines, "Hello" and "Bye". It is the easiest way to get multiple echo statements to produce a single stream.
Evaluating commands as if they were variables: $(expr 1 + 1) will act like a variable, but will produce the value 2.
Performing math: $((5 * 4 / 3 + 2 % 1)) will evaluate like a variable, but will compute the result of that mathematical expression.
The && operator is a list operator - he seperates two commands and only executes when the first is true, but in this case the first is the while and he is expecting his do stuff. And then he reaches do and the while stuff is already history.
Your intention is to put it into the expression. So you put it together with (). E.g. this a solution with just a small change
while ( read -u 4 line && test "$line" != "$header_line" )
I have a task to write simple bash script that adds deletes and views entries from file.
The requirement is to use "dialog"
data structure in file:
Name Surname mymail#mail.com
Another New person#database.loc
basically i have accomplished everything except delete, i know how to do the delete itself(with "sed" i think?)
But i need to use dialog --menu to display the search results.
The menu item should be whole line of text i think as after selection of an item i will use "grep" again to filter out the unique entry.
Maybe anyone can put me on the right direction?
Thanks.
I used dialog never before, but maybe I still can help. Try this:
declare -a args=()
while read
do
args+=("$REPLY" "")
done < <( grep '#' example.txt )
dialog --menu "Please select the line you want to edit" 40 60 34 "${args[#]}"
How does this work?
dialog --menu takes the following arguments:
question text
height and width of the window
height of the menu (which should be 7 less then the window height to use it fully, in my experience)
pairs of tag string and description.
The selected tag string is then output (on stderr) at the end.
How to create such a list strings from our grep output? A failed try is described below, here the working one.
The read command reads one line a time from standard input (to which we redirected the grep output), and puts it (if we don't give other options or arguments) in the REPLY variable.
We then add this value (quoted to be one element) to the array args , and additionally a single "" to add an empty string to the array, too.
We have to use the < <( ... ) syntax for redirection, since the normal | creates a subshell for the second command, which has the effect that changes to the variables are not propagated back to the original shell. (< means read input from file, and <( ... ) creates a pipe to read the output of the command and results in its filename.)
Then we use the "${args[#]}" parameter expansion - # has the effect that each element is individually quoted as the result. So for your example, the command line now looks like
dialog --menu "Please select the line you want to edit" 40 60 34 "Name Surname mymail#mail.com" "" "Another New person#database.loc" ""
This creates a two line menu, with the complete lines as the "tag", and an empty string as the clarification.
You will need some way to capture it's standard error output, as it puts the result there.
Another idea which does not work:
The question is how to to get the output of grep in the command line of dialog, so that it forms two arguments for each line.
What helps here are the following syntactic constructs:
Command substitution: $( cmd ) executes the command and converts the result to a string, which is then used at the point in the command line.
So, we need some command which produces two "words" for each line of grep output (since your file would give three words). As you are already using sed, why not use it here too?
The sed command s/^.*$/"&" ""/ replaces each line with the line enclosed in "", followed by another two quotes.
"Name Surname mymail#mail.com" ""
"Another New person#database.loc" ""
The idea would now be to use
dialog --menu "Please select the line you want to edit" 40 60 34 $( sed -e 's/^.*$/"&" ""/' < input )
but unfortunately the word-splitting of bash does not respect "" after command-substitution, so bash gives the six arguments "Name, Surname, mymail#mail.com", "", "Another, New, person#database.loc" and "" to the dialog program. (In fact, using "" to inhibit splitting seems to work only for quotes given literal in the source or in eval - but eval does not work here since we have line breaks, too.)