Trying to compare the process list before and after running a bash script of tests. Having trouble, since ps returns 1, and I'm not sure how to compare the before and after when I have them.
Ideally, it would look something like this. Forgive the crude pseudo-code:
run-tests:
ps -x
export before=$?
# run tests and scripts
ps -x
export after=$?
# compare before and after
Suggests and advice appreciated.
I'm assuming you want to count the number of running processes before and after (your question wasn't overly clear on that). If so, you can pipe ps into wc:
export before=`ps --no-headers | wc -l`
-- EDIT ---
I reread the question, and it may be that you're looking for the actual processes that differ. If that's the case, then, you can capture the output in variables and compare those:
target:
# before=$$(ps --no-headers); \
run test; \
after=$$(ps --no-headers); \
echo "differing processes:"; \
comm -3 <(echo "$before") <(echo "$after")
A few quick notes on this: I concatenated all the lines using \'s as you mentioned you used makefiles, and the scope of a variable is only the recipe line in which it's defined. By concatenating the lines, the variables have a scope of the whole recipe.
I used double $$ as your original post suggested a makefile, and a makefile $$ will expand to a single $ in the bash code to be run.
Doing var=$(command) in bash assigns var the output of command
I used the <() convention which is specific to bash. This lets you treat the output of a command as file, without having to actually create a file. Notice that I put quotes around the variables -- this is required, otherwise the bash will ignore newlines when expanding the variable.
Related
I need to take specific variables from one script and use them in a different script.
Example:
Original script:
VARA=4 # Some description of VARA
VARB=6 # Some description of VARB
SOMEOTHERVAR="Foo"
/call/to/some/program
I want to write a second script that needs VARA and VARB, but not SOMEOTHERVAR or the call to the program.
I can already do:
eval $(grep 'VARA=' origscript.sh)
eval $(grep 'VARB=' origscript.sh)
This seems to work, but when I want to do both, like this, it only sets the first:
eval $(grep 'VAR[AB]=' origscript.sh)
because it seems to concatenate the two lines that grep returns. (Which probably means that the comments save the first assignments.)
Put quotes around it, so that the newlines in the output of grep will not be turned into spaces.
eval "$(grep 'VAR[AB]=' origscript.sh)"
I have two aliases:
alias ls="ls -G"
alias la="ls -aFhlT"
I know that after you type your alias, but before you execute, you can type Meta-Control-e (probably Alt-Control-e, but possibly Esc-Control-e) to expand what you've typed.
So, if I expand my alias la using this method I get:
ls -aFhlT
However, what I really want is to see:
ls -G -aFhlT
Is there any way to achieve this besides typing Meta-Control-e a second time?
--OR--
Is there any way to confirm that my execution of la actually executed ls -G -aFhlT (other than knowing how nested aliases work and trusting that it did what I think it did)?
I'm trying to do this on macOS, but a general bash solution will also be accepted.
This question rides the fine line between using an alias and using a function. When aliases get even slightly complicated, it is generally better to write a function instead. That being said, I did find a solution for this question that allows for expanding aliases as desired.
I wrote a bash function for this:
xtrace() {
local eval_cmd
printf -v eval_cmd '%q ' "${#}"
{ set -x
eval "${eval_cmd}"
} 2>&1 | grep '^++'
return "${PIPESTATUS[0]}"
}
The -v flag of printf will store the output of printf in the specified variable.
The printf format string %q will print the associated argument ($# in this case) shell-quoted, reusable as input. This eliminates the dangers associated with passing arbitrary code/commands to eval.
I then use a command group { ... } so I can control the functionality of set -x, which tells bash to print a trace of all executed commands. For my purposes, I do not care about any output except for the fully expanded command, so I redirect stderr and grep for the output line that starts with "++". This will be the line that shows the fully expanded command.
Finally, I return the value of PIPESTATUS[0], which contains the return code of the last command executed in the command group (i.e. the eval command).
Thus, we will get something like the following:
$ xtrace la; echo $?
++ ls -G -aFhlT
0
Much thanks to #CharlesDuffy for the set -x recommendation as well as the input sanitation for eval.
I am using make's shell command to populate some variables, and on output it indicates that they are being set to the values I expect. However when I pass them to my make recipes they all show up empty. I suspect make is doing some odd parsing on the results. Eg:
MyTarget: $(MySources)
LINE='$(shell cat $< | grep GIMME_THE_LINE_I_WANT)'
CH9=$(shell echo $(LINE) | cut -b9)
echo $(CH9) # doesn't print anything
I checked my generator commands manually by setting SHELL=sh -XV and when I run identical commands I get the right values, it just looks like bash is 'zeroing' my variables. Any idea what's wrong?
There are several things going on here. The first is that when you have:
MyTarget: $(MySources)
LINE='$(shell cat $< | grep GIMME_THE_LINE_I_WANT)'
You are setting a shell variable called LINE, not a make variable. The build instructions for a target are all shell commands. So after the first line, the shell variable $LINE contains GIMME_THE_LINE_I_WANT. However...
...each line in the build instructions for a target runs in a separate shell process. So if you have:
mytarget:
MYVAR=foo
echo $$MYVAR
You'll see no output, because $MYVAR isn't set in the context of the second command. Also note the use of $$ here, because otherwise the $ would be interpreted by Make (that is, writing $MYVAR would actually be the make expression $M followed by the text YVAR). You can resolve this by logically joining your lines into a single shell script, like this:
mytarget:
MYVAR=foo; \
echo $$MYVAR
The \ is Makefile syntax that extends a single logical line over multiple physical lines, and of course ; is simply shell syntax for combining multiple commands on one line.
With all this in mind, we could rewrite your target like this:
MyTarget: $(MySources)
LINE=$$(cat $< | grep GIMME_THE_LINE_I_WANT); \
CH9=$$(echo $$LINE | cut -b9); \
echo $$CH9
Notice that since we are already running a shell script I'm not using Make's $(shell ...) construct, and that I'm making sure to escape all of the $ characters to ensure that the shell, not Make, is handling variable expansion.
Taking it just a little further, you don't need to use cat in that script; you could simply write:
MyTarget: $(MySources)
LINE=$$(grep GIMME_THE_LINE_I_WANT $<); \
CH9=$$(echo $$LINE | cut -b9); \
echo $$CH9
Or:
MyTarget: $(MySources)
CH9=$$(grep GIMME_THE_LINE_I_WANT $< | cut -b9); \
echo $$CH9
(NB: While not germane to this solution, it's although worth noting that each invocation of $(shell ...) is also run in a separate process, so a variable set in one won't be available in another.)
The make runs every command in its separate shell. So, the values are not carried over.
When in doubt, you could always debug it with -d option. Also, a site note, the debug option is very useful when you are trying to figure out why a rule did not fire the way you had intended it.
~> cat Makefile
MyTarget:
LINE="somevalue"
echo ${LINE}
~>
~>
~> make -d MyTarget | tail -10
LINE="somevalue"
Putting child 0x2004fbb0 (MyTarget) PID 4052 on the chain.
Live child 0x2004fbb0 (MyTarget) PID 4052
Reaping winning child 0x2004fbb0 PID 4052
echo
Live child 0x2004fbb0 (MyTarget) PID 4192
Reaping winning child 0x2004fbb0 PID 4192
Removing child 0x2004fbb0 PID 4192 from chain.
Successfully remade target file 'MyTarget'.
~>
Just clarifying for the people who downvoted and commented that my above solution doesn't work: First, I apologize for my laziness. May be I should have been clear. Let me try again.
The "make -d" is not a solution to OP's problem.
I tried to show OP how he/she could use debug option to solve a variety of problems that people come across while using makefiles (which, I admit, goes in a slight tangent than just solving the OP's problem at hand).
The above debug shows that the first command was executed in a shell with PID=4052 and the second command was executed in another shell with PID=4192 (which doesn't carry the value of that variable). Also it shows that using a variable with single dollar (${LINE}) just gives you a blank (because the makefile doesn't interpret it as a shell variable).
Again, to be clear: "make -d" is not a solution. Just combine the commands in one line, separated by commas, use double dollars; if the line is long, escape the new lines.
MyTarget:
LINE="somevalue"; \
echo $${LINE}
I have variable "one" which contains following
avi,mkw,dvd,cd
im trying to dynamicly create directories that would look like this
type-avi
type-mkw
type-dvd
type-cd
I have tried to achieve wanted result with following code
mkdir type-{"$one"}
but instead of creating 4 directories , it created one directory called
type-{avi,mkw,dvd,cd}
I suppose this is wrong method.. if so , how can i create dynamicly directories with "suffixes" stored in variabe?
Use an array instead of your string variable for this.
IFS=, read -a onearr <<<"$one"
mkdir "${onearr[#]/#/type-}"
Or if you don't need the $one string in the first place just create the array manually.
onearr=(avi mkw dvd cd)
mkdir "${onearr[#]/#/type-}"
If you aren't worried about spaces or anything in the values in $one and can trust your input to be "safe" and not exploitative and can't use read then you could use this to create the array instead (but it is just flat out a worse soluton).
onearr=($(tr , ' ' <<<"$one"))
A way to do this without reading into the shell, in a traditional tools pipeline approach:
echo "$one" |
tr ',' '\n' |
sed "s/^/mkdir 'type-/; s/$/'/" |
sh -x
Your original attempt was very close. To make it work, you can use the shell eval command:
eval mkdir type-{$one}
or
echo mkdir type{"$one"} | bash
In either case, the effect causes bash to re-evaluate the line.
I personally would not recommend this approach for these reasons:
eval can be a security risk and is little used, maintainers will have to do a double-take.
Brace Expansion is a bash-type shell extension and while I love bash, I write all shell scripts to run with the POSIX /bin/sh.
These will not handle unusual characters in filenames, such as spaces.
The eval causes the shell to re-evaluate the string after the variable substition has been performed. To gain more understanding on these topics, see "Brace Expansion" and also the eval command, both on the bash man page.
I am trying to automate the set up of site creation for our in-house development server.
Currently, this consists of creating a system user, mysql user, database, and apache config. I know how I can do everything in a single bash file, but I wanted to ask if there was a way to more cleanly generate the apache config.
Essentially what I want to do is generate a conf file based on a template, similar to using printf. I could certainly use printf, but I thought there might be a cleaner way, using sed or awk.
The reason I don't just want to use printf is because the apache config is about 20 lines long, and will take up most of the bash script, as well as make it harder to read.
Any help is appreciated.
Choose a way of marking parameters. One possibility is :parameter:, but any similar pair of markers that won't be confused with legitimate text for the template file(s) is good.
Write a sed script (in sed, awk, perl, ...) similar to the following:
sed -e "s/:param1:/$param1/g" \
-e "s/:param2:/$param2/g" \
-e "s/:param3:/$param3/g" \
httpd.conf.template > $HTTPDHOME/etc/httpd.conf
If you get to a point where you need sometimes to edit something and sometimes don't, you may find it easier to create the relevant sed commands in a command file and then execute that:
{
echo "s/:param1:/$param1/g"
echo "s/:param2:/$param2/g"
echo "s/:param3:/$param3/g"
if [ "$somevariable" = "somevalue" ]
then echo "s/normaldefault/somethingspecial/g"
fi
} >/tmp/sed.$$
sed -f /tmp/sed.$$ httpd.conf.template > $HTTPDHOME/etc/httpd.conf
Note that you should use a trap to ensure the temporary doesn't outlive its usefulness:
tmp=/tmp/sed.$$ # Consider using more secure alternative schemes
trap "rm -f $tmp; exit 1" 0 1 2 3 13 15 # aka EXIT HUP INT QUIT PIPE TERM
...code above...
rm -f $tmp
trap 0
This ensures that your temporary file is removed when the script exits for most plausible signals. You can preserve a non-zero exit status from previous commands and use exit $exit_status after the trap 0 command.
I'm surprised nobody mentioned here documents. This is probably not what the OP wants, but certainly a way to improve legibility of the script you started out with. Just take care to escape or parametrize away any constructs which the shell will perform substitutions on.
#!/bin/sh
# For example's sake, a weird value
# This is in single quotes, to prevent substitution
literal='$%"?*=`!!'
user=me
cat <<HERE >httpd.conf
# Not a valid httpd.conf
User=${user}
Uninterpolated=${literal}
Escaped=\$dollar
HERE
In this context I would recommend ${variable} over the equivalent $variable for clarity and to avoid any possible ambiguity.
Use sed like for example
sed s/%foo%/$foo/g template.conf > $newdir/httpd.conf