I have a program that is a compiled binary that calls a whole bunch (~300) of child bash scripts. I would just like a way to time each of those child processes to find out which ones take the longest. The problem comes from the fact that I cannot change the compiled binary (by adding for example time foo.sh >> bar_log.txt). I can change the child scripts, so in theory I could move them to a different location and replace them with scripts that just call the ones from the new location with the time command, but again -- there are 300+ with unique names and whatnot. I was wondering if there was a way to call the original binary and get a log of the times to execute the child processes.
Thanks in advance.
EDIT: I also have limited access to additional programs and cannot download/install new ones. For example, I do not have atop.
The environment variable ENV (for POSIX shells; for bash when not started under the name sh, BASH_ENV) is parsed even by noninteractive shells as the location of a script to run on startup.
Thus, you can create a file that initializes tracing:
if [ -n "$BASH_VERSION" ]; then
# SECONDS is time since the start of this individual script
PS4=':$BASH_SOURCE:$LINENO:$SECONDS+'
else
# note that calling $(date) slows your code substantially
# nothing to do about it without shell extensions, however.
PS4=':${0}:$(date)+'
fi
set -x
...and set both ENV and BASH_ENV to point to that file.
Related
I have searched the forum couldn't find one.can we define a variable that only increments on every cronjob run?
for example:
i have a script that runs every 5minutes so i need a variable that increments based on the cron run
Say if the job ran 5minutes for minutes. so 6 times the script got executed so my counter variable should be 6 now
Im expecting in bash/shell
Apologies if a duplicate question
tried:
((count+1))
You can do it this way:
create two scripts: counter.sh and increment_counter.sh
add execution of increment_counter.sh in your cron job
add . /path/to/counter.sh into /etc/profile or /etc/bash.bashrc or wherever you need
counter.sh
declare -i COUNTER
COUNTER=1
export COUNTER
increment_counter.sh
#!/bin/bash
echo "COUNTER=\$COUNTER+1" >> /path/to/counter.sh
The shell that you've run the command in has exited; any variables it has set have gone away. You can't use variables for this purpose.
What you need is some sort of permanent data store. This could be a database, or a remote network service, or a variety of things, but by far the simplest solution is to store the value in a file somewhere on disk. Read the file in when the script starts and write out the incremented value afterwards.
You should think about what to do if the file is missing and what happens if multiple copies of the script are run at the same time, and decide whether those are situations you care about at all. If they are, you'll need to add appropriate error handling and locking, respectively, in your script.
Wouldn't this be a better solution?
...to define a file under /tmp, such that a command like:
echo -n "." > $MyCounterFilename
Tracks the number of times something is invoked, in my particular case of app.:
#!/bin/bash
xterm [ Options ] -T "$(cat $MyCounterFilename | wc -c )" &
echo -n "." > $MyCounterFilename
Because i had to modify the way xterm is invoked for my purposes and i found already that having opened many of these concurrently one waste less time if knowing exactly what is running on each one by its number (without having to cycle alt+tab and eye inspect through everything).
NOTE: /etc/profile, or better either ~/.profile or ~/.bash_profile needs only a env. variable name defined containing the full path to your counter file.
Anyway, if you dont like the idea above, experiments might be performed to determine a) 1st time out of all that /etc/profile is executed since machine is powered on and system boots. 2) Wether /etc/profile is executed or not, and how many times (Each time we open an xterm?, for instance). ... thereafter the same sort of testing for the other dudes lesser general than /etc one.
So I have a gem that has two executables say, run and run_nohup. I have created an executable file where I add all the environment stuff required to execute the run and have added this on the path.
Example:
env variable1=value variable2=value /opt/my_gem/bin/run "$#"
Now my question is, is there another way to do the same for run_nohup without duplicating this work? I ask this because, am installing all of this with chef and it would require me to create more templates, basically duplicating the old template except for the last part where I call run_nohup.
$0 is the name used to invoke the current program; thus, you can look at it to determine how you were called, or manipulate it (in the below case, stripping the directory name and using only the filename):
#!/bin/sh
exec env variable1=value variable2=value /opt/my_gem/bin/"${0##*/}" "$#"
You can take this single executable, save it in two files named run and run_nohup (which can be hardlinked together, if you like), and it'll call the appropriate tool from /opt/my_gem/bin for the name it's invoked with.
Aside: It would be slightly more efficient (save a few microseconds) to have the shell export the environment updates rather than calling through env:
#!/bin/sh
variable1=value variable2=value exec /opt/my_gem/bin/"${0##*/}" "$#"
I have fairly often seen that PATH variable is exported at the beginning of the script. For example in /etc/init.d/rc script in Debian Wheezy:
PATH=/sbin:/usr/sbin:/bin:/usr/bin
export PATH
While I understand that this ensures that executables used in the script are started from correct directories, I don't fully understand which shells are affected by this export statement. For example here I start the script named rc(PID 6582; command is "/bin/sh /etc/init.d/rc") in bash(PID 3987):
init(1)-+-acpid(1926)
|-sshd(2139)-+-sshd(2375)---bash(2448)---screen(3393)---screen(3394)-+-bash(3395)---vim(3974)
| | |-bash(3397)---pstree(6584)
| | `-bash(3987)---rc(6582)---sleep(6583)
Am I correct that this PATH export statement in rc script affects only the /bin/sh with PID 6582 because parent shells(bash with PID 3987 in my example) do not inherit variables from children? In addition, am I correct that all the commands executed in script rc are started under the /bin/sh with PID 6582 and thus use this PATH=/sbin:/usr/sbin:/bin:/usr/bin variable? If yes, then hasn't the simple PATH=/sbin:/usr/sbin:/bin:/usr/bin been enough?
The environment variables are inherited by all the processes run from the script. PATH in particular affects the behaviour of the C functions execlp() and execvp(), so all the processes launched by the init.d script that started sshd and their descendants are impacted, but only until the point where one of these descendants changes and exports it.
In particular, bash(2448) most probably changes it, as it is a login shell, to match the system's and the user's config, so all it descendans are impacted by this change.
Then when you run manually the /etc/init.d/rc script by hand, the change is inherited by the sleep command (but that one never tries to run anithing).
If yes, then hasn't the simple PATH=/sbin:/usr/sbin:/bin:/usr/bin been enough?
If you mean just setting the variable instead of also exporting it, it depends on what the rc script runs. If it launches anything that tries to run commands with any of those functions, then no, only after exporting PATH it affects the children.
PATH should already be exported by the parent shell when the script run, so indeed, there is no need.
I can imagine corner cases where the shell which runs your script might not be properly initialized, such as for a startup script running very early in the boot process, but for regular userspace scripts, things should be set up the way you want them already.
I ask because I recently made a change to a KornShell (ksh) script that was executing. A short while after I saved my changes, the executing process failed. Judging from the error message, it looked as though the running process had seen some -- but not all -- of my changes. This strongly suggests that when a shell script is invoked, the entire script is not read into memory.
If this conclusion is correct, it suggests that one should avoid making changes to scripts that are running.
$ uname -a
SunOS blahblah 5.9 Generic_122300-61 sun4u sparc SUNW,Sun-Fire-15000
No. Shell scripts are read either line-by-line, or command-by-command followed by ;s, with the exception of blocks such as if ... fi blocks which are interpreted as a chunk:
A shell script is a text file containing shell commands. When such a
file is used as the first non-option argument when invoking Bash, and
neither the -c nor -s option is supplied (see Invoking Bash), Bash
reads and executes commands from the file, then exits. This mode of
operation creates a non-interactive shell.
You can demonstrate that the shell waits for the fi of an if block to execute commands by typing them manually on the command line.
http://www.gnu.org/software/bash/manual/bashref.html#Executing-Commands
http://www.gnu.org/software/bash/manual/bashref.html#Shell-Scripts
It's funny that most OS'es I know, do NOT read the entire content of any script in memory, and run it from disk. Doing otherwise would allow making changes to the script, while running. I don't understand why that is done, given the fact :
scripts are usually very small (and don't take many memory anyway)
at some point, and shown in this thread, people would start making changes to a script that is already running anyway
But, acknowledging this, here's something to think about: If you decided that a script is not running OK (because you are writing/changing/debugging), do you care on the rest of the running of that script ? you can go ahead making the changes, save them, and ignore all output and actions, done by the current run.
But .. Sometimes, and that depends on the script in question, a subsequent run of the same script (modified or not), can become a problem since the current/previous run is doing an abnormal run. It would typically skip some stuff, or sudenly jump to parts in the script, it shouldn't. And THAT may be a problem. It may leave "things" in a bad state; particularly if file manipulation/creation is involved.
So, as a general rule : even if the OS supports the feature or not, it's best to let the current run finish, and THEN save the updated script. You can change it already, but don't save it.
It's not like in the old days of DOS, where you actually have only one screen in front of you (one DOS screen), so you can't say you need to wait on run completion, before you can open a file again.
No they are not and there are many good reasons for that.
One of the things you should keep in mind is that a shell is not an interpreter even if there are some similarities. Shells are designed to work with a stream of commands. Either from the TTY ,a PIPE, FIFO or even a socket.
The shell reads from its resource line by line until a EOF is returned by the kernel.
The most shells have no extra support for interpreting files. they work with a file as they would work with a terminal.
In fact this is considered to be a nice feature because you can do interesting stuff like this How do Linux binary installers (.bin, .sh) work?
You can use a binary file and prepend shell scripts. You can't do this with an interpreter. because it parses the whole file or at least it would try it and fail. A shell would just interpret it line by line and doesnt care about the garbage at the end of the file. You just have to make sure the execution of the script gets terminated before it reaches the binary part.
I have a load of bash scripts that backup different directories to different locations. I want each one to run every day. However, I want to make they don't run simultaneously.
I've wrote a script that basically just calls each script in succession and sits in cron.daily, but I want a way for this script to work even if I add and remove backup scripts without having to manually edit it.
So what I need to go is generate a list of the scripts (e.g. "dir -1 /usr/bin/backup*.sh") and then run each script it finds in turn.
Thanks.
#!/bin/sh
for script in /usr/bin/backup*.sh
do
$script
done
#!/bin/bash
for SCRIPT in /usr/bin/backup*.sh
do
[ -x "$SCRIPT" ] && [ -f "$SCRIPT" ] && $SCRIPT
done
If your system has run-parts then that will take care of it for you. You can name your scripts like "10script", "20anotherscript" and they will be run in order in a manner similar to the rc*.d hierarchy (which is run via init or Upstart, however). On some systems it's a script. On mine it's a binary executable.
It is likely that your system is using it to run hourly, daily, etc., cron jobs just by dropping scripts into directories such as /etc/cron.hourly/
Pay particular attention, though, to how you name your scripts. (Don't use dots, for example.) Check the man page specific to your system, since file naming restrictions may vary.