I am learning the bash environment and cannot understand what I get when running this command:
wc filename.txt &
It returns an array with a 1-digital integer and another integer, neither of them matches any other result I can get from wc commands (-l, -m, -w, -c).
Besides the second integer is much bigger than for example the bytes counts. So I terribly wonder.
I browsed forums and found some explanations on the multiple uses of the ampersand in a Unix/Linux environment, but there was nothing that I could relate.
I don't need it, but I won't flush this mystery away, I wish to understand!
Thanks
I imagine the integers you see are similar to this:
[1] 1830
& launches a command in the background, and the shell prints its job number (1) and process id (1830). On a longer-running job, you can use those two numbers to control its execution. See the JOB CONTROL section of the bash man page for more details.
An ampersand at the end of a WC command tells the shell to start executing the command in the background and to get ready for further command line commands.
Related
I have a bash script that executes a series of commands, some involving redirection. See cyrus-mark-ham-spam.
I want the script to have a test mode, where all the commands run are printed instead of executing them. As you can see, I have tried to do that by just putting "echo" on the front of each command in test mode.
Unfortunately this doesn't deal with redirection - any redirections are still done, so the program leaves lots of temp files littered about the place when run in test mode.
I have tried various ways to get round this, like quoting the whole command and passing it to a function that either prints it or runs it, but either the redirections work in test mode, or they don't work in run mode.
I thought this must have come up before, and wonder if there is a known solution which does not involve every command being repeated with an if TEST round the pair?
Please note, this is NOT a duplicate of show commands without executing them because neither that question, nor its answers, covers redirection (which is the essence of this question).
I see that it is not a duplicate but there is not general solution to this. You need to look at each command separately.
As long as the command doesn't use arguments enclosed in spaces, like
cmd -a -b -c > filename
, you can quote it:
echo 'cmd -a -b -c > filename'
But real life code is more complex, sure.
Say you have a shell command like
cat file1 | ./my_script
Is there any way from inside the 'my_script' command to detect the command run first as the pipe input (in the above example cat file1)?
I've been digging into it and so far I've not found any possibilities.
I've been unable to find any environment variables set in the process space of the second command recording the full command line, the command data the my_script commands sees (via /proc etc) is just _./my_script_ and doesn't include any information about it being run as part of a pipe. Checking the process list from inside the second command even doesn't seem to provide any data since the first process seems to exit before the second starts.
The best information I've been able to find suggests in bash in some cases you can get the exit codes of processes in the pipe via PIPESTATUS, unfortunately nothing similar seems to be present for the name of commands/files in the pipe. My research seems to be saying it's impossible to do in a generic manner (I can't control how people decide to run my_script so I can't force 3rd party pipe replacement tools to be used over build in shell pipes) but it just at the same time doesn't seem like it should be impossible since the shell has the full command line present as the command is run.
(update adding in later information following on from comments below)
I am on Linux.
I've investigated the /proc/$$/fd data and it almost does the job. If the first command doesn't exit for several seconds while piping data to the second command can you read /proc/$$/fd/0 to see the value pipe:[PIPEID] that it symlinks to. That can then be used to search through the rest of the /proc//fd/ data for other running processes to find another process with a pipe open using the same PIPEID which gives you the first process pid.
However in most real world tests I've done of piping you can't trust that the first command will stay running long enough for the second one to have time to locate it's pipe fd in /proc before it exits (which removes the proc data preventing it being read). So if this method will return any information is something I can't rely on.
How can one loop a command/program in a Unix shell without writing the loop into a script or other application.
For example, I wrote a script that outputs a light sensor value but I'm still testing it right now so I want it run it in a loop by running the executable repeatedly.
Maybe I'd also like to just run "ls" or "df" in a loop. I know I can do this easily in a few lines of bash code, but being able to type a command in the terminal for any given set of command would be just as useful to me.
You can write the exact same loop you would write in a shell script by writing it in one line putting semicolons instead of returns, like in
for NAME [in LIST ]; do COMMANDS; done
At that point you could write a shell script called, for example, repeat that, given a command, runs it N times, by simpling changing COMMANDS with $1 .
I recommend the use of "watch", it just do exactly what you want, and it cleans the terminal before each execution of the commands, so it's easy to monitor changes.
You probably have it already, just try watch ls or watch ./my_script.sh. You can even control how much time to wait between each execution, in seconds, with the -n option, and you can use -d to highlight the difference in the output of consecutive runs.
Try:
Run ls each second:
watch -n 1 ls
Run my_script.sh each 3 seconds, and highlight differences:
watch -n 3 -d ./my_script.sh
watch program man page:
http://linux.die.net/man/1/watch
This doesn't exactly answer your question, but I felt it was relavent. One of the great things with shell looping is that some commands return lists of items. Of course that is obvious, but a something you can do using the for loop is execute a command on that list of items.
for $file in `find . -name *.wma`; do cp $file ./new/location/ done;
You can get creative and do some very powerful stuff.
Aside from accepting arguments, anything you can do in a script can be done on the command line. Earlier I typed this directly in to bash to watch a directory fill up as I transferred files:
while sleep 5s
do
ls photos
end
My $SHELL is tcsh. I want to run a C shell script that will call a program many times with some arguments changed each time. The program I need to call is in Fortran. I do not want to edit it. The program only takes arguments once it is executed, but not on the command line. Upon calling the program in the script, the program takes control (this is where I am stuck currently, I can never get out because the script will not execute anything until after the program process stops). At this point I need to pass it some variables, then after several iterations I will need to Ctrl+C out of the program and continue with the script.
How can this be done?
To add to what #Toybuilder said, you can use a "here document". I.e. your script could have
./myfortranprogram << EOF
first line of input
second line of input
EOF
Everything between the "<<EOF" and the "EOF" will be fed to the program's standard input (does Fortran still use "read (5,*)" to read from standard input?)
And because I think #ephemient's comment deserves to be in the answer:
Some more tips: <<'EOF' prevents
interpolation in the here-doc body;
<<-EOF removes all leading tabs (so
you can indent the here-doc to match
its surroundings), and EOF can be
replaced by any token. An empty token
(<<"") indicates a here-doc that stops
at the first empty line.
I'm not sure how portable those ones are, or if they're just tcsh extensions - I've only used the <<EOF type "here document" myself.
What you want to use is Expect.
Uhm, can you feed your Fortran code with a redirection? You can create a temporary file with your inputs, and then pipe it in with the stdin redirect (<).
This is a job for the unix program expect, which can nicely and easily interactively command programs and respond to their prompts.
I was sent here after being told my question was close to being a duplicate of this one.
FWIW, I had a similar problem with a csh C shell script.
This bit of code was allowing the custom_command to execute without getting ANY input arguments:
foreach f ($forecastTimes)
custom_command << EOF
arg1=x$f;2
arg2=ya
arg3=z,z$f
run
exit
EOF
end
It didn't work the first time I tried it, but after I backspaced out all of the white space in that section of the code I removed the space between the "<<" and the "EOF". I also backspaced the closing "EOF" all the way to the left margin. After that it worked:
foreach f ($forecastTimes)
custom_command <<EOF
arg1=x$f;2
arg2=ya
arg3=z,z$f
run
exit
EOF
end
Not a tcsh user, but if the program runs then reads in commands via stdin then you can use shell redirection < to feed it the required commands. If you run it in the background with & you will not block when it is executed. Then you can sleep for a bit, then use whatever tools you have (ps, grep, awk, etc) to discover the program's PID, then use kill to send it SIGTERM which is the same as doing a Ctrl-C.
I once read that one way to obtain a unique filename in a shell for temp files was to use a double dollar sign ($$). This does produce a number that varies from time to time... but if you call it repeatedly, it returns the same number. (The solution is to just use the time.)
I am curious to know what $$ actually is, and why it would be suggested as a way to generate unique filenames.
$$ is the process ID (PID) in bash. Using $$ is a bad idea, because it will usually create a race condition, and allow your shell-script to be subverted by an attacker. See, for example, all these people who created insecure temporary files and had to issue security advisories.
Instead, use mktemp. The Linux man page for mktemp is excellent. Here's some example code from it:
tempfoo=`basename $0`
TMPFILE=`mktemp -t ${tempfoo}` || exit 1
echo "program output" >> $TMPFILE
In Bash $$ is the process ID, as noted in the comments it is not safe to use as a temp filename for a variety of reasons.
For temporary file names, use the mktemp command.
$$ is the id of the current process.
Every process in a UNIX like operating system has a (temporarily) unique identifier, the PID. No two processes running at the same time can have the same PID, and $$ refers to the PID of the bash instance running the script.
This is very much not a unique idenifier in the sense that it will never be reused (indeed, PIDs are reused constantly). What it does give you is a number such that, if another person runs your script, they will get a different identifier whilst yours is still running. Once yours dies, the PID may be recycled and someone else might run your script, get the same PID, and so get the same filename.
As such, it is only really sane to say "$$ gives a filename such that if someone else runs the same script whist my instance is still running, they will get a different name".
$$ is your PID. It doesn't really generate a unique filename, unless you are careful and no one else does it exactly the same way.
Typically you'd create something like /tmp/myprogramname$$
There're so many ways to break this, and if you're writing to locations other folks can write to it's not too difficult on many OSes to predict what PID you're going to have and screw around -- imagine you're running as root and I create /tmp/yourprogname13395 as a symlink pointing to /etc/passwd -- and you write into it.
This is a bad thing to be doing in a shell script. If you're going to use a temporary file for something, you ought to be using a better language which will at least let you add the "exclusive" flag for opening (creating) the file. Then you can be sure you're not clobbering something else.
$$ is the pid (process id) of the shell interpreter running your script. It's different for each process running on a system at the moment, but over time the pid wraps around, and after you exit there will be another process with same pid eventually.As long as you're running, the pid is unique to you.
From the definition above it should be obvious that no matter how many times you use $$ in a script, it will return the same number.
You can use, e.g. /tmp/myscript.scratch.$$ as your temp file for things that need not be extremely reliable or secure. It's a good practice to delete such temp files at the end of your script, using, for example, trap command:
trap "echo 'Cleanup in progress'; rm -r $TMP_DIR" EXIT
$$ is the pid of the current shell process. It isn't a good way to generate unique filenames.
It's the process ID of the bash process. No concurrent processes will ever have the same PID.
The $$ is the process id of the shell in which your script is running. For more details, see the man page for sh or bash. The man pages can be found be either using a command line "man sh", or by searching the web for "shell manpage"
Let me second emk's answer -- don't use $$ by itself as a "unique" anything. For files, use mktemp. For other IDs within the same bash script, use "$$$(date +%s%N)" for a reasonably good chance of uniqueness.
-k
In Fish shell (3.1.2):
The $ symbol can also be used multiple times, as a kind of "dereference" operator (the * in C or C++)
set bar bazz
set foo bar
echo $foo # bar
echo $$foo # same as echo $bar → bazz
Also, You can grab login username via this command. Eg.
echo $(</proc/$$/login id). After that, you need to use getent command.