I am a newbie to unix scripting, I want to do following and I have little clue how to proceed.
I want to log the input and output of certain set of commands, given on the terminal, to a trace file. I should be able to switch it on and off.
E.g.
switch trace on
user:echo Hello World
user:Hello World
switch trace off
Then the trace log file, e.g. trace.log, it's content should be
echo Hello World
Hello World
One thing that I can think to do is to use set -x, redirecting its output to some file, but couldn't find a way to do that. I did man set, or man -x but I found no entry. Maybe I am being too naive, but some guidance will be very helpful.
I am using bash shell.
See script(1), "make typescript of terminal session". To start a new transcript in file xyz: script xyz. To add on to an existing transcript in file xyz: script -a xyz.
There will be a few overhead lines, like Script started on ... and Script done on ... which you could use awk or sed to filter out on printout. The -t switch allows a realtime playback.
I think there might have been a recent question regarding how to display a transcript in less, and although I can't find it, this question and this one address some of the same issues of viewing a file that contains control characters. (Captured transcripts often contain ANSI control sequences and usually contain Returns as well as Linefeeds.)
Update 1 A Perl program script-declutter is available to remove special characters from script logs.
The program is about 45 lines of code found near the middle of the link. Save those lines of code in a file called script-declutter, in a subdirectory that's on your PATH (for example, $HOME/bin if that's on your search path, else (eg) /usr/local/bin) and make the file executable. After that, a command like
script-declutter typescript > out
will remove most special characters from file typescript,
while directing the result to file out.
Related
A formely working bash script no longer works after switching computers. I get the following error:
No such file or directory.
Before going on, please excuse any mistakes you may find since english is not my native language.
The script was used in cygwin under Windows XP. I now had to switch to cygwin64 under Windwos 7 (64bit).
The script is used as a checkhandler for the program SMSTools3 to split a file with a specific format into multiple smaller ones, which the program then uses to send SMS to multiple recipients. The script was copied directly from the page of SMSTools3 and uses the package formail.
After looking up the error the most likely problem was that the environmantle path was not set up to look in the right path (/usr/bin). I therefore added it to the path but to no avail.
I then deleted other entries in the enviromental path of windows which contained spaces because that could have been another explanation, but again to no avail.
Following is a minimal example of the code which produces the error.
#!/bin/bash
# Sample script to allow multiple recipients in one message file.
# Define this script as a checkhandler.
echo $PATH
which formail
outgoing="/var/spool/sms/outgoing"
recipients=`formail -zx "To:" < "$1"`
I added the lines the lines echo $Path and which formail to show if the script can find the correct file. Both results look fine, the second command gives me the right output '/usr/bin/formail'
But the line recipients=... throws me the error:
No such file or directory.
I do not have much experience with bash scripting, or cygwin in general. So if someone on this wonderful board could help me solve this problem, I would be really grateful. Thank you all for your help.
EDIT:
First of all thank you all for your comments.
Secondly, I would like to apologize for the late reply. The computer in question is also used for other purposes and my problem is part of a background routine, so I have to wait for "free time" on the pc to test things.
For the things #shellter pruposed: The ls command returned an error: '': No such file or directory.
The which -a formail as well as the echo $(which -a formail) commands that #DougHenderson pruposed returned the 'right' path of /usr/bin/formail. echo \$1 = $1 before the recipent line returned the path to the checkhandler file (/usr/local/bin/smsd_checkhandler.sh), the same command after the recipent line seems to show a empty string ($1 = ). Also, the pruposed change to the recipent line did not change the error.
For the dos2unix conversion that #DennisWilliamson pruposed, I opened the file in notepad++ to use their build in converion, but it showed me that the file is in unix format with Unix style line endings.
I'm trying to use a Bash script to take certain log entries and generate a more user friendly notification email via exec with swatch. However, what I'm finding is when swatch picks up the corresponding log file lines to match for a specific application, the Bash script is erroring out with sh: script: No such file or directory.
This seems to be due to the log line in question outputting something like:
[2017-05-22 20:00:41] somehost someapp[3999]: INFO: <script>: bad stuff happened bruh
I've tested the script out with output from things like rsyslog to /var/log/messages and secure, which don't cause problems. This specific application I'm trying to make these notifications for is only problematic because the log lines themselves include <script>, which I can't exclude. Specifically, this seems to trip up with just <script as I've messed around with removing chars in those log lines that might be interpreted as something besides text.
Any ideas on how to not get the Bash script to try to interpret <script> as a file/directory in the original log? It's even fine if the suggested answer is to simply rip that out of the line. I've tried using sed -i 's/\<script\>//g ${#} to remove <script> to strip and store to a temp variable with the intention of doing something like: echo -e "${plainenglish}\n\nThe original log message is:${logline}" >> $outputfile but I get the same error noted above.
Edit: More info. The problem application in question is Kamailio, where most of the logging from routing execution is written by the xlog module. By default, xlog shoves <script> in front of everything you log. The module does include a parameter override (modparam) for prefix, which defaults to <script>.
I realize this is a bit specific for question and solution, but here's a summary followed by suggestion:
-Using Kamailio (SIP server/proxy, etc.) with xlog configuration logging calls to some logfile (e.g.: /var/log/kamailio)
-Using swatch or some tool to grab specific Kamailio logs with the intent of notifying, such as routing failure or suspicious requests received (e.g.: +18928751123#1.1.1.1:5060)
-Using Bash shell script to sanitize/normalize the corresponding log line in some manner for dispatch.
The solution: set xlog's prefix modparam to something that will not be interpreted by Bash as a legal command (script is a legal command) when attempting to process the log lines.
It seems Bash processing gets thrown astray as soon as it hits the <script> line in the default xlog produced logfile lines.
I need to start up a Golang web server and leave it running in the background from a bash script. If the script in question in syntactically correct (as it will be most of the time) this is simply a matter of issuing a
go run /path/to/index.go &
However, I have to allow for the possibility that index.go is somehow erroneous. I should explain that in Golang this for something as "trival" as importing a module that you then fail to use. In this case the go run /path/to/index.go bit will return an error message. In the terminal this would be something along the lines of
index.go:4:10: expected...
What I need to be able to do is to somehow change that command above so I can funnel any error messages into a file for examination at a later stage. I tried variants on go run /path/to/index.go >> errors.txt with the terminating & in different positions but to no avail.
I suspect that there is a bash way to do this by altering the priority of evaluation of the command via some judiciously used braces/brackets etc. However, that is way beyond my bash capabilities. I would be most obliged to anyone who might be able to help.
Update
A few minutes later... After a few more experiments I have found that this works
go run /path/to/index.go &> errors.txt &
Quite apart from the fact that I don't in fact understand why it works there remains the issue that it produces a 0 byte errors.txt file when the command goes to completion without Golang throwing up any error messages. Can someone shed light on what is going on and how it might be improved?
Taken from man bash.
Redirecting Standard Output and Standard Error
This construct allows both the standard output (file descriptor 1) and the standard error output (file descriptor 2) to be redirected to the file whose name is the expansion of word.
There are two formats for redirecting standard output and standard error:
&>word
and
>&word
Of the two forms, the first is preferred. This is semantically equivalent to
>word 2>&1
Appending Standard Output and Standard Error
This construct allows both the standard output (file descriptor 1) and the standard error output (file descriptor 2) to be appended to the file whose name is the expansion of word.
The format for appending standard output and standard error is:
&>>word
This is semantically equivalent to
>>word 2>&1
Narūnas K's answer covers why the &> redirection works.
The reason why the file is created anyway is because the shell creates the file before it even runs the command in question.
You can see this by trying no-such-command > file.out and seeing that even though the shell errors because no-such-command doesn't exist the file gets created (using &> on that test will get the shell's error in the file).
This is why you can't do things like sed 'pattern' file > file to edit a file in place.
I have a bash script that prints a line of text into a file, and then calls a second script that prints some more data into the same file. Lets call them script1.sh and script2.sh. The reason it's split into two scripts, is because I have different versions of script2.sh.
script1.sh:
rm -f output.txt
echo "some text here" > output.txt
source script2.sh
script2.sh:
./read_time >> output.txt
./run_program
./read_time >> output.txt
Variations on the three lines in script2.sh are repeated.
This seems to work most of the time, but every once in a while the file output.txt does not contain the line "some text here". At first I thought it was because I was calling script2.sh like this: ./script2.sh. But even using source the problem still occurs.
The problem is not reproducible, so even when I try to change something I don't know if it's actually fixed.
What could be causing this?
Edit:
The scripts are very simple. script1 is exactly as you see here, but with different file names. script 2 is what I posted, but then the same 3 lines repeated, and ./run_program can have different arguments. I did a grep for the output file, and for > but it doesn't show up anywhere unexpected.
The way these scripts are used is that script1 is created by a program (the only difference between the versions is the source script2.sh line. This script1.sh is then run on a different computer (linux on an FPGA actually) using ssh. Before that is done, the output file is also deleted using ssh. I don't know why, but I didn't write all of this. Also, I've checked the code running on the host. The only mention of the output file is when it is deleted using ssh, and when it is copied back to the host after the script1 is done.
Edit 2:
I finally managed to make the problem reproducible at a reasonable rate by stripping script2.sh of everything but a single line printing into the file. This also let me do the testing a bit faster. Once I had this I got the problem between 1 and 4 times for every 10 runs. Removing the command that was deleting the file over ssh before the script was run seems to have solved the problem. I will test it some more to be sure, but I think it's solved. Although I'm still not sure why it would be a problem. I thought that the ssh command would not exit before all the remove commands were executed.
It is hard to tell without seeing the real code. Most likely explanation is that you have a typo, > instead of >>, somewhere in one of the script2.sh files.
To verify this, set noclobber option with set -o noclobber. The shell will then terminate when trying to write to existing file with >.
Another possibility, is that the file is removed under certain rare conditions. Or it is damaged by some command which can have random access to it - look for commands using this file without >>. Or it is used by some command both as input and output which step on each other - look for the file used with <.
Lastly, you can have a racing condition with a command outputting to the file in background, started before that echo.
Can you grep all your scripts for 'output.txt'? What about scripts called inside read_time and run_program?
It looks like something in one of the script2.sh scripts must be either overwriting, truncating or doing a substitution on output.txt.
For example,there could be a '> output.txt' burried inside a conditional for a condition that rarely obtains. Just a guess, but it would explain why you don't always see it.
This is an interesting problem. Please post the solution when you find it!
I'm trying to write a script that contains this
screen -S demo -d -m which should start a new screen session named demo and detach it.
Putting screen -S demo -d -m in the command line works.
If I put it in a file named boot.sh, and run it ./boot.sh I get
Error: Unknown option m
Why does this work in the command line but not as a shell script?
This file was transferred from windows and had ctrl-M characters.
Running "screen" on my Linux machine, a bad option (Screen version 4.00.03jw4 (FAU) 2-May-06) gives the error,
Error: Unknown option -z"
while your description includes no dash before the offending option. I'd check that the characters in your script file are what you expect them to be. There are many characters that look like a dash but which are not.
cat -v boot.sh
may show something interesting as it'll show codes for non-ascii characters.
This may seem a little like the "make sure your printer is plugged in" kind of help, but anyway:
have you tried to check if the screen you're invoking from the script is the same as the one invoked from the command line ?
I'm thinking you may change the PATH variable inside your script somewhere and perhaps screen from the script would be something else (a different version, perhaps ?).