I am trying to do a kubectl exec and store value into a variable. But the variable displays null. If i use -it in the exec, it works. Since while running with terraform says can't allocate terminal, I need to run it without -it.
Command I run: abc=$(kubectl exec nginx-6799fc88d8-xpzs9 -- nginx -v)
You need to modify your command as following(-it):
abc=$(kubectl exec -it nginx-6799fc88d8-xpzs9 -- nginx -v)
The reason for this behavior is nginx -v is displaying the output over stderr not stdout. This means, the command you were using would work for any other commands like ls , cat etc which display output over stdout.
Alternate approach by redirecting stderr to stdout(2>&1) :
abc=$(kubectl exec nginx-6799fc88d8-xpzs9 -- nginx -v 2>&1)
Example:
x=$(k exec nginx -- nginx -v 2>&1)
echo "$x"
nginx version: nginx/1.21.3
As #P.... Mentioned alternative approach for replacing -it is using 2>&1
File descriptor 1 is the standard output (stdout).
File descriptor 2 is the standard error (stderr).
-i Pass stdin to the container(stdin)
-t Stdin is a TTY(tty)
Here is one way to remember this construct: at first, 2>1 may look like a good way to redirect stderr to stdout. However, it will actually be interpreted as "redirect stderr to a file named 1". & indicates that what follows and precedes is a file descriptor and not a filename. So the construct becomes: 2>&1.
Consider >& as a redirect merger operator.
For more details, refer this document on kubectl exec command.
Related
If I set -x in my bash session ( v4.1.2(2) - CentOS 6.10), I get :
$ ls /root
+ ls --color=auto /root
ls: cannot open directory /root: Permission denied
Great, it echo's the command I ran and prints out the terminal. This is expected. Now if I redirect both stdout and stderr to the another file.
$ ls /root &> stuff.txt
+ ls --color=auto /root
It still prints the command to the terminal.
QUESTION
Where is set -x having bash print to if it isn't stderr or stdout?
The set -x command prints tracing information to stderr.
When you run this command...
ls /root &> stuff.txt
You're only redirecting stdout and stderr for the ls command. You're not changing either for your current shell, which is where you have run set -x.
As Mad Physicist points out, the technical answer is "it logs to BASH_XTRACEFD", which defaults to stderr. You can redirect trace logging for the current shell to another file by doing something like:
# open a new file descriptor for logging
exec 4> trace.log
# redirect trace logs to fd 4
BASH_XTRACEFD=4
# enable tracing
set -x
When you execute a command, you can redirect the standard output (known as /dev/stdout) of the command directly to the file. Also if the command generates error-output (generally send to /dev/stderr) you can also redirect it to a file as:
$ command > /path/to/output.txt 2> /path/to/error.txt
When you execute the command set -x, you ask it to generate a trace of the commands being executed. It does this by sending messages to /dev/stderr. In contrast to a normal command, you cannot easily redirect this in a similar way as with a normal command. This is because bash executes the script and at the same time generates the trace to /dev/stderr. So if you would like to catch the trace, you would have to redirect the error output of bash directly. This can be done by the command
exec 2> /path/to/trace.txt
note: this will at the same time also contain all the error output of any command executed in the script.
Examples:
#!/usr/bin/env bash
set -x
command
This sends all output and error output to the terminal
#!/usr/bin/env bash
set -x
command 2> /path/to/command.err
This sends the output of command and the trace of bash to the terminal but catches the error output of command in a file
#!/usr/bin/env bash
set -x
exec 2> /path/to/trace.err
command 2> /path/to/command.err
This sends the output of command to the terminal, the error output of command to a file, and the trace of the script to /path/to/trace.err
#!/usr/bin/env bash
set -x
exec 2> /path/to/trace_and_command.err
command
This sends the output of command to the terminal, the trace and the error of command to a file.
I know how to redirect stdout to a file:
exec > foo.log
echo test
this will put the 'test' into the foo.log file.
Now I want to redirect the output into the log file AND keep it on stdout
i.e. it can be done trivially from outside the script:
script | tee foo.log
but I want to do declare it within the script itself
I tried
exec | tee foo.log
but it didn't work.
#!/usr/bin/env bash
# Redirect stdout ( > ) into a named pipe ( >() ) running "tee"
exec > >(tee -i logfile.txt)
# Without this, only stdout would be captured - i.e. your
# log file would not contain any error messages.
# SEE (and upvote) the answer by Adam Spiers, which keeps STDERR
# as a separate stream - I did not want to steal from him by simply
# adding his answer to mine.
exec 2>&1
echo "foo"
echo "bar" >&2
Note that this is bash, not sh. If you invoke the script with sh myscript.sh, you will get an error along the lines of syntax error near unexpected token '>'.
If you are working with signal traps, you might want to use the tee -i option to avoid disruption of the output if a signal occurs. (Thanks to JamesThomasMoon1979 for the comment.)
Tools that change their output depending on whether they write to a pipe or a terminal (ls using colors and columnized output, for example) will detect the above construct as meaning that they output to a pipe.
There are options to enforce the colorizing / columnizing (e.g. ls -C --color=always). Note that this will result in the color codes being written to the logfile as well, making it less readable.
The accepted answer does not preserve STDERR as a separate file descriptor. That means
./script.sh >/dev/null
will not output bar to the terminal, only to the logfile, and
./script.sh 2>/dev/null
will output both foo and bar to the terminal. Clearly that's not
the behaviour a normal user is likely to expect. This can be
fixed by using two separate tee processes both appending to the same
log file:
#!/bin/bash
# See (and upvote) the comment by JamesThomasMoon1979
# explaining the use of the -i option to tee.
exec > >(tee -ia foo.log)
exec 2> >(tee -ia foo.log >&2)
echo "foo"
echo "bar" >&2
(Note that the above does not initially truncate the log file - if you want that behaviour you should add
>foo.log
to the top of the script.)
The POSIX.1-2008 specification of tee(1) requires that output is unbuffered, i.e. not even line-buffered, so in this case it is possible that STDOUT and STDERR could end up on the same line of foo.log; however that could also happen on the terminal, so the log file will be a faithful reflection of what could be seen on the terminal, if not an exact mirror of it. If you want the STDOUT lines cleanly separated from the STDERR lines, consider using two log files, possibly with date stamp prefixes on each line to allow chronological reassembly later on.
Solution for busybox, macOS bash, and non-bash shells
The accepted answer is certainly the best choice for bash. I'm working in a Busybox environment without access to bash, and it does not understand the exec > >(tee log.txt) syntax. It also does not do exec >$PIPE properly, trying to create an ordinary file with the same name as the named pipe, which fails and hangs.
Hopefully this would be useful to someone else who doesn't have bash.
Also, for anyone using a named pipe, it is safe to rm $PIPE, because that unlinks the pipe from the VFS, but the processes that use it still maintain a reference count on it until they are finished.
Note the use of $* is not necessarily safe.
#!/bin/sh
if [ "$SELF_LOGGING" != "1" ]
then
# The parent process will enter this branch and set up logging
# Create a named piped for logging the child's output
PIPE=tmp.fifo
mkfifo $PIPE
# Launch the child process with stdout redirected to the named pipe
SELF_LOGGING=1 sh $0 $* >$PIPE &
# Save PID of child process
PID=$!
# Launch tee in a separate process
tee logfile <$PIPE &
# Unlink $PIPE because the parent process no longer needs it
rm $PIPE
# Wait for child process, which is running the rest of this script
wait $PID
# Return the error code from the child process
exit $?
fi
# The rest of the script goes here
Inside your script file, put all of the commands within parentheses, like this:
(
echo start
ls -l
echo end
) | tee foo.log
Easy way to make a bash script log to syslog. The script output is available both through /var/log/syslog and through stderr. syslog will add useful metadata, including timestamps.
Add this line at the top:
exec &> >(logger -t myscript -s)
Alternatively, send the log to a separate file:
exec &> >(ts |tee -a /tmp/myscript.output >&2 )
This requires moreutils (for the ts command, which adds timestamps).
Using the accepted answer my script kept returning exceptionally early (right after 'exec > >(tee ...)') leaving the rest of my script running in the background. As I couldn't get that solution to work my way I found another solution/work around to the problem:
# Logging setup
logfile=mylogfile
mkfifo ${logfile}.pipe
tee < ${logfile}.pipe $logfile &
exec &> ${logfile}.pipe
rm ${logfile}.pipe
# Rest of my script
This makes output from script go from the process, through the pipe into the sub background process of 'tee' that logs everything to disc and to original stdout of the script.
Note that 'exec &>' redirects both stdout and stderr, we could redirect them separately if we like, or change to 'exec >' if we just want stdout.
Even thou the pipe is removed from the file system in the beginning of the script it will continue to function until the processes finishes. We just can't reference it using the file name after the rm-line.
Bash 4 has a coproc command which establishes a named pipe to a command and allows you to communicate through it.
Can't say I'm comfortable with any of the solutions based on exec. I prefer to use tee directly, so I make the script call itself with tee when requested:
# my script:
check_tee_output()
{
# copy (append) stdout and stderr to log file if TEE is unset or true
if [[ -z $TEE || "$TEE" == true ]]; then
echo '-------------------------------------------' >> log.txt
echo '***' $(date) $0 $# >> log.txt
TEE=false $0 $# 2>&1 | tee --append log.txt
exit $?
fi
}
check_tee_output $#
rest of my script
This allows you to do this:
your_script.sh args # tee
TEE=true your_script.sh args # tee
TEE=false your_script.sh args # don't tee
export TEE=false
your_script.sh args # tee
You can customize this, e.g. make tee=false the default instead, make TEE hold the log file instead, etc. I guess this solution is similar to jbarlow's, but simpler, maybe mine has limitations that I have not come across yet.
Neither of these is a perfect solution, but here are a couple things you could try:
exec >foo.log
tail -f foo.log &
# rest of your script
or
PIPE=tmp.fifo
mkfifo $PIPE
exec >$PIPE
tee foo.log <$PIPE &
# rest of your script
rm $PIPE
The second one would leave a pipe file sitting around if something goes wrong with your script, which may or may not be a problem (i.e. maybe you could rm it in the parent shell afterwards).
I run a bash python command as current user, by doing this:
su $USER -c 'python3 -m site --user-site'
This works properly and prints the following:
/Users/standarduser7/Library/Python/3.6/lib/python/site-packages
I want to assign this output to a variable, so I'm using "$(command)":
target="$(su $USER -c 'python3 -m site --user-site')"
At this point, the OSX terminal hangs and has to be killed. Using backticks instead of "$(command)" leads to same result.
However, if I run the command without user, everything works as it should:
target="$(python3 -m site --user-site)"
echo target
output: /Users/standarduser7/Library/Python/3.6/lib/python/site-packages
How can I assign the output from a command run as the current user to a variable?
I don’t think it’s hanging; I think it’s showing a blank (prompt-less) command-line and is waiting for input. When I key in the user password, it returns this result:
Password:/Users/CK/Library/Python/2.7/lib/python/site-packages
and this is what ends up being stored in the target variable. A quick parameter substitution can rectify this anomalous output:
target=${target#*:}
The other solution (credit given to this answer) is to create a file descriptor as a copy of stdout, then tee the command to the copy, which then allows stdout to be piped to grep in order to process the output:
exec 3>&1 # create a copy of stdout
target=$(su $USER -c "python -m site --user-site" | tee /dev/fd/3 | grep -o '\/.*')
exec 3>&- # close copy
I used
exec > /dev/null
to suppress output.
Is there a command to undo this? (Without restarting the script.)
To do it right, you need to copy the original FD 1 somewhere else before repointing it to /dev/null. In this case, I store a backup on FD 5:
exec 5>&1 >/dev/null
...
exec 1>&5
Another option is to redirect stdout within a block rather than using exec:
{
...
} >/dev/null
If you just want to get output again at the command prompt, you can do this:
exec >/dev/tty
If you are creating a script, and you want to have the output of a certain group of commands redirected, put those commands in braces:
{
command
command
} >/dev/null
Save the original output targets beforehand.
# $$ = the PID of the running script instance
STDOUT=`readlink -f /proc/$$/fd/1`
STDERR=`readlink -f /proc/$$/fd/2`
And restore them again using exec.
exec 1>$STDOUT 2>$STDERR
If you use /dev/tty for restoration as in the answers above, contrary to this, you won't be able to do redirections in call-level, e.g. bash script.sh &>/dev/null won't work.
Not really, as that would require changing the state of a running process. Even assuming you could, whatever you wrote before resetting standard output is truly, completely gone, as it was sent to the bit bucket.
To restore stdout I use
unset &1
I'm attempting to write a bash script that uses nohup and passes errors to rsyslog.
I've tried this command with different variations of the log variable (see below) but can't get the output passed to anything but a std txt file. I can't get it to pipe.
nohup imageprocessor.sh > "$LOG" &
Is it possible to pipe nohup output or do I need a different command.
A couple of variations of log that I have tried
LOG="|/usr/bin/logger -t workspaceworker -p LOCAL5.info &2"
or
LOG="|logtosyslog.sh"
or
LOG="logtosyslog.sh"
A way in bash to redirect output to syslog is:
exec > >(logger -t myscript)
stdout is then sent to logger command
exec 2> >(logger -t myscript)
for stderr
Not directly. nohup will detach the child process, so piping the output of the nohup command isn't helpful. This is what you want:
nohup sh -c 'imageprocessor.sh | logger'