This question already has answers here:
How to store the output of a command in a variable at the same time as printing the output?
(4 answers)
Closed 2 years ago.
When I run the following script, output of ls -la is stored to variable $output.
#! /bin/sh
output=$(ls -la)
How can I pipe output of ls -la to stdout and $output?
I am asking in the context of running borgbackup which can output for a long time during backups.
I would like to be able to track progress when I manually run script while storing output in $output to send it to sysadmin via email.
Use tee;
output=$(ls -lta | tee /dev/tty)
Another way of doing this is by creating a copy of STDOUT, and again using tee to send the output there;
# Create copy of stdout
exec 3>&1
# Run command
output=$(ls -lta | tee /dev/fd/3)
# Close copy
exec 3>&-
Use tee util and pass it to /dev/tty to print stdout.
box: ~/demo
➜ out=$(ls -la | tee /dev/tty)
total 16
drwxr-xr-x 4 chen staff 128 Feb 15 15:59 .
drwxr-xr-x+ 103 chen staff 3296 Feb 15 15:59 ..
-rw-r--r-- 1 chen staff 141 Sep 1 12:38 docker-compose.yml
-rw-r--r-- 1 chen staff 84 Sep 1 12:31 ubuntu.Dockerfile
box: ~/demo
➜ echo $out
total 16
drwxr-xr-x 4 chen staff 128 Feb 15 15:59 .
drwxr-xr-x+ 103 chen staff 3296 Feb 15 15:59 ..
-rw-r--r-- 1 chen staff 141 Sep 1 12:38 docker-compose.yml
-rw-r--r-- 1 chen staff 84 Sep 1 12:31 ubuntu.Dockerfile
Related
Let's say I run a command in Bash like:
ls -l | grep filename
How can I save output of both command to a variable? In another words output of "ls -l" and "grep filename" to the same variable?
The command must use a pipe.
Thank you in advance!
combinedOutput = ""
ls -l
total 4
-rw-r--r-- 1 webmaster webmaster 0 Nov 16 20:34 a
-rw-r--r-- 1 webmaster webmaster 0 Nov 16 20:34 b
-rw-r--r-- 1 webmaster webmaster 0 Nov 16 20:34 file1.txt
-rw-r--r-- 1 webmaster webmaster 0 Nov 16 20:34 file2.txt
-rw-r--r-- 1 webmaster webmaster 5 Nov 16 20:34 main.sh
ls -l | grep main.sh
-rw-r--r-- 1 webmaster webmaster 20 Nov 16 20:35 main.sh
echo $combinedOutput
total 4
-rw-r--r-- 1 webmaster webmaster 0 Nov 16 20:34 a
-rw-r--r-- 1 webmaster webmaster 0 Nov 16 20:34 b
-rw-r--r-- 1 webmaster webmaster 0 Nov 16 20:34 file1.txt
-rw-r--r-- 1 webmaster webmaster 0 Nov 16 20:34 file2.txt
-rw-r--r-- 1 webmaster webmaster 5 Nov 16 20:34 main.sh
-rw-r--r-- 1 webmaster webmaster 20 Nov 16 20:35 main.sh
UPDATE #1:
A better example: let's say I am trying to archive and compress a directory using the following command:
tar cvf - /some/directory/ | pigz --verbose -1 -p 4 >compressed_archive.tgz;
The question is how to put outputs of "tar cvf - /some/directory/" and "pigz --verbose -1 -p 4 >compressed_archive.tgz" to a variable.
Simply:
combinedOutput=$(( ls -l | tee /dev/stderr | grep filename ) 2>&1)
echo "$combinedOutput"
Just run two commands.
combinedOutput=$(
ls -l
ls -l | grep filename
)
But anyway, you may be more comfortable with just:
one=$(ls -l)
two=$(ls -l | grep filename)
combinedOutput="$one
$two"
It's not possible to get the output you want with echo $combinedOutput. You can do echo "$combinedOutput". Consider researching shell quoting. Check your scripts with shellcheck.
I was toying with the idea of creating an alias that would allow me to list the contents of a target subdirectory, without changing to that directory.
I have successfully been able to create both an alias and a script to change directory and display contents. I call the script leap and it is simply:
#!/bin/bash
PATH=/bin:/usr/bin:.
# script to change to a directory, then display current contents
cd $1 && ls -l -a
The alias I use to trigger leap is defined: alias lp='. ~/scripts/leap'
My hope was I could simply create an alias named pk (pk is for peek) and concatenate my leap script and a standard bash command using &&. I could not be more wrong.
Current Directory
For reference, here is the contents of my current home directory (my user id has been replaced with $$$$):
drwxr-xr-x 3 $$$$$$ 46374 23 Aug 30 11:40 Fall_2019
drwxr-xr-x 5 $$$$$$ 46374 66 Aug 28 09:01 PAST_COURSES
drwxr-xr-x 3 $$$$$$ 46374 22 Aug 30 12:03 repos
drwxr-xr-x 3 $$$$$$ students 117 Aug 31 09:06 scripts
The Attempt(s)
Using alias pk='lp $1 && cd ..'
Entering [$$$$#host ~]$ pk PAST_COURSES results in:
-rw------- 1 $$$$$$ students 1766 Feb 28 2018 ~
drwx------ 10 $$$$$$ students 4096 Aug 31 09:06 .
drwx--x--x 1232 root root 28672 Aug 30 16:03 ..
-rw------- 1 $$$$$$ students 11368 Aug 30 12:20 .bash_history
-rw------- 1 $$$$$$ students 18 Aug 21 2017 .bash_logout
-rw------- 1 $$$$$$ students 180 Mar 8 2018 .bash_profile
-rw------- 1 $$$$$$ students 526 Aug 30 11:19 .bashrc
-rw------- 1 $$$$$$ students 266 Aug 21 2017 .cshrc
drwxr-xr-x 3 $$$$$$ 46374 23 Aug 30 11:40 Fall_2019
drwxr-xr-x 8 $$$$$$ students 155 Aug 30 12:14 .git
-rw-r--r-- 1 $$$$$$ students 87 Apr 11 2018 .gitconfig
-rw-r--r-- 1 $$$$$$ students 12288 Jan 29 2018 .hello.swp
-rw------- 1 $$$$$$ students 172 Aug 21 2017 .kshrc
-rw------- 1 $$$$$$ students 189 Mar 13 2018 .lesshst
-rw-r--r-- 1 $$$$$$ students 20480 Jan 29 2018 .ls.swn
drwxr-xr-x 5 $$$$$$ 46374 66 Aug 28 09:01 PAST_COURSES
drwxr----- 3 $$$$$$ 46374 18 Aug 30 12:16 .pki
drwxr-xr-x 3 $$$$$$ 46374 22 Aug 30 12:03 repos
drwxr-xr-x 3 $$$$$$ students 117 Aug 31 09:06 scripts
drwx------ 3 $$$$$$ students 103 Aug 30 11:12 .ssh
-rw------- 1 $$$$$$ students 12288 Sep 6 2017 .swp
drwxr-xr-x 2 $$$$$$ 46374 23 Aug 31 09:06 .vim
-rw-r--r-- 1 $$$$$$ students 8129 Aug 31 09:06 .viminfo
-rw-r--r-- 1 $$$$$$ students 142 Feb 14 2018 .vimrc
[$$$$#host home]$
As you can see, this displays the current directory ( ~ ) rather than switching to PAST_COURSES and displaying it. Additionally, the alias jumps up one directory above the current directory ( ~ ) rather than returning to it from PAST_COURSES.
Incidentally, I also get this exact result when I try using the following aliases:
pk='. ~/scripts/leap $1 && cd ..'
(using the script for leap rather than the alias)
pk='cd $1 && ls -l -a && cd ..'
(using the exact code inside leap )
Findings
In my tinkering I have noticed a few things:
First, if I simply type $$$$#host ~]$ ~/scripts/leap *[dir-name]* I actually get EXACTLY what I want - a look into a directory without changing to it. All by omitting the leading ., which boggles me.
Second, I can fix the current pk alias by changing the trailing cd .. to cd $(pwd), though it will not display the target directory instead of the current one.
At this point, I'd like a little help - not just in a script or alias that will do the job. An explanation that explains this behavior that I'm seeing would be marvelous.
Don't use . (the source builtin)
alias pk='. ~/scripts/leap $1 && cd ..' (using the script for leap rather than the alias)
is not equivalent to
alias pk='cd $1 && ls -l -a && cd ..'
In the first one, the . builtin (also known as source) is used, while in the second it is not. . doesn't just execute a command, it executes it in the current shell context. From the documentation:
. (a period)
. filename [arguments]
Read and execute commands from the filename argument in the current shell context.
That means anything the script does effects your current shell context. If the script changes directories, so does your current context.
If, on the other hand, the first version omitted the . like this:
alias pk='~/scripts/leap $1 && cd ..'
then the contents of the leap script would run in it's own bash context, but your current context would move up one directory (since the cd .. isn't inside the leap script).
Additional Recommendation on Functions vs Aliases
You could implement pk using a function like this:
pk() {
pushd $1
if [[ $? == 0 ]]; then
# Successfully changed directories.
# Run command
$2
# Return from the pre-pushd directory.
popd
fi
}
From the Bash Manual | 6.6 Aliases:
For almost every purpose, shell functions are preferred over aliases.
Example Alias
alias foo="echo bar"
Equivalent Function
foo() {
echo bar
}
I have this small script:
#!/bin/bash
for file in "$(ls | grep -v $0)";do
cat $file > "${file}-test"
done
On this directory:
total 40
-rwxr-xr-x 1 root root 783 Dec 11 09:19 appendToLog.sh
-rwxr-xr-x 1 root root 3995 Dec 11 13:22 con2dd.py
-rwxr-xr-x 1 root root 362 Dec 11 13:26 dd.py
-rw-r--r-- 1 root root 566 Dec 11 13:26 dd.pyc
-rw-r--r-- 1 root root 18558 Dec 25 11:24 moshe.log
-rw------- 1 root root 0 Dec 11 09:20 nohup.out
-rwxr-xr-x 1 root root 88 Dec 25 11:28 task.sh
-rwxr-xr-x 1 root root 560 Dec 11 10:33 test.py
Nevermind that I can achieve that with cp, I want to understand why this exactly is producing this file:
-rw-r--r-- 1 root root 24912 Dec 25 11:28 appendToLog.sh?con2dd.py?dd.py?dd.pyc?moshe.log?nohup.out?task.sh?test.py-test
And nothing else.
The problem is parsing output of ls is just wrong (see Why you shouldn't parse the output of ls(1), filenames in unix can have almost any special characters including whitespace, newlines, commas, pipe symbols. Its because you've quoted the output of ls in one construct, you have a single list of all the files concatenated as one string in the value of "${file}-test" which is quite not what you wanted to do.
Also notice how ls sometimes garbles your filename data (in our case, it turned the \n character in between the words into a ? question mark (could indicate a character that cannot be displayed).
Just use the glob expansion in bash to list the files and do actions on them.
for f in *; do
[[ -e $f ]] || continue
...
done
That said, You could probably have some non-printable characters on end of lines (eg. CRLF imported rom Windows)
Run cat -A scriptname it'll show you all characters in your script. Then, you can convert to unix-like format running dos2unix scriptname.
I'm a little bit confused, i changed my time on one file with the shell command :
touch -t = touch -t 201606012135 trial01
But after the ls -lt, I got this :
-rw-r--r-- 1 CharleyD staff 87 1 jun 2016 trial01
drwxr-xr-x 15 CharleyD staff 510 3 apr 12:57 Hybrid_proj
Why the shell doesn't write the hours like the "Hybrid_proj" directory for the "trial01" ? The trial01 file have the hour : 21:35, so itsn't empty.
Indeed, I search to get this in output :
-rw-r--r-- 1 CharleyD staff 87 1 jun 21:35 trial01
drwxr-xr-x 15 CharleyD staff 510 3 apr 12:57 Hybrid_proj
How I can do this ?
Thx a lot buddies to enlighten my way ! ;)
If a file is not from the current year, ls defaults to showing the year instead of the time. The time is still correctly set, just formatted differently.
To always show the full time, with GNU ls, you can use ls --time-style=long-iso -l:
$ ls --time-style=full-iso -l
total 0
-rw-r--r-- 1 user user 0 2017-04-04 13:20 newfile
-rw-r--r-- 1 user user 0 2016-04-03 12:34 oldfile
With BusyBox ls, you can use -e:
$ busybox ls -e
total 0
-rw-r--r-- 1 user user 0 Tue Apr 4 13:20:42 2017 newfile
-rw-r--r-- 1 user user 0 Sun Apr 3 12:34:00 2016 oldfile
With macOS ls, you can use -lT:
$ ls -lT
total 0
-rw-r--r-- 1 user group 0 Apr 4 13:19:35 2017 myfile
-rw-r--r-- 1 user group 0 Apr 3 12:34:00 2016 oldfile
In each case, you get a long timestamp with the same format for older and newer files.
Use the -T option if your ls supports it.
What does the command cp $1/. $2 do? I know cp is used for copying from source(stored in variable $1) to destination(stored in variable $2). I am just confused with the /. used along with the variable. Can someone please help me understand this?
The command:
$ cp -R $1/. $2
copies contents of directory pointed by $1 to the directory $2.
Without -R switch this command would fail both when $1 is a file or directory.
In general, . points to the current directory. You can see that by comparing inode's shown by ls:
$ mkdir test
$ ls -ali
9525121 drwxr-xr-x 3 IU wheel 102 23 mar 12:31 .
771046 drwxrwxrwt 21 root wheel 714 23 mar 12:30 ..
9525312 drwxr-xr-x 2 IU wheel 68 23 mar 12:31 test
$ cd test
$ ls -ali
9525312 drwxr-xr-x 2 IU wheel 68 23 mar 12:31 .
9525121 drwxr-xr-x 3 IU wheel 102 23 mar 12:31 ..
Note that inode 9525312 points to test when viewed from the parent directory, and points to . when viewed from inside the test directory.