For the command ls, the option -1 is supposed to do run ls yielding an output with only one column (yielding one file per line). But if I put it in a script, it just shows every file jammed on one line, separated with spaces.
Script:
#!/bin/ksh
text=`ls -1`
echo $text
Folder contents:
test
|--bob
|--coolDir
|--file
|--notThisDirectoryAgain
|--script.sh
|--spaces are cool
|--thatFile
Script Output:
bob coolDir file notThisDirectoryAgain script.sh spaces are cool thatFile
Output if I run ls -1 in the terminal (not in a script)
bob
coolDir
file
notThisDirectoryAgain
script.sh
spaces are cool
thatFile
it just shows every file jammed on one line, separated with spaces.
You have to consider what it is.
When you do
text=`ls -1`
that runs the program ls and presents the output as if you typed it in. So the shell gets presented with:
ls=file1
file2
file3
etc.
The shell splits command-line tokens on whitespace, which by default includes a space, a tab, and a newline. So each filename is seen as a separate token by the shell.
These tokens are then passed into echo as separate parameters. The echo command is unaware that the newlines were ever there.
As I'm sure you know, all echo does is to write each parameter to stdout, with a space between each one.
This is why the suggestion given by #user5228826 works. Change IFS if you don't want a newline to separate tokens.
However, all you really had to do is to enclose the variable in quotes, so that it didn't get split:
echo "$text"
By the way, using `backticks` is deprecated and poor practice because it can be difficult to read, particularly when nested. If you run ksh -n on your script it will report this to you (assuming you are not using an ancient version). Use:
text=$(ls -1)
Having said all that, this is a terrible way to get a list of files. UNIX shells do globbing, this is an unnecessary use of the ls external program. Try:
text=(*) # Get all the files in current directory into an array
oldIFS="$IFS" # Save the current value of IFS
IFS=$'\n' # Set IFS to a newline
echo "${text[*]}" # Join the elements of the array by newlines, and display
IFS="$oldIFS" # Reset IFS to previous value
That's because you're capturing ls output into a variable. Bash does the same.
Related
I'm learning bash so this is a simple question probably. I'd like to understand what happens in this case, there's no real use for this script. Let DTEST be a directory that contains some random files.
for filename in " `ls DTEST/*` " ; do
touch "$filename".txt
done
So the command substitution happens and the ls output should be file1 file2 file3. Then because the command substitution is double quoted the first line of the for loop should be for filename in "file1 file2 file3". I think it should create only one file named file1 file2 file3.txt.
But I've seen it wants to create a file named something like file1'$'\n''file2'$'\n''file3.txt.
I don't understand where the '$'\n'' come from. I read in bash manual that with double quotes \ followed by special characters like n for newline retains its special meaning, but why \n are generated?
ls DTEST/* outputs each file on a separate line. Also, the output would contain the directory name (i.e. DEST/file1 etc.).
Of course you would never use ls in this way in a real script. Instead you would just use something like
for filename in DEST/*
do
...
done
I want to list all the files in a directory one line after another, for which I am using a sample shell script as follows
#!/bin/sh
MY_VAR="$(ls -1)"
echo "$MY_VAR"
This works out as expected, however if the same is executed using csh as follows:
#!/bin/csh
set MY_VAR = `ls -1`
echo $MY_VAR
it outputs all files in one single line, than printing one file per line.
Can any one explain why in csh ls -1 is not working as expected.
From man csh, emphasis mine:
Command substitution
Command substitution is indicated by a command enclosed in ``'. The
output from such a command is broken into separate words at blanks,
tabs and newlines, and null words are discarded. The output is vari‐
able and command substituted and put in place of the original string.
Command substitutions inside double quotes (`"') retain blanks and
tabs; only newlines force new words. The single final newline does not
force a new word in any case. It is thus possible for a command sub‐
stitution to yield only part of a word, even if the command outputs a
complete line.
By default, the shell since version 6.12 replaces all newline and car‐
riage return characters in the command by spaces. If this is switched
off by unsetting csubstnonl, newlines separate commands as usual.
The entries are assigned in a list; you can access a single entry with e.g. echo $MY_VAR[2].
This is different from the Bourne shell, which doesn't have the concept of a "list" and variables are always strings.
To print all the entries on a single line, use a foreach loop:
#!/bin/csh
set my_var = "`ls -1`"
foreach e ($my_var)
echo "$e"
end
Adding double quotes around the `ls -1` is recommended, as this will make sure it works correctly when you have filenames with a space (otherwise such files would show up as two words/list entries).
I have very simple directory with "directory1" and "file2" in it.
After
out=`ls`
I want to print my variable: echo $out gives:
directory1 file2
but echo "$out" gives:
directory1
file2
so using quotes gives me output with each record on separate line. As we know ls command prints output using single line for all files/dirs (if line is big enough to contain output) so I expected that using double quotes prevents my shell from splitting words to separate lines while ommitting quotes would split them.
Pls tell me: why using quotes (used for prevent word-splitting) suddenly splits output ?
On Behavior Of ls
ls only prints multiple filenames on a single line by default when output is to a TTY. When output is to a pipeline, a file, or similar, then the default is to print one line to a file.
Quoting from the POSIX standard for ls, with emphasis added:
The default format shall be to list one entry per line to standard output; the exceptions are to terminals or when one of the -C, -m, or -x options is specified. If the output is to a terminal, the format is implementation-defined.
Literal Question (Re: Quoting)
It's the very act of splitting your command into separate arguments that causes it to be put on one line! Natively, your value spans multiple lines, so echoing it unmodified (without any splitting) prints it precisely that manner.
The result of your command is something like:
out='directory1
file2'
When you run echo "$out", that exact content is printed. When you run echo $out, by contrast, the behavior is akin to:
echo "directory1" "file2"
...in that the string is split into two elements, each passed as completely different argument to echo, for echo to deal with as it sees fit -- in this case, printing both those arguments on the same line.
On Side Effects Of Word Splitting
Word-splitting may look like it does what you want here, but that's often not the case! Consider some particular issues:
Word-splitting expands glob expressions: If a filename contains a * surrounded by whitespace, that * will be replaced with a list of files in the current directory, leading to duplicate results.
Word-splitting doesn't honor quotes or escaping: If a filename contains whitespace, that internal whitespace can't be distinguished from whitespace separating multiple names. This is closely related to the issues described in BashFAQ #50.
On Reading Directories
See Why you shouldn't parse the output of ls. In short -- in your example of out=`ls`, the out variable (being a string) isn't able to store all possible filenames in a useful, parsable manner.
Consider, for instance, a file created as such:
touch $'hello\nworld"three words here"'
...that filename contains spaces and newlines, and word-splitting won't correctly detect it as a single name in the output from ls. However, you can store and process it in an array:
# create an array of filenames
names=( * )
if ! [[ -e $names || -L $names ]]; then # this tests only the FIRST name
echo "No names matched" >&2 # ...but that's good enough.
else
echo "Found ${#files[#]} files" # print number of filenames
printf '- %q\n' "${names[#]}"
fi
I tried to do something tricky today with bash scripting, which made me question my knowledge of bash scripting.
I have the following script called get_ftypes.sh, where the first input argument is a file containing file globs:
for ftype in `cat $1`
do
echo "this is ftype $ftype"
done
For example, the script would be called like this get_ftypes.sh file_types, and file_types would contain something like this:
*.txt
*.sh
I would expect the echo to print each line in the file, which in this example would be *.txt, *.sh, etc. But, instead it expands the globbing, *, and it echos the actual file names, instead of the globb as I would expect.
Any reason for this behavior? I cannot figure out why. Thank you.
On the line for ftype in `cat $1`, the shell performs both word splitting and pathname expansion. If you don't want that, use a while loop:
while read -r ftype
do
echo "this is ftype $ftype"
done <"$1"
This loop reads one line at a time from the file $1 and, while leading and trailing whitespace are removed from each line, no expansions are performed.
(If you want to keep the leading and trailing whitespace, use while IFS= read -r ftype).
Typically, for loops are useful when you are looping over items that are already shell-defined variables, like for x in "$#". If you are reading something in from an external command or file, you typically want a while read loop.
Alternative not using shell
When processing files line-by-line, the goal can often be accomplished more efficiently using sed or awk. As an example using awk, the above loop simplifies to:
$ awk '{print "this is ftype " $0}' filetypes
this is ftype *.txt
this is ftype *.sh
echo $(cat foo)
will produce the content of foo, split them into words, do globs on each word - i.e. treat the content of foo as parameters - before it interpolates it into the current command line.
echo "$(cat foo)"
will produce the content of foo as a single argument, does not treat them as parameters, will not glob (but you will only get one pass through the loop).
You want to read foo one line at a time; use while read -r ftype for that.
Having some trouble here. I want to capture output from ls command into variable. Then later use that variable and count the number of lines in it. I've tried a few variations
This works, but then if there are NO .txt files, it says the count is 1:
testVar=`ls -1 *.txt`
count=`wc -l <<< $testVar`
echo '$count'
This works for when there are no .txt files, but the count comes up short by 1 when there are .txt files:
testVar=`ls -1 *.txt`
count=`printf '$testVar' | wc -l`
echo '$count'
This variation also says the count is 1 when NO .txt files exist:
testVar=`ls -1 *.txt`
count=`echo '$testVar' | wc -l`
echo '$count'
Edit: I should mention this is korn shell.
The correct approach is to use an array.
# Use ~(N) so that if the match fails, the array is empty instead
# of containing the pattern itself as the single entry.
testVar=( ~(N)*.txt )
count=${#testVar[#]}
This little question actually includes the result of three standard shell gotchas (both bash and korn shell):
Here-strings (<<<...) have a newline added to them if they don't end with a newline. That makes it impossible to send a completely empty input to a command with a here-string.
All trailing newlines are removed from the output of a command used in command substitution (cmd or preferably $(cmd)). So you have no way to know how many blank lines were at the end of the output.
(Not really a shell gotcha, but it comes up a lot). wc -l counts the number of newline characters, not the number of lines. So the last "line" is not counted if it is not terminated with a newline. (A non-empty file which does not end with a newline character is not a Posix-conformant text file. So weird results like this are not unexpected.)
So, when you do:
var=$(cmd)
utility <<<"$var"
The command substitution in the first line removes all trailing newlines, and then the here-string expansion in the second line puts exactly one trailing newline back. That converts an empty output to a single blank line, and otherwise removes blank lines from the end of the output.
So utility is wc -l, then you will get the correct count unless the output was empty, in which case it will be 1 instead of 0.
On the other hand, with
var=$(cmd)
printf %s "$cmd" | utility
The trailing newline(s) are removed, as before, by the command substitution, so the printf leaves the last line (if any) unterminated. Now if utility is wc -l, you'll end up with 0 if the output was empty, but for non-empty files, the count will not include the last line of the output.
One possible shell-independent work-around is to use the second option, but with grep '' as a filter:
var=$(cmd)
printf %s "${var}" | grep '' | utility
The empty pattern '' will match every line, and grep always terminates every line of output. (Of course, this still won't count blank lines at the end of the output.)
Having said all that, it is always a bad idea to try to parse the output of ls, even just to count the number of files. (A filename might include a newline character, for example.) So it would be better to use a glob expansion combined with some shell-specific way of counting the number of objects in the glob expansion (and some other shell-specific way of detecting when no file matches the glob).
I was going to suggest this, which is a construct I've used in bash:
f=($(</path/to/file))
echo ${#f[#]}
To handle multiple files, you'd just .. add files.
f=($(</path/to/file))
f=+($(</path/to/otherfile))
or
f=($(</path/to/file) $(</path/to/otherfile))
To handle lots of files, you could loop:
f=()
for file in *.txt; do
f+=($(<$file))
done
Then I saw chepner's response, which I gather is more korn-y than mine.
NOTE: loops are better than parsing ls.
You can also use like this:
#!/bin/bash
testVar=`ls -1 *.txt`
if [ -z "$testVar" ]; then
# Empty
count=0
else
# Not Empty
count=`wc -l <<< "$testVar"`
fi
echo "Count : $count"