I am not a great bash scripter and hence have a few questions. One of which is how (or even whether) bash understands that a variable is a "file" or simply a local variable.
file=/usr/share/lib
Obviously this is a file to be saved, etc and can be used like so:
echo "$output" > $file
To save the output of $output to $file.
But where in bash does it calculate whether it's a file or not, is it only a file once it's been passed to a 'writing method'?
If you treat that variable as a file name, bash will simply do what it's told e.g.
echo "Test output" > $file
will work regardless of file being set to /tmp/myfile.txt, or to abcd. In the above you're using bash's file redirection to write out the standard out to the file you've named.
Consequently if you use the wrong variable in the above pattern, or have the value set incorrectly, bash will simply follow your instructions and you may end up with incorrectly named/located files.
You need to check this yourself, bash is only aware of the contents of the variable. If you want to check if a file location is held within a variable, you can test for it using the -f test operator
if [ -f "$file" ]
then
echo "$file is a file"
echo "$output" > "$file"
else
echo "$file is not a file"
fi
Variables are strings - nothing more, nothing less. (I'm ignoring array variables here, WLOG).
Bash (or any shell) expands variables blindly, without considering what you intend to do with them. It's only in the next stage of command processing that the contents of the string matter.
To use your example:
output="foo bar"
file=/usr/share/lib
echo "$output" >"$file"
(I've quoted $file even though it's not necessary here, simply because I've been bitten too many times by changing the value and breaking everything).
The line
echo "$output" >"$file"
gets transformed into
echo "foo bar" >"/usr/share/lib"
and only then does bash consider the > and attempt to open /usr/share/lib for writing.
Related
(This question is a follow-up on this comment, in an answer about git hooks)
I'm far too unskilled in bash (so far) to understand fully the remark and how to act accordingly. More specifically, I've been advised to avoid using bash command cat this way :
echo "$current_branch" $(cat "$1") > "$1"
because the order of operations depends on the specific shell and it could end up destroying the contents of the passed argument, so the commit message itself if I got it right?
Also, how to "save the contents in a separate step"?
Would the following make any sense?
tmp = "$1"
echo "$current_branch" $(cat $tmp) > "$1"
The proposed issue is not about overwriting variables or arguments, but about the fact that both reading from and writing to a file at the same time is generally a bad idea.
For example, this command may look like it will just write a file to itself, but instead it truncates it:
cat myfile > myfile # Truncates the file to size 0
However, this is not a problem in your specific command. It is guaranteed to work in a POSIX compliant shell because the order of operations specify that redirections will happen after expansions:
The words that are not variable assignments or redirections shall be expanded. If any fields remain following their expansion, the first field shall be considered the command name and remaining fields are the arguments for the command.
Redirections shall be performed as described in Redirection.
Double-however, it's still a bit fragile in the sense that seemingly harmless modifications may trigger the problem, such as if you wanted to run sed on the result. Since the redirection (> "$1") and command substitution $(cat "$1") are now in separate commands, the POSIX definition no longer saves you:
# Command may now randomly result in the original message being deleted
echo "$current_branch $(cat "$1")" | sed -e 's/(c)/©/g' > "$1"
Similarly, if you refactor it into a function, it will also suddenly stop working:
# Command will now always delete the original message
modify_message() {
echo "$current_branch $(cat "$1")"
}
modify_message "$1" > "$1"
You can avoid this by writing to a temporary file, and then replace your original.
tmp=$(mktemp) || exit
echo "$current_branch $(cat "$1")" > "$tmp"
mv "$tmp" "$1"
In my opinion, it's better to save to another file.
You may try something like
echo "$current_branch" > tmp
cat "$1" >> tmp # merge these into
# echo "$current_branch" $(cat "$1") > tmp
# may both OK
mv tmp "$1"
However I am not sure if my understanding is right, or there are some better solutions.
This is what I considered as the core of question. It is hard to decide the "precedence" of $() block and >. If > is executed "earlier", then echo "$current_branch" will rewrite "$1" file and drop the original content of "$1", which is a disaster. If $() is executed "earlier", then everything works as expected. However, there exists a risk, and we should avoid it.
A command group would be far better than a command substitution here. Note the similarity to Geno Chen's answer.
{
echo "$current_branch"
cat "$1"
} > tmp && mv tmp "$1"
I only know what I have read so far, and I am confused about how to actually echo a variable as is.
echo "$var" might fail if var='-n'
printf '%s\n' "$var" might fail because of shell not implementig printf
echo -- "$var" might fail because it is a gnu extension
So if i would have to guess:
echo x"$var"|sed 's#^x##1' would be the only way, but I have never encountered that pattern. Why?
As a concrete question:
for source; do
target="$(echo "$source"|sed 's#[^a-z0-9]\+#.#')"
# do stuff with $source and $target
done
Does this work, or could someone "hack" / "break" my script by putting a file named '-n' somewhere, assuming my script is executed by some my_script * cron?
How do I write echo "$var" so it does not break?
Does this work, or could someone "hack" / "break" my script by putting
a file named '-n' somewhere?
There is nothing wrong with:
target="$(echo "$source"|sed 's#[^a-z0-9]\+#.#')"
What is happening:
"$(...)" is a command substitution which will substitute the results of the command within as the value -- in which case the result is assigned to target.
echo "$source"|sed 's#[^a-z0-9]\+#.#' simply pipes the output of echo (e.g. what is in source) to sed for the simple substitution of every character not lowercase or a digit followed by + with a period 1. Note: the quotes ".." around $source ARE proper within the command substitution.
There is no inherent reason assigning -n to a variable will cause any mischief. What you do with the variable is another question, but suffice it to say it is hard to see any problem.
"POSIX-shell's out there not implementing printf" -- Huh? Any shell not implementing printf would be more an exception rather than the rule. See printf - The Open Group Library that is POSIX.
If you are attempting to printf output that begins with '-' simply precede the output with "--" to indicate End-of-Options before the string your want to print and things will go fine. With your example of "-n", printf is about the only way you will output a variable beginning with the single '-', for example:
$ t="-n"
$ printf -- "%s\n" "$t"
-n
(note: you don't have to include "--" in printf "%s\n" "$var", the only time you must include it is with printf -- "-foo\n" or you will receive an "invalid option error".
For echo you can enable interpretation of backslash escapes with -e and include a backspace, e.g.
$ echo -e " \b$t"
-n
I think that has covered all issues. If not, let me know. Also, if you have any additional questions, drop a comment below or edit and add to your question.
footnotes:
note: + isn't part of basic regular expressions and it need not be escaped, but if there is any question, it is safer to include in a character class of its own, e.g. [^a-z0-9][+].
First of all, hi to everyone, that's my first post here.
I swear I have checked the site for similar questions to avoid the "double post about same argument" issue but none of them answered exactly to my question.
The problem is that in the code below I always get the "There are no files with this extension" message when I call the script passing it an extension as first argument.
#!/bin/bash
if [ "$1" ];
then
file=*."$1";
if [ -f "$file" ];
then
for i in "$file";
[...do something with the each file using "$i" like echo "$i"]
else
echo "There are no files with this extension";
fi;
else
echo "You have to pass an extension"
fi;
I tried using the double parenthesis, using and not using the quotes in the nested if, using *."$1" directly in the if, but none of this solution worked.
One problem is that you're not quoting a variable when you first assign a value to file. In this statement:
file=*."$1";
The * will be interpreted by the shell, so for example if you passed in .py on the command line, file might end up with the value file1.py file2.py, which will throw off your file existence test later on.
Another problem, as #sideshowbarker points out, is that you can't use wildcards with the [ -f ... ].
Another variable quoting issue is that quoting inhibits wildcard expansion, such that even without the file existence test, if $file is, e.g., *.txt, then this:
for x in "$file"; do ...
Will loop over a single argument with the literal value *.txt, while this:
for x in $file; do ...
Will loop over all files that end with a .txt extension (unless there are none, in which case it will loop once with $x set to the literal value *.txt).
Typically, you would write your script to expect a list of arguments, and allow the user to call it like myscript *.txt...that is, leave wildcard handling to the interactive shell, and just let your script process a list of arguments. Then it becomes simply:
for i in "$#"; do
echo do something with this file named "$x"
done
If you really want to handle the wildcard expansion in your script, something like this might work:
#!/bin/bash
if [ "$1" ];
then
ext="$1"
for file in *.$ext; do
[ -f "$file" ] || continue
echo $file
done
else
echo "You have to pass an extension"
fi;
The statement [ -f "$file" ] || continue is necessary there because of the case I mentioned earlier: if there are no files, the loop will still execute once with the literal expansion of *.$ext.
How do I use variables with dot in a Unix (Bash) shell script? I have the script below:
#!/bin/bash
set -x
"FILE=system.properties"
FILE=$1
echo $1
if [ -f "$FILE" ];
then
echo "File $FILE exists"
else
echo "File $FILE does not exist"
fi
This is basically what I need ⟶ x=propertyfile, and propertyfile=$1. Can someone please help me?
You can't declare variable names with dots but you can use associative arrays to map keys, which is the more appropriate solution. This requires Bash 4.0.
declare -A FILE ## Declare variable as an associative array.
FILE[system.properties]="somefile" ## Assign a value.
echo "${FILE[system.properties]}" ## Access the value.
Note that the line:
"FILE=system.properties"
tries to execute a command FILE=system.properties which most likely doesn't exist. To assign to a variable, the quote must come after the equals:
FILE="system.properties"
It is a bit hard to tell from the question what you are after, but it sounds as if you might be after indirect variable names. Unfortunately, standard editions of bash don't allow dots in variable names.
However, if you used an underscore instead, then:
FILE="system_properties"
system_properties="$1"
echo "${FILE}"
echo "${!FILE}"
will echo:
system_properties
what-was-passed-as-the-first-argument
I've written a small script in bash that parses either the provided files or stdin if no file is given to produce some output. What is the best way to redirect the parsed output to stdout (at the end of the script the result is stored in a variable). Should I use cat or echo, or is there another preferred method?
Use the printf command:
printf '%s\n' "$var"
echo is ok for simple cases, but it can behave oddly for certain arguments. For example, echo has a -n option that tells it not to print a newline. If $var happens to be -n, then
echo "$var"
won't print anything. And there are a number of different versions of echo (either built into various shells or as /bin/echo) with subtly different behaviors.
echo. You have your parsed data in a variable, so just echo "$var" should be fine. cat is used to print the contents of files, which isn't what you want here.
echo is a fine way to do it. You will have to jump through a few hoops if you want cat to work.