I am trying to write a program in bash which takes as input argument a file name and then takes every line of that file and appends it to a string. At the end, I want to print this string. This is what I wrote (consider that the program name is concat.sh):
#!/bin/bash
FILE=$1
STR=""
cat $FILE | while read USER; do
STR="${STR}${USER}"
done
echo "$STR"
And I have the file stuff.txt, which has the following:
a
b
c
And after I run ./concat.sh stuff.txt I get the empty string. I tried multiple variations of that concatenation, with space, without space (like above), with newline, etc. Still doesn't work. If I try to simply print each line, it works. So if I simply add the line echo "$USER" inside the loop I get the correct output, i.e., a, b, c (each on a different line). But that string concatenation still doesn't work. It doesn't work even if I stop using that $USER variable in the concatenation. So if I do something like STR="${STR} abc" the characters abc are surprisingly not concatenated to STR. But if I take this outside of the while loop it actually works (so if I comment the while loop and simply do STR="${STR} abc" I will get abc in the string STR). I am a newbie in bash and this looks like really weird behaviour to me and have no idea what is going on/how to fix it.
Just do not use cat - ie. do not use pipe. And do not use USER.
while read var; do
str="${str}${var}"
done < "$file"
Do not use upper case variables in your scripts. USER is variable set by bash to the name of current user.
Check scripts with http://shellcheck.net
Read https://mywiki.wooledge.org/BashFAQ/024
Read https://mywiki.wooledge.org/BashFAQ/001
Do not uselessly use cat.
In bash you can also use str+="$var".
Quote variable expansions.
Related
I am trying to use sed to use as input for a variable. The user will choose from a list of files that have numbers before each to identify individual files. Then they choose a number corresponding to a name. I need to get the name of that file. My code is:
for entry in *; do
((i++))
echo "$i) $entry: "
done
echo What file # do you want to choose?:
read filenum
fileName=$(./myscript.sh | sed -n "${filenum}p")
echo $fileName ###this is to see if anything goes into fileName. nothing is ever output
echo What do you want to do with $fileName?
Ideally I would use () instead of the backtick but I can't seem to figure out how. I've looked at the links below, but can't get those ideas to work. I believe a problem may be that I am trying to include the filenum variable inside my sed.
https://www.linuxquestions.org/questions/linux-newbie-8/storing-output-of-sed-in-a-variable-in-shell-script-499997/
Store output of sed into a variable
Don't put backticks around $filenum. That will try to execute the contents of $filenum as a command. Put variables inside double quotes.
And if you do want to nest a backtick expression inside another set of backticks, you have to escape them. That's where $() becomes useful -- they nest without any hassle.
When you use sed -n, you need to use the p command to print the lines that you want to show in the output.
fileName=$(sed -n "${filenum}p" myscript.sh)
This will put the contents of line $filenum of myscript.sh in the variable.
If you actually wanted to execute myscript.sh and print the selected line of its output, you need to pipe to sed:
fileName=$(./myscript.sh | sed -n "${filenum}p")
I am using read to make some file in a while loop. The code is:
while read var val
do
sed -i 's/$var/$val/g' Hhh300_4L_gen.sh
echo $var $val
done < "Hhh300_4L_config.txt"
Where in Hhh300_4L_config.txt, there is a line, for instance,
PROCESSNAME Hhh;
and in Hhh300_4L_gen.sh, there is one element: PROCESSNAME. So if it works, PROCESSNAME in Hhh300_4L_gen.sh should be replaced by Hhh. But it doesn't. However the output of echo prints correctly.
Variables are not expanded inside single quotes. If you want var and val to expand you need double quotes (and make sure they don't have the sed separator, here / in them):
sed -i "s/$var/$val/g" Hhh300_4L_gen.sh
though if you're modifying a shell script (as I'm guessing might be from the .sh) there are probably better ways to do it, like having your .txt file store things as var=val instead of with white space separating them, then just source it from the script.
I was looking to try and figure out how trim a string in Bash, from the trailing end, once I hit a certain character.
Example: if my string is this (or any link): https://www.cnpp.usda.gov/Innovations/DataSource/MyFoodapediaData.zip
(I'll set that as my variable).
(I.e. if I echo $var it will return that link:)
I'm looking to use Bash, I'm guessing I will need to utilize sed or awk, but I want to trim, starting from the end until I see the first / (since the will be the file name) and strip that out.
So using that link, I'm trying to just get after the / so jus "MyFoodapediaData.zip" and set that to a different variable.
So in the end, if I echo $var2 (if I call it that) it will just return: MyFoodapediaData.zip"
I tried working with sed 's.*/" and that will start from the beginning until it finds the first slash. I was looking for the reverse order if possible.
You can use bash builtin parameter substitution for this:
$ var='https://www.cnpp.usda.gov/Innovations/DataSource/MyFoodapediaData.zip'
$ echo "$var"
https://www.cnpp.usda.gov/Innovations/DataSource/MyFoodapediaData.zip
$ var2=${var##*/}
$ echo "$var2"
MyFoodapediaData.zip
${var##*/} means "from the beginning of the value of the var variable, remove everything up to the last slash."
See parameter substitution in the manual
echo $fbname | awk -F'[__]' '{print $2 $A_name = $2}'
echo $A_name
I am trying to extract a name within the fbname variable like for example,
newlist_sammy_card.csv So I am trying to get the name sammy which is between the two underscores and assign it to a variable I can use for rest of the script.
The first line prints out sammy, which is what I need, but the second line does not.
Can anyone show me where I am not assinging the variable correctly?
There is a fundamental flaw in your understanding and reasoning. If you invoke awk in your script it is spawned as a program in its own individual right. Therefore all the variables that exist in your current shell are not available to awk and vice versa. As such you can not 'define' variables in awk that are then visible to your shell. What you should do is 'capture' the output of awk, by using the notation $(), and assign it to a variable. Consider this example:
var=$(awk '{print "test"}')
echo $var
This will output
test
Now in your case, we are actually facing an xy-problem. You want to extract sammy from the string newlist_sammy_card.csv and use that as a variable. One possible solution in pure bash is the following:
name="newlist_sammy_card.csv"
temp=${name#*_}
var=${temp%%_*}
echo $var
This will output
sammy
There's a LOT of ways to do what you're asking for, e.g. here's a couple more in addition to the other ideas you've received so far:
$ fbname='newlist_sammy_card.csv'
$ A_name=$(echo "$fbname" | cut -d_ -f2)
$ echo "$A_name"
sammy
or:
$ IFS=_
$ set -- $fbname
$ A_name="$2"
$ echo "$A_name"
sammy
but I wonder if you're approaching your problem completely the wrong way. We can't tell without more info on what you're trying to do though.
You can simply use bash:
str1="newlist_sammy_card.csv"
# replace 'newlist_' from front of the string
str2=${str1#*_}
# replace '_card.csv' from back of the string:
str2=${str2%%_*}
echo "$str2" # Output: sammy
Unfortunately it can't be done in a single run in bash. However it should still perform a lot better than launching any kind of external program.
Pankrates's answer explains the problem with the OP's approach well and offers a pure shell solution using shell parameter expansion.
Here's another pure shell solution, using a single command based on the read builtin:
Using bash, with a here-string:
IFS=_ read -r _ A_name _ <<<"$fbname"
POSIX-compliant equivalent, using a here-doc:
IFS=_ read -r _ A_name _ <<EOF
$fbname
EOF
If $fbname contains 'newlist_sammy_card.csv', $A_name will contain 'sammy' afterward.
IFS=_ tells read to split the input into tokens by _ instances.
Note that by directly prepending IFS=... to read, the effect of setting $IFS is localized to the read command - no need to restore the original $IFS later.
read -r _ A_name _ ... reads input unmodified (-r - no interpretation of backslash escape sequences)
Note that _ on either side of A_name is the - customary - name of a dummy variable that is used for input that is of no interest, but is needed to positionally extract the token of interest.
It is a mere coincidence in this case that the name of this dummy variable is the same as the $IFS character.
In this case: $_ receives the 1st field (before the first _ char. in the input), and is then overwritten with any remaining fields after the 2nd field, where the 2nd field is read into the variable of interest, $A_name.
I need to write a bash function which, given a string that represents a command line, returns just the first token in the command line (i.e. the program being called, which may have spaces in its name), dropping any arguments. I want to do this without using sed or awk or anything but bash builtins and variable manipulation.
e.g.:
drop_args "ls" # prints "ls"
drop_args "ls -al" # prints "ls"
drop_args "spaces\ in\ name --bad-idea" # prints "spaces\ in\ name"
What I've tried is:
drop_args () { echo $1; }
and then I call drop_args ls -al, i.e. without quoting the string, and that works nicely for all cases that I can see except for drop_args spaces\ in\ name.
I'm not too terribly concerned if I can't do this in a way that correctly handles the spaces case with the restrictions I have stipulated, but I need to at least reliably detect that situation and display an appropriate error message, I guess.
This is related to my earlier question about dereferencing shell aliases; it's sort of a subproblem of what I am ultimately trying to accomplish there.
drop_args()
{
read cmd args <<< "$1"
echo "$cmd"
}
This loses the backslashes, but they were never really meant to be "there" in the first place.