I am trying to use sed to use as input for a variable. The user will choose from a list of files that have numbers before each to identify individual files. Then they choose a number corresponding to a name. I need to get the name of that file. My code is:
for entry in *; do
((i++))
echo "$i) $entry: "
done
echo What file # do you want to choose?:
read filenum
fileName=$(./myscript.sh | sed -n "${filenum}p")
echo $fileName ###this is to see if anything goes into fileName. nothing is ever output
echo What do you want to do with $fileName?
Ideally I would use () instead of the backtick but I can't seem to figure out how. I've looked at the links below, but can't get those ideas to work. I believe a problem may be that I am trying to include the filenum variable inside my sed.
https://www.linuxquestions.org/questions/linux-newbie-8/storing-output-of-sed-in-a-variable-in-shell-script-499997/
Store output of sed into a variable
Don't put backticks around $filenum. That will try to execute the contents of $filenum as a command. Put variables inside double quotes.
And if you do want to nest a backtick expression inside another set of backticks, you have to escape them. That's where $() becomes useful -- they nest without any hassle.
When you use sed -n, you need to use the p command to print the lines that you want to show in the output.
fileName=$(sed -n "${filenum}p" myscript.sh)
This will put the contents of line $filenum of myscript.sh in the variable.
If you actually wanted to execute myscript.sh and print the selected line of its output, you need to pipe to sed:
fileName=$(./myscript.sh | sed -n "${filenum}p")
Related
I am new to shell, I have a case where I am trying to evaluate a particular column unique values to check if they are valid in a shell script which will be invoked later.
From my searches I think cut along with sort & unique is good to do it
So my attempt is
file=/filepath/*vendor.csv
file_categories = `cut -d, -f1 $file |sort |unique`
$file should hold file which has vendor in its filename
but even after using command substitution (`) the $file is not getting replaced with the correct filename , it just places what is present in file
Another example for what I am attempting is
a=/stage/scripts/vendor/*.out
echo $a
/stage/Scripts/ecommerce/oneclick/nohup.out /stage/Scripts/ecommerce/Vendor/Vendor_Automate_Ingestion_Process.out
wc-l
wc: /stage/Scripts/ecommerce/vendor/*.out:
$(wc -l "$a")
wc: /stage/Scripts/ecommerce/vendor/*.out:No such file or directory
I want to understand how we can pass wild characters in command substitution and what I can do to rectify.
No, file will contain the literal string with the wildcard. When you interpolate the value $file without quotes around it, that's when the shell evaluates it as a wildcard. echo "$file" with proper quoting shows you the actual value of the variable.
There is no good way to store a list of file names in a regular shell variable. Ksh and some other shells have arrays for this purpose, but it's not portable back to generic sh and may be something else than what you actually need, depending on what end goal you are trying to accomplish. If you want to extract unique values from a field in the files matching the wildcard into a string, just make sure you don't have spaces around the equals sign in the assignment and you're done.
file_categories=$(cut -d, -f1 $file | sort -u)
# ^ no spaces around the equals sign!
Storing the wildcard in a variable is dubious here; probably simply use the wildcard directly if this is the problem you want to solve.
Everywhere you don't specifically want the shell to expand wildcards and tokenize a string, you need to put double quotes around it.
echo "$file_categories"
This string isn't properly machine readable, and so it's of limited use to capture it in a variable at all. I'll wager a small sum of money that you actually simply want to display the output directly instead of storing it in a variable so that you can then echo its value:
cut -d, -f1 /filepath/*vendor.csv | sort -u
If you want to loop over the values, pipe this further to while read -r ...
In order to make your wc -l command to work, call it as such:
wc -l $a
Do not quote the a variable, the shell needs to expand it to read its * wildcard value
I'm trying to remove everything after a specific_string in a path string in Bash. I've tried using sed to no avail so far.
variable="specific_string"
input_string="/path/to/some/specific_string/specific_string.something/specific_string.something-else"
output=$(sed 's/$variable//' $input_string)
Output should be "/path/to/some/specific_string/"
Would be better if I didn't have to use commands such as sed!
The Problems
There are many problems
Variables are not evaluated inside single quotes. 's/$variable//' will be treated as a literal string, which does not contain specific_string
sed can modify text from files or STDIN, but not text given via parameters. With sed 's/...//' $input_string the /path/to/some/specific_string/.../file is opened and its content is read, instead of the path itself.
s/string// deletes only string, not the words afterwards.
Also remember to double quote your variables. cmd $variable is dangerous if the variable contains spaces. cmd "$variable" is safe.
Sed Solution
output="$(sed "s/$variable.*/$variable/" <<< "$input_string")"
GNU Grep Solution
output="$(grep -Po "^.*?$variable" <<< "$input_string")"
Pure Bash Solution
output="${input_string%%$variable*}$variable"
If you want to remove everything after "specific_string" it will remove the "/" also as it does with the following example:
output=$(echo $input_string|sed "s/${variable}.*$/${variable}/")
try with simple sed:
variable="specific_string"
input_string="/path/to/some/specific_string/specific_string.something/specific_string.something-else"
output=$(echo "$input_string" | sed "s/\(.*$variable\/\).*/\1/")
Output of variable output will be as follows.
echo $output
/path/to/some/specific_string/
I am using read to make some file in a while loop. The code is:
while read var val
do
sed -i 's/$var/$val/g' Hhh300_4L_gen.sh
echo $var $val
done < "Hhh300_4L_config.txt"
Where in Hhh300_4L_config.txt, there is a line, for instance,
PROCESSNAME Hhh;
and in Hhh300_4L_gen.sh, there is one element: PROCESSNAME. So if it works, PROCESSNAME in Hhh300_4L_gen.sh should be replaced by Hhh. But it doesn't. However the output of echo prints correctly.
Variables are not expanded inside single quotes. If you want var and val to expand you need double quotes (and make sure they don't have the sed separator, here / in them):
sed -i "s/$var/$val/g" Hhh300_4L_gen.sh
though if you're modifying a shell script (as I'm guessing might be from the .sh) there are probably better ways to do it, like having your .txt file store things as var=val instead of with white space separating them, then just source it from the script.
I don't usually work in bash but grep could be a really fast solution in this case. I have read a lot of questions on grep and variable assignment in bash yet I do not see the error. I have tried several flavours of double quotes around $pattern, used `...`` or $(...) but nothing worked.
So here's what I try to do:
I have two files. The first contains several names. Each of them I want to use as a pattern for grep in order to search them in another file. Therefore I loop through the lines of the first file and assign the name to the variable pattern.
This step works as the variable is printed out properly.
But somehow grep does not recognize/interpret the variable. When I substitute "$pattern" with an actual name everything is fine as well. Therefore I don't think the variable assignment has a problem but the interpretation of "$pattern" as the string it should represent.
Any help is greatly appreciated!
#!/bin/bash
while IFS='' read -r line || [[ -n $line ]]; do
a=( $line )
pattern="${a[2]}"
echo "Text read from file: $pattern"
var=$(grep "$pattern" 9606.protein.aliases.v10.txt)
echo "Matched Line in Alias is: $var"
done < "$1"
> bash match_Uniprot_StringDB.sh ~/Chromatin_Computation/.../KDM.protein.tb
output:
Text read from file: "UBE2B"
Matched Line in Alias is:
Text read from file: "UTY"
Matched Line in Alias is:
EDIT
The solution drvtiny suggested works. It is necessary to get rid of the double quotes to match the string. Adding the following lines makes the script work.
pattern="${pattern#\"}"
pattern="${pattern%\"}"
Please, look at "-f FILE" option in man grep.
I advise that this option do exactly what you need without any bash loops or such other "hacks" :)
And yes, according to the output of your code, you read pattern including double quotes literally. In other words, you read from file ~/Chromatin_Computation/.../KDM.protein.tb this string:
"UBE2B"
But not
UBE2B
as you probably expect.
Maybe you need to remove double quotes on the boundaries of your $pattern?
Try to do this after reading pattern:
pattern=${pattern#\"}
pattern=${pattern%\"}
I'd like to use sed to do a replace, but not by searching for what to replace.
Allow me to explain. I have a variable set to a default value initially.
VARIABLE="DEFAULT"
I can do a sed to replace DEFAULT with what I want, but then I would have to put DEFAULT back when I was all done. This is becuase what gets stored to VARIABLE is unique to the user. I'd like to use sed to search for somthing else other than what to replace. For example, search for VARIABLE=" and " and replace whats between it. That way it just constantly updates and there is no need to reset VARIABLE.
This is how I do it currently:
I call the script and pass an argument
./script 123456789
Inside the script, this is what happens:
sed -i "s%DEFAULT%$1%" file_to_modify
This replaces
VARIABLE="DEFAULT"
with
VARIABLE="123456789"
It would be nice if I didn't have to search for "DEFAULT", because then I would not have to reset VARIABLE at end of script.
sed -r 's/VARIABLE="[^"]*"/VARIABLE="123456789"/' file_to_modify
Or, more generally:
sed -r 's/VARIABLE="[^"]*"/VARIABLE="'"$1"'"/' file_to_modify
Both of the above use a regular expression that looks for 'VARIABLE="anything-at-all"' and replaces it with, in the first example above 'VARIABLE="123456789"' or, in the second, 'VARIABLE="$1"' where "$1" is the first argument to your script. The key element is [^"]. It means any character other than double-quote. [^"]* means any number of characters other than double-quote. Thus, we replace whatever was in the double-quotes before, "[^"]*", with our new value "123456789" or, in the second case, "$1".
The second case is a bit tricky. We want to substitute $1 into the expression but the expression is itself in single quotes. Inside single-quotes, bash will not substitute for $1. So, the sed command is broken up into three parts:
# spaces added for exposition but don't try to use it this way
's/VARIABLE="[^"]*"/VARIABLE="' "$1" '"/'
The first part is in single quotes and bash passes it literally to sed. The second part is in double-quotes, so bash will subsitute in for the value of `$``. The third part is in single-quotes and gets passed to sed literally.
MORE: Here is a simple way to test this approach on the command line without depending on any files:
$ new=1234 ; echo 'VARIABLE="DEFAULT"' | sed -r 's/VARIABLE="[^"]*"/VARIABLE="'"$new"'"/'
VARIABLE="1234"
The first line above is the command run at the prompt ($). The second is the output from running the command..