I have a script that contains 2 variables. One of the variables is a directory path and the other is a user input variable. Once the user enters the variable, in this case a certificate name, I use sed to replace the text of xxxxx with both the script_path separated by a / and the the certfile variable.
Still new at programming but I've managed to get my script working aside from the issue below. I've tried escaping the variables the "/" and nothing seems to work.
I've also tried changing the delimiter that sed uses as well with no luck. I did search around quite a bit and didn't find anything specific with using a "/" and concatenating variables so apologies up front if this has already been solved.
#!/bin/bash
script_path=/opt/ceflog
read -p 'Enter the name of the certificate file: ' certfile
sed -e "s/pkcs12_file = xxxxxx/pkcs12_file = $script_path/$certfile/g" \$script_path/cef.conf
What is should look like is something like the below.
pkcs12_file = /opt/ceflog/192.168.1.1_1.pkcs12
As always thanks in advance for you help.
I guess you want to do something like this
$ path='/opt/ceflog'; cert='192.168.1.1_1.pkcs12';
$ echo pkcs12_file = xxxxxx/pkcs12_file |
sed -E 's~(pkcs12_file =) (xxxxxx/pkcs12_file)~\1 '"${path}/${cert}"'~'
pkcs12_file = /opt/ceflog/192.168.1.1_1.pkcs12
use a different sed delimiter (here I picked ~) than the default one (/), since you may have it in your data.
So looks like there were 2 issues. I escaped the use of the variables on the end "\$script_path/cef.conf" and changing the dilemeter for sed I was able to get it to work.
sed -e "s|pkcs12_file = xxxxxx|pkcs12_file = $script_path/$certfile|g" $script_path/cef.conf
Thanks again all.
Related
Where the word is repeated and only wanted to be removed from specific word location
Lets say my path is - /opt/xyz/config/config.xml
Solution I want after using sed is: /opt/xyz/config/
how can this be obtained?
I am sick of using {sed 's/config.*//'} >> This actually removes both config words
such as it looks
/opt/xyz/
I have tried using this in multiple ways
> sed 's/config.*//'
Maybe with something like this?
sed 's/[^/]*$//'
But if the filepath is in a shell variable then you might as well use:
mydir=${myfilepath%/*}/
Another potential solution, depending on your use-case, is the dirname bash function, e.g.
dirname /opt/xyz/config/config.xml
/opt/xyz/config
Using BASH parameter expansion:
p="/opt/xyz/config/config.xml"; echo "${p%/*}"
/opt/xyz/config
My Linux repository file contain a link that until now was using http with a port number to point to it repository.
baseurl=http://host.domain.com:123/folder1/folder2
I now need a way to replace that URL to use https with no port or a different port .
I need also the possibility to change the server name for example from host.domain.com to host2.domain.com
So my idea was to use sed to search for the start of the http until the first / that come after the 2 // thus catching whatever in between and will give me the ability to change both server name port or http\s usage.
Im now using this code (im using echo just for the example):
the example shows how in 2 cases where one time i have a link with http and port 123 converted to https and the second time the other way around
and both code i was using the same sed for generic reasons.
WANTED_URL="https://host.domain.com"
echo 'http://host.domain.com:123/folder1/folder2' | sed -i "s|http.*://[^/]*|$WANTED_URL|"
OR
WANTED_URL="http://host.domain.com:123"
echo 'https://host.domain.com/folder1/folder2' | sed -i "s|http.*://[^/]*|$WANTED_URL|"
is that the correct way doing so?
sed regexes are greedy by default. You can tell sed to consume only non-slashes, like this:
echo 'http://host.domain.com:123/folder1/folder2' | sed -e 's|http://[^/]*|https://host.domain.com|'
result:
https://host.domain.com/folder1/folder2
(BTW you don't have to escape slashes because you are using an alternate separating character)
the key is using [^/]* which will match anything but slashes so it stops matching at the first slash (non-greedy).
You used /.*/ and .* can contain slashes, not that you wanted (greedy by default).
Anyway my approach is different because expression does not include the trailing slash so it is not removed from final output.
Assuming it doesn't really matter if you have 1 sed script or 2 and there isn't a good reason to hard-code the URLs:
$ echo 'http://host.domain.com:123/folder1/folder2' |
sed 's|\(:[^:]*\)[^/]*|s\1|'
https://host.domain.com/folder1/folder2
$ port='123'; echo 'https://host.domain.com/folder1/folder2' |
sed 's|s\(://[^/]*\)|\1:'"$port"'|'
http://host.domain.com:123/folder1/folder2
If that isn't what you need then edit your question to clarify your requirements and in particular explain why:
You want to use hard-coded URLs, and
You need 1 script to do both transformations.
and provide concise, testable sample input and expected output that demonstrates those needs (i.e. cases where the above doesn't work).
wrt what you had:
WANTED_URL="https://host.domain.com"
echo 'http://host.domain.com:123/folder1/folder2' | sed -i "s|http.*://[^/]*|$WANTED_URL|"
The main issues are:
Don't use all-upper-case for non-exported shell variable names to avoid clashes with exported variables and to avoid obfuscating your code (this convention has been around for 40 years so people expect all upper case variables to be exported).
Never enclose any script in double quotes as it exposes the whole script to the shell for interpretation before the command you want to execute even sees it. Instead just open up the single quotes around the smallest script segment possible when necessary, i.e. to expand $y in a script use cmd 'x'"$y"'z' not cmd "x${y}z" because the latter will fail cryptically and dangerously given various input, script text, environment settings and/or the contents of the directory you run it from.
The -i option for sed is to edit a file in-place so you can't use it on an incoming pipe because you can't edit a pipe in-place.
When you let a shell variable expand to become part of a script, you have to take care about the possible characters it contains and how they'll be interpreted by the command given the context the variable expands into. If you let a whole URL expand into the replacement section of a sed script then you have to be careful to first escape any potential backreference characters or script delimiters. See Is it possible to escape regex metacharacters reliably with sed. If you just let the port number expand then you don't have to deal with any of that.
I read some file line by looking for some specific string. When I find I assign it to the var.
But when I try to append to that string I have problem. Instead of add to the end of var i get var where characters at the beginning are replaced by new characters.
Example:
echo $fileToGet
newVar=$fileToGet".xml"
echo $newVar
Output:
c024z160205
.xmlz160205
And what I want is: c024z160205.xml
I think I tried everything what is on Stack, several ways of appending but nothing works.
The problem was \r at the end of var. Before assigning line to var fileToGet I do something like:
newVar=$(echo "$fileToGet" | tr -d '\r')
and after concatenation I have what I want.
I edit this answer because it contains false informations. What really was the problem #chepner described in comment: 'The terminal simply displays any characters following the carriage return at the beginning of the line, overwriting the earlier characters.'
Thanks!
Your code is working for me, try using:
newVar="${fileToGet}.xml"
Instead of newVar=$fileToGet".xml"; you gotta put the variable also inside the quotes.
Tell me if it works now.
I have a script that I am writing that checks a value and then based on the value modifies it.
I am trying to understand why it works this one way and not the other. Based on the google and stackoverflow searches I did, nothing really fits what I am trying to understand.
The value it is trying to change is test_value = 5 and changes the 5 to a 6. In the file there are two like lines.
test_value = 5
test_value_action = log
Script content is below:
#!/bin/bash
CHECK="test_value"
sed -i 's|^\("$CHECK" = \).*|\1'6'|' /user/file.txt
That doesn't work. But if I hard code the value for check it works
#!/bin/bash
CHECK="test_value"
sed -i 's|^\(test_value = \).*|\1'6'|' /user/file.txt
What am I failing to understand? Also for other ones I have wrote where there is only one single line in the file that it can match, it works with $CHECK.
Your variable is still within single quote hence not getting expanded. Use:
sed -i 's|^\('"$CHECK"' = \)*.|\1'6'|' /user/file.txt
Apologies for a seemingly inane question. But I have spent the whole day trying to figure it out and it drives me up the walls. I'm trying to write a seemingly simple bash script that would take a list of files in the directory from ls, replace part of the file names using sed, get unique names from the list and pass them onto some command. Like so:
inputs=`ls *.ext`
echo $inputs
test1_R1.ext test1_R2.ext test2_R1.ext test2_R2.ext
Now I would like to put it through sed to replace 1.ext and 2.ext with * to get test1_R* etc. Then I'd like to remove resulting duplicates by running sort -u to arrive to the following $outputs variable:
echo $outputs
test1_R* test2_R*
And pass this onto a command, like so
cat $outputs
I can do something like this in a command line:
ls *.ext | sed s/..ext/\*/g | sort -u
But if I try to assign the above to a variable in the script it just returns the output from the ls. I have tried several ways to do it: including the whole pipe in the script. Running each command separately and assigning it to a variable, then passing that variable to the next command and writing the outputs to files then passing the file to the next command. But so far none of this managed to achieve what I aimed to. I think my problem lies in (except general cluelessness aroung bash scripting) inability to run seq on a variable within script. There seems to be a lot of advice around in how to pass variables to pattern or replacement string in sed, but they all seem to take files as input. But I understand that it might not be the proper way of doing it anyway. Therefore I would really appreciate if someone could suggest an elegant way to achieve, what I'm trying to.
Many thanks!
Update 2/06/2014
Hi Barmar, thanks for your answer. Can't say it solved the problem, but it helped pin-pointing it. Seems like the problem is in me using the asterisk. I have to say, I'm very puzzled. The actual file names I've got are:
test1_R1.fastq.gz test1_R2.fastq.gz test2_R1.fastq.gz test2_R2.fastq.gz
If I'm using the code you suggested, which seems to me the right way do to it:
ins=$(ls *.fastq.gz | sed 's/..fastq.gz/\*/g' | sort -u)
Sed doesn't seem to do anything and I'm getting the output of ls:
test1_R1.fastq.gz test1_R2.fastq.gz test2_R1.fastq.gz test2_R2.fastq.gz
Now if I replace that backslash with anything else, the sed works, but it also returns whatever character I'm putting in front (or after) the asteriks:
ins=$(ls *.fastq.gz | sed 's/..fastq.gz/"*/g' | sort -u)
test1_R"* test2_R"*
That's odd enough, but surely I can just put an "R" in front of the asteriks and then replace R in the search pattern string, right? Wrong! If I do that whichever way: 's/R..fastq.gz/R*/g' 's/...fastq.gz/R*/g' 's/[A-Z]..fastq.gz/R*/g' I'm back to the original names! And even if I end up with something like test1_RR* test2_RR* and try to run it through sed again and replace "_R" for "_" or "RR" for "R", I'm having no luck and I'm back to the original names. And yet I can replace the rest of the file name no problem, just not to get me test1_R* I need.
I have a feeling I should be escaping that * in some very clever way, but nothing I've tried seems to work. Thanks again for your help!
This is how you capture the result of the whole pipeline in a variable:
var=$(ls *.ext | sed s/..ext/\*/g | sort -u)