In GitLab CI script I wanted to
Remove any character other then numbers and dot (.) and
remove all the text after 2nd dot [Ex 5.6.7.8 -> 5.6] if dot exists in text.
So for that I have tried to use sed , but it's not working in GitLab Script (working on bash shell locally)
export BRANCH_NAME={$CI_COMMIT_REF_NAME} | sed 's/\.[^.]*$//' | sed 's/[^0-9.]//g'
any idea what is missing here ?
If sed is not going to work here then any other option to achieve the same?
Edit :
As per #WiktorStribiżew
- echo "$CI_COMMIT_REF_NAME" | sed 's/.[^.]$//' | sed 's/[^0-9.]//g'
- export BRANCH_NAME=${CI_COMMIT_REF_NAME} | sed 's/.[^.]$//' | sed 's/[^0-9.]//g'
echo is working but export is not
Using sed
$ export BRANCH_NAME=$(echo "$CI_COMMIT_REF_NAME" |sed 's/[A-Za-z]*//g;s/\([0-9]*\.[0-9]*\)\.[^ ]*/\1/g')
First of all, to remove all text after and including second dot you need
sed 's/\([^.]*\.[^.]*\).*/\1/'
The sed 's/\.[^.]*$//' sed command removes the last dot and any text after it.
Next, you must have made a typo and you actually meant to write ${CI_COMMIT_REF_NAME} and not {$CI_COMMIT_REF_NAME}.
So, you might be better off with
export BRANCH_NAME=$(echo "$CI_COMMIT_REF_NAME" | sed 's/\([^.]*\.[^.]*\).*/\1/' | sed 's/[^0-9.]//g')
Related
I have a no_proxy string I'd like to modify to fit with Java Opts.
❯ export no_proxy_test=localhost,127.0.0.1,.myfancy.domain,.myotherfancy.domain
❯ echo $no_proxy_test
localhost,127.0.0.1,.myfancy.domain,.myotherfancy.domain
I need for each value to be delimited with pipes "|" instead and I need to remove the dots for the wildcard. So should be:
localhost|127.0.0.1|myfancy.domain|myotherfancy.domain
When I use:
❯ echo $no_proxy_test | sed 's/,./|/g'
localhost|27.0.0.1|myfancy.domain|myotherfancy.domain
For some reason it cuts the 1 in 127.0.0.1 and I don't understand why? I thought I may achieve it with a double sed:
❯ echo $no_proxy_test | sed 's/,/|/g' | sed 's/|./|/g'
localhost|27.0.0.1|myfancy.domain|myotherfancy.domain
But same problem there. Does anyone have an idea? I don't want to sed replace 27 with 127. Would be interesting if I ran into a bug or if anyone could explain why 1 is being cut from the string.
Thank you!
You can use
sed -E 's/,\.?/|/g' # POSIX ERE
sed 's/,\.\{0,1\}/|/g' # POSIX BRE
This will replace a comma followed with an optional . char with a pipe symbol.
See the online demo:
#!/bin/bash
s='localhost,127.0.0.1,.myfancy.domain,.myotherfancy.domain'
sed 's/,\.\{0,1\}/|/g' <<< "$s"
# => localhost|127.0.0.1|myfancy.domain|myotherfancy.domain
I'm trying to automate generating a README.md.
The idea is:
Generate markdown table string like...
table="| Image | Name | Description | Notes |\n"
table+="| --- | --- | --- | --- |\n"
table+="| $img1 | $name1 | $desc1 | $notes1 |\n"
table+="| $img2 | $name2 | $desc2 | $notes2 |\n"
...
*simplified
*contains special characters like e.g. |-()[]/<>
Replace <!-- insert-table-here --> in a readme_template.md file with the full table
## Header
<!-- insert-table-here -->
<sub>More info...</sub>
Save new file as README.md
I can't get step 2 working.
How do you replace a line in a file with a multi-line, special char ridden string?
Every sed, awk, perl, or even head/tail command I try seems to not work. Are heredocs the better approach?
I have found some hack solutions for specific cases with specific chars but I want to identify a more robust method.
EDIT: Thanks to #potong, this is what ended up working for me.
echo -e ${table} | sed -e '/<!-- insert-table-here -->/{r /dev/stdin' -e 'd}' readme_template.md > README.md
EDIT 2: After spending some more time on this, I found a nice multi-match option through awk
awk \
-v t1="$(generate_table1)" \
-v t2="$(generate_table2)" \
'{
gsub(/<!-- insert-table-1 -->/,t1)
gsub(/<!-- insert-table-2 -->/,t2)
}1' \
readme_template.md > README.md
This might work for you (GNU sed and bash):
cat <<\! | sed -e '/<!-- insert-table-here -->/{r /dev/stdin' -e 'd}' file
Here is a heredoc
with special symbols
{}-()[]/<>
!
The heredoc is piped through to the sed command using /dev/stdin as a file for the r command, then the original line is deleted using the d command.
N.B. The use of the -e command line option to split the two parts of the sed script (oneliner). This is necessary because the r command needs to be terminated by a newline and the -e option provides this functionality.
Here is what is the file beast.txt:
savage
beast
tank
beauty
I am looking to get it to look like:
tbwbhf
cfbtu
ubol
cfbvuz
This what I have but when I run this it changes the letters to "[a-z]+1".
cat beast.txt | sed 's/[a-z]/[a-z]+1/g' > savage.txt
Do I have to use other special characters to in the sed command or should this be done with a loop? Thank you for your help.
This is not a job for sed:
$ echo 'savage beast tank beauty' | tr a-z b-za
tbwbhf cfbtu ubol cfbvuz
If you really want to use sed, you can do:
$ b2z=bcdefghijklmnopqrstuvwxyz
$ echo 'savage beast tank beauty' | sed -e "y/a$b2z/${b2z}a/"
tbwbhf cfbtu ubol cfbvuz
I used this '$ echo 'savage beast tank beauty' | tr a-z b-za' and it worked.
I am now having trouble reversing what I just did.
I use this to generate svn logs for a specific user:
svn log | sed -n '/bob/,/-----$/ p'
which works fine. Now I would like to use a parameter/variable for the user:
USER="bob"
svn log | sed -n '/$USER/,/-----$/ p'
but it just give an empty log. How do I pass a parameter to the above command?
The correct way to do this is to use double quotes so that the bash variable can be expanded:
USER="bob"
svn log | sed -n "/$USER/,/-----$/p"
Adding more single quotes to "insert" a bash variable (as per the accepted answer) is not best practice. HTH.
Use single quotes:
USER="bob"
svn log | sed -n '/'$USER'/,/-----$/ p'
try:
svn log | sed -n '/"$USER"/,/-----$/ p'
Here's my problem, I have written the following line of code to format properly a list of files found recursively in a directory.
find * | sed -e '/\(.*\..*\)/ !d' | sed -e "s/^.*/\${File} \${INST\_FILES} &/" | sed -e "s/\( \)\([a-zA-Z0-9]*\/\)/\/\2/" | sed -e "s/\(\/\)\([a-zA-Z0-9\_\-\(\)\{\}\$]*\.[a-zA-Z0-9]*\)/ \2/"
The second step is to write the output of this command in a script. While the code above has the expected behavior, the problem occurs when I try to store its output to a variable, I get a bad substitution error from the first sed command in the line.
#!/bin/bash
nsisscript=myscript.sh
FILES=*
for f in $(find $FILES); do
v=`echo $f | sed -e '/\(.*\..*\)/ !d' | sed -e "s/^.*/\${File} \${INST\_FILES} &/" | sed -e "s/\( \)\([a-zA-Z0-9]*\/\)/\/\2/" | sed -e "s/\(\/\)\([a-zA-Z0-9\_\-\(\)\{\}\$]*\.[a-zA-Z0-9]*\)/ \2/"`
sed -i.backup -e "s/\;Insert files here/$v\\n&/" $nsisscript
done
Could you please help me understand what the difference is between the two cases and why I get this error ?
Thanks in advance!
Well my guess was that your escaping of underscore in INST_FILES is strange as underscore is not a special character in shell nor in sed. The error disappear when you delete the '\' before '_'
my 2 cents
Parsing inside of backquote-style command substitution is a bit weird -- it requires an extra level of escaping (i.e. backslashes) to control when expansions take place. Ugly solution: add more backslashes. Better solution: use $() instead of backquotes -- it does the same thing, but without the weird parsing and escaping issues.
BTW, your script seems to have some other issues. First, I don't know about the sed on your system, but the versions I'm familiar with don't interpret \n in the substitution as a newline (which I presume you want), but as a literal n character. One solution is to include a literal newline in the substitution (preceded by a backslash).
Also, the loop executes for each found file, but for files that don't have a period in the name, the first sed command removes them, $v is empty, and you add a blank line to myscript.sh. You should either put the filtering sed call in the for statement, or add it as a filter to the find command.
#!/bin/bash
nsisscript=myscript.sh
nl=$'\n'
FILES=*
for f in $(find $FILES -name "*.*"); do
v=$(echo $f | sed -e "s/^.*/\${File} \${INST\_FILES} &/" | sed -e "s/\( \)\([a-zA-Z0-9]*\/\)/\/\2/" | sed -e "s/\(\/\)\([a-zA-Z0-9\_\-\(\)\{\}\$]*\.[a-zA-Z0-9]*\)/ \2/")
sed -i.backup -e "s/\;Insert files here/$v\\$nl&/" $nsisscript
done