I have some troubles with my bash script. The end of my file doesn't work but every commands work outside the file. I have two strings as argument $1 and $2. $acl_line and $usebackend_line are numbers and they are good.
Here is my end file :
sed -i "$((acl_line+1))i \ \tacl\t\t is_$2_$1\t\thdr_com(host)\t-i $2.$1" /my_doc/haproxy/haproxy.cfg
sed -i "$((usebackend_line+1))i \ \tuse_backend\t$2_$1\tif is_$2_$1" /my_doc/haproxy/haproxy.cfg
echo -en "\nbackend $2_$1\n\tserver $2_$1 163.172.167.52:$3 maxconn 1024" >> /my_doc/haproxy/haproxy.cfg
cp -r "./model/*" "./script/lp_domains/$1/$2/"
sed -i 's/lp_ports/$ports/g' "./script/lp_domains/$1/$2/my_doc.yml"
sed -i 's/lp_name/$2-$1/g' "./script/lp_domains/$1/$2/my_doc.yml"
Thanks for your anwsers :)
If $1 and $2 should be interpolated, you cannot use single quotes.
Moreover, copying a file and then running sed -i on it is wasteful and error-prone. Just run sed and perform your substitutions at the same time.
sed -i -e "$((acl_line+1))i \ \tacl\t\t is_$2_$1\t\thdr_com(host)\t-i $2.$1" \
-e "$((usebackend_line+1))i \ \tuse_backend\t$2_$1\tif is_$2_$1" /my_doc/haproxy/haproxy.cfg \
-e "\$a\
backend $2_$1\n\tserver $2_$1 163.172.167.52:$3 maxconn 1024" /my_doc/haproxy/haproxy.cfg
# remove ./model/my_doc.yml; instead have a template ./my_doc.yml.in
cp -r "./model/*" "./script/lp_domains/$1/$2/"
sed -e "s/lp_ports/$ports/g" -e "s/lp_name/$2-$1/g" \
my_doc.yml.in >"./script/lp_domains/$1/$2/my_doc.yml"
(You should probably do something similar with haproxy.cfg.in actually.)
I have fixed my errors. It was just permissions errors, Sed create some temporary files so i add permissions to my user. Thanks for your help !
Related
This question already has answers here:
How can I parse a YAML file from a Linux shell script?
(23 answers)
Closed 26 days ago.
Here is our yaml:
network:
ethernets:
ens160:
addresses:
- 10.200.2.11/22
gateway4: 10.200.0.1
nameservers:
addresses:
- 8.8.8.8
- 4.4.4.4
search:
- cybertax.live
version: 2
I want to change the dns severs only.
From:
- 8.8.8.8
- 4.4.4.4
to:
- 10.10.10.10
- 10.10.10.11
How can I do this? Note: we cannot use or install yq so this needs to be done through sed or awk. Also, yes I know, this is not recommended, but its what needs to be done right now.
What I have tried so far:
sed -i '/ addresses:/,/ search:/ s/^/# /' $netplan_yaml
sed -i '/ nameservers:/a\ \ \ \ \ \ \ \ addresses:' $netplan_yaml
for i in ${!asar_dns[#]}; do
sed -i "/ addresses:/a\ \ \ \ \ \ \ \ - ${asar_dns[$i]}" $netplan_yaml
done
But this does three things wrong (that I can see).
It matches between addresses and search including the line wiht addresses and search. I only want what is AFTER addresses, and BEFORE search.
It puts the DNS addresses in the associative array between the older addresses that is commented out anywhere there is an "addresses". I dont want to do that on the commented out line.
i dont like how I have to use \ \ \ \ \ \ would much rather use a .* if possible but also need to use the addresses in the associative array.
I found a very hacky solution but it works. Open for feedback.
netplan_yaml=/etc/netplan/00-installer-config.yaml
baddns="$(sed -n '/.*nameservers:/,/.*search:/p' /etc/netplan/00-installer-config.yaml | grep -v 'nameservers\|addresses' | grep -v 'search' | grep -v 10.200 | awk '{print$2}')"
mapfile -t arr_baddns <<<$baddns
for i in ${arr_baddns[#]}; do
sed -i "/ - $i/s/^/#/g" $netplan_yaml
done
for i in ${!asar_dns[#]}; do
sed -i "/ addresses:/a\ \ \ \ \ \ \ \ - ${asar_dns[$i]}" $netplan_yaml
done
If ed is available/acceptable, with your given input.
Something like.
ed -s file.yaml <<-'EOF'
g/nameservers:/;/addresses:/;/search:/-s/[[:digit:].]\{1,\}/10.10.10.11/\
-s/[[:digit:].]\{1,\}/10.10.10.10/
,p
Q
EOF
Using variables for the values, something like:
#!/bin/sh
dns1=10.10.10.10
dns2=10.10.10.11
ed -s file.yaml <<-EOF
g/nameservers:/;/addresses:/;/search:/-s/[[:digit:].]\{1,\}/$dns2/\\
-s/[[:digit:].]\{1,\}/$dns1/
,p
Q
EOF
Change Q to w if in-place editing is required, ala sed -i
Remove the ,p to silence the output.
have such file to execute
exec java -Dcom.sun.management.jmxremote=true \
-Dcom.sun.management.jmxremote.port=12345 \
-Djava.rmi.server.hostname=127.0.0.1 \
-Dcom.sun.management.jmxremote.authenticate=false \
-jar somefile.jar
The question how to change IP with using of sed?
for in-place change
sed -i 's/127\.0\.0\.1'/new_ip/' file
For a copy, you can use
sed -i 's/127\.0\.0\.1'/new_ip/' file > new_file
I am trying to automate a procedure where the system will fetch the contents of a file (1 Url per line), use wget to grab the files from the site (https folder) and then remove the line from the file.
I have made several tries but the sed part (at the end) cannot understand the string (I tried escaping characters) and remove it from that file!
cat File
https://something.net/xxx/data/Folder1/
https://something.net/xxx/data/Folder2/
https://something.net/xxx/data/Folder3/
My line of code is:
cat File | xargs -n1 -I # bash -c 'wget -r -nd -l 1 -c -A rar,zip,7z,txt,jpg,iso,sfv,md5,pdf --no-parent --restrict-file-names=nocontrol --user=test --password=pass --no-check-certificate "#" -P /mnt/USB/ && sed -e 's|#||g' File'
It works up until the sed -e 's|#||g' File part..
Thanks in advance!
Dont use cat if it's posible. It's bad practice and can be problem with big files... You can change
cat File | xargs -n1 -I # bash -c
to
for siteUrl in $( < "File" ); do
It's be more correct and be simpler to use sed with double quotes... My variant:
scriptDir=$( dirname -- "$0" )
for siteUrl in $( < "$scriptDir/File.txt" )
do
if [[ -z "$siteUrl" ]]; then break; fi # break line if him empty
wget -r -nd -l 1 -c -A rar,zip,7z,txt,jpg,iso,sfv,md5,pdf --no-parent --restrict-file-names=nocontrol --user=test --password=pass --no-check-certificate "$siteUrl" -P /mnt/USB/ && sed -i "s|$siteUrl||g" "$scriptDir/File.txt"
done
#beliy answers looks good!
If you want a one-liner, you can do:
while read -r line; do \
wget -r -nd -l 1 -c -A rar,zip,7z,txt,jpg,iso,sfv,md5,pdf \
--no-parent --restrict-file-names=nocontrol --user=test \
--password=pass --no-check-certificate "$line" -P /mnt/USB/ \
&& sed -i -e '\|'"$line"'|d' "File.txt"; \
done < File.txt
EDIT:
You need to add a \ in front of the first pipe
I believe you just need to use double quotes after sed -e. Instead of:
'...&& sed -e 's|#||g' File'
you would need
'...&& sed -e '"'s|#||g'"' File'
I see what you trying to do, but I dont understand the sed command including pipes. Maybe some fancy format that I dont understand.
Anyway, I think the sed command should look like this...
sed -e 's/#//g'
This command will remove all # from the stream.
I hope this helps!
I would like to implement this as a Makefile task:
# step 1:
curl -u username:password -X POST \
-d '{"name": "new_file.jpg","size": 114034,"description": "Latest release","content_type": "text/plain"}' \
https://api.github.com/repos/:user/:repo/downloads
# step 2:
curl -u username:password \
-F "key=downloads/octocat/Hello-World/new_file.jpg" \
-F "acl=public-read" \
-F "success_action_status=201" \
-F "Filename=new_file.jpg" \
-F "AWSAccessKeyId=1ABCDEF..." \
-F "Policy=ewogIC..." \
-F "Signature=mwnF..." \
-F "Content-Type=image/jpeg" \
-F "file=#new_file.jpg" \
https://github.s3.amazonaws.com/
In the first part however, I need to get the file size (and content type if it's easy, not required though), so some variable:
{"name": "new_file.jpg","size": $(FILE_SIZE),"description": "Latest release","content_type": "text/plain"}
I tried this but it doesn't work (Mac 10.6.7):
$(shell du path/to/file.js | awk '{print $1}')
Any ideas how to accomplish this?
If you have GNU coreutils:
FILE_SIZE=$(stat -L -c %s $filename)
The -L tells it to follow symlinks; without it, if $filename is a symlink it will give you the size of the symlink rather than the size of the target file.
The MacOS stat equivalent appears to be:
FILE_SIZE=$(stat -L -f %z)
but I haven't been able to try it. (I've written this as a shell command, not a make command.) You may also find the -s option useful:
Display information in "shell output", suitable for initializing variables.
For reference, an alternative method is using du with -b bytes output and -s for summary only. Then cut to only keep the first element of the return string
FILE_SIZE=$(du -sb $filename | cut -f1)
This should return the same result in bytes as #Keith Thompson answer, but will also work for full directory sizes.
Extra: I usually use a macro for this.
define sizeof
$$(du -sb \
$(1) \
| cut -f1 )
endef
Which can then be called like,
$(call sizeof,$filename_or_dirname)
I think this is a case where parsing the output of ls is legitimate:
% FILE_SIZE=`ls -l $filename | awk '{print $5}'`
(no it's not: use stat, as noted by Keith Thompson)
For the type, you can use
% FILE_TYPE=`file --mime-type --brief $filename`
Is it possible to Wget a page's title from the command line?
input:
$ wget http://bit.ly/rQyhG5 <<code>>
output:
If it’s broke, fix it right - Keeping it Real Estate. Home
This script would give you what you need:
wget --quiet -O - http://bit.ly/rQyhG5 \
| sed -n -e 's!.*<title>\(.*\)</title>.*!\1!p'
But there are lots of situations where it breaks, including if there is a <title>...</title> in the body of the page, or if the title is on more than one line.
This might be a little better:
wget --quiet -O - http://bit.ly/rQyhG5 \
| paste -s -d " " \
| sed -e 's!.*<head>\(.*\)</head>.*!\1!' \
| sed -e 's!.*<title>\(.*\)</title>.*!\1!'
but it does not fit your case as your page contains the following head opening:
<head profile="http://gmpg.org/xfn/11">
Again, this might be better:
wget --quiet -O - http://bit.ly/rQyhG5 \
| paste -s -d " " \
| sed -e 's!.*<head[^>]*>\(.*\)</head>.*!\1!' \
| sed -e 's!.*<title>\(.*\)</title>.*!\1!'
but there is still ways to break it, including no head/title in the page.
Again, a better solution might be:
wget --quiet -O - http://bit.ly/rQyhG5 \
| paste -s -d " " \
| sed -n -e 's!.*<head[^>]*>\(.*\)</head>.*!\1!p' \
| sed -n -e 's!.*<title>\(.*\)</title>.*!\1!p'
but I am sure we can find a way to break it. This is why a true xml parser is the right solution, but as your question is tagged shell, the above it the best I can come with.
The paste and the 2 sed can be merged in a single sed, but is less readable. However, this version has the advantage of working on multi-line titles:
wget --quiet -O - http://bit.ly/rQyhG5 \
| sed -n -e 'H;${x;s!.*<head[^>]*>\(.*\)</head>.*!\1!;T;s!.*<title>\(.*\)</title>.*!\1!p}'
Update:
As explain in the comments, the last sed above uses the T command which is a GNU extension. If you do not have a compatible version, you can use:
wget --quiet -O - http://bit.ly/rQyhG5 \
| sed -n -e 'H;${x;s!.*<head[^>]*>\(.*\)</head>.*!\1!;tnext;b;:next;s!.*<title>\(.*\)</title>.*!\1!p}'
Update 2:
As above still not working on Mac, try:
wget --quiet -O - http://bit.ly/rQyhG5 \
| sed -n -e 'H;${x;s!.*<head[^>]*>\(.*\)</head>.*!\1!;tnext};b;:next;s!.*<title>\(.*\)</title>.*!\1!p'
and/or
cat << EOF > script
H
\$x
\$s!.*<head[^>]*>\(.*\)</head>.*!\1!
\$tnext
b
:next
s!.*<title>\(.*\)</title>.*!\1!p
EOF
wget --quiet -O - http://bit.ly/rQyhG5 \
| sed -n -f script
(Note the \ before the $ to avoid variable expansion.)
It seams that the :next does not like to be prefixed by a $, which could be a problem in some sed version.
The following will pull whatever lynx thinks the title of the page is, saving you from all of the regex nonsense. Assuming the page you are retrieving is standards compliant enough for lynx, this should not break.
lynx -dump example.com | sed '2q;d'