How to use sed command to replace word in file - bash

I have a text file:
org.jitsi.videobridge.xmpp.user.shard-1.HOSTNAME=localhost
org.jitsi.videobridge.xmpp.user.shard-1.DOMAIN=auth.jc.name.com
org.jitsi.videobridge.xmpp.user.shard-1.USERNAME=name
org.jitsi.videobridge.xmpp.user.shard-1.PASSWORD=Hfr*7462
org.jitsi.videobridge.xmpp.user.shard-1.MUC_JIDS=JvbBredjoy#internal.auth.jc.name.com
org.jitsi.videobridge.xmpp.user.shard-1.MUC_NICKNAME=7896aee5-fgre-4b02-4569-0bcc75ed1d0d
I created a bash script:
#!/bin/bash
DPATH="/etc/jitsi/videobridge/sip-communicator.properties"
k=$(grep -o 'shard-1' $DPATH) # shard ends by a number#
i=$(grep -o 'shard-1' $DPATH | cut -c7)
m=$((i+1))
n="shard-$m"
sed -i "s|${k}|${n}|g" $DPATH
But I get error:
/home/scripts# ./shard_number
./shard_number: line 5: 1
1
1
1
1
1: syntax error in expression (error token is "1
1
1
1
1")
sed: -e expression #1, char 9: unterminated `s' command
Could you please help to solve this issue? Thank you.

If you call your script with bash -x /path/to/your/script or add set -x somewhere at the start of your script (after the #!shebang, but before the commands you want to debug), you will see that your grep commands return not a single 'shard-1' but rather one 'shard-1' per line :
++ grep -o shard-1 /etc/jitsi/videobridge/sip-communicator.properties
+ k='shard-1
shard-1
shard-1
shard-1
shard-1
shard-1'
Once cut, that gives the 1\n1\n1\n1\n1\n string that is mentionned in your error output as an invalid token for the $(( ... )) expression, which also breaks the syntax of your sed substitution :
++ cut -c7
++ grep -o shard-1 /etc/jitsi/videobridge/sip-communicator.properties
+ i='1
1
1
1
1
1'
Make that string a single number (for instance piping your grep into sort -u to unicize all the shards found) and your script will work just fine :
#!/bin/bash
DPATH="/etc/jitsi/videobridge/sip-communicator.properties"
k=$(grep -o 'shard-1' $DPATH | sort -u) # shard ends by a number#
i=$(grep -o 'shard-1' $DPATH | sort -u | cut -c7)
m=$((i+1))
n="shard-$m"
sed -i "s|${k}|${n}|g" $DPATH
You can try it here. Also check this test if you want to see your initial script debugged.

Related

bash: ERROR: batch file could not be read

Inside C:\cygwin64\home\Administrator I have 3 files
script.sh
links_da_scaricare.txt
cookies.txt
To download from dplay I use successfully if I use this command, videos are downloaded
yt-dlp --referer "https:www.discoveryplus.com/" --add-header "Cookie:gcl_au=1.1.192985240.1664694131; _scid=62382352-4832-438c-a74d-30699989ca96; _fbp=fb.1.1664694351012.468612256; __telemetric.v=1599968982.1664694367.81679; __zlcmid=1CFlGmqJqevtvtz; st=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJVU0VSSUQ6ZHBsYXk6MzEzNzY0NDYtOTkwMC00ODVlLWE3YjYtYTkyYzc2MDJkZWE4IiwianRpIjoidG9rZW4tNTA5ZTdiZWYtOTBmMC00ZmVjLWJlNWMtNzJhZmNiZThlYzZmIiwiYW5vbnltb3VzIjpmYWxzZSwiaWF0IjoxNjY0Njk0MzkzfQ.kH5QmwI2CH0-baBgJMGVoalTHbMcwOtH8Go_G2mEz6c; aam_uuid=50034166116354787000738543341305359891; _tt_enable_cookie=1; _ttp=45eb20c2-a55e-4740-be63-01b5e4166bd3; _hjSessionUser_2533509=eyJpZCI6IjM5NDAxNDA3LTllZDQtNWI1ZC1iYjY4LTRiNDlhOWIzMzZhOSIsImNyZWF0ZWQiOjE2NjQ2OTQzNjcxNDQsImV4aXN0aW5nIjp0cnVlfQ==; AMCV_9AE0F0145936E3790A495CAA#AdobeOrg=-1124106680|MCIDTS|19268|MCMID|50189272745869278580790010530934545485|MCAAMLH-1665300361|6|MCAAMB-1665300361|RKhpRz8krg2tLO6pguXWp5olkAcUniQYPHaMWWgdJ3xzPWQmdj0y|MCOPTOUT-1664702762s|NONE|MCSYNCSOP|411-19275|vVersion|5.2.0; __telemetric.s=1.1664980917.1664982717; _hjSession_2533509=eyJpZCI6ImVhNmZmNTA3LTM1ZGYtNDBlZC1iMWFjLTA0MzRlYzZlYjJjNCIsImNyZWF0ZWQiOjE2NjQ5ODA5MTc5MTEsImluU2FtcGxlIjpmYWxzZX0=; _hjAbsoluteSessionInProgress=0; _clck=ji7lnf|1|f5g|0; fs_uid=#161CV1#6518423911501824:5976792460267520:::#/1696517470; _uetsid=2596f2d044bd11ed80c355f240f55627; _uetvid=c8269d50422011ed952c2140169cc91d; ABTastySession=mrasn=&sen=9&lp=https%3A%2F%2Fwww.discoveryplus.com%2Fit; ABTasty=uid=4hnc3e5beewkxmkz&fst=1664694364438&pst=1664813235740&cst=1664981468233&ns=4&pvt=21&pvis=10&th=; _clsk=6lrbo9|1664981584916|8|0|j.clarity.ms/collect" --batch-file links_da_scaricare.txt
ok, I try to download using Cygwin and I use this command
bash -x script.sh
But I get this error
$ bash -x script.sh
+ IFS=
+ read -r url
++ echo https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno
++ sed 'shttps://www.discoveryplus.com/it/video/\(.*\)/.*~\1'
sed: -e expression #1, char 52: unterminated `s' command
+ folder_name=
+ echo 'folder_name: '
folder_name:
+ mkdir -p /cygdrive/c/cygwin64/home/Administrator/
+ echo 'The script is running and creating folder: '
+ yt-dlp --referer https://www.discoveryplus.com/ --cookies /cygdrive/c/cygwin64/home/Administrator/cookies.txt -a https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno -o '/cygdrive/c/cygwin64/home/Administrator//%(title)s.%(ext)s'
ERROR: batch file https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno could not be read
+ IFS=
+ read -r url
++ echo https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas
++ sed 'shttps://www.discoveryplus.com/it/video/\(.*\)/.*~\1'
sed: -e expression #1, char 52: unterminated `s' command
+ folder_name=
+ echo 'folder_name: '
folder_name:
+ mkdir -p /cygdrive/c/cygwin64/home/Administrator/
+ echo 'The script is running and creating folder: '
+ yt-dlp --referer https://www.discoveryplus.com/ --cookies /cygdrive/c/cygwin64/home/Administrator/cookies.txt -a https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas -o '/cygdrive/c/cygwin64/home/Administrator//%(title)s.%(ext)s'
ERROR: batch file https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas could not be read
+ IFS=
+ read -r url
script.sh have this code
#!/bin/bash
while IFS= read -r url; do
folder_name=$(echo "$url" | sed 'shttps://www.discoveryplus.com/it/video/\(.*\)/.*~\1')
echo "folder_name: $folder_name"
mkdir -p "/cygdrive/c/cygwin64/home/Administrator/$folder_name"
echo "The script is running and creating folder: $folder_name" > ~/script.log
yt-dlp --referer "https://www.discoveryplus.com/" --cookies "/cygdrive/c/cygwin64/home/Administrator/cookies.txt" -a "$url" -o "/cygdrive/c/cygwin64/home/Administrator/$folder_name/%(title)s.%(ext)s"
done < /cygdrive/c/cygwin64/home/Administrator/links_da_scaricare.txt
links_da_scaricare.txt have these 3 lines - I use notepad to copy paste these links
https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno
https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas
https://www.discoveryplus.com/it/video/il-boss-delle-cerimonie/stagione-5-il-quarantesimo-compleanno-di-nello
cookies.txt is this - is the same that I use in powershell command so it should download because with powershell I can do it but with bash i have problems, I don't know why
gcl_au=1.1.192985240.1664694131; _scid=62382352-4832-438c-a74d-30699989ca96; _fbp=fb.1.1664694351012.468612256; __telemetric.v=1599968982.1664694367.81679; __zlcmid=1CFlGmqJqevtvtz; st=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJVU0VSSUQ6ZHBsYXk6MzEzNzY0NDYtOTkwMC00ODVlLWE3YjYtYTkyYzc2MDJkZWE4IiwianRpIjoidG9rZW4tNTA5ZTdiZWYtOTBmMC00ZmVjLWJlNWMtNzJhZmNiZThlYzZmIiwiYW5vbnltb3VzIjpmYWxzZSwiaWF0IjoxNjY0Njk0MzkzfQ.kH5QmwI2CH0-baBgJMGVoalTHbMcwOtH8Go_G2mEz6c; aam_uuid=50034166116354787000738543341305359891; _tt_enable_cookie=1; _ttp=45eb20c2-a55e-4740-be63-01b5e4166bd3; _hjSessionUser_2533509=eyJpZCI6IjM5NDAxNDA3LTllZDQtNWI1ZC1iYjY4LTRiNDlhOWIzMzZhOSIsImNyZWF0ZWQiOjE2NjQ2OTQzNjcxNDQsImV4aXN0aW5nIjp0cnVlfQ==; AMCV_9AE0F0145936E3790A495CAA#AdobeOrg=-1124106680|MCIDTS|19268|MCMID|50189272745869278580790010530934545485|MCAAMLH-1665300361|6|MCAAMB-1665300361|RKhpRz8krg2tLO6pguXWp5olkAcUniQYPHaMWWgdJ3xzPWQmdj0y|MCOPTOUT-1664702762s|NONE|MCSYNCSOP|411-19275|vVersion|5.2.0; __telemetric.s=1.1664980917.1664982717; _hjSession_2533509=eyJpZCI6ImVhNmZmNTA3LTM1ZGYtNDBlZC1iMWFjLTA0MzRlYzZlYjJjNCIsImNyZWF0ZWQiOjE2NjQ5ODA5MTc5MTEsImluU2FtcGxlIjpmYWxzZX0=; _hjAbsoluteSessionInProgress=0; _clck=ji7lnf|1|f5g|0; fs_uid=#161CV1#6518423911501824:5976792460267520:::#/1696517470; _uetsid=2596f2d044bd11ed80c355f240f55627; _uetvid=c8269d50422011ed952c2140169cc91d; ABTastySession=mrasn=&sen=9&lp=https%3A%2F%2Fwww.discoveryplus.com%2Fit; ABTasty=uid=4hnc3e5beewkxmkz&fst=1664694364438&pst=1664813235740&cst=1664981468233&ns=4&pvt=21&pvis=10&th=; _clsk=6lrbo9|1664981584916|8|0|j.clarity.ms/collect
In other words script.sh should be do this
--> read urls from links_da_scaricare.txt and extract only these strings
citta-mortale
il-boss-delle-cerimonie
--> make folders with that names
--> download
https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno
https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas
in this folder
citta-mortale
--> download
https://www.discoveryplus.com/it/video/il-boss-delle-cerimonie/stagione-5-il-quarantesimo-compleanno-di-nello
--> in this folder
il-boss-delle-cerimonie
Script creates the first folder but doesn't download anything inside it
You are looking to download files into a directory named according to the 6th field delimited by "/" in the url specification (between slash #5 and #6).
That is accomplished as
folder_name=$( echo "${url}" | cut -f6 -d/ )
Regarding sed, you must specify a delimiter character which does not conflict with any possible characters that might be found in the text string that you are trying to edit.
For URLs that always have "/", you cannot use "/" as that delimiter. You must choose something else. I tend to use either the "+" or the "|" depending on circumstances.
So to strip the prefix from the URL string, you would use
echo "${url}" | sed 's+https://www.discoveryplus.com/it/video/++'
to obtain
citta-mortale/stagione-1-episodio-1-strada-per-linferno
citta-mortale/stagione-1-episodio-2-non-in-kansas
il-boss-delle-cerimonie/stagione-5-il-quarantesimo-compleanno-di-nello
You would make a second, similar, usage of sed to strip the tail end after the "/".
OR ... you can again use cut in this manner
id_string=$( echo "${url}" | cut -f7 -d/ )
to obtain
stagione-1-episodio-1-strada-per-linferno
stagione-1-episodio-2-non-in-kansas
stagione-5-il-quarantesimo-compleanno-di-nello

Bash script: Using variables in executing sed in Freebsd which expects \ afters a

I'm trying to use variables in a sed command in Freebsd. sed in Freebsd expects \ after a. Basically I want to append a line if a particular line in the file matches a pattern. I'm using sed's append for that.
#!/usr/bin/bash
SYSLOG_SERVER="192.168.1.36"
SYSLOG_PORT="514"
syslog_conf_file="/etc/syslog.conf"
send_logs() {
logs=(messages auth.log )
send_logs[0]=`awk '(index($2, "messages") != 0) {print $1}' $syslog_conf_file`
send_logs[1]=`awk '(index($2, "auth.log") != 0) {print $1}' $syslog_conf_file`
for (( i = 0 ; i < ${#send_logs[#]} ; i++ ))
do
if [ ! -z "${send_logs[$i]}" ]; then
send_logs[i]=${send_logs[i]}" \t"#$SYSLOG_SERVER:$SYSLOG_PORT
sed "/${logs[$i]}$/a\
${send_logs[$i]} \
" $syslog_conf_file
fi
done
}
I'm facing this error. The variables are printed properly but the way in which I'm running the script is wrong. How can I fix this ?
root#Great# bash temp.sh
send_logs *.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err \t#192.168.1.36:514
logs messages
sed: 1: "/messages$/a ...": command a expects \ followed by text
send_logs auth.info;authpriv.info \t#192.168.1.36:514
logs auth.log
sed: 1: "/auth.log$/a ...": command a expects \ followed by text
Sample expected input for sed:
root#Great# sed '/messages$/a\
*.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err #192.168.1.36:514 \
' /etc/syslog.conf
Expected output:
*.err;kern.warning;auth.notice;mail.crit /dev/console
*.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err /var/log/messages
*.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err #192.168.1.36:514
security.* /var/log/security
auth.info;authpriv.info /var/log/auth.log
It's because \ followed by a newline character means following line to be joined, to avoid escape : \\ :
sed "/${logs[$i]}$/a\\
${send_logs[$i]} \\
" $syslog_conf_file

Net::OpenSSH command remote with multi pipeline

i have a problem when attempt run multi command in remote linux, with Perl and Module Net::OpenSSH. .
use Net::OpenSSH;
my $new_connect = Net::OpenSSH->new($D_CONNECT{'HOST'}, %OPT);
my $file = "file.log.gz"
my ($output2, $error2) = $new_connect->capture({ timeout => 10 }, "gunzip -c /path/to/file/$file | tail -n1 | awk '/successfully completed/ {print \$NF}'");
the output that i get is:
bash: -c: line 1: syntax error near unexpected token |'
bash: -c: line 1: |tail -n1 |awk '/successfully completed/ {print $NF}''
;;;
any idea or suggestion, thanks.
Fcs
Probably a quoting error. Just let Net::OpenSSH take care of the quoting for you:
my ($output2, $error2) = $new_connect->capture({ timeout => 10 },
'gunzip', '-c', "/path/to/file/$file", \\'|',
'tail', '-n1', \\'|',
'awk', '/successfully completed/ {print $NF}');
Note how the pipes (|) are passed as a double reference so that they are passed unquoted to the remote shell. The module documentation has a section on quoting.
That looks like the error message you'd get if you had a newline at the end of your $file string, causing the pipe character to be at the start of a second line (interpreted as the start of a second command).
This test deomstrates the same error:
bash -c 'echo foo
| cat'
So I guess your bug doesn't really occur with $file = "file.log.gz" and whatever your real $file is, you need to chomp it.
A bigger mystery is why bash says the error is on line 1 of the -c. ash, ksh, and zsh all correctly report it on line 2.

sh random error when saving function output

Im trying to make a script that takes a .txt file containing lines
like:
davda103:David:Davidsson:800104-1234:TNCCC_1:TDDB46 TDDB80:
and then sort them etc. Thats just the background my problem lies here:
#!/bin/sh -x
cat $1 |
while read a
do
testsak = `echo $a | cut -f 1 -d :`; <---**
echo $testsak;
done
Where the arrow is, when I try to run this code I get some kind of weird error.
+ read a
+ cut -f+ echo 1 -d :davda103:David:Davidsson:800104-1234:TNCCC_1:TDDB46
TDDB80:
+ testsak = davda103
scriptTest.sh: testsak: Det går inte att hitta
+ echo
(I have my linux in swedish because school -.-) Anyways that error just says that it cant find... something. Any ideas what could be causing my problem?
You have extra spaces around the assignment operator, remove them:
testsak=`echo $a | cut -f 1 -d :`; <---**
The spaces around the equal sign
testsak = `echo $a | cut -f 1 -d :`; <---**
causes bash to interpret this as a command testak with arguments = and the result of the command substitution. Removing the spaces will fix the immediate error.
A much more efficient way to extract the value from a is to let read do it (and use input redirection instead of cat):
while IFS=: read testak the_rest; do
echo $testak
done < "$1"

Looking for exact match using grep

Suppose that I have a file like this:
tst.txt
fName1 lName1-a 222
fname1 lName1-b 22
fName1 lName1 2
And I want to get the 3rd column only for "fName1 lName1", using this command:
var=`grep -i -w "fName1 lName1" tst.txt`
However this returns me every line that starts with "fName1 lName1", how can I look for the exact match?
Here you go:
#!/bin/bash
var=$(grep -Po '(?<=fName1 lName1 ).+' tst.txt)
echo $var
The trick is to use the o option of the grep command. The P option tells the interpreter to use Perl-compatible regular expression syntax when parsing the pattern.
var=$(grep "fName1 lName1 " tst.txt |cut -d ' ' -f 3)
you can try this method:
grep -i -E "^fName1 lName1\s" tst.txt | cut -f3,3- -d ' '
But you must be sure that line starts with fName1 and you have space after lName1.

Resources