I have a text file:
org.jitsi.videobridge.xmpp.user.shard-1.HOSTNAME=localhost
org.jitsi.videobridge.xmpp.user.shard-1.DOMAIN=auth.jc.name.com
org.jitsi.videobridge.xmpp.user.shard-1.USERNAME=name
org.jitsi.videobridge.xmpp.user.shard-1.PASSWORD=Hfr*7462
org.jitsi.videobridge.xmpp.user.shard-1.MUC_JIDS=JvbBredjoy#internal.auth.jc.name.com
org.jitsi.videobridge.xmpp.user.shard-1.MUC_NICKNAME=7896aee5-fgre-4b02-4569-0bcc75ed1d0d
I created a bash script:
#!/bin/bash
DPATH="/etc/jitsi/videobridge/sip-communicator.properties"
k=$(grep -o 'shard-1' $DPATH) # shard ends by a number#
i=$(grep -o 'shard-1' $DPATH | cut -c7)
m=$((i+1))
n="shard-$m"
sed -i "s|${k}|${n}|g" $DPATH
But I get error:
/home/scripts# ./shard_number
./shard_number: line 5: 1
1
1
1
1
1: syntax error in expression (error token is "1
1
1
1
1")
sed: -e expression #1, char 9: unterminated `s' command
Could you please help to solve this issue? Thank you.
If you call your script with bash -x /path/to/your/script or add set -x somewhere at the start of your script (after the #!shebang, but before the commands you want to debug), you will see that your grep commands return not a single 'shard-1' but rather one 'shard-1' per line :
++ grep -o shard-1 /etc/jitsi/videobridge/sip-communicator.properties
+ k='shard-1
shard-1
shard-1
shard-1
shard-1
shard-1'
Once cut, that gives the 1\n1\n1\n1\n1\n string that is mentionned in your error output as an invalid token for the $(( ... )) expression, which also breaks the syntax of your sed substitution :
++ cut -c7
++ grep -o shard-1 /etc/jitsi/videobridge/sip-communicator.properties
+ i='1
1
1
1
1
1'
Make that string a single number (for instance piping your grep into sort -u to unicize all the shards found) and your script will work just fine :
#!/bin/bash
DPATH="/etc/jitsi/videobridge/sip-communicator.properties"
k=$(grep -o 'shard-1' $DPATH | sort -u) # shard ends by a number#
i=$(grep -o 'shard-1' $DPATH | sort -u | cut -c7)
m=$((i+1))
n="shard-$m"
sed -i "s|${k}|${n}|g" $DPATH
You can try it here. Also check this test if you want to see your initial script debugged.
Have this input:
bar foo
foo ABC/DEF
BAR ABC
ABC foo DEF
foo bar
on the above I need do 4 (sequential) actions:
select only lines containing "foo" (lowercase)
on the selected lines, remove everything but UPPERCASE letters
delete empty lines (if some is created by the previous action)
and on the remaining from the above - enclose every char with [x]
I'm able to solve the above, but need two sed invocations piped together. Script:
#!/bin/bash
data() {
cat <<EOF
bar foo
foo ABC/DEF
BAR ABC
ABC foo DEF
foo bar
EOF
}
echo "Result OK"
data | sed -n '/foo/s/[^A-Z]//gp' | sed '/^\s*$/d;s/./[&]/g'
# in the above it is solved using 2 sed invocations
# trying to solve it using only one invocation,
# but the following doesn't do what i need.. :( :(
echo "Variant 2 - trying to use only ONE invocation of sed"
data | sed -n '/foo/s/[^A-Z]//g;/^\s*$/d;s/./[&]/gp'
output from the above:
Result OK
[A][B][C][D][E][F]
[A][B][C][D][E][F]
Variant 2 - trying to use only ONE invocation of sed
[A][B][C][D][E][F]
[B][A][R][ ][A][B][C]
[A][B][C][D][E][F]
The variant 2 should be also only
[A][B][C][D][E][F]
[A][B][C][D][E][F]
It is possible to solve the above using only by one sed invocation?
sed -n '/foo/{s/[^A-Z]//g;/^$/d;s/./[&]/g;p;}' inputfile
Output:
[A][B][C][D][E][F]
[A][B][C][D][E][F]
Alternative sed approach:
sed '/foo/!d;s/[^A-Z]//g;/./!d;s/./[&]/g' file
The output:
[A][B][C][D][E][F]
[A][B][C][D][E][F]
/foo/!d - deletes all lines that don't contain foo
/./!d - deletes all empty lines
I have file, and I need replace one word in it.
I can find this word with many grep's and pipelines.
cat file | <many times grep here>
for example:
>> cat test.cpp | grep -o "\-\-window [0-9]* \"" | grep -o "[0-9]*"
>> 71303214
How can i change result number in this pipeline?
You can use look-ahead and look-behind as follows:
grep -Po '(?<=\-\-window )\d*(?= \")' file
It looks for digits (\d*) in between a block of --window_ and _" (_ stands for space).
If you want to replace with sed, use:
sed 's/--window \([0-9]*\) \"/\1/g' file
It looks for --window_digits_* and replaces them with digits (_ stands for space).
Update
If you want to replace it with another number, do:
sed 's/--window [0-9]* \"/new_number/g' file
Or you can even use a bash variable if you use double quotes (with single, it wouldn't expand the variable).
sed "s/--window [0-9]* \"/$new_number/g" file
Test
$ cat a
hello --window 234234 " a
hello --window 234234a " a
hello this is another thing
$ grep -Po '(?<=\-\-window )\d*(?= \")' a
234234
$ sed 's/--window \([0-9]*\) \"/\1/g' a
hello 234234 a
hello --window 234234a " a
hello this is another thing
$ sed 's/--window [0-9]* \"/XXX/g' a
hello XXX a
hello --window 234234a " a
hello this is another thing
$ number=22
$ sed "s/\(--window \)[0-9]*\( \"\)/\1$number\2/g" a
hello --window 22 " a
hello --window 234234a " a
hello this is another thing
sed -n '/--window [0-9]* \"/ {
s/[^[:digit:]]//gp
}' file
sed with file as input (so no need of cat pipe before)
use of -n for not printing output unless specific request (P in command)
first pattern between // (regex reduced so you need to escape some meta char like \ / . *)
(here) extraction of digit by removing other char of the line and print the result if it occur
As you see, grep is better in this case (more readeable and efficient)
I THINK all you're asking for is:
$ cat file
#define CLICK(x,y) system("xdotool mousemove --window 71303214 " #x" "#y " click 1");
$ var="888999"; sed "s/\(.*--window \)[^ ]*/\1$var/" file
#define CLICK(x,y) system("xdotool mousemove --window 888999 " #x" "#y " click 1");
If that's not it, update your question to show some representative sample input and expected output.
How to I concatenate stdin to a string, like this?
echo "input" | COMMAND "string"
and get
inputstring
A bit hacky, but this might be the shortest way to do what you asked in the question (use a pipe to accept stdout from echo "input" as stdin to another process / command:
echo "input" | awk '{print $1"string"}'
Output:
inputstring
What task are you exactly trying to accomplish? More context can get you more direction on a better solution.
Update - responding to comment:
#NoamRoss
The more idiomatic way of doing what you want is then:
echo 'http://dx.doi.org/'"$(pbpaste)"
The $(...) syntax is called command substitution. In short, it executes the commands enclosed in a new subshell, and substitutes the its stdout output to where the $(...) was invoked in the parent shell. So you would get, in effect:
echo 'http://dx.doi.org/'"rsif.2012.0125"
use cat - to read from stdin, and put it in $() to throw away the trailing newline
echo input | COMMAND "$(cat -)string"
However why don't you drop the pipe and grab the output of the left side in a command substitution:
COMMAND "$(echo input)string"
I'm often using pipes, so this tends to be an easy way to prefix and suffix stdin:
echo -n "my standard in" | cat <(echo -n "prefix... ") - <(echo " ...suffix")
prefix... my standard in ...suffix
There are some ways of accomplish this, i personally think the best is:
echo input | while read line; do echo $line string; done
Another can be by substituting "$" (end of line character) with "string" in a sed command:
echo input | sed "s/$/ string/g"
Why i prefer the former? Because it concatenates a string to stdin instantly, for example with the following command:
(echo input_one ;sleep 5; echo input_two ) | while read line; do echo $line string; done
you get immediatly the first output:
input_one string
and then after 5 seconds you get the other echo:
input_two string
On the other hand using "sed" first it performs all the content of the parenthesis and then it gives it to "sed", so the command
(echo input_one ;sleep 5; echo input_two ) | sed "s/$/ string/g"
will output both the lines
input_one string
input_two string
after 5 seconds.
This can be very useful in cases you are performing calls to functions which takes a long time to complete and want to be continuously updated about the output of the function.
You can do it with sed:
seq 5 | sed '$a\6'
seq 5 | sed '$ s/.*/\0 6/'
In your example:
echo input | sed 's/.*/\0string/'
I know this is a few years late, but you can accomplish this with the xargs -J option:
echo "input" | xargs -J "%" echo "%" "string"
And since it is xargs, you can do this on multiple lines of a file at once. If the file 'names' has three lines, like:
Adam
Bob
Charlie
You could do:
cat names | xargs -n 1 -J "%" echo "I like" "%" "because he is nice"
Also works:
seq -w 0 100 | xargs -I {} echo "string "{}
Will generate strings like:
string 000
string 001
string 002
string 003
string 004
...
The command you posted would take the string "input" use it as COMMAND's stdin stream, which would not produce the results you are looking for unless COMMAND first printed out the contents of its stdin and then printed out its command line arguments.
It seems like what you want to do is more close to command substitution.
http://www.gnu.org/software/bash/manual/html_node/Command-Substitution.html#Command-Substitution
With command substitution you can have a commandline like this:
echo input `COMMAND "string"`
This will first evaluate COMMAND with "string" as input, and then expand the results of that commands execution onto a line, replacing what's between the ‘`’ characters.
cat will be my choice: ls | cat - <(echo new line)
With perl
echo "input" | perl -ne 'print "prefix $_"'
Output:
prefix input
A solution using sd (basically a modern sed; much easier to use IMO):
# replace '$' (end of string marker) with 'Ipsum'
# the `e` flag disables multi-line matching (treats all lines as one)
$ echo "Lorem" | sd --flags e '$' 'Ipsum'
Lorem
Ipsum#no new line here
You might observe that Ipsum appears on a new line, and the output is missing a \n. The reason is echo's output ends in a \n, and you didn't tell sd to add a new \n. sd is technically correct because it's doing exactly what you are asking it to do and nothing else.
However this may not be what you want, so instead you can do this:
# replace '\n$' (new line, immediately followed by end of string) by 'Ipsum\n'
# don't forget to re-add the `\n` that you removed (if you want it)
$ echo "Lorem" | sd --flags e '\n$' 'Ipsum\n'
LoremIpsum
If you have a multi-line string, but you want to append to the end of each individual line:
$ ls
foo bar baz
$ ls | sd '\n' '/file\n'
bar/file
baz/file
foo/file
I want to prepend my sql script with "set" statement before running it.
So I echo the "set" instruction, then pipe it to cat. Command cat takes two parameters : STDIN marked as "-" and my sql file, cat joins both of them to one output. Next I pass the result to mysql command to run it as a script.
echo "set #ZERO_PRODUCTS_DISPLAY='$ZERO_PRODUCTS_DISPLAY';" | cat - sql/test_parameter.sql | mysql
p.s. mysql login and password stored in .my.cnf file
I have the following log file which contains lines like this
1345447800561|FINE|blah#13|txReq
1345447800561|FINE|blah#13|Req
1345447800561|FINE|blah#13|rxReq
1345447800561|FINE|blah#14|txReq
1345447800561|FINE|blah#15|Req
I am trying extract the first field from each line and depending on whether it belongs to blah#13 or blah#14, blah#15 i am creating the corresponding files using the following script, which seems quite in-efficient in terms of the number of temp files creates. Any suggestions on how I can optimize it ?
cat newLog | grep -i "org.arl.unet.maca.blah#13" >> maca13
cat newLog | grep -i "org.arl.unet.maca.blah#14" >> maca14
cat newLog | grep -i "org.arl.unet.maca.blah#15" >> maca15
cat maca10 | grep -i "txReq" >> maca10TxFrameNtf_temp
exec<blah10TxFrameNtf_temp
while read line
do
echo $line | cut -d '|' -f 1 >>maca10TxFrameNtf
done
cat maca10 | grep -i "Req" >> maca10RxFrameNtf_temp
while read line
do
echo $line | cut -d '|' -f 1 >>maca10TxFrameNtf
done
rm -rf *_temp
Something like this ?
for m in org.arl.unet.maca.blah#13 org.arl.unet.maca.blah#14 org.arl.unet.maca.blah#15
do
grep -i "$m" newLog | grep "txReq" | cut -d' ' -f1 > log.$m
done
I've found it useful at times to use ex instead of grep/sed to modify text files in place without using temps ... saves the trouble of worrying about uniqueness and writability to the temp file and its directory etc. Plus it just seemed cleaner.
In ksh I would use a code block with the edit commands and just pipe that into ex ...
{
# Any edit command that would work at the colon prompt of a vi editor will work
# This one was just a text substitution that would replace all contents of the line
# at line number ${NUMBER} with the word DATABASE ... which strangely enough was
# necessary at one time lol
# The wq is the "write/quit" command as you would enter it at the vi colon prompt
# which are essentially ex commands.
print "${NUMBER}s/.*/DATABASE/"
print "wq"
} | ex filename > /dev/null 2>&1