I'm running a dozen of sed commands for each Capistranio deploy and I was wondering, if it's possible to chain them into 1 single sed command, instead of firing dozens at the server.
task :taskname do
{:'foo' => foo, :'bar' => bar, :'foobar' => foobar, :'fubar' => fubar }.each do |search, replace|
run "sed -i 's/#{search}/#{replace}/' file.ext"
end
end
sed natively accepts a dozen of patterns (if you for some reason prefer sed):
{:foo => foo, :bar => bar, :foobar => foobar, :fubar => fubar}.inject("") do |acc, k, v|
acc += " -e 's/#{k}/#{v}'"
end
run "sed #{acc} file.ext"
Does mudasobwa's code work? With my Ruby (v1.9.3), it has to be:
acc = {:foo => foo, :bar => bar, :foobar => foobar, :fubar => fubar}.inject("") do |m, p|
m + " -e 's/#{p[0]}/#{p[1]}'"
end
run "sed #{acc} file.ext"
Related
I would like to take a string that contains positional argument markers (not named), supply it with an array (not hash) of values, and have it evaluated.
The use case as an example would be somewhat like ARGV.
For example,
# given:
string = "echo $1 ; echo $#"
values = ["hello", "world"]
# expected result:
"echo hello ; echo hello world"
The below function is the best I could come up with:
def evaluate_args(string, arguments)
return string unless arguments.is_a? Array and !arguments.empty?
# Create a variable that can replace $# with all arguments, and quote
# arguments that had "more than one word" originally
all_arguments = arguments.map{|a| a =~ /\s/ ? "\"#{a}\"" : a}.join ' '
# Replace all $1 - $9 with their respective argument ($1 ==> arguments[0])
string.gsub!(/\$(\d)/) { arguments[$1.to_i - 1] }
# Replace $# or $* with all arguments
string.gsub!(/\$[*|#]/, all_arguments)
return string
end
And it seems to me like it can and should be simpler.
I was hoping to find something that is closer to the Kernel.sprintf method of doing things - like "string with %{marker}" % {marker: 'value'}
So, although this issue is almost solved for me (I think), I would love to know if there is something I missed that can make it more elegant.
It seems like you're trying to reproduce Bash-style variable expansion, which is an extremely complex problem. At the very least, though, you can simplify your code in two ways:
Use Kernel.sprintf's built in positional argument feature. The below code does this by substituting e.g. $1 with the sprintf equivalent %1$s.
Use Shellwords from the standard library to escape arguments with spaces etc.
require 'shellwords'
def evaluate_args(string, arguments)
return string unless arguments.is_a? Array and !arguments.empty?
tmpl = string.gsub(/\$(\d+)/, '%\1$s')
(tmpl % arguments).gsub(/\$[*#]/, arguments.shelljoin)
end
string = "echo $1 ; echo $#"
values = ["hello", "world"]
puts evaluate_args(string, values)
# => echo hello ; echo hello world
If you didn't have the $* requirement I'd suggest just dropping the Bash-like format and just using sprintf, since it covers everything else you mentioned. Even so, you could further simplify things by using sprintf formatting for everything else:
def evaluate_args(string, arguments)
return string unless arguments.is_a? Array and !arguments.empty?
string.gsub('%#', arguments.shelljoin) % arguments
end
string = "echo %1$s ; echo %#"
values = ["hello", "world"]
puts evaluate_args(string, values)
# => echo hello ; echo hello world
Edit
If you want to use %{1} with sprintf you could turn the input array into a hash where the integer indexes are turned into symbol keys, e.g. ["hello", "world"] becomes { :"1" => "hello", :"2" => "world" }:
require "shellwords"
def evaluate_args(string, arguments)
return string unless arguments.is_a? Array and !arguments.empty?
string % {
:* => arguments.shelljoin,
**arguments.map.with_index {|val,idx| [ :"#{idx + 1}", val ] }.to_h
}
end
string = "echo %{1} ; echo %{*}"
values = ["hello", "world"]
puts evaluate_args(string, values)
# => echo hello ; echo hello world
string = "echo $1 ; echo $# ; echo $2 ; echo $cat"
values = ["hello", "World War II"]
vals = values.map { |s| s.include?(' ') ? "\"#{s}\"" : s }
#=> ["hello", "\"World War II\""]
all_vals = vals.join(' ')
#=> "hello \"World War II\""
string.gsub(/\$\d+|\$[#*]/) { |s| s[/\$\d/] ? vals[s[1..-1].to_i-1] : all_vals }
#=> "echo hello ; echo hello \"World War II\" ; echo \"World War II\" ; echo $cat" $cat"
I have this line in a bash script and I can't figure out what the "=>" means? I don't think it means equal to or greater than but maybe it does. Thoughts?
"echo '\"postgres\" => { \"archive_timeout\" => 300, \"backup\" => 1, \"base_backup_interval\" => 3600, \"restore\" => 1 },' >> /tmp/user_data.config\n",
It doesn't mean anything, because it's inside a string. Consider:
$ echo 'foo'
foo
$ echo 'foo => bar'
foo => bar
The => doesn't have any significance; it's just part of the string that echo writes to its output.
In the case of your code, the echo command and its string argument are followed by >> /tmp/user_data.config, which just means that the output will be appended to the user_data.config file. Like so:
$ touch /tmp/out.txt
$ echo 'foo => bar' >> /tmp/out.txt
$ echo 'baz => qux' >> /tmp/out.txt
$ cat /tmp/out.txt
foo => bar
baz => qux
The => is part of the string that is being echo'd. Try running the command (without the " at the beginning and the \n", at the end) in BASH and you'll see it just echoes the string and appends it to /tmp/user_data.config
$ echo '\"postgres\" => { \"archive_timeout\" => 300, \"backup\" => 1, \"base_backup_interval\" => 3600, \"restore\" => 1 },' >> /tmp/user_data.config
$ cat /tmp/user_data.config
\"postgres\" => { \"archive_timeout\" => 300, \"backup\" => 1, \"base_backup_interval\" => 3600, \"restore\" => 1 },
I have spent hours on this but can't crack it. I am using sed on OSX .
This is the code:
sed -i.bak "s/^\(\$dokuwiki_hash.*\)$/\1\n '"$date"' => '"$hash"',/" install.php
And the output that I get which is wrong is (see the first line):
$dokuwiki_hash = array(n '2013-03-17' => '7b62b75245f57f122d3e0f8ed7989623',
'2005-09-22' => 'e33223e957b0b0a130d0520db08f8fb7',
'2006-03-05' => '51295727f79ab9af309a2fd9e0b61acc',
'2006-03-09' => '51295727f79ab9af309a2fd9e0b61acc',
'2006-11-06' => 'b3a8af76845977c2000d85d6990dd72b',
'2007-05-24' => 'd80f2740c84c4a6a791fd3c7a353536f',
'2007-06-26' => 'b3ca19c7a654823144119980be73cd77',
'2008-05-04' => '1e5c42eac3219d9e21927c39e3240aad',
'2009-02-14' => 'ec8c04210732a14fdfce0f7f6eead865',
'2009-12-25' => '993c4b2b385643efe5abf8e7010e11f4',
'2010-11-07' => '7921d48195f4db21b8ead6d9bea801b8',
'2011-05-25' => '4241865472edb6fa14a1227721008072',
'2011-11-10' => 'b46ff19a7587966ac4df61cbab1b8b31',
'2012-01-25' => '72c083c73608fc43c586901fd5dabb74',
'2012-09-10' => 'eb0b3fc90056fbc12bac6f49f7764df3',
'2013-04-06' => '7b62b75245f57f122d3e0f8ed7989623',
);
It should be on a new line like below:
$dokuwiki_hash = array(
'2013-03-17' => '7b62b75245f57f122d3e0f8ed7989623',
'2005-09-22' => 'e33223e957b0b0a130d0520db08f8fb7',
'2006-03-05' => '51295727f79ab9af309a2fd9e0b61acc',
'2006-03-09' => '51295727f79ab9af309a2fd9e0b61acc',
'2006-11-06' => 'b3a8af76845977c2000d85d6990dd72b',
'2007-05-24' => 'd80f2740c84c4a6a791fd3c7a353536f',
'2007-06-26' => 'b3ca19c7a654823144119980be73cd77',
'2008-05-04' => '1e5c42eac3219d9e21927c39e3240aad',
'2009-02-14' => 'ec8c04210732a14fdfce0f7f6eead865',
'2009-12-25' => '993c4b2b385643efe5abf8e7010e11f4',
'2010-11-07' => '7921d48195f4db21b8ead6d9bea801b8',
'2011-05-25' => '4241865472edb6fa14a1227721008072',
'2011-11-10' => 'b46ff19a7587966ac4df61cbab1b8b31',
'2012-01-25' => '72c083c73608fc43c586901fd5dabb74',
'2012-09-10' => 'eb0b3fc90056fbc12bac6f49f7764df3',
'2013-04-06' => '7b62b75245f57f122d3e0f8ed7989623',
);
Any help would be greatly appreciated !
Don't use substitute for this, sed has a perfectly good append command for adding a line after the current one, without fiddling around with newline or storage of regex results:
pax> echo '$dokuwiki_hash = array(
'"'"'2013-03-17'"'"' => '"'"'7b62b75245f57f122d3e0f8ed7989623'"'"'
);' | sed '/^\$dokuwiki_hash = /a\ blah '"'"'blah'"'"' blah'
$dokuwiki_hash = array(
blah 'blah' blah
'2013-03-17' => '7b62b75245f57f122d3e0f8ed7989623'
);
The machinations with the quotes are to allow you to put literal single quotes within the command.
Alternatively, you can use double quotes on the outside, you just have to be careful that the shell doesn't interpret your dollar-variables:
pax> echo "\$dokuwiki_hash = array(
'2013-03-17' => '7b62b75245f57f122d3e0f8ed7989623'
);" | sed "/^\$dokuwiki_hash = /a\ blah 'blah' blah"
$dokuwiki_hash = array(
blah 'blah' blah
'2013-03-17' => '7b62b75245f57f122d3e0f8ed7989623'
);
There's also an opposing insert command i for inserting before the current line but it's append you want in this case.
And, if you're having trouble with mixing quote types (perhaps due to an older bash under OSX), you can put the sed commands into a file and use sed -f to run them:
pax> cat qq.sed
/^$dokuwiki_hash = /a\ blah 'blah' blah
pax> echo '$dokuwiki_hash = array(
'"'"'2013-03-17'"'"' => '"'"'7b62b75245f57f122d3e0f8ed7989623'"'"'
);' | sed -f qq.sed
$dokuwiki_hash = array(
blah 'blah' blah
'2013-03-17' => '7b62b75245f57f122d3e0f8ed7989623'
);
That gets around any quoting battles between the shell and sed. If that still doesn't work, see this link, which suggests installing GNU sed instead.
\n is not supported as a newline character in the replacement part of the substitute command of regular sed (only in GNU sed).
For example to prepend a newline to a pattern, instead of
sed 's/pattern/\n&/' file
use
sed 's/pattern/\
&' file
The \ should be the last character on the line.
Why doesn't the regex (?<=fo).* match foo (whereas (?<=f).* does)?
"foo" =~ /(?<=f).*/m => 1
"foo" =~ /(?<=fo).*/m => nil
This only seems to happen with singleline mode turned on (dot matches newline); without it, everything is OK:
"foo" =~ /(?<=f).*/ => 1
"foo" =~ /(?<=fo).*/ => 2
Tested on Ruby 1.9.3 and 2.0.0.
See it on Rubular
EDIT: Some more observations:
Adding an end-of-line anchor doesn't change anything:
"foo" =~ /(?<=fo).*$/m => nil
But together with a lazy quantifier, it "works":
"foo" =~ /(?<=fo).*?$/m => 2
EDIT: And some more observations:
.+ works as does its equivalent {1,}, but only in Ruby 1.9 (it seems that that's the only behavioral difference between the two in this scenario):
"foo" =~ /(?<=fo).+/m => 2
"foo" =~ /(?<=fo).{1,}/ => 2
In Ruby 2.0:
"foo" =~ /(?<=fo).+/m => nil
"foo" =~ /(?<=fo).{1,}/m => nil
.{0,} is busted (in both 1.9 and 2.0):
"foo" =~ /(?<=fo).{0,}/m => nil
But {n,m} works in both:
"foo" =~ /(?<=fo).{0,1}/m => 2
"foo" =~ /(?<=fo).{0,2}/m => 2
"foo" =~ /(?<=fo).{0,999}/m => 2
"foo" =~ /(?<=fo).{1,999}/m => 2
This has been officially classified as a bug and subsequently fixed, together with another problem concerning \Z anchors in multiline strings.
Given the following code:
options = {}
optparse = OptionParser.new do |opts|
opts.on('-t', '--thing [THING1,THING2]', Array, 'Set THING1, THING2') do |t|
options[:things] = t
end
end
If THING1 has a comma in it, how can I prevent OptionParser from splitting on it?
Sample case: ./scrit.rb -t 'foo,bar',baz. In this case I want options[:things] should be ['foo,bar', 'baz']
Is this even possible?
If your run:
./scrit.rb -t 'foo,bar',baz
shell pass ARGV:
["-t", "foo,bar,baz"]
Shell converts 'foo,bar',baz to foo,bar,baz:
$ strace -e trace=execve ./scrit.rb -t 'foo,bar',baz
execve("./scrit.rb", ["./scrit.rb", "-t", "foo,bar,baz"], [/* 52 vars */]) = 0
execve("/home/scuawn/bin/ruby", ["ruby", "./scrit.rb", "-t", "foo,bar,baz"], [/* 52 vars */]) = 0
You can use other delimiter:
opts.on('-t', '--thing [THING1,THING2]', Array, 'Set THING1, THING2') do |t|
options[:things] = t
options[:things][0] = options[:things][0].split(":")
end
$ ./scrit.rb -t foo:bar,baz
[["foo", "bar"], "baz"]
Or:
opts.on('-t', '--thing [THING1,THING2]', Array, 'Set THING1, THING2') do |t|
options[:things] = t
options[:things] = options[:things].length == 3 ? [[options[:things][0],options[:things][1]],options[:things][2]] : options[:things]
end
$ ./scrit.rb -t foo,bar,baz
[["foo", "bar"], "baz"]
First of all, the shell1 yields the same final value for all of the following quoting variations:
./scrit.rb -t 'foo,bar',baz
./scrit.rb -t foo,'bar,baz'
./scrit.rb -t 'foo,bar,baz'
./scrit.rb -t foo,bar,baz
./scrit.rb -t fo"o,b"ar,baz
./scrit.rb -t foo,b\ar,baz
# obviously many more variations are possible
You can verify this like so:
ruby -e 'f=ARGV[0];ARGV.each_with_index{|a,i|puts "%u: %s <%s>\n" % [i,a==f,a]}'\
'foo,bar',baz foo,'bar,baz' 'foo,bar,baz' foo,bar,baz fo"o,b"ar,baz foo,b\ar,baz
1 I am assuming a Bourne-like shell (some sh-variant like zsh, bash, ksh, dash, et cetera).
If you want to switch to some other separator, you might do it like this:
split_on_semicolons = Object.new
OptionParser.accept split_on_semicolons do |s,|
s.split ';'
end
⋮
opts.on('-t', '--thing [THING1;THING2]', split_on_semicolons, 'Set THING1, THING2 (semicolon must be quoted to protect it from the shell)') do |t|
options[:things] = t
end
The shell gives special meaning to the semicolon, so it must be escape or quoted (otherwise it serves as an unconditional command separator (e.g. echo foo; sleep 2; echo bar)):
./scrit.rb -t foo,bar\;baz
./scrit.rb -t foo,bar';'baz
./scrit.rb -t 'foo,bar;baz'
# et cetera
The “parsing” done when you specify Array is almost exactly a basic str.split(',') (it also drops empty string values), so there is no way to directly specify an escape character.
If you want to stick with commas but introduce an “escape character”, then you could post-process the values a bit in your OptionParser#on block to stitch certain values back together:
# use backslash as an after-the-fact escape character
# in a sequence of string values,
# if a value ends with a odd number of backslashes, then
# the last backslash should be replaced with
# a command concatenated with the next value
# a backslash before any other single character is removed
#
# basic unsplit: (note doubled backslashes due to writing these as Ruby values)
# %w[foo\\ bar baz] => %w[foo,bar baz]
#
# escaped, trailing backslash is not an unsplit:
# %w[foo\\\\ bar baz] => %w[foo\\ bar baz]
#
# escaping [other, backslash, split], also consecutive unsplits
# %w[f\\o\\\\o\\ \\\\\\bar\\\\\\ baz] => %w[fo\\o,\\bar\\,baz]
def unsplit_and_unescape(orig_values)
values = []
incompleteValue = nil
orig_values.each do |val|
incomplete = /\\*$/.match(val)[0].length.odd?
val.gsub! /\\(.)/, '\1'
val = incompleteValue + ',' + val if incompleteValue
if incomplete
incompleteValue = val[0..-2]
else
values << val
incompleteValue = nil
end
end
if incompleteValue
raise ArgumentError, 'Incomplete final value'
end
values
end
⋮
opts.on('-t', '--thing [THING1,THING2]', Array, 'Set THING1, THING2 (use \\, to include a comma)') do |t|
options[:things] = unsplit_and_unescape(t)
end
You could then run it from the shell like this (the backslash is also special to the shell, so it must be escaped or quoted2):
./scrit.rb -t foo\\,bar,baz
./scrit.rb -t 'foo\,bar,baz'
./scrit.rb -t foo'\,'bar,baz
./scrit.rb -t "foo\\,bar,baz"
./scrit.rb -t fo"o\\,ba"r,baz
# et cetera
2 Unlike in Ruby, the shell’s single quote is completely literal (e.g. no backslashes are interpreted), so it is often a good choice when you need to embed any other shell-special characters (like backslashes and double quotes).