I have a string:
string = "\\"
puts string
# => \
I am interpolating this into a new string and sending to a database. However the database (InfluxDB) uses backslashes as escape characters so pushing this string can cause an error.
For example, if I pass the following to Influx it will cause an "unterminated string" error:
insert_cmd = <<-TXT
INSERT INTO my_db.default.my_measurement,my_tag=1 my_val="#{string}"
TXT
My question is how can I replace \ in a string with \\ (two actual backslashes).
I have it working with gsub("\\", "\\\\\\") but I don't understand why this works and the following doesn't:
string.gsub("\\", "\\\\")
# SyntaxError: (irb):10: syntax error, unexpected $undefined, expecting end-of-input
Why doesn't this work? Why does gsub("\\", "\\\\\\") work? Is there a better way?
solved
As I mentioned in a comment, actually I am not manually interpolating into a INSERT INTO string. I am using influxdb-ruby:
INFLUXDB_CLIENT.write_point("things", time: Time.now.to_i, values: { foo: "\\" })
It turns out this is a bug with that gem: https://github.com/influxdata/influxdb-ruby/issues/200
It is fixed in v 0.4.2 and i was using 0.4.1
You just use parameterized query strings:
INSERT INTO my_db.default.my_measurement,my_tag=1 my_val=%{1}
Where when you call it you do this:
influxdb.query("...query...", params: [ string ])
What you did was create a classic injection bug by sending unescaped data into a query. The same principle applies in any database with a plain-text string representation, or even other data formats like HTML and JavaScript.
Related
I am trying to use file name path (Ex: C:\Document\Report.txt) as a parameter through uipath orchastrator api. I tried different approach and In each approach I am getting Bad request error "{"message":"Argument Values validation failed.","errorCode":2003,"resourceIds":null}"
Below is my Example code
FileListUploaded ="C\\Documents\\report.txt";
string parameter1 = "{\"startInfo\": {\"ReleaseKey\": \"xxxxx-xxx-xxx-xxx-xxxxxx\"," +
"\"RobotIds\": [xxxxx]," +
"\"JobsCount\": 0," +
"\"InputArguments\": \"{ "+
"\\\"reports_or_other_files\\\": \\\" + FileListUploaded + \\\"}\"}}";
request_startRobot.AddParameter("application/json; charset=utf-16", parameter, ParameterType.RequestBody);
IRestResponse response_startRobot = client_startRobot.Execute(request_startRobot);
That's a bit messy to look at, but it appears you are not quoting and escaping your JSON correctly.
I might suggest building an array and serializing it into JSON to make it easier to read, or using a HEREDOC or string formatting. If you do continue to concatenate your JSON body string together, dump out the results to see how it is coming together.
The final results of the JSON should look something like
{
"startInfo": {
"ReleaseKey":"{{uipath_releaseKey}}",
"Strategy":"JobsCount",
"JobsCount":1,
"InputArguments":"{\"reports_or_other_files\":\"C:\\\\Documents\\\\report.txt\"}"
}
}
With the InputArguments:
Looks like you are missing some quotes
Might need to double escape your backslashes in the FileListUploaded variable
Missing a colon after the C in the path
I have a method Embed.toggler that takes a hash argument. With the following code, I'm trying to use a heredoc in the hash.
Embed.toggler({
title: <<-RUBY
#{entry['time']}
#{entry['group']['who']
#{entry['name']}
RUBY
content: content
})
However, I'm getting the following error trace:
syntax error, unexpected ':', expecting tSTRING_DEND
content: content
^
can't find string "RUBY" anywhere before EOF
syntax error, unexpected end-of-input, expecting tSTRING_CONTENT or tSTRING_DBEG or tSTRING_DVAR or tSTRING_END
title: <<-RUBY
^
How I can avoid getting this error?
Add a comma after your <<-RUBY:
Embed.toggler({
title: <<-RUBY,
#{entry['time']}
#{entry['group']['who']
#{entry['name']}
RUBY
content: content
})
this does work in general. I am not sure why it wasn't working in my code though.
It didn't work because hashes require key/value pair to be separated by a comma, like {title: 'my title', content: 'my content' } and your code just didn't have the comma. It was hard to see that because of the cumbersome HEREDOC syntax.
Do you know if there is a way to perform operations on the string?
You're playing with fire. It's always safer (and usually cleaner) to extract a variable and do post-processing on a variable itself:
title = <<-RUBY
#{entry['time']}
#{entry['group']['who']
#{entry['name']}
RUBY
Embed.toggler(title: title.upcase, content: content)
However, if you feel dangerous today, you can just add operations after opening HEREDOC literal, just as you've added the comma:
Embed.toggler({
title: <<-RUBY.upcase,
#{entry['time']}
#{entry['group']['who']
#{entry['name']}
RUBY
content: content
})
But I discourage you from this because it destroys readability.
The Logstash filter regular expression to parse our syslog stream is getting more and more complicated, which led me to write tests. I simply copied the structure of a Grok test in the main Logstash repository, modified it a bit, and ran it with bin/logstash rspec as explained here. After a few hours of fighting with the regular expression syntax, I found out that there is a difference in how modifier characters have to be escaped. Here is a simple test for a filter involving square brackets in the log message, which you have to escape in the filter regular expression:
require "test_utils"
require "logstash/filters/grok"
describe LogStash::Filters::Grok do
extend LogStash::RSpec
describe "Grok pattern difference" do
config <<-CONFIG
filter {
grok {
match => [ "message", '%{PROG:theprocess}(?<forgetthis>(: )?(\\[[\\d:|\\s\\w/]*\\])?:?)%{GREEDYDATA:message}' ]
add_field => { "process" => "%{theprocess}" "forget_this" => "%{forgetthis}" }
}
}
CONFIG
sample "uwsgi: [pid: 12345|app: 0|req: 21/93281] BLAHBLAH" do
insist { subject["tags"] }.nil?
insist { subject["process"] } == "uwsgi"
insist { subject["forget_this"] } == ": [pid: 12345|app: 0|req: 21/93281]"
insist { subject["message"] } == "BLAHBLAH"
end
end
end
Save this as e.g. grok_demo.rb and test it with bin/logstash rspec grok_demo.rb, and it will work. If you remove the double escapes in the regexp, though, it won't.
I wanted to try the same thing in straight Ruby, using the same regular expression library that Logstash uses, and followed the directions given here. The following test worked as expected, without the need for double escape:
require 'rubygems'
require 'grok-pure'
grok = Grok.new
grok.add_patterns_from_file("/Users/ulas/temp/grok_patterns.txt")
pattern = '%{PROG:theprocess}(?<forgetthis>(: )?(\[[\d:|\s\w/]*\])?:?)%{GREEDYDATA:message}'
grok.compile(pattern)
text1 = 'uwsgi: [pid: 12345|app: 0|req: 21/93281] BLAHBLAH'
puts grok.match(text1).captures()
I'm not a Ruby programmer, and am a bit lost as to what causes this difference. Is it possible that the heredoc config specification necessitates double escapes? Or does it have to do with the way the regular expression gets passed to the regexp library within Logstash?
I never worked with writing tests for logstash before, but my guess is the double escape is due to the fact that you have strings embedded in strings.
The section:
<<-CONFIG
# stuff here
CONFIG
Is a heredoc in ruby (which is a fancy way to generate a string). So the filter, grok, match, add and all the brackets/braces are actually part of the string. Inside this string, you are escaping the escape sequence so the resulting string has a single literal escape sequence. I'm guessing that this string gets eval'd somewhere so that all the filter etc. stuff gets implemented as needed and that's where the single escape sequence is getting used.
When using "straight ruby" you aren't doing this double interpretation. You're just passing a string directly into the method to compile it.
I’m confused as to why this throws an error:
s = <<JSON
{"s": "This is \"valid\" JSON"}
JSON
JSON.parse(s) # => JSON::ParserError: 757: unexpected token at '{"s": "This is "valid" JSON"}'
Based on using http://jsonlint.com I can confirm that this is valid JSON, so what’s the deal? I get the feeling that I could be using %q{} here and things would be escaped properly, but I’d really rather use a heredoc here.
It turns out that Ruby supports disabling interpolation in heredocs by surrounding the opening identifier with single quotes, so in my example above, it would look like this:
s = <<'JSON'
{"s": "This is \"valid\" JSON"}
JSON
JSON.parse(s) # => {"s"=>"This is \"valid\" JSON"}
So here is my problem.
I want to retrieve a string stored in a model and at runtime change a part of it using a variable from the rails application. Here is an example:
I have a Message model, which I use to store several unique messages. So different users have the same message, but I want to be able to show their name in the middle of the message, e.g.,
"Hi #{user.name}, ...."
I tried to store exactly that in the database but it gets escaped before showing in the view or gets interpolated when storing in the database, via the rails console.
Thanks in advance.
I don't see a reason to define custom string helper functions. Ruby offers very nice formatting approaches, e.g.:
"Hello %s" % ['world']
or
"Hello %{subject}" % { subject: 'world' }
Both examples return "Hello world".
If you want
"Hi #{user.name}, ...."
in your database, use single quotes or escape the # with a backslash to keep Ruby from interpolating the #{} stuff right away:
s = 'Hi #{user.name}, ....'
s = "Hi \#{user.name}, ...."
Then, later when you want to do the interpolation you could, if you were daring or trusted yourself, use eval:
s = pull_the_string_from_the_database
msg = eval '"' + s + '"'
Note that you'll have to turn s into a double quoted string in order for the eval to work. This will work but it isn't the nicest approach and leaves you open to all sorts of strange and confusing errors; it should be okay as long as you (or other trusted people) are writing the strings.
I think you'd be better off with a simple micro-templating system, even something as simple as this:
def fill_in(template, data)
template.gsub(/\{\{(\w+)\}\}/) { data[$1.to_sym] }
end
#...
fill_in('Hi {{user_name}}, ....', :user_name => 'Pancakes')
You could use whatever delimiters you wanted of course, I went with {{...}} because I've been using Mustache.js and Handlebars.js lately. This naive implementation has issues (no in-template formatting options, no delimiter escaping, ...) but it might be enough. If your templates get more complicated then maybe String#% or ERB might work better.
one way I can think of doing this is to have templates stored for example:
"hi name"
then have a function in models that just replaces the template tags (name) with the passed arguments.
It can also be User who logged in.
Because this new function will be a part of model, you can use it like just another field of model from anywhere in rails, including the html.erb file.
Hope that helps, let me know if you need more description.
Adding another possible solution using Procs:
#String can be stored in the database
string = "->(user){ 'Hello ' + user.name}"
proc = eval(string)
proc.call(User.find(1)) #=> "Hello Bob"
gsub is very powerful in Ruby.
It takes a hash as a second argument so you can supply it with a whitelist of keys to replace like that:
template = <<~STR
Hello %{user_email}!
You have %{user_voices_count} votes!
Greetings from the system
STR
template.gsub(/%{.*?}/, {
"%{user_email}" => 'schmijos#example.com',
"%{user_voices_count}" => 5,
"%{release_distributable_total}" => 131,
"%{entitlement_value}" => 2,
})
Compared to ERB it's secure. And it doesn't complain about single % and unused or inexistent keys like string interpolation with %(sprintf) does.