using raw SQL when I use the IN statement inside a query using sequel's fetch function, I can't escape a single quote by writing where stuff IN ...
#values='stuff1\'','stuff2\''
db.fetch("query...where IN (?)", "#{#values}")
outputs query...where stuff IN ('stuff1'',''stuff2') instead of ('stuff1','stuff2')
Quite frustrating that I'd probably have to write a Sequel equivalent for the raw query or use a different ORM just because of this escape issue. Any thoughts?
You should probably do something like:
#values = ['stuff1', 'stuff2']
db.fetch("query...where IN ?", #values)
If I understand the Sequel documentation correctly, using String#lit or Sequel.lit should turn a Ruby string into a literal string and bypass the automatic escaping mechanism; therefore, this should work (untested):
#values='stuff1\'','stuff2\''.lit
db.fetch("query...where IN (?)", "#{#values}")
The usual caveats when working with raw SQL strings (SQL injection attacks, inefficient SQL due to forced re-parsing of statements etc.) apply :-)
This works:
values = Sequel.lit("'stuff1', 'stuff2'")
db.fetch("SELECT * FROM TABLE1 WHERE COL1 IN ?", values)
See also: How to pass a list to an IN clause via a placeholder with Ruby Sequel
Related
Easy example:
Zlib::Inflate.inflate(PG::Connection.unescape_bytea(PG::Connection.escape_bytea(Zlib::Deflate.deflate('["128,491,128,487"]'))))
Zlib::DataError: incorrect data check
This is not issue with zlib because following succeeds:
Zlib::Inflate.inflate(Zlib::Deflate.deflate('["128,491,128,487"]'))
=> "[\"128,491,128,487\"]"
Parsing fails/succeeds dependant on provided string:
Zlib::Inflate.inflate(PG::Connection.unescape_bytea(PG::Connection.escape_bytea(Zlib::Deflate.deflate('["128,491,128,487", "128,491,128,490", "38,465,40,463"]'))))
=> "[\"128,491,128,487\", \"128,491,128,490\", \"38,465,40,463\"]"
Am i doing something wrong or is postgres bytea field escaping in ruby broken? What can i do as alternative?
Tried on: Ruby 2.2.3p173, gem pg-0.18.4; Ruby 2.4.1p111, gem pg-0.21.0
Answer from https://bitbucket.org/ged/ruby-pg/issues/262/postgres-bytearray-escaping :
The description of (un)escape_bytea is misleading. It's not the exact opposite of each other.
PG::Connection.escape_bytea uses the old and deprecated escaping mechanism, which must be double escaped - first as a BYTEA and secondary as a string literal (per adding ' before and after the string) for insertion into the SQL string. This is done in one step for convenience and performance reasons.
On contrary PG::Connection.unescape_bytea does only BYTEA unescaping, because it is meant to decode column data retrieved by a query. This data is not escaped as a string literal.
You now have two options to make the above work:
enco = PG::TextEncoder::Bytea.new
deco = PG::TextDecoder::Bytea.new
Zlib::Inflate.inflate(deco.decode(enco.encode(Zlib::Deflate.deflate('["128,491,128,487"]'))))
This makes use of the type encoders of the pg gem instead of libpq. The encoder uses the newer BYTEA escaping machanism and is a bit faster than libpq's functions. This is still independent of a server connection.
conn = PG.connect
Zlib::Inflate.inflate(conn.unescape_bytea(conn.escape_bytea(Zlib::Deflate.deflate('["128,491,128,487"]'))))
This makes use of the connection bound escaping functions of libpq. They also use the newer escaping mechanism, so that no double escaping occurs.
I am writing a script that exports results into a SQLite database. I can not get the code to work when I use variables.
Here is an excerpt:
require 'sqlite3'
require 'shodan'
table_name = "#{Date.today.strftime("%B")}#{Time.now.year}_Findings"
db = SQLite3::Database.new "shodan_test.db"
db.execute ("CREATE TABLE #{table_name} (ip string , ports string)")
results = api.host(target)
ip = results["ip_str"].to_s
ports = results["ports"].to_s
db.execute ("insert into #{table_name} (ip, ports) values (#{ip}, #{ports})")
The code fails at the last line. I can remove the variables and the code works. I'm a bit confused since the CREATE TABLE works with a variable.
Your problem is that you're not quoting your strings. The result is that SQLite sees things like
insert into ... values (192.168.0.6, whatever is in ports)
and that's not valid SQL.
SQLite3::Database#execute understands placeholders so you should use them:
db.execute("insert into #{table_name} (ip, ports) values (?, ?)", ip, ports)
The placeholders (?) in the query will be replaced with properly quoted and escaped versions of their values. Using placeholders avoids all the usual quoting and injection problems that plague using string interpolation for SQL.
Also note that I've removed the space between execute and (. If you put a space there then Ruby thinks your "method calling parentheses" are actually "expression grouping parentheses" and it will complain about not understanding what those commas are doing. If you're only passing one argument (such as in db.execute ("CREATE ...")) then it won't matter because the parentheses are effectively ignored but it does matter when there are multiple arguments. In any case, o.m (arg) is a bad habit to get into, you should say o.m(arg) or leave the parentheses out.
I want SQLAlchemy to generate the following SQL code:
SELECT t171 AS "3Harm" FROM production
I've been playing around with something similar to this SQLAlchemy ORM snippet:
session.query(Production.t171.label('3harm'))
The problem here is that this doesn't properly quote "3harm" in the generated SQL. Instead of "3harm" this generates the unquoted 3harm, which is invalid because it starts with a numerical character and therefore raises the following Oracle exception:
ORA-00923: FROM keyword not found where expected
I can get this to work by capitalizing any character in the alias name:
session.query(Production.t171.label('3Harm'))
But I would still prefer to use all lowercase column names since the rest of my program is standardized for all lowercase. Any idea how to force quote the lowercase version?
Found the solution while looking for something else.
Any column can be forced to use quotes with column.quote = True.
So for the original example:
column = Production.t171.label('3harm')
column.quote = True
session.query(column)
Success!
The SQL you want to generate isn't valid; rather than this:
SELECT t171 AS '3Harm' FROM production
... you need the identifier to be enclosed in double quotes, not single quotes:
SELECT t171 AS "3Harm" FROM production
So it looks like you should be able to do this:
session.query(Production.t171.label('"3harm"'))
or maybe:
session.query(Production.t171.label("3harm"))
But I don't use SQLAlchemy and I don't have any way to check if either is valid; you might need to escape the double quotes in the first one, for instance, though from this perhaps the second is more likely to work... and I don't understand why 3Harm would work unquoted.
I want to execute my INSERT strings within a Ruby file that has the following array:
names_arr = ["Jack", "Susan", "Peter"]
In the same Ruby file, I've required the SQLite3 gem and have this code:
names_arr.each do |each_name|
db.execute('INSERT INTO users (name) VALUES ("#{each_name}");')
end
I am expecting this in my "users" table:
id name
1 Jack
2 Susan
3 Peter
Instead, I'm getting this:
id name
1 #{each_name}
2 #{each_name}
3 #{each_name}
It's clear I'm not interpolating properly, but I've tried a lot of different things and can't figure out where I'm wrong.
I am using SQLite3 and Ruby.
You have several problems in this code:
names_arr.each do |each_name|
db.execute('INSERT INTO users (name) VALUES ("#{each_name}");')
end
You're trying to use Ruby's string interpolation inside a single quoted string but string interpolation only works in double quoted strings (and heredocs, %Q{...} strings, regex literals, ...).
Standard SQL uses single quotes for string literals, not double quotes. Some databases let you get away with double quotes but it is a really bad habit to get into.
You shouldn't use string interpolation to build SQL anyway, you're not hacking PHP in 1999 so you should use placeholders and let the database library deal with the quoting.
The SQLite driver's execute understands ? placeholders so you can say this:
names_arr.each do |each_name|
db.execute('INSERT INTO users (name) VALUES (?)', each_name)
end
so I'm making a little ruby script to parse a .csv and output a large MySQL query. The problem is some of the names I'm working with have single quotes, which messes up my SQL...
My attempt to gsub the quotes into escape quotes has failed, though I'm not sure why. But sure enough when I open my query.txt file a name like D'vinci would be just that, and not D\'vinci.
What am I doing wrong?
require 'csv'
filename = ARGV[0]
q = "INSERT INTO `table` (`username`, `password`, `firstname`, `lastname`, `email`) VALUES "
CSV.foreach(filename) do |row|
last_name, first_name, email, id = row.each {|c| c.gsub "'", "\'"}
q += "('#{id}', SHA1('password'), '#{first_name}', '#{last_name}', '#{email}'),\n"
end
q += ";"
File.open("query.txt", 'w').write(q)
Do not write your own SQL escaping. Please, always use the methods provided by the appropriate database driver.
If you're using MySQL, either mysql or the newer mysql2 gem, there's an escape function that should handle this for you.
It's not entirely clear why you're writing SQL in the first place when most databases have some kind of import-from-file function. MySQL in particular has LOAD DATA INFILE which can read in many different formats if used correctly.
each returns the receiver. So, your gsub inside that block is doing nothing. Change it to map, and it shall be fine. Or, you can keep the each and use gsub! instead.