I want to replace 'hoge' to 'foo' with regex. But the user's value is dynamic so I can't use str.gsub('hoge', 'foo').
str = '?user=hoge&tab=fuga'
What should I do?
Don't do this with a regular expression.
This is how to manipulate URIs using the existing wheels:
require 'uri'
str = 'http://example.com?user=hoge&tab=fuga'
uri = URI.parse(str)
query = URI.decode_www_form(uri.query).to_h # => {"user"=>"hoge", "tab"=>"fuga"}
query['user'] = 'foo'
uri.query = URI.encode_www_form(query)
uri.to_s # => "http://example.com?user=foo&tab=fuga"
Alternately:
require 'addressable'
uri = Addressable::URI.parse('http://example.com?tab=fuga&user=hoge')
query = uri.query_values # => {"tab"=>"fuga", "user"=>"hoge"}
query['user'] = 'foo'
uri.query_values = query
uri.to_s # => "http://example.com?tab=fuga&user=foo"
Note that in the examples the order of the parameters changed, but the code handled the difference without problems.
The reason you want to use URI or Addressable is because parameters and values have to be correctly encoded when they contain illegal characters. URI and Addressable know the rules and will follow them, whereas naive code assumes it's OK to not bother with encoding, causing broken URIs.
URI is part of the Ruby Standard Library, and Addressable is more full-featured. Take your pick.
You can try below regex
([?&]user=)([^&]+)
DEMO
You probably want to find out what the user query maps to first before using a .gsub to replace whatever value it is.
First, parse the URL string into an URI object using the URI module. And then, you can use the CGI query methods to get the key value pairs of the query params off the URI object using the CGI module. And finally, you can .gsub off the values in that hash.
Related
I have a URL like below.
/shows/the-ruby-book/meta-programming/?play=5b35a825-d372-4375-b2f0-f641a38067db"
I need to extract only the id of the play (i.e. 5b35a825-d372-4375-b2f0-f641a38067db) using regular expression. How can I do it?
I would not use a regexp to parse a url. I would use Ruby's libraries to handle URLs:
require 'uri'
url = '/shows/the-ruby-book/meta-programming/?play=5b35a825-d372-4375-b2f0-f641a38067db'
uri = URI.parse(url)
params = URI::decode_www_form(uri.query).to_h
params['play']
# => 5b35a825-d372-4375-b2f0-f641a38067db
You can do:
str = '/shows/the-ruby-book/meta-programming/?play=5b35a825-d372-4375-b2f0-f641a38067db'
match = str.match(/.*\?play=([^&]+)/)
puts match[1]
=> "5b35a825-d372-4375-b2f0-f641a38067db"
The regex /.*\?play=([^&]+)/ will match everything up until ?play=, and then capture anything that is not a & (the query string parameter separator)
A match will create a MatchData object, represented here by match variable, and captures will be indices of the object, hence your matched data is available at match[1].
url = '/shows/the-ruby-book/meta-programming/?play=5b35a825-d372-4375-b2f0-f641a38067db'
url.split("play=")[1] #=> "5b35a825-d372-4375-b2f0-f641a38067db"
Ruby's built-in URI class has everything needed to correctly parse, split and decode URLs:
require 'uri'
uri = URI.parse('/shows/the-ruby-book/meta-programming/?play=5b35a825-d372-4375-b2f0-f641a38067db')
URI::decode_www_form(uri.query).to_h['play'] # => "5b35a825-d372-4375-b2f0-f641a38067db"
If you're using an older Ruby that doesn't support to_h, use:
Hash[URI::decode_www_form(uri.query)]['play'] # => "5b35a825-d372-4375-b2f0-f641a38067db"
You should use URI, rather than try to split/extract using a regexp, because the query of a URI will be encoded if any values are not within the characters allowed by the spec. URI, or Addressable::URI, will decode those back to their original values for you.
I have this code for send a request to an url, and I wanted to place on the url two variables :
talksList = open('http://yolo.com/?action=cp_list&id=#{variable1}&key=#{variable2}')
But when I insert my variables like this, it doesn't work. Can you help me ?
Thanks in advance.
As #YuHao said, you're trying to interpolate a variable into a non-interpreted string. But you have a bigger long-term problem.
Don't try to inject unencoded variables into a URL. While it will work, you run the risk of generating nonsensical URLs, which a browser would accept, but code won't. Instead, use the appropriate tools to modify the URL, which will maintain appropriate encoding for you.
Here's an example using URI:
require 'uri'
variable1 = 'foo'
variable2 = 'bar'
uri = URI.parse('http://yolo.com/?action=cp_list')
params = URI.decode_www_form(uri.query)
params << ['id', variable1]
params << ['key', variable2]
uri.query = URI.encode_www_form(params)
uri.to_s # => "http://yolo.com/?action=cp_list&id=foo&key=bar"
You can do the same thing using the Addressable gem, which is more full-featured:
require 'addressable/uri'
variable1 = 'foo'
variable2 = 'bar'
uri = Addressable::URI.parse('http://yolo.com/?action=cp_list')
params = uri.query_values
uri.query_values = params.merge('id' => variable1, 'key' => variable2)
uri.to_s # => "http://yolo.com/?action=cp_list&id=foo&key=bar"
That's because you are using strings with single quotes. In single quoted strings, nothing is replaced except \\ and \'.
Specifically, interpolation is only available in double quoted strings, try
talksList = open("http://yolo.com/?action=cp_list&id=#{variable1}&key=#{variable2}")
Say I have a string like this: "http://something.example.com/directory/"
What I want to do is to parse this string, and extract the "something" from the string.
The first step, is to obviously check to make sure that the string contains "http://" - otherwise, it should ignore the string.
But, how do I then just extract the "something" in that string? Assume that all the strings that this will be evaluating will have a similar structure (i.e. I am trying to extract the subdomain of the URL - if the string being examined is indeed a valid URL - where valid is starts with "http://").
Thanks.
P.S. I know how to check the first part, i.e. I can just simply split the string at the "http://" but that doesn't solve the full problem because that will produce "http://something.example.com/directory/". All I want is the "something", nothing else.
I'd do it this way:
require 'uri'
uri = URI.parse('http://something.example.com/directory/')
uri.host.split('.').first
=> "something"
URI is built into Ruby. It's not the most full-featured but it's plenty capable of doing this task for most URLs. If you have IRIs then look at Addressable::URI.
You could use URI like
uri = URI.parse("http://something.example.com/directory/")
puts uri.host
# "something.example.com"
and you could then just work on the host.
Or there is a gem domainatrix from Remove subdomain from string in ruby
require 'rubygems'
require 'domainatrix'
url = Domainatrix.parse("http://foo.bar.pauldix.co.uk/asdf.html?q=arg")
url.public_suffix # => "co.uk"
url.domain # => "pauldix"
url.subdomain # => "foo.bar"
url.path # => "/asdf.html?q=arg"
url.canonical # => "uk.co.pauldix.bar.foo/asdf.html?q=arg"
and you could just take the subdomain.
Well, you can use regular expressions.
Something like /http:\/\/([^\.]+)/, that is, the first group of non '.' letters after http.
Check out http://rubular.com/. You can test your regular expressions against a set of tests too, it's great for learning this tool.
with URI.parse you can get:
require "uri"
uri = URI.parse("http://localhost:3000")
uri.scheme # http
uri.host # localhost
uri.port # 3000
I have a bunch of URLs which I would like to clean. They all contain UTM parameters, which are not necessary, or rather harmful in this case. Example:
http://houseofbuttons.tumblr.com/post/22326009438?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+HouseOfButtons+%28House+of+Buttons%29
All potential parameters begin with utm_.
How can I remove them easily with a ruby script / structure without destroying other potentialy "good" URL parameters?
You can apply a regex to the urls to clean them up. Something like this should do the trick:
url = 'http://houseofbuttons.tumblr.com/post/22326009438?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+HouseOfButtons+%28House+of+Buttons%29&normal_param=1'
url.gsub(/&?utm_.+?(&|$)/, '') => "http://houseofbuttons.tumblr.com/post/22326009438?normal_param=1"
This uses the URI lib to deconstruct and change the querystring (no regex):
require 'uri'
str ='http://houseofbuttons.tumblr.com/post/22326009438?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+HouseOfButtons+%28House+of+Buttons%29&normal_param=1'
uri = URI.parse(str)
clean_key_vals = URI.decode_www_form(uri.query).reject{|k, _| k.start_with?('utm_')}
uri.query = URI.encode_www_form(clean_key_vals)
p uri.to_s #=> "http://houseofbuttons.tumblr.com/post/22326009438?normal_param=1"
How do I encode or 'escape' the URL before I use OpenURI to open(url)?
We're using OpenURI to open a remote url and return the xml:
getresult = open(url).read
The problem is the URL contains some user-input text that contains spaces and other characters, including "+", "&", "?", etc. potentially, so we need to safely escape the URL. I saw lots of examples when using Net::HTTP, but have not found any for OpenURI.
We also need to be able to un-escape a similar string we receive in a session variable, so we need the reciprocal function.
Don't use URI.escape as it has been deprecated in 1.9.
Rails' Active Support adds Hash#to_query:
{foo: 'asd asdf', bar: '"<#$dfs'}.to_query
# => "bar=%22%3C%23%24dfs&foo=asd+asdf"
Also, as you can see it tries to order query parameters always the same way, which is good for HTTP caching.
Ruby Standard Library to the rescue:
require 'uri'
user_text = URI.escape(user_text)
url = "http://example.com/#{user_text}"
result = open(url).read
See more at the docs for the URI::Escape module. It also has a method to do the inverse (unescape)
The main thing you have to consider is that you have to escape the keys and values separately before you compose the full URL.
All the methods which get the full URL and try to escape it afterwards are broken, because they cannot tell whether any & or = character was supposed to be a separator, or maybe a part of the value (or part of the key).
The CGI library seems to do a good job, except for the space character, which was traditionally encoded as +, and nowadays should be encoded as %20. But this is an easy fix.
Please, consider the following:
require 'cgi'
def encode_component(s)
# The space-encoding is a problem:
CGI.escape(s).gsub('+','%20')
end
def url_with_params(path, args = {})
return path if args.empty?
path + "?" + args.map do |k,v|
"#{encode_component(k.to_s)}=#{encode_component(v.to_s)}"
end.join("&")
end
def params_from_url(url)
path,query = url.split('?',2)
return [path,{}] unless query
q = query.split('&').inject({}) do |memo,p|
k,v = p.split('=',2)
memo[CGI.unescape(k)] = CGI.unescape(v)
memo
end
return [path, q]
end
u = url_with_params( "http://example.com",
"x[1]" => "& ?=/",
"2+2=4" => "true" )
# "http://example.com?x%5B1%5D=%26%20%3F%3D%2F&2%2B2%3D4=true"
params_from_url(u)
# ["http://example.com", {"x[1]"=>"& ?=/", "2+2=4"=>"true"}]
Ruby has the built-in URI library, and the Addressable gem, in particular Addressable::URI
I prefer Addressable::URI. It's very full featured and handles the encoding for you when you use the query_values= method.
I've seen some discussions about URI going through some growing pains so I tend to leave it alone for handling encoding/escaping until these things get sorted out:
http://osdir.com/ml/ruby-core/2010-06/msg00324.html
http://osdir.com/ml/lang-ruby-core/2009-06/msg00350.html
http://osdir.com/ml/ruby-core/2011-06/msg00748.html