This question already has answers here:
Closed 12 years ago.
Possible Duplicates:
Regex to match URL
regex to remove the webpage part of a url in ruby
I am in search of a regular expression for parsing all the urls in a file.
i tried many of the regular expression i got after googling but it fails in one or the other case . my idea is to write one which checks the presense of http or https at the begening and it will match everything untill it sees a blank space .
any ideas ?
NOTE : i dont need to parse the url but erase all the urls from a file or atleast make it unreadable .
The standard URI library provides URI.regexp which is the regular expression for url string.
require 'uri'
string.scan(URI.regexp)
http://ruby-doc.org/stdlib/libdoc/uri/rdoc/index.html
You can try this:
/https?:\/\/[\S]+/
The \S means any non-whitespace character.
(Rubular)
Related
This question already has answers here:
Regular expression to stop at first match
(9 answers)
Closed 2 years ago.
I have this gigantic ugly string:
J0000000: Transaction A0001401 started on 8/22/2008 9:49:29 AM
J0000010: Project name: E:\foo.pf
J0000011: Job name: MBiek Direct Mail Test
J0000020: Document 1 - Completed successfully
I'm trying to extract pieces from it using regex. In this case, I want to grab everything after Project Name up to the part where it says J0000011: (the 11 is going to be a different number every time).
Here's the regex I've been playing with:
Project name:\s+(.*)\s+J[0-9]{7}:
The problem is that it doesn't stop until it hits the J0000020: at the end.
How do I make the regex stop at the first occurrence of J[0-9]{7}?
Make .* non-greedy by adding '?' after it:
Project name:\s+(.*?)\s+J[0-9]{7}:
Using non-greedy quantifiers here is probably the best solution, also because it is more efficient than the greedy alternative: Greedy matches generally go as far as they can (here, until the end of the text!) and then trace back character after character to try and match the part coming afterwards.
However, consider using a negative character class instead:
Project name:\s+(\S*)\s+J[0-9]{7}:
\S means “everything except a whitespace and this is exactly what you want.
Well, ".*" is a greedy selector. You make it non-greedy by using ".*?" When using the latter construct, the regex engine will, at every step it matches text into the "." attempt to match whatever make come after the ".*?". This means that if for instance nothing comes after the ".*?", then it matches nothing.
Here's what I used. s contains your original string. This code is .NET specific, but most flavors of regex will have something similar.
string m = Regex.Match(s, #"Project name: (?<name>.*?) J\d+").Groups["name"].Value;
I would also recommend you experiment with regular expressions using "Expresso" - it's a utility a great (and free) utility for regex editing and testing.
One of its upsides is that its UI exposes a lot of regex functionality that people unexprienced with regex might not be familiar with, in a way that it would be easy for them to learn these new concepts.
For example, when building your regex using the UI, and choosing "*", you have the ability to check the checkbox "As few as possible" and see the resulting regex, as well as test its behavior, even if you were unfamiliar with non-greedy expressions before.
Available for download at their site:
http://www.ultrapico.com/Expresso.htm
Express download:
http://www.ultrapico.com/ExpressoDownload.htm
(Project name:\s+[A-Z]:(?:\\w+)+.[a-zA-Z]+\s+J[0-9]{7})(?=:)
This will work for you.
Adding (?:\\w+)+.[a-zA-Z]+ will be more restrictive instead of .*
This question already has answers here:
Capturing groups don't work as expected with Ruby scan method
(3 answers)
Closed 5 years ago.
This is a following up question regarding Lazy (ungreedy) matching multiple groups using regex. I try to use the method but not very successful.
I grab a string from gitlab API and try to extract all the repos. The name of repo follows the format of "https://gitlab.example.com/foo/xxx.git".
So far, if I try this, it works OK.
gitlab_str.scan(/\"https\:\/\/gitlab\.example\.com\/foo\//)
But to add name wildcard is tricky, I use the method from the previous question:
gitlab_str.scan(/\"https\:\/\/gitlab\.example\.com\/foo\/(.*?)\.git\"/)
It says to use (.*?) for lazy matching, but it doesn't seem to work.
Thanks a lot for the help.
If we have the following string:
gitlab_str = "\"https://gitlab.example.com/foo/xxx.git\""
The following RegEx will return [["xxx"]], which is expected:
gitlab_str.scan(/\"https\:\/\/gitlab\.example\.com\/foo\/(.*?)\.git\"/)
Because you had the (.*?). Note the parenthesis, so only what's inside the parenthesis will be returned.
If you want to return the whole string matched, you can just remove the parenthesis:
gitlab_str.scan(/\"https\:\/\/gitlab\.example\.com\/foo\/.*?\.git\"/)
This will return:
["\"https://gitlab.example.com/foo/xxx.git\""]
It also works for multiple occurrences:
> gitlab_str = "\"https://gitlab.example.com/foo/xxx.git\" and \"https://gitlab.example.com/foo/yyy.git\""
> gitlab_str.scan(/\"https\:\/\/gitlab\.example\.com\/foo\/.*?\.git\"/)
=> ["\"https://gitlab.example.com/foo/xxx.git\"", "\"https://gitlab.example.com/foo/yyy.git\""]
Finally, if you want to remove the https:// part from the resulting matches, then just wrap everything but that part with () in the RegEx:
gitlab_str.scan(/\"https\:\/\/(gitlab\.example\.com\/foo\/.*?\.git)\"/)
This question already has answers here:
How to extract URLs from text
(6 answers)
Closed 8 years ago.
I am tring to extract a link from a phrase and it could be any where last, first or middle so I am usig this regex
link=text.scan(/(^| )(http.*)($| )/)
but the problem is when the link is in the middle it gets the whole phrase until the end.
What should I do ?
It's because .* next to http is greedy. I suggest you to use lookarounds.
link=text.scan(/(?<!\S)(http\S+)(?!\S)/)
OR
link=text.scan(/(?<!\S)(http\S+)/)
Example:
> "http://bar.com foo http://bar.com bar http://bar.com".scan(/(?<!\S)http\S+(?!\S)/)
=> ["http://bar.com", "http://bar.com", "http://bar.com"]
DEMO
(?<!\S) Negative lookbehind which asserts that the match won't be preceeded by a non-space character.
http\S+ Matches the substring http plus the following one or more non-space characters.
Do all the links you are trying to match follow some simple pattern? We'd need to see more context to confidently provide a good solution to your problem.
For example, the regex:
link=text.scan(/http.*\.com/)
...might be good enough for the job (this assumes all links end in ".com"), but I can't say for sure without more information.
Or again, for example, perhaps you could use something like:
link=text.scan(/http[a-z./:]*) - this assumes all links contain only lower case letters, ".", "/" and ":".
This question already has answers here:
How to match all occurrences of a regular expression in Ruby
(6 answers)
Closed 8 years ago.
I've got the following regex:
regex = /\$([a-zA-Z.]+)/
and the following query
query = "Show me the PE Ratio for $AAPL, $TSLA"
Now regex.match(query) should capture AAPL and TSLA, but instead I get the following:
#<MatchData "$AAPL" 1:"AAPL">
which is completely wrong. Anyone know why?
Note that this regex works fine on Rubular: http://rubular.com/r/j0maQHnVFF
In Ruby the .match method will only return the first capture. You need it to return all captured matches, like the /g flag in PCRE
You can use the scan method. The scan method will either give you an array of all the matches or, if you pass it a block, pass each match to the block.
Code
query.scan(/\$([a-zA-Z.]+)/)
Fixed it, needed to use .scan instead of .match
In my Ruby app, I've used the following method and regular expression to remove all HTML tags from a string:
str.gsub(/<\/?[^>]*>/,"")
This regular expression did just about all I was expecting it to, except it caused all quotation marks to be transformed into “
and all single quotes to be changed to ”
.
What's the obvious thing I'm missing to convert the messy codes back into their proper characters?
Edit: The problem occurs with or without the Regular Expression, so it's clear my problem has nothing to do with it. My question now is how to deal with this formatting error and correct it. Thanks!
Use CGI::unescapeHTML after you perform your regular expression substitution:
CGI::unescapeHTML(str.gsub(/<\/?[^>]*>/,""))
See http://www.ruby-doc.org/core/classes/CGI.html#M000547
In the above code snippet, gsub removes all HTML tags. Then, unescapeHTML() reverts all HTML entities (such as <, “) to their actual characters (<, quotes, etc.)
With respect to another post on this page, note that you will never ever be passed HTML such as
<tag attribute="<value>">2 + 3 < 6</tag>
(which is invalid HTML); what you may receive is, instead:
<tag attribute="<value>">2 + 3 < 6</tag>
The call to gsub will transform the above to:
2 + 3 < 6
And unescapeHTML will finish the job:
2 + 3 < 6
You're going to run into more trouble when you see something like:
<doohickey name="<foobar>">
You'll want to apply something like:
gsub(/<[^<>]*>/, "")
...for as long as the pattern matches.
This regular expression did just about
all I was expecting it to, except it
caused all quotation marks to be
transformed into “ and all
single quotes to be changed to ”
.
This doesn't sound as if the RegExp would be doing this. Are you sure it's different before?
See this question here for information about the problem, it has got an excellent answer:
Get non UTF-8 form fields as UTF-8 in php.
I've run into a similar problem with character changes, this happened when my code ran through another module that enforced UTF-8 encoding and then when it came back, I had a different file (slurped array of lines) on my hands.
You could use a multi-pass system to get the results you are looking for.
After running your regular expression, run an expression to convert &8220; to quotes and another to convert &8221; to single quotes.