Is it possible to load an Image from a remote URL using Images.jl, the same way you would load from a local file?
For reference:
using Images
yuss = load("mylocalimage.jpg") # Hooray
fail = load("https://www.wikipedia.org/portal/wikipedia.org/assets/img/Wikipedia-logo-v2#2x.png") # Sad
Trying to use the yuss success in the question to fix the fail problem:
using Images
img = mktemp() do fn,f
download("https://www.wikipedia.org/portal/wikipedia.org/assets/img/Wikipedia-logo-v2#2x.png", fn)
load(fn)
end
Or instead of using an external download tool, the HTTP package can be used (also this avoids the do syntax which might confuse):
using Images
using HTTP
t = tempname()
HTTP.download("https://www.wikipedia.org/portal/wikipedia.org/assets/img/Wikipedia-logo-v2#2x.png", t)
img = load(t)
rm(t)
Admittedly, using a temporary file is not the prettiest code. Is this enough for Hooray?
ADDITION
See other answer for an even shorter version thanks to Simon Danish.
#Simon Danisch sent me a pretty good solution offline:
julia> using FileIO
julia> download("https://www.wikipedia.org/portal/wikipedia.org/assets/img/Wikipedia-logo-v2#2x.png") |> load
Apparently download() defaults to creating a temporary file, and returns the full-name of it as a string. Basically the same as #Dan Getz's answer, but a bit simpler.
It's too bad we have to interact with the file system at all, but I suppose this will do for now.
Using HTTP.jl to stream the image without saving a temporary file on disk:
using HTTP, Images
url = "https://user-images.githubusercontent.com/6933510/107239146-dcc3fd00-6a28-11eb-8c7b-41aaf6618935.png"
philip = HTTP.get(url).body |> IOBuffer |> load
Related
class torch.FloatStorage[source]
byte()
Casts this storage to byte type
char()
Casts this storage to char type
Im trying to get some documentation done, i have managed to to get the format like the one shown above, But im not sure how to give that link of source code which is at the end of that function!
The link takes the person to the file which contains the code,But im not sure how to do it,
This is achieved thanks to one of the builtin sphinx extension.
The one you are looking for in spinx.ext.viewcode. To enable it, add the string 'sphinx.ext.viewcode' to the list extensions in your conf.py file.
In summary, you should see something like that in conf.py
extensions = [
# other extensions that you might already use
# ...
'sphinx.ext.viewcode',
]
I'd recommend looking at the linkcode extension too. Allows you to build a full HTTP link to the code on GitHub or such like. This is sometimes a better option that including the code within the documentation itself. (E.g. may have stronger permission on it than the docs themselves.)
You write a little helper function in your conf.py file, and it does the rest.
What I really like about linkcode is that it creates links for enums, enum values, and data elements, which I could not get to be linked with viewcode.
I extended the link building code to use #:~:text= to cause the linked-to page to scroll to the text. Not perfect, as it will only scroll to the first instance, which may not always be correct, but likely 80~90% of the time it will be.
from urllib.parse import quote
def linkcode_resolve(domain, info):
# print(f"domain={domain}, info={info}")
if domain != 'py':
return None
if not info['module']:
return None
filename = quote(info['module'].replace('.', '/'))
if not filename.startswith("tests"):
filename = "src/" + filename
if "fullname" in info:
anchor = info["fullname"]
anchor = "#:~:text=" + quote(anchor.split(".")[-1])
else:
anchor = ""
# github
result = "https://<github>/<user>/<repo>/blob/master/%s.py%s" % (filename, anchor)
# print(result)
return result
So I'm working on a crawler to get a bunch of images on a page that are saved as links. The relevant code, at the moment, is:
def parse_html(html)
html_doc = Nokogiri::HTML(html)
nodes = html_doc.xpath("//a[#href]")
nodes.inject([]) do |uris, node|
uris << node.attr('href').strip
end.uniq
end
I am current getting a bunch of links, most of which are images, but not all. I want to narrow down the links before downloading with a regex. So far, I haven't been able to come up with a Ruby-Friendly regex for the job. The best I have is:
^https?:\/\/(?:[a-z0-9\-]+\.)+[a-z]{2,6}(?:/[^\/?]+)+\.(?:jpg|gif|png)$.match(nodes)
Admittedly, I got that regex from someone else, and tried to edit it to work and I'm failing. One of the big problems I'm having is the original Regex I took had a few "#"'s in it, which I don't know if that is a character I can escape, or if Ruby is just going to stop reading at that point. Help much appreciated.
I would consider modifying your XPath to include your logic. For example, if you only wanted the a elements that contained an img you can use the following:
"//a[img][#href]"
Or even go further and extract just the URIs directly from the href values:
uris = html_doc.xpath("//a[img]/#href").map(&:value)
As some have said, you may not want to use Regex for this, but if you're determined to:
^http(s?):\/\/.*\.(jpeg|jpg|gif|png)
Is a pretty simple one that will grab anything beginning with http or https and ending with one of the file extensions listed. You should be able to figure out how to extend this one, Rubular.com is good for experimenting with these.
Regexp is a very powerful tool but - compared to simple string comparisons - they are pretty slow.
For your simple example, I would suggest using a simple condition like:
IMAGE_EXTS = %w[gif jpg png]
if IMAGE_EXTS.any? { |ext| uri.end_with?(ext) }
# ...
In the context of your question, you might want to change your method to:
IMAGE_EXTS = %w[gif jpg png]
def parse_html(html)
uris = []
Nokogiri::HTML(html).xpath("//a[#href]").each do |node|
uri = node.attr('href').strip
uris << uri if IMAGE_EXTS.any? { |ext| uri.end_with?(ext) }
end
uris.uniq
end
I have the following line in a long loop
page = Nokogiri::HTML(open(topic[:url].first)).xpath('//ul[#class = "pages"]//li').first
Sometimes my Ruby application crashes raising the "End of file reached " exception in this line.
How can I resolve this problem? Just a begin;raise;end block?
Is a script that performs a forum backup, so is important that doesn't skip any thread.
Thanks in advance.
In addition to #Phrogz's excellent advice (in particular about at_css with the simpler expression), I would pull the raw xml [content] separately:
page = if (content = open(topic[:url].first)).strip.length > 0
Nokogiri::HTML(content).xpath('//ul[#class = "pages"]//li').first
end
I would suggest that you should first to fix the underlying issue so that you do not get this error.
Does the same URL always cause the problem? (Output it in your log files.) If so, perhaps you need to URI encode the URL.
Is it random, and therefor likely related to a connection hiccup or server problem? If so, you should rescue the specific error and then retry one or more times to get the crucial data.
Secondarily, you should know that the CSS syntax for that query is far simpler:
page = Nokogiri.HTML(...).at_css('ul.pages li')
Not only is this less than half the bytes, it allows for cases like <ul class="foo pages"> that the XPath would miss.
Using at_css (or at_xpath) is the same as .css(...).first, but is faster and simpler.
I'm attempting to create a standalone application (independent of the Rails asset pipeline) using less.rb to output CSS files based upon Twitter Bootstrap.
The following results in an empty document
parser = Less::Parser.new :paths => [Rails.root + '/public/bootstraps/twitter-bootstrap-857b8fb/less']
tree = parser.parse("#import 'bootstrap.less'")
tree.to_css
Which results in an empty string being returned. I've tried variations of altering the #import to be the full path etc, with no success. I think I must be missing something simple.
I believe you have an issue with how you are specifying your path. As far as I can tell, Less is looking for an array of String objects, not Path obejcts.
Use the following:
parser = Less::Parser.new paths: [Rails.root.join('public', 'bootstraps', 'twitter-bootstrap-857b8fb', 'less').to_s]
tree = parser.parse("#import 'bootstrap.less'")
tree.to_css
You can simple run make as you have described here: https://github.com/twitter/bootstrap/wiki/Contributing-to-Bootstrap
If performance is not of utmost concern, you can always include less.js, which will compile the less files at runtime. Detailed instructions here.
Have you considered using the filename syntax from the less.rb github page? https://github.com/cowboyd/less.rb/
parser = Less::Parser.new :paths => ['./lib', 'other/lib'], :filename => 'mystyles.less'
I was hoping to be able to generate montages using PythonMagick. The documentation seems very sparse, but I've been trying to hunt it down using the code completion part of Eclipse at least, as well as a few other questions' suggestions here on Stack Overflow. It seems that the MagickWand API has the function I am looking for, according to this:
http://www.imagemagick.org/api/MagickWand/montage_8c.html
However, I cannot seem to find it in PythonMagick. Is this simply unavailable? If so I might just ditch the rest of my PythonMagick code and rely on subprocess.call on a portable ImageMagick distribution or something like that (this program will have to be portable, and run on Windows with an easy port to Mac OS... so far I have a few other PythonMagick commands working so I'd like to keep this route going if possible).
Thanks!
Using the python imagemagick/graphicsmagick bindings helps a lot, but unfortunately not all of the functionality is there yet. I actually had the same problem with #FizxMike. I needed to use montage and then do some further operations, but saving the file on hard disk and then reloading it in a proper pgmagick object in order to do the rest of the operations and saving it again was slow.
Eventually I used the subprocess solution, but instead of saving in a file, I redirect the output in stdout. Then, I use the stdout to load the image from a pgmagick.Blob in a pgmagick.Image object and do the rest of the processing in python code.
The procedure looks like this in code:
import os
import pgmagick
import subprocess
my_files = []
# Dir with the images that you want to operate on
dir_with_images = "."
for file in os.listdir(dir_with_images):
if file.endswith(".png"):
my_files.append(os.path.join(dir_with_images, file))
montage_cmd = ['gm', 'montage']
montage_cmd.extend(my_files)
# The trick is in the next line of code. Instead of saving in a file, e.g. myimage.png
# the montaged file will just be "printed" in the stdout with 'png:-'
montage_cmd.extend(['-tile', '2x2', '-background', 'none', '-geometry', '+0+0', 'png:-'])
# Use the command line 'gm montage' since there are not python bindings for it :(
p = subprocess.Popen(montage_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
# Get the stdout in a variable
stdout, stderr = p.communicate()
# Load the stdout in a python pgmagick Image object using the pgmagick.Blob
# and do the rest of the editing on python code
img = pgmagick.Image(pgmagick.Blob(stdout))
# Display the image
img.display()
geometry = pgmagick.Geometry(300, 200)
geometry.aspect(True)
# Resize the montaged image to 300x200, but keep the aspect ratio
img.scale(geometry)
# Display it again
img.display()
# And finally save it <- Only once disk access at this point.
img.write('myimage.png')
I have the same problem, even pgmagick lacks the montageImage() function needed (Magick++ montage example)
This is what I do (in a Django View):
#ImageMagick CLI is better documented anyway (-background none preserves transparency)
subprocess.call("montage -border 0 -geometry "+str(cols)+"x -tile 1x"+str(len(pages))+" "+target_path[0:len(target_path)-4]+"[0-9]*.png -background none "+target_path,shell=True)`
Not fun because I have to juggle around a bunch of files first... writing to hard disk is not the fastest thing to do, then delete the temp files.
I would much rather do it all in ram.
I am still in search of a better answer myself.