I am trying to write a crawler in a bash to download images from bing. Everything works fine except the fact that bing loads only 30 images in the beginning (uses infinite scrolling) and hence I am not able to download more than 30 images using wget.
Any workaround to download more than 30 images from bing images at once.?
--
Thanks for any help. :)
Bing has an API, perhaps that would work better than screen scraping? It would certainly be more allowed/supported by Bing than screen scraping, and less prone to breaking changes as they change their offerings over time.
If you're really set on the screen scraping for whatever reason, you might try loading the page with debugging (such as with Firebug) and watch what happens when the infinite scroll is triggered. Then try to manually trigger it with code.
Related
I use the Facebook Open Graph to publish URLs from a website to their corresponding Facebook Page. The main image is specified using the Open Graph tag og:image. Sometimes the images show up on Facebook as a very zoomed-in, and granular/pixelated. Upon further examination, the images which do poorly appear to have been rotated (I noticed this because a Lightbox plugin I use isn't correctly rotating them, but all other software: browser, Mac Preview, Facebook are).
An example page from the site with the problem (click to zoom and see the orientation issue in that 3rd party library but that's another matter). This issue I'm trying to resolve on Facebook can be seen by using the Facebook Debugger against this page. (As a side-note, it's oriented correctly).
At first I thought I could perhaps fix this with og:image:width and og:image:height, but it had no effect (it did fix a different the pre-caching/crawling issue).
Note that providing the raw image URL directly into the Debugger works fine; the problem is only when providing this URL via the Open Graph tags.
My one workaround idea is to use my image processing library (sorl-thumbnail) to produce a version of the image which FB might like better.
What is the specific characteristic of these rotated images causing a problem and how do I work around it?
Turns out this is a legitimate Facebook bug. Kudos to them for responding and addressing it promptly.
https://developers.facebook.com/bugs/1080037355413437/
For some reason when I use Chrome to test out my website, no images load at all, not even tiny ones like loading icons or the simple "back to top of page" arrow icon at the bottom. The browser tab just hangs on the spinning circle loading state. If I refresh it like 10 times or something then a portion of the images might load. Interestingly my Nivo-Slider images never load... there is a large empty space at where the Nivo-Slider gallery should be at the top left of my page no matter how many times I refresh.
The console is completely clean of errors if you check.
Can anyone check for me why my website is not working in Chrome? It works perfectly in Firefox/IE. I admit some images might have large file sizes but it shouldn't cause Chrome to hang for like half an hour?
I'm using latest Chrome/IE/Firefox. Windows 7 64-bit.
My website: www.symphonyofpromise.com/inspiration/en
By the way my website is a completely fictional museum/gallery project, it's not the official site for some museum!
I cannot be 100% certain on this, but judging from the Network panel, you might be loading too many MP3 files at the same time, thus maxing out the number of simultaneous requests. Chrome might use a different prioritization algorithm.
I would recommend removing some of those MP3 files and deferring loading until after the initial load.
EDIT:
Nevermind that, it appears that the MP3s load fine, but other resources are definitely blocking it. A request should never be pending this long. Poke around further in the Network panel and you'll find the problem.
I have created a gif in photoshop. It has two frames displayed for 4 seconds each and is on a loop forever.
For some reason the image freezes on the second frame and does not continue to load in some browsers (The exact information about browsers and / or versions is unclear at the moment but it has happened in the latest version of chrome).
The image is embedded in the code in the standard way, no javascript is used and nothing is changing the element.
Does anyone have any idea what could be causing this? Everything I have searched for so far seems to result in people saying "Oh, I forgot to put make the gif loop forever" instead of any real reasons for causing this problem.
Thank you for your help!
Edit: Sorry I didn't think about putting an image in the original post. He is an example of one advert which doesn't seem to work.
Because the current image displays correctly in everything I've tryed so far, try to create image in different application.
Some applications you may try:
GIMP - tutorial
Some freeware crap.
I'd vote for GIMP. Its complicated, but its free and it has wide user suport.
If you still fail...
Now its time to check the enviroment. First, open the image on different machine. Maybe there is something wrong with your browser instalation. In this case, reinstall.
Try to upload on different server. Maybe your browser does not correctly decompress deflated data or has other communication issues.
I have a web app I'm building in Dashcode, currently every thing is working as advertised. I'm using the browser template and was able to modify it a bit, got the xml working to connect images and their description. Now I want to add a simple OnClick event that sends the image to fullscreen but I can't find any documentation for that. I'm pretty sure this is easily done so I'm missing something here? Any help greatly appreciated.
Tom
Taking over the screen is a bit complicated (if your talking about a full screen experience like youtube), but full screen in the browser window is defiantly possible. The simplest method is to have the click event open a link to the image location. This would load the image in full resolution in the browser window. A more elegant approach would be to load the image url into the source of a image dashcode object. and have that objects visibility only appear overs the whole web app when the Oncick event occurs. If you are talking about a built in function in dashcode to achieve a full screen/browser image, your out of luck.
Anyone know of a Flash file (image) uploader that will force a user to resize and/or crop their image BEFORE uploading it? To then upload it as well.
Basically, I don't want my server processing the image resize/crop. I want to specify a target aspect ratio and have the user resize and crop their image to make it fit.
I've seen cropping uploaders before but they all seem to be server side. I saw a Flex one but I'm not sure it's "mandatory" -- Basically if the user just uploads the image without making edits, then I'd like the Flash to scale and fit the image into set dimensions...Leaving it I guess short in one direction to not stretch.
Anything like this out there?
Thanks!
I have a bounty running with a very similar question, be sure to take a peek - there isn't anything there yet that does client side resizing, though.
Also, SWFUpload is said to support it in the new Beta, but the feature is very sparsely documented right now. You would probably have to work on it to get it running the way you want.
http://www.adylevy.com/index.php/2009/07/22/multiple-files-uploader-with-preview-on-client-side/
there was another one called resize before upload but the site went down. though while I'm updating this and we're on the subject -- I don't see why you'd want to use Flash these days for this task. There are plenty of JavaScript options and now we also have Google's Dart (which builds JavaScript).