I am able to upload the test data from the Quickstart example but when I try and upload data from my own google cloud storage I get
"Error: Cannot find the referenced file: in request."
I tried taking the data from the quick start that I know works and putting it on my own google cloud storage and uploading it from there and I get the same thing.
I am able to see all of the files when I browse for objects and I am able to see it all just fine. I even tried making the files public to all users thinking maybe it was a permissions thing but it doesn't appear to be that either.
Any advice?
Related
The project I am currently working on needs a search engine to search a couple of 10.000 pdf files. When the user searches via the website for a certain keyword, the search engine will return a snippet of the pdf files matching his search criteria. The user then has the option to click on a button to view the entire pdf file.
I figured that the best way to do this was using elasticsearch + fscrawler (https://fscrawler.readthedocs.io/en/fscrawler-2.7/). Running some tests today and was able to crawl to a folder on my local machine.
For serving the PDF files (via a website), I figured I could store the PDF files in a google cloud storage and then use the link of the google cloud storage to let the users view the pdf files. However, FS Crawler does not seem to be able to access the bucket. Any tips or ideas on how to solve this. Feel free to criticize the work method described above. If there are better ways to make the users of the website access the PDF files, I would love to hear it.
Thanks in advance and kind regards!
You can use s3fs-fuse to mount s3 bucket into your file system and then use normal Local FS crawler.
Alternatively, you can fork fscrawler and implement a crawler for s3 similar to crawler-ftp.
I am trying to implement Dropbox on my website and so far I've been able to upload, fetch file metadata, user details and also download the file on my local machine(using the Dropbox API v2).
But, I would like to import the file directly from Dropbox and upload it to the server to be processed further....I'm able to generate the link for the chosen file using the "Chooser"
Dropbox API explorer lists all the possible APIs dropbox can provide.
To build the website I'm using laravel 5.6.17
Your help would be much appreciated. Thanks in advance
I am creating an android application that is connected to the parse back end at the moment. My question is how do i upload data like text, images,and video from my computer directly to parse and then be able to call it from my app. ive been doing some research and i seen things like rest api and accessing it through Http. i just want to see whats best for my case.
so basically my question is how do i upload data to parse from my computer.
im kind of new to this and any help would be appreciated
Thanks
if you just need to upload a few files then you can do it from the Parse dashboard. For a large number of files write a script that call the Parse REST API
In my Ruby on Rails app, I use the imdb gem (https://rubygems.org/gems/imdb) to search for a movie by title and grab the poster url and add it to the movie model I have in my database. Then in my view, I put that url in an image source tag and display the image to the user.
I don't have any problems when I'm running my application locally, but when I deploy it to Heroku, sometimes a few images are rendered successfully but for the most part, they aren't displayed properly. I've tried multiple browsers and as it turns out when I try to load the image, I get a "Referral Denied" message saying:
You don't have permission to access "[poster url here]" on this server. Reference #[some ref. number here]
How would I go about fixing this? I'm guessing it's because the IMDB server is denying my access because either I'm making too many requests from my application or because my application doesn't have the necessary credentials to get the data or maybe some combination of both. Is there a way to bypass this at all?
IMDB blocks the direct linking of images from their site on other sites, I think this previous question covers the topic.
The easiest way to get around this is to download the image and host it yourself rather than linking IMDB's copy. Alternatively you could investigate alternative movie DBs to see if they can offer what you want - the answers to this question on IMDB APIs lists a few. The Movie DB API looks like a good bet.
I have a model which *I want* to contain an image blob. I have the images on my local filesystem, but due to the nature of my application, I need to get them in the datastore. Here's my model:
class JeanImage(db.Model):
type = db.StringProperty(required=True, choices=set(["main","front","back","detail"]))
image = db.BlobProperty(required=True)
I haven't tried anything yet because I'm not great when dealing with images.
How can/should I convert my images to blobs so that I can get them in my bulkupload csv file?
Mark
You can do it, just not with the bulk uploader. You need to access the remote api directly.
This site has a basic example of how to use it:
http://www.billkatz.com/2009/2/Remote-API-Hello-World
Its pretty slow and a good idea to have a retry mechanism.
A more detailed description can be found here:
http://code.google.com/appengine/articles/remote_api.html
I believe that what you are trying to achieve is not possible using the app engine bulkloader.
Instead try to create some kind of uploader yourself. For example you could upload the images as a zip file and then extract it an store it in the datastore. The code for that should be fairly straightforward if you can map your images to the datastore entity (e.g. by using a naming convention).