I am following this tutorial an have a lot of trouble with it as many parts are not explained to beginners: https://cloud.google.com/developers/articles/how-to-build-mobile-app-with-app-engine-backend-tutorial
I came to the part where we need to upload some data to datastore. They are using a .csv file to accomplish that.
This is part of the tutorial where I've stuck:
Create Upload Script
This is the script that actually uploads the simulated information into the backend datastore.
For information about the bulkloader.yaml and more, refer to the App Engine documentation in: Uploading and Downloading Data. Also, you can find the bulkloader.yaml file and the test data here MobileAssistant-Data.
Create a new directory and make it your current directory.
In your editor, create a script file and call it upload_data.sh.
Copy and paste the following code into the file.
#!/bin/sh
appcfg.py upload_data
--config_file bulkloader.yaml --url=http://localhost:8888/remote_api --filename $1 --kind=$2 -e
nobody#nowhere.com
The previous script accepts two arguments separated by a blank space:
./upload_data.sh<data.csv> <class entity name>
The first argument is the name of the csv file that contains the data you want to upload, The second is the name of the entity class to handle the data.
Close and save the file. Notice that the script at this time is intended to be used on local server. It must be modified when the backend application is deployed to the cloud. For example, e-mail “nobody#nowhere.com ” will be changed to an actual e-mail address of the App Engine administrator before deployment.
Places Simulated Data
The simulated data is contained in the file places.csv that you downloaded earlier from the MobileAssistant-Data directory in Mobile Shopping Assistant.
The file contains the following comma separated fields:
name: <place’s name>
placeId: <number that identifies the place>
location: <place’s latitude and longitude coordinates (two values separated by a comma) >
key: <unique key associated with the place>
address: <the place’s address >
Modify Web.xml
To upload data to the datastore, reference the “remote API servlet” and associate it with a URL. You’ll define these mappings in the Web.xml file of the MobileAssistant-Appengine application, as shown next. The web server uses this configuration to identify the servlet to handle a given request.
Note. Notice, you might need to restart Eclipse. if you skip th e Web.xml modification you will get Error 404 Not Found.
<servlet>
<display-name>Remote API Servlet</display-name>
<servlet-name>RemoteApiServlet</servlet-name>
<servlet-class>com.google.apphosting.utils.remoteapi.RemoteApiServlet</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
<servlet-name>RemoteApiServlet</servlet-name>
<url-pattern>/remote_api</url-pattern>
</servlet-mapping>
Upload Places Simulated Data
To upload the test data to the data store, run the script that uploads the data from your script directory.
Before you upload your data, make sure to start MobileAssistant-AppEngine as a Web Application (in Debug mode) in Eclipse.
./upload_data.sh <places.csv> Place
At the password request, just press enter.
I don't understand that part with the script. I created a folder age on Desktop, placed inside their files bulkloader.yaml and places.csv. Inside that folder is that upload script.
After I run ./upload_data.sh <places.csv> Place, in my terminal I receive the following error:
upload_data.sh: line 2: appcfg.py: command not found
upload_data.sh: line 3: --config_file: command not found
upload_data.sh: line 4: nobody#nowhere.com: command not found
What should I do to upload that data to Google App Engine datastore? Thank you.
Is appcfg.py in your path, if not then set the path explicitly in your script or add the path the appcfg.py to your PATH
Related
We have usecase where we need to call an API that uploads its respective category of unique file.
For every API call we need to use a unique FileName. I mean File once used in an API call should not be used again.
For Example
CarAPI will be called by uploading a file-name from list of files (CarAP_1.xml to CarAP_1000.xml files)
File CarAP_1.xml once used in an API call should not be used again in next call
BikeAPI will be called by uploading a file-name from list of files (BikeAP_1.xml to BikeAP_1000.xml files)
File once used in an API call should not be used again.
Any thoughts or inputs on how we can achieve this using JMeter.
You can put these filenames
either to a CSV file and use HTTP Simple Table Server, its READ endpoint has KEEP=false mode so once the data is used it will be removed from memory hence you can avoid duplicate requests
or to Redis and use Redis Data Set Config which also provides possibility to remove the data from the list once it has been utilized
Both plugins can be installed using JMeter Plugins Manager
Suppose a total of 50 users and one HTTP request/sampler were created so in that case we provide one input file when run Api then all 50 users use the same input files (same input file location )and some requests failed.
We want to make ideas or scripts for users to use different file locations so not chance to fail request or file not found exception.
You can parameterize the file name/location just like any other request parameter, username, etc.
The configuration element which is most frequently user for parameterization is CSV Data Set Config.
However if you don't want to manually manage the file content and protect yourself from "file not founds" it makes sense to consider going for Directory Listing Config plugin which reads the files in the given folder and stores them into a JMeter Variable so each thread/iteration will pick the next file from the pool.
You can install Directory Listing Config plugin using JMeter Plugins Manager
I have developed a small web application that runs a web server in golang.
Each user can login, view the list of their docs (previously uploaded) and click on an item to view an html page that shows some fields of the document plus an tag with a src attribute
The src attribute includes an url like "mydocuments/download/123-456-789.pdf"
On the server side I handle the URL ("mydocuments/download/*") via an http Handler
mymux.HandleFunc(pat.Get("/mydocuments/download/:docname"), DocDownloadHandler)
where:
I check that the user has the rights to view the document in the url
Then I create a fileserver that obviously re-maps the url to the real path of the folder where the files are stored on the filesystem of the server
fileServer := http.StripPrefix("/mydocs/download/",http.FileServer(http.Dir("/the-real-path-to-documents-folder/user-specific-folder/)))
and of course I serve the files
fileServer.ServeHTTP(w, r)
IMPORTANT: the directory where the documents are stored is not the static-files directory I sue for the website but a directory where all files end after being uploaded by users.
My QUESTION
As I am trying to convert the code for it to work also on Google Cloud, I am trying to change the code so that files are stored in a bucket (or, better in "sub-directories" -as they do not properly exist- of a bucket).
How can I modify the code so to map the real document url as available via the cloud storage bucket?
Can I still use the http.FileServer technique above (if so what should I use instead of http.Dir to map the bucket "sub-folder" path where the documents are stored)?
I hope I was enough clear to explain my issue...
I apologise in advance for any unclear point...
Some options are:
Give the user direct access to the resource using a signed URL.
Write code to proxy the request to GCS.
Use http.FS with an fs.FS backed by GCS.
It's possible that a fs.FS for GCS already exists, but you may need to write one.
You can use http.FileSystem since it is an interface and can be implemented however you like.
I am currently writing a Go web app using Revel.
My app needs to read the content of an XML file which is stored on the server. At the moment, I store this file in the "public" folder where some other resources (css, js...) lie.
I am using ioutil.ReadFile to read the content of this file. While this is working when the server is run from the main app folder itself, I cannot figure how to access the file when the server is run from another location (say by running "revel run myapp" from $GOPATH).
Is there any way to deal with this situation in revel?
is there a generic way to know the path of the "public" folder?
Any hint would be appreciated.
Thanks! :)
The base path of the application is stored and accessible through revel.BasePath.
The "public" folder can thus be accessed through revel.BasePath + "/public/<...>".
This BasePath value is used, for example, in Static.Serve.
so I am working on a school project in which we have designed a web application that takes in much user info and creates a pdf then should display that pdf to the user so they can print it off or save it. We are using Play! Framework 2.1.3 as our framework and server and Java for the server side. I create the pdf with Apache's PDFbox library. Every thing works as it should in development mode ie launching it on a localhost with plays run command. the issue is when we put it up to the server and launch with plays start command I it seems to take a snapshot of the directory (or at least the assets/public folder) which is where I am housing the output.pdf file/s (i have attempted to move the file elsewhere but that still seems to result in a 404 error). Initially I believed this to be something with liunx machine we were deploying to which was creating a caching problem and have tried many of the tricks to defeat the browser from caching the pdf
like using javascript to append on a time stamp to the filename,
using this cache-control directive in the play! documentation,
"assets.cache./public/stylesheets/output.pdf"="max-age=0",
then I tried to just save the pdf as a different filename each time and pass back the name of that file and call it directly through the file structure in the HTML
which also works fine with the run command but not the start.
finally I came to the conclusion that when the start command is issued it balls up the files so only the files that are there when the start command is issued can be seen.
I read the documentation here
http://www.playframework.com/documentation/2.1.x/Production
which then I noticed this part
When you run the start command, Play forks a new JVM and runs the
default Netty HTTP server. The standard output stream is redirected to
the Play console, so you can monitor its status.
so it looks like the fact that it forks a new JVM is what is causing my pain.
so my question really is can this be gotten around in some way that a web app can create and display a pdf form? (if I cannot get this to work my only solution
that I can see is that I will have to simulate the form with HTML and fill it out from there) --which I really think is a bad way to do this.
this seems like something that should have a solution but I cannot seem to find or come up with one please help.
i have looked here:
http://www.playframework.com/documentation/2.1.x/JavaStream
the answer may be in there but Im not getting it to work I am pretty novice with this Play! Framework still
You are trying to deliver the generated PDF file to the user by placing it in the assets directory, and putting a link to it in the HTML. This works in development mode because Play finds the assets in the directory. It won't work in production because the project is wrapped up into a jar file when you do play dist, and the contents of the jar file can't be modified by the Play application. (In dev mode, Play has a classpath entry for the directory. In production, the classpath points to the jar file).
You are on the right lines with JavaStream. The way forward is:
Generate the PDF somewhere in your local filesystem (I recommend the temp directory).
Write a new Action in your Application object that opens the file you generated, and serves it instead of a web page.
Check out the Play docs for serving files. This approach also has the advantage that you can specify the filename that the user sees. There is an overloaded function Controller.ok(File file, String filename) for doing this. (When you generate the file, you should give it a unique name, otherwise each request will overwrite the file from a previous request. But you don't want the user to see the unique name).