I have a remote file on my server, It's writable. I know how to read and edit local files, but I have a problem with remote files. For example: www.mysite.com/myfile.txt
There is no standard way of using HTTP to edit a file, so you will have to:
Write a PHP script that accepts HTTP requests with data in them and writes the data to the file. You will probably want some form of security and you will probably need to mark the file as writable by all (chmod a+wfilename). You will probably want it to be a POST request so the file length is not limited by the max length of a URI.
Make a little HTML form to test the script.
Write a ruby script that sends that HTTP request. The built-in Net::HTTP library will probably work for you.
Related
I am sending a HTTP request to upload a file. And the request is setup like this:
uploadFile
And, the Directory Listing plugin pointing to a directory with all files and the request picks one file at a time. It works fine when run with one thread but, when i run in multiple threads, i see that already uploaded file is picked again to upload which leads to error.
I have added regular expression extrator to get the filename from the request body like this:
extract-filename-from-requestbody
And then, I am trying to use a post processor beanshell script to either delete the file from the folder or move to a different folder. But, not been successful. Need some help on this.
The first issue is i am not sure if i am extracting the value the right way. The value is to be got from request body and not request header. But, i dont see that option in the extractor.
Second, i am unable to use/retrieve the value from the extractor. Tried vars.get, vars.getObject and simply "${fileName}". Nothing works.
I don't think that deleting the file will help because Directory Listing Config reads the folder at the beginning of the test (see Execution Order chapter) so no matter whether the file is physically present or not JMeter will try to upload it
If you want to get unique files without repetitions just untick "Rewind on end of list" box:
This way each virtual user will read the next value so there will be no duplicates. When the last file will be used - the test will stop.
More information: Introducing the Directory Listing Config Plugin on JMeter
Also going forward consider using JSR223 Test Elements and Groovy language instead of Beanshell, it's the recommended option since JMeter 3.1
Bit of Background, I am using Jmeter to search for a download url from a rest API with an Oauth authorisation token set from the rest API. Once I have this URL I am doing a HTTP request, GET with redirect automatically, keepAlive and browser-compatible headers all checked.
Hanging of this I have then attached a "Save response to a file" with the file name prefix set to "blob" (this will be a filename set by a parameter later) and Add timestamp to the file name checked.
The url in question points to a zip file that needs OAuth header token (which is set successfully) the whole test plan succeeds.
This is great and you can see looking at the results
You can see that there are 67821343 bytes downloaded by the HTTP Request this is what we are expecting to see as this is the size of the file around 67Mb
This is were it starts to go wrong however as the save file only has 10,240 KB is an OCTET-Stream File and renaming this file to a zip just does not work as it is not a complete zip file.
This is my issue the Save file is not saving all the information and only save 10,240KB, every single time.
What am i doing wrong?
Is there a better way to get this zip file?
Please help it is driving me mad.
As once I have downloaded it I then need to assess the contents of the zip file to prove that the download URL that we are being directed to contains the correct ZIP.
here's the RUB if i do it manually through a browser i get the file downloaded successfully and it is exactly the same size (67821343 Bytes) but it just doesn't save in Jmeter.
Add the following property to user.properties file:
httpsampler.max_bytes_to_store_per_request=73400320
This will allow JMeter to save files up to 70MB
You can also set this property to 0 - in this case JMeter will not truncate data (make sure you amend JVM heap so responses could fit in memory)
References:
Configuring JMeter
Apache JMeter Properties Customization Guide
I'm making an upload form for zips in a ruby webmachine app. My idea is to have an upload through my backend where I can add some extra params and then upload it to amazons s3 service with RestClient.
I did successfully create a direct upload (web based form post) to a s3bucket, but in that way I'm unable to handle the variables which are needed in the request, the way I want.
I've tried several things but I can't figure out, how to handle the request, as soon as it gets in my backend. I've created a resource and I'm debugging directly in the process_post method.
My #request variable represents a Webmachine::Request, with a Webmachine::Adapters::Rack::RequestBody and a Rack::Request, but I can't get the file out of it to use it as input for my RestClient request.
I think; #request.body.to_s and #request.body.to_io, represent the uploaded file in some way, and I tried to use them as input for Rack::Multipart methods, but that doesn't give me the file.
I also tried to work with the rack-raw-upload gem, but I can't get the mime-type something else than "application/x-www-form-urlencoded" or multipart. I do explicitly set it to; application/octet-stream
Things like File.new(filename, 'rb') gave me `rrno::ENOENT: No such file or directory # rb_sysopen'. For filename I just used 'example.zip'.
I guess I'm missing something which has to do with the Rack::Request call(env) method.
Does somebody have an idea, on how to handle the Rack uploads? Or give me any hints for a new direction? Thanks.
I've created a gist which shows how to retrieve the multipart stream. You'll need further parsing in order to get the uploaded file.
https://gist.github.com/jewilmeer/eb40abd665b70f53e6eb60801de24342
I am building an application which will allow users to upload images. Mostly, it will work with mobile browsers with slow internet connections. I was wondering if there are best practices for this. Does doing some encryption and than doing the transfer and decoding on server is a trick to try ? OR something else?
You would want something preferably with resumable uploads. Since your connections is slow you'd need something that can be resumed where you left off. A library i've come across over the many years is Nginx upload module:
http://www.grid.net.ru/nginx/upload.en.html
According to the site:
The module parses request body storing all files being uploaded to a directory specified by upload_store directive. The files are then being stripped from body and altered request is then passed to a location specified by upload_pass directive, thus allowing arbitrary handling of uploaded files. Each of file fields are being replaced by a set of fields specified by upload_set_form_field directive. The content of each uploaded file then could be read from a file specified by $upload_tmp_path variable or the file could be simply moved to ultimate destination. Removal of output files is controlled by directive upload_cleanup. If a request has a method other than POST, the module returns error 405 (Method not allowed). Requests with such methods could be processed in alternative location via error_page directive.
What is the best practice when serving files from the Zend Framework MVC? These files have to be served from the MVC as they are protected.
I know you can read in the file and place it into the Response object but this seems like a bad practice as you would be reading the entire file into memory then serving it. Right now I usually do:
header('Content-type: image/jpeg');
fpassthru(fopen($path, 'rb'));
exit;
But this also doesn't seem right as I'm stopping the execution of the script. Any suggestions?
I see nothing wrong with just exit(); What you will need to be careful of is any output buffering layers you may have on (gzip compression, etc). Large files could blow up those buffers pretty quick, so you'll want to close them out and potentially 'chunk' your output with a fopen/fread loop.
I would suggest building a super-simple script for retrieving files based on ticket system like in CMS you generate ticket to DB - filename, unique-hash and than redirect to the super-simple file-retieving script (file.php?hash=asd52ad3as1g5). It will get the hash from query and based on it fetch the real filename and push that to output as you have written using fpassthru. The hash need to be unique and hard to guess...
You could try using the X-Sendfile header. It is supported by lighttpd and newer versions of apache. Basically the webserver will replace the output of the script with the file you specified. The downside being that it is specific to the configuration of the webserver, so you may be on a host that doesn't support it.