Multiple file Upload - Multipart form data maximum limit - spring

I am trying to upload multiple files with the following request along with stringed JSON object:
Content-Disposition: form-data; name="params"
{"data":{"userName":"jim","description":"test","email":"jim#ox.com"}}
-----------------------------5366762814869373672043632099
Content-Disposition: form-data; name="file0"
VBORw0KGgoAAAANSUhEUgAAAFwAAAA/CAYAAABtj6+sAAAYJ2lDQ1BJQ0MgUHJvZmlsZQAAWIWVeQdUFE2zds
..... (content of file0)
-----------------------------5366762814869373672043632099
Content-Disposition: form-data; name="file1"
cBORw0KGgoAAAANSUhEUgAAAEsAAABpCAYAAAByKt7XAAAYJ2lDQ1BJQ0MgUHJvZmlsZQAAWIWVeQdUFE2zds
..... (content of file1)
-----------------------------5366762814869373672043632099--
file0 is of 16KB and file1 is of 12KB
and My spring controller method looks like this:
#POST
#Path("/addFile")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public void addFilesWithParams(MultipartFormDataInput filesData)
{
//some logic
}
But always ends up with one file saved and second file is ignored.
So i went ahead debuged my code and found out that On REST side I am only getting 2 Parts of the form i.e params and file0 and file1 is totaly ignored.
As you can see in the above snap it says size is 2, even if 3 or 4 parts are being sent from ui.
So my guess is there is some Size cap that I have to override or specify.
Can anyone help me with this issue.

Related

Google Cloud DLP - CSV inspection

I'm trying to inspect a CSV file and there are no findings being returned (I'm using the EMAIL_ADDRESS info type and the addresses I'm using are coming up with positive hits here: https://cloud.google.com/dlp/demo/#!/). I'm sending the CSV file into inspect_content with a byte_item as follows:
byte_item: {
type: :CSV,
data: File.open('/xxxxx/dlptest.csv', 'r').read
}
In looking at the supported file types, it looks like CSV/TSV files are inspected via Structured Parsing.
For CSV/TSV does that mean one can't just sent in the file, and needs to use the table attribute instead of byte_item as per https://cloud.google.com/dlp/docs/inspecting-structured-text?
What about for XSLX files for example? They're an unspecified file type so I tried with a configuration like so, but it still returned no findings:
byte_item: {
type: :BYTES_TYPE_UNSPECIFIED,
data: File.open('/xxxxx/dlptest.xlsx', 'rb').read
}
I'm able to do inspection and redaction with images and text fine, but having a bit of a problem with other file types. Any ideas/suggestions welcome! Thanks!
Edit: The contents of the CSV in question:
$ cat ~/Downloads/dlptest.csv
dylans#gmail.com,anotehu,steve#example.com
blah blah,anoteuh,
aonteuh,
$ file ~/Downloads/dlptest.csv
~/Downloads/dlptest.csv: ASCII text, with CRLF line terminators
The full request:
parent = "projects/xxxxxxxx/global"
inspect_config = {
info_types: [{name: "EMAIL_ADDRESS"}],
min_likelihood: :POSSIBLE,
limits: { max_findings_per_request: 0 },
include_quote: true
}
request = {
parent: parent,
inspect_config: inspect_config,
item: {
byte_item: {
type: :CSV,
data: File.open('/xxxxx/dlptest.csv', 'r').read
}
}
}
dlp = Google::Cloud::Dlp.dlp_service
response = dlp.inspect_content(request)
The CSV file I was testing with was something I created using Google Sheets and exported as a CSV, however, the file showed locally as a "text/plain; charset=us-ascii". I downloaded a CSV off the internet and it had a mime of "text/csv; charset=utf-8". This is the one that worked. So it looks like my issue was specifically due the file being an incorrect mime type.
xlsx is not yet supported. Coming soon. (Maybe that part of the question should be split out from the CSV debugging issue.)

Get the random file name

With a folder contained around 10 files (like 1.csv, 2.csv....10.csv) And I am uploading them to my http request using a beanshell preprocessor using the following script,
File folder = new File("C:\\User\\SYSTEMTESTING\\SAMPLENEWFILES\\REUPLOADFILES");
File[] fileForUpload = folder.listFiles();
Random rnd = new Random();
vars.put("CURRENT_FILE", fileForUpload[rnd.nextInt(fileForUpload.length)].getAbsolutePath());
want to get the file name which is uploaded using a JSR223 post processer
log.info("File Uploaded Is :"+${CURRENT_FILE});
I am getting,
javax.script.ScriptException: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script8.groovy: 1: Missing ')' # line 1, column 79.
"File Uploaded Is ------->"+C:\Users\SY
In the request body:
PUT data:
--RhvwJL7ZdnIMBIaE0CoKVhsE68UNUiH
Content-Disposition: form-data; name="file"; filename="CSV_10_MB.csv"
Content-Type: application/vnd.ms-excel
Content-Transfer-Encoding: binary
<actual file content, not shown here>
--RhvwJL7ZdnIMBIaE0CoKVhsE68UNUiH--
I want the CSV_10_MB.csv name.
Use vars.get:
log.info("File Uploaded Is :" + vars.get("CURRENT_FILE"));
If you want to use Groovy's GString you need to declare this CURRENT_FILE property, if this is what you're looking for you need to amend your code like:
def CURRENT_FILE = vars.get("CURRENT_FILE")
log.info("File Uploaded Is: ${CURRENT_FILE}")
or use the same vars shorthand for printing the filename to JMeter log like:
log.info("File Uploaded Is: " + vars.get("CURRENT_FILE"))

Uploading a file to hdfs through hdfs API results in file getting appended and prepended with signature

My goal is to upload a file, this is what my code looks like:
headers = {
'Some_Auth_Stuff': _get_ca_cert(ROLE),
'Host': host,
}
files = {'upload_file': file}
params = (
('op', 'create'),
('permission', '755')
)
r = requests.put(
'https://proxystuff.hostname.com/fs%s' % hue_path,
headers=headers, files=files, params=params)
if r.status_code == 201:
return True
return False
and I'm uploading this file:
i am a test file
I get a 201 response, which is great but when I look at the file, it looks like so:
--04dc34a8a49d4b83878473d6d78e683d
Content-Disposition: form-data; name="upload_file"; filename="testfile"
i am a test file
--04dc34a8a49d4b83878473d6d78e683d--
Am I missing something when it comes down to uploading content? Any way to disable the file from getting stuff prepended and appended?
EDIT:
If I use this curl command it works fine
curl -c cookie -b cookie -T "test.txt" "https://proxystuff.hostname.com/fs/user/stupidfatcat/test.txt?op=create&permission=755" -H "Some_Auth_Stuff:blahblah" -H "Host:someotherhost_with_hadoop.com:4443"
After trying some stuff out, if I changed it to
headers=headers, data=file.read(), params=params
and I set the Content-Type to plain/text it works fine, seems like it doesn't like file param.

JMeter: Body & File content

The WebAPI request has a POST method which expects Content body. I've tried to use both Parameters and Body options but I receive error responses - 'Invalid Request' with 400 Status code, etc.
JMeter request Sample Content Body:
{
"ParamA": 111,
"ParamB": "Char String",
"ParamC": "VarType"
}
OR
{ "ParamA": 111, "ParamB": "Char String", "ParamC": "VarType"}
Listener Request:
POST data:
--8vpH3B6WcV4f1La46_wccVi4c25lrLJaGcN--
Listener Response:
{"message":"The request is invalid.","modelState":{"value":["An error
has occurred."]}}
Any insight into viable options? Eventually, I'm planning on reading the Body string from a .csv file so I can parameterize the request. Reading from a .CSV file only reads the first line of the request body - for example: '{'
Any help would be greatly appreciated.
Best,
Ray
HTTP Request
Request
Uncheck in HTTP request the option:
Use multipart/form data for POST
Also check your CSV does not contain some data that contains the CSV separator which is '\t' by default.
Ensure it doesn't by changing separator to '|' for example if you're sure your JSON will never contain it.

Ruby/Rails - Accessing params values

A newbie question I assume but here we go: I have the following params:
{"utf8"=>"✓",
authenticity_token"=>".........",
"import"=>
{"csv"=>
#<ActionDispatch::Http::UploadedFile:0x007fb59092a660
#content_type="text/csv",
#headers="Content-Disposition: form-data; name=\"import[csv]\"; filename=\"Users.csv\"\r\nContent-Type: text/csv\r\n",
#original_filename="DemoUsers.csv",
#tempfile=#<File:/var/folders/_p/w29hlx3x0cs6h026txv_rqhc0000gn/T/RackMultipart20141211-8204-1ha0i1u>>,
"datatype"=>"users"},
"commit"=>"Import",
"action"=>"create",
"controller"=>"imports"}
In my code, I need to assigns the value of #tempfile to a local variable but I just cant figure out how. ;-)
Most part of params are in params. So try
local_val = params["import"]["csv"].tempfile
suppose you assign response to a variable res
res = {"utf8"=>"✓",
authenticity_token"=>".........",
"import"=>
{"csv"=>
#<ActionDispatch::Http::UploadedFile:0x007fb59092a660
#content_type="text/csv",
#headers="Content-Disposition: form-data; name=\"import[csv]\"; filename=\"Users.csv\"\r\nContent-Type: text/csv\r\n",
#original_filename="DemoUsers.csv",
#tempfile=#<File:/var/folders/_p/w29hlx3x0cs6h026txv_rqhc0000gn/T/RackMultipart20141211-8204-1ha0i1u>>,
"datatype"=>"users"},
"commit"=>"Import",
"action"=>"create",
"controller"=>"imports"}
Now,
res["import"]["csv"].tempfile

Resources