I am trying to send a multi part post request from my local Marklogic server using this xquery script:
xquery version "1.0-ml";
let $uri := "http://uri_to_service"
let $data :=
'------WebKitFormBoundaryq3PKI0zfbTv08rDu
Content-Disposition: form-data; name="projectId"
1DBC9DEE-6B5D-0001-D76C-13F516502B00
------WebKitFormBoundaryq3PKI0zfbTv08rDu
Content-Disposition: form-data; name="language"
de
------WebKitFormBoundaryq3PKI0zfbTv08rDu
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain
Das ist ein Test.
------WebKitFormBoundaryq3PKI0zfbTv08rDu
Content-Disposition: form-data; name="submit"
Submit
------WebKitFormBoundaryq3PKI0zfbTv08rDu--
'
let $options :=
<options xmlns="xdmp:http">
<authentication method="basic">
<username>abc</username>
<password>123</password>
</authentication>
<data>{$data}</data>
<headers>
<Connection>keep-alive</Connection>
<Cache-Control>max-age=0</Cache-Control>
<Accept>text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8</Accept>
<Origin>null</Origin>
<User-Agent>Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36</User-Agent>
<Content-Type>multipart/form-data; boundary=----WebKitFormBoundaryq3PKI0zfbTv08rDu</Content-Type>
<Accept-Encoding>gzip,deflate,sdch</Accept-Encoding>
<Accept-Language>en-US,en;q=0.8,de;q=0.6</Accept-Language>
</headers>
</options>
return
xdmp:http-post($uri, $options)
In Wireshark I see the following outgoing request:
POST /api/extract HTTP/1.1
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Origin: null
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryq3PKI0zfbTv08rDu
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8,de;q=0.6
From: admin#marcos-macbook-pro.local
Host: some_service_uri
Content-Length: 509
Authorization: Basic some_basic_code
------WebKitFormBoundaryq3PKI0zfbTv08rDu
Content-Disposition: form-data; name="projectId"
1DBC9DEE-6B5D-0001-D76C-13F516502B00
------WebKitFormBoundaryq3PKI0zfbTv08rDu
Content-Disposition: form-data; name="language"
de
------WebKitFormBoundaryq3PKI0zfbTv08rDu
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain
Das ist ein Test.
------WebKitFormBoundaryq3PKI0zfbTv08rDu
Content-Disposition: form-data; name="submit"
Submit
------WebKitFormBoundaryq3PKI0zfbTv08rDu--
HTTP/1.1 400 Bad Request
Server: nginx/1.6.1
Date: Fri, 22 Aug 2014 14:18:47 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 1246
Connection: keep-alive
Set-Cookie: JSESSIONID=B677A4E57DB4E8E8076EB4C9DD3E0AF4; Path=/extractor
<html><head><title>Apache Tomcat/6.0.36 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 400 - Could not parse multipart servlet request; nested exception is org.apache.commons.fileupload.FileUploadException: Stream ended unexpectedly</h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u>Could not parse multipart servlet request; nested exception is org.apache.commons.fileupload.FileUploadException: Stream ended unexpectedly</u></p><p><b>description</b> <u>The request sent by the client was syntactically incorrect.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/6.0.36</h3></body></html>
As you can see, the request is failing for some reason.
Then I tried the same request from Chrome Browser with a small html form.
<html>
<head>
<title>Test Form</title>
</head>
<body>
<form action="service_uri" method="post" enctype="multipart/form-data">
<input type="text" name="projectId" value="1DBC9DEE-6B5D-0001-D76C-13F516502B00">
<input type="text" name="language" value="de">
<label for="file">Filename:</label>
<input type="file" name="file" id="file"><br>
<input type="submit" name="submit" value="Submit">
</form>
</body>
</html>
The browser asks me for username/password and everything goes fine.
In Wireshark I can see:
POST /api/extract HTTP/1.1
Host: some_service_uri
Connection: keep-alive
Content-Length: 527
Cache-Control: max-age=0
Authorization: Basic some_basic_code
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Origin: null
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryq3PKI0zfbTv08rDu
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8,de;q=0.6
Cookie: JSESSIONID=469591D4A83F24BB77547B6C235F25B7
------WebKitFormBoundaryq3PKI0zfbTv08rDu
Content-Disposition: form-data; name="projectId"
1DBC9DEE-6B5D-0001-D76C-13F516502B00
------WebKitFormBoundaryq3PKI0zfbTv08rDu
Content-Disposition: form-data; name="language"
de
------WebKitFormBoundaryq3PKI0zfbTv08rDu
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain
Das ist ein Test.
------WebKitFormBoundaryq3PKI0zfbTv08rDu
Content-Disposition: form-data; name="submit"
Submit
------WebKitFormBoundaryq3PKI0zfbTv08rDu--
HTTP/1.1 200 OK
Server: nginx/1.6.1
Date: Fri, 22 Aug 2014 13:38:04 GMT
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
80
{"text":"Das ist ein Test.\n","metadata":{},"document":{"freeTerms":[{"textValue":"test","score":100,"frequencyInDocument":1}]}}
0
Why is my request failing when send from Marklogic? Based on the outgoing request (captured by Wireshark) I can find no difference, except Content-Length.
What is happening? Or what could be the reason? Could encoding change the Content-Length?
The server error was Stream ended unexpectedly so content-length seems like a good candidate: 527 vs 509.
Rather than writing your own multipart code, have you tried this library? https://github.com/ableasdale/consultant-tools/blob/da92815e1322fe8e479ba2ba3741857670498c2a/src/main/resources/modules/example/lib-multipart-post.xqy
One key difference is that it uses https://docs.marklogic.com/xdmp:multipart-encode
Related
I have a portal page that is stuck on the preloader. In the Inspector, it throws this error:
POST https://####/wp-admin/admin-ajax.php[HTTP/2 500 Internal Server Error 5675ms]
Below that, there is these error messages:
Status: 500 Internal Server Error
Version: HTTP/2
Transferred: 726 B (241 B size)
Referrer Policy: strict-origin-when-cross-origin
RESPONSE HEADERS
access-control-allow-credentials: true
access-control-allow-origin: https://####
cache-control: no-cache, must-revalidate, max-age=0
content-type: text/html; charset=UTF-8
date: Sun, 30 Oct 2022 08:45:32 GMT
expires: Wed, 11 Jan 1984 05:00:00 GMT
referrer-policy: strict-origin-when-cross-origin
server: Apache
x-content-type-options: nosniff
X-Firefox-Spdy: h2
x-frame-options: SAMEORIGIN
x-robots-tag: noindex
REQUEST HEADERS
Accept: application/json, text/javascript, */*; q=0.01
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.5
Connection: keep-alive
Content-Length: 1242
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Cookie: wordpress_sec_43463c74c2bcdfa8f63c308437d1e9a2=########.com%7C1668185768%7C3RDSuJIwsueKkV0XMKCFD8aYWn14tchxzW5ygy5ZT2V%7Cc98e394e223dbf6e592589722ee40693641c42b27df660dc8bf0d307d26d1bb8; wordpress_logged_in_43463c74c2bcdfa8f63c308437d1e9a2=rreis%40ejme.com%7C1668185768%7C3RDSuJIwsueKkV0XMKCFD8aYWn14tchxzW5ygy5ZT2V%7Caa83aa2070723dd4047efdf67a1d3681a827359dfac09e97fe3ebf5156f6c343; wp-settings-175=editor%3Dtinymce; wp-settings-time-175=1666975646; wpc_key=BcD%26y9QHSMt%21mygJDcMm; PHPSESSID=ac2505ebc1aa49f5748064cb9bbfc233; sc_is_visitor_unique=rx11212407.1667119532.8D619E70C6CF4F2F03D9DB342E78DFB3.2.2.2.2.2.2.2.1.1
Host: www.####.co.uk
Origin: https://www.####.co.uk
Referer: https://www.####.co.uk/portal/portal-hub/
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
TE: trailers
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:106.0) Gecko/20100101 Firefox/106.0
X-Requested-With: XMLHttpRequest
Can anybody tell what is throwing the error? And is this why the page wont load?
Thanks
Request:
POST data:
--RSTXvNAvF6qiZc6NOcKWqw9HHxFBxbOOrs
Content-Disposition: form-data; name="ProjectUID"
Content-Type: form-data; charset=UTF-8
Content-Transfer-Encoding: 8bit
127
--RSTXvNAvF6qiZc6NOcKWqw9HHxFBxbOOrs
Content-Disposition: form-data; name="Customer"
Content-Type: form-data; charset=UTF-8
Content-Transfer-Encoding: 8bit
23
--RSTXvNAvF6qiZc6NOcKWqw9HHxFBxbOOrs
Content-Disposition: form-data; name="LoansInsertedVia"
Content-Type: form-data; charset=UTF-8
Content-Transfer-Encoding: 8bit
Single
--RSTXvNAvF6qiZc6NOcKWqw9HHxFBxbOOrs
Content-Disposition: form-data; name="LoanNumber"
Content-Type: form-data; charset=UTF-8
Content-Transfer-Encoding: 8bit
loan0106_3
--RSTXvNAvF6qiZc6NOcKWqw9HHxFBxbOOrs
Content-Disposition: form-data; name="ProductUID"
Content-Type: form-data; charset=UTF-8
Content-Transfer-Encoding: 8bit
8
Response:
{"validation_error":1,"message":" Please complete all mandatory fields","Customer":"This Field is required","LoanNumber":"This Field is required","Single-ProductUID":"This Field is required","Single-ProjectUID":"This Field is required"}
Try unticking Use multipart/form-data box in the HTTP Request sampler, it might be the case your server expects "normal" form-data parameters, not the multipart ones.
Also if you're capable of successfully executing your request with the browser you can just record it using JMeter's HTTP(S) Test Script Recorder or JMeter Chrome Extension
I am sending this request to the server:
POST /stream.php?K=pparam HTTP/1.0
Content-Type: multipart/form-data;boundary=MY_BOUNDARY_STRING_NOONE_HAS
Host: someweb.com
Content-Length:22182
--MY_BOUNDARY_STRING_NOONE_HAS
Content-Disposition: form-data; name="XXXXX16" filename="YYYYY17"
Content-Type: application/octet-stream
[IMAGE 22182 BUFFER]
--MY_BOUNDARY_STRING_NOONE_HAS
Content-Disposition: form-data; name="XXXXX12" filename="YYYYY13"
Content-Type: application/octet-stream
[IMAGE 22163 BUFFER]
--MY_BOUNDARY_STRING_NOONE_HAS
But at the server:
The php $_FILES is empty
Array
(
)
<?php
print_r($_FILES);
?>
The whole request should have the Accept: header
Now I am able to upload a camera stream
POST /stream.php?K=127.0.0.1_camera1 HTTP/1.1
Host: localhost
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Encoding: gzip, deflate
Liveimage: camera1
Content-Type: multipart/form-data; boundary=212367691630846570233068106084
Content-Length:20209
Connection: keep-alive
--212367691630846570233068106084
Content-Disposition: form-data; name="file10"; filename="S11"
Content-Type: application/octet-stream
[IMAGE 20209]
boundary=--212367691630846570233068106084
Content-Disposition: form-data; name="file12"; filename="S13"
Content-Type: application/octet-stream
[IMAGE 20209]
boundary=--212367691630846570233068106084
RRRRRR POST
Content-Disposition: form-data; name="file14"; filename="S15"
Content-Type: application/octet-stream
[IMAGE 20209]
boundary=--212367691630846570233068106084
I am facing some weird problem while replaying my recorded script. I have correlated everything and for rest of the steps the CSRF token is working fine but for the below step its giving error for me.
The recorded script is:
web_add_header("X-CSRF-TOKEN",
"1f285aef-f9b1-4709-a76b-6789e785ca8a");
web_add_header("X-Requested-With",
"XMLHttpRequest");
lr_think_time(33);
web_custom_request("saveScheduleAcademyMapping",
"URL=http://localhost:8080/ams/saveScheduleAcademyMapping",
"Method=POST",
"Resource=0",
"RecContentType=application/json",
"Referer=http://localhost:8080/ams/manage_academy?scheduleInstanceId={scheduleID}",
"Snapshot=t1553.inf",
"Mode=HTTP",
"EncType=application/json; charset=UTF-8",
"Body=[{JSON values]",
LAST);
Below are the request and response while recording and replaying the script
For Recording
Request:
POST /ams/saveScheduleAcademyMapping HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Content-Length: 791
Accept: */*
Origin: http://localhost:8080
X-CSRF-TOKEN: 1f285aef-f9b1-4709-a76b-6789e785ca8a
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36
Content-Type: application/json; charset=UTF-8
Referer: http://localhost:8080/ams/manage_academy?scheduleInstanceId=230044
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cookie: JSESSIONID=B9C8BB02040E49246E1C7BA4CC16F6CF
[{JSON VALUE}]
Response:
HTTP/1.1 200
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Frame-Options: DENY
X-Application-Context: application:dev
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked
Date: Wed, 25 Apr 2018 12:09:17 GMT
[{JSON VALUE}]
For Replay:
Request:
POST /ams/saveScheduleAcademyMapping HTTP/1.1
Content-Type: application/json; charset=UTF-8
Referer: http://localhost:8080/ams/manage_academy?scheduleInstanceId=230046
Origin: http://localhost:8080
X-CSRF-TOKEN: 1f285aef-f9b1-4709-a76b-6789e785ca8a
X-Requested-With: XMLHttpRequest
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Accept: */*
Connection: Keep-Alive
Host: localhost:8080
Cookie: JSESSIONID=E61AF0BA93B173F3D597244508FE11DD
Content-Length: 791
[{JSON VALUE}]
Response:
HTTP/1.1 403
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Frame-Options: DENY
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked
Date: Wed, 25 Apr 2018 12:25:05 GMT
{"timestamp":"2018-04-25T12:25:05.690+0000","status":403,"error":"Forbidden","message":"Invalid CSRF Token '1f285aef-f9b1-4709-a76b-6789e785ca8a' was found on the request parameter '_csrf' or header 'X-CSRF-TOKEN'.","path":"/ams/saveScheduleAcademyMapping"}
Can anyone help me with this.
Its weird to answer my question itself but need to close it hence postintg it.
Just correlated one more time and added the lines in script like below and its working now. using web_reg_save_param_ex() function fetched the CSRF and used it.
web_add_header("X-CSRF-TOKEN",
"{CSRF}");
web_add_header("X-Requested-With",
"XMLHttpRequest");
I have a h:commandButton associated with a method, on clicking the button the action method is executed but the redirect does not take place.
'xhtml snippet'
<h:commandButton id="update" value="save"
onclick="if( validate('submitForm')==true){ updateProperty();} else{ return false;}"
action="#{propertyBean.update}"
class="btn btn-green btn-lg arabic"
rendered="#{propertyBean.isNew == false}">
<f:param name="propertyId" value="#{propertyBean.property.id}">
</f:param>
</h:commandButton>
'bean snippet'
public String update() {
System.out.println("in update method");
return "/success.xhtml";
}
Update
This is a snippet of the network monitor, the response is correct even though the page displayed and the URL in the browser are wrong
POST /Newgar_revamp/pages/add.xhtml HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Content-Length: 2207
Pragma: no-cache
Cache-Control: no-cache
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Origin: http://localhost:8080
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryFE3725BjJXsA1xKf
Referer: http://localhost:8080/Newgar_revamp/pages/add.xhtml?propertyId=3
Accept-Encoding: gzip, deflate
Accept-Language: en-GB,en-US;q=0.8,en;q=0.6,ar;q=0.4
Cookie: JSESSIONID=B81FF21F23988CC8C98D84DA14B36F54
HTTP/1.1 302 Found
Server: Apache-Coyote/1.1
Location: http://localhost:8080/Newgar_revamp/pages/status/success.xhtml
Content-Length: 0
Date: Fri, 20 Nov 2015 21:24:32 GMT