Mule 4.1.4 Failing upload compressed xml file content through HTTP POST request - mule-component

Question

I am migrating Mule 3.9 file upload logic to Mule 4.1.4 version. For simplicity, in Mule 4.1.4 I am trying with basic logic using http connector to upload compressed xml file content to post to HTTP POST request, it keep failing with BAD_REQUEST, not getting what is wrong with my input.
Please suggest what am I missing in it???
Mule 3.9 existing working code:
<flow name="Post_XML_To_ExtSystem" processingStrategy="synchronous">
<timer-interceptor/>
<object-to-byte-array-transformer doc:name="Object to Byte Array"/>
<gzip-compress-transformer doc:name="Gzip Compress"/>
<logger message="gZip compression completed for Part: #[flowVars.partId]" level="INFO" doc:name="gZip completed"/>
<flow-ref name="WriteToFile_Flow" doc:name="Write to File Optionally"/>
<set-variable variableName="fileContentgzip" value="#[payload]" doc:name="fileContentgzip"/>
<flow-ref name="SetAttachments_PostPayload_Flow" doc:name="SetAttachments_PostPayload_Flow - FlowRef"/>
<exception-strategy ref="Global_Errorflow_Choice_Exception_Strategy" doc:name="Reference Exception Strategy"/>
</flow>
<sub-flow name="SetAttachments_PostPayload_Flow">
<logger message="Post Payload Flow with vars: #[flowVars]" level="DEBUG" doc:name="Logger"/>
<set-attachment attachmentName="TenantID" value="#['${http.ext.system.tenant}']" contentType="text/plain" doc:name="Tenant ID"/>
<set-attachment attachmentName="Category" value="#[flowVars.Category]" contentType="text/plain" doc:name="Category"/>
<set-attachment attachmentName="Data" value="#[flowVars.fileContentgzip]" contentType="application/xml" doc:name="Data"/>
<scripting:component doc:name="filename attachment">
<scripting:script engine="Groovy"><![CDATA[import org.mule.message.ds.ByteArrayDataSource;
import javax.activation.DataHandler;
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream;
String category = message.getInvocationProperty("Category")
String fileName=category + '.xml'
String attachmentName='Data'
byte[] compressed = flowVars.fileContentgzip
ByteArrayDataSource attachment = new ByteArrayDataSource(compressed, "application/xml",fileName);
message.addOutboundAttachment(attachmentName, new DataHandler(attachment))
return payload;]]></scripting:script>
</scripting:component>
<copy-attachments attachmentName="*" doc:name="All attachments together"/>
<set-payload value="#[null]" doc:name="Nullify Payload"/>
<logger message="before ingestion call: ${http.by.ingestion.basepath}, ${http.by.ingestion.host}, ${http.by.ingestion.port}" level="DEBUG" doc:name="Log Ingestion basepath, host, port"/>
<logger message="Begin Posting #[flowVars.Category] for Part: #[flowVars.partId]" level="INFO" doc:name="Begin Posting data"/>
<flow-ref name="Ingestion_with_retries_Flow" doc:name="Flow Ref Ingestion_with_retries" doc:description="retry injestion api call"/>
</sub-flow>
<flow name="Ingestion_with_retries_Flow" >
<until-successful objectStore-ref="objectStore" maxRetries="${max.retries}" deadLetterQueue-ref="Failed_Payload_To_ErrorDir_And_Notify"
failureExpression="#[(exception != null) and (exception.causedBy(java.net.ConnectException) || exception.causedBy(java.net.SocketTimeoutException) || exception.causedBy(java.net.SocketException) || exception.causedBy(java.io.IOException))]"
doc:name="Until Successful" millisBetweenRetries="${millis.between.retries}">
<processor-chain doc:name="Processor Chain">
<logger message="Posting data to Server" level="INFO" doc:name="Logger"/>
<http:request config-ref="HTTPS_Ingestion_Service_ExtSystem" path="/delivery" method="POST" doc:name="ExtSystem Data Delivery Post">
<http:request-builder>
<http:header headerName="Accept" value="${http.by.interface.version}"/>
<http:header headerName="Content-Encoding" value="gzip"/>
</http:request-builder>
<http:success-status-code-validator values="200"/>
</http:request>
<json:xml-to-json-transformer doc:name="XML to JSON"/>
<flow-ref name="Subflow_Extract_Ingestion_Response" doc:name="Extract Ingestion Response"/>
</processor-chain>
</until-successful>
</flow>
<sub-flow name="Subflow_Extract_Ingestion_Response">
<object-to-string-transformer returnClass="java.lang.String" mimeType="application/json" doc:name="Response_to_String"/>
<dw:transform-message doc:name="Extract DeliveryId">
<dw:set-payload resource="classpath:ingestion\ingestion-delivery.dwl"/>
</dw:transform-message>
<json:json-to-object-transformer returnClass="java.lang.Object" doc:name="JSON to Object"/>
<set-variable variableName="ExtSystemDeliveryID" value="#[payload.DeliveryID]" doc:name="ExtSystemDeliveryID"/>
<logger message="Delivery ID: #[payload.DeliveryID]" level="INFO" doc:name="Log Delivery Id"/>
<set-variable variableName="ExtSystemStatus" value="#[payload.Status]" doc:name="ExtSystemStatusStatus"/>
<flow-ref name="Update_DeliveryID_Category_in_Part_Flow" doc:name="Update Part with DeliveryID and Category"/>
<set-payload value="#[payload + '\n']" doc:name="Set Payload"/>
<file:outbound-endpoint path="${write.folderpath}#[flowVars.correlationId]" outputPattern="HttpResponse_IngestionIDs.txt" connector-ref="File" responseTimeout="10000" doc:name="Write Ingestion Response"/>
<logger message="Ingestion response stored at ${write.folderpath}#[flowVars.batchJobInstanceId]/#[flowVars.Category]_#[flowVars.partId].gz" level="INFO" doc:name="Log response path"/>
</sub-flow>
Mule 4.1.4 XML compressed File Upload Logic
<flow name="storeStocksFlow" doc:id="2d611c4c-edec-4b75-aa94-25474d145040" >
<http:listener doc:name="POST/payloadtest" doc:id="a5ed0fce-aa12-4e00-a68c-fe99008f1559" allowedMethods="POST" config-ref="HTTP_Listener_config" path="/payloadtest" outputMimeType="application/json">
</http:listener>
<logger level="INFO" doc:name="Logger" doc:id="61545cb4-8d94-4af8-a08e-9bfd0667b77f" message="Input json request: #[payload]"/>
<set-variable value="#[payload]" doc:name="Set Variable" doc:id="3923534b-7482-4a8c-ad46-948fda597550" variableName="origJsonInPayload"/>
<set-variable value="#[uuid()]" doc:name="Set Variable correlationId" doc:id="49798fd3-3175-44f8-9443-368b9a018207" variableName="correlationId"/>
<logger level="INFO" doc:name="Logger before transformation" doc:id="c72a768f-58c4-4947-b29c-93aa955b18a5" message="Before transformation: #[payload]"/>
<logger level="INFO" doc:name="Logger after transformation" message="Logger after transformation: #[payload]" doc:id="287ee190-47f6-4af6-982e-6f93a66cc052"/>
<!-- Tried both compressed and plain xml format both giving BAD_REQUEST error
<compression:compress doc:name="Gzip Compress" doc:id="bf8e4d8e-dbce-43f8-982a-ff68b87839c0" >
<compression:compressor >
<compression:gzip-compressor />
</compression:compressor>
</compression:compress> -->
<logger message="gzip compression completed - payload:#[payload]" level="INFO" doc:name="gZip completed" />
<set-variable variableName="fileContentgzip" value="#[payload]" doc:name="fileContentgzip" />
<set-variable variableName="TenantID" value="#['${http.ext.system.tenant}']" mimeType="application/json" doc:name="Tenant ID"/>
<set-variable variableName="Category" value="Stocks" mimeType="application/json" doc:name="Category"/>
<set-variable variableName="Data" value="#[vars.fileContentgzip]" mimeType="application/xml" doc:name="Data"/>
<logger level="INFO" doc:name="Logger" doc:id="29a22776-354e-42b0-b486-36bedcf8d6f0" message="JOB entry created in JOB table."/>
<flow-ref name="BY_API_Call_SubFlow1" doc:name="ExtSystem Ingestion API Test"/>
</flow>
<sub-flow name="API_Call_SubFlow1">
<logger message="Posting data to Server" level="INFO" doc:name="Logger" />
<http:request config-ref="HTTPS_Ingestion_Service_ExtSystem" path="/delivery" method="POST" doc:name="Ext System Data Delivery Post" outputMimeType="application/xml">
<http:body><![CDATA[#[%dw 2.0
output application/xml
input payload multipart/form-data
---
{
parts : {
Data : {
headers : {
"Content-Disposition" : {
"name" : "Data",
"filename": "Stocks.xml"
},
"Accept" : 'application/xml',
"Content-Encoding": 'gzip',
"TenantID": "xxxx-yyyy-aaaa-bbbb-ccccccc",
"Category": "Stocks"
},
content : payload
}
}
}]]]></http:body>
<http:headers ><![CDATA[#[output application/java
---
{
"Content-Type" : "application/com.ext-system.xxx_and_yyy-v1.14.17+xml"
}]]]></http:headers>
<http:response-validator >
<http:success-status-code-validator values="200" />
</http:response-validator>
</http:request>
<logger level="INFO" doc:name="Ingestion API Response" doc:id="a9ece74e-4b86-486f-9f3c-16272d1d00d1" message="Ingestion API Response: #[payload]"/>
</sub-flow>
Error logs:
0-6ffbd441-5963-11e9-8d2b-0a0027000005] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: Posting data to Server
ERROR 2019-04-08 00:02:04,461 [[MuleRuntime].cpuLight.16: [adapter].storePersonFlow.CPU_LITE #427b75e6] [event: ] org.mule.runtime.core.internal.exception.OnErrorContinueHandler:
********************************************************************************
Message : HTTP POST on resource 'https://api.ext-system.com:443/xxxx/delivery' failed: bad request (400).
Error type : HTTP:BAD_REQUEST
Element : API_Call_SubFlow1/processors/1 # adapter:exposing-a-restful-resource-using-the-http-connector.xml:142 (Ext System Data Delivery Post)
Element XML : <http:request config-ref="HTTPS_Ingestion_Service_ExtSystem" path="/delivery" method="POST" doc:name="Ext System Data Delivery Post" outputMimeType="application/xml">
<http:body>#[%dw 2.0
output application/xml
input payload multipart/form-data
---
{
parts : {
Data : {
headers : {
"Content-Disposition" : {
"name" : "Data",
"filename": "Stocks.xml"
},
"Accept" : 'application/com.ext-system.xxx_and_yyy-v1.14.17+xml',
"Content-Encoding": 'gzip',
"TenantID": "xxxx-yyyy-aaaa-bbbb-ccccccc",
"Category": "Stocks"
},
content : payload
}
}
}]</http:body>
<http:headers>#[output application/xml
---
{
"Content-Type" : "application/com.ext-system.xxx_and_yyy-v1.14.17+xml"
}]</http:headers>
<http:response-validator>
<http:success-status-code-validator values="200"></http:success-status-code-validator>
</http:response-validator>
</http:request>
(set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
********************************************************************************
INFO 2019-04-08 00:02:04,466 [[MuleRuntime].cpuLight.16: [adapter].storePersonFlow.CPU_LITE #427b75e6] [event: 0-6ffbd441-5963-11e9-8d2b-0a0027000005] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: In HTTP:BAD_REQUEST

Answer 1

The error is returned by the host because it didn't like something about the request. It is very difficult to understand what could be the problem just by reviewing the code snippets. A complete picture requeries to know the data and details of the server validation that failed.
A way to resolve this if you have a working case (the 3.9 version) is to enable HTTP wire logging (https://support.mulesoft.com/s/article/How-to-Enable-HTTP-Wire-Logging) in both versions, execute them and compare both HTTP requests. Then you can see what is different between them and adjust the Mule 4 version to match the other request.

Answer 2

I got the root cause, headers were missing in the request, I could trace this after enabling DEBUG in log4j2.xml as alejandro-dobniewski suggested. In my use case TenantID, Category (values for these both keys are String) and Data are the keys which are part of multipart/form-data. Data value will be gzip file content. Below json is self explanatory.
Solution:
The correct format of headers and body (importantly the parts payload json) to HTTP Request is:
<http:body ><![CDATA[#[%dw 2.0
output multipart/form-data
---
{
parts: {
TenantID : {
headers : {
"Content-Type": "text/plain"
},
content : "xxxxxxxxx"
},
Category : {
headers : {
"Content-Type": "text/plain"
},
content : "MyCategory"
},
Data: {
headers: {
"Content-Disposition": {
"name": "Data",
"filename": "MyCategory_gzip"
},
"Content-Type": payload.^mimeType,
},
content: payload
}
}
}]]]></http:body>
<http:headers ><![CDATA[#[output application/java
---
{
"Accept" : "application/com.xxxxx-v1.1+xml",
"Content-Encoding" : "gzip"
}]]]></http:headers>