CXF JAX-RS client always sends empty PUT requests in chunking mode regardles of AllowChunking setting - chunked-encoding

We perform PUT request to our party using CXF JAX-RS client. Request body is empty.
A simple request invocation leads to server response with code 411.
Response-Code: 411
"Content-Length is missing"
Our party's REST-server requires Content-Length HTTP-header to be set.
We switched chunking off according to note about chunking but this did not solve the problem. The REST-server still answers with 411 error.
Here is our conduit configuration from cxf.xml file
<http-conf:conduit name="{http://myhost.com/ChangePassword}WebClient.http-conduit">
<http-conf:client AllowChunking="false"/>
</http-conf:conduit>
Line in the log confirms that execution of our request bound to our conduit configuration:
DEBUG o.a.cxf.transport.http.HTTPConduit - Conduit '{http://myhost.com/ChangePassword}WebClient.http-conduit' has been configured for plain http.
Adding Content-Length header explicitly also did not help.
Invocation.Builder builder = ...
builder = builder.header(HttpHeaders.CONTENT_LENGTH, 0);
A CXF Client's log entry confirms header setting, however when we sniffed packets, we have surprisingly found that header setting has been completely ignored by CXF client. Content-Length header was not sent.
Here is the log. Content-Length header is present:
INFO o.a.c.i.LoggingOutInterceptor - Outbound Message
---------------------------
ID: 1
Address: http://myhost.com/ChangePassword?username=abc%40gmail.com&oldPassword=qwerty123&newPassword=321ytrewq
Http-Method: PUT
Content-Type: application/x-www-form-urlencoded
Headers: {Accept=[application/json], client_id=[abcdefg1234567890abcdefg12345678], Content-Length=[0], Content-Type=[application/x-www-form-urlencoded], Cache-Control=[no-cache], Connection=[Keep-Alive]}
--------------------------------------
DEBUG o.apache.cxf.transport.http.Headers - Accept: application/json
DEBUG o.apache.cxf.transport.http.Headers - client_id: abcdefg1234567890abcdefg12345678
DEBUG o.apache.cxf.transport.http.Headers - Content-Length: 0
DEBUG o.apache.cxf.transport.http.Headers - Content-Type: application/x-www-form-urlencoded
DEBUG o.apache.cxf.transport.http.Headers - Cache-Control: no-cache
DEBUG o.apache.cxf.transport.http.Headers - Connection: Keep-Alive
And here is an output of the packet sniffer. Content-Length header is not present:
PUT http://myhost.com/ChangePassword?username=abc%40gmail.com&oldPassword=qwerty123&newPassword=321ytrewq HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Accept: application/json
client_id: abcdefg1234567890abcdefg12345678
Cache-Control: no-cache
User-Agent: Apache-CXF/3.1.8
Pragma: no-cache
Host: myhost.com
Proxy-Connection: keep-alive
Does anyone know how actually disable chunking?
Here is our code:
public static void main(String[] args)
{
String clientId = "abcdefg1234567890abcdefg12345678";
String uri = "http://myhost.com";
String user = "abc#gmail.com";
Client client = ClientBuilder.newBuilder().newClient();
WebTarget target = client.target(uri);
target = target.path("ChangePassword").queryParam("username", user).queryParam("oldPassword", "qwerty123").queryParam("newPassword", "321ytrewq");
Invocation.Builder builder = target.request("application/json").header("client_id", clientId).header(HttpHeaders.CONTENT_LENGTH, 0);
Response response = builder.put(Entity.form(new Form()));
String body = response.readEntity(String.class);
System.out.println(body);
}
Versions:
OS: Windows 7 Enterprise SP1
Arch: x86_64
Java: 1.7.0_80
CXF: 3.1.8

I had a very similar issue that I was not able to solve as you did by trying to turn off chunking.
What I ended up doing was setting the Content-Length to 1 and adding some white space " " as the body. For me it seemed that the proxy servers before the server application was rejected the request and by doing that got me past the proxy servers and the server was able to process the request as it was only operating based on the URL.

Related

Can't send http/2 request in Gatling

I've got a problem using gatling tool when trying to send http/2 request.
I've enable http/2 setting in protocol settings, even added mapping to make sure that client will communicate with server using http2 but still request is send using http/1.1.
In console output you'll see that server can communicate using http2 but for some reason request are sent only with http 1.1.
All headers are written according to the ones from browser.
Could someone help me please with this issue?
package test
import scala.concurrent.duration._
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import io.gatling.jdbc.Predef._
class RecordedSimulation extends Simulation {
val httpProtocol = http
.baseUrl("https://sitename")
.inferHtmlResources(BlackList(""".*\.js""", """.*\.css""", """.*\.gif""", """.*\.jpeg""", """.*\.jpg""", """.*\.ico""", """.*\.woff""", """.*\.woff2""", """.*\.(t|o)tf""", """.*\.png""", """.*detectportal\.firefox\.com.*"""), WhiteList())
.acceptHeader("*/*")
.acceptEncodingHeader("gzip, deflate")
.acceptLanguageHeader("en-US,en;q=0.5")
.userAgentHeader("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:85.0) Gecko/20100101 Firefox/85.0")
.enableHttp2
.http2PriorKnowledge(Map("sitename" -> true))
.disableCaching
.disableWarmUp
val headers_0 = Map(
"Accept" -> "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
"Cache-Control" -> "max-age=0",
"sec-fetch-dest" -> "document",
"sec-fetch-mode" -> "navigate",
"sec-fetch-site" -> "none",
"sec-fetch-user" -> "?1",
"Upgrade-Insecure-Requests" -> "1")
val scn = scenario("RecordedSimulation")
.exec(http("request_0")
.get("/auth?r=%2F&m=NOT_AUTHENTICATED")
.headers(headers_0))
.pause(2)
setUp(scn.inject(atOnceUsers(1))).protocols(httpProtocol)
}
In console I see next:
DEBUG io.gatling.http.client.impl.DefaultHttpClient - ALPN led to HTTP/2 with remote sitename
DEBUG io.gatling.http.client.impl.Http2AppHandler - Write request WritableRequest{request=DefaultFullHttpRequest(decodeResult: success, version: HTTP/1.1, content: UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeHeapByteBuf(ridx: 0, widx: 0, cap: 0))
GET https://sitename/auth?r=%2F&m=NOT_AUTHENTICATED HTTP/1.1
sec-fetch-site: none
Upgrade-Insecure-Requests: 1
If-Modified-Since: Fri, 29 Jan 2021 06:40:06 GMT
sec-fetch-dest: document
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Cache-Control: max-age=0
accept-encoding: gzip, deflate
If-None-Match: "1d6f60994776ee0"
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:85.0) Gecko/20100101 Firefox/85.0
accept-language: en-US,en;q=0.5
sec-fetch-mode: navigate
sec-fetch-user: ?1
accept: */*
Those are internal Gatling logs. We use Netty in a way that it converts HTTP/1.1 payloads into HTTP/2 ones.
From the logs you provided, it looks that Gatling is correctly using HTTP/2 here.

Feign Client: How to log the server name which the request was sent to?

I am using a Feign Client to call a REST endpoint with success, and have logging turned on to FULL. This is helpfully shows me the request sent and the response received. However, I cannot see which server the request was sent to. It only shows me that it was POSTed to http://foo-service which is the name of the service, not the name of the server.
How can I log which server name this request was sent to?
This is what I see in the logs:
c.l.l.r.service.FooFeignClient : [FooFeignClient#bar] ---> POST http://foo-service/some-endoint HTTP/1.1
c.l.l.r.service.FooFeignClient : [FooFeignClient#bar] Accept: application/json
c.l.l.r.service.FooFeignClient : [FooFeignClient#bar] Content-Type: application/json
c.l.l.r.service.FooFeignClient : [FooFeignClient#bar] Content-Length: 15
c.l.l.r.service.FooFeignClient : [FooFeignClient#bar]
c.l.l.r.service.FooFeignClient : [FooFeignClient#bar] {"name":"John"}
c.l.l.r.service.FooFeignClient : [FooFeignClient#bar] ---> END HTTP (15-byte body)
c.l.l.r.service.FooFeignClient : [FooFeignClient#bar] <--- HTTP/1.1 200 (8ms)
c.l.l.r.service.FooFeignClient : [FooFeignClient#bar] Transfer-Encoding: chunked
c.l.l.r.service.FooFeignClient : [FooFeignClient#bar] X-Application-Context: fooService:9006
c.l.l.r.service.FooFeignClient : [FooFeignClient#bar] Date: Tue, 10 Oct 2017 09:25:36 GMT
c.l.l.r.service.FooFeignClient : [FooFeignClient#bar] Content-Type: application/json;charset=UTF-8
c.l.l.r.service.FooFeignClient : [FooFeignClient#bar]
c.l.l.r.service.FooFeignClient : [FooFeignClient#bar] {"result":"Hello John"}
c.l.l.r.service.FooFeignClient : [FooFeignClient#bar] <--- END HTTP (23-byte body)
To turn on logging, I declare this bean:
#Bean
public feign.Logger.Level feignLoggerLevel() {
return Logger.Level.FULL;
}
We are using spring-cloud-netflix-core:1.1.0.M4
You will need to provide your own Logger to achieve this. The default Logger is very simple and logs the host before the request is sent. When using Ribbon the information will be available with the Response. If you override the logAndRebufferResponse method, the response.url property will be the entire url submitted, including host name.
protected Response logAndRebufferResponse(
String configKey,
Level logLevel,
Response response,
long elapsedTime) throws IOException {
/* ask the response for the request and log the uri */
log(response.request.url());
}
Enable logging via logback and set log level of debug or these classes
<logger name="com.netflix.loadbalancer.BaseLoadBalancer" level="DEBUG"/>
<logger name="com.netflix.loadbalancer.LoadBalancerContext" level="DEBUG"/>
<logger name="com.netflix.loadbalancer.reactive.LoadBalancerCommand" level="DEBUG"/>
I can see where each request is going (server:port) and request failed on which server
DEBUG [ c.a.m.c.f.CommonProducerClient][72] - [CommonProducerClient#findAllEmployee] ---> GET http://COMMON-PRODUCER/allemployee HTTP/1.1
DEBUG [ c.n.loadbalancer.LoadBalancerContext][492] - COMMON-PRODUCER using LB returned Server: localhost:7001 for request http:///allemployee
DEBUG [ c.n.l.reactive.LoadBalancerCommand][314] - Got error java.net.ConnectException: Connection refused: connect when executed on server localhost:7001
DEBUG [ c.n.l.reactive.LoadBalancerCommand][314] - Got error java.net.ConnectException: Connection refused: connect when executed on server localhost:7001
DEBUG [ c.n.l.reactive.LoadBalancerCommand][314] - Got error java.net.ConnectException: Connection refused: connect when executed on server localhost:7001
DEBUG [ c.n.loadbalancer.LoadBalancerContext][492] - COMMON-PRODUCER using LB returned Server: localhost:7003 for request http:///allemployee
DEBUG [ c.a.m.c.f.CommonProducerClient][72] - [CommonProducerClient#findAllEmployee] <--- HTTP/1.1 200 (4010ms)
From Logs its clear, first request went to port 7001, connection refused there, second request goes to 7003, all good there.

ElasticSearch is unable to recognize Context-Type header with encoding defined

I've spent some time trying to fix the elastic search bulk upload warning:
Content type detection for rest requests is deprecated. Specify the content type using the [Content-Type] header
My request is below:
POST http://elasticserver/_bulk HTTP/1.1
Authorization: xxx
Content-Type: application/x-ndjson; charset=utf-8
Host: elasticserver
Content-Length: 8559
... new line delimited json content ...
And my valid response with 200 status is below:
HTTP/1.1 200 OK
Warning: 299 Elasticsearch-5.5.1-19c13d0 "Content type detection for rest requests is deprecated. Specify the content type using the [Content-Type] header." "Mon, 14 Aug 2017 00:46:21 GMT"
content-type: application/json; charset=UTF-8
content-length: 4183
{"took":5538,"errors":false,...}
By experimenting I discovered that the issue is in content type charset definition Content-Type: application/x-ndjson; charset=utf-8 and if I change it to Content-Type: application/x-ndjson I get no warning.
Is it an elastic search issue or I'm forming the request incorrectly?
The official documentation explicitly states that
When sending requests to this endpoint the Content-Type header should be set to application/x-ndjson.
The RestController source code also shows that they are ignoring the charset:
final String lowercaseMediaType = restRequest.header("Content-Type").toLowerCase(Locale.ROOT);
// we also support newline delimited JSON: http://specs.okfnlabs.org/ndjson/
if (lowercaseMediaType.equals("application/x-ndjson")) {
restRequest.setXContentType(XContentType.JSON);
return true;
}

Filtering with Quickbooks Online V2 returns 401 ErrorCode=003200, Empty body posts work perfectly

We're attempting to integrate with the QuickBooks Online V2 api using Ruby 1.9.3 (not RoR).
Using the API Explorer and the Employee endpoint documentation we were able to get a simple list of test employees by using the Google Signet OAuth Gem.
require 'signet'
require 'signet/oauth_1/client'
#intialize oauth1 client
#client = Signet::OAuth1::Client.new(
:temporary_credential_uri => "https://oauth.intuit.com/oauth/v1/get_request_token",
:authorization_uri => "https://appcenter.intuit.com/Connect/Begin",
:token_credential_uri => "https://oauth.intuit.com/oauth/v1/get_access_token",
:client_credential_key => 'qyprdPEfJqU7eOze0Fby9iYhrUS5DQ',
:client_credential_secret => 'fuXsasJo4TrTEd3Yhv4TeMUizmtguh0JioIB5r2I',
:callback => "http://localhost:3000/callback/general"
)
#client.token_credential_key = 'qyprdJUtDSk7owxVfZlq7JeWO1mtpHBkSMD5GhB02PwIC6N0'
#client.token_credential_secret = 'Rq2ekgQWWL9frZAKpcgWef291mR0J5HBE354u5F3'
#setup request
original_request = [
'POST',
'https://qbo.sbfinance.intuit.com/resource/employees/v2/791630875',
# we also tried this url 'https://qbo.intuit.com/qbo28/resource/employees/v2/791630875',
[
['Content-Type', 'application/x-www-form-urlencoded'],
],
[]
]
#execute request
response = #client.fetch_protected_resource(:request => original_request)
puts response.body
As you can see the request is pretty straight-forward.
However once we create a request with a Filter in the body, we get an HTML page with the following error: HTTP Status 401 - message=Exception authenticating OAuth; errorCode=003200; statusCode=401
#setup request
original_request = [
'POST',
'https://qbo.intuit.com/qbo28/resource/employees/v2/791630875',
#'https://qbo.sbfinance.intuit.com/resource/employees/v2/791630875',
[
['Content-Type', 'application/x-www-form-urlencoded'],
],
["Filter=Name :EQUALS: Doe"]
]
We're using the Google OAuth gem, and I've verified the signature generation to be correct using these tools: LinkedIn Oauth Test Console and Beginners guide to OAuth signing requests. They both verify that the signature that Signet is generating is correct for the body I provide.
I've looked at a few SO Questions:
QuickBooks Online querying with filter returns 401 everytime
Unable to create(POST) objects (Account, customer...) on QB Windows using IDS and Sync Manager
But nothing has worked. Any help would be appreciated, we're willing to use a third party gem such as quickeebooks but we would rather not. I assume I'm just missing something simple here.
Please provide me with the following items so I can verify a working answer:
Your request parameters, including uri, header, body and exact client and access tokens (developer app tokens only please, I'll need to verify that I can generate the exact same request, including signature)
You basestring used for generating the HMAC-SHA1 signature. it will look something like
POST&https%3A%2F%2Fqbo.intuit.com%2Fqbo28%2Fresource%2Femployees%2Fv2%2F7916308‌​75&Filter%3DName%2520%253AEQUALS%253A%2520David%2520Test%26oauth_consumer_key%3Dq‌​yprdPEfJqU7eOze0Fby9iYhrUS5DQ%26oauth_nonce%3D-1787433535548338293%26oauth_signat‌​ure_method%3DHMAC-SHA1%26oauth_timestamp%3D1380089100%26oauth_token%3DqyprdJUtDSk‌​7owxVfZlq7JeWO1mtpHBkSMD5GhB02PwIC6N0%26oauth_version%3D1.0
Your response, including header and body data
I had tried to use filter query with employee endpoint. It works fine.
EDIT - Sharing endpoint, filter and resultset related to Employee API Endpoint
https://qbo.intuit.com/qbo28/resource/employees/v2/791926875
Filter= Name :EQUALS: Manas Mukherjee
header - "Authorization: OAuth oauth_token="2eRrd7LhEtHrM1CrqWvy1kmSgeukEgFxW99E1xwhSsLCp1JB", oauth_consumer_key="qyprdXsaKh0a132eNs7NTJLufjfrzm", oauth_version="1.0", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1380084612", oauth_nonce="1556081845430558974", oauth_signature="IMjh%2FTx%2F7GMFDE6WQqZK8b6apjI%3D"[\r][\n]"
Content-Type: application/x-www-form-urlencoded
Data Set
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<qbo:SearchResults xmlns="http://www.intuit.com/sb/cdm/v2" xmlns:qbp="http://www.intuit.com/sb/cdm/qbopayroll/v1" xmlns:qbo="http://www.intuit.com/sb/cdm/qbo">
<qbo:CdmCollections xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="Employees">
<Employee>
<Id idDomain="QBO">20</Id>
<SyncToken>0</SyncToken>
<MetaData>
<CreateTime>2013-09-24T21:37:22-07:00</CreateTime>
<LastUpdatedTime>2013-09-24T21:37:22-07:00</LastUpdatedTime>
</MetaData>
<Name>Manas Mukherjee</Name>
<Address>
<Line1>ABC Str</Line1>
<City>London</City>
<PostalCode>4353543</PostalCode>
<GeoCode>LAT=51.5148382,LNG=-0.1264144</GeoCode>
</Address>
<GivenName>Manas</GivenName>
<MiddleName>Kr</MiddleName>
<FamilyName>Mukherjee</FamilyName>
<ShowAs>Manas Kr Mukherjee</ShowAs>
<BillableTime>false</BillableTime>
</Employee>
</qbo:CdmCollections>
<qbo:Count>1</qbo:Count>
<qbo:CurrentPage>1</qbo:CurrentPage>
</qbo:SearchResults>
OAuth header using your tokens
"Authorization: OAuth oauth_token="qyprdJUtDSk7owxVfZlq7JeWO1mtpHBkSMD5GhB02PwIC6N0", oauth_consumer_key="qyprdPEfJqU7eOze0Fby9iYhrUS5DQ", oauth_version="1.0", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1380089100", oauth_nonce="-1787433535548338293", oauth_signature="Vj67xMVhSKGjVSmGyOxt7SVv0i8%3D"[\r][\n]"
Endpoint - https://qbo.intuit.com/qbo28/resource/employees/v2/791630875
Post data to end point: Filter= Name :EQUALS: David Test
Content-Type: application/x-www-form-urlencoded
It works fine
Thanks
See this sample fiddler request with Filter for items in QBO. I cannot paste the fiddler log here. You can do it for similarly for Employee. The filters should go into the body and encode your header:
Request-
POST https://qbo.intuit.com/qbo1/resource/items/v2/723488155
HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Authorization: OAuth oauth_token="lvprdgF9q4mSQx5A6lKNm3NISXvwIpF16z",oauth_nonce="3740352e-20a4-4d45-af4f-2b783ee20e60",oauth_consumer_key="qyprd7I5WvVnPoiBh1ejZn",oauth_signature_method="HMAC-SHA1",oauth_timestamp="1377106651",oauth_version="1.0",oauth_signature="1OAJXk5uH0sEpYpdhh%2BDMzjQFEs%3D"
Host: qbo.intuit.com
Content-Length: 28
Expect: 100-continue
PageNum=1&ResultsPerPage=100
Response Header-
HTTP/1.1 200 OK
Date: Wed, 21 Aug 2013 17:37:31 GMT
Server: Apache
Set-Cookie: qboeuid=10.129.32.5.1377106651774076; path=/; expires=Thu, 21-Aug-14 17:37:31 GMT; domain=.intuit.com
Set-Cookie: JSESSIONID=82DE11473B5246497B9FDCD8A6DA4C45.c1-pprdqboas30j; Path=/; Secure; HttpOnly
Vary: Accept-Encoding
Content-Type: application/xml;charset=UTF-8
Content-Length: 32525

Tyrus websocket server handshake issue?

I followed the User Guide available here : i added this in my pom :
<dependency>
<groupId>org.glassfish.tyrus</groupId>
<artifactId>tyrus-server</artifactId>
<version>1.2</version>
</dependency>
<dependency>
<groupId>org.glassfish.tyrus</groupId>
<artifactId>tyrus-container-grizzly</artifactId>
<version>1.2</version>
</dependency>
I wrote this in my main class :
Server server = new Server("localhost", 8624, "/", EchoEndPoint.class);
try
{
server.start();
BufferedReader reader = new BufferedReader(new InputStreamReader(System.in));
System.out.print("Please press a key to stop the server.");
reader.readLine();
}
catch(Exception ex) { ex.printStackTrace(); }
finally
{
server.stop();
}
The content of my EchoEndPoint class is the same as described in the guide.
I tried to connect to this with a HTML5 websocket :
var ws = new WebSocket("ws://localhost:8624/echo");
It seems that, browser side, it doesn't connect (it calls the onClose callback directly). And, server side, i get this in the console :
Grave: Invalid Connection header returned: 'keep-alive'
org.glassfish.tyrus.websockets.HandshakeException: Invalid Connection header returned: 'keep-alive'
at org.glassfish.tyrus.websockets.HandShake.validate(HandShake.java:254)
at org.glassfish.tyrus.websockets.HandShake.checkForHeader(HandShake.java:246)
at org.glassfish.tyrus.websockets.HandShake.<init>(HandShake.java:97)
at org.glassfish.tyrus.websockets.draft06.HandShake06.<init>(HandShake06.java:63)
[...]
org.glassfish.grizzly.filterchain.DefaultFilterChain execute
Avertissement: Exception during FilterChain execution
java.lang.ClassCastException: org.glassfish.grizzly.http.HttpContent cannot be cast to org.glassfish.tyrus.websockets.DataFrame
at org.glassfish.tyrus.container.grizzly.WebSocketFilter.handleWrite(WebSocketFilter.java:330)
If it's of any help, i copy the request header caught with the browser inspector :
GET /echo HTTP/1.1
Host: localhost:8624
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0 FirePHP/0.7.2
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
DNT: 1
Sec-WebSocket-Version: 13
Origin: null
Sec-WebSocket-Key: yhGPwJ26c5fYEZ5/abvtqw==
x-insight: activate
Connection: keep-alive, Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
Is this a handshake problem ?
EDIT : i've tried in Chrome (28.0.1500.72) and it's working. Maybe the issue comes from Firefox when it builds the header ?
Tyrus is complaining about the Connection: keep-alive, Upgrade header.
Firefox isn't doing anything wrong here.
Tyrus is being too restrictive and not following the WebSocket Spec (RFC-6455) with regards to how to handle the Connection header.
The RFC states in Section 4.1:
6. The request MUST contain a |Connection| header field whose value
MUST include the "Upgrade" token.
and
3. If the response lacks a |Connection| header field or the
|Connection| header field doesn't contain a token that is an
ASCII case-insensitive match for the value "Upgrade", the client
MUST _Fail the WebSocket Connection_.
This seems like a bug in Tyrus.

Resources