I am using spring boot with jersey implementation in that we are logging every request and response. Now it is logging everything but I need to mask the password in the request. in the configuration file, we are using this
register(new LoggingFeature(java.util.logging.Logger.getLogger(LoggingFeature.DEFAULT_LOGGER_NAME),
Level.INFO, LoggingFeature.Verbosity.PAYLOAD_ANY, 10000));**
I cannot able to mask the specific field from the POST request. Please help
SAMPLE LOGS:
2 > POST http://localhost:9092/sampleapi/login
2 > accept: */*
2 > accept-encoding: gzip, deflate, br
2 > connection: keep-alive
2 > content-length: 69
2 > content-type: application/json
2 > host: localhost:9092
2 > postman-token: 24c72655-6f97-4e19-9503-6f013c859e5f
2 > user-agent: PostmanRuntime/7.26.8
{
"userId": "1111",
"password": "TEST#1",
"yob": "1997"
}
In this, I need to mask the password field alone
Related
I went through the other similar questions but couldn't find any solution. We have a backend service running by Spring Boot and has been working for a while now. But there is a new user of this service recently and they are using MuleSoft to send their request. But all attempts to send a file to our service fails with this error:
Failed to parse multipart servlet request; nested exception is java.io.IOException: org.apache.tomcat.util.http.fileupload.FileUploadException: the request was rejected because no multipart boundary was found"
The only difference we could find between a request from MuleSoft and say a curl command is that MuleSoft always sends the request with boundary value wrapped with double quotes
Mule request:
<Header name="Content-Type">multipart/form-data; charset=UTF-8; boundary="--------------------------669816398596264398718285"</Header>
Versus Postman/curl request:
* SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fd656810a00)
> POST /api/upload HTTP/2
> Host: myhost
> user-agent: curl/7.79.1
> accept: */*
> content-length: 97255
> content-type: multipart/form-data; boundary=------------------------111fd08cb4fafe1c
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
* We are completely uploaded and fine
< HTTP/2 200
< date: Mon, 19 Dec 2022 04:56:25 GMT
< content-length: 0
Our controller in Spring is very simple:
#RestController
class MyController {
#PostMapping("/upload", consumes = [MediaType.MULTIPART_FORM_DATA_VALUE])
#ResponseStatus(HttpStatus.OK)
fun uploadDocument(#RequestPart("file") file: MultipartFile) {
logger.info { "ContentType: ${file.contentType}" }
logger.info { "Name: ${file.name}" }
logger.info { "Byte: ${String(file.bytes)}" }
}
}
The following curl command works fine:
curl -v -X POST -F file=#/Users/myhomefolder/Documents/some-file.jpg https://host-name/api/upload
But this script from MuleSoft doesn't (Sorry I'm not familiar with Mule, I got this code from their team):
import dw::module::Multipart
output multipart/form-data boundary = "---WebKitFormBoundary7MA4YWxkTrZu0gW"
---
{
parts : {
file : {
headers : {
"Content-Disposition" : {
"name": "file",
"filename": payload.name
},
"Content-Type" : "multipart/form-data"
},
content : payload.byteArray
}
}
}
Is there any configuration in Spring that accepts double quotes for boundary? Is there anything missing in our backend configuration that should be added to support different HTTP client?
Environment details
OS: macOS Big Sur Version 11.6 (Apple M1 Chip)
Node.js version: v16.4.1
npm version: 7.23.0
#google-cloud/talent version: v4
Intruduction
I'm having a hard time to get the job search in Google Cloud Talent Solution to work.
I already can create/read/update, tenants/companies/jobs indicating that the credentials are ok.
But I don't find any jobs searching them.
Facts
Currently I have one job stored in Google Cloud Talent Solution.
This ist the job export, done with the Google Console:
{
"name":"projects/insurancepunk/tenants/75f8ac52-6e7c-4b00-9220-03771d25e9c5/jobs/135317048994472646",
"requisition_id":"f9bffe6e-3c8c-40d5-b3c3-672d30485745",
"title":"IT-Berater"
}
This is the JSON stringifyed request, passed to "searchJobs":
{
"parent":"projects/insurancepunk/tenants/75f8ac52-6e7c-4b00-9220-03771d25e9c5",
"searchMode":"JOB_SEARCH",
"requestMetadata":{"domain":"insurancepunk.com","sessionId":"8f47bbab-5c15-4bd9-9008-60f79030ab3b","userId":"vCobKcXPFdf6zlVibjnb"},
"jobQuery":{"query":"IT-Berater"}
}
As you can see the project id and the tenant id match the exported job.
This is my very simple code:
const talent = require('#google-cloud/talent').v4;
const client = new talent.JobServiceClient();
client.searchJobs(request)
.then(responses => {
const resources = responses[0];
for (const resource of resources) {
console.log(`Job summary: ${resource.jobSummary}`);
console.log(`Job title snippet: ${resource.jobTitleSnippet}`);
const job = resource.job;
console.log(`Job name: ${job.name}`);
console.log(`Job title: ${job.title}`);
}
})
.catch(err => {
console.error(err);
});
The code enters the then-path, but "responses" is empty.
Google OAuth Play Ground
When testing it in Google OAuth Play Ground, i get this results.
Googel OAuth Play Ground: https://developers.google.com/oauthplayground/
Google Talent Solution Scope: [https://www.googleapis.com/auth/jobs2
The Scope was found here:
https://cloud.google.com/talent-solution/job-search/docs/reference/rpc/google.cloud.talent.v4
Output from Google OAuth Play Ground:
Request:
POST /v4/projects/insurancepunk/tenants/75f8ac52-6e7c-4b00-9220-03771d25e9c5/jobs:search HTTP/1.1
Host: jobs.googleapis.com
Content-length: 277
Content-type: application/json
Authorization: Bearer ya29.a0ARrda...
{
"parent":"projects/insurancepunk/tenants/75f8ac52-6e7c-4b00-9220-03771d25e9c5",
"searchMode":"JOB_SEARCH",
"requestMetadata":{"domain":"insurancepunk.com","sessionId":"8f47bbab-5c15-4bd9-9008-60f79030ab3b","userId":"vCobKcXPFdf6zlVibjnb"},
"jobQuery":{"query":"IT-Berater"}
}
Response:
HTTP/1.1 200 OK
Content-length: 117
X-xss-protection: 0
X-content-type-options: nosniff
Transfer-encoding: chunked
Vary: Origin, X-Origin, Referer
Server: ESF
-content-encoding: gzip
Cache-control: private
Date: Tue, 30 Nov 2021 09:44:40 GMT
X-frame-options: SAMEORIGIN
Alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Content-type: application/json; charset=UTF-8
{
"metadata": {
"requestId": "0e7330e1-ee15-404e-85d2-b8174679583f:APAb7ITvUURH6nrwLrbLLBCB9Zg6NKPMfg=="
}
}
Listing companies works well!
Request:
GET /v4/projects/insurancepunk/tenants/75f8ac52-6e7c-4b00-9220-03771d25e9c5/companies HTTP/1.1
Host: jobs.googleapis.com
Content-length: 0
Authorization: Bearer ya2...
Response:
HTTP/1.1 200 OK
Content-length: 1124
X-xss-protection: 0
Content-location: https://jobs.googleapis.com/v4/projects/insurancepunk/tenants/75f8ac52-6e7c-4b00-9220-03771d25e9c5/companies
X-content-type-options: nosniff
Transfer-encoding: chunked
Vary: Origin, X-Origin, Referer
Server: ESF
-content-encoding: gzip
Cache-control: private
Date: Tue, 30 Nov 2021 10:08:49 GMT
X-frame-options: SAMEORIGIN
Alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Content-type: application/json; charset=UTF-8
{
"companies": [
{
"displayName": "Pompadour GmbH",
"name": "projects/insurancepunk/tenants/75f8ac52-6e7c-4b00-9220-03771d25e9c5/companies/a6f34dd2-76f1-40e8-8175-1274f49f5977",
"headquartersAddress": "Am Burgweg 1, 97346 Iphofen",
"imageUri": "http://www.pompadour.info/bar",
"derivedInfo": {
"headquartersLocation": {
"locationType": "STREET_ADDRESS",
"postalAddress": {
"postalCode": "97346",
"regionCode": "DE",
"administrativeArea": "BY",
"addressLines": [
"Am Burgweg 1, 97346 Iphofen, Germany"
],
"locality": "Iphofen"
},
"radiusMiles": 6.892640659556388e-05,
"latLng": {
"latitude": 49.7102381,
"longitude": 10.254041
}
}
},
"externalId": "9a1ebd16-886c-40ac-ae0a-d5a4e288f867",
"websiteUri": "http://www.pompadour.info",
"hiringAgency": true
}
],
"metadata": {
"requestId": "5780a724-ca05-4f56-8a88-d74fdc04a24e:APAb7IS/J4Hs1KThU2G0nCZk5fOdBT3sJw=="
}
}
All hints are welcome.
Wow!
This don't quite smell like Artificial Intelligence...
My one and only job had the Title "IT-Berater".
Searching for "IT-Berater" returned an empty result set.
However, searching for "Berater" returned the job...
The results are equal no matter if I used the Node.js API ore the original Google HTTP-API...
I've got a problem using gatling tool when trying to send http/2 request.
I've enable http/2 setting in protocol settings, even added mapping to make sure that client will communicate with server using http2 but still request is send using http/1.1.
In console output you'll see that server can communicate using http2 but for some reason request are sent only with http 1.1.
All headers are written according to the ones from browser.
Could someone help me please with this issue?
package test
import scala.concurrent.duration._
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import io.gatling.jdbc.Predef._
class RecordedSimulation extends Simulation {
val httpProtocol = http
.baseUrl("https://sitename")
.inferHtmlResources(BlackList(""".*\.js""", """.*\.css""", """.*\.gif""", """.*\.jpeg""", """.*\.jpg""", """.*\.ico""", """.*\.woff""", """.*\.woff2""", """.*\.(t|o)tf""", """.*\.png""", """.*detectportal\.firefox\.com.*"""), WhiteList())
.acceptHeader("*/*")
.acceptEncodingHeader("gzip, deflate")
.acceptLanguageHeader("en-US,en;q=0.5")
.userAgentHeader("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:85.0) Gecko/20100101 Firefox/85.0")
.enableHttp2
.http2PriorKnowledge(Map("sitename" -> true))
.disableCaching
.disableWarmUp
val headers_0 = Map(
"Accept" -> "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
"Cache-Control" -> "max-age=0",
"sec-fetch-dest" -> "document",
"sec-fetch-mode" -> "navigate",
"sec-fetch-site" -> "none",
"sec-fetch-user" -> "?1",
"Upgrade-Insecure-Requests" -> "1")
val scn = scenario("RecordedSimulation")
.exec(http("request_0")
.get("/auth?r=%2F&m=NOT_AUTHENTICATED")
.headers(headers_0))
.pause(2)
setUp(scn.inject(atOnceUsers(1))).protocols(httpProtocol)
}
In console I see next:
DEBUG io.gatling.http.client.impl.DefaultHttpClient - ALPN led to HTTP/2 with remote sitename
DEBUG io.gatling.http.client.impl.Http2AppHandler - Write request WritableRequest{request=DefaultFullHttpRequest(decodeResult: success, version: HTTP/1.1, content: UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeHeapByteBuf(ridx: 0, widx: 0, cap: 0))
GET https://sitename/auth?r=%2F&m=NOT_AUTHENTICATED HTTP/1.1
sec-fetch-site: none
Upgrade-Insecure-Requests: 1
If-Modified-Since: Fri, 29 Jan 2021 06:40:06 GMT
sec-fetch-dest: document
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Cache-Control: max-age=0
accept-encoding: gzip, deflate
If-None-Match: "1d6f60994776ee0"
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:85.0) Gecko/20100101 Firefox/85.0
accept-language: en-US,en;q=0.5
sec-fetch-mode: navigate
sec-fetch-user: ?1
accept: */*
Those are internal Gatling logs. We use Netty in a way that it converts HTTP/1.1 payloads into HTTP/2 ones.
From the logs you provided, it looks that Gatling is correctly using HTTP/2 here.
I've spent some time trying to fix the elastic search bulk upload warning:
Content type detection for rest requests is deprecated. Specify the content type using the [Content-Type] header
My request is below:
POST http://elasticserver/_bulk HTTP/1.1
Authorization: xxx
Content-Type: application/x-ndjson; charset=utf-8
Host: elasticserver
Content-Length: 8559
... new line delimited json content ...
And my valid response with 200 status is below:
HTTP/1.1 200 OK
Warning: 299 Elasticsearch-5.5.1-19c13d0 "Content type detection for rest requests is deprecated. Specify the content type using the [Content-Type] header." "Mon, 14 Aug 2017 00:46:21 GMT"
content-type: application/json; charset=UTF-8
content-length: 4183
{"took":5538,"errors":false,...}
By experimenting I discovered that the issue is in content type charset definition Content-Type: application/x-ndjson; charset=utf-8 and if I change it to Content-Type: application/x-ndjson I get no warning.
Is it an elastic search issue or I'm forming the request incorrectly?
The official documentation explicitly states that
When sending requests to this endpoint the Content-Type header should be set to application/x-ndjson.
The RestController source code also shows that they are ignoring the charset:
final String lowercaseMediaType = restRequest.header("Content-Type").toLowerCase(Locale.ROOT);
// we also support newline delimited JSON: http://specs.okfnlabs.org/ndjson/
if (lowercaseMediaType.equals("application/x-ndjson")) {
restRequest.setXContentType(XContentType.JSON);
return true;
}
We perform PUT request to our party using CXF JAX-RS client. Request body is empty.
A simple request invocation leads to server response with code 411.
Response-Code: 411
"Content-Length is missing"
Our party's REST-server requires Content-Length HTTP-header to be set.
We switched chunking off according to note about chunking but this did not solve the problem. The REST-server still answers with 411 error.
Here is our conduit configuration from cxf.xml file
<http-conf:conduit name="{http://myhost.com/ChangePassword}WebClient.http-conduit">
<http-conf:client AllowChunking="false"/>
</http-conf:conduit>
Line in the log confirms that execution of our request bound to our conduit configuration:
DEBUG o.a.cxf.transport.http.HTTPConduit - Conduit '{http://myhost.com/ChangePassword}WebClient.http-conduit' has been configured for plain http.
Adding Content-Length header explicitly also did not help.
Invocation.Builder builder = ...
builder = builder.header(HttpHeaders.CONTENT_LENGTH, 0);
A CXF Client's log entry confirms header setting, however when we sniffed packets, we have surprisingly found that header setting has been completely ignored by CXF client. Content-Length header was not sent.
Here is the log. Content-Length header is present:
INFO o.a.c.i.LoggingOutInterceptor - Outbound Message
---------------------------
ID: 1
Address: http://myhost.com/ChangePassword?username=abc%40gmail.com&oldPassword=qwerty123&newPassword=321ytrewq
Http-Method: PUT
Content-Type: application/x-www-form-urlencoded
Headers: {Accept=[application/json], client_id=[abcdefg1234567890abcdefg12345678], Content-Length=[0], Content-Type=[application/x-www-form-urlencoded], Cache-Control=[no-cache], Connection=[Keep-Alive]}
--------------------------------------
DEBUG o.apache.cxf.transport.http.Headers - Accept: application/json
DEBUG o.apache.cxf.transport.http.Headers - client_id: abcdefg1234567890abcdefg12345678
DEBUG o.apache.cxf.transport.http.Headers - Content-Length: 0
DEBUG o.apache.cxf.transport.http.Headers - Content-Type: application/x-www-form-urlencoded
DEBUG o.apache.cxf.transport.http.Headers - Cache-Control: no-cache
DEBUG o.apache.cxf.transport.http.Headers - Connection: Keep-Alive
And here is an output of the packet sniffer. Content-Length header is not present:
PUT http://myhost.com/ChangePassword?username=abc%40gmail.com&oldPassword=qwerty123&newPassword=321ytrewq HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Accept: application/json
client_id: abcdefg1234567890abcdefg12345678
Cache-Control: no-cache
User-Agent: Apache-CXF/3.1.8
Pragma: no-cache
Host: myhost.com
Proxy-Connection: keep-alive
Does anyone know how actually disable chunking?
Here is our code:
public static void main(String[] args)
{
String clientId = "abcdefg1234567890abcdefg12345678";
String uri = "http://myhost.com";
String user = "abc#gmail.com";
Client client = ClientBuilder.newBuilder().newClient();
WebTarget target = client.target(uri);
target = target.path("ChangePassword").queryParam("username", user).queryParam("oldPassword", "qwerty123").queryParam("newPassword", "321ytrewq");
Invocation.Builder builder = target.request("application/json").header("client_id", clientId).header(HttpHeaders.CONTENT_LENGTH, 0);
Response response = builder.put(Entity.form(new Form()));
String body = response.readEntity(String.class);
System.out.println(body);
}
Versions:
OS: Windows 7 Enterprise SP1
Arch: x86_64
Java: 1.7.0_80
CXF: 3.1.8
I had a very similar issue that I was not able to solve as you did by trying to turn off chunking.
What I ended up doing was setting the Content-Length to 1 and adding some white space " " as the body. For me it seemed that the proxy servers before the server application was rejected the request and by doing that got me past the proxy servers and the server was able to process the request as it was only operating based on the URL.