Log Api Gateway request/response body/headers to ElasticSearch - aws-lambda

It's easy to setup it by default, but i cannot change the format of messages. It sends to CloudWatch
Api Gateway works as proxy
2019-03-15 13:31:22.248 (REQ_ID) Method completed with status: 200
2019-03-15 13:31:22.248 (REQ_ID) Successfully completed execution
2019-03-15 13:31:22.248 (REQ_ID) Method response headers:
{Cache-Control=max-age=0, private, must-revalidate,
Content-Type=application/json; charset=utf-8, Date=Fri, 15 Mar 2019
13:31:22 GMT, ETag=W/"607046c7bacdeee5e27e99f48fa2e053",
Server=nginx/1.12.1 + Phusion Passenger 4.0.60, Status=200 OK,
Vary=Accept-Encoding, Origin, X-Content-Type-Options=nosniff,
X-Frame-Options=SAMEORIGIN, X-Powered-By=Phusion Passenger 4.0.60,
X-Request-Id=88c8f3c2-0a25-4445-9010-30d4753b855a, X-Runtime=0.030736,
X-XSS-Protection=1; mode=block, Connection=keep-alive}
2019-03-15 13:31:22.248 (REQ_ID) Method response body after
transformations: {"success":1,"message":"Token is invalid"}
2019-03-15 13:31:22.248 (REQ_ID) Endpoint response headers:
{Cache-Control=max-age=0, private, must-revalidate,
Content-Type=application/json; charset=utf-8, Date=Fri, 15 Mar 2019
13:31:22 GMT, ETag=W/"607046c7bacdeee5e27e99f48fa2e053",
Server=nginx/1.12.1 + Phusion Passenger 4.0.60, Status=200 OK,
Vary=Accept-Encoding, Origin, X-Content-Type-Options=nosniff,
X-Frame-Options=SAMEORIGIN, X-Powered-By=Phusion Passenger 4.0.60,
X-Request-Id=88c8f3c2-0a25-4445-9010-30d4753b855a, X-Runtime=0.030736,
X-XSS-Protection=1; mode=block, transfer-encoding=chunked,
Connection=keep-alive}
2019-03-15 13:31:22.248 (REQ_ID) Endpoint response body before
transformations: {"success":1,"message":"Token is invalid"}
2019-03-15 13:31:22.248 (REQ_ID) Received response. Integration
latency: 43 ms
2019-03-15 13:31:22.205 (REQ_ID) Sending request to
https://proxy.domain.com/api/endpoint
2019-03-15 13:31:22.205 (REQ_ID) Endpoint request body after
transformations: {"message":"hello world"}
2019-03-15 13:31:22.205 (REQ_ID) Endpoint request headers:
{x-amzn-apigateway-api-id=API, User-Agent=curl/7.54.0,
X-Forwarded-Proto=https, X-Forwarded-For=195.24.159.254,
content-type=application/json, X-Forwarded-Port=443,
X-Amzn-Trace-Id=Root=1-5c8ba92a-7fb7fa94b9991bb8f1561d55, accept=/}
2019-03-15 13:31:22.205 (REQ_ID) Endpoint request URI:
https://proxy.domain.com/api/endpoint
2019-03-15 13:31:22.119 (REQ_ID) Method request body before
transformations: {"message":"hello world"}
2019-03-15 13:31:22.119 (9f76246a-4726-11e9-a18d-25bda2ff1f7b) Method
request headers: {User-Agent=curl/7.54.0, X-Forwarded-Proto=https,
X-Forwarded-For=195.24.159.254, content-type=application/json,
Host=api.execute-api.eu-west-1.amazonaws.com, X-Forwarded-Port=443,
X-Amzn-Trace-Id=Root=1-5c8ba92a-7fb7fa94b9991bb8f1561d55, accept=/}
2019-03-15 13:31:22.119 (REQ_ID) Method request query string: {}
2019-03-15 13:31:22.119 (REQ_ID) Method request path:
{proxy=api/endpoint}
I need to send in JSON these 4 columns only: request_header, request_body, response_header, response_body to ES. Now it sends all the logs above for each request.

Related

When calling Google Admin Directory API to write recovery email, I get response status code 400 BAD REQUEST

Here is my code:
Here is the response:
2020/09/14 12:19:51.425: INFO Results for adding ppsEmailAddress as recovery email: {data=[object Object], finalURL=https://www.googleapis.com/admin/directory/v1/users/ttestaccount#pps.net, headers={Alt-Svc=h3-29=":443"; ma=2592000,h3-27=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-T050=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43", Content-Length=357, Content-Type=application/json; charset=UTF-8, Date=Mon, 14 Sep 2020 19:19:51 GMT, Server=ESF, Vary=[Origin, X-Origin, Referer], X-Content-Type-Options=nosniff, X-Frame-Options=SAMEORIGIN, X-XSS-Protection=0}, initialURL=https://www.googleapis.com/admin/directory/v1/users/ttestaccount#pps.net, statusCode=400.0, statusReason=Bad Request}
Full code:
Trying to follow the Google API reference here, though something wrong with my syntax:
https://developers.google.com/admin-sdk/directory/v1/reference/users#resource
I made a similar request using the Google OAuth Playground
This was the format of my request:
PATCH /admin/directory/v1/users/youremail#joshgoldeneagle.com HTTP/1.1
Host: www.googleapis.com
Content-length: 45
Content-type: application/json
Authorization: Bearer {REPLACE WITH ACCESS TOKEN}
{
recoveryEmail: "yourPatchEmail#joshgoldeneagle.com"
}
Ensure that callGoogleAPI() makes a request in this format.
References:
https://developers.google.com/admin-sdk/directory/v1/reference/users/patch

Which IIS config causes the "405 Method not allowed" error for specific calls?

In our current WebAPI calls, only a few specific ones are returning the 405 error message (405 Method not allowed), the rest are working okay. The ones that are failing are only failing in a specific environment, in all the other environments all calls are returning 200 and everything is kosher.
Which IIS setting or system setting can cause this ? I am zooming in on this because this looks more like a systems/config issue.
Request URL : <hostname/GetAllStuff?userId=johndoe&sortByCustom=CreatedDate&sortByOrder=desc
Request Method: GET
Status Code: 405 Method Not Allowed
Remote Address: 172.72.78.33:443
Referrer Policy: no-referrer-when-downgrade
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Methods: GET,POST,PUT,DELETE,OPTIONS
Access-Control-Allow-Origin: *
Allow: GET,POST,DELETE,PUT
Cache-Control: no-cache
Content-Length: 136
Content-Type: application/json; charset=utf-8
Date: Tue, 06 Nov 2018 17:16:28 GMT
Expires: -1
The Response looks like this. What is strange is that ALL the API calls are 1.0 (api-version: 1.0) yet only a couple of calls throw this error and that too in a specific environment.
{
"Error": {
"Code": "UnsupportedApiVersion",
"Message": "The requested resource with API version '1.0' does not support HTTP method 'GET'."
}
}

JMeter recorded sample failing on rerun

I recorded my project website workflow on Jmeter. But On rerunning the recorded samples, all the samples having POST request are failing. I am unable to figure out why the same samples which were passing during recording, are failing on re-run.
Below is request of one of the samples which is failing:
Recorded sample request which passed:
POST https://example.com/live/v1/dashboards/promo_pa/ds/promo_program_dimensions
POST data:
{"query":"promo_program_dimensions.filterby(program_master_id = 'GOGGLE').filterby(dimension_name = 'channel').groupby(dimension_value).aggregate(count(dimension_value) as count).orderby(dimension_value+)"}
[no cookies]
Request Headers:
Connection: keep-alive
Referer: https://example.com/live/v1/dashboards/promo_pa/
Accept-Language: en-US,en;q=0.5
DNT: 1
Accept: */*
X-Requested-With: XMLHttpRequest
Content-Type: application/json
Accept-Encoding: gzip, deflate, br
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0
Content-Length: 216
Host: example.com
Sampler Result:
Thread Name:
Sample Start: 2018-02-22 17:32:44 IST
Load time: 93
Connect Time: 58
Latency: 93
Size in bytes: 647
Sent bytes:1095
Headers size in bytes: 645
Body size in bytes: 2
Sample Count: 1
Error Count: 0
Data type ("text"|"bin"|""): text
Response code: 200
Response message: OK
Response headers:
HTTP/1.1 200 OK
Server: nginx
Date: Thu, 22 Feb 2018 12:02:44 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 2
Connection: keep-alive
Cache-Control: no-cache, no-store, must-revalidate, public, max-age=0
Set-Cookie: live=.eJxNj11vgjAYhf_K0muzMLJdSOIFC6yBQJmuiO1Nw0cdtB0zUEAw_nfRZcbLN2_Oc85zAoGNHA9B9mlDF1gn8JQBCwTm8VD8xH0h3J7CjymFfk92vszHZZklxzpdr1bgvABBBKHrMA-x-MvdzEFVZbzRI1gAtm94WwJLNx2fr6q4wymWr9RRiuJ8iJJtRRLyQsytouJdULwREUQKJfGAkvBWkrfNnulfyes7IhTxQCGqiLkeIqeQaAqn0EElxa5B8feEoK-IkFesDLFnUMd-C4e_zVXBa13p8TntdMn0eODAqjulHj4PY8lua6RwOV6TXcubm8is2XP273q-AI7Dam4.DXBC5A.CJMLbN0B4HC4U8703ZQS50K00lk; HttpOnly; Path=/live
HTTPSampleResult fields:
ContentType: text/html; charset=utf-8
DataEncoding: utf-8
Request of the same sample which is failing on re-run:
POST https://example.com/live/v1/dashboards/promo_pa/ds/promo_program_dimensions
POST data:
{"query":"promo_program_dimensions.filterby(program_master_id = 'GOGGLE').filterby(dimension_name = 'channel').groupby(dimension_value).aggregate(count(dimension_value) as count).orderby(dimension_value+)"}
Cookie Data:
live=.eJxNj11vgjAYhf_K0muzuE4uJPHCBNZgaB0O1Pam4aOu5WsGCqMY__uYy4yXJ2_O857nAvw1cTyC-PsaucC-gKcE2MCHwzmroj7L3Z6htzFGm54eN0VqljI5DHUcrFbgOgP-FiHX4R7h0Ye7m4qlSkSjDZgBfmpEK4Gtm05MSWV3OK5YgR3XohXJcR4tyCgVzvdTSi0WUoshOt86HiRhcHuSts2J669C1HcEgdhgJ10wx3shI4a02uUYRiMNpSIwGFguCxyuzYR6ZaEs8SH4xp9_m1Umaq20eY47Lbk2ZwHsuivLh8vDWHrcz2O0NL_NrhXNTWTS7AX_d73-AAjCaeE.DXBNjA.Fy0Fs7zpVKg-f1qSbAljATOn64E
Request Headers:
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 216
Host: example.com
User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_161)
Sampler Result:
Thread Name: Liberty 1-1
Sample Start: 2018-02-22 18:24:46 IST
Load time: 26
Connect Time: 0
Latency: 26
Size in bytes: 709
Sent bytes:868
Headers size in bytes: 652
Body size in bytes: 57
Sample Count: 1
Error Count: 1
Data type ("text"|"bin"|""): text
Response code: 400
Response message: BAD REQUEST
Response headers:
HTTP/1.1 400 BAD REQUEST
Server: nginx
Date: Thu, 22 Feb 2018 12:54:46 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 57
Connection: keep-alive
Cache-Control: no-cache, no-store, must-revalidate, public, max-age=0
Set-Cookie: live=.eJxNj1tvgjAAhf_K0mezGAcPkvjg0q7B0DKciO1Lw6WGcpuBwizG_z50mfHx5ORcvgvw1hS6FIvPNUbAuYCXBDjAW5xPWR0OWYEGjj_GGG8GdtiUqVnmSXRu4mC1AtcZ8HyMERQuFeEX2k7BSiWy1QbMgDi2ssuBo9teTkplj3JS85JAZLOaFqQILTrmihT7SaU23zGbYzb3obugu-A-knbtUejvUjaPCh9Wio7vFY22ikBm0SKrCeSlDwOL79I3OpajjwOLRTRnUWh4jSz28_dZZbLRSpvXuNe50OYkgdP0VfXkPJ1lh_08xktzS_adbO8gE-YgxT_r9Rcq0WoP.DXBPFg.4Rll1tgU0GdWHWyE73uLkKM8z9c; HttpOnly; Path=/live
HTTPSampleResult fields:
ContentType: text/html; charset=utf-8
DataEncoding: utf-8
Response Data:
{"error": "Failed to authenticate user for data access."}
I have a login sample before this which passed successfully, So user has logged-in successfully before this sample.
I have used
HTTP Cache Manager, HTTP Cookie Manager and HTTP Header Manager. HTTP Cookie Manager will manage the session automatically. Still i am getting authentication error.
i have used the default settings for the sample i.e. "Redirect Automatically" and "Use KeepAlive". I am using JMeter 3.3
Appreciate any pointers.
In the "successful" request you have the following HTTP Headers:
X-Requested-With: XMLHttpRequest
Content-Type: application/json
In the "failing" one you have:
Content-Type: application/x-www-form-urlencoded
So my expectation is that you need to add HTTP Header Manager as a child of the "failing" request and configure it to send Content-Type header with the value of application/json

Applozic Platform Chat API - Uploading message attachments

I'm integrating with applozic for a client, and I need to send messages with attachments to users. I'm following the steps here: https://docs.applozic.com/docs/1-1-user-chat-and-group-chat-api#section-send-message-with-attachment on how to do this.
I'm having trouble with step 2:
Step 2. Call Url With multipart :
Call API with your file object attached to files[] array:
the requests I send are rejected with a 405 error, for example....
Request:
POST /_ah/upload/AMmfu6ZQrGP3Szfk1GuQAb_2a3J7PPWhQoiRbTnEjLp2MIzpuoeHrYryXhlzI6NW9JikjpJbT-HEtHAIk3og-Gl5EesCzBASipgtq1Hvh-PN90sjvasjRBvtO5XIFWi08gGfqTYUNT0C/ALBNUaYAAAAAWocIx4JPtA2a7LU00w1_pRui2Q3NjLR5/
application-key: XXXX
authorization: Basic XXXXX
cache-control: no-cache
accept: */*
host: applozic.appspot.com
accept-encoding: gzip, deflate
content-type: multipart/form-data; boundary=--------------------------523557777486909202804628
content-length: 286288
--------------------------523557777486909202804628
Content-Disposition: form-data; name="file"; filename="attachment.pptx"
Content-Type: application/vnd.openxmlformats-officedocument.presentationml.presentation
....file data....
Response:
HTTP/1.1 405
status: 405
x-guploader-uploadid: AEnB2UpLhLC9VKz0ysfP-WcNTgGCFc_67dVEp_-ANZsLTvWfEOFgyMWKKvpehGa3I6E9Q_s8S7LQAcYFlTt-J8LwVqRosha6lNros6eECUP5JdJ_RsZMW9g
access-control-allow-origin: *
access-control-allow-methods: POST, GET, OPTIONS, DELETE
access-control-max-age: 3600
access-control-allow-headers: UserId-Enabled, Authorization, Application-Key, Source, Content-Type
allow: GET
x-cloud-trace-context: 728352eed99001ff946db65f68daf518;o=1
x-appengine-estimated-cpm-us-dollars: $0.000026
x-appengine-resource-usage: ms=93 cpu_ms=605
date: Fri, 16 Feb 2018 16:29:37 GMT
pragma: no-cache
expires: Fri, 01 Jan 1990 00:00:00 GMT
cache-control: no-cache, must-revalidate
content-length: 0
server: UploadServer
content-type: text/html; charset=UTF-8
alt-svc: hq=":443"; ma=2592000; quic=51303431; quic=51303339; quic=51303338; quic=51303337; quic=51303335,quic=":443"; ma=2592000; v="41,39,38,37,35"
Could someone tell me what i'm missing here? It's responding with allow: GET which doesn't make sense, am I failing auth somehow?
Is there any more documentation available on this feature?
Step 1. Get Url to Upload File
Call API:
https://applozic.appspot.com/rest/ws/aws/file/url?data=1478763491992
where data= currentTime in long
API Response String:
https://applozic.appspot.com/_ah/upload/AMmfu6YAZpXFUYvC7wqIcW7msh8-YF1d7Tsh1UOTSCzpx2vinrcLQRtVfWbFHHXLFunUqsSLe1dYsDbsJxIO28cNcGrECf7LfFaNSycct-Sybd9KAZWk0yk7HybzxbBp4YQEDmMLi4Uf/ALBNUaYAAAAAWovz3TcYX24yam5K3embIkgQ6Q1pGIRf/
Step 2. Call Url With multipart :
Call API with your file object attached to files[] array(Parameter:files[]):
https://applozic.appspot.com/_ah/upload/AMmfu6YAZpXFUYvC7wqIcW7msh8-YF1d7Tsh1UOTSCzpx2vinrcLQRtVfWbFHHXLFunUqsSLe1dYsDbsJxIO28cNcGrECf7LfFaNSycct-Sybd9KAZWk0yk7HybzxbBp4YQEDmMLi4Uf/ALBNUaYAAAAAWovz3TcYX24yam5K3embIkgQ6Q1pGIRf/
filetMeta json Response:
{"fileMeta":{"blobKey":"AMIfv96n1wlMLpa3R_1i4nbFc4L1RLG81W5RovnPqMhVspzzJv5WBbnYgI4uwZkNjvzszNqsWwEQU6mrYoYsaoa2Vhi45p3P7bvQhAO1ciEL1K1yZJ2HB-goYPULYumC7LA8h33p_Ry
JBewFK8FogMDPR4_4zjClIg","contentType":"image/png","createdAtTime":"1478763491698","name":"applozic.png","size":"8694","thumbnailUrl":"https://lh3.googleusercontent
.com/EfnmKkzLtwBgYQq9UWc26oVqSZUiGukhXQgq7ns9a3G53ZAveFOszamvsqD-tbOfuirqERBO0QR60xFgYiGr=s120"}}
Try this request :
Post request
Url:-
https://applozic.appspot.com/_ah/upload/AMmfu6ZB1z1BBDQMh_ztllvkde5mest9aFeqDHoSmCLzGH3vEtqQLKKOZG820ONgNCOc3BatKJL-59Tppm76zvyfw773R4lEa7m3gaM4cdKGbDU5oy8R_9zt_PT12j8xYSK2oh3rO3xa/ALBNUaYAAAAAWoq31zwU986GLyomPgxjoJb6qHuf4iIx/
Param:files[]

JMeter HTTP request showing Invalid credential and not getting Pass

I am testing a site having login page and then some blogs on which we can comment/post.
I first login to the site and then select a blog and make a comment on that blog.
I am able to pass the login request and select blog request but comment on the blog request showing error Invalid credential.
Here is the response message:
Thread Name: Thread Group 1-1
Sample Start: 2014-09-17 12:41:42 IST
Load time: 9
Latency: 9
Size in bytes: 286
Headers size in bytes: 286
Body size in bytes: 0
Sample Count: 1
Error Count: 1
Response code: 403
Response message: Invalid credentials
Response headers:
HTTP/1.1 403 Invalid credentials
Cache-Control: private
Server: Microsoft-IIS/8.5
X-FRAME-OPTIONS: SAMEORIGIN
X-UA-Compatible: IE=Edge
X-Telligent-Evolution: 8.0.0.37997
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Wed, 17 Sep 2014 07:11:42 GMT
Content-Length: 0
HTTPSampleResult fields:
ContentType:
DataEncoding: null
And this is the request data:
POST http://192.xx.xx.61/TelligentCommunity/api.ashx/v2/comments.json
POST data:
Body=This+is+the+sample+post+by+Arjun&ContentId=c6bc2886-1fcb-46a4-8120-e344b8d8e4a1&ContentTypeId=f7d226ab-d59f-475c-9d22-
Cookie Data:
AuthorizationCookie=71710e44-7ba7-4af7-b390-6ae81cdec229; .Telligent.Evolution=F426475F7F35ED7250E97697BCC3DE0147BD3D15C494681A9C188922459CC698DD6F6A0CA77D61E2D5C6A56A5239B338B8DC484DAB6A4073B69F9F9139500867A843A9EB39D217825C47672E1B5165214A990F8E9CF519ED6159591B510967F84F6810CBCC4466E0DA5E37D03AB0E341A6DA0970861A2F58EE873E168D1A851D9B956033B10C9856D680FC6AC5736F961631BDD1A66EE89024020BF55A8422B24A485311C87C5074F2507E4FFA9EFBADC36B0DCBC051965ACE32EDA2B5607FEBDE17F0C2F486A42E05680FD90F30494B
Request Headers:
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Rest-Authorization-Code: 7a24113f-2a71-428b-80c7-57234aac67c1
Accept-Language: en-US
Accept: */*
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko
DNT: 1
Pragma: no-cache
Authorization-Code: 7a24113f-2a71-428b-80c7-57234aac67c1
X-Requested-With: XMLHttpRequest
Referer: http://192.xx.xx.61/TelligentCommunity/certification/b/90days2mcsa/archive/2014/09/12/how-to-win-by-contributing-to-the-mcp-community
Accept-Encoding: gzip, deflate
Content-Length: 123
Host: 192.xx.xx.61
Are you sure that your login request is successful? How do you know? ASP.NET applications use dynamic parameters like viewstate or eventvalidation and they need to be extracted from the previous response and added to the next one. See ASP.NET Login Testing with JMeter guide for details.
Looking into api and json URL bits I would expect that Content-Type header should be application/json. I would suggest adding a HTTP Header Manager as a child of the request which posts a blog comment and configure it to send above Content-Type.
This Authorization-Code header value: is it something you recorded or it came from correlation? If first you need to pass the correct value as well.

Resources