Log incoming time of request in Quarkus - quarkus

I would like the log to mention something whenever an incoming request reaches the server, so that I can see its exact time of entry.
Rationale is that I would like to see and follow in the log what happens to the request, from its incoming time up to the response time, so that I can see where the time is spent, in order to take decisions relating to optimisation.
This is on a local server used only for such debug activity, so there would be only one request at a time.
Here is an example (partial) log from the moment a request comes.
Hibernate:
select
user0_."id" as id1_6_
from
"User" user0_
where
(…)
2021-12-20 01:35:43,545 TRACE [org.hib.typ.des.sql.BasicBinder] (executor-thread-192) binding parameter [1] as [VARCHAR] - (…)
Hibernate:
select
(…)
2021-12-20 01:35:43,735 TRACE [org.hib.typ.des.sql.BasicBinder] (executor-thread-192) binding parameter [1] as [INTEGER] - [2]
Hibernate:
select
user0_."id" as id1_6_0_,
(…)
2021-12-20 01:35:44,275 TRACE [org.hib.typ.des.sql.BasicBinder] (executor-thread-192) binding parameter [1] as [VARCHAR] - (…)
2021-12-20 01:35:44,655 INFO [io.git.oli.die.UserService] (executor-thread-192) Got user encoded io.github.oliviercailloux.diet.dao.Base64#4ca04a5c.
2021-12-20 01:35:44,656 INFO [io.git.oli.die.UserService] (executor-thread-192) Got user encoded io.github.oliviercailloux.diet.dao.Base64#4ca04a5c, with events size 0.
2021-12-20 01:35:44,656 INFO [io.git.oli.die.VideoService] (executor-thread-192) Querying for videos.
(…)
I would like to have a supplementary entry that indicates the incoming time (something like 2021-12-20 01:35:43,232 TRACE [org.some-quarkus-class] (executor-thread-192) incoming request), so that I can compute the time taken by each step compared to the starting time. Currently, the intermediate time stamps are visible, but not the incoming time stamp.
This is related to this question and this bug report (or this issue); and probably I could plug into the HttpAuthenticationMechanism Customization mechanism, but these provide only indirect solutions or workarounds. I hope that some simple solution exists, like a simple configuration entry, that allows to log an entry at the start of the request.
Here is the relevant part of my current application.properties.
quarkus.http.auth.basic=true
quarkus.http.auth.proactive=false
quarkus.security.jaxrs.deny-unannotated-endpoints=true
quarkus.hibernate-orm.log.sql=true
quarkus.hibernate-orm.log.bind-parameters=true
quarkus.log.min-level=TRACE
quarkus.log.category."org.apache.http".level=DEBUG
quarkus.http.access-log.enabled=true
quarkus.http.record-request-start-time=true
quarkus.http.access-log.pattern=%{REMOTE_HOST} %l %{REMOTE_USER} %{DATE_TIME} "%{REQUEST_LINE}" %{RESPONSE_CODE} %b %{RESPONSE_TIME} %T

Related

Value cannot be null: Parameter: Source error received from JMeter

I have been receiving this error message on particular request - "Value cannot be null. Parameter name: source".
My Scenario is that 1 user will be transacting 5 transaction (Search and Remit). I'm targeting to run 650 Users to 3250 Transaction. The problem is that one particular request which is getting details from the DB, some data is passed but mostly are the mentioned error message above.
I Have 2 CSV Config, 1 from Users, the 2nd is for the Data where the second csv is inside the Loop Controller.
Your application tells you that the request is not correct, it expects a value and you're failing to provide the value. Use Debug Sampler and View Results Tree listener combination to validate the values of the JMeter Variables which are being sent for successful and not successful requests, it might be a test data issue, for example an extra comma in the CSV file will make CSV Data Set config "think" that this is the delimiter for the next column.

How to know how many request each api handled every day in aws

I have EC2 service (elasticbeanstalk) which my project is located on it. Now Is there any way to see how many requests a specific API handles every day. I'm storing the error, access, ... logs to the cloud watch, Maybe somehow we could use the access logs to see how many requests each API handled every day. But I need to define a chart for it so in one look I could understand, For example, this new endpoint api/user/allowance that I have made, there are some customers started using it. So eventually what I need is something like this
Api | Total number of requests | filter_start_date | filter_end_date
Actually I have dig into the problem more, and I've found a solution for it, At least it's works for me. So I went to cloudWatch and then Insights panel, From there I could define a query to group my log messages by their request Url, The log message I have is sth like this
0.0.0.0 (11.11.11.11) - - [18/Oct/2019:13:33:49 +0000] "GET api/user/allowance HTTP/1.1" 200 2575 "-" "okhttp/3.6.0"
Then I have defined the query to get and group by the Request URL, And after grouping, I counted the grouped.
FIELDS #message
| PARSE #message "* [*] * * *" as ipAddresses, requestTime, RequestAction, RequestUrl, RestOfTheLog
| stats count(*) by RequestUrl
This way you'll have list of endpoints with total number of requests.
As you said, your app is writing the logs to CloudWatch Logs. You can log an unique entrance msg of the restful API you want to trace. Then create a custom metrics for the CloudWatch Log groups with filter matching the entrance msg of API.
See official doc for how to create custom metrics for CloudWatch Log groups.

Running JMeter in Command mode is not creating any log for anything under while loop in .jmx file

I have a .jmx file with two thread groups. The first thread group is for data comparison (DB Vs API) and has a JDBC request where I plugin my SQL script and saves it to a tab delimited file. Then I have a while loop under which I have an HTTP request.
Second thread group for negative scenarios validation.
Below is the structure of the .jmx file
-- Thread Group Name - FX-Rates
-- JDBC Request Name - FX-SQL
-- While loop
-- HTTP Request - FX Rates - API
-- Thread Group Name - Negative Testing
-- Error Codes
I am running JMeter in Non GUI mode using below command.
jmeter -n -t "F:\MY DOCUMENTS\PSM\PSM_Automation\bin\Non_GUI_FX_Rates_Validation.jmx" -l "F:\MY DOCUMENTS\PSM\PSM_Automation\log\Non_GUI_FX_Rates_Validation.jtl"
I see that it is creating log for each individual sampler i.e; SQL and negative scenarios but not for anything under while loop. Below is the log that it created.
Logfile:
timeStamp,elapsed,label,responseCode,responseMessage,threadName,dataType,success,failureMessage,bytes,sentBytes,grpThreads,allThreads,Latency,IdleTime,Connect
1492185939615,12140,FX - SQL,200,OK,FX Rates 1-1,text,true,,18549,0,1,1,12017,0,1566
1492185951933,0,Error 400: Invalid Date Format,Non HTTP response code: java.net.URISyntaxException,Non HTTP response message: Illegal character in query at index 80: https://sys-fxrt-v0.apps.system.pcf.ntrs.com/foreign-exchange-rates?as-of-date=${D_EXCH_RT_EFF},Negative Testing - Error Codes 2-1,text,false,,1105,0,1,1,0,0,0
1492185951935,190,Error 404: No Account,404,Not Found,Negative Testing - Error Codes 2-1,text,true,,354,232,1,1,189,0,170
1492185952127,20,Error 404: Incorrect URL,404,Not Found,Negative Testing - Error Codes 2-1,text,false,,354,241,1,1,19,0,12
1492185952147,19,Error 204: No Data ,404,Not Found,Negative Testing - Error Codes 2-1,text,true,,354,260,1,1,19,0,12
Obviously, the kind of sampler result is there:
Error 400: Invalid Date Format,Non HTTP response code: java.net.URISyntaxException,Non HTTP response message: Illegal
character in query at index 80:
https://sys-fxrt-v0.apps.system.pcf.ntrs.com/foreign-exchange-rates?as-of-date=${D_EXCH_RT_EFF}
Obviously that is because the D_EXCH_RT_EFF is not set/resolved in your URL.
That's the problem you're gonna debug.
Obviously.
If you're already doing that (debugging), and run into a problem during it - then describe it please.
And show two things: how you set (initialize) the variable; how you use it, means, literally, show your HTTP sampler (yes, screenshot, at least).
The reasons of not entering the "While Loop" can be in:
While condition is returning false
While condition is wrong
If your While Condition depends on a variable coming from JDBC Request Name - FX-SQL sampler - double check this variable value using Debug Sampler and View Results Tree listener combination.
General "good practice" is running your test with 1-2 virtual users in GUI mode and inspect requests and responses details in the View Results Tree listener. Also pay attention to any suspicious entries in jmeter.log file. Don't run your JMeter test with the full load until you are totally sure that it is doing what it is supposed to be doing. See How to Debug your Apache JMeter Script article for more information on JMeter tests troubleshooting techniques.

How to print a message in broker log

I've been searching on internet and ESQL/WebSphere MessageBroker documentations, to find a way of printing a variable's value so that i can trace it in Broker Logs. Like System.out.println() in java.
I can't debug the messageflow because of some technical issues, so could you please suggest me how to do it or any workarounds.
UserTrace is supposed to fulfil this role for ESQL but if UserTrace isn't helping then I see a lot of people use static calls out from ESQL to java whcih are then logged.
The java code could be as simple as writing to stdout (which will go in /var/mqsi/components//stdout) but more commonly I see this pattern used with existing java logging frameworks like log4J.
The advantage of this approach is that you unify logging between your JCN's and ESQL compute nodes.
User Trace should suffice your need, Put a trace node at the point where you want to log, select the file trace and give a file path,
pattern give as:
${CURRENT_TIMESTAMP}
${Root}
${Environment}
${LocalEnvironment}
${ExceptionList}
so it logs everything.
If it's in higher environments, then you have to use the mqsichangetrace command to enable the trace on flow.
Probably the easiest way is:
1) Set a temporary location in the Environment to the value of the variable: SET Environment.temp = yourVar ;
2) Subsequently, in a Trace node, set the Pattern on the Basic tab of the Trace node to that temporary location: ${Environment.temp}
3) Configure the Trace node to print to a File, User Trace, or the Local Error Log.
4) Deploy and run your flow. Then look in the output of the Trace node.

Race Condition Warning OpenTSDB

I am getting a race condition warning while running multiple imports simultaneously to openTSDB. Following is one of the log sequences showing the race condition.
2013-08-21 14:34:24,745 INFO [main] UniqueId: Creating an ID for
kind='tagv' name='25447'
2013-08-21 14:34:24,747 INFO [main] UniqueId: Got ID=307 for
kind='tagv' name='25447'
2013-08-21 14:34:24,752 WARN [main] UniqueId: Race condition: tried
to assign ID 307 to tagv:25447, but CAS failed on
PutRequest(table="tsdb-uid", key="25447", family="id",
qualifiers=["tagv"], values=["\x00\x013"],
timestamp=9223372036854775807, lockid=-1, durable=true,
bufferable=true, attempt=0, region=null), which indicates this UID
must have been allocated concurrently by another TSD. So ID 307 was
leaked.
Following questions I have:
Since it is a warning, is it that the record is actually written and not skipped?
At the end it says, 'ID 307 was leaked', so is some other ID assigned to the record?
How to verify that the said record has been written in HBase's table named 'tsdb-uid'? (HBase shell commands, I tried a few but in vain).
This just means that a UID was allocated for nothing, but otherwise everything is fine. If you are worried about the state of your tsdb-uid table, you could run the tsdb uid fsck command, and it will probably report that some UIDs are allocated but unused.
If you see the message only occasionally, you can ignore it. If you see it a lot, then the only undesirable consequence is that you're burning through the UID space faster than you should be, so you may run out of UIDs sooner (there are 16777215 possible UIDs for each of: metric names, tag names, tag values).

Resources