Using Apache HTTPClient - how can one see the raw request string before it's being sent? - apache-commons-httpclient

For debugging purposes, I'd like to see the raw request that is going to be sent. Is there a way to get this without a HTTP monitor directly from the API of HttpPost or HttpClient?
I found some "almost" duplicate questions, but not for this particular one

You can set some environment variables for Apache HttpClient (example tested for 4.3.2).
System.setProperty("org.apache.commons.logging.Log","org.apache.commons.logging.impl.SimpleLog");
System.setProperty("org.apache.commons.logging.simplelog.showdatetime", "true");
System.setProperty("org.apache.commons.logging.simplelog.log.org.apache.http.wire", "DEBUG");
There are also some more variables for debugging:
System.setProperty("org.apache.commons.logging.simplelog.log.org.apache.http.impl.conn", "DEBUG");
System.setProperty("org.apache.commons.logging.simplelog.log.org.apache.http.impl.client", "DEBUG");
System.setProperty("org.apache.commons.logging.simplelog.log.org.apache.http.client", "DEBUG");
System.setProperty("org.apache.commons.logging.simplelog.log.org.apache.http", "DEBUG");

org.apache.http.client.fluent.Request#viaProxy
This method can make your request pass through proxy server, so your can launch a local proxy server, for example Fiddler, so this debugging proxy can show the details of http request and response.

Try enabling DEBUG mode in your logging configurations, if you're using log4j you can add this to the log4j.xml file of your project
<root>
<priority value="DEBUG" />
<appender-ref ref="FILE" />
</root>
You will have the full request headers, params, body, etc, logged in your logs files.

log4j2 and/or slf4j solution
For everybody using log4j2 and/or slf4j all the mentioned solutions didn't work.
I added the following to my maven pom:
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
<version>1.7.30</version>
</dependency>
Add a log4j2.xml file / or use your existing one and add:
<Logger name="org.apache.http">
<Level>DEBUG</Level>
</Logger>
Of course for a valid log4j2.xml configuration you need some appenders defined etc. A simple example can be found here

Try this:
HttpClient client = new HttpClient();
PostMethod method = new PostMethod(url);
method.setParameter(...., ....);
to retrieve the URI
System.out.println("getUri: " + method.getURI());
to retrieve the parameters in POST
method.getRequestEntity().writeRequest(System.out);

Related

Spring logging priority

The Spring logging documentation is available here and describes the different logging possibilities. However, something I have not been able to find is the priority of the different configurations. More concretely, in application.properties I enabled logging for showing basic authentication failures by doing:
logging.level.org.springframework.security.web.authentication.www=DEBUG
This works fines and enabled the logs I expect to see. Then I checked if I could override the configuration above via log4j2.xml by including:
<Logger name="org.springframework.security.web.authentication.www" level="ERROR" additivity="false">
<AppenderRef ref="CONSOLE"/>
</Logger>
The change in log4j2.xml did not cause any effect. So my impression is that the logging configuration included in applications.properties takes priority over log4j2.xml (or it is read after). The only note I can see in the documentation that may be related is:
Since logging is initialized before the ApplicationContext is created,
it is not possible to control logging from #PropertySources in Spring
#Configuration files. The only way to change the logging system or
disable it entirely is through System properties.
So my assumption is that the following will happen:
Logging is configured according to log4j2.xml.
Spring loads the Application context and then the logging configuration in application.properties is taken and "wins".
So my question is, if someone knows how this is supposed to work without guessing (a link will be appreciated). NOTE: I am interested because I want to be sure that no log4j2.xml will disable the spring-security logging.
Thanks in advance!
application.properties/application.yaml take precedence over log4j2.xml - search for precedence here
Environment variables take precedence over application.properties/application.yaml
A fuller precedence list is here

Can Ktor have different log levels per package/file

In spring boot, it's possible to specify a different log level at the package or even file level.
logging.level.com.company.project.monitors=DEBUG
logging.level.com.company.project.controllers=INFO
logging.level.com.company.project.controllers.utils=WARN
Is there anything similar in Ktor where we can set different logging for individual areas of the app (without writing a bunch of code)?
What you describe isn't Spring Boot specific feature as such, but of the underlying logging library both Spring Boot and Ktor use by default.
In order to achieve the same behaviour in Ktor, add the following lines to your logback.xml file:
<logger name="com.company.project.monitors" level="DEBUG" />
<logger name="com.company.project.controllers" level="INFO" />
<logger name="com.company.project.controllers.utils" level="WARN" />
The logback.xml should be located in your src/main/resources directory, if you're using Gradle as your build tool.
You can read more about Logback support in Ktor here: https://ktor.io/docs/logging.html#add_dependencies

How to use Spring WebClient without Spring Boot

I have a very limited need to be able to make HTTP request. I see WebClient is the new replacement for RestTemplate. But it seems it is impossible to use WebClient, without dragging in the whole of spring boot; which is not what I want to do. Any way to use WebClient without Spring boot?
You can make asynchronous HTTP request using Reactor Netty HttpClient (docs). Spring WebClient uses it under the hood.
Just add dependency
<dependency>
<groupId>io.projectreactor.netty</groupId>
<artifactId>reactor-netty</artifactId>
<version>0.9.11.RELEASE</version>
</dependency>
and make request
HttpClient.create()
.request(HttpMethod.GET)
.uri("http://example.com/")
.responseContent()
.asString()
.subscribe(System.out::println);
I had the same problem and I solved it doing this.
You need to create a logback.xml file in the src / main / resources folder
and copy this
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<statusListener class="ch.qos.logback.core.status.NopStatusListener" />
</configuration>
If you already have this file just add this statusLIstener inside your configuration.
More info: http://logback.qos.ch/manual/configuration.html

spring boot logback refresh

I know that with config server and refresh endpoint, it is possible to dynamically change the logging level in spring boot application. To get control over the log rotation policy and json encoding for file, I decided to use logback. But this will stop me from dynamically changing the logging level.
<root level="info">
<appender-ref ref="RollingFile" />
<appender-ref ref="Console" />
</root>
This means that only info will be written to console/file. But what if I want to change it to debug/trace during runtime?
EDIT
I still dont understand the root level tag. But the logback seems to be taking the log level from application.properties, which basically answers my question.
You can change loggig levle using spring-boot-acutuator endpiont.
To check loging level call this GET method endpoint:
http://host:port/contextpath/actuator/loggers
To check root level loggers call this GET method:
http://host:port/contextpath/actuator/loggers/root
To change root log level call this POST method endpoint:
http://host:port/contextpath/actuator/loggers/root
header: content-type: application-json
body:
{"configuredLevel": "TRACE"}
By calling this endpoint u can change root log level.

How to disable logs of Exposed Framework?

I have a kotlin desktop application with gradle builder.
I added Exposed ORM framework for my sqlite DB.
Then I noticed this framework generates a lot of logs that I don't want to see in console (I want to see only my logs generated io.github.microutils:kotlin-logging).
Is there any way to disable logs from Exposed using gradle properties?
To disable (or change logging level) you have to check your logger framework implementation documentation. Both kotlin-logging and slf4j (which used by kotlin-logging) just provide facades for logging.
For example, if you use logback you could update your configuration to show only warns and errors from an Exposed:
<configuration>
// another code here
<logger name="Exposed" level="warn" additivity="false">
<appender-ref ref="STDOUT"/>
</logger>
</configuration>

Resources