Spring boot stackdriver logging is textPayload and not jsonPayload - spring-boot

Question

I have a log filter that logs out essential request information for debugging and log analytics. But as you can see, the text payload is really hard to read.
I don't want to have to copy + paste this text payload into a text editor every single time. Is there a way to make stack driver print this in a collapsable json instead?
More info:
- GKE pod
#Component
class LogFilter : WebFilter {
private val logger = LoggerFactory.getLogger(LogFilter::class.java)
override fun filter(exchange: ServerWebExchange, chain: WebFilterChain): Mono<Void> {
return chain
.filter(exchange)
.doAfterTerminate {
val request = exchange.request
val path = request.uri.path
val routesToExclude = listOf("actuator")
var isExcludedRoute = false
for (r in routesToExclude) { if (path.contains(r)) { isExcludedRoute = true; break; } }
if (!isExcludedRoute) {
val startTime = System.currentTimeMillis()
val statusCode = exchange.response.statusCode?.value()
val requestTime = System.currentTimeMillis() - startTime
val msg = "Served $path as $statusCode in $requestTime msec"
val requestPrintMap = mutableMapOf<Any, Any>()
requestPrintMap["method"] = if (request.method != null) {
request.method.toString()
} else "UNKNOWN"
requestPrintMap["path"] = path.toString()
requestPrintMap["query_params"] = request.queryParams
requestPrintMap["headers"] = request.headers
requestPrintMap["status_code"] = statusCode.toString()
requestPrintMap["request_time"] = requestTime
requestPrintMap["msg"] = msg
logger.info(JSONObject(requestPrintMap).toString())
}
}
}
}

Answer 1

What you will need to do is customize Fluentd in GKE. Pretty much it's creating a Fluend daemonset for logging instead of the default logging method.
Once that is done, you can setup structured logging to send jsonPayload logs to Stackdriver Logging.

Answer 2

The default Stackdriver logging agent configuration for Kubernetes will detect single-line JSON and convert it to jsonPayload. You can configure Spring to log as single-line JSON (e.g., via JsonLayout1) and let the logging agent pick up the JSON object (see https://cloud.google.com/logging/docs/agent/configuration#process-payload).
1Some of the JSON field names are different (e.g., JsonLayout uses "level" for the log level, while the Stackdriver logging agent recognizes "severity"), so you may have to override addCustomDataToJsonMap to fully control the resulting log entries.
See also GKE & Stackdriver: Java logback logging format?