I'm trying to add custom tags - the path variables and their values from each request - to each metric micrometer generates. I'm using spring-boot with java 16.
From my research i've found that creating a bean of type WebMvcTagsContributor alows me to do just that.
This is the code
public class CustomWebMvcTagsContributor implements WebMvcTagsContributor {
private static int PRINT_ERROR_COUNTER = 0;
#Override
public Iterable<Tag> getTags(HttpServletRequest request, HttpServletResponse response,
Object handler,
Throwable exception) {
return Tags.of(getAllTags(request));
}
private static List<Tag> getAllTags(HttpServletRequest request) {
Object attributesMapObject = request.getAttribute(View.PATH_VARIABLES);
if (isNull(attributesMapObject)) {
attributesMapObject = request.getAttribute(HandlerMapping.URI_TEMPLATE_VARIABLES_ATTRIBUTE);
if (isNull(attributesMapObject)) {
attributesMapObject = extractPathVariablesFromURI(request);
}
}
if (nonNull(attributesMapObject)) {
return getPathVariablesTags(attributesMapObject);
}
return List.of();
}
private static Object extractPathVariablesFromURI(HttpServletRequest request) {
Long currentUserId = SecurityUtils.getCurrentUserId().orElse(null);
try {
URI uri = new URI(request.getRequestURI());
String path = uri.getPath(); //get the path
UriTemplate uriTemplate = new UriTemplate((String) request.getAttribute(
HandlerMapping.BEST_MATCHING_PATTERN_ATTRIBUTE)); //create template
return uriTemplate.match(path); //extract values form template
} catch (Exception e) {
log.warn("[Error on 3rd attempt]", e);
}
return null;
}
private static List<Tag> getPathVariablesTags(Object attributesMapObject) {
try {
Long currentUserId = SecurityUtils.getCurrentUserId().orElse(null);
if (nonNull(attributesMapObject)) {
var attributesMap = (Map<String, Object>) attributesMapObject;
List<Tag> tags = attributesMap.entrySet().stream()
.map(stringObjectEntry -> Tag.of(stringObjectEntry.getKey(),
String.valueOf(stringObjectEntry.getValue())))
.toList();
log.warn("[CustomTags] [{}]", CommonUtils.toJson(tags));
return tags;
}
} catch (Exception e) {
if (PRINT_ERROR_COUNTER < 5) {
log.error("[Error while getting attributes map object]", e);
PRINT_ERROR_COUNTER++;
}
}
return List.of();
}
#Override
public Iterable<Tag> getLongRequestTags(HttpServletRequest request, Object handler) {
return null;
}
}
#Bean
public WebMvcTagsContributor webMvcTagsContributor() {
return new CustomWebMvcTagsContributor();
}
In order to test this, i've created a small spring boot app, added an endpoint to it. It works just fine.
The problem is when I add this code to the production app.
The metrics generates are the default ones and i can't figure out why.
What can I check to see why the tags are not added?
local test project
http_server_requests_seconds_count {exception="None", method="GET",id="123",outcome="Success",status="200",test="test",uri="/test/{id}/compute/{test}",)1.0
in prod - different (& bigger) app
http_server_requests_seconds_count {exception="None", method="GET",outcome="Success",status="200",uri="/api/{something}/test",)1.0
What i've tried and didn't work
Created a bean that implemented WebMvcTagsProvider - this one had an odd behaviour - it wasn't creating metrics for endpoints that had path variables in the path - though in my local test project it worked as expected
I added that log there in order to see what the extra tags are but doesn't seem to reach there as i don't see anything in the logs - i know, you might say that the current user id stops it, but it's not that.
I have micro service which support open tracing and that injecting trace-id and span-id in to header. Other micro service support open telemetry. how can I create parent span using trace-id and span-id in second micro service?
Thanks,
You can use W3C Trace Context specifications to achieve this. We need to send traceparent(Ex: 00-8652a752089f33e2659dff28d683a18f-7359b90f4355cfd9-01) from producer via HTTP headres ( or you can create it using the trace-id and span-id in the consumer). Then we can extract the remote context and create the span with traceparent.
This is the consumer controller. TextMapGetter used to map that traceparent data to the Context. ExtractModel is just a custom class.
#GetMapping(value = "/second")
public String sencondTest(#RequestHeader(value = "traceparent") String traceparent){
try {
Tracer tracer = openTelemetry.getTracer("cloud.events.second");
TextMapGetter<ExtractModel> getter = new TextMapGetter<>() {
#Override
public String get(ExtractModel carrier, String key) {
if (carrier.getHeaders().containsKey(key)) {
return carrier.getHeaders().get(key);
}
return null;
}
#Override
public Iterable<String> keys(ExtractModel carrier) {
return carrier.getHeaders().keySet();
}
};
ExtractModel model = new ExtractModel();
model.addHeader("traceparent", traceparent);
Context extractedContext = openTelemetry.getPropagators().getTextMapPropagator()
.extract(Context.current(), model, getter);
try (Scope scope = extractedContext.makeCurrent()) {
// Automatically use the extracted SpanContext as parent.
Span serverSpan = tracer.spanBuilder("CloudEvents Server")
.setSpanKind(SpanKind.SERVER)
.startSpan();
try {
Thread.sleep(150);
} finally {
serverSpan.end();
}
}
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
return "Server Received!";
}
Then when we configuring the OpenTelemetrySdk need to set W3CTraceContextPropagator in Context Propagators.
// Use W3C Propagator(to extract span from HTTP headers) since we use the W3C specifications
TextMapPropagator textMapPropagator = W3CTraceContextPropagator.getInstance();
OpenTelemetrySdk openTelemetrySdk = OpenTelemetrySdk.builder()
.setTracerProvider(tracerProvider)
.setPropagators(ContextPropagators.create(textMapPropagator))
.buildAndRegisterGlobal();
Here is my customer ExtractModel class
public class ExtractModel {
private Map<String, String> headers;
public void addHeader(String key, String value) {
if (this.headers == null){
headers = new HashMap<>();
}
headers.put(key, value);
}
public Map<String, String> getHeaders() {
return headers;
}
public void setHeaders(Map<String, String> headers) {
this.headers = headers;
}
}
You can find more details in the official documentation for manual instrumentation.
Generally you have to propogate the span-id and trace-id if it is available in header. Any request you get in your microservice, check if the headers have span-id and trace-id in them. If yes,extract them and use them in your service.
If it is not present then you create a new one and use it in your service and also add it to requests that go out of your microservice.
I am working on project where I need to validate consumer group is created on topic or not. Is there any way in boldSpring Kafkabold to validate it
Currently, I haven't seen describeConsumerGroups supported in Spring-Kafka KafkaAdmin. So, you may need to create a Kafka AdminClient and call the method by yourself.
E.g: Here, I took advantage of the auto-configuration property class KafkaProperties and autowired it to the service.
#Service
public class KafkaBrokerService implements BrokerService {
private Map<String, Object> configs;
public KafkaBrokerService(KafkaProperties kafkaProperties) { // Autowired
this.configs = kafkaProperties.buildAdminProperties();
}
private AdminClient createAdmin() {
Map<String, Object> configs2 = new HashMap<>(this.configs);
return AdminClient.create(configs2);
}
public SomeDto consumerGroupDescription(String groupId) {
try (AdminClient adminClient = createAdmin()) {
// ConsumerGroup's members
ConsumerGroupDescription consumerGroupDescription = adminClient.describeConsumerGroups(Collections.singletonList(groupId))
.describedGroups().get(groupId).get();
// ConsumerGroup's partitions and the committed offset in each partition
Map<TopicPartition, OffsetAndMetadata> offsets = adminClient.listConsumerGroupOffsets(groupId).partitionsToOffsetAndMetadata().get();
// When you get the information, you can validate it here.
...
} catch (ExecutionException | InterruptedException e) {
//
}
}
}
I follow the instructions on this tutorial (https://www.baeldung.com/spring-boot-redis-cache)
#Cacheable(value = "itemCache")
public UserInfo getUserInfo(String id) {
// without explicit manipulate cache, the value can't be retrieved from Redis
UserInfo res = cacheManager.getCache("itemCache").get(id, UserInfo.class);
if (res != null) {
return res;
}
try {
... retrieve from database ...
res = convertMapToUserInfo(id, userInfoMap);
// without explicit manipulate cache, the value can't be stored in Redis
cacheManager.getCache("itemCache").put(id, res);
return res;
}
} catch (Exception e) {
...
} finally {
...
}
return null;
}
The weird thing is, I have to put/get items from Cache manualy, even I use Cacheable annotation. Without explicit manipulate cache, the returned value of getUserInfo can't be cached.
The RedisConfiguration contains code
#Bean
public RedisCacheManagerBuilderCustomizer redisCacheManagerBuilderCustomizer() {
return (builder) -> builder
.withCacheConfiguration("itemCache",
this.cacheConfiguration());
}
#Bean
public RedisCacheConfiguration cacheConfiguration() {
return RedisCacheConfiguration.defaultCacheConfig()
.disableCachingNullValues()
.serializeValuesWith(RedisSerializationContext.SerializationPair.fromSerializer(new GenericJackson2JsonRedisSerializer()));
}
And I also add #EnableCaching to my Application class. Could anyone help me figure out why my cache doesn't take effect. Thanks!
I've recently found out that if we use just one entry with the builder it does'nt work. But if you have more than one cache in your project and include all these entries in this method or even if you add a 'dummy' entry, it works. Maybe a bug or a known issue in Spring implementation, I don't know.
#Bean
public RedisCacheManagerBuilderCustomizer redisCacheManagerBuilderCustomizer() {
return (builder) -> builder
.withCacheConfiguration("itemCache", this.cacheConfiguration())
.withCacheConfiguration("dummy", this.cacheConfiguration());
}
I hope helps.
I want my response to include this:
"keyMaps":{
"href":"http://localhost/api/keyMaps{/keyMapId}",
"templated":true
}
That's easy enough to achieve:
add(new Link("http://localhost/api/keyMaps{/keyMapId}", "keyMaps"));
But, of course, I'd rather use the ControllerLinkBuilder, like this:
add(linkTo(methodOn(KeyMapController.class).getKeyMap("{keyMapId}")).withRel("keyMaps"));
The problem is that by the time the variable "{keyMapId}" reaches the UriTemplate constructor, it's been included in an encoded URL:
http://localhost/api/keyMaps/%7BkeyMapId%7D
So UriTemplate's constructor doesn't recognise it as containing a variable.
How can I persuade ControllerLinkBuilder that I want to use template variables?
It looks to me like the current state of Spring-HATEOAS doesn't allow this via the ControllerLinkBuilder (I'd very much like to be proven wrong), so I have implemented this myself using the following classes for templating query parameters:
public class TemplatedLinkBuilder {
private static final TemplatedLinkBuilderFactory FACTORY = new TemplatedLinkBuilderFactory();
public static final String ENCODED_LEFT_BRACE = "%7B";
public static final String ENCODED_RIGHT_BRACE = "%7D";
private UriComponentsBuilder uriComponentsBuilder;
TemplatedLinkBuilder(UriComponentsBuilder builder) {
uriComponentsBuilder = builder;
}
public static TemplatedLinkBuilder linkTo(Object invocationValue) {
return FACTORY.linkTo(invocationValue);
}
public static <T> T methodOn(Class<T> controller, Object... parameters) {
return DummyInvocationUtils.methodOn(controller, parameters);
}
public Link withRel(String rel) {
return new Link(replaceTemplateMarkers(uriComponentsBuilder.build().toString()), rel);
}
public Link withSelfRel() {
return withRel(Link.REL_SELF);
}
private String replaceTemplateMarkers(String encodedUri) {
return encodedUri.replaceAll(ENCODED_LEFT_BRACE, "{").replaceAll(ENCODED_RIGHT_BRACE, "}");
}
}
and
public class TemplatedLinkBuilderFactory {
private final ControllerLinkBuilderFactory controllerLinkBuilderFactory;
public TemplatedLinkBuilderFactory() {
this.controllerLinkBuilderFactory = new ControllerLinkBuilderFactory();
}
public TemplatedLinkBuilder linkTo(Object invocationValue) {
ControllerLinkBuilder controllerLinkBuilder = controllerLinkBuilderFactory.linkTo(invocationValue);
UriComponentsBuilder uriComponentsBuilder = controllerLinkBuilder.toUriComponentsBuilder();
Assert.isInstanceOf(DummyInvocationUtils.LastInvocationAware.class, invocationValue);
DummyInvocationUtils.LastInvocationAware invocations = (DummyInvocationUtils.LastInvocationAware) invocationValue;
DummyInvocationUtils.MethodInvocation invocation = invocations.getLastInvocation();
Object[] arguments = invocation.getArguments();
MethodParameters parameters = new MethodParameters(invocation.getMethod());
for (MethodParameter requestParameter : parameters.getParametersWith(RequestParam.class)) {
Object value = arguments[requestParameter.getParameterIndex()];
if (value == null) {
uriComponentsBuilder.queryParam(requestParameter.getParameterName(), "{" + requestParameter.getParameterName() + "}");
}
}
return new TemplatedLinkBuilder(uriComponentsBuilder);
}
}
Which embeds the normal ControllerLinkBuilder and then uses similar logic to parse for #RequestParam annotated parameters that are null and add these on to the query parameters. Also, our client resuses these templated URIs to perform further requests to the server. To achieve this and not need to worry about stripping out the unused templated params, I have to perform the reverse operation (swapping {params} with null), which I'm doing using a custom Spring RequestParamMethodArgumentResolver as follows
public class TemplatedRequestParamResolver extends RequestParamMethodArgumentResolver {
public TemplatedRequestParamResolver() {
super(false);
}
#Override
protected Object resolveName(String name, MethodParameter parameter, NativeWebRequest webRequest) throws Exception {
Object value = super.resolveName(name, parameter, webRequest);
if (value instanceof Object[]) {
Object[] valueAsCollection = (Object[])value;
List<Object> resultList = new LinkedList<Object>();
for (Object collectionEntry : valueAsCollection) {
if (nullifyTemplatedValue(collectionEntry) != null) {
resultList.add(collectionEntry);
}
}
if (resultList.isEmpty()) {
value = null;
} else {
value = resultList.toArray();
}
} else{
value = nullifyTemplatedValue(value);
}
return value;
}
private Object nullifyTemplatedValue(Object value) {
if (value != null && value.toString().startsWith("{") && value.toString().endsWith("}")) {
value = null;
}
return value;
}
}
Also this needs to replace the existing RequestParamMethodArgumentResolver which I do with:
#Configuration
public class ConfigureTemplatedRequestParamResolver {
private #Autowired RequestMappingHandlerAdapter adapter;
#PostConstruct
public void replaceArgumentMethodHandlers() {
List<HandlerMethodArgumentResolver> argumentResolvers = new ArrayList<HandlerMethodArgumentResolver>(adapter.getArgumentResolvers());
for (int cursor = 0; cursor < argumentResolvers.size(); ++cursor) {
HandlerMethodArgumentResolver handlerMethodArgumentResolver = argumentResolvers.get(cursor);
if (handlerMethodArgumentResolver instanceof RequestParamMethodArgumentResolver) {
argumentResolvers.remove(cursor);
argumentResolvers.add(cursor, new TemplatedRequestParamResolver());
break;
}
}
adapter.setArgumentResolvers(argumentResolvers);
}
}
Unfortunately, although { and } are valid characters in a templated URI, they are not valid in a URI, which may be a problem for your client code depending on how strict it is. I'd much prefer a neater solution built into Spring-HATEOAS!
With latest versions of spring-hateoas you can do the following:
UriComponents uriComponents = UriComponentsBuilder.fromUri(linkBuilder.toUri()).build();
UriTemplate template = new UriTemplate(uriComponents.toUriString())
.with("keyMapId", TemplateVariable.SEGMENT);
will give you: http://localhost:8080/bla{/keyMapId}",
Starting with this commit:
https://github.com/spring-projects/spring-hateoas/commit/2daf8aabfb78b6767bf27ac3e473832c872302c7
You can now pass null where path variable is expected. It works for me, without workarounds.
resource.add(linkTo(methodOn(UsersController.class).someMethod(null)).withRel("someMethod"));
And the result:
"someMethod": {
"href": "http://localhost:8080/api/v1/users/{userId}",
"templated": true
},
Also check related issues: https://github.com/spring-projects/spring-hateoas/issues/545
We've run into the same problem. General workaround is we have our own LinkBuilder class with a bunch of static helpers. Templated ones look like this:
public static Link linkToSubcategoriesTemplated(String categoryId){
return new Link(
new UriTemplate(
linkTo(methodOn(CategoryController.class).subcategories(null, null, categoryId))
.toUriComponentsBuilder().build().toUriString(),
// register it as variable
getBaseTemplateVariables()
),
REL_SUBCATEGORIES
);
}
private static TemplateVariables getBaseTemplateVariables() {
return new TemplateVariables(
new TemplateVariable("page", TemplateVariable.VariableType.REQUEST_PARAM),
new TemplateVariable("sort", TemplateVariable.VariableType.REQUEST_PARAM),
new TemplateVariable("size", TemplateVariable.VariableType.REQUEST_PARAM)
);
}
This is for exposing the parameters of a controller response of a PagedResource.
then in the controllers we call this an append a withRel as needed.
According to this issue comment, this will be addressed in an upcoming release of spring-hateoas.
For now, there's a drop-in replacement for ControllerLinkBuilder available from de.escalon.hypermedia:spring-hateoas-ext in Maven Central.
I can now do this:
import static de.escalon.hypermedia.spring.AffordanceBuilder.*
...
add(linkTo(methodOn(KeyMapController.class).getKeyMap(null)).withRel("keyMaps"));
I pass in null as the parameter value to indicate I want to use a template variable. The name of the variable is automatically pulled from the controller.
I needed to include a link with template variables in the root of a spring data rest application, to get access via traverson to an oauth2 token. This is working fine, maybe useful:
#Component
class RepositoryLinksResourceProcessor implements ResourceProcessor<RepositoryLinksResource> {
#Override
RepositoryLinksResource process(RepositoryLinksResource resource) {
UriTemplate uriTemplate = new UriTemplate(
ControllerLinkBuilder.
linkTo(
TokenEndpoint,
TokenEndpoint.getDeclaredMethod("postAccessToken", java.security.Principal, Map )).
toUriComponentsBuilder().
build().
toString(),
new TemplateVariables([
new TemplateVariable("username", TemplateVariable.VariableType.REQUEST_PARAM),
new TemplateVariable("password", TemplateVariable.VariableType.REQUEST_PARAM),
new TemplateVariable("clientId", TemplateVariable.VariableType.REQUEST_PARAM),
new TemplateVariable("clientSecret", TemplateVariable.VariableType.REQUEST_PARAM)
])
)
resource.add(
new Link( uriTemplate,
"token"
)
)
return resource
}
}
Based on the previous comments I have implemented a generic helper method (against spring-hateoas-0.20.0) as a "temporary" workaround. The implementation does consider only RequestParameters and is far from being optimized or well tested. It might come handy to some other poor soul traveling down the same rabbit hole though:
public static Link getTemplatedLink(final Method m, final String rel) {
DefaultParameterNameDiscoverer disco = new DefaultParameterNameDiscoverer();
ControllerLinkBuilder builder = ControllerLinkBuilder.linkTo(m.getDeclaringClass(), m);
UriTemplate uriTemplate = new UriTemplate(UriComponentsBuilder.fromUri(builder.toUri()).build().toUriString());
Annotation[][] parameterAnnotations = m.getParameterAnnotations();
int param = 0;
for (Annotation[] parameterAnnotation : parameterAnnotations) {
for (Annotation annotation : parameterAnnotation) {
if (annotation.annotationType().equals(RequestParam.class)) {
RequestParam rpa = (RequestParam) annotation;
String parameterName = rpa.name();
if (StringUtils.isEmpty(parameterName)) parameterName = disco.getParameterNames(m)[param];
uriTemplate = uriTemplate.with(parameterName, TemplateVariable.VariableType.REQUEST_PARAM);
}
}
param++;
}
return new Link(uriTemplate, rel);
}