Spring Boot ConfigurationProperties metadata generation - wrong default values - spring-boot

So I'm trying to generate my own configuration metadata from #ConfigurationProperties items, following documentation available at https://docs.spring.io/spring-boot/docs/current/reference/html/configuration-metadata.html#configuration-metadata-annotation-processor. Note that I'm using Spring Boot 2.0 and Kotlin.
It works fine, except for one thing: the defaultValue fields are not getting filled with my default values, but instead just contain standard values like 0 or false.
My #ConfigurationProperties file looks like this:
#Component
#ConfigurationProperties("flowr.epg")
class EpgProperties {
/** Whether or not schedules should be computed/refreshed on (re)start */
var refreshOnRestart = true
/** Number of threads used to store new images */
var nImagesUploadThreads = 10
}
The result looks like this:
{
"hints": [],
"groups": [
{
"sourceType": "com.taktik.flowr.epg.properties.EpgProperties",
"name": "flowr.epg",
"type": "com.taktik.flowr.epg.properties.EpgProperties"
}
],
"properties": [
{
"sourceType": "com.taktik.flowr.epg.properties.EpgProperties",
"defaultValue": false,
"name": "flowr.epg.refresh-on-restart",
"description": "Whether or not schedules should be computed\/refreshed on (re)start",
"type": "java.lang.Boolean"
},
{
"sourceType": "com.taktik.flowr.epg.properties.EpgProperties",
"defaultValue": 0,
"name": "flowr.epg.n-images-upload-threads",
"description": "Number of threads used to store new images in Ozone",
"type": "java.lang.Integer"
}
}
The documentation is kind of poor about that defaultValue field. Am I doing something wrong? Is it possible to fill that defaultValue field automatically? If yes, what am I doing wrong?

Related

Can Springfox 3 generate Integer boundaries based on JSR 303 #Min/#Max in OAS3 API docs?

I created a Spring Boot application with a RestController that validates the DTO passed to a POST method based on JSR 303 validation annotations. API docs are generated using Springfox.
Validations are applied correctly and show up in the OAS2 API Docs. They are incomplete in the OAS3 API docs however - no minimum/maximum boundaries are generated for the Integer fields.
I'm using Spring Boot 2.5.2 (this is important as the latest version, 2.6.2, has issues with Springfox) and Springfox 3.0.0.
Since I found no specific hints in documentation + Springfox issue tracking and JSR303 support is working for the most part, I think this is a bug or oversight in Springfox OAS3 support. In the meantime I found a workaround which I will post as an answer - if I missed anything or there are better solutions I'd be happy to hear about that.
Details:
Controller
#Slf4j
#RestController
public class MyController {
#PostMapping
public void send(#Valid #RequestBody MyDTO dto) {
log.info("{}", dto);
}
}
DTO
#Value
public class MyDTO {
#Size(max = 200)
String text;
#Max(2)
#Min(1)
Integer number;
#Max(4)
#Min(3)
int number2;
#Max(6)
#Min(5)
BigDecimal decimal;
}
OAS2 DTO schema (extracted from http://localhost:8080/v2/api-docs)
{
"MyDTO": {
"type": "object",
"properties": {
"decimal": {
"type": "number",
"minimum": 5,
"maximum": 6,
"exclusiveMinimum": false,
"exclusiveMaximum": false
},
"number": {
"type": "integer",
"format": "int32",
"minimum": 1,
"maximum": 2,
"exclusiveMinimum": false,
"exclusiveMaximum": false
},
"number2": {
"type": "integer",
"format": "int32",
"minimum": 3,
"maximum": 4,
"exclusiveMinimum": false,
"exclusiveMaximum": false
},
"text": {
"type": "string",
"minLength": 0,
"maxLength": 200
}
},
"title": "MyDTO"
}
}
OAS3 DTO schema (extracted from http://localhost:8080/v3/api-docs)
{
"schemas": {
"MyDTO": {
"title": "MyDTO",
"type": "object",
"properties": {
"decimal": {
"maximum": 6,
"exclusiveMaximum": false,
"minimum": 5,
"exclusiveMinimum": false,
"type": "number",
"format": "bigdecimal"
},
"number": {
"type": "integer",
"format": "int32"
},
"number2": {
"type": "integer",
"format": "int32"
},
"text": {
"maxLength": 200,
"minLength": 0,
"type": "string"
}
}
}
}
}
After debugging Springfox I learned that the class springfox.documentation.oas.mappers.SchemaMapper in springfox-oas converts a "general model" to a format for OAS3.
In the "general model", field boundaries are represented by an "NumericElementFacet". A specific property that is being mapped is a subclass of "Schema".
The problem seems to happen here:
https://github.com/springfox/springfox/blob/bc9d0cad83e5dfdb30ddb487594fbc33fc1ba28c/springfox-oas/src/main/java/springfox/documentation/oas/mappers/SchemaMapper.java#L385
Properties represented by a "NumberSchema" are handled correctly (e.g. BigDecimal), their boundaries from the "NumericElementFacet" are applied. Integer fields (and further tests have shown: also Short and Long) are however represented by "IntegerSchema", which isn't handled there so the boundaries are not applied to the resulting API.
So what I did as a workaround was subclassing SchemaMapper, post-processing the results of mapProperties and registering the subclass as #Primary to override the springfox component:
import io.swagger.v3.oas.models.media.Schema;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.annotation.Primary;
import org.springframework.stereotype.Component;
import springfox.documentation.oas.mappers.*;
import springfox.documentation.schema.*;
import springfox.documentation.service.ModelNamesRegistry;
import java.util.*;
#Primary
#Component
#Slf4j
public class IntegerBoundarySupportingOasSchemaMapper extends SchemaMapper {
#Override
#SuppressWarnings("rawtypes")
protected Map<String, Schema> mapProperties(
Map<String, PropertySpecification> properties,
ModelNamesRegistry modelNamesRegistry) {
var result = super.mapProperties(properties, modelNamesRegistry);
result.values()
.stream()
// "integer" seems to cover at least Java Short, Integer and Long.
.filter(property -> "integer".equals(property.getType()))
.forEach(property -> properties.get(property.getName())
.getFacets()
.stream()
.filter(NumericElementFacet.class::isInstance)
.map(NumericElementFacet.class::cast)
.findFirst()
.ifPresent(f -> {
log.trace("Adding boundaries to API field {} (min={}, max={})",
property.getName(),
f.getMinimum(),
f.getMaximum());
property.setMaximum(f.getMaximum());
property.exclusiveMaximum(f.getExclusiveMaximum());
property.setMinimum(f.getMinimum());
property.exclusiveMinimum(f.getExclusiveMinimum());
}));
return result;
}
}
In my case, this is working fine, so maybe its helping someone else, too.
Side note: The SchemaMapper is called every time I retrieve http://localhost:8080/v3/api-docs, that might be something to keep in mind when considering other time-consuming modifications of the schema.

Query specific value in array from a metadata entity in Dynamics 365

I'm trying to avoid iterating through this array, but I imagine that is the only way to handle this. Just seeing if there is a way to directly query this value in the array from the Web API URI.
This is the URI example:
https://example.crm.dynamics.com/api/data/v9.0/GlobalOptionSetDefinitions(f4a9de67-1d00-ea11-a811-000d3a33f702)
And this is an example of the response:
{
"#odata.context": "https://example.crm.dynamics.com/api/data/v9.0/$metadata#GlobalOptionSetDefinitions/Microsoft.Dynamics.CRM.OptionSetMetadata/$entity",
"MetadataId": "f4a9de67-1d00-ea11-a811-000d3a33f702",
"Options": [
{
"Value": 799680006,
"Color": "#0000ff",
"IsManaged": false,
"ExternalValue": "",
"ParentValues": [],
"MetadataId": null,
"HasChanged": null,
"Label": {
"LocalizedLabels": [
{
"Label": "ABC123",
"LanguageCode": 1033,
"IsManaged": false,
"MetadataId": "b4eb2c69-b500-ea11-a811-000d3a33fe19",
"HasChanged": null
}
],
"UserLocalizedLabel": {
"Label": "ABC123",
"LanguageCode": 1033,
"IsManaged": false,
"MetadataId": "b4eb2c69-b500-ea11-a811-000d3a33fe19",
"HasChanged": null
}
}
}
]
}
Basically, I have the "Value": 799680006 which is what I want to somehow add to the URI query parameters, so that I can ultimately get "Label": "ABC123".
Any suggestions or is iterating through the array of objects with if Value = x really the only option?
Let me clarify two things:
Querying metadata like you are using GlobalOptionSetDefinitions to get all the localized labels if you have multiple language packs or for verifying customizations or for Devops deployment purpose is one thing
Getting the label for the selected picklist value in one of the transaction database record is another purpose
If you simply need for second purpose, you can get it by selecting the Formatted value, after adding a header in web api request. Read more in my SO answer
Another way to inspect the label is using stringmap entity.
https://crmdev.crm.dynamics.com/api/data/v9.1/stringmaps?$filter=objecttypecode eq 'account' and attributename eq 'accountclassificationcode' and attributevalue eq 1

Spring MVC Converter and Swagger doc: how to?

I use converters in my Spring MCV controllers. In this example, the String from the path variable is mapped into a UserId:
#GetMapping(path = "/user/{user-id}")
public User get(#Parameter(description = "User id", required = true, example = "3fa85f64-5717-4562-b3fc-2c963f66afa6")
#PathVariable("user-id")
UserId userId) {
return userService.get(userId)
}
It seems to annoy Swagger as the generated doc requires an object as parameter and not a plain string:
...
"/api/v1/user/{user-id}": {
"get": {
"operationId": "get",
"parameters": [
{
"name": "user-id",
"in": "path",
"schema": {
"$ref": "#/components/schemas/UserId"
},
}
],
...
with the UserId schema:
"UserId": {
"type": "object",
"properties": {
"value": {
"type": "string",
"format": "uuid"
}
}
}
And thus the Swagger UI cannot be used because either the parameter is considered as invalid when a single string is provided, either the data is actually invalid when the object format is used.
What is an option to fix that?
To achieve that, the schema parameter of the #Parameter annotation is the answer.
The above example becomes:
#Parameter(description = "User id",
required = true,
schema = #Schema(implementation = String.class), // this is new
example = "3fa85f64-5717-4562-b3fc-2c963f66afa6")

How do I use FreeFormTextRecordSetWriter

I my Nifi controller I want to configure the FreeFormTextRecordSetWriter, but I have no Idea what I should put in the "Text" field. I'm getting the text from my source (in my case GetSolr), and just want to write this, period.
Documentation and mailinglist do not seem to tell me how this is done, any help appreciated.
EDIT: Here the sample input + output I want to achieve (as you can see: not ransformation needed, plain text, no JSON input)
EDIT: I now realize, that I can't tell GetSolr to return just CSV data - but I have to use Json
So referencing with attribute seems to be fine. What the documentation omits is, that the ${flowFile} attribute should containt the complete flowfile that is returned.
Sample input:
{
"responseHeader": {
"zkConnected": true,
"status": 0,
"QTime": 0,
"params": {
"q": "*:*",
"_": "1553686715465"
}
},
"response": {
"numFound": 3194,
"start": 0,
"docs": [
{
"id": "{402EBE69-0000-CD1D-8FFF-D07756271B4E}",
"MimeType": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"FileName": "Test.docx",
"DateLastModified": "2019-03-27T08:05:00.103Z",
"_version_": 1629145864291221504,
"LAST_UPDATE": "2019-03-27T08:16:08.451Z"
}
]
}
}
Wanted output
{402EBE69-0000-CD1D-8FFF-D07756271B4E}
BTW: The documentation says this:
The text to use when writing the results. This property will evaluate the Expression Language using any of the fields available in a Record.
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
I want to use my source's text, so I'm confused
You need to use expression language as if the record's fields are the FlowFile's attributes.
Example:
Input:
{
"t1": "test",
"t2": "ttt",
"hello": true,
"testN": 1
}
Text property in FreeFormTextRecordSetWriter:
${t1} k!${t2} ${hello}:boolean
${testN}Num
Output(using ConvertRecord):
test k!ttt true:boolean
1Num
EDIT:
Seems like what you needed was reading from Solr and write a single column csv. You need to use CSVRecordSetWriter. As for the same,
I should tell you to consider to upgrade to 1.9.1. Starting from 1.9.0, the schema can be inferred for you.
otherwise, you can set Schema Access Strategy as Use 'Schema Text' Property
then, use the following schema in Schema Text
{
"name": "MyClass",
"type": "record",
"namespace": "com.acme.avro",
"fields": [
{
"name": "id",
"type": "int"
}
]
}
this should work
I'll edit it into my answer. If it works for you, please choose my answer :)

How can I sort GeoJson file showing defined tags?

Good morning to all and thank you for your help.
I'm working in a map page (map.html) create by leaflet library that take data from a external geojson file called water_well.js. This file, previously generated by overpass service is just a list of markers. every Marker have some proprerties. Follow an exemple:
"properties": {
"operator:type": "international",
"is_in:district": "west_mamprusi",
"is_in:region": "northern",
"source:date": "2012-02-11",
"source:ele": "gps",
"water_wells:source_type": "borehole"
},
The main page extract those data from the file before with this javascript:
var wwMarker = L.geoJson(water_well, {
pointToLayer : function (feature, latlng) {
lat = feature.geometry.coordinates[0];
lng = feature.geometry.coordinates[1];
//following code that make error
op_type = feature.properties.operator_type;
district = feature.properties.is_in:district;
region = feature.properties.is_in:region;
source_date = feature.properties.source:date;
source_ele = feature.properties.source:ele;
source_type = feature.properties.water_wells:source_type;
.....
I'm sure the problem is my Zero javascript knowledge, but I'm not a programmer and I do this map for my NGO engaged in water wells in Burkina Faso.
The script for extraction of the data don't work in this point:
op_type = feature.properties.operator:type;
The problem is ":" because is invalid character.
The second question is that not all markers in the first file called water_well.js have the same "properties" filled ad actually it is possible that someone have different group of "properties like those two:
{
"type": "Feature",
"id": "node/1606958159",
"properties": {
"#id": "node/1606958159",
"amenity": "drinking_water",
"man_made": "water_well",
"name": "puits 4"
},
"geometry": {
"type": "Point",
"coordinates": [
-3.6235696,
12.02171
]
}
},
{
"type": "Feature",
"id": "node/1913126817",
"properties": {
"#id": "node/1913126817",
"ele": "170.8000030517578",
"grid_proximity": "grid_further_500_m",
"is_in:district": "builsa",
"is_in:region": "upper_east",
"man_made": "water_well",
"operational_status": "open",
"operator:type": "individual",
"pipe_connection": "no",
"pump": "manual",
"seasonal": "another_pattern",
"source": "MVP,Columbia University",
"source:date": "2012-02-14",
"source:ele": "gps",
"water_wells:source_type": "unprotected_well"
},
"geometry": {
"type": "Point",
"coordinates": [
-1.2430456,
10.3233693
]
}
},
maybe it is possible to extract all properties of each item independently from which one is present or not..... This can be de better way to solve the problem but I've no idea how to do that.
This is what I do (ckick the water tap to see pop-up): www.h2openmap.org/map
This is almost what I would like to do (ckick the water tap to see pop-up): overpass-turbo.eu/s/7Ov
Thank you for spending your time reading my question.
Have a nice day everyone, Francesco
You can access those properties using the bracketnotation, instead of using:
district = feature.properties.is_in:district;
Use bracketnotation:
district = feature.properties['is_in:district'];
Reference on property-accessors: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Property_Accessors
Now if you want to do something based on if a property exists there is a hasOwnProperty method available on objects. Since feature.properties is an object you can use that in a condition:
if (features.properties.hasOwnProperty('is_in:district')) {
// Property exists, do stuff
}
// or
if (!features.properties.hasOwnProperty('is_in:district')) {
// Property does not exist, do stuff
}
If you want to do something base on wether multiple properties exist you can use the && (and) operator:
if (features.properties.hasOwnProperty('is_in:district') &&
features.properties.hasOwnProperty('source:data')) {
// Properties exist, do stuff
}
// Or
if (!features.properties.hasOwnProperty('is_in:district') &&
!features.properties.hasOwnProperty('source:data')) {
// Properties do not exist, do stuff
}
You could use the || (or) operator to see if at least one of the conditions matches:
if (features.properties.hasOwnProperty('is_in:district') ||
features.properties.hasOwnProperty('source:data')) {
// At least one of the properties exist, do stuff
}
// Or
if (!features.properties.hasOwnProperty('is_in:district') ||
!features.properties.hasOwnProperty('source:data')) {
// At least one of the properties does not exist, do stuff
}
Reference for this can be found here under "Logical operators": https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Expressions_and_Operators
You can use something like to build (or don't build) the data object that you need for your popup. Hope that helps.

Resources