SpringBoot map yaml to map of map - spring

Is it possible to inject below yaml file to Spring Boot application as Map<String, Map<String, String> where tradeType will be the key of outer map and P and B will be the key value for inner map for.
tradeType:
P: B
S: S
securityCode:
ICI: ICICI Bank
BOB: Bank of Baroda
BOA: Bank of America
BOS: Bank of Singapore
As suggested this is how my class look.
#Configuration
#PropertySource("file:data/referenceDataMapping.yaml")
#ConfigurationProperties(prefix = "map")
public class ReferenceDataMapping {
private final Logger logger = LoggerFactory.getLogger(this.getClass());
private Map<String, Map<String, String>> entries;
#Override
public String toString() {
if (entries != null) {
logger.info(entries.toString());
return entries.toString();
} else {
return null;
}
}
public Map<String, Map<String, String>> getEntries() {
return entries;
}
public void setEntries(Map<String, Map<String, String>> entries) {
this.entries = entries;
}
}
from build.gradle
dependencies {
compile 'org.springframework.boot:spring-boot-starter-activemq:2.1.2.RELEASE'
compile 'org.springframework.boot:spring-boot-starter-security:2.1.2.RELEASE'
compile 'org.springframework.boot:spring-boot-starter-web:2.1.2.RELEASE'
compile 'org.apache.camel:camel-spring-boot-starter:2.23.1'
compile 'org.apache.camel:camel-quartz2:2.23.1'
compile 'org.apache.camel:camel-jms:2.23.1'
compile 'org.apache.camel:camel-jacksonxml:2.23.1'
compile 'com.fasterxml.jackson.core:jackson-databind:2.9.8'
compile 'net.sf.saxon:Saxon-HE:9.9.1-1'
testCompile 'org.springframework.boot:spring-boot-starter-test:2.1.2.RELEASE'
testCompile 'org.springframework.security:spring-security-test:5.1.3.RELEASE'
}
referenceDataMapping.yaml
map:
entries:
tradeType:
P: B
S: S
securityCode:
ICI: ICICI Bank
BOB: Bank of Baroda
BOA: Bank of America
BOS: Bank of Singapore

Yes. It is possible.
#Configuration
#ConfigurationProperties(prefix="map")
public class Test {
private Map<String,Map<String,String>> entires;
public Map<String, Map<String, String>> getEntires() {
return entires;
}
public void setEntires(Map<String, Map<String, String>> entires) {
this.entires = entires;
}
}
application.yml:
map:
entires:
tradeType:
P: B
S: S
securityCode:
ICI: ICICI Bank
BOB: Bank of Baroda
BOA: Bank of America
BOS: Bank of Singapore
output:
{tradeType={P=B, S=S}, securityCode={ICI=ICICI Bank, BOB=Bank of Baroda, BOA=Bank of America, BOS=Bank of Singapore}}
Update:
As described in this github-issue. #PropertySource doesnt support yaml files. In that case, kindly follow this guide PropertySource-with-yaml-files

If the YAML is in different file than application.yml,
#Component
public class YourClass {
private Map<String, String> tradeType;
private Map<String, String> securityCode;
public Map<String, String> getTradeType() {
return tradeType;
}
public Map<String, String> getSecurityCode() {
return securityCode;
}
#PostConstruct
public void yamlFactory() {
YamlMapFactoryBean factory = new YamlMapFactoryBean();
factory.setResources(new ClassPathResource("your.yml"));
tradeType = (Map<String, String>) factory.getObject().get("tradeType");
securityCode = (Map<String, String>) factory.getObject().get("securityCode");
}
}

Related

Why are the application config values parsed by my custom Spring PropertySourceLoader not being used?

I am attempting to write a TOML PropertySourceLoader implementation. I took a look at some other examples on GitHub and stackoverflow, all of which seem to eventually parse the result out to a map and then return an OriginTrackedMapPropertySource, which is what I tried below:
public final class TomlPropertySourceFactory implements PropertySourceFactory {
#Override
public PropertySource<?> createPropertySource(String name, EncodedResource resource) throws IOException {
if (!ClassUtils.isPresent("com.fasterxml.jackson.dataformat.toml.TomlFactory", null)) {
throw new IllegalStateException(
"Attempted to load " + name + " but jackson-dataformat-toml was not found on the classpath");
}
if (!ClassUtils.isPresent("com.fasterxml.jackson.core.io.ContentReference", null)) {
throw new IllegalStateException(
"Attempted to load " + name + " but jackson-core was either not found on the classpath or below version 2.13.0");
}
final ObjectMapper mapper = new ObjectMapper(new TomlFactory());
final Map<String, Object> resultMap = mapper.convertValue(mapper.readTree(resource.getInputStream()), new TypeReference<Map<String, Object>>(){});
return new OriginTrackedMapPropertySource(Optional.ofNullable(name).orElseGet(resource.getResource()::getFilename), resultMap);
}
}
public final class TomlPropertySourceLoader implements PropertySourceLoader {
#Override
public String[] getFileExtensions() {
return new String[]{"tml", "toml"};
}
#Override
public List<PropertySource<?>> load(final String name, final Resource resource) throws IOException {
final EncodedResource encodedResource = new EncodedResource(resource);
return Collections.singletonList(new TomlPropertySourceFactory().createPropertySource(name, encodedResource));
}
}
This code does seem to more or less do what is expected; it is executed when application.toml is present, it loads and parses the file out to a <String, Object> map, but from there, none of the actual properties seem to be present in the application — be it when using #Value, #ConfigurationProperties or even when attempting to set stuff like the port Tomcat runs on.
There's not a ton of information available on the internet, without digging into the depths of Spring, about what exactly it is expecting. I'm not sure if the problem is due to how my map is structured or perhaps due to something with the name.
Below you can find my application.toml file:
[spring.datasource]
url = "jdbc:hsqldb:file:testdb"
username = "sa"
[spring.thymeleaf]
cache = false
[server]
port = 8085
[myapp]
foo = "Hello"
bar = 42
aUri = "https://example.org/hello"
targetLocale = "en-US"
[myapp.configuration]
endpoints = ["one", "two", "three"]
[myapp.configuration.connectionSettings]
one = "hello"
[myapp.configuration.connectionSettings.two]
two_sub = "world!"
And my configuration classes:
#Data
#NoArgsConstructor
#Component
#ConfigurationProperties("myapp")
public class AppConfig {
private String foo;
private int bar;
private URI aUri;
private Locale targetLocale;
private SubConfiguration configuration;
}
#Data
#NoArgsConstructor
public class SubConfiguration {
private List<String> endpoints;
private Map<String, Object> connectionSettings;
}
As well as my testing controller:
#RestController
#RequiredArgsConstructor
public final class TomlDemoController {
private final AppConfig appConfig;
#GetMapping("/api/config")
AppConfig getConfig() {
return appConfig;
}
}
The issue was the structure of the property map. The keys have to be flattened in order to work. As an example, for a given table:
[server]
port = 8085
Rather than producing a nested map:
Properties = {
"server" = {
"port" = 8085
}
}
Spring is expecting something more like:
Properties = {
"server.port" = 8085
}
A quick solution using the ObjectToMapTransformer found in Spring Integration:
#Override
#SuppressWarnings("unchecked")
public PropertySource<?> createPropertySource(String name, EncodedResource resource) throws IOException {
if (!ClassUtils.isPresent("com.fasterxml.jackson.dataformat.toml.TomlFactory", null)) {
throw new IllegalStateException(
"Attempted to load " + name + " but jackson-dataformat-toml was not found on the classpath");
}
if (!ClassUtils.isPresent("com.fasterxml.jackson.core.io.ContentReference", null)) {
throw new IllegalStateException(
"Attempted to load " + name + " but jackson-core was either not found on the classpath or below version 2.13.0");
}
final ObjectMapper mapper = new ObjectMapper(new TomlFactory());
final Message<JsonNode> message = new GenericMessage<>(mapper.readTree(resource.getInputStream()));
final ObjectToMapTransformer transformer = new ObjectToMapTransformer();
transformer.setShouldFlattenKeys(true);
Map<String,Object> resultMap = (Map<String, Object>) transformer.transform(message).getPayload();
return new OriginTrackedMapPropertySource(Optional.ofNullable(name).orElseGet(resource.getResource()::getFilename), resultMap);
}

Spring boot consume 2 rest and merge some fields

Im new to Spring Boot and got a problem were i need to consume 2 remote Rest services and merge the results. Would need some insight on the right approach.
I got something like this:
{"subInventories":[
{"OrganizationId": 0,
"OrganizationCode":"",
"SecondaryInventoryName":"",
"Description":""},...{}...],
{"organizations":[
{"OrganizationId":0,
"OrganizationCode":"",
"OrganizationName":"",
"ManagementBusinessUnitId":,
"ManagementBusinessUnitName":""}, ...{}...]}
and need to make it into something like this:
{"items":[
{"OrganizationId":0,
"OrganizationCode":"",
"OrganizationName":"",
"ManagementBusinessUnitId":0,
"ManagementBusinessUnitName":"",
"SecondaryInventoryName":"",
"Description":""},...{}...]
got 2 #Entitys to represent each item, Organizations and Inventories with the attributtes like the JSON fields.
EDIT
Currently trying to get matches with Java8 stream()
#GetMapping("/manipulate")
public List<Organization> getManipulate() {
List<Organization> organization = (List<Organization>)(Object) organizationController.getOrganization();
List<SubInventories> subInventories = (List<SubInventories>)(Object) getSuvInventories();
List<Organization> intersect = organization.stream().filter(o -> subInventories.stream().anyMatch(s -> s.getOrganizationId()==o.getOrganizationId()))
.collect(Collectors.toList());
return intersect;
}
found this searching but i got many classes and I don't know if it would be better to just for each organization get the subinventories and put them in a list of maps like
List<Map<String,Object> myList = new ArrayList<>();
//Loops here
Map<String,Object> a = new HashMap<>();
a.put("OrganizationID", 1231242415)...
myList.add(a)
Quite lost in what the right approach is.
EDIT2
Here the classes I'm using.
Organizations
#Entity
#JsonAutoDetect(fieldVisibility = Visibility.ANY)
#JsonIgnoreProperties(ignoreUnknown = true)
public class Organization implements Serializable{
//#JsonObject("OrganizationId")
#Id
private Long OrganizationId;
private Long ManagementBusinessUnitId;
private String OrganizationCode,OrganizationName,ManagementBusinessUnitName;
public Organization() {
}
//getters setters
}
SubInventories
#Entity
#JsonAutoDetect(fieldVisibility = Visibility.ANY)
#JsonIgnoreProperties(ignoreUnknown = true)
public class SubInventories implements Serializable{
#Id
private Long OrganizationId;
private String OrganizationCode,SecondaryInventoryName,Description;
public SubInventories() {
}
//getters and setters
}
Wrapper to unwrapp consume
#JsonAutoDetect(fieldVisibility = Visibility.ANY)
#JsonIgnoreProperties(ignoreUnknown = true)
public class Wrapper {
//#JsonProperty("items")
private List<Object> items;
public Wrapper() {
}
public List<Object> getOrganization() {
return items;
}
public void setOrganization(List<Object> organization) {
this.items = organization;
}
}
OrganizationController
#RestController
public class OrganizationController {
#Autowired
private RestTemplate restTemplate;
#Autowired
private Environment env;
#GetMapping("/organizations")
public List<Object> getOrganization() {
return getOrganizationInfo();
}
private List<Object> getOrganizationInfo() {
String url = env.getProperty("web.INVENTORY_ORGANIZATIONS");
UriComponentsBuilder builder = UriComponentsBuilder.fromUriString(url);
builder.queryParam("fields", "OrganizationId,OrganizationCode,OrganizationName,ManagementBusinessUnitId,ManagementBusinessUnitName");
builder.queryParam("onlyData", "true");
HttpHeaders headers = new HttpHeaders();
headers.setBasicAuth(env.getProperty("authentication.name"),env.getProperty("authentication.password"));
HttpEntity request = new HttpEntity(headers);
ResponseEntity<Wrapper> temp = restTemplate.exchange(builder.toUriString(), HttpMethod.GET, request, new ParameterizedTypeReference<Wrapper>() {});
List<Object> data = temp.getBody().getOrganization();
return data;
}
}
SubInventoryController
#RestController
public class SubInventoryController {
#Autowired
private RestTemplate restTemplate;
#Autowired
private Environment env;
#GetMapping("/sub")
public List<Object> getSuvInventories() {
UriComponentsBuilder builder = UriComponentsBuilder.fromUriString("this is private :(");
builder.queryParam("onlyData", "true");
builder.queryParam("expand", "subinventoriesDFF");
builder.queryParam("limit", "999999");
builder.queryParam("fields", "OrganizationId,OrganizationCode,SecondaryInventoryName,Description");
HttpHeaders headers = new HttpHeaders();
headers.setBasicAuth(env.getProperty("authentication.name"),env.getProperty("authentication.password"));
headers.set("REST-Framework-Version", "2");
HttpEntity request = new HttpEntity(headers);
ResponseEntity<Wrapper> subInventories = restTemplate.exchange(builder.toUriString(), HttpMethod.GET, request, new ParameterizedTypeReference<Wrapper>() {});
List<Object> data = subInventories.getBody().getOrganization();
return data;
}
}
where I'm right now
#RestController
public class MainController {
#Autowired
private RestTemplate restTemplate;
#Autowired
private Environment env;
#Autowired
private OrganizationController organizationController;
#Autowired
private SubInventoryController subInventoryController;
#GetMapping("/manipulate")
public Map<Organization, List<SubInventories>> getManipulate() {
List<Organization> organizations = (List<Organization>)(Object) organizationController.getOrganization();
List<SubInventories> subInventories = (List<SubInventories>)(Object) subInventoryController.getSuvInventories();
Map<Organization,List<SubInventories>> result = new HashMap<Organization,List<SubInventories>>();
for(Organization organization : organizations) {
List<SubInventories> subInventoryMatched = (List<SubInventories>) subInventories.stream().filter( s -> s.getOrganizationId()== organization.getOrganizationId()).collect(Collectors.toList());
result.put(organizations.get(0), subInventoryMatched);
}
return result;
}
}
From what I understand I need to make a wrapper class for each POJO cause the response looks like this
/organizations
{
"items": [
{
"OrganizationId": 1,
"OrganizationCode": "adasd",
"OrganizationName": "Hotel Bahía Príncipe Sunlight Costa Adeje",
"ManagementBusinessUnitId": 131231,
"ManagementBusinessUnitName": "asdasfdas"
},
{
"OrganizationId": 2,
"OrganizationCode": "adadas",
"OrganizationName": "Hadasd",
"ManagementBusinessUnitId": 1231,
"ManagementBusinessUnitName": "aewfrqaew"
}]}
and /subInventories
{
"items": [
{
"OrganizationId": 1,
"OrganizationCode": "asada",
"SecondaryInventoryName": "adfasdfasdgf",
"Description": "pub"
},
{
"OrganizationId": 2,
"OrganizationCode": "asgfrgtsdh",
"SecondaryInventoryName": "B LOB",
"Description": "pub2"
}
]}
If used the generic one with Object I get a java.lang.ClassCastException: java.util.LinkedHashMap incompatible with com.demo.model.Organization in the stream().filter and for the merge of the fields another class to get the desired
{
"items": [
{
"OrganizationId": 1,
"OrganizationCode": "asdas",
"OrganizationName": "adsadasd",
"ManagementBusinessUnitId": 1,
"ManagementBusinessUnitName": "asdasdf",
"SecondaryInventoryName": "sfsdfsfa",
"Description": "pub1"
}]}
Tons of classes if i get lots of POJO
I assume the following from the information you provide:
You have two Datatypes (Java classes). They should be merged together to one Java class
You have to load this data from different sources
Non of the classes are leading
I can provide you some example code. The code is based on the previos adoptions. This will give you an idea. It's not a simple copy and paste solution.
At first create a class with all fields you want to include in the result:
public class Matched {
private Object fieldA;
private Object fieldB;
// Some getter and Setter
}
The Basic idea is that you load your data. Than find the two corresponding objects. After that do your matching for each field.
public List<Matched> matchYourData() {
// load your data
List<DataA> dataAList = loadYourDataA();
List<DataB> dataBList = loadYourDataB();
List<Matched> resultList = new ArryList<>();
for (dataA: DataA) {
DataB dataB = dataBList.stream()
.filter(data -> data.getId() == dataA.getId())
.findFirst().orElseThrow();
// Now you have your data. Let's match them.
Matched matched = new Matched();
matched.setFieldA(dataB.getFieldA() == dataA.getFieldA() ? doSomething() : doSomethingElse());
// Set all your fields. Decide for everyone the matching strategy
resultList.add(matched);
}
return resultList;
}
This is a quite simple solution. Of course you can use Tools like Mapstruct for mapping purpose. But this depends on your environment.

Consider to dec lare/implement a mapping method: "java.lang.String map(java.lang.Object value)" during to use the Framework MapStruct

#Mapper(componentModel = "spring")
public interface MapperThings {
#MapMapping(keyTargetType = Object.class, valueTargetType = Object.class)
Map<String, String> toDto(Map<Object, Object> mapEntity);
List<Map <String, String>> toListDto(Collection<Map<Object, Object>> listEntity);
#MapMapping(keyTargetType = Object.class, valueTargetType = Object.class)
Map<Object, Object> toEntity(Map<String, String> mapDto);
List<Map<Object, Object> > toListEntity(Collection<Map<String, String>> listDto);
}
There is to generate without mistakes only :
#MapMapping(keyTargetType = Object.class, valueTargetType = Object.class)
Map<Object, Object> toEntity(Map<String, String> mapDto);
List<Map<Object, Object> > toListEntity(Collection<Map<String, String>> listDto);
I found temporary decision. But I would want to use the annotation #MapMapping.
#Mapper(componentModel = "spring")
public abstract class MapperMoviesAbstract {
public Map<String, String> toDto(Map<Object, Object> mapEntity) {
if(mapEntity == null) return null;
Map<String, String> stringMap = new HashMap<>();
for(Map.Entry<Object, Object> entry : mapEntity.entrySet()){
String key = (String) entry.getKey();
stringMap.put(key, mapEntity.get(key).toString());
}
return stringMap;
}
public abstract List< Map<String, String>> toListDto(Collection<Map<Object, Object>> listEntity);
}
According to the MapStruct documentation, using the #MapMapping annotation should generate a class that will perform the conversion.
But I get an error:
Can't map map key "java.lang.Object" to "java.lang.String ". Consider
to dec lare/implement a mapping method: "java.lang.String
map(java.lang.Object value)".
Do anyone have any ideas to do what?
The error message is telling you what to do. You need to provide a way to map Object into a String.
So you'll need a custom method like:
public String objectToString(Object object) {
// Your custom implementation
}

Read multiple properties file in one go using Spring Boot?

I went through the link: How to pass a Map<String, String> with application.properties and other related links multiple times, but still its not working.
I'm using Spring Boot and Spring REST example. Link Question: How to by default execute the latest version of endpoint in Spring Boot REST?.
I've created mapping something like this and simply read the mapping
get.customers={GET: '/app-data/customers', VERSION: 'v1'}
post.customers={POST: '/app-data/customers', VERSION: 'v1'}
get.customers.custId={GET: '/app-data/customers/{custId}', VERSION: 'v2'}
Code:
private String resolveLastVersion() {
// read from configuration or something
return "2";
}
Code:
#Component
#ConfigurationProperties
#PropertySource("classpath:restendpoint.properties")
public class PriorityProcessor {
private final Map<String, String> priorityMap = new HashMap<>();
public Map<String, String> getPriority() {
return priorityMap;
}
}
Code:
I suggest the following implementation:
#ConfigurationProperties(prefix="request")
public class ConfigurationProps {
private List<Mapping> mapping;
public List<Mapping> getMapping() {
return mapping;
}
public void setMapping(List<Mapping> mapping) {
this.mapping = mapping;
}
}
Class Mapping will denote the information about the single mapping:
public class Mapping {
private String method;
private String url;
private String version;
public Mapping(String method, String url, String version) {
this.method = method;
this.url = url;
this.version = version;
}
public Mapping() {
}
// getters setters here
}
On the Configuration or spring boot application class (the one with main method):
#EnableConfigurationProperties(ConfigurationProps.class)
In the properties file put:
request.mapping[0].method=get
request.mapping[0].url=/customers
request.mapping[0].version=1
request.mapping[1].method=post
request.mapping[1].url=/students
request.mapping[1].version=2
In Filter (I assume you followed my suggestion from the linked question):
#Component
#Order(1)
public class LatestVersionFilter implements Filter {
private List<Mapping> mappings;
public LatestVersionFilter(ConfigurationProps props) {
this.mappings = props.getMapping();
}
}

How to get parent POJO class field value using streams

I have two classes - Student and StudentDetails.
My goal is to get data in the form of Map <studentName,Map<subjectName, subjectNo>>.
When using streams, not able to fetch data of parent class(Student).
public class Student {
private String studentName;
private Map<String, Long> studentDetails;
public Map<String, Long> getStudentDetails() {
return studentDetails;
}
public String getStudentName() {
return studentName;
}
}
class StudentDetails {
private String subjectName;
private String subjectNo;
public String getFSubjectName() {
return subjectName;
}
public String getSubjectNo() {
return subjectNo;
}
}
public class Main {
public static void main(String[] args) {
Collection<Student> student = null;
Map<String, Map<String, Long>> sts = student.stream()
.map(st -> st.getStudentDetails().values())
.flatMap(st -> st.stream())
.collect(Collectors.groupingBy(Student::getStudentName,
Collectors.toMap(StudentDetails :: getFatherName, StudentDetails:: getRollNo )));
}
}
Error : The type Student does not define getStudentName(T) that is applicable here.
You "lost" your Student while doing:
.map(st -> st.getStudentDetails().values())
.flatMap(st -> st.stream()) // this is a Stream<StudentDetails> now
Thus if you need it just map it:
.flatMap(st -> st.getStudentDetails()
.values()
.stream()
.map(sd -> new SimpleEntry<>(st, sd)))
.collect(Collectors.groupingBy(
en -> en.getKey().getStudentName(),
Collectors.toMap(
en -> en.getValue().getFartherName(),
en -> en.getValue().getRollNo()
)
))
This can be done without Streams too; you could try to look at Map::compute

Resources