Dynamic Filtering of ArrayList with another ArrayList Java 8 - filter

I have a few Search Results Objects as given below:
public class TradeSearchResult{
private String tradeRefNo;
private String relatedTradeId;
private String custodyDate;
private String orderNumber;
private String odrQty;
private String price : 500;
}
public class CollateralTradesSearchResult{
private String excludeTradeOUT;
private String settlementStatus;
private String fundId;
private String altFundId;
private String apNumber;
private String collateralOrderNumber;
private String componenetIdentifier;
}
Now I have a Search Filter Criteria object
public class CRITERION {
protected String field; //The field denotes any field name of the either
SearchResult object
protected String operator; its will be EQUALS or NOT_EQUALS
protected String value; Value of the field.
}
Now I need to write a Dynamic Filter Method where I will pass the List of Criterion object and can pass List of either SearchResult like below
public static List<Object> applyFilter(List<CRITERION> comp, List<?> objectList){
//The CRITERION.fiedName can be same in more than one in the list
return filteredList;
}
here is one example: Consider the below List of
TradeSearchResult{
tradeRefNo : W12343;
relatedTradeId: N993093;
custodyDate : 2018-12-14;
orderNumber : 0000342343;
String odrQty : 12;
String price : 500;
},
{
tradeRefNo : W12344;
relatedTradeId: N993093;
custodyDate : 2018-12-14;
orderNumber : 0000342344;
String odrQty : 18;
String price : 600;
},
{
tradeRefNo : W12345;
relatedTradeId: N993094;
custodyDate : 2018-12-14;
orderNumber : 0000342345;
String odrQty : 20;
String price : 700;
}
Now the Criterion class is like
CRITERION{
field :relatedTradeId;
operator : EQUALS;
value :N993093;
}
{
field :relatedTradeId;
operator : EQUALS;
value :N993094;
}
{
field :orderNumber ;
operator : EQUALS;
value :0000342344;
}
It will only return one result even relatedTradeId has two filter
TradeSearchResult{
tradeRefNo : W12344;
relatedTradeId: N993093;
custodyDate : 2018-12-14;
orderNumber : 0000342344;
String odrQty : 18;
String price : 600;
}
Now in the same applyFIlter Method I can send a list of Criterion and a list of CollateralTradesSearchResult and returns filtered result.
Here is something I tried
public static List<Object> applyFilter(List<CRITERION> criList, List<?> objectList){
long startTime = Calendar.getInstance().getTimeInMillis();
Set<Object> objectSet = new HashSet<>();
for(CRITERION cri : criList){
String fieldName = cri.getFIELD();
objectList.stream().filter(p->beanProperties(p).get(fieldName).equals(cri.getVALUE())).forEachOrdered(objectSet::add);
//objectList.retainAll(objectSet);
//objectSet.clear();
}
List<Object> ret = new ArrayList<>(objectSet);
long endTime = Calendar.getInstance().getTimeInMillis();
System.out.println("Size"+ ret.size());
System.out.println("Time Taken to Search"+ String.valueOf(endTime-startTime));
return ret;
}
Hereis the beanProperties() method
public static Map<String, Object> beanProperties(Object bean) {
try {
Map<String, Object> map = new HashMap<>();
Arrays.asList(Introspector.getBeanInfo(bean.getClass(), Object.class)
.getPropertyDescriptors())
.stream()
// filter out properties with setters only
.filter(pd -> Objects.nonNull(pd.getReadMethod()))
.forEach(pd -> { // invoke method to get value
try {
Object value = pd.getReadMethod().invoke(bean);
if (value != null) {
map.put(pd.getName(), value);
}
} catch (Exception e) {
// add proper error handling here
}
});
return map;
} catch (IntrospectionException e) {
// and here, too
return Collections.emptyMap();
}
}
Any help using Stream or by any means will be helpful.

Create an interface SearchResult and make both your classes implement it. Then create this class:
public class Filter<T extends SearchResult> {
public List<T> applyFilter(List<Criterion> criteria, List<T> list) {
Map<String, Set<String>> allowedValues = new HashMap<>();
Map<String, Set<String>> prohibitedValues = new HashMap<>();
populateValues(criteria, "EQUALS", allowedValues);
populateValues(criteria, "NOT_EQUALS", prohibitedValues);
prohibitedValues.forEach((k, v) -> list.removeIf(t -> v.contains(getFieldValue(k, t))));
allowedValues.forEach((k, v) -> list.removeIf(t -> !v.contains(getFieldValue(k, t))));
return list;
}
private static void populateValues(List<Criterion> criteria, String operator, Map<String, Set<String>> values) {
criteria.stream()
.filter(c -> c.getOperator().equals(operator))
.forEach(c -> {
values.merge(c.getField(), Stream.of(c.getValue()).collect(Collectors.toSet()),
(set1, set2) -> Stream.concat(set1.stream(), set2.stream()).collect(Collectors.toSet()));
});
}
private String getFieldValue(String fieldName, T object) {
Field field;
try {
field = object.getClass().getDeclaredField(fieldName);
} catch (NoSuchFieldException e) {
e.printStackTrace();
return null;
}
field.setAccessible(true);
try {
return (String) field.get(object);
} catch (IllegalAccessException e) {
e.printStackTrace();
return null;
}
}
}
And use it as:
Filter<TradeSearchResult> filter = new Filter<>(); //or CollateralTradesSearchResult
List<TradeSearchResult> filteredList = filter.applyFilter(criteria, searchResults);

Related

How to import data from csv to elasticsearch in java without using logstash?

I want to import data from a csv file to elasticsearch. But I don't want to use logstatsh. So, what are the ways I can do this ?
Any blogs ? Docs ?
I came across TransportClient, but I'm not getting the point from where to start .
Thanks in advance.
very late answer, however :) ….This is for elasticsearch 7.6.0
//this class for keeping csv each row values
public class Document {
private String id;
private String documentName;
private String name;
private String title;
private String dob;
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getDocumentName() {
return documentName;
}
public void setDocumentName(String documentName) {
this.documentName = documentName;
}
public String getName() {
return name1;
}
public void setName(String name1) {
this.name1 = name1;
}
public String getTitle() {
return title;
}
public void setTitle(String title) {
this.title = title;
}
public String getDob() {
return dob;
}
public void setDob(String dob) {
this.dob = dob;
}
}
public void bulkInsert() {
long starttime = System.currentTimeMillis();
logger.debug("ElasticSearchServiceImpl => bulkInsert Service Started");
BufferedReader br = null;
String line = "";
String cvsSplitBy = ",";
BulkRequest request;
Document document;
//elastic Search Index Name
String esIndex = "post";
try {
br = new BufferedReader(new FileReader(<path to CSV>));
request = new BulkRequest();
while ((line = br.readLine()) != null) {
// use comma as separator
String[] row = line.split(cvsSplitBy);
if(row.length >= 1) {
//filling Document object using csv columns array
document = getDocEntity(row);
//adding each filled obect into BulkRequest
request.add(getIndexRequest(document, esIndex));
} else {
logger.info("ElasticSearchServiceImpl => bulkInsert : null row ="+row.toString());
}
}
br.close();
if(request.numberOfActions()>0) {
BulkResponse bulkResponse = client.bulk(request, RequestOptions.DEFAULT);
if(bulkResponse.hasFailures()) {
logger.error("ElasticSearchServiceImpl => bulkInsert : Some of the record has failed.Please reinitiate the process");
} else {
logger.info("ElasticSearchServiceImpl => bulkInsert : Success");
}
} else {
logger.info("ElasticSearchServiceImpl => bulkInsert : No request for BulkInsert ="+request.numberOfActions());
}
} catch (Exception e) {
logger.error("ElasticSearchServiceImpl => bulkInsert : Exception =" + e.getMessage());
}
long endTime = System.currentTimeMillis();
logger.info("ElasticSearchServiceImpl => bulkInsert End" + Util.DB_AVG_RESP_LOG + "" + (endTime - starttime));
}
public static Document getDocEntity(String[] row)throws Exception {
Document document = new Document();
document.setId(UUID.randomUUID().toString());
for(int i=0;i<row.length;i++) {
switch (i) {
case 0:
document.setDocumentName(row[i]);
break;
case 1:
document.setName(row[i]);
break;
case 7:
document.setTitle(row[i]);
break;
case 8:
document.setDob(row[i]);
break;
}
return document;
}
public static IndexRequest getIndexRequest(Document document,String index)throws Exception {
IndexRequest indexRequest = null;
Map<String, Object> jsonMap = new HashMap<>();
jsonMap.put("doc_name",document.getDocumentName());
jsonMap.put("title",document.getTitle());
jsonMap.put("dob",document.getDob());
indexRequest = new IndexRequest(index).id(document.getId()).source(jsonMap);
return indexRequest;
}
If you need to show each response, you can use the following code for responses
for (BulkItemResponse bulkItemResponse : bulkResponse) {
DocWriteResponse itemResponse = bulkItemResponse.getResponse();
switch (bulkItemResponse.getOpType()) {
case INDEX:
case CREATE:
IndexResponse indexResponse = (IndexResponse) itemResponse;
break;
}
}
For more information please read official link

I want to find element present in list and it compare with Enum value

I have a List it has multiple object is present. for every object we have to find some string which compare to Enum value How we get list of particular object
public enum ObjectType {
CONTACT("Contact"),
BANK_DETAILS("Bank-Details"),
EMPLOYMENT("Employment"),
PRODUCT("Product"),
INCOME_DETAIL("Income-Details");
ObjectType(String values) { this.values = values; }
public String getValues() { return values; }
private String values; }
for this i am using below method
List listOutput = errorsList.stream().
filter(e -> e.getObjType().contains(String.valueOf(ObjectType.values())))
.collect(Collectors.toList());
But it return 0 value are present
Try this (adding check method to enum):
public enum ObjectType {
CONTACT("Contact"),
BANK_DETAILS("Bank-Details"),
EMPLOYMENT("Employment"),
PRODUCT("Product"),
INCOME_DETAIL("Income-Details");
private String values;
ObjectType(String values) {
this.values = values;
}
public String getValues() {
return values;
}
public static boolean isValueExists(String test) {
for (ObjectType o : ObjectType.values()) {
if (o.getValues().equals(test)) {
return true;
}
}
return false;
}
}
And then use it:
List listOutput = errorsList.stream().filter(e -> ObjectType.isValueExists(e.getObjType())).collect(Collectors.toList());

spring data jpa dynamic query which has IN clause

I want to create dynamic query in spring data jpa. Doing many search I can implement it, but I came across a problem when I add IN operator in where clause. I need to check id IN (longlist)
Here is my entity class
#Entity
#Table(name = "view_detail")
public class ViewDetailDom {
#Id
#GeneratedValue(strategy=GenerationType.IDENTITY)
private Long id;
private String name;
#Column(name = "user_id")
private Long userId;
private String description;
Here is specification builder class and specification class
public class ViewDetailSpecificationsBuilder {
private final List<SearchCriteria> params;
public ViewDetailSpecificationsBuilder() {
params = new ArrayList<SearchCriteria>();
}
public ViewDetailSpecificationsBuilder with(String key, Operation operation, Object value) {
params.add(new SearchCriteria(key, operation, value));
return this;
}
public Specification<ViewDetailDom> build() {
if (params.size() == 0) {
return null;
}
List<Specification<ViewDetailDom>> specs = new ArrayList<Specification<ViewDetailDom>>();
for (SearchCriteria param : params) {
specs.add(new ViewDetailSpecification(param));
}
Specification<ViewDetailDom> result = specs.get(0);
for (int i = 1; i < specs.size(); i++) {
result = Specifications.where(result).and(specs.get(i));
}
return result;
}
}
public class ViewDetailSpecification implements Specification<ViewDetailDom> {
private SearchCriteria criteria = new SearchCriteria();
public ViewDetailSpecification(SearchCriteria searchCriteria) {
this.criteria.setKey(searchCriteria.getKey());
this.criteria.setOperation(searchCriteria.getOperation());
this.criteria.setValue(searchCriteria.getValue());
}
#Override
public Predicate toPredicate(Root<ViewDetailDom> root, CriteriaQuery<?> query, CriteriaBuilder builder) {
String value = criteria.getValue().toString().replaceAll(" ", "%");
if (criteria.getOperation() != null && criteria.getOperation() != Operation.DEFAULT) {
if (criteria.getOperation() == Operation.GREATHERTHANEQUALTO) {
return builder.greaterThanOrEqualTo(root.<String>get(criteria.getKey()), value);
} else if (criteria.getOperation() == Operation.LESSTHANEQUALTO) {
return builder.lessThanOrEqualTo(root.<String>get(criteria.getKey()), value);
} else if (criteria.getOperation() == Operation.EQUAL) {
return builder.equal(root.<String>get(criteria.getKey()), value);
} else if (criteria.getOperation() == Operation.IN) {
Path<Long> view = root.<Long>get(criteria.getKey());
return view.in(criteria.getValue());
}
} else {
if (root.get(criteria.getKey()).getJavaType() == String.class) {
return builder.like(builder.lower(root.<String>get(criteria.getKey())),
"%" + value.toLowerCase() + "%");
} else {
return builder.equal(root.get(criteria.getKey()), value);
}
}
return null;
}
}
This method creates specification builder:
public ViewDetailSpecificationsBuilder createSearchSpecifications(ViewSearch view) {
ViewDetailSpecificationsBuilder builder = new ViewDetailSpecificationsBuilder();
if (StringUtils.isNotBlank(view.getName())) {
builder.with("name", Operation.DEFAULT, view.getName());
}
if (StringUtils.isNotBlank(view.getDescription())) {
builder.with("description", Operation.DEFAULT, view.getDescription());
}
return builder;
}
And finally I do this:
ViewDetailSpecificationsBuilder builder = createSearchSpecifications(view);
builder.with("userId", Operation.DEFAULT, userSessionHelper.getUserId());
builder.with("id", Operation.IN, viewids);
Specification<ViewDetailDom> spec = builder.build();
viewDetailDao.findAll(spec);
But I am getting following error:
"Unaware how to convert value [[5, 7, 8] : java.util.ArrayList] to requested type [java.lang.Long]; nested exception is java.lang.IllegalArgumentException: Unaware how to convert value [[5, 7, 8] : java.util.ArrayList] to requested type [java.lang.Long]"
I have resolved this problem in this way:
ViewDetailSpecification class:
if (criteria.getOperation() == Operation.IN) {
final List<Predicate> orPredicates = new ArrayList<Predicate>();
List<Long> viewIds = (List<Long>) criteria.getValue();
for (Long viewid : viewIds) {
orPredicates.add(builder.or(builder.equal(root.<String>get(criteria.getKey()), viewid)));
}
return builder.or(orPredicates.toArray(new Predicate[orPredicates.size()]));
}
In kotlin I have the same error, I change the ArrayList to Array, with this code:
fun values(): Array<String> {
val elems = arrayListOf<String>()
return elems.toTypedArray()
}
Try you convert ArrayList to array, for java see: make arrayList.toArray() return more specific types

java 8 nested null check list of objects

I am trying to do a nested null check and then clear the values in map in the nested object if the map is not null.
The following is my hypothetical code. I am wondering if this is the right way to do it or is there a more elegant solution to this.
package exp.myJavaLab.Experiments;
import java.util.*;
public class OptionalTest {
public Inner inner;
public static void main(String[] args) {
OptionalTest testObj = new OptionalTest();
Pojo pojo1 = new Pojo();
pojo1.id = 1;
Map<String, String> dataMap = new HashMap<>();
dataMap.put("a","b");
pojo1.dataMap = dataMap;
Pojo pojo2 = new Pojo();
pojo2.id = 2;
Inner inner = new Inner();
inner.pojoList = Arrays.asList(pojo1, pojo2);
testObj.inner = inner;
System.out.println(testObj);
Optional.ofNullable(testObj.inner)
.map(Inner::getPojoList)
.ifPresent(pojos -> pojos.forEach(pojo -> {
if (pojo.getDataMap() != null) {
pojo.getDataMap().clear();
}
}));
System.out.println(testObj);
}
#Override
public String toString() {
final StringBuilder sb = new StringBuilder("OptionalTest{");
sb.append("inner=").append(inner);
sb.append('}');
return sb.toString();
}
}
class Inner {
public List<Pojo> pojoList;
public List<Pojo> getPojoList() {
return pojoList;
}
#Override
public String toString() {
final StringBuilder sb = new StringBuilder("Inner{");
sb.append("pojoList=").append(pojoList);
sb.append('}');
return sb.toString();
}
}
class Pojo {
public Map<String, String> dataMap;
public int id;
#Override
public String toString() {
final StringBuilder sb = new StringBuilder("Pojo{");
sb.append("dataMap=").append(dataMap);
sb.append(", id=").append(id);
sb.append('}');
return sb.toString();
}
public Map<String, String> getDataMap() {
return dataMap;
}
public int getId() {
return id;
}
}
In my opinion Collections should never be null.
You could declare your pojoList and dataMap as private and instantiate them.
Your class then needs some add-methods. So you are sure getDataMap() never returns null:
class Pojo {
private Map<String, String> dataMap = new HashMap<>();
public Map<String, String> getDataMap() {
return dataMap;
}
public void add(String key, String value) {
dataMap.put(key, value);
}
}
Then you don't need to check for null:
Optional.ofNullable(testObj.inner)
.map(Inner::getPojoList)
.ifPresent(pojos -> pojos.forEach(pojo -> { pojo.getDataMap().clear(); }));

Trying to string.Join an IList and outputting the results to console

Using "string.Join(",", test);" works but for some reason I get an output of:
"Ilistprac.Location, Ilistprac.Location, Ilistprac.Location"
I tried ToString, Convert.ToString, etc and I still get that output.
All the IList interfaces are implemented with the IEnurmerable too (just not listed here unless someone wants me to).
class IList2
{
static void Main(string[] args)
{
string sSite = "test";
string sBldg = "test32";
string sSite1 = "test";
string sSite2 = "test";
Locations test = new Locations();
Location loc = new Location();
test.Add(sSite, sBldg)
test.Add(sSite1)
test.Add(sSite2)
string printitout = string.Join(",", test); //having issues outputting whats on the list
}
}
string printitout = string.Join(",", test.ToArray<Location>);
public class Location
{
public Location()
{
}
private string _site = string.Empty;
public string Site
{
get { return _site; }
set { _site = value; }
}
}
public class Locations : IList<Location>
{
List<Location> _locs = new List<Location>();
public Locations() { }
public void Add(string sSite)
{
Location loc = new Location();
loc.Site = sSite;
loc.Bldg = sBldg;
_locs.Add(loc);
}
private string _bldg = string.Empty;
public string Bldg
{
get { return _bldg; }
set { _bldg = value; }
}
}
You need to supply a useful ToString implementation for Location as Join is calling that for each element. The default implementation will just return the name of the type. See documentation.
So if you have a type like
class SomeType
{
public string FirstName { get; private set; }
public string LastName { get; private set; }
public SomeType(string first, string last)
{
FirstName = first;
LastName = last;
}
public override string ToString()
{
return string.Format("{0}, {1}", LastName, FirstName);
}
}
You need to specify how that should be represented as a string. If you do that, you can use string.Join like this to produce the output below.
var names = new List<SomeType> {
new SomeType("Homer", "Simpson"),
new SomeType("Marge", "Simpson")
};
Console.WriteLine(string.Join("\n", names));
Output:
Simpson, Homer
Simpson, Marge
You have to override ToString() inc your Location class to provide some meaningful output if you want to keep your current approach, E.g.:
public override string ToString()
{
return Site;
}

Resources