I am using Hibernate Search with Spring Boot to create a searchable rest api. Trying to POST an instance of "Training", I receive the following stack traces. None of the two are very insightful to me which is why I am reaching out for help.
Stack trace:
https://pastebin.com/pmurg1N3
It appears to me that it is trying to index a null entity!? How can that happen? Any ideas?
The entity:
#Entity #Getter #Setter #NoArgsConstructor
#ToString(onlyExplicitlyIncluded = true)
#Audited #Indexed(index = "Training")
#AnalyzerDef(name = "ngram",
tokenizer = #TokenizerDef(factory = StandardTokenizerFactory.class ),
filters = {
#TokenFilterDef(factory = StandardFilterFactory.class),
#TokenFilterDef(factory = LowerCaseFilterFactory.class),
#TokenFilterDef(factory = StopFilterFactory.class),
#TokenFilterDef(factory = NGramFilterFactory.class,
params = {
#Parameter(name = "minGramSize", value = "2"),
}
)
}
)
#Analyzer(definition = "ngram")
public class Training implements BaseEntity<Long>, OwnedEntity {
#Id
#GeneratedValue
#ToString.Include
private Long id;
#NotNull
#RestResourceMapper(context = RestResourceContext.IDENTITY, path = "/companies/{id}")
#JsonProperty(access = Access.WRITE_ONLY)
#JsonDeserialize(using = RestResourceURLSerializer.class)
private Long owner;
#NotNull
#Field(index = Index.YES, analyze = Analyze.YES, store = Store.YES)
private String name;
#Column(length = 10000)
private String goals;
#Column(length = 10000)
private String description;
#Enumerated(EnumType.STRING)
#Field(index = Index.YES, store = Store.YES, analyze = Analyze.NO, bridge=#FieldBridge(impl=EnumBridge.class))
private Audience audience;
#Enumerated(EnumType.STRING)
#Field(index = Index.YES, store = Store.YES, analyze = Analyze.NO, bridge=#FieldBridge(impl=EnumBridge.class))
private Level level;
#ManyToMany
#Audited(targetAuditMode = RelationTargetAuditMode.NOT_AUDITED)
#NotNull #Size(min = 1)
#IndexedEmbedded
private Set<ProductVersion> versions;
#NotNull
private Boolean enabled = false;
#NotNull
#Min(1)
#IndexedEmbedded
#Field(index = Index.YES, store = Store.YES, analyze = Analyze.NO)
#NumericField
private Integer maxStudents;
#NotNull
#ManyToOne(fetch = FetchType.LAZY)
private Agenda agenda;
#NotNull
#Min(1)
#Field(index = Index.YES, store = Store.YES, analyze = Analyze.NO)
#NumericField
private Integer durationDays;
#IndexedEmbedded
#Audited(targetAuditMode = RelationTargetAuditMode.NOT_AUDITED)
#ManyToMany(cascade = CascadeType.PERSIST)
private Set<Tag> tags = new HashSet<>();
I'd say either your versions collection or your tags collection contains null objects, which is generally not something we expect in a Hibernate ORM association, and apparently not something Hibernate Search expects either.
Can you check that in debug mode?
Related
I ve document named plan that correspond plan entity
#Entity
#Table(name = "plan")
#Cache(usage = CacheConcurrencyStrategy.READ_WRITE)
#org.springframework.data.elasticsearch.annotations.Document(indexName = "plan")
public class Plan extends AbstractAuditingEntity implements Serializable {
#Id
#GeneratedValue
#Column(name = "id")
#Field(type = FieldType.Text , fielddata = true)
private UUID id;
#NotNull
#Column(name = "name", nullable = false, unique = true)
private String name;
#Column(name = "description")
private String description;
#Column(name = "current_price")
#Field(type = FieldType.Text , fielddata = true )
private Float currentPrice;
}
Here my method search implementation
public Page<Plan> search(String query, Pageable pageable) {
NativeSearchQuery nativeSearchQuery = new NativeSearchQuery(queryStringQuery("*"+query+"*").defaultOperator(Operator.AND));
nativeSearchQuery.setPageable(pageable);
List<Plan> hits = elasticsearchTemplate
.search(nativeSearchQuery, Plan.class)
.map(SearchHit::getContent)
.stream()
.collect(Collectors.toList());
return new PageImpl<>(hits, pageable, hits.size());
}
Name and Description are searchable but float field isn't .
Marking as FieldType.Float doesn't give expected result .
I have a entity for product:
package com.javaschool.entity;
import lombok.*;
import javax.persistence.*;
import java.util.Set;
#EqualsAndHashCode(of = {"id"})
#ToString(of = { "id", "quantity", "price", "model"})
#Entity
#Table(name = "products")
#Data
#NoArgsConstructor
#AllArgsConstructor
public class Product {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private long id;
#Column(name = "quantity")
private int quantity;
#Column(name = "price")
private int price;
#Column(name = "model")
private String model;
#Column(name = "is_active")
private boolean active;
#Column(name = "picture_url")
private String url;
#ManyToOne(fetch = FetchType.EAGER, optional = false)
#JoinColumn(name = "category_id")
private Category category;
#ManyToOne(fetch = FetchType.EAGER, optional = false)
#JoinColumn(name = "brand_id")
private Brand brand;
#ManyToOne(fetch = FetchType.EAGER, optional = false)
#JoinColumn(name = "season_id")
private Season season;
#ManyToOne(fetch = FetchType.EAGER, optional = false)
#JoinColumn(name = "color_id")
private Color color;
#ManyToOne(fetch = FetchType.EAGER, optional = false)
#JoinColumn(name = "material_id")
private Material material;
#ManyToOne(fetch = FetchType.EAGER, optional = false)
#JoinColumn(name = "size_id")
private Size size;
#ManyToMany(mappedBy = "productSet", fetch = FetchType.LAZY, cascade = {CascadeType.PERSIST, CascadeType.MERGE})
private Set<Order> orderSet;
}
I want to filter by category, season, color, brand and other related parameters
At the moment my filtering function looks like this. It works for parameters such as model, price, quantity. That is, for those that are data in this table and not from others. How can I filter by parameters that are taken from other tables?
#Override
public List<Product> findByParam(List<SearchCriteria> params) {
CriteriaBuilder criteriaBuilder = entityManager.getCriteriaBuilder();
CriteriaQuery<Product> criteriaQuery = criteriaBuilder.createQuery(Product.class);
Root<Product> root = criteriaQuery.from(Product.class);
Predicate predicate = criteriaBuilder.conjunction();
ProductSearchQueryCriteriaConsumer productConsumer = new ProductSearchQueryCriteriaConsumer(predicate, criteriaBuilder, root);
params.stream().forEach(productConsumer);
predicate = productConsumer.getPredicate();
criteriaQuery.where(criteriaBuilder.equal(root.get(Product_.active), true),
predicate);
List<Product> result = entityManager.createQuery(criteriaQuery).getResultList();
return result;
}
I thought that you can make such a call and everything will work. But I was wrong.
List<SearchCriteria> params = new ArrayList<SearchCriteria>();
params.add(new SearchCriteria("season_id", ":", "3"));
List<ProductDto> productDtoList = productService.getProductsByParam(params);
My SearchCriteria
#Data
#AllArgsConstructor
#NoArgsConstructor
public class SearchCriteria {
private String key;
private String operation;
private Object value;
}
Need to make this:
List<SearchCriteria> params = new ArrayList<SearchCriteria>();
params.add(new SearchCriteria("category", ":", categoryRepository.findById(1)));
That is, in the searchcriteria for the value object, pass an object of this class to filter by
Order Entity
#Entity
#Table(name = "Order",
indexes = {
#Index(name = "ORDER_X1", columnList = "REFERENCE_ID,SOURCE_ID"),
#Index(name = "ORDER_X2", columnList = "TYPE,STATUS")
}
)
#DiscriminatorColumn(name="PROCESSOR_TYPE", discriminatorType=DiscriminatorType.STRING, length = 80)
#SequenceGenerator(name="orderSeq", sequenceName="ORDER_SEQ")
#Inheritance(strategy= InheritanceType.JOINED)
public abstract class OrderEntity implements Serializable {
#Id
#GeneratedValue(strategy= GenerationType.SEQUENCE, generator="orderSeq")
private Long id;
#ManyToMany(cascade={CascadeType.MERGE})
#JoinTable(
name = "FILE_ORDER_MAP",
joinColumns = {#JoinColumn(name = "ORDER_ID")},
inverseJoinColumns = {#JoinColumn(name = "FILE_ID")}
)
private Set<TransferFile> transferFiles = new HashSet<>();
#Column(name = "TYPE")
#Enumerated(EnumType.STRING)
private OrderType type;
#Column(name = "AMOUNT", precision = 12, scale = 2)
private LcMoney amount;
#Column(name = "STATUS")
#Enumerated(EnumType.STRING)
private OrderStatus reconStatus;
#Type(type = LcUtc.JPA_JODA_TIME_TYPE)
#Column(name = "STATUS_D", nullable = false)
#LcDateTimeUtc()
private DateTime reconStatusDate;
#Column(name = "REFERENCE_ID")
private Long referenceId;
#Column(name = "SOURCE_ID")
private Long sourceId;
#Column(name = "ACCOUNT_ID")
private Long accountId;
#Column(name = "PROCESSOR_TYPE", insertable = false, updatable = false)
#Enumerated(EnumType.STRING)
private OrderProcessorType processorType;
#Type(type = LcUtc.JPA_JODA_TIME_TYPE)
#Column(name = "TX_EXECUTION_D")
#LcDateTimeUtc()
private DateTime executedDate;
#Type(type = LcUtc.JPA_JODA_TIME_TYPE)
#Column(name = "CREATE_D")
#LcDateTimeUtc()
private DateTime createDate;
#Column(name = "IS_ON_DEMAND")
#Type(type = "yes_no")
private boolean isOnDemand;
#ManyToOne(fetch = FetchType.LAZY, optional = true, cascade = {CascadeType.PERSIST})
#JoinColumn(name="PAYER_ID", nullable=true)
private Payer payer;
#OneToMany(cascade = CascadeType.ALL)
#JoinColumn(name = "ORDER_ID", referencedColumnName = "ID")
private List<OrderTransaction> orderTransactions;
#OneToMany(cascade = {CascadeType.ALL})
#JoinColumn(name = "ORDER_ID", referencedColumnName = "ID",
foreignKey = #ForeignKey(name = "FK_ORDER")
)
private List<MatchResult> matchResults;
#Version
#Column(name = "VERSION")
private Integer version;
#Embedded
#AttributeOverrides({
#AttributeOverride(name = "externalSourceId", column = #Column(name = "TRANS_EXT_SRC_ID")),
#AttributeOverride(name = "externalId", column = #Column(name = "TRANS_EXT_REF_ID"))
})
private LcExternalIdEntity transExtId;
#PreUpdate
#PrePersist
public void beforePersist() {
if (reconStatusDate != null) {
reconStatusDate = reconStatusDate.withZone(DateTimeZone.UTC);
}
if (executedDate != null) {
executedDate = executedDate.withZone(DateTimeZone.UTC);
}
if (createDate != null) {
createDate = createDate.withZone(DateTimeZone.UTC);
}
}
// getters and setters
}
//controller method
public Response processFile(){
// separate trasaction
service.readFileAndCreateOrders(); // read files and create orders in new status
List<Order> newOrders = service.getNewOrders();
for( Order order: newOrders){
service.processOrder(order); // separate transaction
}
}
#Transaction
void processOrder(OrderEntity order){
matchResultJpaRepository.save(orderEntity.id);
log.info("Saving matchId={} for order={}", match.getId(), order.getId());
// create new transaction and add to order
OrderTransaction transaction = createNewTransaction(order);
order.getTransactions().add(transaction);
order.setStatus("PROCESSED");
log.info("Saving Order id={}, Type={}, Status={} ", order.getId(), order.getType(), order.getStatus());
orderRepository.save(order);
}
I am seeing this below error.
ORA-01407: cannot update ("PAYMENTS"."MATCH_RESULT"."ORDER_ID") to NULL
This endpoing is not exposed to user. There is a batch job which invokes this endpoint.
This code has been there for atleast a year and this is the first time i am seeing this.
This happened only once and for only one call. I am seeing both the logs printed. I am puzzled why I am seeing above error complaining about NULL order id. From the logs, we can confirm that the order id is definitely not null.
Any idea why this is happening? What can be done to fix this?
We have the following situation
Given the following two entities
#Indexed
#Spatial(spatialMode = SpatialMode.HASH)
#Entity
#Table(name = "address")
Address{
#Field
#Basic
#Column(name = "state")
private String state;
#Field
#Basic
#Column(name = "town_city")
private String townCity;
#Field
#Longitude
#Basic
#Column(name = "x_coord")
private Double xCoord;
#Field
#Latitude
#Basic
#Column(name = "y_coord")
private Double yCoord;
}
And
#Indexed
#Entity
#Table(name = "person")
Person{
#Field
#Column(name = "weight")
private Double weight;
#Column(name = "age")
private Integer age;
#org.hibernate.annotations.Cache(usage =
org.hibernate.annotations.CacheConcurrencyStrategy.READ_WRITE)
#ManyToMany
#Cascade({org.hibernate.annotations.CascadeType.SAVE_UPDATE})
#JoinTable(name = "person_address",
joinColumns = {#JoinColumn(name = "person_id")},
inverseJoinColumns = {#JoinColumn(name = "address_id")})
private Set<Address> addressSet = new HashSet<>();
}
Getters and Setters rest of the fields omitted
We want to return in our search results as an example people within a 5KM radius of a given position who are also within an age range.
So
FullTextSession fullTextSession = Search.getFullTextSession(entityManagerFactory.unwrap(SessionFactory.class).openSession());
this.queryBuilder = fullTextSession.getSearchFactory()
.buildQueryBuilder().forEntity(Person.class)
.overridesForField("identifiers.identifier_edge", "identifier_query_analyzer")
.get();
this.bool = queryBuilder.bool();
LocalDateTime lowerLocalDateTime = localDateTime.withYear(localDateTime.getYear() - upperAge);
lowerDate = Date.from(lowerLocalDateTime.atZone(ZoneId.systemDefault()).toInstant());
LocalDateTime upperLocalDateTime = localDateTime.withYear(localDateTime.getYear() - lowerAge);
upperDate = Date.from(upperLocalDateTime.atZone(ZoneId.systemDefault()).toInstant());
bool.must(getQueryBuilder().range().onField("datesOfBirth.dateOfBirth").from(lowerDate).to(upperDate).createQuery());
which will give us people within the relevant age range
We have a separte query to get the address id's within a radius around a given point
public Set<Integer> getSpatialAddressResults(SpatialSearchCommand spatialSearchCommand) {
FullTextSession fullTextSession = Search.getFullTextSession(entityManagerFactory.unwrap(SessionFactory.class).openSession());
this.userSearchPreference = userSearchPreference;
this.queryBuilder = fullTextSession.getSearchFactory()
.buildQueryBuilder().forEntity(Address.class)
.get();
this.bool = queryBuilder.bool();
Set<Integer> addressIdSet = new HashSet<>();
bool.must(getQueryBuilder().spatial()
.within(spatialSearchCommand.getRadius(), Unit.KM).ofLatitude
(spatialSearchCommand.getLat()).andLongitude(spatialSearchCommand.getLng()).createQuery());
FullTextQuery fullTextQuery =
fullTextSession.createFullTextQuery(bool.createQuery(), Address.class)
.setProjection("addressId")
.initializeObjectsWith(ObjectLookupMethod.SECOND_LEVEL_CACHE,
DatabaseRetrievalMethod.QUERY);
List results = fullTextQuery.list();
for (Object result : results) {
Object[] arrayResult = (Object[]) result;
addressIdSet.add(((Integer) arrayResult[0]));
}
if (addressIdSet.size() == 0) {
addressIdSet.add(-1);
}
return addressIdSet;
}
Which we use like below (in reality these are done in separate classes but for simplicity I have just shown the relevant code
Set<Integer> localAddressIds = getSpatialAddressResults(new SpatialSearchCommand(userSearchPreference.getRadius(), userSearchPreference.getLat(), userSearchPreference.getLng()));
if(localAddressIds.size() > 0){
BooleanJunction<BooleanJunction> localSquQueryBool = getQueryBuilder().bool();
for (Integer localAddressId : localAddressIds) {
localSquQueryBool.should(getQueryBuilder().keyword().onField("currentLocation.address.indexId").matching(localAddressId).createQuery());
if(!personSearchCommand.getCurrentOnly()){
localSquQueryBool.should(getQueryBuilder().keyword().onField("locations.address.indexId").matching(localAddressId).createQuery());
}
}
bool.must(localSquQueryBool.createQuery());
}
The problem is there can be a huge amount of addresses returned which results in a BooleanQueryTooManyClauses: maxClauseCount is set to 1024
The question really is what is the best way to combined queries on two different indexed entities to avoid problems like above.
Essentially, you are trying to implement a join operation. As you can see, there are technical challenges to joins which are not easily solved on the client side.
Generally, the recommended approach in Elasticsearch and Lucene is to avoid joins when you can. Instead, you will de-normalize your schema: within the document representing each person, embed a copy of every address. Then you will be able to express all your constraints in a single query targeting the person index.
This is done by annotating the addresses property in Person with #IndexedEmbedded.
Now, as you can imagine, this de-normalization comes at a cost: whenever an address is changed, Hibernate Search has to update the relevant person.
To that end, you will need to add a List<Person> property to your Address class and annotate it with #ContainedIn in order for Hibernate Search to be able to fetch the persons to reindex whenever an address is modified.
In short, change your model to this:
//#Indexed // No longer needed
#Spatial(spatialMode = SpatialMode.HASH, name = "location") // Give a name to the spatial field
#Entity
#Table(name = "address")
Address {
// Add this
#ManyToMany(mappedBy = "addressSet")
#ContainedIn
private Set<Person> personSet = new HashSet<>();
#Field
#Basic
#Column(name = "state")
private String state;
#Field
#Basic
#Column(name = "town_city")
private String townCity;
//#Field// This is not necessary
#Longitude
#Basic
#Column(name = "x_coord")
private Double xCoord;
//#Field// This is not necessary
#Latitude
#Basic
#Column(name = "y_coord")
private Double yCoord;
}
#Indexed
#Entity
#Table(name = "person")
Person{
#Field
#Column(name = "weight")
private Double weight;
#Column(name = "age")
private Integer age;
#org.hibernate.annotations.Cache(usage =
org.hibernate.annotations.CacheConcurrencyStrategy.READ_WRITE)
#ManyToMany
#Cascade({org.hibernate.annotations.CascadeType.SAVE_UPDATE})
#JoinTable(name = "person_address",
joinColumns = {#JoinColumn(name = "person_id")},
inverseJoinColumns = {#JoinColumn(name = "address_id")})
#IndexedEmbedded // Add this
private Set<Address> addressSet = new HashSet<>();
#Transient
#IndexedEmbedded // Also add this
public Address getCurrentAddress() {
// This was missing in your schema, I suppose it's a getter that picks the current address from addressSet?
}
}
Then reindex. Your Person documents will now have two new fields: addressSet.location and currentAddress.location.
Then write your query like this:
FullTextSession fullTextSession = Search.getFullTextSession(entityManagerFactory.unwrap(SessionFactory.class).openSession());
this.queryBuilder = fullTextSession.getSearchFactory()
.buildQueryBuilder().forEntity(Person.class)
.overridesForField("identifiers.identifier_edge", "identifier_query_analyzer")
.get();
this.bool = queryBuilder.bool();
LocalDateTime lowerLocalDateTime = localDateTime.withYear(localDateTime.getYear() - upperAge);
lowerDate = Date.from(lowerLocalDateTime.atZone(ZoneId.systemDefault()).toInstant());
LocalDateTime upperLocalDateTime = localDateTime.withYear(localDateTime.getYear() - lowerAge);
upperDate = Date.from(upperLocalDateTime.atZone(ZoneId.systemDefault()).toInstant());
bool.must(getQueryBuilder().range().onField("datesOfBirth.dateOfBirth").from(lowerDate).to(upperDate).createQuery());
SpatialSearchCommand spatialSearchCommand = new SpatialSearchCommand(userSearchPreference.getRadius(), userSearchPreference.getLat(), userSearchPreference.getLng());
// The magic happens below
BooleanJunction<BooleanJunction> localSquQueryBool = getQueryBuilder().bool();
localSquQueryBool.should(getQueryBuilder().spatial()
.onField("currentAddress.location")
.within(spatialSearchCommand.getRadius(), Unit.KM)
.ofLatitude(spatialSearchCommand.getLat())
.andLongitude(spatialSearchCommand.getLng())
.createQuery());
if(!personSearchCommand.getCurrentOnly()) {
localSquQueryBool.should(getQueryBuilder().spatial()
.onField("addressSet.location")
.within(spatialSearchCommand.getRadius(), Unit.KM)
.ofLatitude(spatialSearchCommand.getLat())
.andLongitude(spatialSearchCommand.getLng())
.createQuery());
}
bool.must(localSquQueryBool.createQuery());
I have two tables i.e. users and events. Users table will be filled when new user will sign up. Later same user can create calendar events. so events table will be filled and users_events will keep mapping of events based on user.
I would like to find all events based on logged in userId. so here is query, it should return data based on it.
select * from events where eventid in (select eventId from users_event where id_user=x ). Here is my Users and Event Entity class.
User.java
#Entity
#Table(name = "users")
public class User {
#Id
#GenericGenerator(name = "generator", strategy = "increment")
#GeneratedValue(generator = "generator")
#Column(name = "id", nullable = false)
private Long id;
#Column(name = "first_name", nullable = false)
private String firstName;
#Column(name = "family_name", nullable = false)
private String familyName;
#Column(name = "e_mail", nullable = false)
private String email;
#Column(name = "phone", nullable = false)
private String phone;
#Column(name = "language", nullable = false)
private String language;
#Column(name = "id_picture")
private String pictureId;
#Column(name = "login", nullable = false)
private String login;
#Column(name = "password", nullable = false)
private String password;
#Column(name = "birth_date")
private Date birthDate;
#Column(name = "enabled")
private Boolean enabled;
//getter and setter
Event.java
#Entity
#Table(name = "events")
public class Event {
#Id
#GenericGenerator(name = "generator", strategy = "increment")
#GeneratedValue(generator = "generator")
#Column(name = "eventId", nullable = false)
private Long eventId;
#Column(name = "title", nullable = false)
private String title;
#Column(name = "description", nullable = true)
private String description;
#Column(name = "startAt", nullable = false)
#Temporal(TemporalType.TIMESTAMP)
private Date startAt;
#Column(name = "endAt", nullable = true)
#Temporal(TemporalType.TIMESTAMP)
private Date endAt;
#Column(name = "isFullDay", nullable = false)
private Boolean isFullDay;
#ManyToMany(fetch = FetchType.EAGER)
#JoinTable(name = "users_event", joinColumns = { #JoinColumn(name = "id_event", referencedColumnName = "eventId") }, inverseJoinColumns = { #JoinColumn(name = "id_user", table = "users", referencedColumnName = "id") })
private Set<User> user = new HashSet<User>();
/getter and setter
EventRepo.java
public interface EventRepo extends CrudRepository<Event, Long> {
Event findByUser(Set<User> user);
}
I am trying to implement something, which can give me output of this query.
select * from events where eventid in (select eventId from users_event where id_user=x )
here is my implementation.any input please?
#RequestMapping(value = "/events", method = RequestMethod.GET)
public #ResponseBody List<Event> getEvents() {
logger.debug("get event list");
User x=new User();
x.setId(1);
Set<User> user= new HashSet();
user.add(x);
return (List<Event>) eventRepo.findByUser(user);
}
Just add a following method to your EventRepo:
List<Event> findAllByUserId(Long userId);
And modify your controller to something like this:
#RequestMapping(value = "/events", method = RequestMethod.GET)
public List<Event> getEvents() {
return eventRepo.findAllByUserId(1L);
}