Ok i found myself in a simple but annoying problem. My mongo documents are using java.util.Date as id, and as you might guess the id gets converted (spring converters) to ObjectId, I can't update these documents because every time a new ObjectId(Date) is created get a completely different id even though the date is the same...
how do i force mongo to just use java.util.Date as an id?
providing the sample code:
public void updateNode(...node..) {
final MongoTemplate mongoTemplate = ...
final String collectionName = ...
final Query query = (new Query()).addCriteria(Criteria.where("time").is(node.getTime()));
final Update update = Update.update("time", node.getTime()).set("top", node.getTop())
.set("bottom", node.getBottom()).set("mid", node.getMid())
.set("startTime", node.getStartTime()).set("potential", node.isPotential());
mongoTemplate.upsert(query, update, MyClassNode.class, collectionName);
}
if I ran this code for the first time the objects are inserted into the database but with ObjectId... if the node.getTime() is a java.sql.Date then everything is fine.
if the node.getTime() is not a java.sql.Date I cannot update the document if it exists: why? because everytime the document is prepared it creates a new ObjectId the update and query will have two different _id field values and update fails.
On checking the documentation , i found the following details :
In MongoDB, each document stored in a collection requires a unique _id
field that acts as a primary key. If an inserted document omits the
_id field, the MongoDB driver automatically generates an ObjectId for the _id field.
This also applies to documents inserted through update operations with
upsert: true.
The following are common options for storing values for _id:
Use an ObjectId.
Use a natural unique identifier, if available. This saves space and
avoids an additional index.
Generate an auto-incrementing number.
What i understood from the documentation was that to avoid inserting the same document more than once, only use upsert: true if the query field is uniquely indexed.So, if this flag is set , you will find your id converted using ObjectId() to make it unique.
Related
I'm developing a service that must be able to have a configurable username column in its User field, i.e. different columns can be treated by the service as the "username" (i.e. the actual username column, the ID column etc.). It is a strange requirement to have, but legacy support is a strange thing as you all know by now :)
I've tried to tackle this in a following way, my configuration file contains the name of the column that will be treated as the username, and then that value is used with JPA specifications to find the User (my repository extends JpaSpecificationExecutor).
The code looks something like this:
public UserEntity getUserByUsername(String username) {
String columnName = configuration.getUsernameColumn();
return userRepository.findOne((root, query, builder) ->
builder.and(builder.equal(root.<String>get(columnName), username)));
}
This should work fine... However there is a catch, the column name is specified as the actual column name in the database, and the JPA specification seems to expect field name to be specified, not the database column. My entity is annotated as:
#Column(name = "USER_NAME", length = 100)
private String userName;
So when I try to find the User by searching for "USER_NAME", my code throws an exception because it expected to find a "USER_NAME" field, not column in the database.
I know that the obvious solution is putting "userName" in the configuration instead, however that is not an option. Another way to do this is by using reflection, but that would a last resort approach. Is there a better way to go about this?
One of the another solution can be the criteria query. Pass the column and the searchString and build the criteria query.
Reference : https://eng.zemosolabs.com/dynamic-multi-column-search-with-jpa-criteria-5720fedf13d3
I have a domain object in Spring which I am saving using JpaRepository.save method and using Sequence generator from Postgres to generate id automatically.
#SequenceGenerator(initialValue = 1, name = "device_metric_gen", sequenceName = "device_metric_seq")
public class DeviceMetric extends BaseTimeModel {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "device_metric_gen")
#Column(nullable = false, updatable = false)
private Long id;
///// extra fields
My use-case requires to do an upsert instead of normal save operation (which I am aware will update if the id is present). I want to update an existing row if a combination of three columns (assume a composite unique) is present or else create a new row.
This is something similar to this:
INSERT INTO customers (name, email)
VALUES
(
'Microsoft',
'hotline#microsoft.com'
)
ON CONFLICT (name)
DO
UPDATE
SET email = EXCLUDED.email || ';' || customers.email;
One way of achieving the same in Spring-data that I can think of is:
Write a custom save operation in the service layer that
Does a get for the three-column and if a row is present
Set the same id in current object and do a repository.save
If no row present, do a normal repository.save
Problem with the above approach is that every insert now does a select and then save which makes two database calls whereas the same can be achieved by postgres insert on conflict feature with just one db call.
Any pointers on how to implement this in Spring Data?
One way is to write a native query insert into values (all fields here). The object in question has around 25 fields so I am looking for an another better way to achieve the same.
As #JBNizet mentioned, you answered your own question by suggesting reading for the data and then updating if found and inserting otherwise. Here's how you could do it using spring data and Optional.
Define a findByField1AndField2AndField3 method on your DeviceMetricRepository.
public interface DeviceMetricRepository extends JpaRepository<DeviceMetric, UUID> {
Optional<DeviceMetric> findByField1AndField2AndField3(String field1, String field2, String field3);
}
Use the repository in a service method.
#RequiredArgsConstructor
public class DeviceMetricService {
private final DeviceMetricRepository repo;
DeviceMetric save(String email, String phoneNumber) {
DeviceMetric deviceMetric = repo.findByField1AndField2AndField3("field1", "field", "field3")
.orElse(new DeviceMetric()); // create new object in a way that makes sense for you
deviceMetric.setEmail(email);
deviceMetric.setPhoneNumber(phoneNumber);
return repo.save(deviceMetric);
}
}
A word of advice on observability:
You mentioned that this is a high throughput use case in your system. Regardless of the approach taken, consider instrumenting timers around this save. This way you can measure the initial performance against any tunings you make in an objective way. Look at this an experiment and be prepared to pivot to other solutions as needed. If you are always reading these three columns together, ensure they are indexed. With these things in place, you may find that reading to determine update/insert is acceptable.
I would recommend using a named query to fetch a row based on your candidate keys. If a row is present, update it, otherwise create a new row. Both of these operations can be done using the save method.
#NamedQuery(name="getCustomerByNameAndEmail", query="select a from Customers a where a.name = :name and a.email = :email");
You can also use the #UniqueColumns() annotation on the entity to make sure that these columns always maintain uniqueness when grouped together.
Optional<Customers> customer = customerRepo.getCustomersByNameAndEmail(name, email);
Implement the above method in your repository. All it will do it call the query and pass the name and email as parameters. Make sure to return an Optional.empty() if there is no row present.
Customers c;
if (customer.isPresent()) {
c = customer.get();
c.setEmail("newemail#gmail.com");
c.setPhone("9420420420");
customerRepo.save(c);
} else {
c = new Customer(0, "name", "email", "5451515478");
customerRepo.save(c);
}
Pass the ID as 0 and JPA will insert a new row with the ID generated according to the sequence generator.
Although I never recommend using a number as an ID, if possible use a randomly generated UUID for the primary key, it will qurantee uniqueness and avoid any unexpected behaviour that may come with sequence generators.
With spring JPA it's pretty simple to implement this with clean java code.
Using Spring Data JPA's method T getOne(ID id), you're not querying the DB itself but you are using a reference to the DB object (proxy). Therefore when updating/saving the entity you are performing a one time operation.
To be able to modify the object Spring provides the #Transactional annotation which is a method level annotation that declares that the method starts a transaction and closes it only when the method itself ends its runtime.
You'd have to:
Start a jpa transaction
get the Db reference through getOne
modify the DB reference
save it on the database
close the transaction
Not having much visibility of your actual code I'm gonna abstract it as much as possible:
#Transactional
public void saveOrUpdate(DeviceMetric metric) {
DeviceMetric deviceMetric = metricRepository.getOne(metric.getId());
//modify it
deviceMetric.setName("Hello World!");
metricRepository.save(metric);
}
The tricky part is to not think the getOne as a SELECT from the DB. The database never gets called until the 'save' method.
I am new in writing aggregate queries in Mongo DB + Spring
Scenario: We are storing birthDate(Jjava.uti.Date) in mongo db which got stored as ISO date. Now we are trying to look for the records which are matching with the dayOfMonth and Month only. So that we can corresponding object from the list.
I had gone through few solutions and here is the way I am trying but this is giving me a null set of records.
Aggregation agg = Aggregation.newAggregation(
Aggregation.project().andExpression("dayOfMonth(birthDate)").as("day").andExpression("month(birthDate)")
.as("month"),
Aggregation.group("day", "month"));
AggregationResults<Employee> groupResults = mongoTemplate.aggregate(agg, Employee.class, Employee.class);
I also tried applying a a query with the help of Criteria but this is also giving me a Employee object which all null content.
Aggregation agg = Aggregation.newAggregation(Aggregation.match(Criteria.where("birthDate").lte(new Date())), Aggregation.project().andExpression("dayOfMonth(birthDate)").as("day").andExpression("month(birthDate)")
.as("month"),
Aggregation.group("day", "month"));
AggregationResults<Employee> groupResults = mongoTemplate.aggregate(agg, Employee.class, Employee.class);
I must missing some important thing which is giving me these null data.
Additional Info: Employee object has only birthDate(Date) and email(String) in it
Please try to specify the fields to be included in the $project stage.
project("birthDate", "...").andExpression("...
The _id field is, by default, included in the output documents. To include any other fields from the input documents in the output documents, you must explicitly specify the inclusion in $project.
see: MongoDBReference - $project (aggregation)
I've created DATAMONGO-2200 to add an option to project directly onto the fields of a given domain type via something like project(Employee.class).
I have domain object Person with date fields:
public class Person {
#Id
private Long id;
private Date date
Build example like this:
Person person = new Person();
person.setSomeOtherFields("some fields");
Example<Person> example = Example.of(person);
How i can create example query with date range (search entity contains date greater or equal from some date and less or equal some another date)?
The Spring Data JPA query-by-example technique uses Examples and ExampleMatchers to convert entity instances into the underlying query. The current official documentation clarifies that only exact matching is available for non-string attributes. Since your requirement involves a java.util.Date field, you can only have exact matching with the query-by-example technique.
You could write your own ExampleMatcher that returns query clauses according to your needs.
I have a secondary index on an optional column:
class Sessions extends CassandraTable[ConcreteSessions, Session] {
object matchId extends LongColumn(this) with PartitionKey[Long]
object userId extends OptionalLongColumn(this) with Index[Option[Long]]
...
}
However, the indexedToQueryColumn implicit conversion is not available for optional columns, so this does not compile:
def getByUserId(userId: Long): Future[Seq[Session]] = {
select.where(_.userId eqs userId).fetch()
}
Neither does this:
select.where(_.userId eqs Some(userId)).fetch()
Or changing the type of the index:
object userId extends OptionalLongColumn(this) with Index[Long]
Is there a way to perform such a query using phantom?
I know that I could denormalize, but it would involve some very messy housekeeping and triple our (substantial) data size. The query usually returns only a handful of results, so I'd be willing to use a secondary index in this case.
Short answer: You could not use optional fields in order to query things in phantom.
Long detailed answer:
But, if you really want to work with secondary optional columns, you should declare your entity field as Option but your phantom representation should not be an option in order to query.
object userId extends LongColumn(this) with Index[Long]
In the fromRow(r: Row) you can create your object like this:
Sessions(matchId(r), Some(userId(r)))
Then in the service part you could do the following:
.value(_.userId, t.userId.getOrElse(0))
You also have a better way to do that. You could duplicate the table, making a new kind of query like sessions_by_user_id where in this table your user_id would be the primary key and the match_id the clustering key.
Since user_id is optional, you would end with a table that contains only valid user ids, which is easy and fast to lookup.
Cassandra relies on queries, so use it in your favor.
Take a look up on my github project that helps you get up with multiple queries in the same table.
https://github.com/iamthiago/cassandra-phantom