how to allow filter in cassandra with java datastax - spring-boot

I created a table and two indexes on it i am not able to do filter query from my spring application
first i created table
CREATE TABLE keyspace.favourite_setting (
id uuid,
user_name text,
filter_name text,
columns LIST<FROZEN<columns_def>>,
filters text,
PRIMARY KEY (id)
);
second i added two indexes
CREATE INDEX IF NOT EXISTS user_name_index ON keyspace.favourite_setting (user_name)
CREATE INDEX IF NOT EXISTS filter_name_index ON keyspace.favourite_setting (filter_name);
then i am trying to filter shows me error you must allow filter although i am doing it already...
My Accessory
#Accessor
public interface FavouriteSettingAccessor {
#Query("select * from favourite_setting where user_name =:userName allow filtering " )
Result<FavouriteSetting> getAllByUserName(#Param("userName") String userName );
}
my Service Impl
#Override
public List<FavouriteSetting> getAllFilterNames(String userName) throws Exception {
session = cassandraFactory.getDataSource();
favouriteSettingAccessor =
new MappingManager(session).createAccessor(FavouriteSettingAccessor.class);
return favouriteSettingAccessor.getAllByUserName(userName).all();
}
My Favourite Setting Model
#Table(keyspace = "keyspace", name = "favourite_setting")
public class FavouriteSetting {
#PartitionKey
#Column(name = "id")
private UUID id;
#Column(name = "user_name")
private String userName;
#Column(name = "filter_name")
private String filterName;
#Column(name = "columns")
private List<ColumnsDef> columns;
#Column(name = "filters")
private String filters;
public FavouriteSetting(UUID id, String userName, String filterName,
List<ColumnsDef> columns, String filters) {
this.id = id;
this.userName = userName;
this.filterName = filterName;
this.columns = columns;
this.filters = filters;
}
public FavouriteSetting() {
}
public UUID getId() {
return id;
}
public void setId(UUID id) {
this.id = id;
}
public String getFilterName() {
return filterName;
}
public void setFilterName(String filterName) {
this.filterName = filterName;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
public List<ColumnsDef> getColumns() {
return columns;
}
public void setColumns(List<ColumnsDef> columns) {
this.columns = columns;
}
public String getFilters() {
return filters;
}
public void setFilters(String filters) {
this.filters = filters;
}
}
Cannot execute this query as it might involve data filtering and thus may have unpredictable performance. If you want to execute this query despite the performance unpredictability, use ALLOW FILTERING

For the query
#Query("select * from favourite_setting where user_name =:userName ... " )
userName is not part of the primary key, you can set it as a clustering key. For more information you can look here or here.
Please note that there is no alter statement to add a clustering key, you will need to recreate the table with:
CREATE TABLE keyspace.favourite_setting (
id uuid,
user_name text,
filter_name text,
columns LIST<FROZEN<columns_def>>,
filters text,
PRIMARY KEY (id, username)
The index created for the username column is considered one of the antipatterns for indexes, as it will be expected to be unique (and therefore, it will have high cardinality); you can find more information of why this is a problem here
allow filtering should never be used, as explained here you should resist the urge to just add ALLOW FILTERING to it. You should think about your data, your model and what you are trying to do.

It works Now By adding user_name As Cluster Key and ADD To The End of Query ALLOW FILTERING ..
Thanks :)

Related

Problem when attempting a saveAndFlush commit (JPA ) when primary key is auto-generated from postGres trigger

I am using spring JPA to attempt to write records to a postGres DB. At the time of the commit, I am getting the following error:
Caused by: org.postgresql.util.PSQLException: ERROR: null value in column "col_id" violates not-null constraint
Detail: Failing row contains (null, null, null, null, null)
I have the following repository interface:
public interface MyRepo extends JpaRepository <MyModel, String> {
}
, the following model class:
#Entity
#Validated
#Table(name = "my_table", schema="common")
public class MyModel {
#Id
#Column(name = "col_id")
private String id;
#Column(name = "second_col")
private String secCol;
#Column(name = "third_col")
private String thirdCol;
#Column(name = "fourth_col")
private String fourthCol;
#Column(name = "fifth_col")
private String fifthCol;
public MyModel() {
}
public MyModel(String id, String secCol, String thirdCol, String fourthCol, String fifthCol) {
this.id = id;
this.secCol = secCol;
this.thirdCol = thirdCol;
this.fourthCol = fourthCol;
this.fifthCol = fifthCol;
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getSecCol() {
return secCol;
}
public void setSecCol(String secCol) {
this.secCol = secCol;
}
public String getThirdCol() {
return thirdCol;
}
public void setThirdCol(String thirdCol) {
this.thirdCol = thirdCol;
}
public String getFourthCol() {
return fourthCol;
}
public void setFourthCol(String fourthCol) {
this.fourthCol = fourthCol;
}
public String getFifthCol() {
return fifthCol;
}
public void setFifthCol(String fifthCol) {
this.fifthCol = fifthCol;
}
}
, and the relevant part of the service class:
public MyModel myModel (MyModel myModel) {
MyModel mm = null;
try {
mm = myRepo.saveAndFlush(myModel);
} catch ( Exception e) {
e.printStackTrace();
}
return mm;
}
UPDATE:
I finally realized that my problem is due to a database trigger that auto-generates primary key against a complex business rule. Also, I assume I might need to use a custom save method rather than the default repo.saveAndFlush? I would be grateful for any ideas given new information. Thanks!
I reproduced the exact same code in a test project with Postgres and it worked well for me. You are absolutely correct that the values of the model class are not populated. You must share your controller also. It may really help me to help you if I can get a look where your service is being called from. Only that will help me to deduce why your model values are being passed as null in the service call.

JHipster - Insert in the database with the GET method

I have to create an application with Jhipster but i never use it before.
When a user send a GET request to the address http://localhost:8080/api/newmesure/{mac-address}/{value}
I want to insert a new mesure in my database.
First i created 3 entity "Plantes", "Capteurs" and "Mesures" with this format :
Image here : https://i.stack.imgur.com/zJqia.png (I'm not allowed to post)
I activated the JPA Filtering to create a #Query to insert data in my database but i read that was not possible.
In /src/main/java/com/mycompany/myapp/web/rest/MesuresRessources.java :
/**
* REST controller for managing {#link com.mycompany.myapp.domain.Mesures}.
*/
#RestController
#RequestMapping("/api")
public class MesuresResource {
private final Logger log = LoggerFactory.getLogger(MesuresResource.class);
private static final String ENTITY_NAME = "mesures";
#Value("${jhipster.clientApp.name}")
private String applicationName;
private final MesuresService mesuresService;
private final MesuresQueryService mesuresQueryService;
public MesuresResource(MesuresService mesuresService, MesuresQueryService mesuresQueryService) {
this.mesuresService = mesuresService;
this.mesuresQueryService = mesuresQueryService;
}
#GetMapping("/newMesure/{mac}/{value}")
public String newMesure(#PathVariable String mac,#PathVariable int value) {
log.debug("Adresse MAC : "+mac);
log.debug("Valeur : "+value);
#Query("SELECT valeur FROM Mesures WHERE id = 1") //not working
Mesures getValeur(); //not working
return "Mesure ajoutée";
}
}
In /src/main/java/com/mycompany/myapp/domain/Mesures.java :
/**
* A Mesures.
*/
#Entity
#Table(name = "mesures")
public class Mesures implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(name = "valeur")
private Integer valeur;
#ManyToOne(optional = false)
#NotNull
#JsonIgnoreProperties("macs")
private Capteurs mac;
// jhipster-needle-entity-add-field - JHipster will add fields here, do not remove
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public Integer getValeur() {
return valeur;
}
public Mesures valeur(Integer valeur) {
this.valeur = valeur;
return this;
}
public void setValeur(Integer valeur) {
this.valeur = valeur;
}
public Capteurs getMac() {
return mac;
}
public Mesures mac(Capteurs capteurs) {
this.mac = capteurs;
return this;
}
public void setMac(Capteurs capteurs) {
this.mac = capteurs;
}
// jhipster-needle-entity-add-getters-setters - JHipster will add getters and setters here, do not remove
#Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof Mesures)) {
return false;
}
return id != null && id.equals(((Mesures) o).id);
}
#Override
public int hashCode() {
return 31;
}
#Override
public String toString() {
return "Mesures{" +
"id=" + getId() +
", valeur=" + getValeur() +
"}";
}
}
Louan
Learning java with JHipster is probably not a wise idea, it uses a very rich technology stack which might lose you unless you invest enough time to learn the basics.
There are many things wrong in your code and approach:
You can't use #Query annotation inside the body of method a of your REST controller, it must be used in your #Repository interface, this code can't compile. See https://www.baeldung.com/spring-data-jpa-query for a quick introduction
JPA filtering is not related to inserting into database
In HTTP/REST, GET method is supposed to be idempotent. For making changes in your database you should use POST or PUT methods. See What is idempotency in HTTP methods?
Your entity naming convention is not consistent: use singular for entity classes because each entity object represents one single instance of Mesure. Here you have Plantes (plural), Capteur (singular) and Mesures (plural). For table names, JHipster uses singular but plural is quite common too because a table holds many rows. Of course, this is just a convention and you or your team may decide to apply another (like a prefix for table names) but the key point is to be consistent.

Android room. error: Cannot figure out how to read this field from a cursor

the SQLite database contains three tables 1) employee 2) skills 3) departments. The idea is this - the employee table stores data such as id, name, last_name, salary. Also, an employee has data such as skill and department, but there can be several data for one employee, so I created two separate skills and departments tables and linked them using the key to the employee table where the primary key for employee is id. Now with the help of id I need to display all the information about employee including his skills which can be several and departments. I implement the whole process using the ROOM library.
Here is the request I make
#Query("SELECT employ.id ,employ.name ,employ.last_name, employ.salary, " +
"skill.skill, department.department_name FROM employ INNER JOIN skill,department " +
"ON employ.id = :id AND skill.employ_id = :id AND department.employ_id = :id ")
AllAboutEmployee getAllAboutEmployee(String id);
Here is the AllAboutEmployee class whose object accepts the result of the request
public class AllAboutEmployee {
#ColumnInfo(name = "id")
private String id;
#ColumnInfo(name = "name")
private String name;
#ColumnInfo(name = "last_name")
private String lastName;
#ColumnInfo(name = "salary")
private String salary;
#ColumnInfo(name = "department_name")
private List<String> departmentsList; // THE ERROR IS ON THIS LINE
#ColumnInfo(name = "skill")
private List<String> skillList; // THE ERROR IS ON THIS LINE
public AllAboutEmployee(String id, String name, String lastName, String salary, List<String> departmentsList, List<String> skillList) {
this.id = id;
this.name = name;
this.lastName = lastName;
this.salary = salary;
this.departmentsList = departmentsList;
this.skillList = skillList;
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getLastName() {
return lastName;
}
public void setLastName(String lastName) {
this.lastName = lastName;
}
public String getSalary() {
return salary;
}
public void setSalary(String salary) {
this.salary = salary;
}
public List<String> getDepartmentsList() {
return departmentsList;
}
public void setDepartmentsList(List<String> departmentsList) {
this.departmentsList = departmentsList;
}
public List<String> getSkillList() {
return skillList;
}
public void setSkillList(List<String> skillList) {
this.skillList = skillList;
}
}
So ther are two fields int the AllAboutEmployee class with the List type, in order to put several skills and several departments there. It is in these fields that an error occurs. Thank you in advance for your help
Wow.. that's so cool.. I was coding all day and got this error too!
You have to create a TypeCoverter to store your data into your Room's Database.
In this case you have two List, that are not types recognizable by Database, so you have to create an Converter for it to store as a String, and another method to do the inverse.
Something like:
class TypeCoverter{
#TypeConverter
fun arrayListToString(arrayList: ArrayList<String>?): String? {
if (arrayList.isNullOrEmpty()) return null
val string = StringBuilder()
for (item in arrayList) {
val isNotTheLastItemInTheArrayList = (item == arrayList.last()).not()
if (isNotTheLastItemInTheArrayList) {
string.append(item).append(COMMA)
} else {
string.append(item)
}
}
return string.toString()
}
}
#TypeConverter
fun stringToArrayList(string: String?): ArrayList<String>? {
when {
string.isNullOrEmpty() -> {
return null
}
string.contains(COMMA).not() -> {
val list = ArrayList<String>()
list.add(string)
return list
}
else -> {
return string.split(COMMA.toRegex()).dropLastWhile { it.isEmpty() } as ArrayList<String>
}
}
}
That's actually in Kotlin, but you can see how it works.

Relationship Exists in neo4j but not in Spring #NodeEntity

I have a class in my domain called Activity that looks like the following
#JsonIdentityInfo(generator=ObjectIdGenerators.PropertyGenerator.class, property="id")
#NodeEntity
public class Activity {
#GraphId
private Long id;
private String title;
private String description;
#Relationship(type = "RELATED_TO", direction = Relationship.UNDIRECTED)
private List<Activity> relatedActivities = new ArrayList<>();
public Activity() {
}
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getTitle() {
return title;
}
public String getDescription() {
return description;
}
public Collection<Activity> getRelatedActivities() {
System.out.println("getting relatedActivities");
System.out.println(relatedActivities);
return relatedActivities;
}
public void addRelatedActivity(Activity activity) {
this.relatedActivities.add(activity);
}
}
I create relationships using the following repository class:
#RepositoryRestResource(collectionResourceRel = "relationships", path = "relationships")
public interface RelationshipRepository extends GraphRepository<Relationship> {
#Query("MATCH (a1:Activity), (a2:Activity) " +
"WHERE a1.title = {0} AND a2.title = {1}" +
"CREATE (a1)-[:RELATED_TO]->(a2)")
void addRelationship(String a1Title, String a2Title);
}
I have verified that this code works using the neo4j browser, which lets me see existing nodes and relationships between them. However, when I access getRelatedActivities() on an Activity object, it's always an empty array, even if that Activity has other Activity nodes related to it, clearly visible in neo4j.
How can I get the relatedActivites on an Activity to automatically populate based on its relationships correctly?
The problem in your code is that you define the "target" as an Activity here
#Relationship(type = "RELATED_TO", direction = Relationship.UNDIRECTED)
private List<Activity> relatedActivities = new ArrayList<>();
but you also have a RelationshipEntity class in your code base: Relationship with the same type RELATED_TO.
When OGM gets the result it tries to match every field but since it converts the relationship type RELATED_TO to the RelationshipEntity and not an Activity object, it does not fill the list in the Activity class.

Spring JPA: Locking parent row when inserting one to many child record

We have two tables that have a one to many relationship. When we insert multiple records into the child table across multiple threads (more specifically across multiple REST web requests) we are running into lost update issues due to a race condition.
What we need to be able to do is have JPA recognize that the entity has been updated elsewhere prior to inserting the child record. I've tried using the #Version annotation approach but that doesn't seem to do the trick as the update/insert (I guess...) is happening on another table. I tried adding a version timestamp column on the parent table that is updated on every update but that didn't seem to do the trick either.
I think what I actually need to do is get a reference to the EntityManager directly so that I can issue a lock() command on the record prior to calling save(). I'm just too new to Spring to know if
A) that is indeed the correct approach,
B) if there is a better/easier way to do what we are trying to accomplish, and
C) how to actually do that.
Also, I am aware of the #OneToMany annotation but that didn't seem to do anything.
I've truncated the code below for brevity and I also created a trimmed down version of the code that demonstrates the problem and will hopefully make it easier to see what I am trying to do. In the test if you change the thread pool number to 1 you can see the test pass.
Engagement class:
#Entity
public class Engagement implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id")
private Long id;
#ElementCollection(fetch = EAGER)
private List<String> assignedUsers;
#Version
private Long version;
private LocalDateTime updatedOn;
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public Long getVersion(){return version;}
public void setVersion(Long version){this.version = version;}
public LocalDateTime getUpdatedOn(){
return updatedOn;
}
public void setUpdatedOn(LocalDateTime updatedOn) {
this.updatedOn = updatedOn;
}
public List<String> getAssignedUsers() {
return assignedUsers;
}
public void setAssignedUsers(List<String> assignedUsers) {
this.assignedUsers = assignedUsers;
}
public Engagement() {
}
}
User class:
public final class User {
private final String name;
private final String email;
private final String userId;
private final List<Engagement> engagements;
#ConstructorProperties({"roles", "name", "email", "userId", "engagements"})
User(String name, String email, String userId, List<Engagement> engagements) {
this.name = name;
this.email = email;
this.userId = userId;
this.engagements = engagements;
}
public static User.UserBuilder builder() {
return new User.UserBuilder();
}
public String getName() {
return this.name;
}
public String getEmail() {
return this.email;
}
public String getUserId() {
return this.userId;
}
public List<Engagement> getEngagements() {
return this.engagements;
}
public static final class UserBuilder {
private String name;
private String email;
private String userId;
private List<Engagement> engagements;
UserBuilder() {
}
public User.UserBuilder name(String name) {
this.name = name;
return this;
}
public User.UserBuilder email(String email) {
this.email = email;
return this;
}
public User.UserBuilder userId(String userId) {
this.userId = userId;
return this;
}
public User.UserBuilder engagements(List<Engagement> engagements) {
this.engagements = engagements;
return this;
}
public User build() {
return new User(this.name, this.email, this.userId, this.engagements);
}
public String toString() {
return "User.UserBuilder(name=" + this.name + ", email=" + this.email + ", userId=" + this.userId + ", engagements=" + this.engagements + ")";
}
}
}
Thread test:
#RunWith(SpringJUnit4ClassRunner.class)
#SpringBootTest
public class EngagementTest {
#Mock
UsersAuthService usersService;
#Autowired
EngagementsRepository engagementsRepository;
UsersAuthService authService;
#Before
public void init() {
MockitoAnnotations.initMocks(this);
authService = new UsersAuthServiceImpl(usersService, engagementsRepository);
}
#Test
public void addingMultipleUsersAtOnceSucceeds() throws InterruptedException {
Long engagementId = 1L;
String userId1 = "user1";
String userId2 = "user2";
String userId3 = "user3";
String userId4 = "user4";
String userId5 = "user5";
String auth = "asdf";
User adminUser = User.builder()
.userId("adminUser")
.email("user#user.com")
.name("Admin User")
.build();
Engagement engagement = new Engagement();
engagement.setAssignedUsers(new ArrayList<>());
engagement.getAssignedUsers().add(adminUser.getUserId());
engagementsRepository.save(engagement);
ExecutorService executorService = Executors.newFixedThreadPool(5);//change this to 1 to see the test pass
List<Callable<Engagement>> callableList = Arrays.asList(
addUserThread(engagementId, userId1, auth, adminUser),
addUserThread(engagementId, userId2, auth, adminUser),
addUserThread(engagementId, userId3, auth, adminUser),
addUserThread(engagementId, userId4, auth, adminUser),
addUserThread(engagementId, userId5, auth, adminUser));
executorService.invokeAll(callableList);
Engagement after = engagementsRepository.findById(engagementId);
assertEquals(6, after.getAssignedUsers().size());
}
private Callable<Engagement> addUserThread(Long engagementId, String userId1, String auth, User adminUser) {
return () -> authService.addUserTo(engagementId, userId1, auth, adminUser);
}
}
What's happening here is that you submit the callbacks for execution but never actually wait for their completion before checking the result. You need to use the List<Future<Engagement>> to actually wait for the results to complete before proceeding.
Something like this would do the trick:
executorService.invokeAll(callableList).forEach(it -> {
try {
it.get(500, TimeUnit.MILLISECONDS);
} catch (InterruptedException | ExecutionException | TimeoutException e) {
e.printStackTrace();
}
});
Note that this is not a proper way to deal with the exception case but it causes the code to wait for completion. If you have that in place you see the threads properly rejecting some of the updates with an ObjectOptimisticLockingFailureException:
java.util.concurrent.ExecutionException: org.springframework.orm.ObjectOptimisticLockingFailureException: Object of class [com.example.racecondition.engagement.Engagement] with identifier [1]: optimistic locking failed; nested exception is org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) : [com.example.racecondition.engagement.Engagement#1]
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:206)
at com.example.racecondition.EngagementTest.lambda$0(EngagementTest.java:68)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at com.example.racecondition.EngagementTest.addingMultipleUsersAtOnceSucceeds(EngagementTest.java:66)
What's weird about the test case beyond that is that UsersAuthServiceImpl carries an #Transactional but the test case manually instantiates that class, so that there's no transactional proxy in place already. This causes the calls to findById(…) and save(…) from within addToUser(…) to run in two transactions. Tweaking that doesn't change the output though.
I think what I actually need to do is get a reference to the EntityManager directly so that I can issue a lock() command on the record prior to calling save(). I'm just too new to Spring to know if
A) that is indeed the correct approach,
If I understand you correctly you want to basically force a version increment on an entity so that if multiple threads do that one fails.
You can indeed achieve that by locking the entity in question using LockModeType.PESSIMISTIC_FORCE_INCREMENT or LockModeType.OPTIMISTIC_FORCE_INCREMENT.
B) if there is a better/easier way to do what we are trying to accomplish, and
C) how to actually do that.
With Spring Data probably the best way to do that is using the #Lock annotation on the method you use to load the entity.

Resources