Please help with this magic I observe here:
Map<String, RenditionMeta> map = cache.asMap();
System.out.println("Before iterating: " + map.containsKey(objectId));
for(String s : map.keySet()) {
if(s.equals(objectId)) {
System.out.println(s + " equals " + objectId + ":" + s.equals(objectId) + "-" + map.containsKey(objectId));
System.out.println(objectId.hashCode());
System.out.println(s.hashCode());
}
}
outputs:
Before iterating: false
09009e5d805f6b0b equals 09009e5d805f6b0b:true-false
1453886923
1453886923
Could someone explain how "true-false" is possible above?
cache is defined as
this.cache = CacheBuilder.newBuilder()
.concurrencyLevel(4)
.weakKeys()
.maximumSize(10000)
.expireAfterWrite(10, TimeUnit.MINUTES)
.build();
When you using weak keys option Guava cache uses identity equality for keys hence behaviour you observe.
CacheBuilder#weakKeys
Related
Dears,
I'm new to Streams and want to convert some logic to using them:
this is the logic:
for (String headerKey : map.keySet()) {
int counter = 1;
for (String value : map.get(headerKey)) {
if (map.get(headerKey).size() != 1) {
System.out.println("Response header: " + headerKey + "[" + counter++ + "]: " + value);
} else {
System.out.println("Response header: " + headerKey + ": " + value);
}
}
}
the problem is the "counter"...
I got as far as:
private static long after(Map<String, List<String>> map) {
return map.keySet().stream().map(key -> output(key, map)).count(); // use print() here instead of output()
}
private static String output(String key, Map<String, List<String>> map) {
counter = 1;
long longString = map.get(key).stream().map(value -> print(value, key, map.get(key))).count();
return "" + longString;
}
private static String print(String value, String key, List<String> strings) {
if (strings.size() != 1) {
System.out.println("Response header: " + key + "[" + counter++ + "]: " + value);
} else {
System.out.println("Response header: " + key + ": " + value);
}
return "";
}
And I suppose I can put the print() method at the indicated spot,
but I don't know how to get the counter to behave as in the original code...
All comment/ideas are welcome :)
Thanks beforehand!
Create a helper method like
static Stream<String> values(List<?> list) {
return list.size() == 1? Stream.of(": " + list.get(0)):
IntStream.range(0, list.size()).mapToObj(ix -> "[" + ix + "]: " + list.get(ix));
}
Instead of re-evaluating the list.size() == 1 condition in each iteration, it selects the right shape of the operation upfront. When the size is not one, rather than trying to maintain a counter, stream over the index range in the first place.
This method now can be used when streaming over the map, like
map.entrySet().stream()
.flatMap(e -> values(e.getValue()).map(("Response header: " + e.getKey())::concat))
.forEach(System.out::println);
I'm using spring boot 2.4 and spring-data-elasticsearch 4.1. I have document like this
#Document(indexName = "test", replicas = 3, shards = 2)
public class TestDocument {
#Id
#Field(type = FieldType.Keyword)
private String Id;
#Field(type = FieldType.Object, enabled = false)
private String name;
...
getters
setters
}
And i want to override hardcoded values in replicas and shards in Document annotation from application.yml for index creation because this values can be differnet by environment of my service. Is there any way to do this?
You can disable auto-creation of an index by using
#Document(indexName = "test", createIndex = false)
and then create the index by using the IndexOperations.create(settings). The following code is taken from Spring Data Elasticsearch tests (https://github.com/spring-projects/spring-data-elasticsearch/blob/4.1.x/src/test/java/org/springframework/data/elasticsearch/core/ElasticsearchTemplateTests.java#L2361-L2384):
public void shouldCreateIndexWithGivenClassAndSettings() {
// given
String settings = "{\n" + " \"index\": {\n" + " \"number_of_shards\": \"1\",\n"
+ " \"number_of_replicas\": \"0\",\n" + " \"analysis\": {\n"
+ " \"analyzer\": {\n" + " \"emailAnalyzer\": {\n"
+ " \"type\": \"custom\",\n"
+ " \"tokenizer\": \"uax_url_email\"\n" + " }\n"
+ " }\n" + " }\n" + " }\n" + '}';
// when
indexOperations.delete();
indexOperations.create(parse(settings));
indexOperations.putMapping(SampleEntity.class);
indexOperations.refresh();
// then
Map<String, Object> map = indexOperations.getSettings();
assertThat(operations.indexOps(IndexCoordinates.of(INDEX_NAME_SAMPLE_ENTITY)).exists()).isTrue();
assertThat(map.containsKey("index.number_of_replicas")).isTrue();
assertThat(map.containsKey("index.number_of_shards")).isTrue();
assertThat((String) map.get("index.number_of_replicas")).isEqualTo("0");
assertThat((String) map.get("index.number_of_shards")).isEqualTo("1");
}
Instead of parsing the JSON for the settings you should create a Document in your code:
Document settings = Document.create();
settings.put("index.number_of_replicas", 42);
settings.put("index.number_of_shards", 42);
you can Add a #Setting annotion to you Entity, which is provided by spring-data-elasticsearch, in order to help you custom index info .
https://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/#elasticsearc.misc.index.settings
I use Query by Example and want to know how I can find objects with certain properties in the nested objects.
A plan anyone?
Here is my example Code:
ExampleMatcher matcher = ExampleMatcher.matching()
.withMatcher("offer2product.id.productId", match -> match.exact()
);
Offer2ProductId id = new Offer2ProductId();
id.setProductId(1337L);
Offer2Product offer2Product = new Offer2Product();
offer2Product.setId(id);
Set<Offer2Product> offer2productSet = new HashSet<>();
offer2productSet.add(offer2Product);
Offer probe = new Offer();
probe.setOffer2productSet(offer2productSet);
Example<Offer> example = Example.of(probe, matcher);
List<Offer> offerList = offerRepository.findAll(example);
Quoting Spring data documentation: https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#query-by-example
Currently, only SingularAttribute properties can be used for property matching.
In your example, you want to search by a property that is a Set<> (offer2productSet), which is a PluralAttribute - it is not possible to search by this field. It will be ignored when building a query, as can be seen here:
https://github.com/spring-projects/spring-data-jpa/blob/master/src/main/java/org/springframework/data/jpa/convert/QueryByExamplePredicateBuilder.java#L112
man actually can do it, follow the example below, where demand has a list of labels, I don't have time to explain everything but soon I'll update this post.
#Repository
#RequiredArgsConstructor
public class DemandaFilterRepository {
private final EntityManager entityManager;
public Page<Demanda> findAll(DemandaFilterDTO demandaFilter, Pageable pageable) {
String query = "select d from Demanda d where 1=1";
Map<String, Object> parameters = new HashMap<>();
if (demandaFilter.getId() != null) {
query += " and d.id = :id";
parameters.put("id", demandaFilter.getId());
}
if (!StringUtils.isEmpty(demandaFilter.getTenant())) {
query += " and upper(d.tenant) = upper(:tenant)";
parameters.put("tenant", demandaFilter.getTenant());
}
if (!StringUtils.isEmpty(demandaFilter.getAssunto())) {
query += " and upper(d.assunto) like upper(:assunto)";
parameters.put("assunto", "%" + demandaFilter.getAssunto() + "%");
}
if (!StringUtils.isEmpty(demandaFilter.getDescricao())) {
query += " and upper(d.descricao) like upper(:descricao)";
parameters.put("descricao", "%" + demandaFilter.getDescricao() + "%");
}
if (!StringUtils.isEmpty(demandaFilter.getEtiqueta())) {
query = query.replace("Demanda d", "Demanda d inner join d.etiquetas etiqueta");
query += " and upper(etiqueta.descricao) = upper(:etiqueta)";
parameters.put("etiqueta", demandaFilter.getEtiqueta());
}
if (!StringUtils.isEmpty(demandaFilter.getNomeDemandante())) {
query += " and upper(d.demandante.nome) like upper(:nomeDemandante)";
parameters.put("nomeDemandante", "%" + demandaFilter.getNomeDemandante() + "%" );
}
if (!StringUtils.isEmpty(demandaFilter.getDtInclusao())) {
query += " d.dtInclusao like :dtInclusao";
parameters.put("dtInclusao", "%" + demandaFilter.getDtInclusao() + "%");
}
query += " ORDER BY d.id DESC ";
TypedQuery<Demanda> typedQuery = entityManager.createQuery(query, Demanda.class)
.setMaxResults(pageable.getPageSize())
.setFirstResult(pageable.getPageNumber() * pageable.getPageSize());
parameters.forEach(typedQuery::setParameter);
return new PageImpl<>(typedQuery.getResultList());
}
}
this is a class for one of a small project that I'm working on ...
The current procedure we now often perform is to extract data from an endpoint, perform computational analysis, generate RDF files and manually load them back into the endpoint.
Now I was looking into automating this procedure using Apache Jena ARQ as these dependencies are currently used for the retrieval of information.
I manage to get it partially to work using INSERT statements but performing thousands if not millions of inserts one by one seem a bit inefficient to me. The second issue is that we sometimes have regex or " in a string and this needs to escaped but there are many exceptions.
Is there a way to either parse or iterate over an internal apache jena model statements and inject this directly into an endpoint?
We currently use graphdb but it would be great if this can be applied using a universal approach.
Update
I have updated the code to handle 10 statements at once not sure yet what eventually the limit will be...
public class EndpointTests extends TestCase {
// Only for local testing
public void testEndpoint() throws Throwable {
String endpoint = "http://10.117.11.77:7200/repositories/Test";
Domain domain = new Domain("file:///Users/jasperkoehorst/diana_interproscan_head.nt");
StmtIterator statements = domain.getRDFSimpleCon().getModel().listStatements();
String strInsert = "INSERT DATA { ";
int insertCounter = 0;
while (statements.hasNext()) {
insertCounter = insertCounter + 1;
Statement statement = statements.nextStatement();
String subject = statement.getSubject().getURI();
String predicate = statement.getPredicate().getURI();
String object = statement.getObject().toString();
if (statement.getObject().isURIResource()) {
object = "<" + statement.getObject().toString() + ">";
}
if (statement.getObject().isLiteral()) {
object = statement.getObject().asLiteral().getString();
object = object.replaceAll("\\\\", "\\\\\\\\");
object = object.replaceAll("\"","\\\\\"");
}
if (object.startsWith("http")) {
object = "<" + object + ">";
} else {
object = "\"" + object + "\"";
}
strInsert = strInsert + "<" + subject + "> <" + predicate + "> " + object + " . ";
if (insertCounter % 10 == 0) {
System.out.println(insertCounter);
strInsert = strInsert + " } ";
UpdateRequest updateRequest = UpdateFactory.create(strInsert);
UpdateProcessor updateProcessor = UpdateExecutionFactory.createRemote(updateRequest, endpoint + "/statements");
updateProcessor.execute();
strInsert = "INSERT DATA { ";
}
}
}
}
I am using BaseX (XML based datastore) for its performance benchmarking. For testing it with ,
TestBeds
I) 10,000 users, 10 friends, 10 resources
II) 100,000 users , 10 friends, 10 resources
I faced below issues:
1) Loading of data is too slow. Gets slowed with eh increase in the number of threads.
2) Plus point - Reading/retriving values from BaseX is faster (17k operation per second)
3) Updating the data in BaseX is very slow. Throughput is ~10 operations per second.
Am I correct to say BaseX is 'TOO' slow for write/update operations (20/sec) compared to read/retrieve (10k/sec)?
Please advice me to make it more efficient for the write and update :
I have a function insertEntity (update or insert function) in to the BaseX datastore as follows -
public int insertEntity(String entitySet, String entityPK,
HashMap<String, ByteIterator> values, boolean insertImage) {
String parentTag ="",childTag ="", key="", entryTag="";
StringBuffer insertData = new StringBuffer();
Set<String> keys = values.keySet();
Iterator<String> iterator = keys.iterator();
while(iterator.hasNext()) {
String entryKey = iterator.next();
if(!(entryKey.equalsIgnoreCase("pic") || entryKey.equalsIgnoreCase("tpic")))
insertData.append("element " + entryKey + " {\"" + StringEscapeUtils.escapeXml(values.get(entryKey).toString()) + "\"},");
}
if(entitySet.equalsIgnoreCase("users")&& insertImage){
byte[] profileImage = ((ObjectByteIterator)values.get("pic")).toArray();
String encodedpImage = DatatypeConverter.printBase64Binary(profileImage);
insertData.append(" element pic {\"" + encodedpImage + "\"},");
profileImage = ((ObjectByteIterator)values.get("tpic")).toArray();
encodedpImage = DatatypeConverter.printBase64Binary(profileImage);
insertData.append(" element tpic {\"" + encodedpImage + "\"},");
}
if(entitySet.equalsIgnoreCase("users"))
{
parentTag = "users";
childTag = "members";
entryTag = "member";
key = "mem_id";
insertData.append("element confirmed_friends {}, element pending_friends {}");
}
if(entitySet.equalsIgnoreCase("resources"))
{
parentTag = "resources";
childTag = "resources";
entryTag = "resource";
key = "rid";
insertData.append("element manipulations {}");
}
try {
session.execute(new XQuery(
"insert node element " + entryTag
+ "{ attribute " + key + "{"
+ entityPK + "}, "
+ insertData.toString()
+ "} "
+ "into doc('" + databaseName + "/" + parentTag +".xml')/" + childTag
));
String q1 = "insert node element " + entryTag
+ "{ attribute " + key + "{"
+ entityPK + "}, "
+ insertData.toString()
+ "} "
+ "into doc('" + databaseName + "/" + parentTag +".xml')/" + childTag;
System.out.println(q1);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return 0;
}
And the below function is acceptFriendship (update function)
public int acceptFriend(int inviterID, int inviteeID) {
// TODO Auto-generated method stub
String acceptFriendQuery1 = "insert node <confirmed_friend id = '"
+ inviterID + "'>"
+ " </confirmed_friend>"
+ "into doc('"+databaseName+"/users.xml')/members/member[#mem_id = '"+inviteeID+"']/confirmed_friends";
String acceptFriendQuery2 = "insert node <confirmed_friend id = '"
+ inviteeID + "'>"
+ " </confirmed_friend>"
+ "into doc('"+databaseName+"/users.xml')/members/member[#mem_id = '"+inviterID+"']/confirmed_friends";
String acceptFriendQuery3 = "delete node doc('"+databaseName+"/users.xml')/members/member[#mem_id = '"
+ inviteeID + "']/pending_friends/pending_friend[#id = '"+ inviterID +"']";
try {
session.execute(new XQuery(acceptFriendQuery1));
session.execute(new XQuery(acceptFriendQuery2));
session.execute(new XQuery(acceptFriendQuery3));
System.out.println("Inviter: "+inviterID +" AND Invitee: "+inviteeID);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return 0;
}