I have more than a million documents in object store, and I want to know the count of documents for a specific time period. How can I get the count using FileNet CE api's
The code I use is below, which gives me only a maximum of 200 documents.
--Code
SearchScope scope= new SearchScope(obj);
SearchSQL sql= new SearchSQL();
sql.setMaxRecords(100000);
String query="select * from document where datecreated >(date)";
RepositoryRowSet res= scope.fetchRows(sql,1000,null,null);
int count=0;
PageIterator p= result.pageIterator();
while(p.nextPage){
count+=p.getElementCount();a
}
It is possible to use COUNT() function in background searches:
select COUNT(Id) from Document
Link to SQL syntax for background search query
Working with background search queries via API
Or, you can use a direct database connection and find the count of documents using documented database tables schema from DocVersion table.
Table schema - DocVersion
Related
I am new in writing aggregate queries in Mongo DB + Spring
Scenario: We are storing birthDate(Jjava.uti.Date) in mongo db which got stored as ISO date. Now we are trying to look for the records which are matching with the dayOfMonth and Month only. So that we can corresponding object from the list.
I had gone through few solutions and here is the way I am trying but this is giving me a null set of records.
Aggregation agg = Aggregation.newAggregation(
Aggregation.project().andExpression("dayOfMonth(birthDate)").as("day").andExpression("month(birthDate)")
.as("month"),
Aggregation.group("day", "month"));
AggregationResults<Employee> groupResults = mongoTemplate.aggregate(agg, Employee.class, Employee.class);
I also tried applying a a query with the help of Criteria but this is also giving me a Employee object which all null content.
Aggregation agg = Aggregation.newAggregation(Aggregation.match(Criteria.where("birthDate").lte(new Date())), Aggregation.project().andExpression("dayOfMonth(birthDate)").as("day").andExpression("month(birthDate)")
.as("month"),
Aggregation.group("day", "month"));
AggregationResults<Employee> groupResults = mongoTemplate.aggregate(agg, Employee.class, Employee.class);
I must missing some important thing which is giving me these null data.
Additional Info: Employee object has only birthDate(Date) and email(String) in it
Please try to specify the fields to be included in the $project stage.
project("birthDate", "...").andExpression("...
The _id field is, by default, included in the output documents. To include any other fields from the input documents in the output documents, you must explicitly specify the inclusion in $project.
see: MongoDBReference - $project (aggregation)
I've created DATAMONGO-2200 to add an option to project directly onto the fields of a given domain type via something like project(Employee.class).
I have Documents table and Signs table. Document record can be related with many records in Signs table.
Now, I want to get all records of Documents table when document ID appears in Signs table.
Here I get all documents:
var documents = (from c in context.documents select c);
Here I get all my signs and save into List:
var myDocuments = (from s in context.signs where s.UserId== id select s.ID).ToList();
This list contains collection on document ID.
And here, I'm trying to get all documents that exists in myDocuments list:
documents.Where(item => myDocuments.Contains(item.ID));
But, when I do .ToList() allways return all records (in database only exists one compatible record)
What is wrong in LinQ statement?
The problem is that this statement doesn't modify the contents of documents, it merely returns the results (which you're not doing anything with):
documents.Where(item => myDocuments.Contains(item.ID));
documents is still the full list.
Change this line to something like:
var matchingIDDocs = documents.Where(item => myDocuments.Contains(item.ID));
And then use matchingIDDocs in place of "documents" later in your code.
I am working with a Solr instance that is populated from an oracle database. As records are added and deleted from the oracle database they are supposed to also be added and removed from Solr.
The schema.xml has this setup, which we use to store the ID that is also the primary key in oracle:
<uniqueKey>id</uniqueKey>
<field name="id" type="string" indexed="true" stored="true"/>
Furthermore the ids are not in sequential order. The solr admin interface has not been much help, I can only see the IDs along with the rest of each record, a few at a time, paginated.
There are about a million documents in this solr core.
I can easily get the IDs of the records from the oracle database, and so I would like to also get a list of the document id's from the solr index for comparison.
I haven't been able to find any information on how to do this but I may be searching
If you really need to get the id of all your documents, use the fl parameter. Something like that:
SolrQuery q = new SolrQuery("*:*&fl=id");
// ^^^^^
// return only the `id` field
q.setRows(10000000);
// ^^^^^^^^
// insanely high number: retrieve _all_ rows
// see: http://wiki.apache.org/solr/CommonQueryParameters#rows-1
return server.query(q).getResults();
(untested):
For simple comparison between the content in Oracle and in Solr, you might just want to count documents:
SolrQuery q = new SolrQuery("*:*");
q.setRows(0);
// ^
// don't retrieve _any_ row
return server.query(q).getResults().getNumFound();
// ^^^^^^^^^^^^^
// just get the number of matching documents
(untested):
In latest Solr (4.10), you can export large number of records.
However, if you really just want one field, you can make a request with that one field and export as CSV. That minimizes the formatting overhead.
For Solr 7 syntax has changed a bit. This is what worked for me (in Java):
CloudSolrClient solrClient = ...;
solrClient.setDefaultCollection("collection1");
SolrQuery q = new SolrQuery("*:*");
q.set("fl", "id");
q.setRows(10000000);
Set<String> uniqueIds = solrClient.query(q).getResults()
.stream().map(x -> (String) x.get("id"))
.collect(Collectors.toSet());
I am facing a problem in deleting large data from parse.com
Firstly i filtered the data using filter but it displays me only at max 100 rows and then i have to select this 100 rows and delete , and then again select and delete next 100.
Is there any way i can delete all data matching the filter,
something like
DELETE FROM Tablename WHERE fieldname LIKE '%foo%'
or is it possible to execute query in parse.com
or is there a way to deleted it using shell script and parse somehow (any package might help me)
If you want to do this programmatically, you can create a query to get all the objects and then delete them. Here is an example using swift for iOS:
var query = PFQuery(className: TABLENAME)
query.whereKey(fieldname, equals: "%foo%")
query.findObjectsInBackgroundWithBlock(
{(objects: [AnyObject]!, error: NSError!) -> Void in
for object in objects {
object.deleteInBackground()
}
})
The documentation for parse in any of its supported languages can be found here: https://parse.com/docs/
In the table ReservationWorkerPeriods there are records of all workers that are planned to work on a given period on any possible machine.
The additional table WorkerOnMachineOnConstructionSite contains columns workerId, MachineId and ConstructionSiteId.
From the table ReservationWorkerPeriods I would like to retrieve just workers who work on selected machine.
In order to retrieve just relevant records from WorkerOnMachineOnConstructionSite table I have written the following code:
var relevantWorkerOnMachineOnConstructionSite = (from cswm in currentConstructionSiteSchedule.ContrustionSiteWorkerOnMachine
where cswm.MachineId == machineId
select cswm).ToList();
workerOnMachineOnConstructionSite = relevantWorkerOnMachineOnConstructionSite as List<ContrustionSiteWorkerOnMachine>;
These records are also used in the application so I don't want to bypass the above code even if is possible to directly retrieve just workerPeriods for workers who work on selected machine. Anyway I haven't figured out how it is possible to retrieve the relevant workerPeriods once we know which userIDs are relevant.
I have tried the following code:
var userIDs = from w in workerOnMachineOnConstructionSite select new {w.WorkerId};
List<ReservationWorkerPeriods> workerPeriods = currentConstructionSiteSchedule.ReservationWorkerPeriods.ToList();
allocatedWorkers = workerPeriods.Where(wp => userIDs.Contains(wp.WorkerId));
but it seems to be incorrect and don't know how to fix it. Does anyone know what is the problem and how it is possible to retrieve just records which contain userIDs from the list?
Currently, you are constructing an anonymous object on the fly, with one property. You'll want to grab the id directly with (note the missing curly braces):
var userIDs = from w in workerOnMachineOnConstructionSite select w.WorkerId;
Also, in such cases, don't call ToList on it - the variable userIDs just contains the query, not the result. If you use that variable in a further query, the provider can translate it to a single sql query.