I'm running a very simple query in Oracle NoSQL (NodeJs driver):
let count_statement = `SELECT count(*) FROM ${resource}`;
let res = await this.client.query(count_statement, {});
This returns 0 rows (and thus no count). If I run the query without the count, I get back rows I can iterate over. Is there no way to get the total results for a query.
I don't want to count the results WITHIN the row. I need the number of rows that match this query (which is all of them in this query)
In order to get the full result set, the code should call client.query in a loop intil the continuationKey in the result object is null, as in this example from the nosql node sdk:
/*
* Execute a query and print the results. Optional limit parameter is used to
* limit the results of each query API invocation to that many rows.
*/
async function runQuery(client, stmt, limit) {
const opt = { limit };
let res;
console.log('Query results:');
do {
// Issue the query
res = await client.query(stmt, opt);
// Each call to NoSQLClient.query returns a portion of the
// result set. Iterate over the result set, using the
// QueryResult.continuationKey in the query's option parameter,
// until the result set is exhausted and continuation key is
// null.
for(let row of res.rows) {
console.log(' %O', row);
}
opt.continuationKey = res.continuationKey;
} while(res.continuationKey != null);
}
You can also use the new queryIterable syntax introduced in version 5.3.0
const count_statement = `SELECT count(*) FROM ${resource}`
const rows = []
for await(const res of client.queryIterable(count_statement )) {
rows.push.apply(rows, res.rows);
}
return rows
Related
Imagine the following GraphQL request:
{
books(
first:10,
filter: [{field: TITLE, contains: "Potter"}],
orderBy: [{sort: PRICE, direction: DESC}, {sort: TITLE}]
)
}
The result will return a connection with the Relay cursor information.
Should the cursor contain the filter and orderBy details?
Meaning querying the next set of data would only mean:
{
books(first:10, after:"opaque-cursor")
}
Or should the filter and orderBy be repeated?
In the latter case the user can specify different filter and/or orderBy details which would make the opaque cursor invalid.
I can't find anything in the Relay spec about this.
I've seen this done multiple ways, but I've found that with cursor-based pagination, your cursor exists only within your dataset, and to change the filters would change the dataset, making it invalid.
If you're using SQL (or something without cursor-based-pagination), then, you would need to include enough information in your cursor to be able to recover it. Your cursor would need to include all of your filter / order information, and you would need to disallow any additional filtering.
You'd have to throw an error if they sent "after" along with "filter / orderBy". You could, optionally, check to see if the arguments are the same as the ones in your cursor, in case of user error, but there simply is no use-case to get "page 2" of a DIFFERENT set of data.
I came across the same question / problem, and came to the same conclusion as #Dan Crews. The cursor must contain everything you need to execute the database query, except for LIMIT.
When your initial query is something like
SELECT *
FROM DataTable
WHERE filterField = 42
ORDER BY sortingField,ASC
LIMIT 10
-- with implicit OFFSET 0
then you could basically (don't do this in a real app, because of SQL Injections!) use exactly this query as your cursor. You just have to remove LIMIT x and append OFFSET y for every node.
Response:
{
edges: [
{
cursor: "SELECT ... WHERE ... ORDER BY ... OFFSET 0",
node: { ... }
},
{
cursor: "SELECT ... WHERE ... ORDER BY ... OFFSET 1",
node: { ... }
},
...,
{
cursor: "SELECT ... WHERE ... ORDER BY ... OFFSET 9",
node: { ... }
}
]
pageInfo: {
startCursor: "SELECT ... WHERE ... ORDER BY ... OFFSET 0"
endCursor: "SELECT ... WHERE ... ORDER BY ... OFFSET 9"
}
}
The next request will then use after: CURSOR, first: 10. Then you'll take the after argument and set the LIMIT and OFFSET:
LIMIT = first
OFFSET = OFFSET + 1
Then the resulting database query would be this when using after = endCursor:
SELECT *
FROM DataTable
WHERE filterField = 42
ORDER BY sortingField,ASC
LIMIT 10
OFFSET 10
As already mentioned above: This is only an example, and it's highly vulnerable to SQL Injections!
In a real world app, you could simply encode the provided filter and orderBy arguments within the cursor, and add offset as well:
function handleGraphQLRequest(first, after, filter, orderBy) {
let offset = 0; // initial offset, if after isn't provided
if(after != null) {
// combination of after + filter/orderBy is not allowed!
if(filter != null || orderBy != null) {
throw new Error("You can't combine after with filter and/or orderBy");
}
// parse filter, orderBy, offset from after cursor
cursorData = fromBase64String(after);
filter = cursorData.filter;
orderBy = cursorData.orderBy;
offset = cursorData.offset;
}
const databaseResult = executeDatabaseQuery(
filter, // = WHERE ...
orderBy, // = ORDER BY ...
first, // = LIMIT ...
offset // = OFFSET ...
);
const edges = []; // this is the resulting edges array
let currentOffset = offset; // this is used to calc the offset for each node
for(let node of databaseResult.nodes) { // iterate over the database results
currentOffset++;
const currentCursor = createCursorForNode(filter, orderBy, currentOffset);
edges.push({
cursor = currentCursor,
node = node
});
}
return {
edges: edges,
pageInfo: buildPageInfo(edges, totalCount, offset) // instead of
// of providing totalCount, you could also fetch (limit+1) from
// database to check if there is a next page available
}
}
// this function returns the cursor string
function createCursorForNode(filter, orderBy, offset) {
return toBase64String({
filter: filter,
orderBy: orderBy,
offset: offset
});
}
// function to build pageInfo object
function buildPageInfo(edges, totalCount, offset) {
return {
startCursor: edges.length ? edges[0].cursor : null,
endCursor: edges.length ? edges[edges.length - 1].cursor : null,
hasPreviousPage: offset > 0 && totalCount > 0,
hasNextPage: offset + edges.length < totalCount
}
}
The content of cursor depends mainly on your database and you database layout.
The code above emulates a simple pagination with limit and offset. But you could (if supported by your database) of course use something else.
In the meantime I came to another conclusion: I think it doesn't really matter whether you use an all-in-one cursor, or if you repeat filter and orderBy with each request.
There are basically two types of cursors:
(1.) You can treat a cursor as a "pointer to a specific item". This way the filter and sorting can change, but your cursor can stay the same. Kinda like the pivot element in quicksort, where the pivot element stays in place and everything around it can move.
Elasticsearch's Search After works like this. Here the cursor is just a pointer to a specific item in the dataset. But filter and orderBy can change independently.
The implementation for this style of cursor is dead simple: Just concat every sortable field. Done. Example: If your entity can be sorted by price and title (plus of course id, because you need some unique field as tie breaker), your cursor always consists of { id, price, title }.
(2.) The "all-in-one cursor" on the other hand acts like a "pointer to an item within a filtered and sorted result set". It has the benefit, that you can encode whatever you want. The server could for example change the filter and orderBy data (for whatever reason) without the client noticing it.
For example you could use Elasticsearch's Scroll API, which caches the result set on the server and though doesn't need filter and orderBy after the initial search request.
But aside from Elasticsearch's Scroll API, you always need filter, orderBy, limit, pointer in every request. Though I think it's an implementation detail and a matter of taste, whether you include everything within your cursor, or if you send it as separate arguments. The outcome is the same.
I have multiple values for one parameter i want to fetch data for each query for every value of parameter in Birt report. i'm getting data only for one value of parameter not all. m using Scripted data source.Open and fetch methods.Thanks
Open in DataSet
importPackage(Packages.com.abc.test.events);
var TlantNo = params["tlant"].value;
var reqNo = params["Number"].value;
poreEvents = new StdPoreReqEvents();
poreEvents.setReqNo(reqNo);
poreEvents.setTlantNo(TlantNo);
poreEvents.open();
fetch
var poreRO = poreEvents.fetch();
if (poreRO == null) {
return false;
} else
{
row["REQ_NO"] = poreRO.getReqNo();
row["REQ_DATE"] = poreRO.getReqDate();
return true;
}
A report parameter with multiple values is an array, we have to iterate on it through the scripted dataset.
In open event of the dataset, we only have to initialize a global index:
i=0;
In fetch event, process each iteration with something such the script below. Pay a special attention how we get the value of reqNo:
importPackage(Packages.com.abc.test.events);
if (params["Number"].value!=null && i<params["Number"].value.length){
var TlantNo = params["tlant"].value;
var reqNo = params["Number"].value[i];
//ETC. do here your stuff with porevents, declare poreRO, check if result is null
row["REQ_NO"] = poreRO.getReqNo();
row["REQ_DATE"] = poreRO.getReqDate();
i++; //Important: increment this even if poreRO is null, otherwise infinite loop
return true; //should return true even if poreRO was null, to process next rows
}else{
return false;
}
The other approach is to do this problem is by defining a dataset having output column which is the multiple value parameter Fetch that column by IN query of the multiple value parameter then pass that value means(output column value) to the other datasets as a Input/output parameter and give binding to parameter.
It resolved my problem :)
I return the values 10 by 10 in my table using the tablesorterPager.
This ajaxProcessing code treat my values returned by json.
When I use the filters (filter-select or search), it's only applied on the 10 values returned by my controller and I want to apply it to the total rows.
ajaxProcessing: function(data){
if (data && data.hasOwnProperty('rows')) {
var r, row, c, d = data.rows,
total = data.total_rows,
headers = data.headers,
rows = [],
len = total;
for ( r=0; r < len; r++ ) {
row = [];
for ( c in d[r] ) {
if (typeof(c) === "string") {
row.push(d[r][c]);
}
}
rows.push(row);
}
return [ total, rows, headers ];
}
},
Have you some ideas?
Thank you for your time.
Sounds like you may need to use the ajaxUrl and customAjaxUrl options. With the ajaxUrl option you set how the filter url parameters are formatted when a filter is activated. With customAjaxUrl you can convert the filter url parameters set in ajaxURl to whatever you need to properly filter the 1000 records according to how your program works.
For example, let's say you have ajaxUrl set as follows:
ajaxUrl: dataquery.php?{filter:filter}
Let's say you type "John" into the first column filter in the table. This will cause customAjaxUrl to run and the url variable will be dataquery.php?filter[0]=John. Let's say that for the query to properly filter your 1000 records you need the url query to be dataquery.php?name=John. To make that conversion, add coding within the customAjaxUrl option to change filter[0] to name and return the converted url dataquery.php?name=John. This will then filter the 1000 records and return the filtered result to your table.
You may not be using php but the concept would still apply.
I have a SQL statement similar to:
SELECT DATEDIFF(Day, startDate, endDate) FROM Data WHERE ProjectId=#id
In the case where Data doesn't have any records for ProjectId, SQL Server returns null.
In Dapper, I execute this via:
value = conn.Query<int>("...").SingleOrDefault()
In this case, I expect the semantics of SingleOrDefault to mean "if this is null, return zero." In fact, my code is even more zero-friendly:
int toReturn = 0;
using (var conn = ...) {
toReturn = conn.Query<int>("SELECT DATEDIFF(...))");
}
return toReturn;
When I debug and step into this code, I find that the line yield return (T)func(reader) is throwing a null pointer exception.
Am I doing something wrong here, or is this by design?
(FYI, the work-around is to wrap my select in an ISNULL(..., 0))
In the case where Data doesn't have any records for ProjectId, SQL Server returns null.
In the case where Data doesn't have any matching records, SQL server does not really return null - it returns no rows. This scenario works fine:
var result = connection.Query<int>( // case with rows
"select DATEDIFF(day, GETUTCDATE(), #date)", new { date = DateTime.UtcNow.AddDays(20) })
.SingleOrDefault();
result.IsEqualTo(20);
result = connection.Query<int>( // case without rows
"select DATEDIFF(day, GETUTCDATE(), #date) where 1 = 0", new { date = DateTime.UtcNow.AddDays(20) })
.SingleOrDefault();
result.IsEqualTo(0); // zero rows; default of int over zero rows is zero
both of which work fine. The fact that you say ISNULL "fixes" it means that you are talking about a different scenario - the "I returned rows" scenario. Since that is the case, what your code is saying is "take this 1-or-more integers which contains a null, and map it as a non-nullable int" - that isn't possible, and the mapper is correct to throw an exception. Instead, what you want is:
int? result = connection.Query<int?>(...).SingleOrDefault();
Now, if there are rows, it is mapping the value to int?, and then applying the single-or-default. It you want that as an int, then maybe:
int result = connection.Query<int?>(...).SingleOrDefault() ?? 0;
If you need to be able to tell the difference between "zero rows" and "null result", then I would suggest:
class NameMe {
public int? Value {get;set;}
}
var row = connection.Query<NameMe>("select ... as [Value] ...", ...)
.SingleOrDefault();
if(row == null) {
// no rows
} else if(row.Value == null) {
// one row, null value
} else {
// one row, non-null value
}
or something similar
I have a linq query like this:
var accounts =
from account in context.Accounts
from guranteer in account.Gurantors
where guranteer.GuarantorRegistryId == guranteerRegistryId
select new AccountsReport
{
recordIndex = ?
CreditRegistryId = account.CreditRegistryId,
AccountNumber = account.AccountNo,
}
I want to populate recordIndex with the value of current row number in collection returned by the LINQ. How can I get row number ?
Row number is not supported in linq-to-entities. You must first retrieve records from database without row number and then add row number by linq-to-objects. Something like:
var accounts =
(from account in context.Accounts
from guranteer in account.Gurantors
where guranteer.GuarantorRegistryId == guranteerRegistryId
select new
{
CreditRegistryId = account.CreditRegistryId,
AccountNumber = account.AccountNo,
})
.AsEnumerable() // Moving to linq-to-objects
.Select((r, i) => new AccountReport
{
RecordIndex = i,
CreditRegistryId = r.CreditRegistryId,
AccountNumber = r.AccountNo,
});
LINQ to objects has this builtin for any enumerator:
http://weblogs.asp.net/fmarguerie/archive/2008/11/10/using-the-select-linq-query-operator-with-indexes.aspx
Edit: Although IQueryable supports it too (here and here) it has been mentioned that this does unfortunately not work for LINQ to SQL/Entities.
new []{"aap", "noot", "mies"}
.Select( (element, index) => new { element, index });
Will result in:
{ { element = aap, index = 0 },
{ element = noot, index = 1 },
{ element = mies, index = 2 } }
There are other LINQ Extension methods (like .Where) with the extra index parameter overload
Try using let like this:
int[] ints = new[] { 1, 2, 3, 4, 5 };
int counter = 0;
var result = from i in ints
where i % 2 == 0
let number = ++counter
select new { I = i, Number = number };
foreach (var r in result)
{
Console.WriteLine(r.Number + ": " + r.I);
}
I cannot test it with actual LINQ to SQL or Entity Framework right now. Note that the above code will retain the value of the counter between multiple executions of the query.
If this is not supported with your specific provider you can always foreach (thus forcing the execution of the query) and assign the number manually in code.
Because the query inside the question filters by a single id, I think the answers given wont help out. Ofcourse you can do it all in memory client side, but depending how large the dataset is, and whether network is involved, this could be an issue.
If you need a SQL ROW_NUMBER [..] OVER [..] equivalent, the only way I know is to create a view in your SQL server and query against that.
This Tested and Works:
Amend your code as follows:
int counter = 0;
var accounts =
from account in context.Accounts
from guranteer in account.Gurantors
where guranteer.GuarantorRegistryId == guranteerRegistryId
select new AccountsReport
{
recordIndex = counter++
CreditRegistryId = account.CreditRegistryId,
AccountNumber = account.AccountNo,
}
Hope this helps.. Though its late:)