How to increase performance of pouchdb-find queries? - performance

I'm using pouchDB replication from couchDB in react native. This is my package.json:
"pouchdb": "^6.3.4",
"pouchdb-find": "^6.3.4",
"pouchdb-react-native": "^6.3.4",
I've a database named "databaseA" where one of the column is "col1". I've written a query using pouchdb-find to get all records of databaseA with matching
value in col1.
const localDB = new PouchDB('databaseA');
const remoteDB = new PouchDB('databaseA');
PouchDB.plugin(findPlugin);
localDB.replicate.from(
remoteDB,
(err, response) => {
if (err) {
return console.log(err);
}
},
);
localDB.createIndex({
index: {
fields: ['col1', 'col2'],
ddoc: 'col1-col2-index',
},
});
localDB.find({
selector: {
col1: {
$eq: '1',
},
col2: true,
},
fields: ['_id', 'col1', 'col2' ],
limit: 0,
// sort: [‘_id’],
use_index: 'col1-col2-index',
}).then((results) => {
alert(`Index called to increase speed: ${results.docs.length}`);
}).catch((ex) => {
// alert(`Index called to increase speed: Exception: ${JSON.stringify(ex)}`);
});
export function getItems(col1) {
const startTime = new Date();
return (dispatch, getState) => {
return localDB.find({
selector: {
col1: {
$eq: col1,
},
col2: true,
},
fields: ['_id', 'col2', 'col1' ],
// sort: [‘_id’],
use_index: 'col1-col2-index',
}).then((results) => {
const time = new Date().getTime() - startTime;
alert(`Total time ${time}`);
}).catch((ex) => {
console.log(ex);
});
};
}
I've around 900 records in the database. It takes nearly 2minutes which is quite high to query. How can the performance of this query be increased? Please help

Related

Apollo Graphql fetchMore, updateQuery does not update state

I'm currently trying to implement pagination on my posts.
Using Apollo graphql here is my useQuery
const { data: postsData, fetchMore } = useQuery(POSTS_BY_USER_DRAFT, {
fetchPolicy: 'network-only',
variables: {
user: user.id,
start: 0,
limit: limit
},
onCompleted: () => {
setTotal(postsData[model].meta.pagination.total)
}})
and here is my onClick handler for fetching more posts
const loadMorePosts = async () => {
const nextStart = start + limit
setStart(nextStart);
await fetchMore({
variables: {
user: user.id,
offset: nextStart,
limit: limit,
},
updateQuery: (prevResult, { fetchMoreResult }) => {
if (!fetchMoreResult) {
return prevResult
}
const prevData = prevResult[model].data
const moreData = fetchMoreResult[model].data
fetchMoreResult[model].data = [...prevData, ...moreData]
// fetchMoreResult[model].data = [...moreData]
return fetchMoreResult
},
})}
My queries are successful as I do get correctly the data, however postsData does not get updated
[NOTICED]: If I switch fetchMoreResult[model].data = [...prevData, ...moreData] for
fetchMoreResult[model].data = [...moreData] my postsData does get updated.
I have tried return { ...fetchMoreResult } and multiple ways of returning data fearing an immutability/comparaison issue but it does not seem to do the job.
I'm not sure why, but setting a fetchPolicy for Apollo will do the job
const client = new ApolloClient({
link: authLink.concat(httpLink),
cache: new InMemoryCache({
typePolicies: {
Publication: {
merge: true,
},
Post: {
merge: true,
},
},
}),
defaultOptions: defaultOptions,
})

Apollo cache fetchMore & mutation

I'm trying to create a page with a load more and with a button that execute a mutation against a particular item of the list, this is the query:
const { data, fetchMore, networkStatus } =
useSearchItemsQuery({
notifyOnNetworkStatusChange: true,
variables,
});
this is the fetchMore:
fetchMore({
variables: {
offset: data.searchItems.nodes.length,
},
});
this is my apollo config:
return new ApolloClient({
...
cache: new InMemoryCache({
typePolicies: {
Item: {
keyFields: ["uuid"],
merge: true,
},
Query: {
fields: {
searchItems: {
keyArgs: false,
merge(existing = [], incoming, { args }) {
console.log("options", args);
return deepmerge(existing, incoming, {
arrayMerge: (destinationArray, sourceArray) => {
const refs = [...destinationArray, ...sourceArray].map(
(o) => o.__ref
);
const array = [...destinationArray, ...sourceArray].filter(
({ __ref }, index) => !refs.includes(__ref, index + 1)
);
console.log("array", array);
return array;
},
});
},
},
},
},
},
}),
});
};
the fetchMore works correctly but when I try to run a mutation I can see the changes on the screen but the searchItems query is been refetched and the list on the screen update with the initial state, before the fetchMore.
Example:
open the page and 1 item appear
click fetchMore
a new item appear
run a mutation against the 1st item or the 2nd item
the list now contains only the 1st item
I can see on the network tab the after the mutation the initial query it's been executed with the initial offset/limit

Pagination Apollo Client 3 - Cache merges but does not render new results when paginating

I'd love to implement nested pagination within my application. I have been reading the docs and looking at several other examples but I just can't get this to work - any help is appreciated! Thanks!
React component:
I am clicking the button to run the fetchMore function provided by the useQuery hook (apollo). The network request is going through and the new products are merged into the cache... but no new products render on the page.
export const FilterableKit = () => {
const selectedKitId = useReactiveVar(selectedKitIdVar);
const [
getKitProducts,
{ data: getKitProductsData, loading: getKitProductsLoading, fetchMore },
] = useGetKitProductsLazyQuery();
useEffect(() => {
if (selectedKitId) {
getKitProducts({
variables: {
getKitsInput: {
_id: {
string: selectedKitId,
filterBy: "OBJECTID" as StringFilterByEnum,
},
},
getProductsInput: {
config: {
pagination: {
reverse: true,
limit: 3,
},
},
},
},
});
}
}, [getKitProducts, selectedKitId]);
const kitProducts = getKitProductsData?.getKits.data?.find(
(kit) => kit?._id === selectedKitId
)?.products.data;
const handleLoadMore = () => {
if (kitProducts && kitProducts?.length > 0) {
const remaining =
getKitProductsData?.getKits.data[0]?.products.stats?.remaining;
if (remaining && remaining > 0) {
const cursor =
kitProducts[kitProducts.length - 1] &&
kitProducts[kitProducts.length - 1]?.createdAt;
fetchMore({
variables: {
getProductsInput: {
config: {
pagination: {
reverse: true,
createdAt: cursor,
},
},
},
},
});
}
}
};
return (
<CContainer>
<KitItemCards products={kitProducts} loading={getKitProductsLoading} />
<CContainer className="d-flex justify-content-center my-3">
<CButton color="primary" className="w-100" onClick={handleLoadMore}>
Load More
</CButton>
</CContainer>
</CContainer>
);
};
Type Policies: I define the "Kit" typePolicy to merge products into the correct field.
export const cache: InMemoryCache = new InMemoryCache({
typePolicies: {
Kit: {
fields: {
products: {
keyArgs: false,
merge(existing = [] as Product[], incoming: GetProductsResponse) {
if (!incoming) return existing;
if (!existing) return incoming;
const { data: products, ...rest } = incoming;
let result: any = rest;
result = [...existing, ...(products ?? [])];
return result;
},
},
},
},
});
Thanks for any pointers in the right direction! Let me know if there is something else you'd like to see.

how can I find all of the indexes associated with an Elastic search alias?

using elastic search SDK https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/index.html how can find all of the indexes associated with an Elastic search alias
We do have sdk method cat. aliases where I can iterate and find the associated indexes. But is there any other elegant method available to achieve the same?
You can pass the alias name (or array of names) as a parameter. Docs
const { Client } = require("#elastic/elasticsearch");
var client;
client = new Client({
node: "http://localhost:9200",
maxRetries: 5,
requestTimeout: 60000,
sniffOnStart: true,
});
client.cat
.aliases({ format: "json", name: "alias_name" })
.then((result) => {
console.log(result.body);
})
.catch((error) => {
console.log(error);
});
Output
[
{
alias: 'alias_name',
index: 'index_name',
filter: '-',
'routing.index': '-',
'routing.search': '-',
is_write_index: '-'
}
]
And if you want the index names only
const { Client } = require("#elastic/elasticsearch");
var client;
client = new Client({
node: "http://localhost:9200",
maxRetries: 5,
requestTimeout: 60000,
sniffOnStart: true,
});
client.cat
.aliases({ format: "json", name: "alias_name" })
.then((result) => {
const clean_indices = result.body.map(r => r.index)
console.log(clean_indices);
})
.catch((error) => {
console.log(error);
});
This is what I have come up with now.
const { Client } = require('#elastic/elasticsearch');
const async = require('async');
var client;
client = new Client({
"node": "http://localhost:9200",
"maxRetries": 5,
"requestTimeout": 60000,
"sniffOnStart": true
});
client.cat.aliases({format:"json"}).then((result) => {
let indexes={};
result.body.forEach(element => {
if(!indexes[element.alias]){
indexes[element.alias] = [];
}
indexes[element.alias].push(element.index);
});
console.log(JSON.stringify(indexes,null,2));
}).catch((error) => {
console.log(error)
});

GraphQL Stitching - Why would fields from subschemas return null?

I'm attempting to stitch together two GraphQL schemas, one from contentful and one from neo4j.
Each subschema appears to be interrogated during queries across the combined schema but "foreign" fields always come back as null.
I just can't figure this one out.
Sample Query:
query {
//Request data exclusively from the neo4j schema
Product(id:"475e006f-b9cf-4f40-8712-271ceb46d14b"){
id,
name,
weight
},
//This is a contentful schema query which should return weight from neo4j
product(id:"[contentful-native-id]"){
id,
weight,
}
}
Result:
"data": {
"Product": [
{
"id": "475e006f-b9cf-4f40-8712-271ceb46d14b",
"name": "Test product name",
"weight": 14.9
}
],
"product": {
"id": "475e006f-b9cf-4f40-8712-271ceb46d14b",
"weight": null //This shouldn't be null! :(
}
}
Logging:
//First query being executed against neo4j database
neo4j-graphql-js MATCH (`product`:`Product` {id:$id}) RETURN `product` { .id , .name , .weight } AS `product`
neo4j-graphql-js {
"offset": 0,
"first": -1,
"id": "475e006f-b9cf-4f40-8712-271ceb46d14b"
}
//Triggered by the second query correctly trying to resolve weight from neo4j
neo4j-graphql-js MATCH (`product`:`Product` {id:$id}) RETURN `product` { .weight , .id } AS `product`
neo4j-graphql-js {
"offset": 0,
"first": -1,
"id": "475e006f-b9cf-4f40-8712-271ceb46d14b"
}
This seems to suggest something is working, but the result of weight never makes it to the final output.
ApolloServer doesn't report any errors via didEncounterErrors()
Stitching:
const gatewaySchema = stitchSchemas({
subschemas: [{
schema: neoSchema,
merge: {
Product: {
selectionSet: '{id}',
fieldName: 'Product',
args: ({
id
}) => ({
id
}),
}
}
},
{
schema: contentfulSchema,
merge: {
}
}
],
})
Schemas:
const executor = async ({
document,
variables,
context
}) => {
const query = print(document);
//console.log(query);
const fetchResult = await fetch('https://graphql.contentful.com/content/v1/spaces/[SPACE]', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer [AUTHTOKEN]`,
},
body: JSON.stringify({
query,
variables
})
});
return fetchResult.json();
};
const contentfulSchema = wrapSchema({
schema: await introspectSchema(executor),
executor: executor
});
const driver = neo4j.driver(
process.env.NEO4J_URI || 'bolt://localhost:7687',
neo4j.auth.basic(
process.env.NEO4J_USER,
process.env.NEO4J_PASS
), {
encrypted: process.env.NEO4J_ENCRYPTED ? 'ENCRYPTION_ON' : 'ENCRYPTION_OFF',
}
)
const neoSchema = makeAugmentedSchema({
typeDefs: typeDefs,
});
Server:
const server = new ApolloServer({
schema: gatewaySchema,
context: ({ req }) => {
return {
driver,
req
};
},
plugins:[
myPlugin
]
});
Any insight or ideas much appreciated!
This appears to be down to the fact that stitchSchemas is NOT supported in ApolloServer...
Does Apollo Server work with GraphQL Tools stitchSchemas?

Resources