I have tired pagination using endCursor and hasNextPage in github grpahQL API to get more than 100 data. Query I used is:
query {
organization(login:"XXX") {
repository(name:"XX") {
pullRequests(first:100, states:[OPEN], after: "XXX" ) {
pageInfo{
hasNextPage
endCursor
}
}
}
It is working. But in order to access further details,iterative pagination needs to be done. Can anyone provides an efficient alternative to traverse all pages programatically in GraphQL API?
Taking inspiration from Simon Willison's 'Paginating through the GitHub GraphQL API with Python' here's what I've been doing to paginate my queries:
query {
node(id: "PROJECT_ID") {
... on ProjectNext {
items(first: 100 after: CURSOR) {
pageInfo {
hasNextPage
endCursor
}
nodes {
title
fieldValues(first: 8) {
nodes {
value
}
}
content {
... on Issue {
number
labels(first: 50) {
nodes {
name
}}}}}}}}}
In my Python code I'm splicing in PROJECT_ID with a variable set to the project ID I'm referencing.
For the cursor after: CURSOR is replaced with "" initially, and then for the next page I set cursor = 'after:\\"' + response["data"]["node"]["items"]["pageInfo"]["endCursor"] + '\\"'
My full code is in the atdumpmemex module of my dump_cards utility.
The key here is to get pageInfo along with other relevant nodes, and then grab the endCursor each time hasNextPage is true so that it can be fed into the query for the next iteration.
pageInfo will look something like:
"pageInfo": {
"hasNextPage": false,
"endCursor": "Y3Vyc29yOnYyOpHOAAhOsg=="
}
At the moment the endCursor is base64 encoded cursor:v2:XYZ, but don't rely on that as GitHub have moved other IDs from being base64 encoded to other schemes.
Pagination using endCursor is the one solution. Like REST API, you cannot traverse all pages pragmatically. As you specified you can specify endCursor and traverse if there is next page
Related
I am trying to load user information from GitLab so that I can associate usernames with issues. When querying for issues, the assignee username is not directly available. Instead, a user ID is available.
If I execute this GraphQL query using the /-/graphql-explorer endpoint on my GitLab deployment:
query {
users {
nodes {
id
name
username
}
}
}
then 91 users are returned. This is clearly not all users on the deployment, though. There are users I know exist but which are not included in the result. I can query for them individually using this GraphQL query:
query {
user(username: "someusername") {
id
}
}
and receive a result which seems to correctly describe the user.
Why are some users omitted from the results for this query? I know that large result sets require dealing with pagination but the default page size is supposed to be 100 and I am receiving fewer results than this. Additionally, if I request pageinfo and ask for the users after the resulting endCursor I receive no items.
How do I submit a query that gets me all users? Failing that, how do I submit a query that will get me all users which could be assignees for a a group (or, failing that, for a list of projects)?
From the documentation :
By default, GitLab’s GraphQL API will return only the first 100 records of any collection. This can be changed by using first or last arguments. Both arguments take a value, so first: 10 will return the first 10 records, and last: 10 the last 10 records.
So you have to do the pagination yourself, your first query would be for example :
query {
users(first: 10) {
edges {
node {
id
name
username
}
}
pageInfo {
endCursor
hasNextPage
}
}
}
Then use the PageInfo result values to know if you have more pages to fetch and the cursor ID to fetch the next page, for example you could get the following result :
...
"pageInfo": {
"endCursor": "eyJpZCI6Ijc****************zNjk0MDUwMDAgOVRDIn0",
"hasNextPage": true
}
So for the next page, you have to query :
query {
users(first: 10, after: "eyJpZCI6IjcxMj**********************Ni4zNjk0MDUwMDAgOVRDIn0") {
edges {
node {
id
name
username
}
}
pageInfo {
endCursor
hasNextPage
}
}
}
And so on until hasNextPage returns false.
For more info about pagination and cursors, see GraphQL documentation : https://graphql.org/learn/pagination/
Tha database for that queries comes up to a set of YAML files in a folder.
Using GatsbyJS theses files are collected and are available by graphQL requests.
Not all source files defines a label field.
Is it possible to return a default or empty value for that field when missing?
Actually I get following error:
Cannot query field "label" on type "EngagementsYamlTo".
This is my query:
{
stable: allEngagementsYaml(filter: {stable: {eq: true}, to: {}}) {
edges {
...engagementNode
}
}
unstable: allEngagementsYaml(filter: {stable: {eq: false}}) {
edges {
...engagementNode
}
}
}
fragment engagementNode on EngagementsYamlEdge {
node {
heading
description
iconClass
stable
to {
href
label
}
}
}
Hm ... that was not a problem of the query.
None of the source files had a label field, so that schema was never touched. Having at least 1 file with label field will return a valid response.
Clarified for this case!
I have a GraphQL query called myAccounts which returns an array of accounts. When I go to the Playground and call the query:
{
accounts {
email
}
}
I get this result:
"data": {
"accounts": [
{
"email": "zach#email-one.com",
},
{
"email": "zach#email-two.com",
}
]
}
However, when I am in my Component, vue-apollo returns two items in the array, but seems to overwrite the second item with the first. Here is the query (in MyAccounts.gql):
query myAccounts {
accounts: myAccounts {
email
}
}
and here is the Apollo query in the component:
import MY_ACCOUNTS_QUERY from '~/apollo/queries/MyAccounts'
...
apollo: {
accounts: {
query: MY_ACCOUNTS_QUERY,
result(data) {
console.log(JSON.stringify(data))
}
}
}
and here is what vue-apollo logs out through the result:
{
"data":{
"accounts":[
{
"email":"zach#email-one.com",
"__typename":"Account"
},
{
"email":"zach#email-one.com",
"__typename":"Account"
}
]
},
"loading":false,
"networkStatus":7,
"stale":false
}
Expected behavior
I would expect the data returned in the Playground to be identical to what vue-apollo is fetching.
Versions
vue: 2.6.10
vue-apollo: #nuxtjs/apollo: 4.0.0-rc18
Additional context
I thought the result hook would be the best way to debug, but any other suggestions gladly welcomed. I assumed that this was a bug in our code, but I cannot figure out what could be causing the repetition (and mismatch).
Apollo normalizes its cache based on the __typename and the id (or _id) field. You need to include an id or _id field in your selection set alongside email. Failing to do so results in both objects being assigned the same key. If you don't have an id field to request, you'll need to provide a custom dataIdFromObject function as shown here.
From Guillaume Chau (https://github.com/Akryum):
This is because the Apollo Client cache can't compute a different ID
for the two items, so you endup with Account:undefined (or similar)
for both. Open the Apollo devtools and look at the myAccounts key in
the cache.
Learn more:
https://www.apollographql.com/docs/react/caching/cache-configuration/
Below is my GraphQL Query to Fetch Posts from Strapi backend.
Please note I am running this on my Nuxt app.
Now I want to bring only those posts which have post_status = "Publish"
post_status is a ENUM field with two option as Draft and Publish
query GetPosts{
posts {
id
post_title
post_excerpt
post_featured_image{url}
post_content
post_category{category_name}
postingredients{ingredient{ingredient_name}, ingredient_unit}
updated_at
post_author{username}
post_slug
}
}
I did not understand how can I get
How to bring post_status values on my original Query
How to filter on the post_status where I can get only Published posts.
query GetStatusEnum{
__type(name: "ENUM_POST_POST_STATUS") {
name
enumValues {
name
} } }
Result of the above:
{
"data": {
"__type": {
"name": "ENUM_POST_POST_STATUS",
"enumValues": [
{
"name": "Publish"
},
{
"name": "Draft"
}
]
}
}
}
To add your post_status in your original request you just have to add it in the list of the attributes you want to fetch.
{
posts {
id
post_title
post_status <- here /!\
}
}
Here is the query to fetch Posts that have Publish as post_status
{
posts(where: { post_status: "Publish" }) {
id
post_title,
post_status
}
}
You can play with GraphQL playground in your strapi application:
http://localhost:1337/graphql
You will see in the right of you page a docs button that will show you all the information you need to create your GraphQL request.
I had a similar scenario (though I'm using a Prisma layer as well so keep that in mind) and i'm not sure that you can filter for enum values on the call but you can filter what it returns.
const posts = [the array of all posts]
const isPublished = (post) => {
if (post.post_status.includes('Publish')) {
return post;
}
}
let publishedPosts = posts.filter(isPublished);
return publishedPosts;
I'm trying to build a query with the GitHub API v4 (GraphQL) to get the number of contributors.
At the moment I have something of the likes of
query ($owner: String!, $name: String!) {
repository(owner: $owner, name: $name) {
ref(qualifiedName: "master") {
target {
... on Commit {
history(first: 100) {
nodes {
author {
name
}
}
pageInfo {
hasNextPage
endCursor
}
}
}
}
}
}
}
where I'm going through all the commits and get the name of the Authors (at the time I was trying to get the number of commits for contributor), but for repositories with a large amount of commits, this takes a lot of time.
So back to my question, is the a way to get only the number of contributors in a repository?
As far as I know, this is only possible with the REST API (v3), by requesting only one item per page, and extracting the total number of pages in the Response headers.
With this graphql query you can get the:
- total releases count
- total branches count
- total commits count
query{
repository(owner:"kasadawa", name:"vmware-task") {
Releases:refs(first: 0, refPrefix: "refs/tags/") {
totalCount
}
Branches:refs(first: 0, refPrefix: "refs/heads/") {
totalCount
}
object(expression:"master") {
... on Commit {
history {
totalCount
}
}
}
}
}
But if you want to get the contributors, you should do it with the REST API, because currently there is no simple way to implement it with GRAPHQL API.
Here is a solutions with the REST API.
const options = {
url: 'https://api.github.com/repos/vmware/contributors' ,
'json': true ,
headers: {
'User-Agent':'request',
'Authorization': 'token API_KEY_GENERATED_FROM_GITHUB'
}
};
var lastPageNum = 0 ; // global variable
request.getAsync(options).then((res) =>{
this.checkHeaders(res.headers.status) // just check the header status
// check if there are more pages
if(!!res.headers.link){
const [ , lastURL] = res.headers.link.split(','); // getting the last page link from the header
lastPageNum = +lastURL.match(/page=(\d+)/)[1]; // get the number from the url string
options.url = licenseUrl + lastPageNum;
return request.getAsync(options) // make Request with the last page, in order to get the last page results, they could be less than 30
}else{
// if its single page just resolve on to the chain
return Promise.resolve(res.body.length);
}
})
.then((lastPageRes)=>{
return (lastPageNum !== 0
? ( (lastPageNum-1)*30 + lastPageRes.body.length )
: lastPageRes)
})
.catch() // handle errors
Checkout for updates: https://github.community/t5/GitHub-API-Development-and/Get-contributor-count-with-the-graphql-api/td-p/18593