Gatsby warning on build "query takes too long" - graphql

I am building a static website with Gatsby and Netlify CMS. The website is hosted on Netlify as well. I have a blog section and I generate a page per article from a markdown file. I have the following warning during build "query takes too long" for every article page I generate. The site builds eventually but the build time gets longer and longer the more pages I am generating so I'm afraid it will become too long when I start having too many articles in my site.
I am generating a page for each markdown file that is being created by netlify CMS.
Would you mind looking and the code I have in the gatsby-node file and the query I am using in my blog template file to see if I'm doing anything wrong that would explain the build time warning message ?
Thanks
Here is my development environment
npmPackages:
gatsby: ^2.26.1 => 2.26.1
gatsby-image: ^2.10.0 => 2.10.0
gatsby-plugin-netlify-cms: ^4.8.0 => 4.8.0
gatsby-plugin-react-helmet: ^3.8.0 => 3.8.0
gatsby-plugin-sharp: ^2.13.0 => 2.13.0
gatsby-plugin-styled-components: ^3.9.0 => 3.9.0
gatsby-remark-images: ^3.10.0 => 3.10.0
gatsby-remark-prismjs: ^3.12.0 => 3.12.0
gatsby-source-filesystem: ^2.9.1 => 2.9.1
gatsby-transformer-remark: ^2.15.0 => 2.15.0
gatsby-transformer-sharp: ^2.11.0 => 2.11.0
npmGlobalPackages:
gatsby-cli: 2.18.0
here is the code I've got in the gatsby-node file to generate my posts pages
exports.createPages = async ({ actions, graphql, reporter }) => {
const { createPage } = actions
const blogPostTemplate = require.resolve(`./src/templates/blog-post.js`)
const categoryPageTemplate = require.resolve(
`./src/templates/category-page.js`
)
const uncategorizedPageTemplate = require.resolve(
`./src/templates/uncategorized.js`
)
const _ = require("lodash")
const result = await graphql(`
{
posts: allMarkdownRemark(
sort: { fields: [frontmatter___date], order: DESC }
) {
edges {
node {
id
frontmatter {
categories
}
fields {
slug
}
}
}
}
categoriesGroup: allMarkdownRemark {
group(field: frontmatter___categories) {
fieldValue
}
}
}
`)
// Handle errors
if (result.errors) {
reporter.panicOnBuild(`Error while running GraphQL query.`)
return
}
const posts = result.data.posts.edges
const categories = result.data.categoriesGroup.group
posts.forEach(({ node }, index) => {
const nextPostId = index === 0 ? null : posts[index - 1].node.id
const previousPostId =
index === posts.length - 1 ? null : posts[index + 1].node.id
createPage({
path: `blog${node.fields.slug}`,
component: blogPostTemplate,
context: {
// additional data can be passed via context
id: node.id,
index,
nextPostId
previousPostId
},
})
})
}
const { createFilePath } = require(`gatsby-source-filesystem`)
exports.onCreateNode = ({ node, actions, getNode }) => {
const { createNodeField } = actions
if (node.internal.type === `MarkdownRemark`) {
const value = createFilePath({
node,
getNode,
})
createNodeField({
node,
name: `slug`,
value,
})
}
}
and here is the query I've got in my blog-post template file to get the post with the id from the pageContext:
export const pageQuery = graphql`
query($id: String!, $previousPostId: String, $nextPostId: String) {
markdownRemark(id: { eq: $id }) {
id
html
frontmatter {
featuredImage {
childImageSharp {
fluid(maxWidth: 1600) {
...GatsbyImageSharpFluid_withWebp_tracedSVG
}
}
}
title
description
date(formatString: "MMMM DD, YYYY")
categories
}
}
previous: markdownRemark(id: { eq: $previousPostId }) {
frontmatter {
title
}
fields {
slug
}
}
next: markdownRemark(id: { eq: $nextPostId }) {
frontmatter {
title
}
fields {
slug
}
}
}
`

Gatsby's team is actually working on reducing the build time by adding some "cache" features. You can follow the stack trace in their releases notes, they are still in beta testing (some of them are focused mainly on gatsby develop.
If you want to try it them to check if it improves the build-develop time, you just need to upgrade Gatsby to the latest version (^2.28) and:
// In your gatsby-config.js
module.exports = {
// your existing config
flags: {
FAST_DEV: true,
},
}
Regarding the gatsby build, in the meantime, in Netlify, you can activate some plugins (Gatsby Cache for example).
Among all this stuff, you can also add the incremental build feature (described in this Netlify's post by the great Jason Lengstorf). After you've installed the needed dependencies (upgrade Gatsby and cross-env), just customize the build command (also in Netlify's dashboard) to enable the PAGE_BUILD_ON_DATA feature:
"build": "cross-env GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES=true gatsby build --log-pages"

Related

graphql-codegen + typescript-svelte-apollo - Refetch not working?

(Edited)
Refetching with the generated query type seems to work different from the refetch function in Apollo Client useQuery. I don't understand how to phrase it - can anyone provide an example?
I'm realizing the problem is probably either my refetch is not properly phrased, or maybe the store is only hitting the cached query. I've been going over my code for days and I can't figure out what it could be. I've tried await blocks too.
The refetch worked with svelte-apollo, but i'm trying to eliminate that dependency. I've also tried Apollo Client's useQuery, but the whole point of graphql-codegen with typescript-svelte-apollo is to use the generated typescript wrapper for the query.
When I assign the generated query to a reactive constant in my Svelte front-end code,
$: observations = getObservations({ variables: { filter } });
the query does not refetch when i update the query variables, as I would expect.
This is how my svelte template is using the query. The filter object changes based on a form user input. I've tried this with an await block too.
<script lang="ts">
import { getObservations } from '$lib/generated';
$: observations = getObservations({ variables: { filter } });
function handleFilter(event) {
filter = event.detail;
}
</script>
{#if $observations.loading}
Loading...
{:else if $observations.error}
{$observations.error}
{:else if $observations.data}
{#each $observations.data['observations']['edges'] as edge}
<Item node={edge['node']} />
{/each}
{/if}
Since this plugin allows to use the query directly, without Apollo's useQuery, i'm not sure how to phrase a refetch.
If i do $observations.refetch(); inside handleFilter(e), i get an error
Property 'refetch' does not exist on type 'Readable<ApolloQueryResult<GetObservationsQuery> & { query: ObservableQuery<GetObservationsQuery, Exact<{ filter?: FilterObservationsInput; }>>; }>'.ts(2339)
There's nothing fancy in my config. Am I doing something wrong here?
schema: src/graphql/schema.graphql
documents:
- src/graphql/queries.graphql
- src/graphql/mutations.graphql
generates:
src/lib/generated.ts:
plugins:
- typescript
- typescript-operations
- graphql-codegen-svelte-apollo
config:
clientPath: src/lib/shared/client
# asyncQuery: true
scalars:
ISO8601Date: Date
ISO8601DateTime: Date
Here's the client:
export default new ApolloClient({
link: authLink.concat(httpLink),
cache: new InMemoryCache({
typePolicies: {
Query: {
fields: {
observations: relayStylePagination(),
},
},
},
})
});
The generated query:
export const getObservations = (
options: Omit<
WatchQueryOptions<GetObservationsQueryVariables>,
"query"
>
): Readable<
ApolloQueryResult<GetObservationsQuery> & {
query: ObservableQuery<
GetObservationsQuery,
GetObservationsQueryVariables
>;
}
> => {
const q = client.watchQuery({
query: GetObservationsDoc,
...options,
});
var result = readable<
ApolloQueryResult<GetObservationsQuery> & {
query: ObservableQuery<
GetObservationsQuery,
GetObservationsQueryVariables
>;
}
>(
{ data: {} as any, loading: true, error: undefined, networkStatus: 1, query: q },
(set) => {
q.subscribe((v: any) => {
set({ ...v, query: q });
});
}
);
return result;
}
Here's the query document that it's built from:
query getObservations($filter: FilterObservationsInput) {
observations(filter: $filter) {
pageInfo {
startCursor
endCursor
hasNextPage
hasPreviousPage
}
edges {
cursor
node {
id
createdAt
updatedAt
when
where
imgSrcThumb
imgSrcSm
imgSrcMed
thumbImage {
width
height
}
name {
formatName
author
}
user {
name
login
}
rssLog {
detail
}
}
}
}
}

Can't query data from contentful to gatsby project

Don't get the query to contentful to work.
Receive error message:
TypeError: Cannot read property 'allContentfulMagArticle' of undefined
datais undefined inside the Posts component. Can't see what i'm doing wrong here.
import { graphql } from 'gatsby';
import Post from "./post.js";
import './posts.css';
export const query = graphql`
query {
allContentfulMagArticle{
edges{
node{
index
title
name
subHeading
extract {
raw
}
slug
}
}
}
}
`
const Posts = ({ data }) => {
return (
<section className="posts">
<ul className="post-list">
{data.allContentfulMagArticle.edges.map(({ node }) => (
<Post
key={node.index}
id={node.index}
node={node}
title={node.title}
name={node.name}
// image={node.frontmatter.featuredImage.childImageSharp.fluid}
subheading={node.subheading}
body={node.extract.raw}
/>
))}
</ul>
</section>
)
}
export default Posts
Here my gatsby-config.js:
require('dotenv').config({
path: `.env`,
})
module.exports = {
siteMetadata: {
title: `XX`,
description: `XX`,
author: `Lisa Lee`,
url: `https://www.tortmagazine.com`
},
plugins: [
`gatsby-plugin-react-helmet`,
'gatsby-plugin-fontawesome-css',
'gatsby-plugin-sharp',
`gatsby-transformer-sharp`,
`gatsby-transformer-remark`,
{
resolve: `gatsby-source-filesystem`,
options: {
path: `${__dirname}/src/`,
},
},
{
resolve: `gatsby-source-contentful`,
options: {
spaceId: process.env.GATSBY_CONTENTFUL_SPACE_ID,
accessToken: process.env.GATSBY_CONTENTFUL_ACCESS_TOKEN,
},
},
],
}
You used the word "component" to describe your Posts but the query you are using only works in a page, or in the context of createPage (so in a template file as well). If you are indeed in a component, that will be the issue. If not, then I'm not clear what is wrong, I use the same pattern (eg: data.edges.node.map()) and it works for me.
The only other difference I noticed is in gatsby-config, I define an environment key. I'm not sure what the behavior is if none is defined, probably defaults to master so you may also want to confirm you're on the right environment.

Gatsby graphql query works fine in development environment, but not onPostBuild

Problem occured afer updating to Gatbsy v3 and gatsby-plugin-sitemap v3 In my gatsby-config.js, I have configuration for sitemap:
{
resolve: 'gatsby-plugin-sitemap',
options: getSitemapForLanguage('en'),
},
{
resolve: 'gatsby-plugin-sitemap',
options: getSitemapForLanguage('de'),
},
I got the the following function, which generates sitemap based on language passed:
const getSitemapForLanguage = lang => ({
output: `/${lang === 'en' ? '' : lang}/sitemap.xml`,
query: `
{
site {
siteMetadata {
siteUrl
}
}
allMdx(
filter: {frontmatter: {seo: {exclude_from_sitemap: {ne: true}}, languages: {in: "${lang}"}}}
) {
edges {
node {
frontmatter {
pathname
}
}
}
}
}`,
serialize: ({ site, allMdx }) =>
allMdx.edges.map(edge => ({
url: `${site.siteMetadata.siteUrl}${lang === 'en' ? '' : `/${lang}`}${
edge.node.frontmatter.pathname
}`,
changefreq: 'daily',
priority: 0.7,
})),
});
I created this using an in-browser IDE for graphql, that you get when you run gatsby develop. In that IDE I can see results and everything I need, but when I try to build it, I get:
TypeError: Cannot use 'in' operator to search for 'nodes' in undefined
Error of failed build process
For testing purposes I removed , languages: {in: "${lang}"} part in query, but it still doesn't work.
I don't think it even work in gatsby develop since the plugin does only work in production mode. As you can see in the gatsby-plugin-sitemap docs:
NOTE: This plugin only generates output when run in production mode!
To test your sitemap, run: gatsby build && gatsby serve
In gatsby develop is not even firing the plugin, even the query may work in the localhost:8000/___graphql playground.
In my opinion, the issue relies on the language parameter, you can't pass GraphQL variables like this in a Node file, like gatsby-config.js is. The approach is to create separate queries and serialize them somehow. Your configuration should look like this:
{
resolve: `gatsby-plugin-sitemap`,
options: {
query: `
{
site {
siteMetadata {
siteUrl
}
}
deSitemap: allMdx(
filter: {frontmatter: {seo: {exclude_from_sitemap: {ne: true}}, languages: {in: "de"}}}
) {
edges {
node {
frontmatter {
pathname
}
}
}
}
enSitemap: allMdx(
filter: {frontmatter: {seo: {exclude_from_sitemap: {ne: true}}, languages: {in: "en"}}}
) {
edges {
node {
frontmatter {
pathname
}
}
}
}
}`,
serialize: ({ site, allSitePage }) => enSitemap.edges // here you will need to serialize both or append the language at the end
.map(edge => {
return {
url: site.siteMetadata.siteUrl + path, // https://sitemaps.com/page-path
changefreq: 'daily',
priority: 0.7,
links: [
// https://sitemaps.com/page-path
{ lang: 'en', url: site.siteMetadata.siteUrl + path },
// https://sitemaps.com/es/page-path
{ lang: 'de', url: `${site.siteMetadata.siteUrl}/de${path}` },
// The default in case page for user's language is not localized.
{ lang: 'x-default', url: site.siteMetadata.siteUrl + path }
]
};
})
}
}
Source: https://github.com/gatsbyjs/gatsby/issues/4603
Don't try to split it into a separate function, at least until you ensure that is working as expected.

How to get 'Last Update Date' of a blog post in GATSBY.js

Hello I'm not a dev so may the question will be easy for you guys. I used the advance starter from gatsby site. The blog is working perfect but I need to provide the LAST UPDATED time under my title. Searched for some solutions but none of them worked. Could you Provide some help?
gatsby-node.js
exports.onCreateNode = ({ node, getNode, actions }) => {
const { createNodeField } = actions;
if (node.internal.type !== 'MarkdownRemark') {
return;
}
const fileNode = getNode(node.parent);
createNodeField({
node,
name: 'modifiedTime',
value: fileNode.mtime
});
};
`````````````````````````
PostListing.jsx
class PostListing extends React.Component {
getPostList() {
const postList = [];
this.props.postEdges.forEach(postEdge => {
postList.push({
path: postEdge.node.fields.slug,
tags: postEdge.node.frontmatter.tags,
cover: postEdge.node.frontmatter.cover,
title: postEdge.node.frontmatter.title,
date: postEdge.node.fields.date,
excerpt: postEdge.node.excerpt,
timeToRead: postEdge.node.timeToRead,
modifiedTime:postEdge.node.modifiedTime
});
});
return postList;
}
render() {
const postList = this.getPostList();
return (
<div className='posts'>
{/* Your post list here. */
postList.map(post => (
<Fragment>
<div className='singlePost__date'>
<h4 style={{color:'white'}}> {post.modifiedTime}</h4>
</div>
<div className='singlePost__Title'>
<Link classname='singlePost' to={post.path} key={post.title}>
<h1 className='singlePost__title'>{post.title}</h1>
</Link>
</div>
</Fragment>
))}
</div>
);
}
}
export default PostListing;
I expect something like
TITLE
last updated : 3/2/2019
You can use information stored in Git to get the latest time when a file was modified.
1st approach
Track it manually, but this can be error-prone if you forget to edit the modified time. So I would recommend that as the last option if you can't get others to work.
2nd approach
You can edit your gatsby-node.js to pull information from Git like so:
const { execSync } = require("child_process")
exports.onCreateNode = ({ node, actions }) => {
// ...
if (node.internal.type === "MarkdownRemark") {
const gitAuthorTime = execSync(
`git log -1 --pretty=format:%aI ${node.fileAbsolutePath}`
).toString()
actions.createNodeField({
node,
name: "gitAuthorTime",
value: gitAuthorTime,
})
}
// ...
}
Then, in your template, you can fetch it:
query($slug: String!) {
markdownRemark(fields: { slug: { eq: $slug } }) {
# ...
fields {
gitAuthorTime
}
# ...
}
}
And, finally, use it in JSX like so:
import React from "react"
const BlogPost = (props) => {
const { gitAuthorTime } = props.data.markdownRemark.fields
render(<p>Updated at: ${gitAuthorTime}</p>)
}
export default BlogPost
3rd approach
Similar to the previous one but it uses a plugin gatsby-transformer-info. It does a similar thing as in the 2nd approach, but you need to access the modified time differently this time. Like so:
query($slug: String!) {
markdownRemark(fields: { slug: { eq: $slug } }) {
# ...
parent {
... on File {
fields {
gitLogLatestDate
}
}
}
# ...
}
}
I wrote more about this in my blog post "Add Updated At To Your Gatsby Blog" if you want to check it out.
Edit: The answer below is actually wrong, since File. modifiedTime is the modifiedTime of the markdown file itself & not the modifiedTime for your content. For example, if you deploy your blog on say, Netlify, then the modifiedTime of your files there will be different than in your local environment.
I think the right answer is to track it separately. If you're using a CMS like NetlifyCMS, you can create a field that automatically update the date/time on every edit.
Wherever you're querying for your markdown files, you can use the below field:
query {
allMarkdownRemark {
edges {
node {
frontmatter { /* other stuff */ }
parent {
... on File {
modifiedTime(formatString: "MM/DD/YYYY")
}
}
}
}
}
}
And access it in your via postEdge.node.parent.modifiedTime

How to define Gatsby markdown page paths in one file?

Bu default, Gatsby uses frontmatter for defining paths, like:
---
path: /example-page
---
and then works with it via GraphQL.
What is the best way to define those paths for all markdown files by not writing frontmatter part in every file, but in one file, as an instance, like this:
[
{
"title": "Example",
"path": "/example-page.md"
}
]
You can do that by adding the path when the page is created.
Add this in gatsby-node :
const { createFilePath } = require(`gatsby-source-filesystem`);
exports.onCreateNode = ({ node, getNode, boundActionCreators }) => {
const { createNodeField } = boundActionCreators
if (node.internal.type === `MarkdownRemark`) {
const slug = createFilePath({
node,
getNode,
basePath: `pages`
})
createNodeField({
node,
name: `slug`,
value: `/pages${slug}`
})
}
};
createFilePath turn markdown files in pages directory into /pages/slug.
createNodeField creates new query'able field with name of 'slug'.
Now in graphql you can access the slug :
{
allMarkdownRemark {
edges {
node {
fields {
slug
}
}
}
}
}
Then you can create your pages as usual using the new slug field as page path.
With that you can add your title and all you want in data accessible in graphql.
Example here : https://www.gatsbyjs.org/tutorial/part-seven/

Resources