Navigate back to the homepage

How to source images and data from JSON files in Gatsby

Frederick Morin
December 1st, 2019 · 4 min read

When I started building the portfolio section of my personal site, I encountered a small problem.

I was building cards that contained both data and image and I wanted everything to appear under the same GraphQL node. Turns out there’s no way for Gatsby to recognize a path to an image in a JSON file by default.

Additionally, I needed to do some processing on the tag data before it could be usable so I started searching for a solution that would handle both cases.

Here’s what a card looks like:

NessIA.ca

Business website

javascriptreactgatsbybulma

Built for a client operating in the Business Intelligence space, this website outperforms competitors in speed, design and usability. It serves as a showcase for their services and as social proof via the Client and Partners section.

The problem

I wanted to achieve two things:

  1. Process the tag array and turn it into a usable array of objects
  2. Have Sharp process the image and get the resulting node in the same tree

My first reflex was to go for gatsby-transformer-json, but I quickly realized that this transformer wouldn’t understand that I had image paths in my data, much less process them using Sharp.

On top of it, I couldn’t preprocess my tag array nor give a specific shape to my PortfolioCard node.

Back to the docs. I knew that I needed to source images from the filesystem so I started there. The documentation was helpful in that regard.

I recommend you have a read if you’ve never done this, but here’s a summary of what you need to do:

  • Install the following plugins with yarn:
1yarn add gatsby-image gatsby-source-filesystem gatsby-plugin-sharp gatsby-transformer-sharp

or using npm:

1npm install gatsby-image gatsby-source-filesystem gatsby-plugin-sharp gatsby-transformer-sharp
  • Add the plugins to your gatsby-config.js:
1const path = require(`path`);
2
3module.exports = {
4 ...
5 plugins: [
6 `gatsby-transformer-sharp`,
7 `gatsby-plugin-sharp`,
8 {
9 resolve: `gatsby-source-filesystem`,
10 options: {
11 name: `images`,
12 // provide the path to your image folder here:
13 path: path.join(__dirname, `src`, `assets`),
14 },
15 },
16 ...
17 ],
18};

Now if you go to your GraphiQL IDE (by default Gatsby serves it on http://localhost:8000/___graphql) you’ll see an allImageSharp field which you can use to query your images processed by Sharp.

That’s great, but not exactly what I wanted.

These image nodes should appear under the corresponding PortfolioCard tree with the rest of the associated data. Not on a top-level node that I would have to query separately using the name of the image obtained in a previous query.

The solution

With some research, I ended up on sourceNodes in the Gatsby Node APIs docs.

I understood that using this API, I could grab data from the filesystem, transform it and build a node in the shape that suited my needs.

Here’s the result of my work in a step-by-step format. I’ll be using code examples from my own website so feel free to browse the source code on GitHub should you need to.

Sourcing the data

The first thing we need to do is to get the JSON data in the gatsby-node.js file.

If we were to write a plugin handling a number of use cases, we’d need to read from the filesystem using Node to source the data. But because we’re targeting a specific use case, hard coding the paths to our JSON files is the simplest thing to do:

1const path = require('path');
2const portfolio = require('./src/data/portfolio.json');
3const colors = require('./src/data/tag_colors.json');
4
5// relative path from `gatsby-node.js`
6const IMAGE_PATH = './src/assets/';

For reference, here’s portfolio.json:

1[
2 {
3 "title": "NessIA.ca",
4 "category": "Business website",
5 "description": "Built for a client operating in the Business Intelligence space...",
6 "technology": "The tool of choice these days for blazing fast websites is Gatsby...",
7 "link": "https://nessia.ca/en",
8 "image": "nessia.png",
9 "alt": "NessIA homepage",
10 "tags": ["javascript", "react", "gatsby", "bulma"]
11 },
12 ...
13]

and tag_colors.json:

1{
2 "javascript": "bg-yellow-vivid-400 text-gray-900",
3 "react": "bg-blue-400 text-gray-900",
4 "gatsby": "bg-purple-700 text-purple-100",
5 ...
6}

Now that we have access to our data, we need to build a node by adapting the code example from the docs to fit our objects.

In this example, we need to iterate over an array and build a node for each portfolio card, so all our code will be written inside a forEach loop:

1exports.sourceNodes = ({ actions, createNodeId, createContentDigest }) => {
2 portfolio.forEach((card) => {
3 // 1. Extract the card data.
4 const {
5 title,
6 category,
7 description,
8 technology,
9 link,
10 image,
11 alt,
12 tags,
13 } = card;
14
15 // 2. Build the PortfolioCard node. Note that most fields simply correspond to
16 // to our JSON data.
17 const node = {
18 title,
19 category,
20 description,
21 technology,
22 link,
23 image, // <----- Problem here
24 alt,
25 tags, // <------ and here
26 id: createNodeId(`card-${title}`),
27 internal: {
28 type: 'PortfolioCard',
29 contentDigest: createContentDigest(card),
30 },
31 };
32
33 // 3. Create the node
34 actions.createNode(node);
35 });
36};

That’s a good start. Our data now exists in a top-level node called allPortfolioCard. But what happens if we query it?

1query {
2 allPortfolioCard {
3 nodes {
4 alt
5 category
6 description
7 image
8 link
9 tags
10 technology
11 title
12 }
13 }
14}

As expected, we have a couple of issues. Both image and tags need to be processed before they can be usable.

1{
2 "data": {
3 "allPortfolioCard": {
4 "nodes": [
5 {
6 "alt": "NessIA homepage",
7 "category": "Business website",
8 "description": "Built for a client operating in the Business Intelligence space...",
9 "image": "nessia.png",
10 "link": "https://nessia.ca/en",
11 "tags": ["javascript", "react", "gatsby", "bulma"],
12 "technology": "The tool of choice these days for blazing fast websites is Gatsby...",
13 "title": "NessIA.ca"
14 },
15 ...
16 ]
17 }
18 }
19}

Let’s start with our tags array.

Processing JSON data

Right now, we have an array of strings that needs to be transformed into an array of objects with the properties name and color:

1[
2 {
3 "name": "javascript",
4 "color": "bg-yellow-vivid-400 text-gray-900"
5 },
6 ...
7]

This is a fairly simple fix. We have the color data already imported in our gatsby-node.js file. All we need to do is to map over the tag array when building our node and return an object:

1const colors = require('./src/data/tag_colors.json');
2
3exports.sourceNodes = ({ actions, createNodeId, createContentDigest }) => {
4 portfolio.forEach((card) => {
5 // ...
6
7 const node = {
8 // ...
9 tags: tags.map((name) => ({
10 name,
11 color: colors[name],
12 })),
13 // ...
14 };
15
16 actions.createNode(node);
17 });
18};

The following query will now be possible:

1query {
2 allPortfolioCard {
3 nodes {
4 tags {
5 color
6 name
7 }
8 }
9 }
10}

And will return the following data:

1{
2 "data": {
3 "allPortfolioCard": {
4 "nodes": [
5 {
6 "tags": [
7 {
8 "color": "bg-yellow-vivid-400 text-gray-900",
9 "name": "javascript"
10 },
11 {
12 "color": "bg-blue-400 text-gray-900",
13 "name": "react"
14 }
15 // ...
16 ]
17 }
18 // ...
19 ]
20 }
21 }
22}

As you can see, when building a custom node, we can do whatever we want with the data. We can pass it down as it is or transform it into a desired shape.

But what about the image field?

Transforming an image path into a childImageSharp node

Now that’s the tricky part.

It took a bit of digging, but I learned from this Stack Overflow answer that in order for Sharp to transform an image, it needs to be a File node.

So all we need to do is to create such a node by giving it the required fields.

gatsby-transformer-sharp only checks if a node has the field ‘extension’ and — if it is one of the valid file types — processes it.
Derek Nguyen on Stack Overflow

The whole answer is worth reading to deepen your understanding of how Gatsby operates under the hood. Let’s go ahead and implement it.

1exports.sourceNodes = ({ actions, createNodeId, createContentDigest }) => {
2 portfolio.forEach((card) => {
3 // ...
4
5 // 1. name, extension and absolute path are required to build a File node
6 const { name, ext } = path.parse(image);
7 const absolutePath = path.resolve(__dirname, IMAGE_PATH, image);
8
9 // 2. Build a data shape that corresponds to a File node that Sharp can process
10 const data = {
11 name,
12 ext,
13 absolutePath, // <-- required
14 extension: ext.substring(1), // <-- required, remove the dot in `ext`
15 };
16
17 // 3. Build the image node using our data
18 const imageNode = {
19 ...data,
20 id: createNodeId(`card-image-${name}`),
21 internal: {
22 type: 'PortfolioCardImage',
23 contentDigest: createContentDigest(data),
24 },
25 };
26
27 // 4. Create the node. When imageNode is created,
28 // Sharp adds childImageSharp to the node
29 actions.createNode(imageNode);
30
31 const node = {
32 // ...
33 // 5. Add the image node to our tree
34 image: imageNode,
35 // ...
36 };
37
38 actions.createNode(node);
39 });
40};

Now when you go back to your Graph Explorer, your should see a childImageSharp node under the image field.

You can then query for it and use it in conjunction with gatsby-image.

Here’s what the final query looks like:

1query {
2 allPortfolioCard {
3 nodes {
4 image {
5 childImageSharp {
6 fluid(maxWidth: 384) {
7 ...GatsbyImageSharpFluid
8 }
9 }
10 }
11 alt
12 category
13 description
14 technology
15 link
16 tags {
17 color
18 name
19 }
20 title
21 }
22 }
23}

Gatsby is pretty powerful out of the box, and even more once you start understanding how it works. I’m still barely scratching the surface but the more I play with it, the more I’m amazed with what it can do.

The key takeaway here is that whether you need to process your data before sending it to your GraphQL tree, or you need images processed by gatsby-plugin-sharp under a specific node, using the sourceNodes API in conjunction with the createNode action will help you achieve your goals.


I hope this post was instructive. If you found it helpful, follow me on Twitter to be notified when I post the next one! I’ll have more posts about Gatsby and the modern front end stack coming soon.

More articles from freddydumont

Master Git, not the command line

Learning Git isn't about memorizing commands. It's about understanding version control. I suggest using a graphical user interface (GUI…

October 12th, 2019 · 2 min read

How to learn web development the right way

Every now and then I meet someone who wants to get into software development. Most of the time they don't know where to start. My advice to…

August 28th, 2019 · 2 min read
© 2019 freddydumont
Link to $https://twitter.com/_freddydumontLink to $https://github.com/freddydumontLink to $https://www.linkedin.com/in/freddydumont/