How to generate unlimited CDN-cached pages with Netlify and Contentful
Everybody's enthusiastic about the Jamstack and all these new ways to build for the web. And while the JAM in Jamstack initially stood for JavaScript, APIs and pre-rendered Markup, its meaning has evolved.
Today, even though the pre-rendering of pages still plays a significant role in Jamstack land, the approach of pre-rendering every HTML page comes with some disadvantages if your project grows.
Consider an ecommerce site that includes thousands, if not millions, of products; pre-rendering all the product pages will take time, and your build process duration will grow with every added product. That's not great!
In an ideal scenario, you could define the URLs that should be pre-rendered in the build process and generate and cache all the other million pages lazily when they're requested. Is this possible on the Jamstack?
Heck, yeah — say hello to Netlify's On-demand Builders! On-demand builders are serverless functions used to generate web content as needed that’s automatically cached on Netlify’s Edge CDN.
On-demand Builders are part of the Jamstack evolution. You can pre-render and cache the essential and most trafficked pages in your build pipeline and then fill up your CDN cache over time with all the other pages using On-demand Builders. It's the best of both worlds. And I don't need to tell you that I'm super excited about it.
To learn more about this new way of page caching, I paired with Netlify's Jason Lengstorf in a recent webinar, "Stop playing cache-up: How to render content on the fly for faster builds with Jamstack."
Watch it online to learn more about how we built it, or pick the interesting nuggets from this summary blog post.
Ready? Three, two, one… let's go!
A site to send virtual postcards
We didn't want to set up a site with thousands of pages just to play around with On-demand Builders, so we went back to the drawing board. What if we could set up a site that renders an unlimited number of pages depending on URL parameters? And what if all of you could generate these pages?
Let me introduce you to Goofy Postcard!
Goofy Postcard is a site that lets you create virtual postcards like this one. Every postcard is generated when someone hits the URL and is then cached on the CDN for all the following requests.
Folks can visit goofy-postcard.netlify.app, choose their preferred postcard motif (me or Jason), and enter a message that should go onto the postcard.
You also see that Jason and I had a little slipper argument and couldn't decide if hippos or bunnies are the best feet warmers. On that note, I hope you're #TeamHippos!
But how did we get there? Let me walk you through the steps and explain how we made Contentful the platform powering the unlimited numbers of generated and cached HTML pages.
How to create a configurable form with Contentful
One of Contentful's strengths is that it can provide content to anything that speaks HTTP. And while most of our customers use Contentful to power their web pages, you can also leverage the APIs to power forms, chat messages, microcopy; you name it. Define the required content structure and get rolling!
And that's what we did for our Goofy Postcard project. We went into Contentful and set up a new content type called Postcard option. The content type included three fields. The title of the postcard option (Jason or my name), our preferred greeting to render it on the postcard, and an image to show off our slipper preferences. Again, go Hippos! 🦛
And with this quick content model setup, we created two entries that would power our Goofy Postcard submission form and the later rendered virtual postcards. If someone else wants to join the postcard party at a later stage, we could go into Contentful, create another entry, and adjust our project without any code change.
How did the content end up on our Netlify site?
How to write Contentful GraphQL queries
Contentful provides two ways to fetch the data you put into the platform. You can use our helper libraries such as contentful.js or use the provided GraphQL API. Jason and I are big GraphQL fans, so this decision was quickly taken.
If you haven't used GraphQL yet, check out this course that teaches you how to use GraphQL, React, and Contentful in detail.
Suppose you're planning to use our GraphQL API. You don't need to install anything or read any additional docs. Every GraphQL API is self-documenting, and there's tooling that you can use to write your GraphQL queries. Our GraphQL API provides the tool GraphiQL, which enables you to poke around the content, see what data is available, and write your GraphQL queries.
I prefer to write my Contentful GraphQL queries next to the content right within the Contentful UI. So let me tell you a secret: If you want to have a GraphQL explorer within the Contentful interface, head over to the Contentful Marketplace and install the GraphQL Playground. Follow the instructions, configure the required API token and create and fetch content in the same place.
And with GraphiQL or the GraphQL playground, you'll see the power of GraphQL in action. To write your query, you can either open the documentation on the right side of the explorer to see what data is available or write your queries using the query auto-completion.
Because we knew the data structure we were dealing with, we quickly found the postcardOptionCollection field, accessed its items, and were ready to fetch the data!
How to fetch GraphQL data
For simple use cases like ours, the cross-browser supported fetch method does the trick to call the Contentful API. Define the POST HTTP method, set the API token as Authorization header, and pass the query as post body. And that's it to fetch your Contentful data!
To avoid the complexity of a build process to render the form, we decided to render it client-side right within an index.html file. Some inline JavaScript requested the data and did some basic DOM manipulation to build the form.
If you want to look at the rendering logic, check the GitHub repo.
How to render a page in a serverless function
At this stage, the Contentful-powered form performed a GET request to /.netlify/functions/postcard?type=xxx&message=hello when it was submitted. type held the id of the postcard option and message included the message that should be rendered onto the postcard.
It was time to create a new serverless route that would:
Use the provided option id to fetch the image and greeting from Contentful.
Render the postcard that includes an image, a greeting and the custom message.
Reading the URL query variables was quickly done by accessing the provided event.queryStringParameters property. And to not reinvent the wheel and reuse the index.html GraphQL code, we leveraged the cross-fetch npm package, which provides the browser fetch functionality in a Node.js environment.
With this, we had all the ingredients we needed. We used the coming in typeId to pass it over to Contentful, assembled another GraphQL query, and used the already familiar fetch method.
The HTML rendering was done with template string interpolation so it wouldn’t increase the complexity. And that was it!
We had a configurable form using Contentful data that led users to a custom Netlify serverless function to render all our goofy postcards.
One problem remained, though. The serverless function would run over and over again. If people would generate a postcard with a unique URL and share it on social media arguing that hippos are the better slippers, the serverless function would run and generate the same HTML page all the time. There was no caching in place. How could we cache the response with the first request, then?
How to cache the function response using Netlify On-demand Builders
To cache the response of a Netlify serverless function, transform it into a Netlify On-demand builder by using the @netlify/functions npm package.
Wrap your exported handler function with the builder method, and that's it! 😲
The builder method accepts all the code you've already written and then augments the response of your serverless function. It's done super quickly. Great job, Netlify!
But you have to consider some things before transforming all serverless functions into On-demand Builders.
Three differences between serverless functions and On-demand Builders
After enabling the On-demand functionality, Jason shared things to watch out for.
1. On-demand handlers only work for GET and HEAD requests
If you're considering caching POST requests, I have bad news. Not only could caching POST (or PUT and DELETE) requests lead to massive headaches, but it's also not possible using Netlify On-demand Builders.
It's a good thing to avoid folks shooting themselves in the foot.
2. On-demand handlers don't have access to query parameters
Second, at the webinar, I learned that Netlify On-demand Builders could not access query string parameters. Jason told me that they're debating whether this should be possible internally, but at this point, we needed to tweak our serverless function setup.
We relied on native browser functionality to make the postcard generation request. The form included a method="GET" attribute, and browsers will then reflect the included form inputs in URL query parameters. We liked that because we didn't need to add any client-side JavaScript to submit the form and hit our function.
But then how could we pass the query parameters to the function? Enter configurable redirects!
The following lines defined in a _redirects file will transform the query parameters to path segments in the form of a 301 redirect. /generated/postcard?type=abc&message=hello will be forwarded /postcard/abc/message/hello/.
And with some function code adjustments, we were back on track. We split the path (/postcard/abc/message/hello/) into its parts and were able to access the same information embedded in the URL! 🎉
Are you ready for the finish line? There was one last difference between serverless functions and On-demand Builders.
3. On-demand Builders live in a different directory
If you inspect the _redirects file closely, you'll see that the URL path of the serverless function matches our directory structure. Our function code was located at /netlify/functions/postcard.js and it was available online at /.netlify/functions/postcard/.
If you wrap your serverless function with the builder method to make it On-demand, the location changes from functions to builders. We had to adjust our redirect logic to point to builders.
Learn more about this fact in the Netlify docs.
And voilà — we made it.
Conclusion: A gazillion cached pages with a speedy build process
We've come a long way, friends! I'm so excited about the current state of web development. When I started building for the web, there were backend and frontend developers. And everything was incredibly complicated.
But now, I feel like I have superpowers. Jason nailed it in the webinar when he said, "I'm not a Fullstack developer, but I can build Fullstack applications." And while we were just arguing over animals in the webinar, we made it to build a flexible CMS-powered and CDN-cached demo application in not even an hour of live-coding. It's magical!
With Netlify's powerful hosting and SaaS services such as Contentful, I feel like whatever I want to build, it's just an API call away. Do you feel the same? Let us know in the Contentful community Slack and connect with fellow Jamstack developers. I’ll see you there!
PS. And for the last time — go Hippos!
