RFC 100: Enhancing headless support in Wagtail core

[thibaudcolas]

View as an HTML document. This RFC attempts to set a direction for Wagtail’s future headless support improvement. It covers:

1. A definition of Wagtail’s support goals when it comes to headless support. Long-term, that’ll help us decide what headless-related capabilities should be in Wagtail itself, vs. in packages, vs. for site implementers to build. This is the most crucial thing to validate as part of the RFC process.
2. An overview of the current areas of interest and known opportunities for improvement. This is meant more as a recap of current points of friction, gaps, opportunities, than a mandate that we must address them all.

I’ve also published a 2024 Wagtail headless survey so we get input from a bigger group of developers. Some of you might recall the 2022 survey, which helped us tremendously back then in documenting the current state. Those survey results will help us understand where it’d be most helpful to direct headless support contributions.

[ahosgood]

I had to add extra code to get redirects to work in the API - would that be helpful to have included?

[thibaudcolas]

@ahosgood I would assume so? Redirects is a contrib module so not as "core" as some of the more fundamental aspects of the CMS, but certainly something that many sites would consider core CMS functionality.

[lb-]

I think it would be good to somehow include the pattern for JSON rendering using normal Page routing. Essentially each Page path can have a different request header and then return JSON instead of HTML.

It's quite an intuitive approach and aligns with how other CMSs (e.g. Adobe Experience Manager) can provide their APIs. It's not going to make sense for every installation but worth reviewing as part of this RFC.

In the abstract, this also could align with applications that may want to build out HTMX style applications, still returning HTML but providing a partial render of 'inner' HTML based on request headers.

See wagtail/wagtail#11752 & wagtail/wagtail#8374

[saevarom]

Headless was on the agenda for Wagtail Space NL but this only involved incorporating the areweheadlessyet.org website into the Wagtail documentation. The resulting PR is here: wagtail/wagtail#12039
The sprint topic discussion is here: https://paper.dropbox.com/doc/Sprint-topics-Wagtail-Space-NL-2024--CUVkcdc_H1K~k2BWz8SS9UUbAg-3rkJ5imATtyajKY2Ey1Cr#:uid=172696553243645599787735&h2=Headless-Wagtail

[lb-]

Another long-standing issue with the current API is that we cannot easily generate an OpenAPI specification

See wagtail/wagtail#6209

I would say that we must have a documented or even out of the box way to generate specifications for our APIs if we want to truly say we have headless support.

[thibaudcolas]

Thank you everyone for the feedback! I’ve heavily updated the RFC. Now’s the time for further reviews and feedback (approval?) The RFC now has two well-separated sections:

1. A definition of Wagtail’s support goals when it comes to headless support. Long-term, that’ll help us decide what headless-related capabilities should be in Wagtail itself, vs. in packages, vs. for site implementers to build. This is the most crucial thing to validate as part of the RFC process.
2. An overview of the current areas of interest and known opportunities for improvement. This is meant more as a recap of current points of friction, gaps, opportunities, than a mandate that we must address them all.

I’ve also published a 2024 Wagtail headless survey so we get input from a bigger group of developers. Some of you might recall the 2022 survey, which helped us tremendously back then in documenting the current state. Those survey results will help us understand where it’d be most helpful to direct headless support contributions.

If you want to help

1. Review the RFC. You’re here already so you’re aware of how this all works and how important input is.
2. Take the survey and share it with colleagues. Should be a quick one.
3. Upvote headless issues?

We’re not big on issue emoji reactions / votes as a way to make decision, but there’s a big corpus of existing tickets here so could be interesting. Here are all open issues sorted by theme:

Headless-related existing issues

• REST API
  • Allow unlimited API results with limit=-1 #12604
  • Wagtail API: Greater than/less than filters #7497
  • Wagtail API: OR filtering related models #5755
  • Add ability to include and filter via detail fields on pages' list endpoints #11404
  • Random order in the API leading to an AttributeError #3512
  • Wagtail API should have a browsable root URL #1921
  • Duplicate db query on api detail page #6133
  • Adding per-subclass filter backend/class to the v2 API Pages Endpoint #3403
  • Future release enhancement: Implement APIField for listing_default_fields, too #3562
  • Document nested_default_fields in APIv2 #11587
  • Add a UUIDField to the Page model #6162
  • Support customizing/encoding IDs in the API #6917
• API schemas
  • Support OpenAPI Schema generation for Wagtail API #6209.
• Images
  • ImagesAPIViewSet query params for rendition? #7973
  • No Image URL in the API for ImageChooserBlock #2087
  • Feature request: Include image's title in ImageRenditionField #5435
  • Documentation for using the "Dynamic image serve" view for generating image renditions on a headless site #6113
• Page URL Routing
  • Keep queries other than html_path in /api/v2/pages/find/ #6577
  • Finding pages by HTML Path - redirect missing port #7595
  • Improvements to API router reverse lookups. #11310
  • Add ability to support custom renderers for Page views (e.g. JSON API response at page URLs) #11752
  • Accept page change in before_serve_page handling #5753
• Redirects
  • Refactor redirect middleware to use its lookup logic for redirects API #11524.
• Rich Text
  • API should provide access to the HTML-converted versions of rich text fields #2695
• Multi-site support
• Form submissions
  • Ability to accept Form Builder Page as a POST request to the API (headless) #6950
• Password-protected Pages
• Internationalisation
  • Support internationalized routes in API pages/find/?html_path view #6679.
• StreamField
  • Override serializing StreamBlocks to JSON api #3454
  • Apply custom-per-block-type StreamField serializers #5174
  • parent_context or just "parent Page" for Block.get_api_representation() #7976

---

Last thing, re long-standing issues and schema specifications: At this stage I think we need to move past the No True Scotsman chain of thought, asking ourselves whether Wagtail is truly headless or not. Wagtail is a hybrid system, it’s pretty clear. Some features aren’t headless-compatible, but clearly there’s hundreds if not thousands of headless sites out there built with Wagtail, some pretty high-profile. So yes it’s truly headless. We don’t want to mislead people, so there are gaps to fill (docs in particular), but it’s already a thing and has been for years.

[itzomen]

A frontend client like wagtail-js will make integrating wagtail for many frontend developers easier.

[thibaudcolas]

@itzomen I’m not sure I follow your point, as you shared it, that front-end client already exists? Is there a need for more than one JS client?

[itzomen]

@itzomen I’m not sure I follow your point, as you shared it, that front-end client already exists? Is there a need for more than one JS client?

Oh no, I meant the continuous development of a JS client (could be that one or something else) will be a great addition to the efforts to promote the usage of Wagtail as a headless cms.

And this is definitely something I will like to provide support with

[thibaudcolas]

Ah yes! that makes a lot of sense :) I just looked at the analytics on our now-retired "Are we headless yet?" website, the most-viewed page was REST API support, so a JS client working with the REST API is a big win.