Hydrogen Mutations RFC

[blittle]

Hydrogen Mutations RFC

Last updated: June 29, 2022

Demo video: https://screenshot.click/29-13-ju7oi-exrd2.mp4

At the moment Hydrogen has mediocre support for updating information on the server. You can create an API Route and manually fetch the from the client, but there are a number of shortcomings.

Consider this contrived example where a client component submits data to an API Route:

// src/routes/counter.server.jsx
import ClientCounter from '../components/counter.client.jsx';

let value = 0;

export default function Counter() {
  return <ClientCounter value={value} />
}

export async function api(request) {
  value += 1;
  return new Response("Success");
}

// src/components/counter.client.jsx
export default function ClientCounter({value}) {
  async function sendAction() {
    const response = await fetch('/counter', {
	  method: 'POST',
	});
  }
  
  return <>   
    <h1>{value}</h1>
    <button onClick={sendAction}>Increment</button>
  </>;
}

There are a few problems with the above example:

1. The App has to load and hydrate before you can interact with the buttons. What if you could click the buttons before JavaScript had finished loading?
2. Usually after a mutation, you need to re-render. In this case, value is defined within a server component. At the moment the rendered value remains stale. How do we re-render that server component after the mutation? We could refresh the whole page, or come up with a server prop hack, but there is no easy performant way to do it. Either of those approaches also require at least an additional round trip to the server.
3. This example does not handle error or loading states. What if we could automatically provide those?

The goal of this RFC is to introduce new primitives to API Routes that allow them to easily re-render server components. We also introduce an official <Form> component, to make interacting with API Routes easier and more performant.

<Form> component

Inspired by the Remix Form, the Hydrogen <Form> provides a declarative way to perform mutations: creating, updating, and deleting data. Consider the above client component rewritten:

// src/components/counter.client.jsx
import {Form} from '@shopify/hydrogen';

export default function ClientCounter({value}) {  
  return <Form action="/counter" method="post">   
    <h1>{value}</h1>
    <button type="submit">Increment</button>
  </Form>;
}

Notice that we don't have to manually call fetch. Nor is there an onClick handler. In fact, this entire component could be moved into the server component. That's right, no client components! Additionally, using the <Form> component means that the <button> still works even before the page, and JavaScript, has completely loaded and hydrated. The component automatically submits the Form by a fetch request, and falls back to a native form request when JavaScript hasn't loaded.

The <Form> component has all of the standard props of a native <form> element, except for the following:

onSubmit

The form component takes an onSubmit event handler. If event.preventDefault() is called, no fetch request will be made. Also, the native event form submission won't take place.

children

The children can optionally be a render prop, where we pass the form submission status. Arguments for both loading and error states:

// src/components/counter.client.jsx
import { Form } from "@shopify/hydrogen";

export default function ClientCounter({ value }) {
  return (
    <Form action="/counter" method="post">
      {(loading, error) => (
        <>
          <h1>{value}</h1>
          {error ? <div class="error">{error}</div> : null}
          <button type="submit" disabled={loading}>Increment</button>
        </>
      )}
    </Form>
  );
}

Note: Using a render prop is only available within client components, because functions as props are not serializable.

Rendering Server Components from API Routes

After a Form has been submitted, server components need to re-render to ensure that any updates to the data are reflected in the UI. Instead of returning a new Response(), the API route can return rendered server components with renderRsc():

// src/routes/counter.server.jsx
import {renderRsc} from '@shopfiy/hydrogen';
import ClientCounter from '../components/counter.client.jsx';

let value = 0;

export default function Counter() {
  return <ClientCounter value={value} />
}

export async function api(request) {
  value += 1;
  return renderRsc();
}

renderRsc takes an configuration object argument, with the following properties:

pathname?: string

By default, calling renderRsc will re-render the server component for the current route. Optionally, you can pass the pathname to an entirely different route. The rendered results will display in the client, and the browser will update the URL to the new route. Consider a login page, where on success, the user is redirected to an account page:

export default function Login() {
  return <SomeLoginForm />
}
export async function api(request, {session}) {
  const formData = await request.formData();
  const username = formData.get('username');
  const password = formData.get('password');
  if (validate(username, password)) {
    session.set(...);
    return renderRsc({pathname: '/account'}); // On login success, redirect to the account page
  }
}

Note: Data form the <Form> component submission is available from within the API Route by a native FormData implementation.

props?: Record<string, any>

When re-rendering server components, sometimes you might want to update the server props in the server components. You can do so by passing data with props. This is particularly are still useful for form validation:

export default function Login({loginError}) {
  return <SomeLoginForm loginError={loginError} />
}
export async function api(request, {session}) {
  const formData = await request.formData();
  const username = formData.get('username');
  const password = formData.get('password');
  if (validate(username, password)) {
    session.set(...);
    return renderRsc({pathname: '/account'}); // On login success, redirect to the account page
  }
  
  // `loginError` will be passed directly to the Server Component above
  return renderRsc({props: {loginError: "Invalid username or password"}});
}

Note: This prop data is ephemeral and just like all server props, doesn't survive a hard page refresh.

Next steps

The <Form> component and renderRsc function are implemented as outlined in this PR: Shopify/hydrogen#1552. There you can see a more realistic example of a login form and associated login API route.

Long term we anticipate the <Form> component being used for most, if not all, mutations in Hydrogen.

Questions

1. Is there a better way to provide <Form> submission state than a render prop?
2. Is there a better term for renderRsc? Other ideas include: renderRoute, renderPage, getPage. Or because it's really constructing a new Request object, we could make it slightly more verbose:

import {getRscUrl} from '@shopify/hydrogen';

export async function api() {
  ...
  return new Request(getRscUrl());
  ...
}

3. Can we remove renderRsc altogether and instead just have the API route return a new Request():

export async function api() {
  ...
  // If this API Route is hit by the <Form> component, then automatically translate the Request into
  // an RSC Request. If the API Route is hit by anyone else, then instead 303 redirect them to `/login?error=invaliduser`
  // Instead of passing server props with the new request, just pass search params
  return new Request('/login?error=invaliduser');
}

4. Down the road, we will have sub-tree RSC re-rendering. Whatever we do here should not prevent renderRsc from rendering a sub-tree.

[davidhousedev]

Have you considered type safety for the props object in renderRsc? Ideally, if I update the props interface for a server component, I should receive a type error in any API routes that render it with a different props structure. Perhaps renderRsc may be generic on props?

[blittle]

Great question. There would need to be a way to declare the types for the props. At the moment, server props are only set in the client, and transmitted to the server to re-render server components. This proposal has a new mechanism for defining them in the API Route and sending them to the server components. I think we'd need to brainstorm a way to declare the types in a dev friendly way 🤔

On another note, I'm personally starting to question the usefulness of server props. They should only ever be used for UI that is ephemeral, meaning UI state that resets on a full page refresh. Most state you want to be present with the URL, so it is sharable, bookmarkable, and the browser back/forward buttons work. Also, during the scenario where JS is disabled (or not yet hydrated) and a native form request is sent, the server needs to respond with a 303 redirect to prevent weird bookmark behavior. Server props get lost in that redirect, and it's impossible to keep them unless the state is moved to the URL (search params), or a cookie is used instead. Remix uses session.flash for those scenarios.

[davidhousedev]

I think we'd need to brainstorm a way to declare the types in a dev friendly way 🤔

I've been thinking about this too... it's difficult to imagine a method that doesn't require the use of generics and potentially the use of an imported props interface e.g. import { HomePageProps } from 'src/routes/HomePage/types.ts

They should only ever be used for UI that is ephemeral, meaning UI state that resets on a full page refresh. Most state you want to be present with the URL, so it is sharable, bookmarkable, and the browser back/forward buttons work.

Cool! This wasn't intuitive to me, but it makes a lot of sense. Personally, I really like storing UI state in the URL; maybe Hydrogen could support using the URL for state like this at the framework level?

[blittle]

I totally understand they aren't intuitive, and that in itself probably is a code smell, because they are kind of unnatural compared to primitives in other frameworks. They also are never populated when a page initially loads, only after the client sets them. So even if a server component somehow declares the types of the props it accepts, inherently they are always optional, and in most cases won't be populated at all. I've seen some implementations that first look for a server prop, then fall back to a URL search param. But at that point, why not exclusively use search params? I think server props are confusing and easy to misuse.

maybe Hydrogen could support using the URL for state like this at the framework level

This might be a good way forward, we find though that a lot of devs are very particular on how the URL looks, so I'd be a little worried about too much magic in the URL. Maybe instead we just provide better primitives for interacting with (and strictly typing) the URL more generally? 🤔

[blittle]

We are leaning towards removing renderRsc and instead requiring the API route to return a new Request(). Usually an API route returns a Response instance. But it can also return a Request. The idea here is that you are requesting another page to render as the response. So if the API Route is hit by the <Form> component, then automatically translate the Request into an RSC Request. If the API Route is hit by anyone else, then instead 303 redirect them to the destination.

export async function api(request) {
  // Render a GET request to the current URL
  // Useful when the API Route handles the POST and a server component handles the GET for the same route
  return new Request(request.url);

  // Redirect the user to a new route
  return new Request('/account');

  // Most Request's should proxy the headers of the original request
  return new Request('/account', { headers: request.headers });

  // Data with the new request must be passed with search params
  return new Request('/login?error=invaliduser');
}

Pros

1. Use the standard Request API.
2. Data passed by search params follow the 303 redirect scenario. They do not with the original proposal with renderRsc.

Cons

1. Verbose. Most use-cases you will need to pass the headers from the original request. This is easy to forget. Though we could maybe magically proxy those automatically?
2. Without server props, there isn't an easy way to ephemerally render a response. For example on form validation, where you want to render an error in the form submission, but on hard refresh the error message goes away. Though to do this properly we should probably use a session based mechanism, similar to remix session.flash()

[davidhousedev]

This sounds like it'll work well!

As I imagine how this Form component will be used, I imagine use cases where we would fetch cached data but bypass the cache because we know it has been updated in the remote source. For example, we might have a cached request for the customer's preferred shipping address as it's unlikely to change. Then, when we know that a shipping address update has been performed, we'd want to fetch fresh resources. Is this use-case accounted for? Could we potentially disable the cache on queries following a Form request via some sort of prop passed to server components? If so, I'm not sure that the cache would persist that now-current data.

We currently make heavy use of this pattern via React Query in order to reduce API chatter in a single page app. I wonder if we can support a similar pattern in Hydrogen?

[blittle]

@davidhousedev There's full-page cache and also request cache. For full-page cache, these mutations are non-GET, so that alone should mean they shouldn't be cached. Query cache is different. Each useQuery and useShopQuery used to build a page on the server can be cached. For your given scenario, I don't they should be cached at all, because you'd want them to provide different results for different requests to the server (different logged in users). Maybe we will down the road have session aware caching? cc @wizardlyhel

[wizardlyhel]

I'm gonna assume that the cached shipping address is used for pre-filling address forms for cases like making shipping estimates, check location based inventories and initial default address for cart.

In that case, yes I agree that these queries should be cache-able at the session level on the client browser side. I think these could live along side with the SessionStorage? Since all we need is the result data so that we don't need to make chain request to do a certain operation (ie. api: get customer preferred address -> api: make estimate shipping cost).

These queries should never be server cached with useQuery or useShopQuery or full page cache.

[davidhousedev]

Got it, thank you both for setting me straight! I think we can safely say that there are no concerns on our end about this moving forward 🚀

[osseonews]

I came to this post from another discussion. My non-expert opinion is that you seem to be reinventing the wheel here. All the problems you bring up in the original contrived example are easily solved by React-Hook-Form and React-Query, which are 2 proven libraries. Worried about user hitting submit? Reac-Hook-Form easily solves that for you. Worried about re-rendering? React-Query handles that for you easily OnSuccess=>invalidateQueries.

Now admittingly, the above are client-side libraries, and I understand you are approaching this from an RSC standpoint, but I think certain things are just better handled by proven client-side libraries, especially Forms, and you can't use RSC for everything. I think you will end up basically running into the same issues that React-Hook-Form already solved, and probably create new unnecessary bugs. Why not just make things simple and use something proven? This will save alot of time, reduce bugs, and also greatly increase the visibility/value of the platform, given the sheer popularity of libraries like React-Hook-Form and React-Query.

[blittle]

Thank you @osseonews for looking over the RFC. The key problem we are trying to solve is that most data-fetching happens in server components, which inherently only execute on the server. When you mutate data, those server components must get re-rendered. React-query is unable to do that for us. Additionally, moving that logic out of the client has performance benefits:

1. Less JS downloaded to the browser
2. You can interact with the page without the JS loaded. Consider a slow connection, where the "Add to Cart" button is visible, but the page, and subsequently JavaScript, has yet to finish loading and hydrate. If purely on the client, that button will not work. Moving the logic to the server means that it can.

[davidhousedev]

@osseonews Just my $0.02: If you prefer to use React Query and React Hook Form, I think those could totally work in Hydrogen if you used a custom client entry component. There are some nice optimistic update capabilities you'd get out of using React Query that you wouldn't get as easily from this RFC.

[osseonews]

Thanks @blittle I completely understand what you are trying to solve. I just think the solution will bring other new issues, when a simpler solution is already available. I guess I just don't believe that an RSC is necessarily the right solution for every problem, especially Forms, but maybe I'm just misunderstanding the code, so probably best to reserve final judgement once this is actually implemented.

In any case, one more quick question: Since this is just a React platform ultimately, I am assuming there is nothing wrong with just creating a _client.tsx, and then using React-Hook-Form, completely bypassing the Shopify Form component. The onSubmit would then just submit to an API route, and the client component will just re-render based on the success of the API response. This is how we do it now in every other app, and it works fine, so I'm assuming there is no obligation to use shopify Form? As regards to the data shown on the client, we would also just use React-Query on the client side to refetch the data upon a successful API response. We would fetch data with API routes with queryShop. What is wrong with this conventional approach for components that will use Forms?

Thanks again for the quick responses. I am truly impressed with this platform and it's amazing that a company the size of Shopify is building this and doing it open source. It gives me great confidence in the future of the platform.

[osseonews]

@davidhousedev Thanks! I believe we submitted our comments at the same time, so I guess you answered my question!

[KevinMind]

I haven't looked at how you might implement this yet, but in most cases render props can be replaced with hooks. If I were coming to this framework with no context I would see render props as an old pattern and might make me turn away from hydrogen. I wish js development weren't this way, but it is 🤷

For how you could implement this. I could see a number of potential routes.

Hook returns Form

const {Form, state, submit} = useForm(...payload); // payload could be something like a field configuration

ID on form with context tracking state

// src/components/counter.client.jsx
import {Form, useForm} from '@shopify/hydrogen';

const FORM_ID = 'my-form';

export default function ClientCounter({value}) {
   const {loading} = useForm(FORM_ID);
  return <Form action="/counter" method="post" formId={FORM_ID}>   
    <h1>{value}</h1>
    <button type="submit">Increment</button>
  </Form>;
}

I haven't wrapped my head around how exactly this would work, so take this with a grain of salt. It does make me wonder how <Form would relate or interop with field components like <input>? Does a form have a concept of the fields it contains? How is that modelled. The current example is pretty simple and not representative of an actual form with fields.

What about validation? Is that beyond the scope of this discussion?