[FEATURE] Add a method for handling expansion of child objects in jbuilder

[wwahammy]

Currently, there's no way to expand child objects in jbuilder files. Ultimately, we should be able to do this. As an example, look at the following (simplified) transaction object for:

// TRANSACTION OBJECT 1

{
  "object": "transaction",
  "id": "trx_35n1235o",
  // other attributes hidden for simplicity
  "supporter": "supp_35135no",
  "subtransaction": "offlinetrx_t35h23o5"
}

A user may instead want the subtransaction expanded so they don't have to make a second request:

// TRANSACTION OBJECT 2

{
  "object": "transaction",
  "id": "trx_35n1235o",
  // other attributes hidden for simplicity
  "supporter": "supp_35135no",
  "subtransaction": {
    "id": "offlinetrx_t35h23o5",
    "object": "offline_transaction",
    "amount": {
      "cents": 4000,
      "currency": "usd"
    },
    // other attributes hidden for simplicity
    "subtransaction_payments": [
      {
        "id": "offtrxchrg_ewiothat",
        "object": "offline_transaction_charge",
        "type": "payment"
      }
    ]
  }
}

As user may want the supporter and the subtransaction expanded both:

// TRANSACTION OBJECT 3

{
  "object": "transaction",
  "id": "trx_35n1235o",
  // other attributes hidden for simplicity
  "supporter": {
    "id": "supp_35135no",
    "object": "supporter",
    "name": "Penelope Schultz",
    // other attributes hidden for simplicity
  },
  "subtransaction": {
    "id": "offlinetrx_t35h23o5",
    "object": "offline_transaction",
    "amount": {
      "cents": 4000,
      "currency": "usd"
    },
    // other attributes hidden for simplicity
    "subtransaction_payments": [
      {
        "id": "offtrxchrg_ewiothat",
        "object": "offline_transaction_charge",
        "type": "payment"
      }
    ]
  }
}

Additionally, a user may want the subtransaction payments expanded, but now the supporter:

// TRANSACTION OBJECT 4

{
  "object": "transaction",
  "id": "trx_35n1235o",
  // other attributes hidden for simplicity
  "supporter": "supp_35135no",
  "subtransaction": {
    "id": "offlinetrx_t35h23o5",
    "object": "offline_transaction",
    "amount": {
      "cents": 4000,
      "currency": "usd"
    },
    // other attributes hidden for simplicity
    "subtransaction_payments": [
      {
        "id": "offtrxchrg_ewiothat",
        "object": "offline_transaction_charge",
        "type": "payment",
        // other attributes hidden for simplicity
        "gross_amount": {
          "cents": 4000,
          "currency": "usd"
        },

        "net_amount": {
          "cents": 3700,
          "currency": "usd"
        },

        "fee_total": {
          "cents": 300,
          "currency": "usd"
        }
      }
    ]
  }
}

A proposed solution

We should be able to pass an object describing the expansions to a call to render a jbuilder partial. I'll explain the object description and how the call should be used.

Expansion descriptions

I believe a simple mechanism for expansion descriptions would be to provide an array that has the dot paths to parts of the JSON to expand from the root element. Here are the expansions for each of examples:

//TRANSACTION OBJECT 1
obj_1_expansion = [] // nothing to expand

//TRANSACTION OBJECT 2
obj_2_expansion = ["subtransaction"] // expand the subtransaction from the root object

//TRANSACTION OBJECT 3
obj_3_expansion = ["subtransaction", "supporter"] // expand the subtransaction and supporter from the root object

//TRANSACTION OBJECT 4
obj_4_expansion = ["subtransaction", "subtransaction.subtransaction_payments"] // expand the subtransaction from the root object and then subtransaction_payments from the subtransaction object

//OR

obj_4_expansion_ideal = ["subtransaction.subtransaction_payments"] // Since you have to expand the subtransaction in order to expand the subtransaction_payment.

Passing to a jbuilder template

From the level of the controller, you can pass the expansions by setting the @__expand variable. (Named like this to avoid any sort of collisions with something named @expand). Let's show how this would look in TransactionController if we always want to expand the supporter.

class Api::TransactionsController < Api::ApiController

  # OTHER CONTENTS REMOVED FOR SIMPLICITY
	def show
		@transaction = current_transaction
    @__expand = ['supporter'] ## let's assume we want always want to expand the supporter
	end
end

Sanitizing expansion requests

We usually want to allow an API user to provide the specific expansions they want. On a transaction object, users may want supporter expanded while others may want the subtransaction expanded. To that end, we can have the user send a parameter named __expand which contains an array of dot paths to expand.

We probably should not allow an unlimited set of expansions, at least for all users. A user could request a really large set of expansions which might make the requests go slowly and overload the server. For example, let's say from the transaction element, request a super long expansion like: subtransaction.subtransaction_payments.transaction.subtransaction.subtransaction_payments.transaction.subtransaction.subtransaction_payments.transaction.subtransaction.subtransaction_payments.transaction.subtransaction.subtransaction_payments.transaction.subtransaction.subtransaction_payments.transaction.subtransaction.subtransaction_payments.transaction.subtransaction.subtransaction_payments.transaction.subtransaction.subtransaction_payments.transaction.subtransaction.subtransaction_payments.transaction.subtransaction.subtransaction_payments.transaction. That's a valid expansion but it's pointlessly long. Therefore, we should likely prevent an average user from expanding more than one level deep (perhaps super_admins could have more?).

We would create a method for controllers called sanitized_expansions which would:

• turns into params['__expand'] into an containing the dot path to be expanded. This will also remove duplicates, for example. If they request:
  ["supporter", "subtransaction", "subtransaction.subtransaction_payments"], they'll get an object representing a tree that looks, more or less, like this:

{
  "supporter": null, // no children so it's null
  "subtransaction": { // `subtransaction` and `subtransaction.subtransaction_payments` gets combined to `subtransaction.subtransaction_payments`
                      // since `subtransaction` needs to be expanded for `subtransaction.subtransaction_payments`
    "subtransaction_payments": null //no children so it's null
  }
}

• here we also remove any dangerous expansions, which is likely only ones which are too long.
• return the result

It would then could be used as follows:

class Api::TransactionsController < Api::ApiController

  # OTHER CONTENTS REMOVED FOR SIMPLICITY
	def show
		@transaction = current_transaction
    @__expand = sanitized_expansions
	end
end

passing from a template to a partial

Once you're in a jbuilder template, you need to pass the expansions to a partial, you can use it like shown in the app/views/api/transactions/show.json.jbuilder

json.partial! @transaction, as: :transaction, __expand: @__expand ## TODO: I have to verify this works

passing between partials

Once in a partial, you can add use the handle_expansion helper method (maybe a better name exists?) like shown in in this example app/views/api/transactions/_transactions.json.jbuilder

json.(:transaction, :id)

json.created transaction.created.to_i

# irrelevant properties removed for clarity

handle_expansion(:supporter, transaction.supporter, {json:json, __expand:__expand})

handle_expansion here accepts:

• the attribute name, which is supporter
• the object which may or may not be expanded
• a hash with:
  • as: the variable for the object for passing into the partial, defaults to the attribute_name
  • json: the jbuilder object
  • the __expand variable (we might be able to get this automatically)

If __expand is not set to expand supporter, this is the equivalent of:

json.supporter transaction.supporter.id

if __expand is set to expand supporter, this is the equivalent of:

json.supporter do
  json.partial! transaction.supporter, as: :supporter, __expand: __expand.children_of('supporter') # children_of gets the part of the tree below supporter, in this case, an empty tree
end

For array elements, we use the handle_array_expansion and handle_item_expansion methods. Let's assume we're in the subtransaction partial now and this is for subtransaction_payments. we would use it as follows:

handle_array_expansion(:payments, subtransaction.subtransaction_payments, {json:json, __expand:__expand}) do |payment, opts|
  handle_item_expansion(payment, {json:opts.json, as: opts.item_as, __expand: opts.__expand})
end

handle_array_expansion here accepts:

• the attribute name, in this case payments

• the object decide on how to expand, in this case subtransaction.subtransaction_payments

• as hash with:

  • json: the jbuilder object
  • the __expand variable (we might be able to get this automatically)
  • the item_as: the name of the variable for passing the item into handle_item_expansion

• a block for displaying the item which has two parameters:

  • the array item
  • an opts hash which contains:
    • json: the jbuilder object
    • __expand: which is the result of __expand.children_of(attribute_name) as passed into the object
    • as: the name of the variable when passed into a partial for json

handle_item_expansion accepts:

• the object which may or may not be expanded
• a hash with:
  • json: the jbuilder object
  • as: the name of the variable when passed into a partial
  • __expand: the expand variable for what can be expanded in the partial

Result

If payments is not supposed to be expanded, this is the equivalent of:

json.payments subtransaction.subtransaction_payments do |payment|
  json.id payment.id
  json.object 'offline_transaction_charge'
  json.type 'payment'
end

If payments is supposed to be expanded this is the equivalent of:

json.payments subtransaction.subtransaction_payments do |payment|
  json.id payment.id
  json.object 'offline_transaction_charge'
  json.type 'payment'
  json.supporter payment.supporter.id
  # removed other attributes for simplicity
end

If payments.supporter is supposed to be expanded, this is the equivalent of:

json.payments subtransaction.subtransaction_payments do |payment|
  json.id payment.id
  json.object 'offline_transaction_charge'
  json.type 'payment'
  json.supporter do
    json.id payment.supporter.id
    json.object 'supporter'
    json.name payment.supporter.name
    # removed other attributes for simplicity
  end
  # removed other attributes for simplicity
end

[wwahammy]

@clarissalimab can I have a review of this with any feedback?

[github-actions]

Thanks for submitting your first issue to Houdini!

[clarissalimab]

I think it's a very good solution!

Just to make sure I understand correctly, each .json.jbuilder would have a handle_expansion method call to each attribute that can be expanded? For example, app/views/api/transactions/_transactions.json.jbuilder would have a call to :supporter, and a handle_array_expansion call to :subtransactions (and any other calls to other attributes that can be expanded from transactions).

[wwahammy]

Thanks for submitting your first issue to Houdini!

Thanks @github-actions, I'm eager to be involved in such an exciting project. 😆

[wwahammy]

I think it's a very good solution!

Just to make sure I understand correctly, each .json.jbuilder would have a handle_expansion method call to each attribute that can be expanded? For example, app/views/api/transactions/_transactions.json.jbuilder would have a call to :supporter, and a handle_array_expansion call to :subtransactions (and any other calls to other attributes that can be expanded from transactions).

Oops, yes! So the various jbuilder templates would be:

# app/views/api/transactions/_transactions.json.jbuilder

json.(transaction, :id)

json.created transaction.created.to_i

handle_expansion(:supporter, transaction.supporter, {json:json, __expand:__expand})

handle_expansion(:subtransaction, transaction.subtransaction, {json:json, __expand: __expand})


# app/views/api/subtransactions/_subtransactions.json.jbuilder


json.(subtransaction, :id)

json.created subtransaction.created.to_i

handle_expansion(:transaction, subtransaction.trx, {json:json, __expand:__expand})

handle_expansion(:supporter, subtransaction.supporter, {json:json, __expand:__expand})

handle_array_expansion(:payments, subtransaction.subtransaction_payments, {json:json, __expand:__expand}) do |payment, opts|
  handle_item_expansion(payment, {json:opts.json, as: opts.item_as, __expand: opts.__expand})
end

[clarissalimab]

Makes a lot of sense to me, I like it. Excellent design!

[alan-ms]

Hi @wwahammy, @MaiconMares and I will be working on this issue.

[wwahammy]

Hi there @alan-ms, This work is actually sort of completed in https://github.com/CommitChange/houdini/tree/webhooks but it's a pretty complex task. Please look into how its implemented there and reuse it as much as possible.

[alan-ms]

Hi @wwahammy. We've looked into the solution implemented, but we are a bit confused about what is not finished and we need to implement.