Operation-specific inline compilation

[nathanmarz]

transform could run more efficiently by baking the transform-fn into the path. At the callsite, composition could work like this instead:

(reduce
  (fn [curr tn]
    (fn [structure]
      (tn structure curr)))
  (reverse navs+final))

This is more efficient than standard compilation. Standard compilation needs to allocate anonymous function objects at runtime, which is ultimately caused by the fact it is unknown how the path will be used or whether it will be further composed.

By adding some sort of concept of "final path" to the inline compiler, this should be achievable.

It should also be investigated whether something similar is possible for select operations.

[nathanmarz]

This is definitely possible for select-any, since it doesn't have any mutable state to deal with like the other select operations.

This is a big win for static paths. With dynamic paths this style of composition requires composition to happen from the end to the front, so you couldn't precompile [:a :b] together in something like [:a :b (keypath c)]. But this extra object allocation needed during runtime compilation should be offset by the object allocation being removed from runtime.

Seems like this functionality is exactly what's needed to get the (select-any [:a :b :c] data) benchmark on par with (-> data :a :b :c).

[nathanmarz]

Ran some benchmarks by doing this form of compilation manually and the improvement for (select-any [:a :b :c] data) would be at most 30%.

[nathanmarz]

Since sequences of keypath / nthpath with select-any is the only known use case which is a good ways from optimal performance, another possibility is for the inline compiler to treat this as a special case and generate optimal code. This is a departure from how Specter has worked so far since it's treating two particular navigators specially, and only for selection. However, it may be the only way to get the performance to optimal.

[nathanmarz]

Another possibility would be for navigators to be able to provide inline code to use for selection or transformation if the full path is known statically. This would be a navigator agnostic solution that could speed up use cases besides keypath and nthpath. Navigators like META, NAMESPACE, and NAME could benefit from this as well.

[nathanmarz]

Another optimization possibility is removing no-ops for specific operations. For instance, terminal operations are no-ops on select path and could be removed, triggering other removals as well. If one branch of a multi-path were emptied, it could be replaced with just the other branch. These two optimizations would completely remove compact from select calls using that navigator.

[pepe]

As with many other issues, I am not even sure what is the goal with this, but I wanted to thank you for all your work. Specter dramatically helps me with my current project (mainly binary trees stuff).

Bravo!

Happy new year sir.