Flag Definition Language & Flag Evaluation Wire Protocol

[toddbaert]

As agreed on in our Aug 31 community meeting¹, I'd like to use this thread to discuss the possibility of specifying a flag definition language and a flag evaluation wire protocol in OpenFeature.

The Present Situation

Currently, the OpenFeature specification describes a vendor-neutral API for flag evaluation. It does not describe a flag definition language, or a flag evaluation wire protocol. Consequently, the only means of integration with the OpenFeature SDK is to implement a provider in the relevant programming language(s), which serves as an adaptor between the OpenFeature evaluation API and the flag management system powering the provider.

Limitations

• Providers must be implemented for multiple languages and vendors.
• Turnkey, self-hosted applications can't easily leverage OpenFeature (they would need to support some kind of plugin system to allow a provider to be specified and configured)².
• Infrastructure as code solutions and PaaS providers have no standard on which to base vendor-neutral feature flag definitions.

Proposal

I propose that in addition to the existing API, OpenFeature specifies a flag definition language and a flag evaluation wire protocol. Practically, this is a more straightforward proposal than describing an API, and I don't think anyone is better positioned to do it. Defining a flag definition language could amount to publishing a JSON schema which defines a structure for a set of flags (we already do something similar in the flagd project). Defining a flag evaluation wire protocol³ could involve publishing a protobuf schema or OpenAPI specification (another thing we already do in the flagd project). These would allow for the implementation of a unified, generic provider compatible with any conformant flag management system.

Note

Though flagd already does the above, I'm not necessarily proposing we use it's specifications as the OpenFeature standard.
It's only mentioned for reference.

Advantages

• A single OpenFeature provider can be used for any backend that conforms to the standards.
  • Providers that do evaluation in-process could use the flag definition language to do so in a consistent way; the evaluation engine would only need to written once.
  • Providers that do remote-evaluation could use the flag evaluation wire protocol, simplifying implementation and allowing reuse.
• A turnkey, self-hosted application's in-built feature flags can be managed by a conformant system.
• PaaS providers and infrastructure as code solutions could leverage a consistent flag definition language to define flags in a vendor neutral way.
  • A vendor-neutral feature flag type could be added to the Kubernetes API, meaning vendor-built integrations (operators) could be used to synchronize feature flags between vendors and customer K8s clusters.
  • Flag definitions could be portable between vendors and PaaS solutions.

Challenges

• Defining flag variants, states, etc, is relatively simple but defining dynamic targeting is more difficult.
  • Rules are hard to implement consistently.
    • A standard expression language such as CEL or JSONLogic may be required.
      • It may not be possible to map the rule expressions in the standard to existing implementations in vendors.
  • Additional dynamic evaluation features (such as fractional evaluation / pseudorandom assignment) might be difficult to implement consistently.
• Standardization of SDK usage telemetry may be required.
  • Impression metrics, events, etc, may require standardization as well so vendors can maintain visibility.
• Identifying a minimum viable feature set could be difficult.
  • is supporting "rules" and percentage-based splits enough for a bare-minimum?
  • do we need to support "scheduled" releases or progressive rollouts in the definition?
  • do we need to support "big audiences" (external data sets) somehow?

Summary

I think this could constitute the next major phase in the OpenFeature project. Let's use this discussion as a place to brainstorm before taking any more concrete steps, such as OFEPs/issues. Please don't hesitate to comment and share!

I'd like to get a working-group together comprising some vendors and OSS projects who would be willing to work through a draft and then potentially do a PoC if there's a consensus this is a worthwhile effort.

Footnotes

1. Aug 31 meeting notes ↩

2. Relevant Slack discussion ↩

3. Flag Evaluation Wire Protocol OFEP ↩

[justinabrahms]

I'm in support of this. I think it would ease adoption for mulit-language use-cases. Is there a type of feedback you're looking for in particular?

[toddbaert]

I'd love to get feedback from vendors, maintainers of associated OSS projects, and feature flag SMEs. If there's an agreement that this is a worthwhile effort, we can move on to more practical next steps (I've added this to the description).

[liran2000]

Adding some.
Raised previous Slack discussion.

Some Motivation - Advantages

• Being able to export configuration
  • For snapshots
  • For backup
  • For export and import to a different service provider
  • Reduce vendor lock
  • Be familiar with a standard configuration, without having to learn and adopt per service provider

Cons

• As some service providers already implemented, it can be high effort for them to adopt it.
  Adoption by multiple service providers can contribute to usage.
• Hard to fit a scheme to needs of lots of service providers.

[toddbaert]

@justinabrahms I think, specially, I'd like to get a working-group together comprising some vendors and OSS projects who would be willing to work through a draft and then, potentially do a PoC.

[thomaspoignant]

I'll be happy to be part of this group if needed.

[thomaspoignant]

Go Feature Flag is using this format https://github.com/thomaspoignant/go-feature-flag/blob/3a2866c89e5c59b7a0852239c8694654e5705a2f/internal/flag/internal_flag.go#L20

As mention in slack, it has information about the flag, BUT also information on how to rollout it, and I am not sure we are willing to add these extra information in the definition of a standard flag.

[toddbaert]

As mention in slack, it has information about the flag, BUT also information on how to rollout it, and I am not sure we are willing to add these extra information in the definition of a standard flag.

I think this is exactly the sort of thing we'd need to decide on if this is an task we're interested in undertaking. We'd need to decide if features like defining the "rollout schedule" in the schema is something we'd want to support or not.

I've added a specific point about this.

[fahad19]

Featurevisor is all about declarative definitions, which you can find here:

• Project: https://featurevisor.com/
• Features: https://featurevisor.com/docs/features/
• Segments: https://featurevisor.com/docs/segments/

[toddbaert]

Ya I've seen it. I'm a big fan, personally!

Basically what I'm wondering is if there's an interest in seeing something like this become a standard, so that vendors could define flags in such a way and have interoperability (see my reasoning above).

[fahad19]

As much as I would like to see that happen, and that means I don't have to worry about creating new SDKs any more and rely fully on OpenFeature SDKs, I feel all providers (tools) will always have something different than the rest.

some will like it, some won't. and covering that entire base will be a big challenge. feature flag is only one implementation when you look at feature management as a broad umbrella, and I am already struggling to get people to see the "experimentation" side of things in this OFEP here: open-feature/ofep#85

[alexvanboxel]

A flag evaluation wire protocol opens the door for a centralized component to translate between the OpenFeature wire protocol and the proprietary protocol. I always thought this was the purpose of FlagD, but it seems not. We were investigating implementing the FlagD wire protocol to modernize our internal feature flag system, without writing our own SDK for each language we use internally.

I always found that OpenFeature needed a central component like the Collector in OpenTelemetry. This collector probably contributes to the success of the project. Once the flag evaluation wire protocol is defined, someone can start building such a component and use it as a bridge to proprietary systems.

[toddbaert]

A flag evaluation wire protocol opens the door for a centralized component to translate between the OpenFeature wire protocol and the proprietary protocol. I always thought this was the purpose of FlagD, but it seems not.

This was one of the purposes of flagd, but we haven't had much interest for vendors in implementing a custom sync, though it's possible and easy, relatively speaking. Thus far, it's only been the "standard" syncs (HTTP, file, K8s-resource) that have been implemented/used.

[nicklasl]

I believe that flag evaluation wire protocol is an easier thing to "standardize" compared to a flag definition language so I wonder if it would make sense to work on these initiatives independent of each other?

I know that we in Confidence are interested to work with defining and to leverage a flag evaluation wire protocol but we believe that there are big challenges with unifying different vendors under the same flag definition protocols, most of them already defined in this original discussion post.
There is a high risk that we would need to settle for some common denominator of operators, which will make the language only partially compatible with different vendors.

[agardnerIT]

Though flagd already does the above and A flag evaluation wire protocol opens the door for a centralized component to translate between the OpenFeature wire protocol and the proprietary protocol. and This was one of the purposes of flagd

These comments make me wonder:

1. If flagd does this, why reinvent the wheel?
2. Even if the wheel is reinvented, what would make vendors and OSS projects (who by your own admission have not been implementing flagd syncs) any more likely to implement this new thing?
3. If there is genuine interest in this area from vendors, the working group should build on what's already available and (if I read this thread correctly, flagd effectively becomes the equivalent of the OTEL collector).

"we haven't had much interest for vendors in implementing a custom sync": It seems that this is the core problem to solve here.

[konradjniemiec]

I wonder if having the responsibility of translation fall on flagd would make it more of a required aspect of OpenFeature that people maintain in their infrastructure, rather than what I understand it is now which is optional.

[staceypotter]

We have setup a meeting to discuss this topic on October 19, 2023 from 10am - 11am ET (4pm - 5pm CET): https://community.cncf.io/events/details/cncf-openfeature-presents-openfeature-special-meeting/

We hope to see you there! 🙂

[oxddr]

We've raised it already during the project meeting, but we (Google) would like to support both efforts: flag definition language and flag evaluation wire protocol.

We think they can contribute to ease of adoption (as pointed above, supporting multiple languages and frameworks can be much easier) and increase reliability and safety of using feature flags (e.g. with flags definition we could autogenerated client libraries built on top of SDKs).

[beeme1mr]

Autogenerated clients from a common flag definition could be an extremely useful feature, in my opinion. It has the potential to greatly improve the overall developer experience.

[kinyoklion]

I think that a wire protocol is a great idea, but I do have concerns about a flag definition language.

Specifically a flag definition language which supports rules or rollouts, versus just the current states of flags for a context. The primary concern is that evaluating a flag for a specific vendor using either that vendors SDK, or an OpenFeature SDK would currently produce a deterministic result. For a context that is equivalent you would get an equivalent result, it would bucket the same (if the vendor has deterministic bucketing), rules would apply the same, etc.

With a flag definition language, without 100% parity with the rules and rollout mechanism used by the vendor, this wouldn't necessarily be the case. I think it would likely result in a vendor having to make specific "OpenFeature" compatible flags.

[beeme1mr]

We'll definitely need to ensure that deterministic bucketing is supported in the flag definition language. However, I'm not sure how important it is to guarantee that it behaves exactly like a vendor implementation. It seems like this would only be an issue if someone had a multi-tier application with different providers at each tier that relied on consistent bucketing of the same flag. I think we could document that users may experience unexpected behavior if they're using a multi-provider architecture.

Another idea could be to add support for various hashing algorithms and expected bucket sizes in the flag definition. If it turns out that we need to support the same hashing mechanism as the underlying flag management tool, this may be an option.

[beeme1mr]

I don't think the goal should be to have 100% parity with the rules and rollout mechanism used by the vendor. It should, however, support the most common use cases in a vendor-agnostic way. A flag management tool could either maintain OpenFeature providers for each language or expose an API that a universal provider could interact with. This API could essentially be a transpiled output of a vendor's understanding of a flag definition.

[toddbaert]

I don't think the goal should be to have 100% parity with the rules and rollout mechanism used by the vendor. It should, however, support the most common use cases in a vendor-agnostic way

This sums up my thoughts.

[kinyoklion]

I know, for at least LD, there is UI which lets you know what variations a specific context will receive. This UI would need to know that the hashing was different for a flag evaluated using the flag definition language. It is not just multi-tier applications that would be affected, but likely baseline functionality.

Predictability is a very important attribute of a feature flagging system.

[kinyoklion]

In the context of just being able to evaluate something, via some lowest common denominator, it could make sense. Most systems though are more than just that level, and have expectations between different components. With a provider, or with a wire protocol ultimately driven by a provider, those expectations can be maintained.