Skip to content

Commit

Permalink
Updating Location of Experimentation Docs + small Debugger Edit (#708)
Browse files Browse the repository at this point in the history
* Updating Location of Experimentation Docs + small Debugger Edit

- moved Feature Experimentation docs page under Essentials section
- added a clarification on what properties can be updated via debugger

* Updated links

updated links to map to new page location for experimentation

* Missed one link

* (actual) last link update
  • Loading branch information
katemacfarlane authored Jun 28, 2024
1 parent 6a01a3d commit 6b2660e
Show file tree
Hide file tree
Showing 6 changed files with 15 additions and 15 deletions.
4 changes: 2 additions & 2 deletions docs/best-practices/engineering-led-experiments.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ To complete this guide, you may want to review the following topics:

- [Creating and Managing Variations](/essentials/variables)
- [Random Distribution Targeting](/essentials/targeting)
- [Feature Experimentation](/extras/metrics/feature-experimentation)
- [Feature Experimentation](/essentials/feature-experimentation)
- [Creating a Metric](/extras/metrics/creating-and-managing-metrics) and [Metric Types](/extras/metrics/creating-and-managing-metrics#types)

## Metrics in DevCycle
Expand Down Expand Up @@ -116,7 +116,7 @@ Some of the simplest and most precise experiments are basic A/B tests comparing

For more complex features you may compare more variations with several combinations of variable values. You would then randomly distribute the variations according to the number of variations you are including in the experiment.

For more information about comparing multiple variations in an experiment, refer to [Feature Experimentation](/extras/metrics/feature-experimentation#comparing-multiple-variations).
For more information about comparing multiple variations in an experiment, refer to [Feature Experimentation](/essentials/feature-experimentation#comparing-multiple-variations).

### Choosing Metrics

Expand Down
2 changes: 1 addition & 1 deletion docs/best-practices/feature-grouping.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ This guide describes how to manage large amounts of feature flags with DevCycle.

## Why Group Flags?

Feature Flags can accumulate very quickly. Without an organizational mechanism, the feature management dashboard would be extremely cluttered. Imagine developing a complex feature with various parts, such as the [Metrics and Experimentation feature in DevCycle](/extras/metrics/feature-experimentation). Each piece of functionality needs a flag: There’s a flag for the API, the UI, each type of experimentation, string flags for naming, and so on. It would be helpful to have the ability to find all these flags in the same place.
Feature Flags can accumulate very quickly. Without an organizational mechanism, the feature management dashboard would be extremely cluttered. Imagine developing a complex feature with various parts, such as the [Metrics and Experimentation feature in DevCycle](/essentials/feature-experimentation). Each piece of functionality needs a flag: There’s a flag for the API, the UI, each type of experimentation, string flags for naming, and so on. It would be helpful to have the ability to find all these flags in the same place.

Not only do we want flags for each part of a feature, but we also want the ability to turn off the entire feature, all at once, in case something goes wrong. Other Feature Flag management solutions require you to create complex feature flag dependencies to set up a “master switch” for a group of flags. In contrast, DevCycle allows you to easily group feature flags and disable them simultaneously with one switch.

Expand Down
2 changes: 1 addition & 1 deletion docs/best-practices/product-led-experimentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ To complete this guide, you may want to review the following topics:
- [DevCycle Feature Hierarchy](/introduction/core-concepts/feature-hierarchy)
- [Creating and Managing Variations](/essentials/variables)
- [Random Distribution Targeting](/essentials/targeting)
- [Feature Experimentation](/extras/metrics/feature-experimentation)
- [Feature Experimentation](/essentials/feature-experimentation)
- [Creating a Metric](/extras/metrics/creating-and-managing-metrics) and [Metric Types](/extras/metrics/creating-and-managing-metrics#types)

## Conducting Experiments: A Collaborative Approach
Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,8 @@
---
title: Feature Experimentation
sidebar_position: 3
sidebar_position: 10
---

:::info
Experimentation relies on custom events.

Experimentation is available to all customers on any plan. However, to perform experiments, events must be sent to DevCycle to calculate metrics. These events are added to your existing plan. To learn more, read about our pricing, or contact us.
:::

## Overview

At DevCycle we believe that experimentation should be a part of the natural lifecycle of all features. So no matter the [feature type](/essentials/features) selected, can be experimented on. Experiments can be as simple as comparing any target audiences against a metric, or can be fully [randomized](/essentials/targeting#serving-a-random-variation-experimentation--random-distribution) A/B tests using statistical methodologies.
Expand Down Expand Up @@ -49,6 +43,12 @@ To set this up, create a targeting rule in Production that delivers to All Users

## Adding Metrics to Your Feature

:::info
Experimentation relies on custom events.

Experimentation is available to all customers on any plan. However, to perform experiments, events must be sent to DevCycle to calculate metrics. These events are added to your existing plan. To learn more, read about our pricing, or contact us.
:::

Now that you have two segments receiving different experiences, the only other thing you need to run an experiment is a metric to evaluate the comparative performance of those experiences.

To add a metric to your feature, click “Comparative Analysis” under the “Data & Results” section on the sidebar of the feature editing page. Click the “Choose a Metric” dropdown. This will bring up the option to add a metric that has already been created in the project or to create a new one.
Expand Down
6 changes: 3 additions & 3 deletions docs/extras/metrics/creating-and-managing-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ To run a test, the following fields must be set:

**Feature** - This is the specific feature this Metric should be applied to. Any event that has been sent since the creation of this Metric from a user receiving any variation of this feature will be part of this Metric. In the event that an error is shown, this means the event has not been seen from this feature yet.

**Control** - After selecting a Feature, a "control" variation must be selected. This is what will be used to show a comparative analysis against all other variations in a feature. Typically, an "off" or "Baseline" variation would act as the control. For more information on this, please refer to the [Feature Experimentation documentation](/extras/metrics/feature-experimentation).
**Control** - After selecting a Feature, a "control" variation must be selected. This is what will be used to show a comparative analysis against all other variations in a feature. Typically, an "off" or "Baseline" variation would act as the control. For more information on this, please refer to the [Feature Experimentation documentation](/essentials/feature-experimentation).

**Date Range** - This range will default from the moment the Metric (or feature) was created, to now within 30 days. If 30 days have passed since the creation of the Metric or feature, the date range must be a 30 day range.

Expand Down Expand Up @@ -132,7 +132,7 @@ Here are some steps you can follow to track metrics within a feature:

While the setup has some default values, the Metric requires the following fields to be filled:

- **Control** - This is what will be used to show a comparative analysis against all other variations in a feature. Typically, an "off" or "Baseline" variation would act as the control. For more information on this, please refer to the [Feature Experimentation documentation](/extras/metrics/feature-experimentation).
- **Control** - This is what will be used to show a comparative analysis against all other variations in a feature. Typically, an "off" or "Baseline" variation would act as the control. For more information on this, please refer to the [Feature Experimentation documentation](/essentials/feature-experimentation).

- **Date Range** - This range will default from the moment the Metric (or feature) was created, to now within 30 days. If 30 days have passed since the creation of the Metric or feature, the date range must be a 30 day range.

Expand All @@ -158,4 +158,4 @@ To read more on the queries behind the Metrics, see [How Metrics Are Calculated]

With Metrics on a feature, experimentation can be easily executed on any Feature. At DevCycle we believe that experimentation should be a part of the natural lifecycle of all features. So no matter the [feature type](/essentials/features) selected, experimentation will always be available for use.

To learn more on how to run experiments with DevCycle, read [Feature Experimentation](/extras/metrics/feature-experimentation)
To learn more on how to run experiments with DevCycle, read [Feature Experimentation](/essentials/feature-experimentation)
2 changes: 1 addition & 1 deletion docs/extras/web-debugger.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ Once set, you can now view and modify your Overrides for any Feature in your pro
In the "Live Configuration" section, you'll find a list of the Features and Variations that you are currently being served by DevCycle on this page.
Each row can be clicked to expand and see the set of Variables and their values which are associated with that Feature and Variation.

At the top of this section you can also see the current user data that you've been identified with on this page, and modify any property.
At the top of this section you can also see the current user data that you've been identified with on this page, and modify User ID or Email and/or any custom property.
Applying changes to that data will trigger the SDK's `identify` function and trigger a new configuration to be retrieved from DevCycle.

### Events
Expand Down

0 comments on commit 6b2660e

Please sign in to comment.