Prebid Connect Measurement Framework

[jdwieland8282]

Type of issue

Feature Request

Description

Measuring outcomes is the first step needed to justify additional investment in the IDHubConnect concepts discussed in London and NYC. Adding addressability features to Prebid without the necessary means to measure the outcome would create ambiguity that could be exploited by opportunistic actors at the expense of publishers. Empowering publisher's or their agents with the ability to measure cpm lift when a specific piece of data (EID, SDA, etc) enters their bid stream is a first order problem worth solving. This issue seeks to define the needed technical requirements.

In order to measure vendor outcomes there are two broad problems to solve. For the sake of simplicity we'll assume that the new piece of data we want to test is a SharediD.

1. We must link the outbound bid request to the winning bid response. The outbound bid request will contain the EIDs array which is where the SharediD will transit oRTB. Ex.

"user":{
   "ext":{
      "eids":[
         {
            "source":"pubcid.org",
            "uids":[
               {
                  "id":"d88*****19e476",
                  "atype":1
               }
            ]
         },

the winning bid response should contain the winning CPM. EID source value and winning CPM are required.

2. The second problem we need to solve relates to sampling and control. The operator of this feature needs a way to modify how often the independent variable, in this example the SharediD gets added to the bid stream. Finally the operator needs a flag attached to the outbound bid request noting that this auction is part of either the control or sample group.

Requirements
using https://docs.prebid.org/dev-docs/modules/genericAnalyticsAdapter.html as a starting point. We expect analytics adapters will modify these requirements to fit within their existing Analytics Adapter framework.

1. Add independent variable to analytics event. For illustrative purpose I'm adding it to the auctionInit event though there might be better places for it.
2. Add suppression function the userId module. This suppression flag (0-1) will control how often the user id module suppresses a specific EID.

• when suppression occurs send a notification to the auctionInit event.

Expected results

Analytics Adapter

pbjs.enableAnalytics({
    provider: 'generic', // operator name
    options: {
        url: 'https://example.com' //operators endpoint
    }
})

auctionInit
this example is not part of any test

{
   "eventType":"auctionInit",
   "args":{
      "auctionId":"97000db4-ae78-4e93-81d1-66b83ac10a74",
      "timestamp":1666207538126,
      "auctionStatus":"inProgress",
      "adUnits":[
         
      ],
      "noBids":[
         
      ],
      "bidsReceived":[
         
      ],
      "bidsRejected":[
         
      ],
      "winningBids":[
         5.00
      ],
      "timeout":2000,
      "metrics":{
         
      },
      "eids":[
         "pubcid.org",
         "liveramp.com",
         "33across.com"
      ],
      "test_label":"NULL // valid values are sample or control"
   }
}

this example is part of the control group

{
   "eventType":"auctionInit",
   "args":{
      "auctionId":"97000db4-ae78-4e93-81d1-66b83ac10a74",
      "timestamp":1666207538126,
      "auctionStatus":"inProgress",
      "adUnits":[
         
      ],
      "noBids":[
         
      ],
      "bidsReceived":[
         
      ],
      "bidsRejected":[
         
      ],
      "winningBids":[
         5.00
      ],
      "timeout":2000,
      "metrics":{
         
      },
      "eids":[
 
      ],
      "test_label":"control"
   }
}

this example is part of the test group

{
   "eventType":"auctionInit",
   "args":{
      "auctionId":"97000db4-ae78-4e93-81d1-66b83ac10a74",
      "timestamp":1666207538126,
      "auctionStatus":"inProgress",
      "adUnits":[
         
      ],
      "noBids":[
         
      ],
      "bidsReceived":[
         
      ],
      "bidsRejected":[
         
      ],
      "winningBids":[
         5.00
      ],
      "timeout":2000,
      "metrics":{
         
      },
      "eids":[
         "pubcid.org"
      ],
      "test_label":"control"
   }
}

UserId module config

{
   "experiement":{
      "eids":{
         "name":"sharedid",
         "sampleRate":{
            "control":"0.01", //should suppress all EIDs
            "sample":"0.01" //should all EIDs except `SharediD`
         }
      }
   },
   "name":"sharedId",
   "params":{
      "syncTime":60
   },
   "storage":{
      "type":"cookie",
      "name":"sharedid",
      "expires":28
   }
}

Actual results

Platform details

Other information

[patmmccann]

appears to be a dupe of #10705?

I think this is the wrong place to solve this problem, i think testing of various modules in the build or configuration should be handled by software external to prebid, likely developed by the publisher or simply in the implementation examples, however, the picking up the labels solution i think we alredy agreed to is reportedly underway by @3link ?

[patmmccann]

this is from #10705, so this does appear to be an exact dupe

`
how about

pbjs.mergeConfig {

analyticsLabels: {
test1 : a,
test2 : d }
}
Then we could have genericAnalytics pick up analyticsLabels by adding analyticsLabels to each event and making sure it went over the wire. Other analytics modules could be Pr'd to do the same by their maintainer
`

[3link]

@patmmccann Unfortunately, it looks like we have other priorities now.

[patmmccann]

@3link thank you, i changed 10705 from in progress to ready for dev and removed your assignment