RealTimeData and video caching

[matthieularere-msq]

Type of issue

Question

Description

I am trying to add informations in video creatives using a realtimedata module and I am fighting with the video caching that's setup on the publisher side. I wonder when a bid response is send to the cache url ? Is it done after rtdmodule onBidResponseEvent has cleared or before ? Based on the schema at https://docs.prebid.org/prebid-video/video-overview.html#instream-video I would guess that it's done after but I am unsure. Could you please confirm me the process flows ?

Thanks

[dgirardi]

Caching is done before bidResponse. The last event fired before caching is bidAdjustment, but that's currently not exposed by the RTD module API (and using it for editing creatives seems like stretching its purpose).

Are you looking to add impression trackers? this might be related: #9085

[matthieularere-msq]

Indeed, I try to add impressions tracker. It works fine for outstream but not for instream because of caching.

If that's not doable using rtdModule, I guess it could be doable by updating src/videoCache.js with a new cache config parameter which would be a list of urls to add to vast impression tracker. I mean something like

example of the new cache config params :

pbjs.setConfig({
  cache: {
    url: "https://prebid.adnxs.com/pbc/v1/cache"
    vastImp: [{
	  // url to build the impression tracking url with
      url: "https://vasttracking.mydomain.com?vast?", 
	  // list of bid attribute to add to the impression tracking
      list: [ 'adUnitCode', 'auctionId', 'bidder', 'bidId', 'cpm', 'creativeId']
    }],
  }
});

This would be used at the beggining of src/videoCache.js function toStorageRequest to build the complete vast impressions urls and add them, something like :

function toStorageRequest(bid, {index = auctionManager.index} = {}) {
  const vastImpTrackers = buildVastImps(bid);
  const vastValue = bid.vastXml ? insertVastImpTrackers(bid.vastXml, vastImpTrackers) : wrapURI(bid.vastUrl, bid.vastImpUrl, vastImpTrackers);
  const auction = index.getAuction(bid);
  ...
}

function insertVastImpTrackers(vastXml, vastImpTrackers) {
  if (vastImpTrackers.length == 0)
    return vastXml;
  const doc = new DOMParser().parseFromString(vastXml, 'text/xml');
  const wrappers = doc.querySelectorAll('VAST Ad Wrapper, VAST Ad InLine');
  if (wrappers.length) {
    wrappers.forEach(wrapper => {
	  vastImpTrackers.forEach(trackingUrl => {
        const impression = doc.createElement('Impression');
        impression.appendChild(doc.createCDATASection(trackingUrl));
        wrapper.appendChild(impression)
	 });
    });
    return new XMLSerializer().serializeToString(doc);
  }
  return vastXml;
}

+ update wrapURI function to deal with the additional parameter

If I submit such a PR, is that any chances to be accepted ?

[matthieularere-msq]

Hi @dgirardi sorry to insist, but do you think this kind of PR would have a change to be accepted ?

[dgirardi]

@matthieularere-msq yes, but I'm not sure it would do what you're looking for - which I understand to be "I need my RTD module to modify VAST bids before they're cached".

The setConfig approach you're proposing would work well if the publisher (not the module) was the party interested in adding trackers; it's better to avoid setConfig from modules because it's not always clear how that should interact with pub-provided config (by default it would overwrite it, which seems undesirable, but any other strategy would also be confusing).

But the general idea of letting interested parties add impression trackers does make sense; I suggest replacing the setConfig with something like

import {registerVASTTrackers} from 'src/video.js';

registerVASTTrackers(function(bidResponse) {
   return {
        imp: [            
            `https://vasttracking.mydomain.com/vast?auctionId=${bidResponse.auctionId}&...` ,
            // ...
        ]
   }
})

I am not that familiar with VAST - the above is assuming that imp trackers are not the only type of tracker we'll want to eventually support.

Also, in addition to (or instead of) toStorageRequest, we probably want to add the trackers when there is no cache - again I don't understand VAST too well so I am not sure if it can/should be done in a way that is cache independent (wrap every bid as we get it, then wrap it again if we cache - or is it worth trying to wrap only once?) cfr:

Prebid.js/src/auction.js

Lines 586 to 598 in eb18f80

	if (config.getConfig('cache.url') && (useCacheKey || context !== OUTSTREAM)) {
	if (!bidResponse.videoCacheKey || config.getConfig('cache.ignoreBidderCacheKey')) {
	addBid = false;
	callPrebidCache(auctionInstance, bidResponse, afterBidAdded, videoMediaType);
	} else if (!bidResponse.vastUrl) {
	logError('videoCacheKey specified but not required vastUrl for video bid');
	addBid = false;
	}
	}
	if (addBid) {
	addBidToAuction(auctionInstance, bidResponse);
	afterBidAdded();
	}

[dgirardi]

actually, I'm not sure my last point makes sense - if there's no cache, can we wrap a vast URL? we could try packaging the wrapper in a data url but I don't know if that has limitations.

[matthieularere-msq]

@dgirardi
Thanks, so if I understand properly, what you suggest is to add a registerVASTTrackers method to src/video.js which would accumulate various vast trackers from analytics of rtd modules. Or should it be only for rtd modules as there is a creative modification involved ?
Then, I could add the vast modification process in src/video.js . My initial thought was that it could be called from src/adapters/bidderFactory.js before or after the call to addBidWithCode in

Prebid.js/src/adapters/bidderFactory.js

Lines 310 to 320 in 22b9c70

	}
	// creating a copy of original values as cpm and currency are modified later
	bid.originalCpm = bid.cpm;
	bid.originalCurrency = bid.currency;
	bid.meta = bid.meta || Object.assign({}, bid[bidRequest.bidder]);
	const prebidBid = Object.assign(createBid(CONSTANTS.STATUS.GOOD, bidRequest), bid);
	addBidWithCode(bidRequest.adUnitCode, prebidBid);
	} else {
	logWarn(`Bidder ${spec.code} made bid for unknown request ID: ${bid.requestId}. Ignoring.`);
	addBidResponse.reject(null, bid, CONSTANTS.REJECTION_REASON.INVALID_REQUEST_ID);
	}

but if it's too early in the process, this could be done in the tryAddVideoBid func from src/auction.js too I guess.

I agree that this should be doable regardless of cache : cache is the reason why it's not working with my previous approach inside rtdModule, but I am looking for it to work with or without cache.
And indeed while I am focus on impression tracking right now, I should make it so that we can eventually track more events.

In case there is no vastXml but only vastUrl, I could add the same logic than in my rtdModule - concatenate it to vastImpUrl :

Prebid.js/modules/oxxionRtdProvider.js

Lines 80 to 84 in 22b9c70

	if (bidResponse.vastUrl) {
	bidResponse.vastImpUrl = bidResponse.vastImpUrl
	? trackingUrl + '&url=' + encodeURI(bidResponse.vastImpUrl)
	: trackingUrl;
	logInfo(LOG_PREFIX + 'insert into vastImpUrl for adId ' + bidResponse.adId);

However this could be a potential issue if too many of them a concatenated one after the others, so I guess if I add such a thing I would need to make sure url length remains lower than 2048 ?

If this makes sense for you I can work on it.

[patmmccann]

finished in #10191