ReferencePreprocessor
Reference Preprocessor is a microservice responsible for fetching all the content related to API reference pages on the Kentico Kontent documentation portal, and passing it forward using Azure Blob Storage. It is an Azure function written in Typescript.
The service is accessible by its 3 Azure function endpoints.
HTTP trigger that accepts GET request. It has 2 query parameters: codename and isTest. The optional isTest parameter is included in the query with value enabled when the function is supposed to run with test configuration, which is used for the integration tests. Required parameter codename is used to specify the codename of some API Reference's root item.
This endpoint serves for "on demand" processing of a particular API Specification.
Event grid trigger that serves as an endpoint for cases when Blob Provider requests an API reference page - for example, preview of API reference on web.
Event grid trigger similar to the Tutorial Search's update endpoint. This trigger allows the service to react to webhooks from Kentico Kontent project, which are being distributed via Dispatcher.
This endpoint covers the scenario of some API reference being updated - as a reaction to some webhook from Kentico Kontent, this service not only preprocesses the relevant API reference data, but also triggers the BlobProvider's ReferenceUpdateStarter function.
- isTest - optional, see Configuration for integration tests
The content of API reference pages on the documentation portal is modelled in Kentico Kontent. The models are created to mirror (to a certain extent) the way that the OpenAPI specification version 3 describes REST APIs.
The shared/models folder contains all the content types from Kentico Kontent that are related to the API Reference model. Items of API specification content type act as roots in which all the data related to a single API reference page is linked.
The relevant data is fetched from Kentico Kontent using either the Secured Delivery API or the Preview API. In this case, each API specification item and all of its children/linked items get fetched.
Being subscribed to an event grid topic, the service reacts to any changes of published content on Kentico Kontent project. The forwarded webhook from Kentico Kontent contains the following information:
- operation type (subject)
- array of affected content items and their language, codename, and type
Unfortunately, when there is a change to some published content item that is being nested in another items, then the webhook specifies only direct "parents" of the changed content item. Therefore, the service has to compute the codenames of all the root API specification_ items that are altered by itself. It fetches all content items from Kentico Kontent project and iteratively traverses them until it finds all the changed root items.
Starting with some API specification root item, the service's getProcessedData method is responsible for processing and resolving all the API Reference content items. The algorithm traverses the whole model and processes all the content items in a unified fashion (see the example blob below):
- operation - specifies the triggerred endpoint
- zapiSpecificationCodename - root item codename
-
items - contains information about all the items that are linked in the root item or are its children, along with the root item itself:
- key - codename of the item
- all properties of the item, that are processed and simplified as well
- content type and id of the item
- Multiple Choice element - array of names of selected options
- Linked items element - array of codenames of linked items
- Taxonomy element - array of names of selected terms
Finally, the service saves the processed data to an Azure Blob Storage - Each API specification and its linked/child items are stored in a separate blob, which is named by the codename of the root item (with -preview suffix in case of preview endpoint).
When the service processes Code Sample items, it sorts them in order to stay consistent with the Website. The order of items is specified in a special content item within Kentico Content - of type platform_picker. The service thus needs to fetch the item from KK and use it while processing Code Samples.
Some schema items within the API Reference model can be inserted in themselves, which creates a data model that contains infinite recursions. This service has to deal with that: It saves a global map named codenamesMap, which contains a ProcessedSchemaCodenames class for each processed root item's codename (in order to not have multiple instances of the service influence each other).
Within the ProcessedSchemaCodenames class, the service remembers which schemas are being processed, or have already been processed. Therefore, if it encounters a schema that has already been processed, it can omit processing it again.
{
"operation": "PREVIEW",
"zapiSpecificationCodename": "recommendation_api",
"zapiSpecificationId": "<GUID>",
"items": {
"recommended_items": {
"contentType": "zapi__category",
"id": "<GUID>",
"apiReference": [
"Recommendation API"
],
"description": "...",
"name": "Recommended items",
"pathOperations": [
"view_recommended_items__get_",
"view_recommended_items__post_"
],
"url": "recommended-items"
},
"tracking": {
"contentType": "zapi__category",
"id": "<GUID>",
"apiReference": [
"Recommendation API"
],
"description": "...",
"name": "Tracking",
"pathOperations": [
"track_a_conversion",
"track_a_visit",
"track_view_of_portion_of_content"
],
"url": "tracking"
},
"recommendation_api": {
"contentType": "zapi_specification",
"id": "<GUID>",
"apiReference": [
"Recommendation API"
],
"categories": [
"recommended_items",
"tracking"
],
"contact": [],
"description": "<p>The Recommendation API is a secure REST API that provides access to recommendations made by an engine that employs machine learning to improve its accuracy. ...",
"license": [],
"security": [
"bearer_authentication"
],
"servers": "",
"termsOfService": "",
"title": "Recommendation API",
"url": "recommendation-api",
"version": "1"
}
}
}Receiving an event from Dispatcher with the "test": "enabled" attribute makes the service run with an alternative set of environment variables that is used for integration tests.