CDS View Generator for SAP Tables Replication to BigQuery via Datasph…

…ere (#1355)

* Adding the files for CDS Generator

* upload the guide doc

---------

Co-authored-by: Andrew Gold <[email protected]>

examples/sap-bigquery-cds-generator/ABAP_Utility_Input_Inventory_Tables_to_CDSviews.txt

@@ -0,0 +1,36 @@
+TABLENAME	CDSVIEWNAME
+t002	ZFI_T002_CDS
+matdoc	ZFI_MATDOC_CDS
+t134	ZFI_T134_CDS
+ckmlhd	ZFI_CKMLHD_CDS
+t009	ZFI_T009_CDS
+t001w	ZFI_T001W_CDS
+t023	ZFI_T023_CDS
+mch1	ZFI_MCH1_CDS
+marc	ZFI_MARC_CDS
+tcurf	ZFI_TCURF_CDS
+t001k	ZFI_T001K_CDS
+t134t	ZFI_T134T_CDS
+makt	ZFI_MAKT_CDS
+t023t	ZFI_T023T_CDS
+t001l	ZFI_T001L_CDS
+tcurr	ZFI_TCURR_CDS
+t009b	ZFI_T009B_CDS
+t001	ZFI_T001_CDS
+ckmlcr	ZFI_CKMLCR_CDS
+tcurx	ZFI_TCURX_CDS
+mara	ZFI_MARA_CDS
+dd03l	ZFI_DD03L_CDS
+mska	ZFI_MSKA_CDS
+msku	ZFI_MSKU_CDS
+mkol	ZFI_MKOL_CDS
+mslb	ZFI_MSLB_CDS
+mard	ZFI_MARD_CDS
+t148	ZFI_T148_CDS
+t148t	ZFI_T148T_CDS
+t156t	ZFI_T156T_CDS
+t157e	ZFI_T157E_CDS
+t179	ZFI_T179_CDS
+t179t	ZFI_T179T_CDS
+tcurc	ZFI_TCURC_CDS
+tcurt	ZFI_TCURT_CDS

examples/sap-bigquery-cds-generator/K900045.S4S

@@ -0,0 +1,7 @@
+LABUSER      K D01        1   0   4   0   0   0   0   0   0   4  755   .  0   0   0   0   0 100
+#U
+#/1/                          A   G   -   C   R   7   T   -   Z RELE EX.  _   _   _   _   _ CLI
+S4S f 0000 20240322163114 tocsaps4app       s4sadm
+S4S e 0000 20240322163117 tocsaps4app       s4sadm
+D01 < 0000 20240322163120 tocsaps4app       s4sadm
+S4S E 0000 20240322163120 tocsaps4app       s4sadm

examples/sap-bigquery-cds-generator/README.md

@@ -0,0 +1,61 @@
+# ABAP Utility for Bulk Creation of CDS Views
+
+## Summary
+
+This ABAP utility program automates the creation of CDS views for SAP tables that can be used for replicating SAP tables to BigQuery and for Cortex Data Foundation. It eliminates the manual effort and potential errors associated with individual view creation, ensuring a seamless and efficient process.
+
+## Problem Statement
+
+Replicate SAP tables to BigQuery via Datasphere using CDS views provided that enterprise SLT license is not available.
+Manually creating CDS views for each SAP source table intended for replication to BigQuery or Cortex integration is time-consuming and prone to errors. This necessitates a more efficient and automated solution.
+
+## Key Features
+
+* **Automated CDS View Creation:** Generates CDS views for designated SAP tables, eliminating manual development effort.
+* **Bulk Processing:** Efficiently handles large datasets of tables through file upload.
+* **Cortex Compatibility:** Guarantees one-to-one correspondence between CDS views and source tables, adhering to Cortex expectations.
+* **Transport and Package Management:** Integrates with the SAP transport system for controlled deployment and versioning of CDS views.
+* **Comprehensive Error Handling:** Catches and reports potential errors during data processing and view creation.
+
+## Technical Design
+
+### User Interface
+
+* An intuitive selection screen allows users to:
+    * Upload an input file containing a list of SAP tables for CDS view creation.
+    * Provide the target transport request for saving the generated views.
+    * Input the package where the CDS views will be assigned.
+
+### Application Logic
+
+* The program extracts table names from the input file.
+* It reads the corresponding schema/structure of the input tables from the SAP DD03L table.
+* For each table, it dynamically creates the CDS view definition with necessary annotations.
+* It generates the underlying SQL view for the defined CDS view.
+* Utilizes ABAP's built-in capabilities to activate and transport the generated views.
+* Implements robust error handling mechanisms.
+
+### Output Screen
+
+* Provides a summary of processed tables and generated CDS views.
+* Highlights successful creations and reports any failures with detailed reasons.
+* Displays the assigned transport and package information.
+
+## ABAP Utility Import Process
+
+### Mass CDS Generator Utility Program Import and Execution
+
+1. **Import Transport Request:** Import the provided transport request using the attached files:
+    * `K900045.S4S`
+    * `R900045.S4S`
+    * Refer to this guide for importing transport requests: [How to Import an SAP Transport Request](https://kb.theobald-software.com/sap/how-to-import-an-sap-transport-request-with-the-transportmanagement-system-stms)   
+
+2. **Prepare Input File:** Create an input file (CSV or TXT) with two columns: `TABLENAME` and `CDSVIEWNAME`.
+3. **Execute the Program:** Run the imported program, providing the input file from step 2.
+4. **(Optional) Replicate to GCP BQ:** If needed, follow the remaining steps (not provided in this document) to replicate the generated views to Google Cloud BigQuery.
+
+## Observations
+
+* Some existing CDS views in Datasphere might be missing the `MANDT` field, which is crucial for Cortex reporting and Looker dashboards and to identify the client correctly.
+* It is recommended to run this program for all Cortex-required tables to ensure the inclusion of the `MANDT` field where applicable.
+* In some cases, the field may not be `MANDT` but it could be `RCLNT` or `CLIENT`. Make sure to check the CDS views and mark those fields as non key fields.