DATAAPI-36: option to skip data validation and full record data retur…

…n on inserts and updates

CHANGELOG.md

@@ -1,5 +1,9 @@
 # Changelog
 
+## v3.6.1
+
+* new options to skip data validation and the record response on inserts/updates, 4 new interception points added
+
 ## v3.6.0
 
 * Added a UI to show records in the DATA API change queue per subscribed API client

README.md

@@ -52,6 +52,10 @@ Additional _optional_ annotation options at the _object_ level are:
 * `dataApiFilterFields`: Fields to allow as simple filters for paginated GET requests (defaults to foreign keys, boolean and enum fields)
 * `dataApiAllowIdInsert`: Whether or not to allow the ID field to be set during a POST operation to create a new record
 * `dataApiQueueRelevantFields`: Fields that are relevant to trigger a record update to be queued. If not used, then any data change will be queued. If used then only the specified fields will be examined and in case of atomic-changes for the queue only the relevant field changes will be included in the queue item. Inserts and deletes are always queued, only during updates this annotation is evaluated.
+* `dataApiSkipValidationOnInsert`: Whether or not to skip data validation when inserting new records (defaults to `false`)
+* `dataApiSkipValidationOnUpdate`: Whether or not to skip data validation when updating records (defaults to `false`)
+* `dataApiSkipRecordResponseOnInsert`: Whether or not to skip returning the full record(s) data as the response on inserts (defaults to `false`)
+* `dataApiSkipRecordResponseOnUpdate`: Whether or not to skip returning the full record(s) data as the response on updates (defaults to `false`)
 
 ### Property annotations
 
@@ -191,7 +195,7 @@ Fires before inserting data through the API. Receives the following keys in the
 * `insertDataArgs`: Arguments that will be passed to the `insertData()` call
 * `entity`: Name of the entity being operated on
 * `record`: The data that will be inserted (struct)
-
+* `skipRecordResponse`: Whether the API request should skip returning the record data in the response. Defaults to the API or object definition, but can be overwritten here.
 
 ### `postDataApiInsertData`
 
@@ -201,7 +205,7 @@ Fires after inserting data through the API. Receives the following keys in the `
 * `entity`: Name of the entity being operated on
 * `record`: The data that will be inserted (struct)
 * `newId`: Newly created record ID
-
+* `skipRecordResponse`: Whether the API request should skip returning the record data in the response. Defaults to the API or object definition, but can be overwritten here.
 
 ### `preDataApiUpdateData`
 
@@ -211,6 +215,7 @@ Fires before updating data through the API. Receives the following keys in the `
 * `entity`: Name of the entity being operated on
 * `recordId`: ID of the record to be updated
 * `data`: The data that will be inserted (struct)
+* `skipRecordResponse`: Whether the API request should skip returning the record data in the response. Defaults to the API or object definition, but can be overwritten here.
 
 ### `postDataApiUpdateData`
 
@@ -220,6 +225,7 @@ Fires after updating data through the API. Receives the following keys in the `i
 * `entity`: Name of the entity being operated on
 * `recordId`: ID of the record to be updated
 * `data`: The data that will be inserted (struct)
+* `skipRecordResponse`: Whether the API request should skip returning the record data in the response. Defaults to the API or object definition, but can be overwritten here.
 
 ### `preDataApiDeleteData`
 
@@ -237,6 +243,39 @@ Fires after deleting data through the API. Receives the following keys in the `i
 * `entity`: Name of the entity being operated on
 * `recordId`: ID of the record to be deleted
 
+### `onDataApiUpdateRecordDataValidation`
+
+Fires before starting data validation on updates. Receives the following keys in the `interceptData`:
+
+* `entity`: Name of the entity being operated on
+* `data`: The data to be validated
+* `skipValidation`: Whether the whole validation should be skipped. Defaults to the API or object definition, but can be overwritte here.
+
+### `onDataApiInsertRecordDataValidation`
+
+Fires before starting data validation on inserts. Receives the following keys in the `interceptData`:
+
+* `entity`: Name of the entity being operated on
+* `data`: The data to be validated
+* `skipValidation`: Whether the whole validation should be skipped. Defaults to the API or object definition, but can be overwritte here.
+
+### `preDataApiBatchUpdateRecords`
+
+Fires before batch updating multiple records. Receives the following keys in the `interceptData`:
+
+* `entity`: Name of the entity being operated on
+* `records`: Array of record data to be batch updated
+* `skipRecordResponse`: Whether the API request should skip returning the record data in the response. Defaults to the API or object definition, but can be overwritten here.
+
+### `postDataApiBatchUpdateRecords`
+
+Fires after multiple records have been batch updated. Receives the following keys in the `interceptData`:
+
+* `entity`: Name of the entity being operated on
+* `records`: Array of record data to be batch updated
+* `skipRecordResponse`: Whether the API request should skip returning the record data in the response. Defaults to the API or object definition, but can be overwritten here.
+* `updated`: Array of the full record data that was updated (or empty array in case skipRecordResponse=true was already set in `preDataApiBatchUpdateRecords`)
+
 ## Data Change Queue(s)
 
 By default, they system enables a queue feature: `settings.features.dataApiQueue`. When enabled, API users can be subscribed, through the admin UI, to listen for data changes to all, or a number, of entities in the system.

config/Config.cfc

@@ -27,7 +27,13 @@ component {
 			, description  = "Generic Preside REST API for external systems to interact with Preside data"
 			, configHandler = "dataApiManager"
 			, dataApiQueues = { default={ pageSize=1, name="", atomicChanges=false } }
-			, dataApiDefaults = { allowIdInsert=false }
+			, dataApiDefaults = {
+				  allowIdInsert              = false
+				, skipValidationOnInsert     = false
+				, skipValidationOnUpdate     = false
+				, skipRecordResponseOnInsert = false
+				, skipRecordResponseOnUpdate = false
+			}
 		};
 		settings.rest.apis[ "/data/v1/docs" ] = {
 			  description     = "Documentation for REST APIs (no authentication required)"
@@ -57,6 +63,10 @@ component {
 			, "postDataApiSelectData"
 			, "preValidateUpsertData"
 			, "postValidateUpsertData"
+			, "onDataApiUpdateRecordDataValidation"
+			, "onDataApiInsertRecordDataValidation"
+			, "preDataApiBatchUpdateRecords"
+			, "postDataApiBatchUpdateRecords"
 		];
 		conf.interceptorSettings.customInterceptionPoints.append( conf.settings.dataApiInterceptionPoints, true );
 	}

handlers/rest-apis/data/v1/SingleRecord.cfc

@@ -72,13 +72,17 @@ component {
 
 
 		var updated = dataApiService.updateSingleRecord(
-			  entity   = entity
-			, recordId = recordId
-			, data     = validationData
+			  entity       = entity
+			, recordId     = recordId
+			, data         = validationData
+			, returnRecord = true
 		);
 
-		if ( updated ) {
-			get( argumentCollection=arguments );
+		if ( IsNumeric( updated ) && updated > 0 ) {
+			// skipRecordResponseOnUpdate=true - either in the API config or in the entity definition or dynamically set by an interceptor
+			restResponse.noData();
+		} else if ( IsStruct( updated ) ) {
+			restResponse.setData( updated );
 		} else {
 			restResponse.setError(
 				  errorCode = 404

handlers/rest-apis/data/v1/WholeEntity.cfc

@@ -160,7 +160,11 @@ component {
 			, records = body
 		);
 
-		restResponse.setData( updated );
+		if ( IsArray( updated ) ) {
+			restResponse.setData( updated );
+		} else {
+			restResponse.noData();
+		}
 	}
 
 }

services/DataApiConfigurationService.cfc

@@ -40,6 +40,22 @@ component {
 		} );
 	}
 
+	public boolean function entitySkipValidationOnInsert( required string entity ) {
+		return _entityIsBooleanConfigOptionTrue( arguments.entity, "skipValidationOnInsert" );
+	}
+
+	public boolean function entitySkipValidationOnUpdate( required string entity ) {
+		return _entityIsBooleanConfigOptionTrue( arguments.entity, "skipValidationOnUpdate" );
+	}
+
+	public boolean function entitySkipRecordResponseOnInsert( required string entity ) {
+		return _entityIsBooleanConfigOptionTrue( arguments.entity, "skipRecordResponseOnInsert" );
+	}
+
+	public boolean function entitySkipRecordResponseOnUpdate( required string entity ) {
+		return _entityIsBooleanConfigOptionTrue( arguments.entity, "skipRecordResponseOnUpdate" );
+	}
+
 	public string function getEntityObject( required string entity, string namespace=_getDataApiNamespace() ) {
 		var args     = arguments;
 		var cacheKey = "getEntityObject" & args.namespace & args.entity;
@@ -227,27 +243,35 @@ component {
 			for( var objectName in objects ) {
 				var isEnabled = objectIsApiEnabled( objectName, args.namespace );
 				if ( _isTrue( isEnabled ) ) {
-					var namespace           = _getNamespaceWithSeparator( args.namespace );
-					var entityName          = getObjectEntity( objectName, args.namespace );
-					var supportedVerbs      = poService.getObjectAttribute( objectName, "dataApiVerbs#namespace#", getDefaultConfigForApiNamespace( "verbs", namespace ) );
-					var selectFields        = poService.getObjectAttribute( objectName, "dataApiFields#namespace#", "" );
-					var upsertFields        = poService.getObjectAttribute( objectName, "dataApiUpsertFields#namespace#", "" );
-					var excludeFields       = _getExcludedFields( objectName, namespace );
-					var upsertExcludeFields = _getExcludedFields( objectName, namespace, "upsert" );
-					var allowIdInsert       = poService.getObjectAttribute( objectName, "dataApiAllowIdInsert#namespace#", getDefaultConfigForApiNamespace( "allowIdInsert", namespace ) );
-					var allowQueue          = poService.getObjectAttribute( objectName, "dataApiQueueEnabled#namespace#", true );
-					var queueName           = poService.getObjectAttribute( objectName, "dataApiQueue#namespace#", "default" );
-					var category            = poService.getObjectAttribute( objectName, "dataApiCategory#namespace#", "" );
+					var namespace                  = _getNamespaceWithSeparator( args.namespace );
+					var entityName                 = getObjectEntity( objectName, args.namespace );
+					var supportedVerbs             = poService.getObjectAttribute( objectName, "dataApiVerbs#namespace#", getDefaultConfigForApiNamespace( "verbs", namespace ) );
+					var selectFields               = poService.getObjectAttribute( objectName, "dataApiFields#namespace#", "" );
+					var upsertFields               = poService.getObjectAttribute( objectName, "dataApiUpsertFields#namespace#", "" );
+					var excludeFields              = _getExcludedFields( objectName, namespace );
+					var upsertExcludeFields        = _getExcludedFields( objectName, namespace, "upsert" );
+					var allowIdInsert              = poService.getObjectAttribute( objectName, "dataApiAllowIdInsert#namespace#", getDefaultConfigForApiNamespace( "allowIdInsert", namespace ) );
+					var skipValidationOnInsert     = poService.getObjectAttribute( objectName, "dataApiSkipValidationOnInsert#namespace#", getDefaultConfigForApiNamespace( "skipValidationOnInsert", namespace ) );
+					var skipValidationOnUpdate     = poService.getObjectAttribute( objectName, "dataApiSkipValidationOnUpdate#namespace#", getDefaultConfigForApiNamespace( "skipValidationOnUpdate", namespace ) );
+					var skipRecordResponseOnInsert = poService.getObjectAttribute( objectName, "dataApiSkipRecordResponseOnInsert#namespace#", getDefaultConfigForApiNamespace( "skipRecordResponseOnInsert", namespace ) );
+					var skipRecordResponseOnUpdate = poService.getObjectAttribute( objectName, "dataApiSkipRecordResponseOnUpdate#namespace#", getDefaultConfigForApiNamespace( "skipRecordResponseOnUpdate", namespace ) );
+					var allowQueue                 = poService.getObjectAttribute( objectName, "dataApiQueueEnabled#namespace#", true );
+					var queueName                  = poService.getObjectAttribute( objectName, "dataApiQueue#namespace#", "default" );
+					var category                   = poService.getObjectAttribute( objectName, "dataApiCategory#namespace#", "" );
 
 					entities[ entityName ] = {
-						  objectName    = objectName
-						, category      = category
-						, verbs         = ListToArray( LCase( supportedVerbs ) )
-						, selectFields  = ListToArray( LCase( selectFields ) )
-						, upsertFields  = ListToArray( LCase( upsertFields ) )
-						, allowIdInsert = _isTrue( allowIdInsert )
-						, allowQueue    = _isTrue( allowQueue    )
-						, queueName     = queueName
+						  objectName                 = objectName
+						, category                   = category
+						, verbs                      = ListToArray( LCase( supportedVerbs ) )
+						, selectFields               = ListToArray( LCase( selectFields ) )
+						, upsertFields               = ListToArray( LCase( upsertFields ) )
+						, allowIdInsert              = _isTrue( allowIdInsert )
+						, skipValidationOnInsert     = _isTrue( skipValidationOnInsert )
+						, skipValidationOnUpdate     = _isTrue( skipValidationOnUpdate )
+						, skipRecordResponseOnInsert = _isTrue( skipRecordResponseOnInsert )
+						, skipRecordResponseOnUpdate = _isTrue( skipRecordResponseOnUpdate )
+						, allowQueue                 = _isTrue( allowQueue    )
+						, queueName                  = queueName
 					};
 
 					if ( !entities[ entityName ].selectFields.len() ) {
@@ -796,6 +820,18 @@ component {
 		return excluded.toList();
 	}
 
+	private boolean function _entityIsBooleanConfigOptionTrue( required string entity, required string option ) {
+		var args     = arguments;
+		var cacheKey = "entityIsBooleanConfigOptionTrue" & _getDataApiNamespace() & args.entity & args.option;
+
+		return _simpleLocalCache( cacheKey, function(){
+			var entities           = getEntities();
+			var booleanOptionValue = entities[ args.entity ].booleanOptionValue ?: "";
+
+			return _isTrue( booleanOptionValue );
+		} );
+	}
+
 // GETTERS AND SETTERS
 	private any function _getPresideFieldRuleGenerator() {
 		return _presideFieldRuleGenerator;

services/DataApiService.cfc

@@ -111,7 +111,7 @@ component {
 		return created;
 	}
 
-	public struct function createRecord( required string entity, required any record ) {
+	public any function createRecord( required string entity, required any record ) {
 		var objectName = _getConfigService().getEntityObject( arguments.entity );
 		var dao        = $getPresideObject( objectName );
 		var namespace  = _getInterceptorNamespace();
@@ -121,11 +121,16 @@ component {
 			, bypassTrivialInterceptors = true
 		};
 
-		$announceInterception( "preDataApiInsertData#namespace#", { insertDataArgs=args, entity=arguments.entity, record=arguments.record } );
+		var interceptDataArgs = { insertDataArgs=args, entity=arguments.entity, record=arguments.record };
+
+		interceptDataArgs.skipRecordResponse = _getConfigService().entitySkipRecordResponseOnInsert( arguments.entity ); // default config on api or object level
+
+		$announceInterception( "preDataApiInsertData#namespace#", interceptDataArgs );
 		var newId = dao.insertData( argumentCollection=args );
-		$announceInterception( "postDataApiInsertData#namespace#", { insertDataArgs=args, entity=arguments.entity, record=arguments.record, newId=newId } );
+		interceptDataArgs.newId = newId;
+		$announceInterception( "postDataApiInsertData#namespace#", interceptDataArgs );
 
-		return getSingleRecord( arguments.entity, newId, [] );
+		return interceptDataArgs.skipRecordResponse ? newId : getSingleRecord( arguments.entity, newId, [] );
 	}
 
 	public any function batchUpdateRecords( required string entity, required array records ) {
@@ -135,19 +140,31 @@ component {
 		var updated    = [];
 		var recordId   = "";
 
+		var interceptDataArgs = { entity=arguments.entity, records=arguments.records };
+
+		interceptDataArgs.skipRecordResponse = _getConfigService().entitySkipRecordResponseOnUpdate( arguments.entity ); // default config on api or object level
+
+		$announceInterception( "preDataApiBatchUpdateRecords#namespace#", interceptDataArgs );
+
 		for( var record in records ) {
 			recordId = record[ idField ] ?: "";
 			if ( Len( Trim( recordId ) ) ) {
 				if ( updateSingleRecord( arguments.entity, record, recordId ) ) {
-					updated.append( getSingleRecord( entity, recordId, [] ) );
+					if ( !interceptDataArgs.skipRecordResponse ) {
+						ArrayAppend( updated, getSingleRecord( entity, recordId, [] ) );
+					}
 				}
 			}
 		}
 
-		return updated;
+		interceptDataArgs.updated = updated;
+
+		$announceInterception( "postDataApiBatchUpdateRecords#namespace#", interceptDataArgs );
+
+		return interceptDataArgs.skipRecordResponse ? "" : interceptDataArgs.updated;
 	}
 
-	public any function updateSingleRecord( required string entity, required struct data, required string recordId ) {
+	public any function updateSingleRecord( required string entity, required struct data, required string recordId, boolean returnRecord=false ) {
 		var objectName = _getConfigService().getEntityObject( arguments.entity );
 		var dao        = $getPresideObject( objectName );
 		var namespace  = _getInterceptorNamespace();
@@ -157,12 +174,19 @@ component {
 			, updateManyToManyRecords = true
 		};
 
-		$announceInterception( "preDataApiUpdateData#namespace#", { updateDataArgs=args, entity=arguments.entity, recordId=arguments.recordId, data=arguments.data } );
+		var interceptDataArgs = { updateDataArgs=args, entity=arguments.entity, recordId=arguments.recordId, data=arguments.data };
+
+		interceptDataArgs.skipRecordResponse = _getConfigService().entitySkipRecordResponseOnUpdate( arguments.entity ); // default config on api or object level
+
+		$announceInterception( "preDataApiUpdateData#namespace#", interceptDataArgs );
 		var recordsUpdated = dao.updateData( argumentCollection=args );
-		$announceInterception( "postDataApiUpdateData#namespace#", { updateDataArgs=args, entity=arguments.entity, recordId=arguments.recordId, data=arguments.data } );
+		$announceInterception( "postDataApiUpdateData#namespace#", interceptDataArgs );
 
+		if ( !returnRecord || interceptDataArgs.skipRecordResponse ) {
+			return recordsUpdated;
+		}
 
-		return recordsUpdated;
+		return getSingleRecord( arguments.entity, arguments.recordId, [] );
 	}
 
 	public numeric function deleteSingleRecord( required string entity, required string recordId ) {
@@ -178,11 +202,28 @@ component {
 	}
 
 	public any function validateUpsertData( required string entity, required any data, boolean ignoreMissing=false, boolean isUpdate=false ) {
-		var ruleset   = _getConfigService().getValidationRulesetForEntity( arguments.entity );
 		var namespace = _getInterceptorNamespace();
+		var args      = arguments;
+
+		if ( args.isUpdate ) {
+			args.skipValidation = _getConfigService().entitySkipValidationOnUpdate( arguments.entity ); // default config on api or object level
+			$announceInterception( "onDataApiUpdateRecordDataValidation#namespace#", args );
+		}
+		else {
+			args.skipValidation = _getConfigService().entitySkipValidationOnInsert( arguments.entity ); // default config on api or object level
+			$announceInterception( "onDataApiInsertRecordDataValidation#namespace#", args );
+		}
+		// to skip the validation completely it could either use the default or could have been overwritten by interceptor (e.g. using a dynamic skip of validation based on a request parameter)
+
+		if ( args.skipValidation ) {
+			return IsArray( arguments.data ) ? { validated=true, validationResults=[] } : [];
+		}
+
+		var ruleset = _getConfigService().getValidationRulesetForEntity( arguments.entity );
 
 		if ( IsArray( arguments.data ) ) {
 			var result = { validated=true, validationResults=[] };
+
 			for( var record in arguments.data ) {
 
 				var prepped = _prepRecordForInsertAndUpdate( arguments.entity, record, arguments.isUpdate );