adding rate limiting for command engine

doc/release-notes/9356-rate-limiting.md

@@ -0,0 +1,20 @@
+## Rate Limiting using JCache (with Hazelcast as provided by Payara)
+The option to rate limit has been added to prevent users from over taxing the system either deliberately or by runaway automated processes.
+Rate limiting can be configured on a tier level with tier 0 being reserved for guest users and tiers 1-any for authenticated users.
+Superuser accounts are exempt from rate limiting.
+Rate limits can be imposed on command APIs by configuring the tier, the command, and the hourly limit in the database.
+Two database settings configure the rate limiting.
+Note: If either of these settings exist in the database rate limiting will be enabled.
+If neither setting exists rate limiting is disabled.
+
+`:RateLimitingDefaultCapacityTiers` is a comma separated list of default values for each tier.
+In the following example, the default for tier `0` (guest users) is set to 10,000 calls per command per hour and tier `1` (authenticated users) is set to 20,000 calls per command per hour.
+Tiers not specified in this setting will default to `-1` (No Limit). I.e., -d "10000" is equivalent to -d "10000,-1,-1,..."
+`curl http://localhost:8080/api/admin/settings/:RateLimitingDefaultCapacityTiers -X PUT -d '10000,20000'`
+
+`:RateLimitingCapacityByTierAndAction` is a JSON object specifying the rate by tier and a list of actions (commands).
+This allows for more control over the rate limit of individual API command calls.
+In the following example, calls made by a guest user (tier 0) for API `GetLatestPublishedDatasetVersionCommand` is further limited to only 10 calls per hour, while an authenticated user (tier 1) will be able to make 30 calls per hour to the same API.
+`curl http://localhost:8080/api/admin/settings/:RateLimitingCapacityByTierAndAction -X PUT -d '[{"tier": 0, "limitPerHour": 10, "actions": ["GetLatestPublishedDatasetVersionCommand", "GetPrivateUrlCommand", "GetDatasetCommand", "GetLatestAccessibleDatasetVersionCommand"]}, {"tier": 0, "limitPerHour": 1, "actions": ["CreateGuestbookResponseCommand", "UpdateDatasetVersionCommand", "DestroyDatasetCommand", "DeleteDataFileCommand", "FinalizeDatasetPublicationCommand", "PublishDatasetCommand"]}, {"tier": 1, "limitPerHour": 30, "actions": ["CreateGuestbookResponseCommand", "GetLatestPublishedDatasetVersionCommand", "GetPrivateUrlCommand", "GetDatasetCommand", "GetLatestAccessibleDatasetVersionCommand", "UpdateDatasetVersionCommand", "DestroyDatasetCommand", "DeleteDataFileCommand", "FinalizeDatasetPublicationCommand", "PublishDatasetCommand"]}]'`
+
+Hazelcast is configured in Payara and should not need any changes for this feature

doc/sphinx-guides/source/_static/installation/files/examples/rate-limit-actions-setting.json

@@ -0,0 +1,40 @@
+[
+    {
+      "tier": 0,
+      "limitPerHour": 10,
+      "actions": [
+        "GetLatestPublishedDatasetVersionCommand",
+        "GetPrivateUrlCommand",
+        "GetDatasetCommand",
+        "GetLatestAccessibleDatasetVersionCommand"
+      ]
+    },
+    {
+      "tier": 0,
+      "limitPerHour": 1,
+      "actions": [
+        "CreateGuestbookResponseCommand",
+        "UpdateDatasetVersionCommand",
+        "DestroyDatasetCommand",
+        "DeleteDataFileCommand",
+        "FinalizeDatasetPublicationCommand",
+        "PublishDatasetCommand"
+      ]
+    },
+    {
+      "tier": 1,
+      "limitPerHour": 30,
+      "actions": [
+        "CreateGuestbookResponseCommand",
+        "GetLatestPublishedDatasetVersionCommand",
+        "GetPrivateUrlCommand",
+        "GetDatasetCommand",
+        "GetLatestAccessibleDatasetVersionCommand",
+        "UpdateDatasetVersionCommand",
+        "DestroyDatasetCommand",
+        "DeleteDataFileCommand",
+        "FinalizeDatasetPublicationCommand",
+        "PublishDatasetCommand"
+      ]
+    }
+]

doc/sphinx-guides/source/installation/config.rst

@@ -1373,6 +1373,33 @@ Before being moved there,
   on your machine, large file uploads via API will cause RAM and/or swap usage bursts. You might want to point this to
   a different location, restrict maximum size of it, and monitor for stale uploads.
 
+.. _cache-rate-limiting:
+
+Configure Your Dataverse Installation to Use JCache (with Hazelcast as Provided by Payara) for Rate Limiting
+------------------------------------------------------------------------------------------------------------
+
+Rate limiting has been added to prevent users from over taxing the system either deliberately or by runaway automated processes.
+Rate limiting can be configured on a tier level with tier 0 being reserved for guest users and tiers 1-any for authenticated users.
+Superuser accounts are exempt from rate limiting.
+Rate limits can be imposed on command APIs by configuring the tier, the command, and the hourly limit in the database.
+Two database settings configure the rate limiting.
+Note: If either of these settings exist in the database rate limiting will be enabled (note that a Payara restart is required for the setting to take effect). If neither setting exists rate limiting is disabled.
+
+- :RateLimitingDefaultCapacityTiers is the number of calls allowed per hour if the specific command is not configured. The values represent the number of calls per hour per user for tiers 0,1,...
+  A value of -1 can be used to signify no rate limit. Tiers not specified in this setting will default to `-1` (No Limit). I.e., -d "10000" is equivalent to -d "10000,-1,-1,..."
+
+.. code-block:: bash
+
+  curl http://localhost:8080/api/admin/settings/:RateLimitingDefaultCapacityTiers -X PUT -d '10000,20000'
+
+- :RateLimitingCapacityByTierAndAction is a JSON object specifying the rate by tier and a list of actions (commands). This allows for more control over the rate limit of individual API command calls.
+  In the following example, calls made by a guest user (tier 0) for API GetLatestPublishedDatasetVersionCommand is further limited to only 10 calls per hour, while an authenticated user (tier 1) will be able to make 30 calls per hour to the same API.
+
+:download:`rate-limit-actions.json </_static/installation/files/examples/rate-limit-actions-setting.json>`  Example json for RateLimitingCapacityByTierAndAction
+
+.. code-block:: bash
+
+  curl http://localhost:8080/api/admin/settings/:RateLimitingCapacityByTierAndAction -X PUT -d '[{"tier": 0, "limitPerHour": 10, "actions": ["GetLatestPublishedDatasetVersionCommand", "GetPrivateUrlCommand", "GetDatasetCommand", "GetLatestAccessibleDatasetVersionCommand"]}, {"tier": 0, "limitPerHour": 1, "actions": ["CreateGuestbookResponseCommand", "UpdateDatasetVersionCommand", "DestroyDatasetCommand", "DeleteDataFileCommand", "FinalizeDatasetPublicationCommand", "PublishDatasetCommand"]}, {"tier": 1, "limitPerHour": 30, "actions": ["CreateGuestbookResponseCommand", "GetLatestPublishedDatasetVersionCommand", "GetPrivateUrlCommand", "GetDatasetCommand", "GetLatestAccessibleDatasetVersionCommand", "UpdateDatasetVersionCommand", "DestroyDatasetCommand", "DeleteDataFileCommand", "FinalizeDatasetPublicationCommand", "PublishDatasetCommand"]}]'
 
 .. _Branding Your Installation:
 
@@ -4496,3 +4523,17 @@ tab. files saved with these headers on S3 - since they no longer have
 to be generated and added to the streamed file on the fly.
 
 The setting is ``false`` by default, preserving the legacy behavior. 
+
+:RateLimitingDefaultCapacityTiers
++++++++++++++++++++++++++++++++++
+Number of calls allowed per hour if the specific command is not configured. The values represent the number of calls per hour per user for tiers 0,1,...
+A value of -1 can be used to signify no rate limit. Also, by default, a tier not defined would receive a default of no limit.
+
+:RateLimitingCapacityByTierAndAction
+++++++++++++++++++++++++++++++++++++
+JSON object specifying the rate by tier and a list of actions (commands). This allows for more control over the rate limit of individual API command calls.
+In the following example, calls made by a guest user (tier 0) for API GetLatestPublishedDatasetVersionCommand is further limited to only 10 calls per hour, while an authenticated user (tier 1) will be able to make 30 calls per hour to the same API.
+{"rateLimits":[
+{"tier": 0, "limitPerHour": 10, "actions": ["GetLatestPublishedDatasetVersionCommand", "GetPrivateUrlCommand", "GetDatasetCommand", "GetLatestAccessibleDatasetVersionCommand"]},
+{"tier": 0, "limitPerHour": 1, "actions": ["CreateGuestbookResponseCommand", "UpdateDatasetVersionCommand", "DestroyDatasetCommand", "DeleteDataFileCommand", "FinalizeDatasetPublicationCommand", "PublishDatasetCommand"]},
+{"tier": 1, "limitPerHour": 30, "actions": ["CreateGuestbookResponseCommand", "GetLatestPublishedDatasetVersionCommand", "GetPrivateUrlCommand", "GetDatasetCommand", "GetLatestAccessibleDatasetVersionCommand", "UpdateDatasetVersionCommand", "DestroyDatasetCommand", "DeleteDataFileCommand", "FinalizeDatasetPublicationCommand", "PublishDatasetCommand"]}]}

pom.xml

@@ -210,6 +210,18 @@
             <scope>provided</scope>
         </dependency>
 
+        <!-- JSON-B -->
+        <dependency>
+            <groupId>jakarta.json.bind</groupId>
+            <artifactId>jakarta.json.bind-api</artifactId>
+        </dependency>
+        <!-- Rope in an implementation for unit tests - is provided at runtime by appserver -->
+        <dependency>
+            <groupId>org.eclipse</groupId>
+            <artifactId>yasson</artifactId>
+            <scope>test</scope>
+        </dependency>
+
         <!-- Jakarta Faces & related -->
         <dependency>
             <groupId>org.glassfish</groupId>
@@ -542,6 +554,10 @@
             <artifactId>dataverse-spi</artifactId>
             <version>2.0.0</version>
         </dependency>
+        <dependency>
+            <groupId>javax.cache</groupId>
+            <artifactId>cache-api</artifactId>
+        </dependency>
         <!-- TESTING DEPENDENCIES -->
         <dependency>
             <groupId>org.junit.jupiter</groupId>
@@ -653,6 +669,11 @@
             <version>3.9.0</version>
             <scope>test</scope>
         </dependency>
+        <dependency>
+            <groupId>com.hazelcast</groupId>
+            <artifactId>hazelcast</artifactId>
+            <scope>test</scope>
+        </dependency>
     </dependencies>
     <build>
         <testResources>

src/main/java/edu/harvard/iq/dataverse/EjbDataverseEngine.java

@@ -4,6 +4,7 @@
 import edu.harvard.iq.dataverse.actionlogging.ActionLogServiceBean;
 import edu.harvard.iq.dataverse.authorization.AuthenticationServiceBean;
 import edu.harvard.iq.dataverse.authorization.providers.builtin.BuiltinUserServiceBean;
+import edu.harvard.iq.dataverse.util.cache.CacheFactoryBean;
 import edu.harvard.iq.dataverse.engine.DataverseEngine;
 import edu.harvard.iq.dataverse.authorization.Permission;
 import edu.harvard.iq.dataverse.authorization.groups.GroupServiceBean;
@@ -16,6 +17,7 @@
 import edu.harvard.iq.dataverse.engine.command.DataverseRequest;
 import edu.harvard.iq.dataverse.engine.command.exception.CommandException;
 import edu.harvard.iq.dataverse.engine.command.exception.PermissionException;
+import edu.harvard.iq.dataverse.engine.command.exception.RateLimitCommandException;
 import edu.harvard.iq.dataverse.ingest.IngestServiceBean;
 import edu.harvard.iq.dataverse.pidproviders.PidProviderFactoryBean;
 import edu.harvard.iq.dataverse.privateurl.PrivateUrlServiceBean;
@@ -176,7 +178,9 @@ public class EjbDataverseEngine {
 
     @EJB
     EjbDataverseEngineInner innerEngine;
-
+
+    @EJB
+    CacheFactoryBean cacheFactory;
 
     @Resource
     EJBContext ejbCtxt;
@@ -202,7 +206,11 @@ public <R> R submit(Command<R> aCommand) throws CommandException {
 
         try {
             logRec.setUserIdentifier( aCommand.getRequest().getUser().getIdentifier() );
-
+            // Check for rate limit exceeded. Must be done before anything else to prevent unnecessary processing.
+            if (!cacheFactory.checkRate(aCommand.getRequest().getUser(), aCommand)) {
+                throw new RateLimitCommandException(BundleUtil.getStringFromBundle("command.exception.user.ratelimited", Arrays.asList(aCommand.getClass().getSimpleName())), aCommand);
+            }
+
             // Check permissions - or throw an exception
             Map<String, ? extends Set<Permission>> requiredMap = aCommand.getRequiredPermissions();
             if (requiredMap == null) {

src/main/java/edu/harvard/iq/dataverse/UserServiceBean.java

@@ -147,6 +147,8 @@ private AuthenticatedUser createAuthenticatedUserForView (Object[] dbRowValues,
         user.setMutedEmails(Type.tokenizeToSet((String) dbRowValues[15]));
         user.setMutedNotifications(Type.tokenizeToSet((String) dbRowValues[15]));
 
+        user.setRateLimitTier((int)dbRowValues[17]);
+
         user.setRoles(roles);
         return user;
     }
@@ -419,7 +421,7 @@ private List<Object[]> getUserListCore(String searchTerm,
         qstr += " u.createdtime, u.lastlogintime, u.lastapiusetime, ";
         qstr += " prov.id, prov.factoryalias, ";
         qstr += " u.deactivated, u.deactivatedtime, ";
-        qstr += " u.mutedEmails, u.mutedNotifications ";
+        qstr += " u.mutedEmails, u.mutedNotifications, u.rateLimitTier ";
         qstr += " FROM authenticateduser u,";
         qstr += " authenticateduserlookup prov_lookup,";
         qstr += " authenticationproviderrow prov";

src/main/java/edu/harvard/iq/dataverse/api/AbstractApiBean.java

@@ -20,6 +20,7 @@
 import edu.harvard.iq.dataverse.engine.command.impl.GetLatestAccessibleDatasetVersionCommand;
 import edu.harvard.iq.dataverse.engine.command.impl.GetLatestPublishedDatasetVersionCommand;
 import edu.harvard.iq.dataverse.engine.command.impl.GetSpecificPublishedDatasetVersionCommand;
+import edu.harvard.iq.dataverse.engine.command.exception.RateLimitCommandException;
 import edu.harvard.iq.dataverse.externaltools.ExternalToolServiceBean;
 import edu.harvard.iq.dataverse.license.LicenseServiceBean;
 import edu.harvard.iq.dataverse.locality.StorageSiteServiceBean;
@@ -421,7 +422,7 @@ public Command<DatasetVersion> handleLatestPublished() {
         }));
         return dsv;
     }
-    
+
     protected DataFile findDataFileOrDie(String id) throws WrappedResponse {
         DataFile datafile;
         if (id.equals(PERSISTENT_ID_KEY)) {
@@ -575,6 +576,8 @@ protected <T> T execCommand( Command<T> cmd ) throws WrappedResponse {
         try {
             return engineSvc.submit(cmd);
 
+        } catch (RateLimitCommandException ex) {
+            throw new WrappedResponse(rateLimited(ex.getMessage()));
         } catch (IllegalCommandException ex) {
             //for 8859 for api calls that try to update datasets with TOA out of compliance
                 if (ex.getMessage().toLowerCase().contains("terms of use")){
@@ -776,11 +779,15 @@ protected Response notFound( String msg ) {
     protected Response badRequest( String msg ) {
         return error( Status.BAD_REQUEST, msg );
     }
-    
+
     protected Response forbidden( String msg ) {
         return error( Status.FORBIDDEN, msg );
     }
-
+
+    protected Response rateLimited( String msg ) {
+        return error( Status.TOO_MANY_REQUESTS, msg );
+    }
+
     protected Response conflict( String msg ) {
         return error( Status.CONFLICT, msg );
     }

src/main/java/edu/harvard/iq/dataverse/authorization/users/AuthenticatedUser.java

@@ -16,6 +16,7 @@
 import edu.harvard.iq.dataverse.util.BundleUtil;
 import edu.harvard.iq.dataverse.util.json.JsonPrinter;
 import static edu.harvard.iq.dataverse.util.StringUtil.nonEmpty;
+
 import edu.harvard.iq.dataverse.util.json.NullSafeJsonBuilder;
 import java.io.Serializable;
 import java.sql.Timestamp;
@@ -42,6 +43,7 @@
 import jakarta.persistence.PostLoad;
 import jakarta.persistence.PrePersist;
 import jakarta.persistence.Transient;
+import jakarta.validation.constraints.Min;
 import jakarta.validation.constraints.NotBlank;
 import jakarta.validation.constraints.NotNull;
 
@@ -146,6 +148,10 @@ public class AuthenticatedUser implements User, Serializable {
     @Transient
     private Set<Type> mutedNotificationsSet = new HashSet<>();
 
+    @Column(nullable=false)
+    @Min(value = 1, message = "Rate Limit Tier must be greater than 0.")
+    private int rateLimitTier = 1;
+
     @PrePersist
     void prePersist() {
         mutedNotifications = Type.toStringValue(mutedNotificationsSet);
@@ -397,6 +403,13 @@ public void setDeactivatedTime(Timestamp deactivatedTime) {
         this.deactivatedTime = deactivatedTime;
     }
 
+    public int getRateLimitTier() {
+        return rateLimitTier;
+    }
+    public void setRateLimitTier(int rateLimitTier) {
+        this.rateLimitTier = rateLimitTier;
+    }
+
     @OneToOne(mappedBy = "authenticatedUser")
     private AuthenticatedUserLookup authenticatedUserLookup;
 
@@ -435,7 +448,6 @@ public void setShibIdentityProvider(String shibIdentityProvider) {
 
     public JsonObjectBuilder toJson() {
         //JsonObjectBuilder authenicatedUserJson = Json.createObjectBuilder();
-
         NullSafeJsonBuilder authenicatedUserJson = NullSafeJsonBuilder.jsonObjectBuilder();
 
         authenicatedUserJson.add("id", this.id);

src/main/java/edu/harvard/iq/dataverse/engine/command/exception/RateLimitCommandException.java

@@ -0,0 +1,16 @@
+package edu.harvard.iq.dataverse.engine.command.exception;
+
+import edu.harvard.iq.dataverse.engine.command.Command;
+
+/**
+ * An exception raised when a command cannot be executed, due to the
+ * issuing user being rate limited.
+ *
+ * @author
+ */
+public class RateLimitCommandException extends CommandException {
+
+    public RateLimitCommandException(String message, Command aCommand) {
+        super(message, aCommand);
+    }
+}

src/main/java/edu/harvard/iq/dataverse/settings/SettingsServiceBean.java

@@ -239,6 +239,10 @@ public enum Key {
 
         CVocConf,
 
+        // Default calls per hour for each tier. csv format (30,60,...)
+        RateLimitingDefaultCapacityTiers,
+        // json defined list of capacities by tier and action list. See RateLimitSetting.java
+        RateLimitingCapacityByTierAndAction,
         /**
          * A link to an installation of https://github.com/IQSS/miniverse or
          * some other metrics app.

src/main/java/edu/harvard/iq/dataverse/util/SystemConfig.java

@@ -1146,11 +1146,21 @@ public Long getTestStorageQuotaLimit() {
         return settingsService.getValueForKeyAsLong(SettingsServiceBean.Key.StorageQuotaSizeInBytes);
     }
     /**
-     * Should we store tab-delimited files produced during ingest *with* the 
-     * variable name header line included? 
+     * Should we store tab-delimited files produced during ingest *with* the
+     * variable name header line included?
      * @return boolean - defaults to false.
      */
     public boolean isStoringIngestedFilesWithHeaders() {
         return settingsService.isTrueForKey(SettingsServiceBean.Key.StoreIngestedTabularFilesWithVarHeaders, false);
     }
+
+    /**
+     * RateLimitUtil will parse the json to create a List<RateLimitSetting>
+     */
+    public String getRateLimitsJson() {
+        return settingsService.getValueForKey(SettingsServiceBean.Key.RateLimitingCapacityByTierAndAction, "");
+    }
+    public String getRateLimitingDefaultCapacityTiers() {
+        return settingsService.getValueForKey(SettingsServiceBean.Key.RateLimitingDefaultCapacityTiers, "");
+    }
 }