Sync changes between master and release branch (#194)

* 2024-04-11-Release (#192)

* [ALS-5827] Enable search by CA id

- If a uuid does not match a query, instead search for common area UUID
- If we have a collision, I'm buying a lottery ticket

* [ALS-5787] Open Access StatViz is not showing "Other" (#180)

* Update key length handling in limitKeySize

The limitKeySize method in the VisualizationUtil class has been updated to ensure key uniqueness when keys exceed a certain length. A more robust method of shortening the keys has been implemented: if the key is longer than 45 characters, it is cut off and replaced with "..." and additional characters are appended to ensure uniqueness if needed. This helps limit key size while preserving uniqueness.

* Add VisualizationUtilTests and refine key shortening logic

Added a new test class, VisualizationUtilTests to validate the functionality of the VisualizationUtil class. Adjusted the 'limitKeySize' method to ensure key uniqueness when keys are shortened to a maximum length of 45 characters. If shortened keys are not unique, additional trailing characters are included until uniqueness is achieved.

* Add check on length before substring

* Update pic-sure-util/src/main/java/edu/harvard/dbmi/avillach/util/VisualizationUtil.java

* Update pic-sure-util/src/main/java/edu/harvard/dbmi/avillach/util/VisualizationUtil.java

* Update pic-sure-util/src/main/java/edu/harvard/dbmi/avillach/util/VisualizationUtil.java

* Update pic-sure-resources/pic-sure-visualization-resource/src/test/java/edu/harvard/hms/dbmi/avillach/resource/visualization/service/VisualizationUtilTests.java

Co-authored-by: Luke Sikina <[email protected]>

* Fix import for modified unit test

* Refactor and add test cases for key size limiting function

Refactors existing code by simplifying key size limiting logic in VisualizationUtil and added several new unit tests to ensure its correct behavior with different input scenarios including long keys, empty maps, null maps, and uniqueness near middle.

* Update null handling in VisualizationUtil's limitKeySize

Changed the handling of null input in VisualizationUtil's limitKeySize from returning a new HashMap to throwing an IllegalArgumentException. Also, updated the corresponding test to check for this exception instead of comparing with an empty map.

---------

Co-authored-by: Luke Sikina <[email protected]>

* Refine the condition for empty return in DataProcessingService (#184)

Adjusted the condition under which an empty HashMap is returned in the DataProcessingService. Now, an empty HashMap will only be returned when both the maximum and minimum values in the data are 0, and the number of bins is also 0. This change takes into account situations where the data contains values but the min and max are both 0.

* ALS-5387: Remove stack from resources table (#186)

* [ALS-5422] Add persistence.xml to visualization resource (#161) (#162)
The visualization resource is failing to start due to an error injecting persistence unit into CDI managed bean. It is unable to find a persistence unit named ''.

* ALS-5387: Remove stack from resources table
* ALS-5387: More robust implementation of feature

---------

Co-authored-by: Gcolon021 <[email protected]>
Co-authored-by: GeorgeC <[email protected]>

* [ALS-6222] Status endpoints now filters resources (#189)

The database contains both open and auth hpds even if it wasn't deployed. This means we attempt to check for a resource that doesn't exist. This results in our service always showing degraded.

---------

Co-authored-by: Luke Sikina <[email protected]>
Co-authored-by: Gcolon021 <[email protected]>
Co-authored-by: Luke Sikina <[email protected]>
Co-authored-by: ramari16 <[email protected]>
Co-authored-by: GeorgeC <[email protected]>

* ALS-6375: Suppress showing user 0 results in open-pic-sure (#193)

---------

Co-authored-by: Luke Sikina <[email protected]>
Co-authored-by: Gcolon021 <[email protected]>
Co-authored-by: Luke Sikina <[email protected]>
Co-authored-by: ramari16 <[email protected]>
Co-authored-by: GeorgeC <[email protected]>

pic-sure-resources/pic-sure-aggregate-data-sharing-resource/src/main/java/edu/harvard/hms/dbmi/avillach/AggregateDataSharingResourceRS.java

@@ -522,6 +522,10 @@ protected String processContinuousCrossCounts(String continuousCrossCountRespons
         int generatedVariance = this.generateVarianceWithCrossCounts(crossCounts);
         boolean mustObfuscate = isCrossCountObfuscated(crossCounts, generatedVariance);
 
+        if (canShowContinuousCrossCounts(crossCounts)) {
+            return null;
+        }
+
         // Handle the case where there is no visualization service UUID
         if (properties.getVisualizationResourceId() != null) {
             Map<String, Map<String, Integer>> continuousCrossCounts = objectMapper.readValue(continuousCrossCountResponse, new TypeReference<>() {});
@@ -709,6 +713,17 @@ private boolean isCrossCountObfuscated(Map<String, String> crossCounts, int gene
         return mustObfuscate;
     }
 
+
+    private boolean canShowContinuousCrossCounts(Map<String, String> crossCounts) {
+        String lessThanThresholdStr = "< " + this.threshold;
+
+        String v = crossCounts.get("\\_studies_consents\\");
+        if (v.contains(lessThanThresholdStr) || v.equals("0")) {
+            return true;
+        }
+        return false;
+    }
+
     /**
      * This method will generate a random variance for the request based on the passed entityString. The variance
      * will be between -variance and +variance. The variance will be generated by adding a random salt to the
@@ -765,7 +780,7 @@ private Stream<String> generateParents(String key) {
     private Optional<String> aggregateCount(String actualCount) {
         try {
             int queryResult = Integer.parseInt(actualCount);
-            if (queryResult > 0 && queryResult < threshold) {
+            if (queryResult < threshold) {
                 return Optional.of("< " + threshold);
             }
         } catch (NumberFormatException nfe) {

pic-sure-resources/pic-sure-aggregate-data-sharing-resource/src/test/java/edu/harvard/hms/dbmi/avillach/AggregateDataSharingResourceRSAcceptanceTests.java

@@ -90,7 +90,7 @@ public void testNoObfuscationOnLargeCounts() throws IOException {
 
 	@Test
 	public void testZeroNotObfuscated() throws IOException {
-		expect_original_result_to_become_obfuscated_result("all_zero_open_access_cross_count_result", "all_zero_open_access_cross_count_result");
+		expect_original_result_to_become_obfuscated_result("all_zero_open_access_cross_count_result", "obfuscated_all_less_ten_open_access_cross_count_result");
 	}
 
 	@Test