From 2cb05a2591b2c20447f0b21c8b358cf971e60914 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Tue, 14 May 2024 11:33:24 -0400 Subject: [PATCH 1/4] add 3DViewer by openforestdata.pl to list of external tools #10561 --- doc/release-notes/10561-3dviewer.md | 1 + .../source/_static/admin/dataverse-external-tools.tsv | 1 + 2 files changed, 2 insertions(+) create mode 100644 doc/release-notes/10561-3dviewer.md diff --git a/doc/release-notes/10561-3dviewer.md b/doc/release-notes/10561-3dviewer.md new file mode 100644 index 00000000000..47da10f8837 --- /dev/null +++ b/doc/release-notes/10561-3dviewer.md @@ -0,0 +1 @@ +3DViewer by openforestdata.pl has been added to the list of external tools: https://preview.guides.gdcc.io/en/develop/admin/external-tools.html#inventory-of-external-tools diff --git a/doc/sphinx-guides/source/_static/admin/dataverse-external-tools.tsv b/doc/sphinx-guides/source/_static/admin/dataverse-external-tools.tsv index fe256828d44..3df5ed5d24f 100644 --- a/doc/sphinx-guides/source/_static/admin/dataverse-external-tools.tsv +++ b/doc/sphinx-guides/source/_static/admin/dataverse-external-tools.tsv @@ -7,3 +7,4 @@ Data Curation Tool configure file "A GUI for curating data by adding labels, gro Ask the Data query file Ask the Data is an experimental tool that allows you ask natural language questions about the data contained in Dataverse tables (tabular data). See the README.md file at https://github.com/IQSS/askdataverse/tree/main/askthedata for the instructions on adding Ask the Data to your Dataverse installation. TurboCurator by ICPSR configure dataset TurboCurator generates metadata improvements for title, description, and keywords. It relies on open AI's ChatGPT & ICPSR best practices. See the `TurboCurator Dataverse Administrator `_ page for more details on how it works and adding TurboCurator to your Dataverse installation. JupyterHub explore file The `Dataverse-to-JupyterHub Data Transfer Connector `_ is a tool that simplifies the transfer of data between Dataverse repositories and the cloud-based platform JupyterHub. It is designed for researchers, scientists, and data analysts, facilitating collaboration on projects by seamlessly moving datasets and files. The tool is a lightweight client-side web application built using React and relies on the Dataverse External Tool feature, allowing for easy deployment on modern integration systems. Currently optimized for small to medium-sized files, future plans include extending support for larger files and signed Dataverse endpoints. For more details, you can refer to the external tool manifest: https://forgemia.inra.fr/dipso/eosc-pillar/dataverse-jupyterhub-connector/-/blob/master/externalTools.json +3DViewer by openforestdata.pl explore file The 3DViewer by openforestdata.pl can be used to explore 3D files (e.g. STL format). It was presented by Kamil Guryn during the 2020 community meeting (`slide deck `_, `video `_) and can be found at https://github.com/OpenForestData/open-forest-data-previewers From e7a0e375f4082aa3816370535861e724225c6c97 Mon Sep 17 00:00:00 2001 From: jeromeroucou Date: Mon, 10 Jun 2024 15:41:17 +0200 Subject: [PATCH 2/4] #10466: quick fix for math challenge contact form on 403 error page (#10602) --- doc/release-notes/10466-math-challenge-403-error-page.md | 1 + .../java/edu/harvard/iq/dataverse/SendFeedbackDialog.java | 4 ++++ 2 files changed, 5 insertions(+) create mode 100644 doc/release-notes/10466-math-challenge-403-error-page.md diff --git a/doc/release-notes/10466-math-challenge-403-error-page.md b/doc/release-notes/10466-math-challenge-403-error-page.md new file mode 100644 index 00000000000..160c760dc9d --- /dev/null +++ b/doc/release-notes/10466-math-challenge-403-error-page.md @@ -0,0 +1 @@ +On forbidden access error page, also know as 403 error page, the math challenge is now correctly display to submit the contact form. diff --git a/src/main/java/edu/harvard/iq/dataverse/SendFeedbackDialog.java b/src/main/java/edu/harvard/iq/dataverse/SendFeedbackDialog.java index 5a522eb7e45..46941c8b5b6 100644 --- a/src/main/java/edu/harvard/iq/dataverse/SendFeedbackDialog.java +++ b/src/main/java/edu/harvard/iq/dataverse/SendFeedbackDialog.java @@ -129,6 +129,10 @@ public void setUserSum(Long userSum) { } public String getMessageTo() { + if (op1 == null || op2 == null) { + // Fix for 403 error page: initUserInput method doesn't call before + initUserInput(null); + } if (feedbackTarget == null) { return BrandingUtil.getSupportTeamName(systemAddress); } else if (feedbackTarget.isInstanceofDataverse()) { From 3934c3f4a968e60350075546a280c8e72caf0fd6 Mon Sep 17 00:00:00 2001 From: Jose Lucas Cordeiro Date: Mon, 10 Jun 2024 11:35:18 -0300 Subject: [PATCH 3/4] API: Handling creation of duplicate role for a dataset object (#10474) * 9729: Handling repeated role creation error for datasets * 9729: Fixing unit test * 9729: Optimizing Imports * 9729: Deleting white space * 9729: Deleting redundant line * Update doc/release-notes/9729-release-notes.md Co-authored-by: Philip Durbin * 9375: Rolling back wrong code removal and reformatting * 9375: Refactoring and adding test * 9375: Changing message to use property file * 9887: rolling back formatting * 9729: rolling back formatting * 9729: rolling back formatting --------- Co-authored-by: Philip Durbin --- doc/release-notes/9729-release-notes.md | 1 + .../engine/command/impl/AssignRoleCommand.java | 14 +++++++++++++- src/main/java/propertyFiles/Bundle.properties | 1 + .../edu/harvard/iq/dataverse/api/DatasetsIT.java | 11 +++++++++++ .../command/impl/CreatePrivateUrlCommandTest.java | 10 ++++++++-- 5 files changed, 34 insertions(+), 3 deletions(-) create mode 100644 doc/release-notes/9729-release-notes.md diff --git a/doc/release-notes/9729-release-notes.md b/doc/release-notes/9729-release-notes.md new file mode 100644 index 00000000000..9dc27995405 --- /dev/null +++ b/doc/release-notes/9729-release-notes.md @@ -0,0 +1 @@ +An error is now correctly reported when an attempt is made to assign an identical role to the same collection, dataset, or file. #9729 #10465 \ No newline at end of file diff --git a/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/AssignRoleCommand.java b/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/AssignRoleCommand.java index e4edb973cd9..121af765737 100644 --- a/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/AssignRoleCommand.java +++ b/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/AssignRoleCommand.java @@ -3,7 +3,6 @@ */ package edu.harvard.iq.dataverse.engine.command.impl; -import edu.harvard.iq.dataverse.DataFile; import edu.harvard.iq.dataverse.Dataset; import edu.harvard.iq.dataverse.Dataverse; import edu.harvard.iq.dataverse.authorization.DataverseRole; @@ -18,6 +17,8 @@ import edu.harvard.iq.dataverse.engine.command.DataverseRequest; import edu.harvard.iq.dataverse.engine.command.exception.CommandException; import edu.harvard.iq.dataverse.engine.command.exception.IllegalCommandException; +import edu.harvard.iq.dataverse.util.BundleUtil; + import java.util.Collections; import java.util.HashSet; import java.util.Map; @@ -68,11 +69,22 @@ public RoleAssignment execute(CommandContext ctxt) throws CommandException { throw new IllegalCommandException("User " + user.getUserIdentifier() + " is deactivated and cannot be given a role.", this); } } + if(isExistingRole(ctxt)){ + throw new IllegalCommandException(BundleUtil.getStringFromBundle("datasets.api.grant.role.assignee.has.role.error"), this); + } // TODO make sure the role is defined on the dataverse. RoleAssignment roleAssignment = new RoleAssignment(role, grantee, defPoint, privateUrlToken, anonymizedAccess); return ctxt.roles().save(roleAssignment); } + private boolean isExistingRole(CommandContext ctxt) { + return ctxt.roles() + .directRoleAssignments(grantee, defPoint) + .stream() + .map(RoleAssignment::getRole) + .anyMatch(it -> it.equals(role)); + } + @Override public Map> getRequiredPermissions() { // for data file check permission on owning dataset diff --git a/src/main/java/propertyFiles/Bundle.properties b/src/main/java/propertyFiles/Bundle.properties index 7bc3ec33a80..2996ccb509b 100644 --- a/src/main/java/propertyFiles/Bundle.properties +++ b/src/main/java/propertyFiles/Bundle.properties @@ -2694,6 +2694,7 @@ datasets.api.datasize.ioerror=Fatal IO error while trying to determine the total datasets.api.grant.role.not.found.error=Cannot find role named ''{0}'' in dataverse {1} datasets.api.grant.role.cant.create.assignment.error=Cannot create assignment: {0} datasets.api.grant.role.assignee.not.found.error=Assignee not found +datasets.api.grant.role.assignee.has.role.error=User already has this role for this dataset datasets.api.revoke.role.not.found.error="Role assignment {0} not found" datasets.api.revoke.role.success=Role {0} revoked for assignee {1} in {2} datasets.api.privateurl.error.datasetnotfound=Could not find dataset. diff --git a/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java b/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java index 5b603d88c6d..d2d14b824bd 100644 --- a/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java +++ b/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java @@ -1717,6 +1717,9 @@ public void testAddRoles(){ giveRandoPermission.prettyPrint(); assertEquals(200, giveRandoPermission.getStatusCode()); + //Asserting same role creation is covered + validateAssignExistingRole(datasetPersistentId,randomUsername,apiToken, "fileDownloader"); + // Create another random user to become curator: Response createCuratorUser = UtilIT.createRandomUser(); @@ -1853,6 +1856,14 @@ public void testListRoleAssignments() { assertEquals(UNAUTHORIZED.getStatusCode(), notPermittedToListRoleAssignmentOnDataverse.getStatusCode()); } + private static void validateAssignExistingRole(String datasetPersistentId, String randomUsername, String apiToken, String role) { + final Response failedGrantPermission = UtilIT.grantRoleOnDataset(datasetPersistentId, role, "@" + randomUsername, apiToken); + failedGrantPermission.prettyPrint(); + failedGrantPermission.then().assertThat() + .body("message", containsString("User already has this role for this dataset")) + .statusCode(FORBIDDEN.getStatusCode()); + } + @Test public void testFileChecksum() { diff --git a/src/test/java/edu/harvard/iq/dataverse/engine/command/impl/CreatePrivateUrlCommandTest.java b/src/test/java/edu/harvard/iq/dataverse/engine/command/impl/CreatePrivateUrlCommandTest.java index 33f9acd0e1a..b67fc8cb4c3 100644 --- a/src/test/java/edu/harvard/iq/dataverse/engine/command/impl/CreatePrivateUrlCommandTest.java +++ b/src/test/java/edu/harvard/iq/dataverse/engine/command/impl/CreatePrivateUrlCommandTest.java @@ -3,8 +3,10 @@ import edu.harvard.iq.dataverse.Dataset; import edu.harvard.iq.dataverse.DatasetVersion; import edu.harvard.iq.dataverse.DataverseRoleServiceBean; +import edu.harvard.iq.dataverse.DvObject; import edu.harvard.iq.dataverse.RoleAssignment; import edu.harvard.iq.dataverse.authorization.DataverseRole; +import edu.harvard.iq.dataverse.authorization.RoleAssignee; import edu.harvard.iq.dataverse.authorization.users.PrivateUrlUser; import edu.harvard.iq.dataverse.engine.TestCommandContext; import edu.harvard.iq.dataverse.engine.TestDataverseEngine; @@ -18,8 +20,8 @@ import org.junit.jupiter.api.Test; import static org.junit.jupiter.api.Assertions.assertEquals; -import static org.junit.jupiter.api.Assertions.*; -import static org.junit.jupiter.api.Assertions.*; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertNull; import static org.junit.jupiter.api.Assertions.assertTrue; public class CreatePrivateUrlCommandTest { @@ -73,6 +75,10 @@ public RoleAssignment save(RoleAssignment assignment) { // no-op return assignment; } + @Override + public List directRoleAssignments(RoleAssignee roas, DvObject dvo) { + return List.of(); + } }; } From 5bf6b6defb1c22971951233f30b679d762496832 Mon Sep 17 00:00:00 2001 From: qqmyers Date: Mon, 10 Jun 2024 10:36:35 -0400 Subject: [PATCH 4/4] Solr: Try Soft Commit on Indexing (#10547) * try soft commit * keep softcommit short to avoid delays in visibility * add test delay for autosoft, make hardcommit 30s like auto setting * add 1-2 second delays in tests for softAutocomplete at 1s * more sleeps * more delays * remove commented out deletes * more commented out code to remove * add 1 sec on failing tests * add missing perm reindex * change waiting * fix index object and add null check for unit test * remove test-specific null check * reindex linking dv * general solr release note * more fixes * revert change - was correct * another sleepforsearch * test adding explicit reindexing * avoid other uses of cache in test that looks for exact counts * Adding longer max sleep, add count param to sleep method * Revert "add missing perm reindex" This reverts commit 317038ae8083e5d421e91c65176df0a806b04011. --- conf/solr/9.3.0/solrconfig.xml | 4 +- doc/release-notes/10547-solr-updates.md | 1 + .../iq/dataverse/search/IndexServiceBean.java | 34 ++++------------- .../search/SolrIndexServiceBean.java | 13 ++----- .../iq/dataverse/api/DataversesIT.java | 2 +- .../edu/harvard/iq/dataverse/api/LinkIT.java | 6 +++ .../harvard/iq/dataverse/api/SearchIT.java | 37 ++++++------------- .../edu/harvard/iq/dataverse/api/UtilIT.java | 19 ++++++++-- .../impl/CreatePrivateUrlCommandTest.java | 16 ++++++++ .../util/cache/CacheFactoryBeanTest.java | 2 + 10 files changed, 66 insertions(+), 68 deletions(-) create mode 100644 doc/release-notes/10547-solr-updates.md diff --git a/conf/solr/9.3.0/solrconfig.xml b/conf/solr/9.3.0/solrconfig.xml index 36ed4f23390..34386375fe1 100644 --- a/conf/solr/9.3.0/solrconfig.xml +++ b/conf/solr/9.3.0/solrconfig.xml @@ -290,7 +290,7 @@ have some sort of hard autoCommit to limit the log size. --> - ${solr.autoCommit.maxTime:15000} + ${solr.autoCommit.maxTime:30000} false @@ -301,7 +301,7 @@ --> - ${solr.autoSoftCommit.maxTime:-1} + ${solr.autoSoftCommit.maxTime:1000}