From 378e3937baaece123e25686a9f1fdabb48117d4b Mon Sep 17 00:00:00 2001 From: milanmajchrak <90026355+milanmajchrak@users.noreply.github.com> Date: Wed, 28 Feb 2024 13:26:40 +0100 Subject: [PATCH] customer/uk-it-16 (#544) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * ufal/downloading-restricted-bitstreams-not-working-properly (#457) * The admin could download restricted bitstreams. * The user cannot download bitstream if he/she is not signed in. * Cannot obtain context with user. * The user is already fetched from the context. * Added docs. * Do not use endpoint for downloading the single file because it already exists - remove it. * Fixed AuthorizationRestControllerIT tests. * ufal/be-cannot-download-and-preview-files-after-migration (#454) * Find bitstream format using mimetype not ID. * Updated tests according to fix. * ufal/update-canonical-prefix-to-hdl (#460) * Updated `handle.canonical.prefix` to hdl handle. * Integration test cannot have changed handle.canonical.prefix. * ufal/be-user-registration-missing (#463) * The user registration is added when the eperson is created by the ui. * try using newer checkout, same as upstream, since error is with git --------- Co-authored-by: MajoBerger <88670521+MajoBerger@users.noreply.github.com> * ufal/be-license-download-statistics (#462) * Add logging of downloading restricted bitstreams. * Log downloading of every bitstream not only restricted ones. * Added docs * Refactored logging. * ufal/be-shibboleth-headers-encoding (#464) * givenname and last name values are converted to UTF-8 encoding * Delete eperson in tests * using our dspace-dependencies because of https://github.com/dataquest-dev/DSpace/issues/466 * ufal/be-fix-email-parameters Fixed emails sending - added arguments and cfg properties that are sent in the email. (#470) * ufal/be-provenance-subbmitter-missing (#469) * #8585 Add submitter information to provenance metadata * Cherry picked fix from vanilla and added test to check if the Item provenance metadata is added --------- Co-authored-by: Adán Román Ruiz * ufal/be-get-user-ip-address (#468) * Created endpoint for fetching IP address with tests. * Made the code more understable * ufal/publisher-ok-fix (#473) * fixes ufal/clarin-dspace#1096 (#471) * Added publisher filter in the integration test check --------- Co-authored-by: Ondřej Košarko * ufal/zip-preview-configurable (#475) * Made previewing of the file configurable. * The default value of the previewing the file must be true in the integration tests. * ufal/fe-oversized-file-upload-message Exposed max file limit for upload from the cfg. (#477) * ufal/be-email-restricted-download * Fixed comments in the clarin emails (#484) * ufal/be-not-show-shib-welcome-page (#485) * Added cfg property to enable/disable showing of the page with attributes passed from the idp. * ufal/be-s3-customization (#481) * The bitstream data are stored in to local store after uploading to S3 * The bitstream is removed from the S3 and local assetstore * Store number is specific if all storages are synchronized. * Return store number in the BitstreamRest object * Find out which store is used - it could be synchronized stores number. * Constant is moved to the Bitstream class * Synchronization of storages is not allowed by default - set up it in the test environment. * Added docs * Removed constant from the Bitstream class - it wasn't consistent * Overriden BitstreamStorageServiceImpl by custom SyncBitstreamStorageServiceImpl * Removed ClarinS3BitStoreService.java to SyncS3BitStoreService * Added doc and refactoring. * ufal/be-s3-checker-sync (#486) * Fixed UserCheck - There was created a wrong select * Fixed ChecksumCheck - the session was closed but there were some requests do the database then * internal/load-version-from-file (#488) * Load version from the specific file and show content in the root endpoint. * Fixed checkstyle issue * DO not log anything if the VERSION file does not exist. Added VERSION file path into cfg property. * Append every line into final String. * ufal/be-s3-checksum-fix (#487) * Cherry picked fix S3 optimalization fixes from Vanilla * Cherry picked adding of pagination on cleanup * Updated Sync services according to S3 optimization fix * The checksum value is compared between S3 and local assetstore and the new result is inserted to the database. * Added docs and refactoring * The checksum values are exposed by REST API * Fixed failing tests --------- Co-authored-by: Tim Donohue Co-authored-by: Luca Giamminonni * internal/versioning (#491) * add version to docker * Allow shell because it is using double quotes in the command string. * removed cat and ls commands --------- Co-authored-by: MajoBerger * ufal/comment-ever-failing-test * Commented still failing tests (#490) * internal/verion-update-path (#492) * Updated `build.version.file.path` because the server cannot find `VERSION_D.txt` file * Added VERSION_D.txt info `.gitignore` * User ${dspace.dir} instead of absolute path * internal/be-upgrade-clarin-dspace-7.6 (#497) * Split docker image builds into separate jobs to allow them to run in parallel. * Ensure 'main' code is tagged as 'latest' in DockerHub * Fixes #8558 - set Solr UTC timezone Set the timezone of the Solr date formatter to UTC * remove obsolete code fragments * Add action to automatically create a port PR when specified (cherry picked from commit f6a898c3d13360286c416b2588ab0447d9e3d81b) * Run PR Port action as 'dspace-bot' to allow new PRs to trigger CI checks * Minor update to label_merge_conflicts to ignore any errors (seem random at this time) * Fix typo. Config must be a valid regex (cherry picked from commit 799528963e3c0391852ecbaf82ef21ec7d477342) * Bump h2 from 2.1.214 to 2.2.220 Bumps [h2](https://github.com/h2database/h2database) from 2.1.214 to 2.2.220. - [Release notes](https://github.com/h2database/h2database/releases) - [Commits](https://github.com/h2database/h2database/compare/version-2.1.214...version-2.2.220) --- updated-dependencies: - dependency-name: com.h2database:h2 dependency-type: direct:production ... Signed-off-by: dependabot[bot] * Update FullTextContentStreams.java Fix NPE if bitstream is null * Update FullTextContentStreams.java Add additional NPE checks * Update ItemUtils.java Prevent npe if bitstream is null * dspace.cfg: remove old webui.itemlist properties These properties are no longer used in DSpace 7: webui.itemlist.widths webui.itemlist.*.widths webui.itemlist.tablewidth (cherry picked from commit 16c46c49797cf4a15b8ef51efdd365610bdf73ab) * dspace.cfg: remove old webui.browse.thumbnail.show property The webui.browse.thumbnail.show property is no longer used as of DSpace 7. Also remove subsequent references to adding thumbnails to item view columns since these are from legacy DSpace. (cherry picked from commit 213a546486073f09e73d91d014c489ed300bf59d) * dspace.cfg: remove old webui.itemlist.browse.* property The webui.itemlist.browse.* properties are no longer used in DSpace 7. (cherry picked from commit 35f72bc9d0c1a01aa0b9313216bbbe63e6960a81) * Fix #8963: Remove deletion constraint from Groomer * Enable entity type to submission form mapping by default (cherry picked from commit b71eee89c1e1dd7569e800e13eb8878548853ce6) * [DURACOM-179] replaced 'null' value with exception actual value in sendErrorResponse method calls having 'null' (cherry picked from commit aa35a47add5565a9302d276da2ceb22b8dbc320f) * DS-8935. webui.browse.link CrossLinks - Fix for multiple exact matches Fixes #8935 when multiple exact match "webui.browse.link" configuration entries are present that point to different indexes. Modified the code to return the index associated with the given metadata (which is used as the key in the hash map), instead of the key from the keySet (which may not actually be the metadata value being searched for). https://github.com/DSpace/DSpace/issues/8935 (cherry picked from commit b846c53baaeae1e19fbbafa3dc7ca724bcaf32c1) * #9006 fix referenced configuration file (cherry picked from commit 29a88d7e2dcfc36d2cd7991de3b84ef5f5623b63) * #9006 fix referenced configuration file (Test) (cherry picked from commit 309b0b355e4bffd6a1be3e6341dd6d17f99892c8) * Fix #8933: Only add the base statistic core if it hasn't already been added * Remove duplicate code * On failure log the name of the assetstore file and trace causes of exception. (cherry picked from commit 22974e982c99b7faa9d287ddc5bef4715f19849a) * Report Throwable's type too. (cherry picked from commit d6b612fc5cf84fe6b7226649451b7b927ded8997) * More description on OutOfMemoryError too. (cherry picked from commit bbe5df3f7dd4a33423fdf47702e23f3eb9ef821f) * Remove useless log.info (cherry picked from commit 1f3ad993cc4d10694112227245be3de1ec7b3762) * 103837: Refactor GA config to list bundles * README.md: Fix typo (cherry picked from commit ca8abddff1230e581501b482623966e64016d609) * Bump up versions of buildnumber-maven-plugin & build-helper-maven-plugin. add configuration for SCM failure (cherry picked from commit 78ea9e86df5e6737a4ce129120a6e1e6c74a371c) * 3331 - remove the --optimize feature of 'dspace stats-util' (cherry picked from commit 08c650921d7ce5210906db846ff29a053c2155a7) * fix MissingOptionException on help (cherry picked from commit 8ae5ffbf9b41fe2ad4e1146eceeff47e6ba985a0) * fix stylecheck (cherry picked from commit b1377ca1ef82d80f2ece9b48b8f1571e786c4525) * ingore unrecognized arguments on help (cherry picked from commit 82c9b6fc9baee9f4f5d8b4cc967b5d12b63cdd39) * add DSpaceSkipUnknownArgumentsParser as parser to ignore/skip unknown arguments in cli by help, fix not necessary ParseException in help (cherry picked from commit 86285d78aa0844b9811dcebdefa897ceeb944226) * remove not necessary else (cherry picked from commit 064e2caa37dfa283c3c08dee0e7321e36073bfa2) * Enable new skip merge commit feature * dspace-api: remove unnecessary trailing whitespace * 103818 ItemServiceImpl#inheritCollectionDefaultPolicies now clears item READ policies if new parent collection has a default READ policy * 103818 Add boolean parameter to ItemServiceImpl#inheritCollectionDefaultPolicies to decide whether to override item read policies * 103818 Add boolean parameters to ItemServiceImpl methodds to decide whether to override read policies * 103818 Extend ItemServiceTest#testMoveItemToCollectionWithMoreRestrictiveReadPolicy * 104878 Fix error in ItemServiceTest related to inheriting collection policies upon item move * 104878 Adjust ItemServiceTest to expect correct behavior of bundles when item is migrated * Fix failing IT in BulkAccessControlIT * 103837: Fix isContentBitstream() in GoogleAsyncEventListener * Rename test suites that are really integration testing. (cherry picked from commit f66ca33b0627c1b0789c9c3ce407463f5dc3356e) * Bump org.eclipse.jetty:jetty-xml Bumps [org.eclipse.jetty:jetty-xml](https://github.com/eclipse/jetty.project) from 9.4.51.v20230217 to 9.4.52.v20230823. - [Release notes](https://github.com/eclipse/jetty.project/releases) - [Commits](https://github.com/eclipse/jetty.project/compare/jetty-9.4.51.v20230217...jetty-9.4.52.v20230823) --- updated-dependencies: - dependency-name: org.eclipse.jetty:jetty-xml dependency-type: direct:production ... Signed-off-by: dependabot[bot] (cherry picked from commit 2bcc0b38a9436b0abc4c54e419f0fa6ae194269c) * Add note that rebooting Tomcat required * Add websvc.opensearch.autolink and websvc.opensearch.shortname to exposed REST configuration properties (cherry picked from commit 80b35c9650ec2f40bea3b497b65ce77d0c97bfcf) * Add a "container friendly" log4j2 cfg and output compose dspace log to console (cherry picked from commit 9eefd56cd7013730ff43969b321d6cbee2a3ae64) * Fix ClassCastException (Collection cannot be cast to Item) in Handle identifier classes (cherry picked from commit 1271374d37a3f7d9cec270e54ec106895aa934bc) * Fix checkstyle. Correct grammar of comment while doing so. (cherry picked from commit ffa2683c632cfab59b0432c203434ac15a6eb85f) * Avoid double slashes in sitemap paths. (cherry picked from commit eae4463eaa5916bd9b20f4e4132398aceeba1f02) * Remove 'cross join' from count query. Updates "countHandlesByPrefix" to use a query similar to existing "findByPrefix" (cherry picked from commit 14223bd712ce91cf97096f2201924baea8456814) * fix logical bug when checking if field is controlled authority * remove optimize option from oai import * added authorization check for license bitstream in OAI import * OAI: add support to extract embargo from bitstreams and expose it in OAI metadata (cherry picked from commit db81d758a947a9bdbb63fea9e872bc9b52a377ff) * ItemUtils.java: added method doc (cherry picked from commit 51e60fbcf92ea731c4e355c9cf080d251ffbf68f) * ItemUtils.java: improved method to account for multiple embargo policies and select the longest embargo (cherry picked from commit 538be7f09ba790a4ab7099e7027e1e8f6a9c62ea) * Refactored access-status to include embargo date based on the DefaultAccessStatusHelper logic (look at primary or first bitstream) (cherry picked from commit 895926f021a355181faef47b5c41e78031700475) * Remove unused imports (cherry picked from commit 4bd2cfdf0f931aec7a05db42f255423fe806ea77) * Fix style issues (cherry picked from commit 724a4ffb0ed9ffefb2866930655767590b462bb5) * Fix style issues (cherry picked from commit 6e2c8a4ae0068d844d0fc796001c170c8849babf) * Add null check (cherry picked from commit 0de4c3945ed7f30d41841cda4bf01acf9ffc130f) * ItemUtils.java: refactored addEmbargoField (cherry picked from commit 291afa765d29836a67727fdd2f82ac0c9f9310c4) * uketd_dc.xsl: also expose access-status if embargo or restricted (cherry picked from commit 4b40872a6d5a3934c1f79c6babf439a21ce25f66) * DefaultAccessStatusHelper: fix logic to take shortest embargo (cherry picked from commit d17ef09082aa237cffdc928d9560667487c2c976) * Remove currently unused customisation of ItemUtils (cherry picked from commit 490a982e8055991a6b8cbacece22b924466e22df) * DefaultAccessStatusHelper: getEmbargoFromItem return null embargo if status than embargo (cherry picked from commit e05e73a112ce60bd0689ce68af442382712bd5fc) * Additional Item class cast fixes in handle providers DSOs were not properly checked if they were instanceof Item before attempting the cast in HandleIdentifierProvider and VersionedHandleIdentifierProviderWithCanonicalHandles * Remove Oracle script that accidentally made it in via #8800 (cherry picked from commit 5e04edf41e452cd383597680da9c3101211156b8) * 8968 - request-a-copy email: non ASCII characters are encoded as HTML character entity references (cherry picked from commit db36d5eeae3e76b61178c2c7ac4243bc2fc20a97) * unused import (cherry picked from commit bf6e042085140e305d43d61ddce564fbfe819c7f) * 8968 - added custom StringEscapper (cherry picked from commit 103c8ee75771d3d9e58e530b8855d07cc14598c2) * checkstyle (cherry picked from commit 2c2b3b18dc781054539add48ca74e4bf688c400c) * 8968 - implementated using HtmlUtils scaping (cherry picked from commit 090beedb6f692df29d1a61d4c2e6fde09d4b4c1d) * checkstiye (cherry picked from commit d12fbe2c340e18e42dba4380ee9976bccb4ca421) * Define _version_ * #8585 Add submitter information to provenance metadata (cherry picked from commit c15ac0eb4a3d39a0de47adbfa5260a6f3b396837) * 8585 - added provenance to metadata-import and itemImport (cherry picked from commit ea6307dcc68a75c935049a02022145691693cff4) * dspace-api: fix misaligned comment (cherry picked from commit 4fba787322803cc36ef267f0d6913b92c1eaeca4) * oai_openaire.xsl : change resourceTypeGeneral for thesis Thesis are "Literature" resource type (resourceTypeGeneral), not "other research product" ref: https://github.com/openaire/guidelines-literature-repositories/issues/43#issuecomment-1318262914 and https://api.openaire.eu/vocabularies/dnet:result_typologies/publication (cherry picked from commit 669ff343503539aa6fc8b23600989ab958a403b9) * Bump org.eclipse.jetty:jetty-http Bumps [org.eclipse.jetty:jetty-http](https://github.com/eclipse/jetty.project) from 9.4.52.v20230823 to 9.4.53.v20231009. - [Release notes](https://github.com/eclipse/jetty.project/releases) - [Commits](https://github.com/eclipse/jetty.project/compare/jetty-9.4.52.v20230823...jetty-9.4.53.v20231009) --- updated-dependencies: - dependency-name: org.eclipse.jetty:jetty-http dependency-type: direct:production ... Signed-off-by: dependabot[bot] * Bump org.json:json from 20230227 to 20231013 in /dspace-api Bumps [org.json:json](https://github.com/douglascrockford/JSON-java) from 20230227 to 20231013. - [Release notes](https://github.com/douglascrockford/JSON-java/releases) - [Changelog](https://github.com/stleary/JSON-java/blob/master/docs/RELEASES.md) - [Commits](https://github.com/douglascrockford/JSON-java/commits) --- updated-dependencies: - dependency-name: org.json:json dependency-type: direct:production ... Signed-off-by: dependabot[bot] (cherry picked from commit 11a08f1ac0a9b75bf3f2869d3760b2f0e229aefe) * Reading localized license file. * Reading localized license file. * Applying commit 6108c98d82f3cbf00e248f83535b06da4040ece4 * quote Pattern for thumbnail resolution constructed from bitstream filename (cherry picked from commit a1248074681a7bc4603176fb3e7d989b91edcbcd) * check null value of bitstream name before quoting name for regex (cherry picked from commit a9bcc0c223d0219f464d986d7b7c66b3c4cbc39c) * extract bitstream thumbnail name pattern into own function (cherry picked from commit c5466c2249c092f6638a7072b57c934d1d3581b5) * Make workflow curation tasks actually work. When curation runs, there was no "current user" and no claimed task, so the code broke when trying to find people to notify about curation failures. (cherry picked from commit a76af35a0cd4f0c0e8737c736578b17bcc349691) * Community request: fake EPerson from configuration. (cherry picked from commit bb9e88d1bb452d0865f4827134baf907e6d34044) * Correct some documentation. (cherry picked from commit be22790aad7f627e2ac027773e272b703986f589) * Handle missing role. (cherry picked from commit 2e62fa3fd1f264aac0bb4a12953b6385211e5656) * Fix "Site cannot be indexed" error by ignoring ADD/REMOVE events on Site object (cherry picked from commit ef7f02fe81bc570353c0bf6a43706c77909e30e3) * Add a null check when assigning ldap groups Prevent NullReferenceException by checking if the group list is null Fixes #8920 (cherry picked from commit bb6498ed5e4696201d3e45bd377faa407dca277f) * 9043 use Templates for compiled XSLT instead of Transformer - use Templates are thread-safe and NOT Transformer (cherry picked from commit 1160341cb2a2c163c8fddc04ddab46de9041e1b8) * Update to newly released XOAI 3.4.0 (cherry picked from commit 160ebbd791c0545db6516403da40cb191a2c8b99) * add test and fix (cherry picked from commit 48b0b71c6301b6eb46c387c47b71d0729cc2f889) * unset primary bitstream on bitstream service (cherry picked from commit 47ca74bc4220249b95de9b8e71186277c9ac31ca) * adding sql expression to fix deleted primary bitstreams from bundle (cherry picked from commit 8a531ad0c7e8fdf09fa9a3870024687e6708a9a1) * add bundle remove authorization (cherry picked from commit 3255e073fa110a3354f1265853bbf531c677f6ea) * adding missing bundle REMOVE authorization (cherry picked from commit 4a05600194fb9be7e19084f3a9106a0152fd0d80) * add missing head style check (cherry picked from commit caba4bbb96f56c103c4dd8ac9f9fa5863b40e04c) * fix style errors (cherry picked from commit 74605f159af5e53a3e890f578732a858cef12e51) * new testDeleteBitstreamAndUnsetPrimaryBitstreamID test for primary bitstream verification (cherry picked from commit e6d108a94e41e58d6d701f3ef0429fda438e6555) * new testDeleteBitstreamAndUnsetPrimaryBitstreamID test for primary bitstream verification (cherry picked from commit ad0d22a13a35a2167557efeb5ddea7a3a504424d) * new testDeleteBitstreamAndUnsetPrimaryBitstreamID remove unnecessary stubs (cherry picked from commit a3e506c7f452133e3cc973705d671dba61a469d6) * make comments more clear to understand (cherry picked from commit c0bbd9d91f894fbe26f8cf7c4f166da8ba1cefd3) * typo (cherry picked from commit 74cce86afcc163c52502892556679e6175fa1948) * Add basic pagination to /groups/[uuid]/epersons endpoint (cherry picked from commit 74c72354b405ed266b65cdd50b594d25bea0e87f) * Bug fix. Only use pageSize and offset if >0 (cherry picked from commit 15de2d0074b56f421b3bbb9f3955814497985aef) * Add missing pagination test for /groups/[uuid]/epersons (cherry picked from commit 457dd9ae441fa084ff7cc3eaf9213e5497a2b298) * Add pagination to /groups/[uuid]/subgroups endpoint, along with tests (cherry picked from commit e7c4b9eba2d8148e07543c3b6c61dde359018da2) * Add basic unit test for new EpersonService methods (cherry picked from commit c000e54116498030261d988f87a496beef7d21d1) * Minor unit test fix. Use isEqualCollection to compare list with Hibernate results (cherry picked from commit cdb68a6fdc925fcbb76f9265e64771497b3f78bc) * Add countAllMembers() with tests. Update tests to use try/catch (cherry picked from commit 58a15b72975940d48ae450e6b46557b4443f2978) * Replace several usages of allMembers() with count methods to avoid performance issues (cherry picked from commit 2c9165afb08126189ee3367347e7011f89227b7c) * Fix bug in logic for determining whether a workflow group will be left empty. Need to check *both* EPerson and subgroup counts. (cherry picked from commit 9832259aa06d9fe140407ed54c4687989e98f7b2) * Use join instead of subquery as join seems slightly faster. (cherry picked from commit 9c0bf08cf4c3ab7e941ebe1bae66cf2aea720697) * Address feedback. Initialize HashSet sizes to avoid resizing. Correct comment about indeterminante ordering. (cherry picked from commit f011a5a5dbcd2def47dde7830981cf282ca660aa) * Allow users with write permission to view hidden metadata (cherry picked from commit 65a17d4390aeab69c191fb75559646aec9dda512) * Test modification: allow users with write rights to see hidden metadata (cherry picked from commit df7f6e9f4082e5aef3392932f8a87177ac202655) * Add test to check that user with read rights can see hidden metadata (cherry picked from commit 03496c36d4d47138bcd51badf8daca720d4cc484) * Verify optional message is not missing or a literal "null" value (cherry picked from commit 534ee3a699937eedd11aa5cb54f97b081bcda621) * Change the database mode to READ_ONLY during the indexing by discovery consumer (IndexEventConsumer) (cherry picked from commit 94822b50af4098d990d63e27bb3906cfa9c0ec37) * Add functions to do a manual flush of the db session and call flush before change to READ_ONLY mode to be sure we index the current object (cherry picked from commit c33d3fa87d6c29533d379939bd23b29ff3d9b5c9) * Flush database changes after switching to READONLY mode (cherry picked from commit 00a65312ccb52481cd72653b4c5465b7d16c760e) * Add test to check retrieving of policies after changing mode to READ_ONLY (cherry picked from commit d19a9599b5f08a567c93d2e167e219673518fb78) * Change class name to ContextIT and correct a test (cherry picked from commit a5567992bbe456cd33c68f695a2364f507149e7a) * dspace/config: update spider agent list Update list of spider user agents from the COUNTER-Robots project. See: https://github.com/atmire/COUNTER-Robots (cherry picked from commit 7566a79d906b5050bef01d22c5f4b3e4ab6e4b58) * removed options to ping search engines when generating sitemaps (cherry picked from commit f8f88060408c30314cdcf38ba5bbac0f367ee3fd) * XmlWorkflowCuratorServiceImpl: add check to queue task if configured; Curation: remove obsolete code preventing curation running on workflow tasks as #3157 is now implemented * Return both user and operational LDAP attributes Explicitly request both user and operation attributes for LDAP group search as the default searching does not include operational attributes. This is required to fetch the memberOf attribute when checking LDAP group membership. Fixes #9151 (cherry picked from commit 56b7cbf4dbcc4a1ec201518f291c119470cc4e93) * [DURACOM-200] improvement of checker script (cherry picked from commit 5a7c7956cd4e8b47f6a6f53adbc646adeddb0f88) * 108055: isClosed method should use xml configuration (cherry picked from commit d800d800d5346ea9a526ba2a880fc93a6892da98) * [DURACOM-192] Authentication Method related special groups are put in claim set even if a different authentication method is used (cherry picked from commit 6504d749b91508096300e4545069a0554eb5934b) * [DURACOM-192] Added test (cherry picked from commit fa39251071156a6eeb1030000f50761663e128e2) * 107891: Cache administrator group (cherry picked from commit 1e82ca7998c45bd628cd84cefce9ae3f0a0ce046) * DURACOM-199 improved test to show bug related to restricted content (cherry picked from commit dac4df9c1a0b813d2b7578a17c79dd1e9f798a55) * DURACOM-199 fix sitemap generator for restricted content and improve performance (cherry picked from commit 6d9ca388dac3a632530eccbdeda955f7842aae84) * [Port dspace-7_x] Event consumer for submission config reloading when a collection changes (#9196) * initialization with refactoring (cherry picked from commit e93dc1cf4586f684bb441eb0341b4f14393abde5) * also consider SubmissionConfigReaderException (cherry picked from commit 8a04b8775088a82ded154e325a6cb0ce250f1a2e) * rename consumer file (cherry picked from commit f6c92a4c8b25fccc6920646f99e09dd967a2ccf2) * init submission service factory (cherry picked from commit fcc52390b98e3a92927d7a99d9fcf57579a8ed97) * set submissionconfig config settings by default (cherry picked from commit e343d515198950dae7ddb705fee5b34cc7ee0eec) * renaming SubmissionConfigReaderService (cherry picked from commit 9ea7c321c374a0058935efa33e1c2941d6aeb602) * support for SubmissionConfigService (cherry picked from commit 61389fb7876a0757eaac497737aa2cfcd4d4c55a) * fixing style errors and renaming submissionConfigService (cherry picked from commit 5f49491b53c2570fa12bf725a7324e7d7338fc9c) * fixing style errors and unused imports (cherry picked from commit 31d92519508f676659c2efb9e6b9a9089305016d) * set default submission event configs (cherry picked from commit 578198c588f5e7f0da6448f1b4497a041df7fddd) * adding force indexing action to Consumer (cherry picked from commit b91236434f38e8d3510aae816f6e2e871517fb70) * stylecheck fixes (cherry picked from commit 89e89c3d8a7ef8b77a8388cde9905c7fdc1b4bba) * undo event.dispatcher.noindex.consumers (cherry picked from commit ae9dc5f0058f8c7ceb4fb960fc3ca3343fac3763) --------- Co-authored-by: Paulo Graça * Add isNotMemberOf for groups, including unit and integration tests (cherry picked from commit 9d271b24b9721741a53142a690b86287efb738fe) * Implement searchNonMembers for EPersonService. Add tests to prove it works (and tests for search()). Requires minor bug fix to AbstractHibernateDSODAO to allow for additional OR/AND clauses to be appended. (cherry picked from commit f186dcf4ca17f56478ce27946acdc2c269d8bd50) * Add /epersons/search/isNotMemberOf endpoint to REST API along with integration tests (cherry picked from commit 5208a355d69c86dc7cb3ea372656c6959664fd9a) * Bug fix to EPersonDAOImpl. Correctly determine if excluded group needs to be preceded by AND or WHERE (cherry picked from commit e5e0eaa9999a96f499c131e02877d4280f7b5263) * Updated IIIF Controller IT to text bitstream and bundle exclusions (cherry picked from commit e92b4b7bfdc08efab9aee9b8f07506273ee2bfcb) * Simplified the process of fixing the tests after adding new sidebar facets/search filters and sort options to discover.xml (cherry picked from commit b40ad0dfc23040f335d6c6be0fcd0ae6e68a318f) * 107671: Expose the handle.canonical.prefix to the frontend (cherry picked from commit 6d86e65b720b5108e94b1df85e6038394c183214) * Remove line breaks from default.license because they are being rendered in the frontend (cherry picked from commit 56aae347c2a7012af912a8893142fc04809e7ff3) * [Port dspace-7_x] subscription email: do not send email if nothing has changed (#9204) * improved subscriptions email template (cherry picked from commit 6e7b32795930ea3c0758875c654e95b8602cf9b3) * do not send emails without content (cherry picked from commit 926b2421302587d69318f208b48e334cab57b204) * fixed coding style violations (cherry picked from commit fdacec08df8f930ff7c45745a20297a10eff3ad2) * removed unnecessary isEmpty check as suggested by reviewer (cherry picked from commit 30a837f85403332c31761880a0ed936996ba1b5a) * moved null check on indexableObjects in generateBodyMail (cherry picked from commit b43c340b182a62ddc51e3fd7e294101f6d355e90) * fixed unhandled IOException (cherry picked from commit 9b3f7b698c3efbff165fb727ac91609dbe569218) * fixed typo in bodyCommunities (cherry picked from commit ac3d02eb1ce83f9e1fee55f69f1c0fd2c0e03332) * do not use != to compare strings (cherry picked from commit e46018333508215fce7225b75a5af50d4d7beb59) * fixed improper handling of empty list (cherry picked from commit ac72aae44b5ce49ec1a8ddfa2b7986d0d580a8ac) --------- Co-authored-by: Sascha Szott * [maven-release-plugin] prepare release dspace-7.6.1 * Fixed checkstyle issues * Fixed compilation errors * Removed redundant service definitions in the `core-factory-services.xml` because unit tests was failing * Removed redundant bean in the `discovery.xml` * Removed redundant `latestVersion` from `schema.xml` * Fixed wrong resolved conflicts - the app cannot start running * Used ClarinVersionedHandleIdentifierProvider instead of Vanilla one in the `identifier-service.xml` * Fixed tests - the user has an authorization to remove bundle * Fixed failing test in the `MetadataExportSearchIT` it was trying to test dateIssued filter, but the CLARIN DSpace doesn't use that. * controlled.authority.dc.relation is changed in the configuration by CLARIN so the test VersioningWithRelationshipsIT is failing because of that. * Fixed wrong license validation class * Removed test which was accidentaly added durign resolving conflicts * The ACL tag restricted loading of some input field which was missing in the assert * The test cannot convert `Oct` from the xml to normal number because of Slovak default locale which doesn't know what `Oct` means in the date. I set up EN locale for the current test * Added clarin set up of search Facets into integration tests - because src/test/java/org/dspace/app/rest/DiscoveryScopeBasedRestControllerIT.java was failing * Added method `areSpecialGroupsApplicable` into ClarinShibAuthentication because after shib authentication the user wasn't assinged into special group because of some dspace new feature * Change DEFAULT_AMOUNT_FORMATS in the BitstreamFormatRestRepositoryIT.java because vanila has added a new format. * Ignored still failing RegistrationRestRepositoryIT test - it is ignored also in the 7.5 version * fix: Failure of org.dspace.app.rest.SitemapRestControllerIT when running locally * Ignore pubmedImportMetadataGetRecordsTest or allow ssl * Let the pubmedImportMetadataGetRecordsTests ignored * Fixed CurationIT - it failed on the Stream because Stream has null param. Null param was because of null handle on the Site object - the site handle was removed in the `ClarinHandleImportControllerIT.java#createHandleWithoutDSpaceObjectTest`. * Do not delete site handle in the HandleRestRepositoryIT * Updated scripts for fast building 7.6.1 * Moved flyway script for updating checksum_results into separate file * Added missing headers --------- Signed-off-by: dependabot[bot] Co-authored-by: Tim Donohue Co-authored-by: Sean Kalynuk Co-authored-by: Sascha Szott Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Agustina Martinez Co-authored-by: Alan Orth Co-authored-by: Martin Walk Co-authored-by: corrado lombardi Co-authored-by: David P. Steelman Co-authored-by: Christian Bethge <54576195+ChrisBethgster@users.noreply.github.com> Co-authored-by: max.nuding Co-authored-by: Max Nuding Co-authored-by: Mark H. Wood Co-authored-by: Nona Luypaert Co-authored-by: Christian Clauss Co-authored-by: Hrafn Malmquist Co-authored-by: aroman-arvo Co-authored-by: Christian Bethge Co-authored-by: Koen Pauwels Co-authored-by: Mark Cooper Co-authored-by: nwoodward Co-authored-by: Kim Shepherd Co-authored-by: Marie-Hélène Vézina Co-authored-by: damian Co-authored-by: Gantner, Florian Klaus Co-authored-by: wwuck Co-authored-by: Paulo Graça Co-authored-by: Toni Prieto Co-authored-by: mohamed eskander Co-authored-by: Jens Vannerum Co-authored-by: Luca Giamminonni Co-authored-by: Andrea Bollini Co-authored-by: DSpace Bot <68393067+dspace-bot@users.noreply.github.com> Co-authored-by: Michael Spalti Co-authored-by: Alexandre Vryghem Co-authored-by: Shankeerthan Kasilingam * The cas.init() method is called in the VocabularyRestRepositoryIT#setup() method * Update docker.yml (#502) dspace-dependencies job was missing * Update docker.yml (#503) Fixed dspace-dependencies error * Update docker.yml (#504) * Empty commit - run actions * use checkstyle before commit * remove fix duplicate dependency * remove discofeed related fetching from obsolete discojuice servers (#508) * remove discofeed related fetching from obsolete discojuice servers * check style, turned off in testing, turned of by default * fix failing test --------- Co-authored-by: jm * ufal/cannot-upload-big-file (#533) * Do not compute localChecksum because it takes too long for the big files. The checksum is computed using DigestInputStream object. Copy content into sync local assetstore using buffered copying. * Fixed checkstyle issue * Updated comment about creating required directories * ufal/license-agreement-wrong-behaviour (#534) * The all transactions are not updated in the user metadata after a new download.. * Return the user metadata depending on user registration, bitstream, mapping and the last translation. * Fixed checkstyle issues * Fixed failing integration tests - do not fetch user metadata for testing by ID where there are more tests which creates the user metadata * internal/discofeed-it (#535) * Disable SSL check for specific discofeed requests * Fixed checkstyle issues * Do not throw error in private constructor * Fixed failing IT - ssl check wasn't disabled * Ignore integration test which is trying to connect to our private network. * allow running from branch rework-actions for testing * ufal/s3-check-etag (#537) * Temp commit - upload a file by uploading 50MB parts * Added uploading the file by parts - for every part is computed checksum and compared with UploadPartResult's ETag. This feature could be enabled or disabled by cfg. * Undo S3BitStoreService changes * Fixed checkstyle issues * Prettify the code * Changed cfg property to be better understandable and file size is converted into constant. * ufal/allow-impersonating Allow impersonating by default (#539) * correct branches for main gh actions * Rework actions (#540) * run correct actions on correct repos * remove old docker job, separating it to several * reusable build yml from upstream * upstream docker.yml version * first test run * steppify docker.yml action to be able to add more steps * added python version to reusable build * fix incorrect usage of steps in gh action * corect the type in reusable wf * correct condition in python version script * corrected the branch * corrected image dockerhub owner * no linux/arm64 * comment out unused tasks; minor corrections * do not redepoy becuse jm said so * do not use codecov * disable deploy differently * extend possibilities for using version script (#542) * parametrized output of python version script * RUUUN * correct version placement * correct branches for run of gh action * Fix all mistakes in the /home page (#543) * Configured browsing by language * Updated tests following the browsing cfg --------- Signed-off-by: dependabot[bot] Co-authored-by: MajoBerger <88670521+MajoBerger@users.noreply.github.com> Co-authored-by: MajoBerger Co-authored-by: Adán Román Ruiz Co-authored-by: Ondřej Košarko Co-authored-by: Tim Donohue Co-authored-by: Luca Giamminonni Co-authored-by: Sean Kalynuk Co-authored-by: Sascha Szott Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Agustina Martinez Co-authored-by: Alan Orth Co-authored-by: Martin Walk Co-authored-by: corrado lombardi Co-authored-by: David P. Steelman Co-authored-by: Christian Bethge <54576195+ChrisBethgster@users.noreply.github.com> Co-authored-by: max.nuding Co-authored-by: Max Nuding Co-authored-by: Mark H. Wood Co-authored-by: Nona Luypaert Co-authored-by: Christian Clauss Co-authored-by: Hrafn Malmquist Co-authored-by: Christian Bethge Co-authored-by: Koen Pauwels Co-authored-by: Mark Cooper Co-authored-by: nwoodward Co-authored-by: Kim Shepherd Co-authored-by: Marie-Hélène Vézina Co-authored-by: damian Co-authored-by: Gantner, Florian Klaus Co-authored-by: wwuck Co-authored-by: Paulo Graça Co-authored-by: Toni Prieto Co-authored-by: mohamed eskander Co-authored-by: Jens Vannerum Co-authored-by: Andrea Bollini Co-authored-by: DSpace Bot <68393067+dspace-bot@users.noreply.github.com> Co-authored-by: Michael Spalti Co-authored-by: Alexandre Vryghem Co-authored-by: Shankeerthan Kasilingam Co-authored-by: jm Co-authored-by: Jozef Misutka <332350+vidiecan@users.noreply.github.com> --- .github/workflows/build.yml | 52 +- .github/workflows/docker.yml | 618 +++--------------- .github/workflows/reusable-docker-build.yml | 238 +++++++ .../app/rest/BrowsesResourceControllerIT.java | 11 +- .../app/rest/matcher/BrowseIndexMatcher.java | 6 +- dspace/config/dspace.cfg | 6 +- 6 files changed, 381 insertions(+), 550 deletions(-) create mode 100644 .github/workflows/reusable-docker-build.yml diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index ac3aeac41a33..59cf1935fb64 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -92,31 +92,31 @@ jobs: path: 'dspace/target/site/jacoco-aggregate/jacoco.xml' retention-days: 14 - # Codecov upload is a separate job in order to allow us to restart this separate from the entire build/test - # job above. This is necessary because Codecov uploads seem to randomly fail at times. - # See https://community.codecov.com/t/upload-issues-unable-to-locate-build-via-github-actions-api/3954 - codecov: - # Must run after 'tests' job above - needs: tests - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v3 + # # Codecov upload is a separate job in order to allow us to restart this separate from the entire build/test + # # job above. This is necessary because Codecov uploads seem to randomly fail at times. + # # See https://community.codecov.com/t/upload-issues-unable-to-locate-build-via-github-actions-api/3954 + # codecov: + # # Must run after 'tests' job above + # needs: tests + # runs-on: ubuntu-latest + # steps: + # - name: Checkout + # uses: actions/checkout@v3 - # Download artifacts from previous 'tests' job - - name: Download coverage artifacts - uses: actions/download-artifact@v3 + # # Download artifacts from previous 'tests' job + # - name: Download coverage artifacts + # uses: actions/download-artifact@v3 - # Now attempt upload to Codecov using its action. - # NOTE: We use a retry action to retry the Codecov upload if it fails the first time. - # - # Retry action: https://github.com/marketplace/actions/retry-action - # Codecov action: https://github.com/codecov/codecov-action - - name: Upload coverage to Codecov.io - uses: Wandalen/wretry.action@v1.0.36 - with: - action: codecov/codecov-action@v3 - # Try upload 5 times max - attempt_limit: 5 - # Run again in 30 seconds - attempt_delay: 30000 + # # Now attempt upload to Codecov using its action. + # # NOTE: We use a retry action to retry the Codecov upload if it fails the first time. + # # + # # Retry action: https://github.com/marketplace/actions/retry-action + # # Codecov action: https://github.com/codecov/codecov-action + # - name: Upload coverage to Codecov.io + # uses: Wandalen/wretry.action@v1.0.36 + # with: + # action: codecov/codecov-action@v3 + # # Try upload 5 times max + # attempt_limit: 5 + # # Run again in 30 seconds + # attempt_delay: 30000 diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml index 371bfe00b023..6984b85d942a 100644 --- a/.github/workflows/docker.yml +++ b/.github/workflows/docker.yml @@ -1,13 +1,14 @@ # DSpace Docker image build for hub.docker.com name: Docker images -# Run this Build for all pushes to customer/uk branch +# Run this Build for all pushes to dtq-dev branch # Also run for PRs to ensure PR doesn't break Docker build process on: push: branches: - dtq-dev - customer/* + pull_request: workflow_dispatch: @@ -15,229 +16,22 @@ permissions: contents: read # to fetch code (actions/checkout) jobs: - docker: - # Ensure this job never runs on forked repos. It's only executed for our repo - if: github.repository == 'dataquest-dev/dspace' - runs-on: ubuntu-latest - env: - # Define tags to use for Docker images based on Git tags/branches (for docker/metadata-action) - # For a new commit on default branch (main), use the literal tag 'dspace-7_x' on Docker image. - # For a new commit on other branches, use the branch name as the tag for Docker image. - # For a new tag, copy that tag name as the tag for Docker image. - IMAGE_TAGS: | - type=raw,value=dspace-7_x,enable=${{ endsWith(github.ref, github.event.repository.default_branch) }} - type=ref,event=branch,enable=${{ !endsWith(github.ref, github.event.repository.default_branch) }} - type=ref,event=tag - # Define default tag "flavor" for docker/metadata-action per - # https://github.com/docker/metadata-action#flavor-input - # We turn off 'latest' tag by default. - TAGS_FLAVOR: | - latest=false - # Architectures / Platforms for which we will build Docker images - # If this is a PR, we ONLY build for AMD64. For PRs we only do a sanity check test to ensure Docker builds work. - # If this is NOT a PR (e.g. a tag or merge commit), also build for ARM64. NOTE: The ARM64 build takes MUCH - # longer (around 45mins or so) which is why we only run it when pushing a new Docker image. - PLATFORMS: linux/amd64${{ github.event_name != 'pull_request' && ', linux/arm64' || '' }} - - steps: - # https://github.com/actions/checkout - - name: Checkout codebase - uses: actions/checkout@v3 - - - name: Add version - run: python scripts/sourceversion.py > dspace/config/VERSION_D.txt - - # https://github.com/docker/setup-buildx-action - - name: Setup Docker Buildx - uses: docker/setup-buildx-action@v2 - - # https://github.com/docker/setup-qemu-action - - name: Set up QEMU emulation to build for multiple architectures - uses: docker/setup-qemu-action@v2 - - # https://github.com/docker/login-action - - name: Login to DockerHub - # Only login if not a PR, as PRs only trigger a Docker build and not a push - if: github.event_name != 'pull_request' - uses: docker/login-action@v2 - with: - username: ${{ secrets.DOCKER_USERNAME }} - password: ${{ secrets.DOCKER_ACCESS_TOKEN }} - - #################################################### - # Build/Push the 'dataquest/dspace-dependencies' image - #################################################### - # https://github.com/docker/metadata-action - # Get Metadata for docker_build_deps step below - - name: Sync metadata (tags, labels) from GitHub to Docker for 'dspace-dependencies' image - id: meta_build_deps - uses: docker/metadata-action@v4 - with: - images: dataquest/dspace-dependencies - tags: ${{ env.IMAGE_TAGS }} - flavor: ${{ env.TAGS_FLAVOR }} - - # https://github.com/docker/build-push-action - - name: Build and push 'dspace-dependencies' image - id: docker_build_deps - uses: docker/build-push-action@v4 - with: - context: . - file: ./Dockerfile.dependencies - platforms: ${{ env.PLATFORMS }} - # For pull requests, we run the Docker build (to ensure no PR changes break the build), - # but we ONLY do an image push to DockerHub if it's NOT a PR - push: ${{ github.event_name != 'pull_request' }} - # Use tags / labels provided by 'docker/metadata-action' above - tags: ${{ steps.meta_build_deps.outputs.tags }} - labels: ${{ steps.meta_build_deps.outputs.labels }} - - ####################################### - # Build/Push the 'dataquest/dspace' image - ####################################### - # Get Metadata for docker_build step below - - name: Sync metadata (tags, labels) from GitHub to Docker for 'dspace' image - id: meta_build - uses: docker/metadata-action@v4 - with: - images: dataquest/dspace - tags: ${{ env.IMAGE_TAGS }} - flavor: ${{ env.TAGS_FLAVOR }} - - - name: Build and push 'dspace' image - id: docker_build - uses: docker/build-push-action@v3 - with: - context: . - file: ./Dockerfile - platforms: ${{ env.PLATFORMS }} - # For pull requests, we run the Docker build (to ensure no PR changes break the build), - # but we ONLY do an image push to DockerHub if it's NOT a PR - push: ${{ github.event_name != 'pull_request' }} - # Use tags / labels provided by 'docker/metadata-action' above - tags: ${{ steps.meta_build.outputs.tags }} - labels: ${{ steps.meta_build.outputs.labels }} - - ##################################################### - # Build/Push the 'dataquest/dspace' image ('-test' tag) - ##################################################### - # Get Metadata for docker_build_test step below - - name: Sync metadata (tags, labels) from GitHub to Docker for 'dspace-test' image - id: meta_build_test - uses: docker/metadata-action@v4 - with: - images: dataquest/dspace - tags: ${{ env.IMAGE_TAGS }} - # As this is a test/development image, its tags are all suffixed with "-test". Otherwise, it uses the same - # tagging logic as the primary 'dataquest/dspace' image above. - flavor: ${{ env.TAGS_FLAVOR }} - suffix=-test - - - name: Build and push 'dspace-test' image - id: docker_build_test - uses: docker/build-push-action@v3 - with: - context: . - file: ./Dockerfile.test - platforms: ${{ env.PLATFORMS }} - # For pull requests, we run the Docker build (to ensure no PR changes break the build), - # but we ONLY do an image push to DockerHub if it's NOT a PR - push: ${{ github.event_name != 'pull_request' }} - # Use tags / labels provided by 'docker/metadata-action' above - tags: ${{ steps.meta_build_test.outputs.tags }} - labels: ${{ steps.meta_build_test.outputs.labels }} - - ########################################### - # Build/Push the 'dataquest/dspace-cli' image - ########################################### - # Get Metadata for docker_build_test step below - - name: Sync metadata (tags, labels) from GitHub to Docker for 'dspace-cli' image - id: meta_build_cli - uses: docker/metadata-action@v4 - with: - images: dataquest/dspace-cli - tags: ${{ env.IMAGE_TAGS }} - flavor: ${{ env.TAGS_FLAVOR }} - - - name: Build and push 'dspace-cli' image - id: docker_build_cli - uses: docker/build-push-action@v3 - with: - context: . - file: ./Dockerfile.cli - platforms: ${{ env.PLATFORMS }} - # For pull requests, we run the Docker build (to ensure no PR changes break the build), - # but we ONLY do an image push to DockerHub if it's NOT a PR - push: ${{ github.event_name != 'pull_request' }} - # Use tags / labels provided by 'docker/metadata-action' above - tags: ${{ steps.meta_build_cli.outputs.tags }} - labels: ${{ steps.meta_build_cli.outputs.labels }} - - - name: redeploy - if: '!cancelled()' - run: | - curl -H "Accept: application/vnd.github.everest-preview+json" \ - -H "Authorization: token ${{ secrets.DEPLOY_DEV5_GH_ACTION_DISPATCH }}" \ - --request POST \ - https://api.github.com/repos/dataquest-dev/\ - dspace-angular/actions/workflows/deploy.yml/dispatches \ - --data "{\"ref\":\"refs/heads/dtq-dev\"}" - - #################################################### - # Build/Push the 'dspace/dspace-dependencies' image. - # This image is used by all other jobs. + #################################################### + # Build/Push the 'dataquest/dspace-dependencies' image. + # This image is used by all other DSpace build jobs. #################################################### dspace-dependencies: - # Ensure this job never runs on forked repos. It's only executed for 'dspace/dspace' - if: github.repository == 'dspace/dspace' - runs-on: ubuntu-latest - - steps: - # https://github.com/actions/checkout - - name: Checkout codebase - uses: actions/checkout@v3 - - # https://github.com/docker/setup-buildx-action - - name: Setup Docker Buildx - uses: docker/setup-buildx-action@v2 - - # https://github.com/docker/setup-qemu-action - - name: Set up QEMU emulation to build for multiple architectures - uses: docker/setup-qemu-action@v2 - - # https://github.com/docker/login-action - - name: Login to DockerHub - # Only login if not a PR, as PRs only trigger a Docker build and not a push - if: github.event_name != 'pull_request' - uses: docker/login-action@v2 - with: - username: ${{ secrets.DOCKER_USERNAME }} - password: ${{ secrets.DOCKER_ACCESS_TOKEN }} - - # https://github.com/docker/metadata-action - # Get Metadata for docker_build_deps step below - - name: Sync metadata (tags, labels) from GitHub to Docker for 'dspace-dependencies' image - id: meta_build_deps - uses: docker/metadata-action@v4 - with: - images: dspace/dspace-dependencies - tags: ${{ env.IMAGE_TAGS }} - flavor: ${{ env.TAGS_FLAVOR }} + # Ensure this job never runs on forked repos. It's only executed for 'dataquest/dspace' + if: github.repository == 'dataquest-dev/dspace' + secrets: + DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }} + DOCKER_ACCESS_TOKEN: ${{ secrets.DOCKER_ACCESS_TOKEN }} + uses: ./.github/workflows/reusable-docker-build.yml + with: + build_id: dspace-dependencies + image_name: dataquest/dspace-dependencies + dockerfile_path: ./Dockerfile.dependencies - # https://github.com/docker/build-push-action - - name: Build and push 'dspace-dependencies' image - id: docker_build_deps - uses: docker/build-push-action@v4 - with: - context: . - file: ./Dockerfile.dependencies - platforms: ${{ env.PLATFORMS }} - # For pull requests, we run the Docker build (to ensure no PR changes break the build), - # but we ONLY do an image push to DockerHub if it's NOT a PR - push: ${{ github.event_name != 'pull_request' }} - # Use tags / labels provided by 'docker/metadata-action' above - tags: ${{ steps.meta_build_deps.outputs.tags }} - labels: ${{ steps.meta_build_deps.outputs.labels }} ####################################### # Build/Push the 'dataquest/dspace' image @@ -247,52 +41,20 @@ jobs: if: github.repository == 'dataquest-dev/dspace' # Must run after 'dspace-dependencies' job above needs: dspace-dependencies - runs-on: ubuntu-latest - - steps: - # https://github.com/actions/checkout - - name: Checkout codebase - uses: actions/checkout@v3 - - # https://github.com/docker/setup-buildx-action - - name: Setup Docker Buildx - uses: docker/setup-buildx-action@v2 - - # https://github.com/docker/setup-qemu-action - - name: Set up QEMU emulation to build for multiple architectures - uses: docker/setup-qemu-action@v2 - - # https://github.com/docker/login-action - - name: Login to DockerHub - # Only login if not a PR, as PRs only trigger a Docker build and not a push - if: github.event_name != 'pull_request' - uses: docker/login-action@v2 - with: - username: ${{ secrets.DOCKER_USERNAME }} - password: ${{ secrets.DOCKER_ACCESS_TOKEN }} - - # Get Metadata for docker_build step below - - name: Sync metadata (tags, labels) from GitHub to Docker for 'dspace' image - id: meta_build - uses: docker/metadata-action@v4 - with: - images: dataquest/dspace - tags: ${{ env.IMAGE_TAGS }} - flavor: ${{ env.TAGS_FLAVOR }} - - - name: Build and push 'dspace' image - id: docker_build - uses: docker/build-push-action@v4 - with: - context: . - file: ./Dockerfile - platforms: ${{ env.PLATFORMS }} - # For pull requests, we run the Docker build (to ensure no PR changes break the build), - # but we ONLY do an image push to DockerHub if it's NOT a PR - push: ${{ github.event_name != 'pull_request' }} - # Use tags / labels provided by 'docker/metadata-action' above - tags: ${{ steps.meta_build.outputs.tags }} - labels: ${{ steps.meta_build.outputs.labels }} + uses: ./.github/workflows/reusable-docker-build.yml + with: + build_id: dspace + image_name: dataquest/dspace + dockerfile_path: ./Dockerfile + run_python_version_script: true + python_version_script_dest: dspace/config/VERSION_D.txt + secrets: + DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }} + DOCKER_ACCESS_TOKEN: ${{ secrets.DOCKER_ACCESS_TOKEN }} + # Enable redeploy of sandbox & demo if the branch for this image matches the deployment branch of + # these sites as specified in reusable-docker-build.xml + REDEPLOY_SANDBOX_URL: ${{ secrets.REDEPLOY_SANDBOX_URL }} + REDEPLOY_DEMO_URL: ${{ secrets.REDEPLOY_DEMO_URL }} ############################################################# # Build/Push the 'dataquest/dspace' image ('-test' tag) @@ -300,117 +62,36 @@ jobs: dspace-test: # Ensure this job never runs on forked repos. It's only executed for 'dataquest/dspace' if: github.repository == 'dataquest-dev/dspace' - # Must run after 'dspace-dependencies' job above + # Must run after 'dspace-dependencies' job above needs: dspace-dependencies - runs-on: ubuntu-latest - - steps: - # https://github.com/actions/checkout - - name: Checkout codebase - uses: actions/checkout@v3 - - # https://github.com/docker/setup-buildx-action - - name: Setup Docker Buildx - uses: docker/setup-buildx-action@v2 - - # https://github.com/docker/setup-qemu-action - - name: Set up QEMU emulation to build for multiple architectures - uses: docker/setup-qemu-action@v2 - - # https://github.com/docker/login-action - - name: Login to DockerHub - # Only login if not a PR, as PRs only trigger a Docker build and not a push - if: github.event_name != 'pull_request' - uses: docker/login-action@v2 - with: - username: ${{ secrets.DOCKER_USERNAME }} - password: ${{ secrets.DOCKER_ACCESS_TOKEN }} - - # Get Metadata for docker_build_test step below - - name: Sync metadata (tags, labels) from GitHub to Docker for 'dspace-test' image - id: meta_build_test - uses: docker/metadata-action@v4 - with: - images: dataquest/dspace - tags: ${{ env.IMAGE_TAGS }} - # As this is a test/development image, its tags are all suffixed with "-test". Otherwise, it uses the same - # tagging logic as the primary 'dataquest/dspace' image above. - flavor: ${{ env.TAGS_FLAVOR }} - suffix=-test - - - name: Build and push 'dspace-test' image - id: docker_build_test - uses: docker/build-push-action@v4 - with: - context: . - file: ./Dockerfile.test - platforms: ${{ env.PLATFORMS }} - # For pull requests, we run the Docker build (to ensure no PR changes break the build), - # but we ONLY do an image push to DockerHub if it's NOT a PR - push: ${{ github.event_name != 'pull_request' }} - # Use tags / labels provided by 'docker/metadata-action' above - tags: ${{ steps.meta_build_test.outputs.tags }} - labels: ${{ steps.meta_build_test.outputs.labels }} + uses: ./.github/workflows/reusable-docker-build.yml + with: + build_id: dspace-test + image_name: dataquest/dspace + dockerfile_path: ./Dockerfile.test + # As this is a test/development image, its tags are all suffixed with "-test". Otherwise, it uses the same + # tagging logic as the primary 'dataquest/dspace' image above. + tags_flavor: suffix=-test + secrets: + DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }} + DOCKER_ACCESS_TOKEN: ${{ secrets.DOCKER_ACCESS_TOKEN }} ########################################### # Build/Push the 'dataquest/dspace-cli' image ########################################### dspace-cli: # Ensure this job never runs on forked repos. It's only executed for 'dataquest/dspace' - if: github.repository == 'dspace/dspace' - # Must run after 'dspace-dependencies' job above + if: github.repository == 'dataquest-dev/dspace' + # Must run after 'dspace-dependencies' job above needs: dspace-dependencies - runs-on: ubuntu-latest - - steps: - # https://github.com/actions/checkout - - name: Checkout codebase - uses: actions/checkout@v3 - - # https://github.com/docker/setup-buildx-action - - name: Setup Docker Buildx - uses: docker/setup-buildx-action@v2 - - # https://github.com/docker/setup-qemu-action - - name: Set up QEMU emulation to build for multiple architectures - uses: docker/setup-qemu-action@v2 - - # https://github.com/docker/login-action - - name: Login to DockerHub - # Only login if not a PR, as PRs only trigger a Docker build and not a push - if: github.event_name != 'pull_request' - uses: docker/login-action@v2 - with: - username: ${{ secrets.DOCKER_USERNAME }} - password: ${{ secrets.DOCKER_ACCESS_TOKEN }} - - # Get Metadata for docker_build_test step below - - name: Sync metadata (tags, labels) from GitHub to Docker for 'dspace-cli' image - id: meta_build_cli - uses: docker/metadata-action@v4 - with: - images: dataquest/dspace-cli - tags: ${{ env.IMAGE_TAGS }} - flavor: ${{ env.TAGS_FLAVOR }} - - - name: Build and push 'dspace-cli' image - id: docker_build_cli - uses: docker/build-push-action@v4 - with: - context: . - file: ./Dockerfile.cli - platforms: ${{ env.PLATFORMS }} - # For pull requests, we run the Docker build (to ensure no PR changes break the build), - # but we ONLY do an image push to DockerHub if it's NOT a PR - push: ${{ github.event_name != 'pull_request' }} - # Use tags / labels provided by 'docker/metadata-action' above - tags: ${{ steps.meta_build_cli.outputs.tags }} - labels: ${{ steps.meta_build_cli.outputs.labels }} - - deploy: - needs: docker - uses: dataquest-dev/dspace-angular/.github/workflows/deploy.yml@customer/uk - secrets: inherit + uses: ./.github/workflows/reusable-docker-build.yml + with: + build_id: dspace-cli + image_name: dataquest/dspace-cli + dockerfile_path: ./Dockerfile.cli + secrets: + DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }} + DOCKER_ACCESS_TOKEN: ${{ secrets.DOCKER_ACCESS_TOKEN }} ########################################### @@ -418,161 +99,72 @@ jobs: ########################################### dspace-solr: # Ensure this job never runs on forked repos. It's only executed for 'dataquest/dspace' - if: github.repository == 'dspace/dspace' - runs-on: ubuntu-latest - - steps: - # https://github.com/actions/checkout - - name: Checkout codebase - uses: actions/checkout@v3 - - # https://github.com/docker/setup-buildx-action - - name: Setup Docker Buildx - uses: docker/setup-buildx-action@v2 - - # https://github.com/docker/setup-qemu-action - - name: Set up QEMU emulation to build for multiple architectures - uses: docker/setup-qemu-action@v2 - - # https://github.com/docker/login-action - - name: Login to DockerHub - # Only login if not a PR, as PRs only trigger a Docker build and not a push - if: github.event_name != 'pull_request' - uses: docker/login-action@v2 - with: - username: ${{ secrets.DOCKER_USERNAME }} - password: ${{ secrets.DOCKER_ACCESS_TOKEN }} - - # Get Metadata for docker_build_solr step below - - name: Sync metadata (tags, labels) from GitHub to Docker for 'dspace-solr' image - id: meta_build_solr - uses: docker/metadata-action@v4 - with: - images: dspace/dspace-solr - tags: ${{ env.IMAGE_TAGS }} - flavor: ${{ env.TAGS_FLAVOR }} + if: github.repository == 'dataquest-dev/dspace' + uses: ./.github/workflows/reusable-docker-build.yml + with: + build_id: dspace-solr + image_name: dataquest/dspace-solr + dockerfile_path: ./dspace/src/main/docker/dspace-solr/Dockerfile + # Must pass solrconfigs to the Dockerfile so that it can find the required Solr config files + dockerfile_additional_contexts: 'solrconfigs=./dspace/solr/' + secrets: + DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }} + DOCKER_ACCESS_TOKEN: ${{ secrets.DOCKER_ACCESS_TOKEN }} + # Enable redeploy of sandbox & demo SOLR instance whenever dspace-solr image changes for deployed branch. + # These URLs MUST use different secrets than 'dspace/dspace' image build above as they are deployed separately. + REDEPLOY_SANDBOX_URL: ${{ secrets.REDEPLOY_SANDBOX_SOLR_URL }} + REDEPLOY_DEMO_URL: ${{ secrets.REDEPLOY_DEMO_SOLR_URL }} - - name: Build and push 'dspace-solr' image - id: docker_build_solr - uses: docker/build-push-action@v4 - with: - context: . - file: ./dspace/src/main/docker/dspace-solr/Dockerfile - platforms: ${{ env.PLATFORMS }} - # For pull requests, we run the Docker build (to ensure no PR changes break the build), - # but we ONLY do an image push to DockerHub if it's NOT a PR - push: ${{ github.event_name != 'pull_request' }} - # Use tags / labels provided by 'docker/metadata-action' above - tags: ${{ steps.meta_build_solr.outputs.tags }} - labels: ${{ steps.meta_build_solr.outputs.labels }} ########################################################### # Build/Push the 'dataquest/dspace-postgres-pgcrypto' image ########################################################### dspace-postgres-pgcrypto: # Ensure this job never runs on forked repos. It's only executed for 'dataquest/dspace' - if: github.repository == 'dspace/dspace' - runs-on: ubuntu-latest - - steps: - # https://github.com/actions/checkout - - name: Checkout codebase - uses: actions/checkout@v3 - - # https://github.com/docker/setup-buildx-action - - name: Setup Docker Buildx - uses: docker/setup-buildx-action@v2 - - # https://github.com/docker/setup-qemu-action - - name: Set up QEMU emulation to build for multiple architectures - uses: docker/setup-qemu-action@v2 - - # https://github.com/docker/login-action - - name: Login to DockerHub - # Only login if not a PR, as PRs only trigger a Docker build and not a push - if: github.event_name != 'pull_request' - uses: docker/login-action@v2 - with: - username: ${{ secrets.DOCKER_USERNAME }} - password: ${{ secrets.DOCKER_ACCESS_TOKEN }} - - # Get Metadata for docker_build_postgres step below - - name: Sync metadata (tags, labels) from GitHub to Docker for 'dspace-postgres-pgcrypto' image - id: meta_build_postgres - uses: docker/metadata-action@v4 - with: - images: dspace/dspace-postgres-pgcrypto - tags: ${{ env.IMAGE_TAGS }} - flavor: ${{ env.TAGS_FLAVOR }} - - - name: Build and push 'dspace-postgres-pgcrypto' image - id: docker_build_postgres - uses: docker/build-push-action@v4 - with: - # Must build out of subdirectory to have access to install script for pgcrypto - context: ./dspace/src/main/docker/dspace-postgres-pgcrypto/ - dockerfile: Dockerfile - platforms: ${{ env.PLATFORMS }} - # For pull requests, we run the Docker build (to ensure no PR changes break the build), - # but we ONLY do an image push to DockerHub if it's NOT a PR - push: ${{ github.event_name != 'pull_request' }} - # Use tags / labels provided by 'docker/metadata-action' above - tags: ${{ steps.meta_build_postgres.outputs.tags }} - labels: ${{ steps.meta_build_postgres.outputs.labels }} + if: github.repository == 'dataquest-dev/dspace' + uses: ./.github/workflows/reusable-docker-build.yml + with: + build_id: dspace-postgres-pgcrypto + image_name: dataquest/dspace-postgres-pgcrypto + # Must build out of subdirectory to have access to install script for pgcrypto. + # NOTE: this context will build the image based on the Dockerfile in the specified directory + dockerfile_context: ./dspace/src/main/docker/dspace-postgres-pgcrypto/ + secrets: + DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }} + DOCKER_ACCESS_TOKEN: ${{ secrets.DOCKER_ACCESS_TOKEN }} ######################################################################## # Build/Push the 'dataquest/dspace-postgres-pgcrypto' image (-loadsql tag) ######################################################################## dspace-postgres-pgcrypto-loadsql: # Ensure this job never runs on forked repos. It's only executed for 'dataquest/dspace' - if: github.repository == 'dspace/dspace' + if: github.repository == 'dataquest-dev/dspace' + uses: ./.github/workflows/reusable-docker-build.yml + with: + build_id: dspace-postgres-pgcrypto-loadsql + image_name: dataquest/dspace-postgres-pgcrypto + # Must build out of subdirectory to have access to install script for pgcrypto. + # NOTE: this context will build the image based on the Dockerfile in the specified directory + dockerfile_context: ./dspace/src/main/docker/dspace-postgres-pgcrypto-curl/ + # Suffix all tags with "-loadsql". Otherwise, it uses the same + # tagging logic as the primary 'dataquest/dspace-postgres-pgcrypto' image above. + tags_flavor: suffix=-loadsql + secrets: + DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }} + DOCKER_ACCESS_TOKEN: ${{ secrets.DOCKER_ACCESS_TOKEN }} + + + dspace-redeploy: runs-on: ubuntu-latest - + needs: dspace + if: false steps: - # https://github.com/actions/checkout - - name: Checkout codebase - uses: actions/checkout@v3 - - # https://github.com/docker/setup-buildx-action - - name: Setup Docker Buildx - uses: docker/setup-buildx-action@v2 - - # https://github.com/docker/setup-qemu-action - - name: Set up QEMU emulation to build for multiple architectures - uses: docker/setup-qemu-action@v2 - - # https://github.com/docker/login-action - - name: Login to DockerHub - # Only login if not a PR, as PRs only trigger a Docker build and not a push - if: github.event_name != 'pull_request' - uses: docker/login-action@v2 - with: - username: ${{ secrets.DOCKER_USERNAME }} - password: ${{ secrets.DOCKER_ACCESS_TOKEN }} - - # Get Metadata for docker_build_postgres_loadsql step below - - name: Sync metadata (tags, labels) from GitHub to Docker for 'dspace-postgres-pgcrypto-loadsql' image - id: meta_build_postgres_loadsql - uses: docker/metadata-action@v4 - with: - images: dspace/dspace-postgres-pgcrypto - tags: ${{ env.IMAGE_TAGS }} - # Suffix all tags with "-loadsql". Otherwise, it uses the same - # tagging logic as the primary 'dspace/dspace-postgres-pgcrypto' image above. - flavor: ${{ env.TAGS_FLAVOR }} - suffix=-loadsql - - - name: Build and push 'dspace-postgres-pgcrypto-loadsql' image - id: docker_build_postgres_loadsql - uses: docker/build-push-action@v4 - with: - # Must build out of subdirectory to have access to install script for pgcrypto - context: ./dspace/src/main/docker/dspace-postgres-pgcrypto-curl/ - dockerfile: Dockerfile - platforms: ${{ env.PLATFORMS }} - # For pull requests, we run the Docker build (to ensure no PR changes break the build), - # but we ONLY do an image push to DockerHub if it's NOT a PR - push: ${{ github.event_name != 'pull_request' }} - # Use tags / labels provided by 'docker/metadata-action' above - tags: ${{ steps.meta_build_postgres_loadsql.outputs.tags }} - labels: ${{ steps.meta_build_postgres_loadsql.outputs.labels }} + - name: redeploy + if: '!cancelled()' + run: | + curl -H "Accept: application/vnd.github.everest-preview+json" \ + -H "Authorization: token ${{ secrets.DEPLOY_DEV5_GH_ACTION_DISPATCH }}" \ + --request POST \ + https://api.github.com/repos/dataquest-dev/\ + dspace-angular/actions/workflows/deploy.yml/dispatches \ + --data "{\"ref\":\"refs/heads/dtq-dev\"}" diff --git a/.github/workflows/reusable-docker-build.yml b/.github/workflows/reusable-docker-build.yml new file mode 100644 index 000000000000..870cad225ec5 --- /dev/null +++ b/.github/workflows/reusable-docker-build.yml @@ -0,0 +1,238 @@ +# +# DSpace's reusable Docker build/push workflow. +# +# This is used by docker.yml for all Docker image builds +name: Reusable DSpace Docker Build + +on: + workflow_call: + # Possible Inputs to this reusable job + inputs: + python_version_script_dest: + required: false + default: version.txt + type: string + run_python_version_script: + required: false + default: false + type: boolean + # Build name/id for this Docker build. Used for digest storage to avoid digest overlap between builds. + build_id: + required: true + type: string + # Requires the image name to build (e.g dspace/dspace-test) + image_name: + required: true + type: string + # Optionally the path to the Dockerfile to use for the build. (Default is [dockerfile_context]/Dockerfile) + dockerfile_path: + required: false + type: string + # Optionally the context directory to build the Dockerfile within. Defaults to "." (current directory) + dockerfile_context: + required: false + type: string + default: '.' + # Optionally a list of "additional_contexts" to pass to Dockerfile. Defaults to empty + dockerfile_additional_contexts: + required: false + type: string + default: '' + # If Docker image should have additional tag flavor details (e.g. a suffix), it may be passed in. + tags_flavor: + required: false + type: string + secrets: + # Requires that Docker login info be passed in as secrets. + DOCKER_USERNAME: + required: true + DOCKER_ACCESS_TOKEN: + required: true + # These URL secrets are optional. When specified & branch checks match, the redeployment code below will trigger. + # Therefore builds which need to trigger redeployment MUST specify these URLs. All others should leave them empty. + REDEPLOY_SANDBOX_URL: + required: false + REDEPLOY_DEMO_URL: + required: false + +# Define shared default settings as environment variables +env: + IMAGE_NAME: ${{ inputs.image_name }} + # Define tags to use for Docker images based on Git tags/branches (for docker/metadata-action) + # For a new commit on default branch (main), use the literal tag 'latest' on Docker image. + # For a new commit on other branches, use the branch name as the tag for Docker image. + # For a new tag, copy that tag name as the tag for Docker image. + IMAGE_TAGS: | + type=raw,value=dspace-7_x,enable=${{ github.ref_name == github.event.repository.default_branch }} + type=ref,event=branch,enable=${{ github.ref_name != github.event.repository.default_branch }} + type=ref,event=tag + # Define default tag "flavor" for docker/metadata-action per + # https://github.com/docker/metadata-action#flavor-input + # We manage the 'latest' tag ourselves to the 'main' branch (see settings above) + TAGS_FLAVOR: | + latest=false + ${{ inputs.tags_flavor }} + # When these URL variables are specified & required branch matches, then the sandbox or demo site will be redeployed. + # See "Redeploy" steps below for more details. + REDEPLOY_SANDBOX_URL: ${{ secrets.REDEPLOY_SANDBOX_URL }} + REDEPLOY_DEMO_URL: ${{ secrets.REDEPLOY_DEMO_URL }} + # Current DSpace maintenance branch (and architecture) which is deployed to demo.dspace.org / sandbox.dspace.org + # (NOTE: No deployment branch specified for sandbox.dspace.org as it uses the default_branch) + DEPLOY_DEMO_BRANCH: 'dspace-7_x' + DEPLOY_ARCH: 'linux/amd64' + +jobs: + docker-build: + + strategy: + matrix: + # Architectures / Platforms for which we will build Docker images + # arch: [ 'linux/amd64', 'linux/arm64' ] + arch: [ 'linux/amd64' ] + os: [ ubuntu-latest ] + isPr: + - ${{ github.event_name == 'pull_request' }} + # If this is a PR, we ONLY build for AMD64. For PRs we only do a sanity check test to ensure Docker builds work. + # The below exclude therefore ensures we do NOT build ARM64 for PRs. + # exclude: + # - isPr: true + # os: ubuntu-latest + # arch: linux/arm64 + + runs-on: ${{ matrix.os }} + + steps: + # https://github.com/actions/checkout + - name: Checkout codebase + uses: actions/checkout@v4 + + - name: Add version + if: ${{ inputs.run_python_version_script }} + run: python scripts/sourceversion.py > ${{ inputs.python_version_script_dest }} + + # https://github.com/docker/setup-buildx-action + - name: Setup Docker Buildx + uses: docker/setup-buildx-action@v3 + + # https://github.com/docker/setup-qemu-action + - name: Set up QEMU emulation to build for multiple architectures + uses: docker/setup-qemu-action@v3 + + # https://github.com/docker/login-action + - name: Login to DockerHub + # Only login if not a PR, as PRs only trigger a Docker build and not a push + if: ${{ ! matrix.isPr }} + uses: docker/login-action@v3 + with: + username: ${{ secrets.DOCKER_USERNAME }} + password: ${{ secrets.DOCKER_ACCESS_TOKEN }} + + # https://github.com/docker/metadata-action + # Get Metadata for docker_build_deps step below + - name: Sync metadata (tags, labels) from GitHub to Docker for image + id: meta_build + uses: docker/metadata-action@v5 + with: + images: ${{ env.IMAGE_NAME }} + tags: ${{ env.IMAGE_TAGS }} + flavor: ${{ env.TAGS_FLAVOR }} + + # https://github.com/docker/build-push-action + - name: Build and push image + id: docker_build + uses: docker/build-push-action@v5 + with: + build-contexts: | + ${{ inputs.dockerfile_additional_contexts }} + context: ${{ inputs.dockerfile_context }} + file: ${{ inputs.dockerfile_path }} + platforms: ${{ matrix.arch }} + # For pull requests, we run the Docker build (to ensure no PR changes break the build), + # but we ONLY do an image push to DockerHub if it's NOT a PR + push: ${{ ! matrix.isPr }} + # Use tags / labels provided by 'docker/metadata-action' above + tags: ${{ steps.meta_build.outputs.tags }} + labels: ${{ steps.meta_build.outputs.labels }} + + # Export the digest of Docker build locally (for non PRs only) + # - name: Export Docker build digest + # if: ${{ ! matrix.isPr }} + # run: | + # mkdir -p /tmp/digests + # digest="${{ steps.docker_build.outputs.digest }}" + # touch "/tmp/digests/${digest#sha256:}" + + # Upload digest to an artifact, so that it can be used in manifest below + # - name: Upload Docker build digest to artifact + # if: ${{ ! matrix.isPr }} + # uses: actions/upload-artifact@v3 + # with: + # name: digests-${{ inputs.build_id }} + # path: /tmp/digests/* + # if-no-files-found: error + # retention-days: 1 + + # If this build is NOT a PR and passed in a REDEPLOY_SANDBOX_URL secret, + # Then redeploy https://sandbox.dspace.org if this build is for our deployment architecture and 'main' branch. + # - name: Redeploy sandbox.dspace.org (based on main branch) + # if: | + # !matrix.isPR && + # env.REDEPLOY_SANDBOX_URL != '' && + # matrix.arch == env.DEPLOY_ARCH && + # github.ref_name == github.event.repository.default_branch + # run: | + # curl -X POST $REDEPLOY_SANDBOX_URL + + # If this build is NOT a PR and passed in a REDEPLOY_DEMO_URL secret, + # Then redeploy https://demo.dspace.org if this build is for our deployment architecture and demo branch. + # - name: Redeploy demo.dspace.org (based on maintenace branch) + # if: | + # !matrix.isPR && + # env.REDEPLOY_DEMO_URL != '' && + # matrix.arch == env.DEPLOY_ARCH && + # github.ref_name == env.DEPLOY_DEMO_BRANCH + # run: | + # curl -X POST $REDEPLOY_DEMO_URL + + # Merge Docker digests (from various architectures) into a manifest. + # This runs after all Docker builds complete above, and it tells hub.docker.com + # that these builds should be all included in the manifest for this tag. + # (e.g. AMD64 and ARM64 should be listed as options under the same tagged Docker image) + # docker-build_manifest: + # if: ${{ github.event_name != 'pull_request' }} + # runs-on: ubuntu-latest + # needs: + # - docker-build + # steps: + # - name: Download Docker build digests + # uses: actions/download-artifact@v3 + # with: + # name: digests-${{ inputs.build_id }} + # path: /tmp/digests + + # - name: Set up Docker Buildx + # uses: docker/setup-buildx-action@v3 + + # - name: Add Docker metadata for image + # id: meta + # uses: docker/metadata-action@v5 + # with: + # images: ${{ env.IMAGE_NAME }} + # tags: ${{ env.IMAGE_TAGS }} + # flavor: ${{ env.TAGS_FLAVOR }} + + # - name: Login to Docker Hub + # uses: docker/login-action@v3 + # with: + # username: ${{ secrets.DOCKER_USERNAME }} + # password: ${{ secrets.DOCKER_ACCESS_TOKEN }} + + # - name: Create manifest list from digests and push + # working-directory: /tmp/digests + # run: | + # docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \ + # $(printf '${{ env.IMAGE_NAME }}@sha256:%s ' *) + + # - name: Inspect image + # run: | + # docker buildx imagetools inspect ${{ env.IMAGE_NAME }}:${{ steps.meta.outputs.version }} diff --git a/dspace-server-webapp/src/test/java/org/dspace/app/rest/BrowsesResourceControllerIT.java b/dspace-server-webapp/src/test/java/org/dspace/app/rest/BrowsesResourceControllerIT.java index 1618cbf001ab..57a8dbb3c2b7 100644 --- a/dspace-server-webapp/src/test/java/org/dspace/app/rest/BrowsesResourceControllerIT.java +++ b/dspace-server-webapp/src/test/java/org/dspace/app/rest/BrowsesResourceControllerIT.java @@ -64,15 +64,15 @@ public void findAll() throws Exception { //We expect the content type to be "application/hal+json;charset=UTF-8" .andExpect(content().contentType(contentType)) - //Our default Discovery config has 5 browse indexes, so we expect this to be reflected in the page + //Our default Discovery config has 6 browse indexes, so we expect this to be reflected in the page // object .andExpect(jsonPath("$.page.size", is(20))) - .andExpect(jsonPath("$.page.totalElements", is(5))) + .andExpect(jsonPath("$.page.totalElements", is(6))) .andExpect(jsonPath("$.page.totalPages", is(1))) .andExpect(jsonPath("$.page.number", is(0))) - //The array of browse index should have a size 5 - .andExpect(jsonPath("$._embedded.browses", hasSize(5))) + //The array of browse index should have a size 6 + .andExpect(jsonPath("$._embedded.browses", hasSize(6))) //Check that all (and only) the default browse indexes are present .andExpect(jsonPath("$._embedded.browses", containsInAnyOrder( @@ -80,7 +80,8 @@ public void findAll() throws Exception { BrowseIndexMatcher.contributorBrowseIndex("asc"), BrowseIndexMatcher.titleBrowseIndex("asc"), BrowseIndexMatcher.subjectBrowseIndex("asc"), - BrowseIndexMatcher.hierarchicalBrowseIndex("srsc") + BrowseIndexMatcher.hierarchicalBrowseIndex("srsc"), + BrowseIndexMatcher.languageBrowseIndex("asc") ))) ; } diff --git a/dspace-server-webapp/src/test/java/org/dspace/app/rest/matcher/BrowseIndexMatcher.java b/dspace-server-webapp/src/test/java/org/dspace/app/rest/matcher/BrowseIndexMatcher.java index 59a53f475a0a..e7664ceaad78 100644 --- a/dspace-server-webapp/src/test/java/org/dspace/app/rest/matcher/BrowseIndexMatcher.java +++ b/dspace-server-webapp/src/test/java/org/dspace/app/rest/matcher/BrowseIndexMatcher.java @@ -117,9 +117,9 @@ public static Matcher publisherBrowseIndex(final String order) { public static Matcher languageBrowseIndex(final String order) { return allOf( - hasJsonPath("$.metadata", contains("dc.language.iso")), - hasJsonPath("$.metadataBrowse", Matchers.is(true)), - hasJsonPath("$.dataType", equalToIgnoringCase("iso_lang")), + hasJsonPath("$.metadata", contains("local.language.name")), + hasJsonPath("$.browseType", equalToIgnoringCase(BROWSE_TYPE_VALUE_LIST)), + hasJsonPath("$.dataType", equalToIgnoringCase("text")), hasJsonPath("$.order", equalToIgnoringCase(order)), hasJsonPath("$.sortOptions[*].name", containsInAnyOrder("title", "dateissued", "dateaccessioned")), hasJsonPath("$._links.self.href", is(REST_SERVER_URL + "discover/browses/language")), diff --git a/dspace/config/dspace.cfg b/dspace/config/dspace.cfg index 0a53d98f2ab9..60e46607c21f 100644 --- a/dspace/config/dspace.cfg +++ b/dspace/config/dspace.cfg @@ -942,7 +942,7 @@ registry.metadata.load = metashare-schema.xml # can login as another user from the "edit eperson" page. This is useful for # debugging problems in a running dspace instance, especially in the workflow # process. The default value is false, i.e. no one may assume the login of another user. -#webui.user.assumelogin = true +webui.user.assumelogin = true # whether to display the contents of the licence bundle (often just the deposit # licence in standard DSpace installation) @@ -1160,8 +1160,8 @@ webui.browse.index.1 = dateissued:item:dateissued webui.browse.index.2 = author:metadata:dc.contributor.*\,dc.creator:text webui.browse.index.3 = title:item:title webui.browse.index.4 = subject:metadata:dc.subject.*:text -# webui.browse.index.5 = publisher:metadata:dc.publisher:text -# webui.browse.index.6 = language:metadata:dc.language.iso:iso_lang +webui.browse.index.5 = language:metadata:local.language.name:text +# webui.browse.index.6 = publisher:metadata:dc.publisher:text # webui.browse.index.7 = itemtype:metadata:dc.type:text # webui.browse.index.8 = rights:metadata:dc.rights.label:text