Oai elg crosswalk #798
Merged
Oai elg crosswalk #798
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 tasks
milanmajchrak
previously approved these changes
Nov 13, 2024
|
@milanmajchrak @vidiecan PR needed |
milanmajchrak
added a commit
that referenced
this pull request
Dec 6, 2024
* UFAL/Removed duplicities of the bitstreams in the cmdi (#766) * Removed duplicities of the bitstreams in the cmdi. * Fixed checkstyle violation * used lindat code instead of vanilla. * Ufal/Preview issues (#764) * Ensure the content preview doesn't overload maximum length of the column in the database. And encode the input stream into UTF-8. * Do not store HTML content in the database because it could be longer than the limit of the database column. * UFAL/Encoded the UTF-8 characters from the redirect URL to UTF (#758) * Encoded the UTF-8 characters from the redirect URL to UTF * Moved ClarinUtils into Utils class * Added a new `dq` package into ComponentScan * Moved dq.Utils into DSpace utils.Utils because the components with the same name causes conflicts. * Removed *.dq component scan from the App * Merge pull request DSpace#9790 from DSpace/backport-9775-to-dspace-7_x (#769) [Port dspace-7_x] Make statistics autocommit much more frequently Co-authored-by: Tim Donohue <[email protected]> * test for bitstream with null value of sizebytes * Update README.md * UFAL/Shibboleth - load more net-id headers e.g. persistent-id (#772) * Load netid from more than one header. authentication-shibboleth.netid-header could be list, not only single value * Shibboleth login - sort the emails passed in the shibboleth email header and get the first one. * The user is redirected to the login page when it is trying to update eperson email which is already assigned to another eperson. * Sorting emails is moved into specific method and ShibbolethLoginFilter is updated following the ShibAuthentication changes * Fixed failing tests * The ClarinShibbolethLoginFilter and ClarinShibAuthentication has duplicate code, I put it into static method. * Propagate the verification token to the DB after the email is successfully sent. (#786) * UFAL/Enhanced type-bind feature (#762) * type bind is correctly rendered in the FE, but BE is still not working * Synchronized the `submission-forms_cs.xml` * Added doc into `submission-forms` about enhanced type-bind `field` * Updated `local.cfg` for tests - added type-bind property * Updated docs for the customized type-bind configuration property. * Updated MetadataValidation following the type-bind customization. * Added isAllowed function for multiple type-bind definitions * Added some docs for the new method * The values of the input wasn't loaded. * Allowed fields could be empty when they should have values. * Used isEmpty function and created constant for the `=>`. * create preview content for tar files (#759) * create preview content for tar files * Added right logs * devided extractFile funs into several separated smaller funs * added comment and removed empty line * added empty lines and removed unwanted comments * removed empty line * used consts * try incorrect identification level * log errors and removed unneeded consts --------- Co-authored-by: milanmajchrak <[email protected]> * Internal/fix failing Clarin integration test (#796) * Initial commit * Ignore the test class from where the tests has started failing. * Ignored half of tests in the ClarinShibbolethLoginFilterIT file * Ignored all tests * unignore some tests * 3 tests ignored 3 allowed * Maybe the problematic test is hidden between 3 unignored tests * two candidates * The last candidate * Ignore just wrong test and all test should pass * Clean up object created in the test. * Removed unused import. * Check the user which is going to be deleted is not null. * Rest api for handle resolution with metadata * decoded rawvalues and response json modification * used static extractMetadata funct in HandlePlugin * return dict: * removed property for test from local * Add default licenses - from ZCU update (#801) * Added flyway file to insert default licenses with license labels and mappings * Added required header * UFAL/share submission by email (#780) * Updated table workspace with share token, created endpoint to generate share token and it is sent via email * Added method to get workspaceitem via share token. * Added an endpoint for changing the submission's owner. * Added license headers * Added test for fetching item with share Token * Added tests to check the owner is changed * Added better explanation why the BE must return Page object in the search endpoint * Validate the user in the SubmissionController, it cannot be null * Updated email - some values are fetched directly from the configuration property * Updated preAuthorization method to ADD instead of WRITE (write is used for controlling authorization for modifying the Item) and updated shareURL * Authorize the submitter which is trying to take sharing item via shareToken. * Update integration test following the feature update * Import default licenses only if the license tables are empty. (#808) * Oai elg crosswalk (#798) * problem with language code * fix amount and sizeUnitOther rest-tests errors * added language coding * new language and funding project mishmash array position * added isoCodes * removed handle from item submission (#812) * UFAL/Shibboleth - netid-header should use getArrayProperty everywhere (#807) * Fetch netid as array from the cfg. Now netid as array is used everywhere. Added integration test to ask for an email when the user send only persistent-id in the shib header. * Fixed checkstyle issue * The user is not signed in without using link with the verification token from the email/ (#809) * UFAL/Shibboleth - show error in the UI when shibboleth authentication is failed (#810) * The user is not signed in without using link with the verification token from the email/ * Send a redirect to UI with specific parameter that the Shibboleth authorization wasn't successful * UFAL/Autocomplete enhancement (#768) * Added solr index `handle_title_ac` and `_comp` for the Item * Added support for searching results from specific solr indexes. 1. Updated submission-forms autocomplete definition to specify a specific index. 2. Updated configuration is provided via REST API. 3. Create a new `/suggestions` endpoint for searching values from custom solr index - it returns VocabularyEntry page. * Supported searching Item byHandle when passed a handle as parameter without handle canonical prefix. * Added autocompleteCustom `solr-subject_ac` and `handle_title_ac`. * Added autocompleteCustom `solr-publisher_ac`. * Added cfg property to define a separator from the solr value to get only display value. * Added autocompleteCustom `solr-dataProvider_ac` * Refactored code and created integration test for the SuggestionRestController * Updated suggestion integration tests because it has had a conflict with another IT * Added doc and changed `autocomplete.custom.format` to `autocomplete.custom.separator` for proper naming. * Added support for loading suggestions from the json file - need to refactor. * Refactored and added docs. * Created tests and fixed failing ones due to updated solr definition * Synchronized submission-forms_cs.xml with the original-english one * Added docs about magic constants * Added doc why the handle is updated to canonical form in the searchbyHandle endpoint * Allow searching only within the solr indexes or JSON files permitted by the configuration. * Removed normalization of handle prefix because there could be more prefixes. Expect only handle value. * Fixed if condition and config property default value. * Fixed integration tests. Allowed autocomplete custom was missing in the test cfg. * The suggestion endpoint is allowed only for authorized users * Refactored method for the normalizing the query for the discoverQuery and added unit tests. * UFAL/Changed position of rows in submission-forms.xml following v5 (#802) * Changed position of rows in submission-forms.xml following v5 * Fixed the alignment of some text * Removed rows which are not in the v5 * Removed license selector from the `teaching` and `clariah-data` collections * Updated input differences in the submission-forms.xml following the v5. --------- Co-authored-by: Juraj Roka <[email protected]> Co-authored-by: milanmajchrak <[email protected]> * Show db connection statistics in the log file or the `dbstatistics` endpoint (#815) * Show db statistics in the log file or the `dbstatistics` endpoint * Finding out why github checks are failed - undo hibernate.cfg * Disabled automatic logging * Use scheduled CRON job instead of PostConstruct * hibernate generating property true --------- Co-authored-by: Paurikova2 <[email protected]> * Translation of submission-forms to _cs (#816) * Translation of submission-forms to _cs * Translated bitstream metadata and complex input fields * Translated the rest of submission-froms_cs.xml * Fixed regex... it must contain regex value, not the message. --------- Co-authored-by: Juraj Roka <[email protected]> Co-authored-by: milanmajchrak <[email protected]> * Updated cfg to pretify the author suggestions (#819) * crosswalk-embargo (#821) * added fn for embargo * using of res policy end_date and added comments * fix string format problem with %s * integration tests are falling down * checkstyle violations * removed findHandle duplicity * added deleted line * checkstyle violations * For now the complex input field is without autocomplete for the size and contact person (#823) * Send the custom type bind `field` to the FE configuration (#822) * fix date converting to string (#825) * fix date converting to string * made const from format * checkstyle * cherry-pick clarin v7 into dtq dev (#820) * cherry-picked DataCite related changes from customer/uk * Add a script that adds a file from url to an item intended for large file workflows * Add ways to influence the bitstream name * add more options to specify an item * Expose resourceId (DSpace#1134) A BE part of DSpace#1127 - this exposes the resourceId so it can be used in the handle mgmt table * fixes ufal#1135 - findEpersonByNetId should stop searching when it finds an eperson - moved the `return eperson` inside the for cycle (after eperson non null check). - removed the eperson param (both callers were passing in `null`) * Test release without db logs (#827) * UFAL/Matomo statistics with dimension (#813) * Updated the version of matomo dependency and tried to change request from Custom Variables to Dimension * Added a custom dimension with item's handle URL * Send custom dimension also in oai tracker * Use only IPv4 address, the Matomo tracker has a problem with IPv6 * Do not change custom dimension when the Item is null * First custom dimension should have ID '1'. * Use a valid URL for Matomo tracker in the IT * Configure handle custom dimension ID in the clarin-dspace.cfg * Refactored ipv4 method to be more readable - return null --------- Co-authored-by: Juraj Roka <[email protected]> Co-authored-by: milanmajchrak <[email protected]> Co-authored-by: milanmajchrak <[email protected]> * The `dspace.name.short` is not working properly in the email, use `dspace.shortname` instead (#828) --------- Co-authored-by: Tim Donohue <[email protected]> Co-authored-by: Paurikova2 <[email protected]> Co-authored-by: Jozef Misutka <[email protected]> Co-authored-by: jurinecko <[email protected]> Co-authored-by: Juraj Roka <[email protected]> Co-authored-by: Paurikova2 <[email protected]> Co-authored-by: Ondřej Košarko <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem description
After running OAI tests with Python for the ELG type, differences were identified that can be resolved through crosswalk modifications.
Identified Differences:
amount: Incorrectly separated values fromlocal.size.info.metadataLastDateUpdatedwas not in ignored tagslanguageTagandlanguageId: Values likeengshould be replaced byen.Note: This issue is still in progress. Please update this section if additional differences are identified.
How to run oai test locally
https://github.com/dataquest-dev/DSpace/wiki/OAI-test