diff --git a/cms/data/nav.yml b/cms/data/nav.yml index 8c5aa278ee..ddd6b7cf7e 100644 --- a/cms/data/nav.yml +++ b/cms/data/nav.yml @@ -56,9 +56,9 @@ entries: secondary_mobile: true route: doaj.support # ~~->Support:WebRoute~~ entries: - - label: Support DOAJ + - label: Institutions and libraries route: doaj.support # ~~->Support:WebRoute~~ - - label: Publisher supporters + - label: Publishers route: doaj.publisher_supporters # ~~->PublisherSupporters:WebRoute~~ - label: Supporters route: doaj.supporters # ~~->Supporters:WebRoute~~ diff --git a/cms/pages/support/index.md b/cms/pages/support/index.md index 8c9ad71dc4..cc39fbfd44 100644 --- a/cms/pages/support/index.md +++ b/cms/pages/support/index.md @@ -2,7 +2,7 @@ layout: sidenav sidenav_include: /includes/_sidenav_donation.html include: /includes/contribution_rates.html -title: Support DOAJ +title: Institutional and library supporter model section: Support sticky_sidenav: true featuremap: @@ -11,10 +11,37 @@ featuremap: --- -Support of DOAJ by academic organisations is vital and we are proud to acknowledge that over 80% of our support comes to us this way. We are very grateful to all our supporting academic organisations from around the world. +Support of DOAJ by academic organisations is vital, and we are proud to acknowledge that over 80% of our support comes to us this way. We are very grateful to all our supporting academic organisations worldwide. -The suggested contributions for academic organisations are below. Use the table to find the most appropriate option for your organisation. [Send an email](mailto:joanna@doaj.org) to Joanna Ball, Managing Director, with the details of the support level you have chosen. Alternatively, you can use our invoice-free one-time donation button to send us an amount of your choosing. +### 2024 pricing -(Publishers interested in supporting us should read the [publisher supporters](/support/publisher-supporters/) page.) +For 2024, we have revised and simplified our supporter model to align with the levels recommended by SCOSS. This new model enables us to invest in the organisation's future and to continue to provide a high-quality service to our community. + +| | Euros(€) | USDs($) | GBPs(£) | +|---------------------|----------|---------|---------| +| Large organisations | 4,000 | 4,400 | 3,440 | +| Small organisations | 2,000 | 2,200 | 1,720 | +| Organisations from [low- and middle-income countries](https://datatopics.worldbank.org/world-development-indicators/the-world-by-income-and-region.html) | 500 | 550 | 430 | + +A 30% discount will be applied to institutions supporting via a billing consortium. Please contact [supporters@doaj.org](mailto:supporters@doaj.org) for further information. + +We always have a wishlist of development projects for which we require additional funding. Please contact us if you would like to support us over and above our standard rates. + +### Why you should support us + +- We are community-led and -governed. Your support enables our commitment to being 100% independent. +- Supporting open infrastructure is a strategic choice for libraries and institutions, demonstrating your commitment to open research and sustaining open infrastructure. +- We are seeing a steady increase in demand: the number of applications we receive each year has increased by 60% since 2018, and our investigations into questionable publishing practices are becoming more complex. +- Help us deliver our role in driving standards and best practice in open access publishing, for example through the [Principles of transparency and best practice in scholarly publishing](/apply/transparency/) and the [OA Journals Toolkit](https://www.oajournals-toolkit.org/). +- You rely extensively on our metadata as a source of trusted journals, integrating it into discovery systems and open access services. + +By supporting us, your organisation will join [a growing family of like-minded institutions](/support/supporters/) committed to ensuring quality content is available online for everyone. Supporting DOAJ is a statement of belief in equitable open knowledge and science. + +### Benefits for institutional and library supporters + +- We will add your institution’s name to [our Supporters page](/support/supporters/) +- you can include details of your DOAJ support in marketing activities +- you can use our logo on your institution’s websites and in other communications +- you can integrate into your services the DOAJ metadata via our OAI/PMH service, our API or the public data dump --- diff --git a/cms/pages/support/supporters.md b/cms/pages/support/supporters.md index c7d08ddda8..6cf98e6b0f 100644 --- a/cms/pages/support/supporters.md +++ b/cms/pages/support/supporters.md @@ -7,9 +7,9 @@ featuremap: ~~Supporters:Fragment~~ --- -We are proud that over 80% of DOAJ's funding comes from academic organisations (libraries, library consortia, universities, research centres). Without this vital support, we wouldn't be able to continue the high levels of service that the research community expects of us. We are grateful for the trust shown in us by our supporters. +We are proud that over 80% of our funding comes from academic organisations (libraries, library consortia, universities, research centres). Without this vital support, we couldn't deliver the services the research community expects of us. We are grateful for the trust shown in us by our supporters. - Check [our support page](/support/) for more information on supporter levels and categories. +Check [our Institutions and libraries support page](/support/) for pricing and benefits. --- diff --git a/cms/sass/components/_accordion.scss b/cms/sass/components/_accordion.scss new file mode 100644 index 0000000000..e066ee02f6 --- /dev/null +++ b/cms/sass/components/_accordion.scss @@ -0,0 +1,3 @@ +.accordion:focus-within { + border: $grapefruit solid; +} \ No newline at end of file diff --git a/cms/sass/components/_buttons.scss b/cms/sass/components/_buttons.scss index 1e71d3aceb..061c75c454 100644 --- a/cms/sass/components/_buttons.scss +++ b/cms/sass/components/_buttons.scss @@ -117,3 +117,10 @@ button[type="submit"].button--secondary { color: currentColor; } } + +button.aria-button { + all: inherit; + -webkit-appearance: none; + -moz-appearance: none; + appearance: none; +} diff --git a/cms/sass/components/_filters.scss b/cms/sass/components/_filters.scss index e82883841b..fddb1e6e07 100644 --- a/cms/sass/components/_filters.scss +++ b/cms/sass/components/_filters.scss @@ -4,6 +4,24 @@ margin-bottom: $spacing-04; border: 0; @include typescale-06; + + input[type="checkbox"], + input[type="radio"] { + display: unset; + opacity: 0; + width: 0.8em; + height: 0.8em; + + &:focus + label { + outline: dashed 2px lightgrey; + outline-offset: 1px; + } + + &:focus:not(:focus-visible){ + outline: none; + } + } + } .filters__heading { @@ -50,6 +68,8 @@ max-height: $spacing-07; height: auto; overflow-y: auto; + padding-top: $spacing-01; + @include unstyled-list; li { diff --git a/cms/sass/components/_skip-to-main-content.scss b/cms/sass/components/_skip-to-main-content.scss new file mode 100644 index 0000000000..3542adb62c --- /dev/null +++ b/cms/sass/components/_skip-to-main-content.scss @@ -0,0 +1,35 @@ +/* Back to main content button */ + +.skip-to-main { + position: absolute; + z-index: 10000; + display: flex; + flex-direction: row; + align-items: center; + min-width: min-content; + padding: 5px; + top: 10px; + left: 10px; + background-color: $grapefruit; + + svg { + display: block; + margin: 0 auto; + stroke: $warm-black; + margin-rigth: 10px; + } + &:hover, &:focus { + svg { + margin-right: 10px; + } + } + &:hover:after, &:focus:after { + content: " Skip to main content"; + color: $warm-black; + vertical-align: bottom; + -webkit-font-feature-settings: 'liga' 1; + -moz-font-feature-settings: 'liga' 1; + font-feature-settings: 'liga' 1; + transition: 0.5 smooth; + } +} \ No newline at end of file diff --git a/cms/sass/main.scss b/cms/sass/main.scss index cdd22133b8..661f2a4ae6 100644 --- a/cms/sass/main.scss +++ b/cms/sass/main.scss @@ -28,6 +28,7 @@ "layout/sidenav", "components/alert", + "components/accordion", "components/back-to-top", "components/buttons", "components/card", @@ -52,6 +53,7 @@ "components/review-table", "components/select2", "components/search-results", + "components/skip-to-main-content", "components/stat", "components/stretch-list", "components/tabs", diff --git a/doajtest/testbook/public_site/home_page.yml b/doajtest/testbook/public_site/home_page.yml index c2261786cc..625df716f7 100644 --- a/doajtest/testbook/public_site/home_page.yml +++ b/doajtest/testbook/public_site/home_page.yml @@ -120,3 +120,22 @@ tests: bottom right-hand corner. results: - You are returned to the top of the home page +- title: Skip to main content button (Accessibility) + context: + role: anonymous + steps: + - step: Refresh the page + - step: Click tab key on the keyboard once + results: + - Skip to the main content button is unfolded and focused + - step: Click enter + results: + - Focus is moved to the main content + - step: Turn on screen reader + - step: With the keyboard navigate to Skip to main content button + results: + - Screen reader reads the button title + - step: Click enter + results: + - Focus is moved to the main content + diff --git a/doajtest/testbook/public_site/public_search.yml b/doajtest/testbook/public_site/public_search.yml index 1bce101f8c..6b47834e85 100644 --- a/doajtest/testbook/public_site/public_search.yml +++ b/doajtest/testbook/public_site/public_search.yml @@ -166,3 +166,25 @@ tests: results: - You are taken to the full text of this article on the Web. It opens in a new tab +- title: 'Test Public Search Results Display: Accessibility' + context: + role: anonymous + steps: + - step: Go to the DOAJ search page at /search/articles + - step: Turn on a screen reader + results: + - Extendable facets are focusable and focus is marked with an orange solid border + - The screenreader gives the header role ("button") + - The screenreader gives the state of the facet ("extended" or "folded") + - step: click spacebar to fold/unfold the facet + resuts: + - screenreader gives correct state of the facet ("extended" or "folded") + - step: click tab + results: + - focus is on the list of checkboxes + results: + - focus is clearly marked by the outline + - step: click spacebar to check the filter + results: + - filter is applied + diff --git a/portality/bll/services/journal.py b/portality/bll/services/journal.py index f9b6eefbab..c1d005300d 100644 --- a/portality/bll/services/journal.py +++ b/portality/bll/services/journal.py @@ -8,7 +8,7 @@ from portality import lock from portality.bll.doaj import DOAJ from portality.lib.dates import FMT_DATETIME_SHORT -from portality.store import StoreFactory, prune_container +from portality.store import StoreFactory, prune_container, StoreException from portality.crosswalks.journal_questions import Journal2QuestionXwalk from datetime import datetime @@ -115,7 +115,7 @@ def journal(self, journal_id, lock_journal=False, lock_account=None, lock_timeou return journal, the_lock - def csv(self, prune=True): + def csv(self, prune=True, logger=None): """ Generate the Journal CSV @@ -127,39 +127,51 @@ def csv(self, prune=True): """ # first validate the incoming arguments to ensure that we've got the right thing argvalidate("csv", [ - {"arg": prune, "allow_none" : False, "arg_name" : "prune"} + {"arg": prune, "allow_none" : False, "arg_name" : "prune"}, + {"arg": logger, "allow_none": True, "arg_name": "logger"} ], exceptions.ArgumentException) # ~~->FileStoreTemp:Feature~~ filename = 'journalcsv__doaj_' + dates.now_str(FMT_DATETIME_SHORT) + '_utf8.csv' container_id = app.config.get("STORE_CACHE_CONTAINER") tmpStore = StoreFactory.tmp() - out = tmpStore.path(container_id, filename, create_container=True, must_exist=False) + try: + out = tmpStore.path(container_id, filename, create_container=True, must_exist=False) + logger("Temporary CSV will be written to {x}".format(x=out)) + except StoreException as e: + logger("Could not create temporary CSV file: {x}".format(x=e)) + raise e with open(out, 'w', encoding='utf-8') as csvfile: - self._make_journals_csv(csvfile) + self._make_journals_csv(csvfile, logger=logger) + logger("Wrote CSV to output file {x}".format(x=out)) # ~~->FileStore:Feature~~ mainStore = StoreFactory.get("cache") try: mainStore.store(container_id, filename, source_path=out) url = mainStore.url(container_id, filename) + logger("Stored CSV in main cache store at {x}".format(x=url)) finally: tmpStore.delete_file(container_id, filename) # don't delete the container, just in case someone else is writing to it + logger("Deleted file from tmp store") action_register = [] if prune: + logger("Pruning old CSVs from store") def sort(filelist): rx = "journalcsv__doaj_(.+?)_utf8.csv" return sorted(filelist, key=lambda x: datetime.strptime(re.match(rx, x).groups(1)[0], FMT_DATETIME_SHORT), reverse=True) def _filter(f_name): return f_name.startswith("journalcsv__") - action_register = prune_container(mainStore, container_id, sort, filter=_filter, keep=2) + action_register = prune_container(mainStore, container_id, sort, filter=_filter, keep=2, logger=logger) + logger("Pruned old CSVs from store") # update the ES record to point to the new file # ~~-> Cache:Model~~ models.Cache.cache_csv(url) + logger("Stored CSV URL in ES Cache") return url, action_register def admin_csv(self, file_path, account_sub_length=8, obscure_accounts=True, add_sensitive_account_info=False): @@ -207,11 +219,12 @@ def acc_email(j): self._make_journals_csv(f, extra_cols) @staticmethod - def _make_journals_csv(file_object, additional_columns=None): + def _make_journals_csv(file_object, additional_columns=None, logger=None): """ Make a CSV file of information for all journals. :param file_object: a utf8 encoded file object. """ + logger = logger if logger is not None else lambda x: x YES_NO = {True: 'Yes', False: 'No', None: '', '': ''} def _get_doaj_meta_kvs(journal): @@ -243,6 +256,8 @@ def _get_article_kvs(journal): # ~~!JournalCSV:Feature->Journal:Model~~ cols = {} for j in models.Journal.all_in_doaj(page_size=1000): #Fixme: limited by ES, this may not be sufficient + logger("Exporting journal {x}".format(x=j.id)) + bj = j.bibjson() issn = bj.get_one_identifier(idtype=bj.P_ISSN) if issn is None: @@ -265,6 +280,7 @@ def _get_article_kvs(journal): toc_kv = _get_doaj_toc_kv(j) cols[issn].insert(2, toc_kv) + logger("All journals exported") issns = cols.keys() csvwriter = csv.writer(file_object) @@ -275,4 +291,5 @@ def _get_article_kvs(journal): csvwriter.writerow(qs) vs = [v for _, v in cols[i]] csvwriter.writerow(vs) + logger("CSV Written") diff --git a/portality/forms/application_forms.py b/portality/forms/application_forms.py index d1f8a44d20..33cd4c1eb7 100644 --- a/portality/forms/application_forms.py +++ b/portality/forms/application_forms.py @@ -169,18 +169,21 @@ class FieldDefinitions: "contexts": { "admin": { "widgets": [ + "trim_whitespace", # ~~^-> TrimWhitespace:FormWidget~~ "click_to_copy", # ~~^-> ClickToCopy:FormWidget~~ ] }, "editor": { "disabled": True, "widgets": [ + "trim_whitespace", # ~~^-> TrimWhitespace:FormWidget~~ "click_to_copy", # ~~^-> ClickToCopy:FormWidget~~ ] }, "associate_editor": { "disabled": True, "widgets": [ + "trim_whitespace", # ~~^-> TrimWhitespace:FormWidget~~ "click_to_copy", # ~~^-> ClickToCopy:FormWidget~~ ] }, @@ -212,16 +215,19 @@ class FieldDefinitions: }, "admin": { "widgets": [ + "trim_whitespace", # ~~^-> TrimWhitespace:FormWidget~~ "click_to_copy", # ~~^-> ClickToCopy:FormWidget~~ ] }, "associate_editor": { "widgets": [ + "trim_whitespace", # ~~^-> TrimWhitespace:FormWidget~~ "click_to_copy", # ~~^-> ClickToCopy:FormWidget~~ ] }, "editor": { "widgets": [ + "trim_whitespace", # ~~^-> TrimWhitespace:FormWidget~~ "click_to_copy", # ~~^-> ClickToCopy:FormWidget~~ ] } @@ -474,7 +480,7 @@ class FieldDefinitions: ], "widgets": [ "trim_whitespace", # ~~^-> TrimWhitespace:FormWidget~~ - {"autocomplete": {"type" : "journal", "field": "bibjson.publisher.name.exact"}}, + {"autocomplete": {"type" : "journal", "field": "bibjson.publisher.name.exact"}}, # ~~^-> Autocomplete:FormWidget~~ "full_contents" # ~~^->FullContents:FormWidget~~ ], "help": { @@ -486,16 +492,22 @@ class FieldDefinitions: }, "admin": { "widgets": [ + "trim_whitespace", # ~~^-> TrimWhitespace:FormWidget~~ + {"autocomplete": {"type": "journal", "field": "bibjson.publisher.name.exact"}}, # ~~^-> Autocomplete:FormWidget~~ "click_to_copy", # ~~^-> ClickToCopy:FormWidget~~ ] }, "associate_editor": { "widgets": [ + "trim_whitespace", # ~~^-> TrimWhitespace:FormWidget~~ + {"autocomplete": {"type": "journal", "field": "bibjson.publisher.name.exact"}}, # ~~^-> Autocomplete:FormWidget~~ "click_to_copy", # ~~^-> ClickToCopy:FormWidget~~ ] }, "editor": { "widgets": [ + "trim_whitespace", # ~~^-> TrimWhitespace:FormWidget~~ + {"autocomplete": {"type": "journal", "field": "bibjson.publisher.name.exact"}}, # ~~^-> Autocomplete:FormWidget~~ "click_to_copy", # ~~^-> ClickToCopy:FormWidget~~ ] } @@ -565,7 +577,7 @@ class FieldDefinitions: }, "widgets": [ "trim_whitespace", # ~~^-> TrimWhitespace:FormWidget~~ - {"autocomplete": {"type" : "journal", "field": "bibjson.institution.name.exact"}}, + {"autocomplete": {"type" : "journal", "field": "bibjson.institution.name.exact"}}, # ~~^-> Autocomplete:FormWidget~~ "full_contents" # ~~^->FullContents:FormWidget~~ ] } @@ -1639,7 +1651,7 @@ class FieldDefinitions: "owner_exists" ], "widgets": [ - {"autocomplete": {"type" : "account", "field": "id", "include" : False}}, + {"autocomplete": {"type" : "account", "field": "id", "include" : False}}, # ~~^-> Autocomplete:FormWidget~~ "clickable_owner" ], "contexts" : { @@ -1697,7 +1709,7 @@ class FieldDefinitions: "label": "Group", "input": "text", "widgets": [ - {"autocomplete": {"type" : "editor_group", "field": "name", "include" : False}} + {"autocomplete": {"type" : "editor_group", "field": "name", "include" : False}} # ~~^-> Autocomplete:FormWidget~~ ], "contexts" : { "editor" : { @@ -1705,7 +1717,7 @@ class FieldDefinitions: }, "admin" : { "widgets" : [ - {"autocomplete": {"type": "editor_group", "field": "name", "include" : False}}, + {"autocomplete": {"type": "editor_group", "field": "name", "include" : False}}, # ~~^-> Autocomplete:FormWidget~~ {"load_editors" : {"field" : "editor"}} ] } diff --git a/portality/models/background.py b/portality/models/background.py index ac3d3bfc65..604eccc95d 100644 --- a/portality/models/background.py +++ b/portality/models/background.py @@ -152,13 +152,16 @@ def pretty_audit(self): class StdOutBackgroundJob(BackgroundJob): - def __init__(self, inner): + def __init__(self, inner, force_logging=False): super(StdOutBackgroundJob, self).__init__(**inner.data) + self._force_logging = force_logging def add_audit_message(self, msg, timestamp=None): super(StdOutBackgroundJob, self).add_audit_message(msg, timestamp) - if app.config.get("DOAJENV") == 'dev': - print(msg) + if app.config.get("DOAJENV") == 'dev' or self._force_logging: + if timestamp is None: + timestamp = dates.now_str_with_microseconds() + print("[" + timestamp + "] " + msg) # ~~-> DataObj:Library~~ diff --git a/portality/scripts/journalcsv.py b/portality/scripts/journalcsv.py index 7c00cbdf41..dedfb51c9b 100644 --- a/portality/scripts/journalcsv.py +++ b/portality/scripts/journalcsv.py @@ -9,10 +9,17 @@ exit() user = app.config.get("SYSTEM_USERNAME") + print("Running journal CSV export for user {}".format(user)) + job = journal_csv.JournalCSVBackgroundTask.prepare(user) - job = StdOutBackgroundJob(job) + job = StdOutBackgroundJob(job, force_logging=True) + print("Background Job prepared with id {}".format(job.id)) + task = journal_csv.JournalCSVBackgroundTask(job) + print("Background task created") + BackgroundApi.execute(task) + print("Finished journal CSV export for user {}".format(user)) diff --git a/portality/scripts/manage_background_jobs.py b/portality/scripts/manage_background_jobs.py index fbfa648f8b..4faa18193d 100644 --- a/portality/scripts/manage_background_jobs.py +++ b/portality/scripts/manage_background_jobs.py @@ -22,45 +22,64 @@ from portality.lib import dates from portality.lib.dates import DEFAULT_TIMESTAMP_VAL +from portality.tasks.anon_export import AnonExportBackgroundTask +from portality.tasks.article_bulk_delete import ArticleBulkDeleteBackgroundTask +from portality.tasks.article_cleanup_sync import ArticleCleanupSyncBackgroundTask +from portality.tasks.article_duplicate_report import ArticleDuplicateReportBackgroundTask +from portality.tasks.async_workflow_notifications import AsyncWorkflowBackgroundTask +from portality.tasks.check_latest_es_backup import CheckLatestESBackupBackgroundTask +# from portality.tasks.find_discontinued_soon import FindDiscontinuedSoonBackgroundTask +from portality.tasks.harvester import HarvesterBackgroundTask from portality.tasks.ingestarticles import IngestArticlesBackgroundTask -from portality.tasks.preservation import PreservationBackgroundTask -from portality.tasks.suggestion_bulk_edit import SuggestionBulkEditBackgroundTask -from portality.tasks.sitemap import SitemapBackgroundTask -from portality.tasks.read_news import ReadNewsBackgroundTask +from portality.tasks.journal_bulk_delete import JournalBulkDeleteBackgroundTask +from portality.tasks.journal_bulk_edit import JournalBulkEditBackgroundTask from portality.tasks.journal_csv import JournalCSVBackgroundTask -from portality.tasks.article_cleanup_sync import ArticleCleanupSyncBackgroundTask from portality.tasks.journal_in_out_doaj import SetInDOAJBackgroundTask -from portality.tasks.check_latest_es_backup import CheckLatestESBackupBackgroundTask +from portality.tasks.preservation import PreservationBackgroundTask from portality.tasks.prune_es_backups import PruneESBackupsBackgroundTask from portality.tasks.public_data_dump import PublicDataDumpBackgroundTask -from portality.tasks.harvester import HarvesterBackgroundTask -from portality.tasks.anon_export import AnonExportBackgroundTask +from portality.tasks.read_news import ReadNewsBackgroundTask +from portality.tasks.reporting import ReportingBackgroundTask +from portality.tasks.sitemap import SitemapBackgroundTask +from portality.tasks.suggestion_bulk_edit import SuggestionBulkEditBackgroundTask + +from portality.background import BackgroundApi # dict of {task_name: task_class} so we can interact with the jobs HANDLERS = { - PreservationBackgroundTask.__action__:PreservationBackgroundTask, + AnonExportBackgroundTask.__action__: AnonExportBackgroundTask, + ArticleBulkDeleteBackgroundTask.__action__: ArticleBulkDeleteBackgroundTask, + ArticleCleanupSyncBackgroundTask.__action__: ArticleCleanupSyncBackgroundTask, + ArticleDuplicateReportBackgroundTask.__action__: ArticleDuplicateReportBackgroundTask, + AsyncWorkflowBackgroundTask.__action__: AsyncWorkflowBackgroundTask, + CheckLatestESBackupBackgroundTask.__action__: CheckLatestESBackupBackgroundTask, + # FindDiscontinuedSoonBackgroundTask.__action__: FindDiscontinuedSoonBackgroundTask, + HarvesterBackgroundTask.__action__: HarvesterBackgroundTask, IngestArticlesBackgroundTask.__action__: IngestArticlesBackgroundTask, - SuggestionBulkEditBackgroundTask.__action__: SuggestionBulkEditBackgroundTask, - SitemapBackgroundTask.__action__: SitemapBackgroundTask, - ReadNewsBackgroundTask.__action__: ReadNewsBackgroundTask, + JournalBulkDeleteBackgroundTask.__action__: JournalBulkDeleteBackgroundTask, + JournalBulkEditBackgroundTask.__action__: JournalBulkEditBackgroundTask, JournalCSVBackgroundTask.__action__: JournalCSVBackgroundTask, - ArticleCleanupSyncBackgroundTask.__action__: ArticleCleanupSyncBackgroundTask, SetInDOAJBackgroundTask.__action__: SetInDOAJBackgroundTask, - CheckLatestESBackupBackgroundTask.__action__: CheckLatestESBackupBackgroundTask, + PreservationBackgroundTask.__action__:PreservationBackgroundTask, PruneESBackupsBackgroundTask.__action__: PruneESBackupsBackgroundTask, PublicDataDumpBackgroundTask.__action__: PublicDataDumpBackgroundTask, - HarvesterBackgroundTask.__action__: HarvesterBackgroundTask, - AnonExportBackgroundTask.__action__: AnonExportBackgroundTask, + ReadNewsBackgroundTask.__action__: ReadNewsBackgroundTask, + ReportingBackgroundTask.__action__: ReportingBackgroundTask, + SitemapBackgroundTask.__action__: SitemapBackgroundTask, + SuggestionBulkEditBackgroundTask.__action__: SuggestionBulkEditBackgroundTask } -def manage_jobs(verb, action, status, from_date, to_date): +def manage_jobs(verb, action, status, from_date, to_date, prompt=True): q = JobsQuery(action, status, from_date, to_date) jobs = models.BackgroundJob.q2obj(q=q.query()) print('You are about to {verb} {count} job(s)'.format(verb=verb, count=len(jobs))) - doit = input('Proceed? [y\\N] ') + + doit = "y" + if prompt: + doit = input('Proceed? [y\\N] ') if doit.lower() == 'y': print('Please wait...') @@ -70,7 +89,7 @@ def manage_jobs(verb, action, status, from_date, to_date): continue job.add_audit_message("Job {pp} from job management script.".format( - pp={'requeue': 'requeued', 'cancel': 'cancelled'}[verb])) + pp={'requeue': 'requeued', 'cancel': 'cancelled', "process": "processed"}[verb])) if verb == 'requeue': # Re-queue and execute immediately job.queue() @@ -78,18 +97,24 @@ def manage_jobs(verb, action, status, from_date, to_date): elif verb == 'cancel': # Just apply cancelled status job.cancel() job.save() + elif verb == 'process': + task = HANDLERS[job.action](job) # Just execute immediately without going through huey + BackgroundApi.execute(task) print('done.') else: print('No action.') -def requeue_jobs(action, status, from_date, to_date): - manage_jobs('requeue', action, status, from_date, to_date) +def requeue_jobs(action, status, from_date, to_date, prompt=True): + manage_jobs('requeue', action, status, from_date, to_date, prompt=prompt) + +def cancel_jobs(action, status, from_date, to_date, prompt=True): + manage_jobs('cancel', action, status, from_date, to_date, prompt=prompt) -def cancel_jobs(action, status, from_date, to_date): - manage_jobs('cancel', action, status, from_date, to_date) +def process_jobs(action, status, from_date, to_date, prompt=True): + manage_jobs("process", action, status, from_date, to_date, prompt=prompt) class JobsQuery(object): @@ -127,6 +152,8 @@ def query(self): help='Add these jobs back on the job queue for processing', action='store_true') parser.add_argument('-c', '--cancel', help='Cancel these jobs (set their status to "cancelled")', action='store_true') + parser.add_argument("-p", "--process", + help="Immediately process these jobs on the command line", action="store_true") parser.add_argument('-s', '--status', help='Filter for job status. Default is "queued"', default='queued') @@ -139,15 +166,18 @@ def query(self): parser.add_argument('-t', '--to_date', help='Date to which to look for jobs in the given type and status', default=dates.now_str()) + parser.add_argument("-y", "--yes", help="Answer yes to all prompts", action="store_true") args = parser.parse_args() if args.requeue and args.cancel: print('Use only --requeue OR --cancel, not both.') exit(1) elif args.requeue: - requeue_jobs(args.action, args.status, args.from_date, args.to_date) + requeue_jobs(args.action, args.status, args.from_date, args.to_date, prompt=False if args.yes else True) elif args.cancel: - cancel_jobs(args.action, args.status, args.from_date, args.to_date) + cancel_jobs(args.action, args.status, args.from_date, args.to_date, prompt=False if args.yes else True) + elif args.process: + process_jobs(args.action, args.status, args.from_date, args.to_date, prompt=False if args.yes else True) else: - print('You must supply one of --requeue or --cancel to run this script') + print('You must supply one of --requeue, --cancel or --process to run this script') exit(1) diff --git a/portality/settings.py b/portality/settings.py index c9c399983e..c8bbb99fb2 100644 --- a/portality/settings.py +++ b/portality/settings.py @@ -9,7 +9,7 @@ # Application Version information # ~~->API:Feature~~ -DOAJ_VERSION = "6.4.1" +DOAJ_VERSION = "6.4.3" API_VERSION = "3.0.1" ###################################### @@ -429,7 +429,7 @@ HUEY_SCHEDULE = { "sitemap": {"month": "*", "day": "*", "day_of_week": "*", "hour": "8", "minute": "0"}, "reporting": {"month": "*", "day": "1", "day_of_week": "*", "hour": "0", "minute": "0"}, - "journal_csv": {"month": "*", "day": "*", "day_of_week": "*", "hour": "*", "minute": "20"}, + "journal_csv": CRON_NEVER, # {"month": "*", "day": "*", "day_of_week": "*", "hour": "*", "minute": "20"}, "read_news": {"month": "*", "day": "*", "day_of_week": "*", "hour": "*", "minute": "30"}, "article_cleanup_sync": {"month": "*", "day": "2", "day_of_week": "*", "hour": "0", "minute": "0"}, "async_workflow_notifications": {"month": "*", "day": "*", "day_of_week": "1", "hour": "5", "minute": "0"}, diff --git a/portality/static/js/doaj.fieldrender.edges.js b/portality/static/js/doaj.fieldrender.edges.js index 169ce93133..49faf4b543 100644 --- a/portality/static/js/doaj.fieldrender.edges.js +++ b/portality/static/js/doaj.fieldrender.edges.js @@ -645,13 +645,13 @@ $.extend(true, doaj, { toggle = ''; } var placeholder = 'Search ' + this.component.nodeCount + ' subjects'; - var frag = '

' + this.title + toggle + '

\ - '; // substitute in the component parts frag = frag.replace(/{{FILTERS}}/g, treeFrag); @@ -1832,10 +1832,10 @@ $.extend(true, doaj, { if (this.togglable) { toggle = ''; } - var frag = '

' + this.component.display + toggle + '

\ - '; // substitute in the component parts frag = frag.replace(/{{FILTERS}}/g, filterFrag + results); @@ -2083,10 +2083,10 @@ $.extend(true, doaj, { if (this.togglable) { toggle = ''; } - var frag = '

' + this.component.display + toggle + '

\ - '; // substitute in the component parts frag = frag.replace(/{{FILTERS}}/g, filterFrag + results); diff --git a/portality/store.py b/portality/store.py index 90300fb1aa..2d0935ee19 100644 --- a/portality/store.py +++ b/portality/store.py @@ -292,7 +292,8 @@ def list_container_ids(self): return [x for x in os.listdir(self.dir) if os.path.isdir(os.path.join(self.dir, x))] -def prune_container(storage, container_id, sort, filter=None, keep=1): +def prune_container(storage, container_id, sort, filter=None, keep=1, logger=None): + logger = logger if logger is not None else lambda x: x action_register = [] filelist = storage.list(container_id) @@ -316,7 +317,9 @@ def prune_container(storage, container_id, sort, filter=None, keep=1): #action_register.append("Considering files for retention in the following order: " + ", ".join(filtered_sorted)) remove = filtered_sorted[keep:] - action_register.append("Removed old files: " + ", ".join(remove)) + msg = "Removed old files: " + ", ".join(remove) + action_register.append(msg) + logger(msg) for fn in remove: storage.delete_file(container_id, fn) diff --git a/portality/tasks/journal_csv.py b/portality/tasks/journal_csv.py index e863aeb9c4..9b5b74269d 100644 --- a/portality/tasks/journal_csv.py +++ b/portality/tasks/journal_csv.py @@ -15,12 +15,16 @@ def run(self): Execute the task as specified by the background_job :return: """ + + def logger(msg): + self.background_job.add_audit_message(msg) + job = self.background_job journalService = DOAJ.journalService() - url, action_register = journalService.csv() - for ar in action_register: - job.add_audit_message(ar) + url, action_register = journalService.csv(logger=logger) + # for ar in action_register: + # job.add_audit_message(ar) job.add_audit_message("CSV generated; will be served from {y}".format(y=url)) def cleanup(self): diff --git a/portality/templates/account/forgot.html b/portality/templates/account/forgot.html index 241525adfd..d8f5e9c837 100644 --- a/portality/templates/account/forgot.html +++ b/portality/templates/account/forgot.html @@ -3,7 +3,7 @@ {% block page_title %}Reset your password{% endblock %} {% block content %} -
+
@@ -23,5 +23,5 @@

Reset your password

-
+ {% endblock %} diff --git a/portality/templates/account/login.html b/portality/templates/account/login.html index 247149641a..726831e9e3 100644 --- a/portality/templates/account/login.html +++ b/portality/templates/account/login.html @@ -3,7 +3,7 @@ {% block page_title %}Login to your account{% endblock %} {% block content %} -
+
@@ -17,5 +17,5 @@

Login

-
+ {% endblock %} diff --git a/portality/templates/account/login_to_apply.html b/portality/templates/account/login_to_apply.html index 5d9ee3e8ed..556fbb71cb 100644 --- a/portality/templates/account/login_to_apply.html +++ b/portality/templates/account/login_to_apply.html @@ -3,7 +3,7 @@ {% block page_title %}Login to apply{% endblock %} {% block content %} -
+
@@ -46,5 +46,5 @@

Related help

-
+ {% endblock %} diff --git a/portality/templates/account/register.html b/portality/templates/account/register.html index 4f91d8250a..a497bd396a 100644 --- a/portality/templates/account/register.html +++ b/portality/templates/account/register.html @@ -12,7 +12,7 @@ {% endblock %} {% block content %} -
+
@@ -30,7 +30,7 @@

Register

-
+ {% endblock %} {% block extra_js_bottom %} diff --git a/portality/templates/account/reset.html b/portality/templates/account/reset.html index 2b459de104..fdacc27620 100644 --- a/portality/templates/account/reset.html +++ b/portality/templates/account/reset.html @@ -4,7 +4,7 @@ {% block content %} -
+
@@ -20,6 +20,6 @@

Hi {{ account.name or account.email }}

-
+ {% endblock %} diff --git a/portality/templates/api/current/api_docs.html b/portality/templates/api/current/api_docs.html index 9c5a2bc8e5..42d3d14588 100644 --- a/portality/templates/api/current/api_docs.html +++ b/portality/templates/api/current/api_docs.html @@ -14,7 +14,7 @@ {% endblock %} {% block content %} -
+
{# todo: this nav was bumping into swagger @@ -58,7 +58,7 @@

API

-
+ {% endblock %} {% block extra_js_bottom %} diff --git a/portality/templates/application_form/public_application.html b/portality/templates/application_form/public_application.html index af0e05d63e..31439b94d3 100644 --- a/portality/templates/application_form/public_application.html +++ b/portality/templates/application_form/public_application.html @@ -23,7 +23,7 @@ {% block content scoped %} -
+
{% include "application_form/_backend_validation.html" %}
@@ -64,7 +64,7 @@
-
+ {% endblock %} diff --git a/portality/templates/application_form/readonly_journal.html b/portality/templates/application_form/readonly_journal.html index 2fed49db61..fe429eea08 100644 --- a/portality/templates/application_form/readonly_journal.html +++ b/portality/templates/application_form/readonly_journal.html @@ -20,7 +20,7 @@ {% block content scoped %} -
+
@@ -47,7 +47,7 @@
-
+ {% endblock %} diff --git a/portality/templates/doaj/article.html b/portality/templates/doaj/article.html index 67720a8585..91f3f80200 100644 --- a/portality/templates/doaj/article.html +++ b/portality/templates/doaj/article.html @@ -67,7 +67,7 @@ {% set doi = bibjson.get_one_identifier("doi") %} {% set normalised_doi = article.get_normalised_doi() %} -
+

@@ -224,5 +224,5 @@

Published in {{jtitle}}

-
+ {% endblock %} diff --git a/portality/templates/doaj/articles_search.html b/portality/templates/doaj/articles_search.html index cea8977897..4a518fb3ae 100644 --- a/portality/templates/doaj/articles_search.html +++ b/portality/templates/doaj/articles_search.html @@ -10,10 +10,10 @@ {%- block meta_twitter_description -%}Find open access articles in DOAJ.{%- endblock -%} {% block content %} -
+
{% include "includes/search-help-modal.html" %} -
+ {% endblock %} {% block extra_js_bottom %} diff --git a/portality/templates/doaj/contact.html b/portality/templates/doaj/contact.html index 2a4739dec8..2ffb86985f 100644 --- a/portality/templates/doaj/contact.html +++ b/portality/templates/doaj/contact.html @@ -1,7 +1,7 @@ {% extends "layouts/public_base.html" %} {% block content %} -
+

Submit your feedback and questions here. Feedback submitted about a particular journal is treated as confidential.

@@ -52,7 +52,7 @@
-
+ {% endblock %} diff --git a/portality/templates/doaj/index.html b/portality/templates/doaj/index.html index 02bb78b878..f24f55571b 100644 --- a/portality/templates/doaj/index.html +++ b/portality/templates/doaj/index.html @@ -74,7 +74,7 @@

DOAJ in numbers

{% endblock %} {% block content %} -
+
@@ -246,6 +246,6 @@

Recently-added journals

-
+ {% endblock %} diff --git a/portality/templates/doaj/journals_search.html b/portality/templates/doaj/journals_search.html index e38bedd18e..8eef0d2c63 100644 --- a/portality/templates/doaj/journals_search.html +++ b/portality/templates/doaj/journals_search.html @@ -10,10 +10,10 @@ {%- block meta_twitter_description -%}Find open access journals in DOAJ.{%- endblock -%} {% block content %} -
+
{% include "includes/search-help-modal.html" %} -
+ {% endblock %} {% block extra_js_bottom %} diff --git a/portality/templates/doaj/toc.html b/portality/templates/doaj/toc.html index 276d9392be..adcb4c5ad9 100644 --- a/portality/templates/doaj/toc.html +++ b/portality/templates/doaj/toc.html @@ -42,7 +42,7 @@ } %} -
+
{% if journal.last_manually_updated_since(days=30) %} @@ -450,7 +450,7 @@

Journal metadata

-
+ {% include "includes/_hotjar.html" %} {% endblock %} diff --git a/portality/templates/editor/editor_base.html b/portality/templates/editor/editor_base.html index 1fc7bfbc81..c116f3d622 100644 --- a/portality/templates/editor/editor_base.html +++ b/portality/templates/editor/editor_base.html @@ -9,10 +9,15 @@ {% endblock %} {% block content %} -
- {% block editor_content %} - {% endblock %} -
+
+

Editor dashboard

+ {% include 'editor/nav.html' %} + +
+ {% block editor_content %} + {% endblock %} +
+
{% include "includes/_hotjar.html" %} {% endblock %} diff --git a/portality/templates/includes/contribution_rates.html b/portality/templates/includes/contribution_rates.html index 821fefede9..2449e24e3c 100644 --- a/portality/templates/includes/contribution_rates.html +++ b/portality/templates/includes/contribution_rates.html @@ -1,7 +1,7 @@