DHQ infrastructure meeting notes

DHQ infrastructure meetings

Meeting notes 2024-04-01

Attendees

Joel Lee
Ash Clark
John Walsh
Julia Flanders

Notes

Two workflows:
- Updating preview, internal site (fast churn)
  - If possible, we want this to be completely automated — press a button and update goes through to a separate server instance (?)
  - GitHub Action
- Publishing the DHQ site (slow, intentional)
  - Human can review the PR and its commits
  - When commits are added to the main branch, a GitHub Action can take care of compressing the site and putting it on the server
  - Makes process more transparent to folks outside the DHQ team
We may need to spend some $ on GitHub runner, but we should be able to have GitHub trigger the event and initiate automation on our server
We're leaning toward having editors work in another branch ("in-progress"?)
- They can continue working in the same branch together, as they have been on main
- Have a "PR Czar" editor who coordinates site publication by merging into main
Since we're going for automated workflows, we'll need TOC validation checks. John and Julia suggested Ash use Rebecca Koeser's PR for GH Action validations as a model
Still need to know what the static search will need in terms of processing — do indexing via GitHub action or on the server?
- Once we know more, we should loop in Simon for help on setting this up with GitHub Actions

Meeting notes 2024-02-12

Attendees

Syd Bauman
Julia Flanders
Joel Lee
Ash Clark
John Walsh

Notes

No progress on staticSearch, but progress on the Ant build that will enable it
John placed CSS and JS assets inline into HTML, and that appears to be working!
- The header is was display: none'd, due to the dhq-print.css stylesheet
  - John added a media query? It's fixed
- Some discussion of absolute vs relative links
  - It's non-trivial but not something we have to worry about until the static site is in use
Next meeting we'll start talking about workflows for using the site
We are cutting out the editorial/ directory from the static site
- Generating ZIPs can/will work
- New Ant target for building all previewable articles for authors
- Ash can cut out the @editorial = 'true' section of the TOC from the standard build

Meeting notes 2023-10-30

Attendees

Syd Bauman
Julia Flanders
Joel Lee
Ash Clark
Hoyeol Kim
John Walsh

Updates

This was kind of a catch-up meeting, esp. on where we stand with static site generation.
Syd continues to investigate UVic's static search.
John needs to implement relative URLs in existing DHQ->HTML XSLT code to support a working site on users' local machines. Currently hard-coded links requires site to be hosted on server for everything to work.
Joel showed a mock-up of keyword display (below abstract in article view).
We established a tentative timeline to wrap-up major tasks in static site generation by Dec. 31 2023 and to final testing and tweaks and roll out new site to public in Q1 2024.

Meeting notes 2023-09-18

Attendees

Syd Bauman
Julia Flanders
Joel Lee
John Walsh

Updates

generated a sorted list of recommended articles to get a reality check on which articles are being over/under-recommended
Syd showed us the static search generated index; we made some notes about which files should or should not be indexed (see below)
discussed details of Syd's well-formedness profiling

Site content that should be indexed:

article XML files
issue bios and indexes (with volume & issue metdata; with “article type” (probably renamed to “document type” or some such) of “author bios” and “index” or some such)
maybe site content?
whitelisted files (does not seem that staticSearch currently has this functionality)

Site content that should not be indexed:

contents of resource directories
maybe site content?
blacklisted files (does not seem that staticSearch currently has this functionality)

Well-formedness discoveries:

The University of Victoria Project Endings’ Static Search system (which is abbreviated “uvepss” in filenames our repo) seems to read all files that have “htm” in the name, and requires that each be well-formed XHTML5.
Just under a dozen .html files in our repo end in “.html” but are not well-formed XHTML5. (Listed in #collaberative-development Slack channel.)
Files are tested using Ant <xmlvalidate lenient="true">, and interestingly that finds a file that has whitespace before a DOCTYPE declaration (and no XML declaration) ill-formed. But it turns out we had 1 and only 1 such file, and do not need it anymore, so Syd deleted it (in the static_site_for_uvepss_generation branch).

Meeting notes 2023-08-07

Attendees

Syd Bauman
Ash Clark
Hoyeol Kim
Julia Flanders
John Walsh

Updates

Lunr requires indexed documents to be JSON objects. Seems good for metadata, not so good for long-form documents, like articles.
Ran through ant build.xml file and tested some targets. Worked well on John's machine.
Discussed problem with paths to assets and possible solutions. Preferred solution is to have relatives paths so site works on local machine or server.
John and Syd will look more closely at Victoria's search engine and report at next meeting.
Discussion (mainly Hoyeol and Julia) on automatic keyword extraction.
- Hoyeol will invite Julia and John to next meeting about automated keyword extraction.

Action items

new ant task zipGenerateSite that creates and zips static site. Existing generateSite task will create static site but not zip it.
John and Syd explore Victoria indexing at their 8/14 meeting
Hoyeol invite John and Julia to next keyword meeting
Add ant task `help.
move to relative paths in static site.

Meeting notes 2023-07-17

J and H met with Ben Lee and Benjamin Gray.
- Ben and Benjamin are going to create set of articles to test models on and give output to put on page as proof of concept.
- Setting up GitHub project for this work. Waiting on Ben to create repo.
Ash and Syd are nearing an end point? for static site work.
- outstanding issues:
  - needs testing on windows
  - test output
  - still missing editorial area
  - still needs documentation
  - it is a static site but not stand-alone because of path issues in XSLT. This can be addressed in future.
- we can all test by checking out branch and running Ant tasks.
  - Ash will add some brief testing documentation about Ant tasks to run and what each does. documentation will be in github issue.

Meeting notes 2023-06-26

Attendeees

Syd Bauman
Hoyeol Kim
Julia Flanders
Joel Lee
John Walsh

Notes

Syd reported on static site generation progress
Hoyeol and Joel reported on keyword extraction work. See https://docs.google.com/document/d/1yYJhWlFSbIFGEiN83yT0fZFBmpGUTaepobG8Kx6QXBY/edit?usp=sharing
Discussed need to evaluate search/indexing solutions for new satic site. Options to explore include:
- Solr (Joel will report on Solr)
- Lunr (John will do an in-depth look and report back to group.)
- projectEndings staticSearch generator (John and Syd will explore and report back.)
- elasticSearch (free version?)
Discussed keywords for articles
Evaluation criteria for search/indexing tools
- Will it work in a minimalist/static computing environment (e.g., without requiring something like Tomcat)?
- Does it work with our existing XML/XHTML structure, or do we need new code to create new structures?
- Does it have multilingual support?
- Is it free and open source?

Meeting notes 2023-04-24

Attendees:

Syd Bauman
Ash Clark
Julia Flanders
Joel Lee
John Walsh

Notes

Ash created a new branch (https://github.com/Digital-Humanities-Quarterly/dhq-journal/tree/static_site_generation) with Saxon jar, Saxon licenses, build.xml, build-properties.xml, etc.
The ant task previewArticle is working.
Ash will review where Ash was at end of Ash's recent development and reach out to rest of group for input and coding assistance.

Meeting notes 2023-04-03

Attendees:

Syd Bauman
Ash Clark
Julia Flanders
Hoyeol Kim
Joel Lee
John Walsh

Notes

Hoyeol, Joel, and Ben met recently to discuss UI enhancements and a feature/suggested/recommended articles UI element.
Joel created issue on enhancements to "Previous Issues" index.
Discussed keyword searching; could be implemented in new search/indexing tool for new infrastructure.
Confirmed our decision from last time to proceed with Ant for automating site build
Started specification to guide new infrastructure project.

Meeting notes 2023-01-23

Attendees:

Syd Bauman
Ash Clark
Julia Flanders
Hoyeol Kim
Joel Lee
John Walsh

Summary

First meeting of DHQ Collaborative Development + Ash and Syd. Discussed goals for new infrastructure and next steps.

Notes

desideratum: New infrastructure that generates a static site more or less identical to current dynamically generated Cocoon site. Must facilitate access to existing content at existing URLs.
desideratum: New search engine to replace Lucene and dependency on Java.

Actions

Ash, with assistance from John and Hoyeol, will start designing a meta document or configuration file that can control the process of static site generation. One way to think of this meta document is a union of the Cocoon sitemap.xmap and toc.xml files.

Next Meeting

review progress on actions (above)
start discussing pros and cons of different approaches to generating the static site (e.g., Makefile, Ant, Gulp, Gradle, shell script, or other automation system; XProc; some combination; other?)