Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build a fixture container of Figgy data for indexing. #39

Merged
merged 5 commits into from
Jul 19, 2024

Conversation

tpendragon
Copy link
Contributor

@tpendragon tpendragon commented Jul 15, 2024

The script build-and-push.sh in figgy-fixture-container will create a postgres 15 container with all the records for a single Ephemera Project.

This also sets up Lando to use that container and host it on port 5435.

Work towards #30
closes #42

We should be able to import this into a local container during a docker
build by running COPY FROM.
@tpendragon tpendragon changed the title Create a script to export data for a project. Build a fixture container of Figgy data for indexing. Jul 15, 2024
@tpendragon tpendragon marked this pull request as ready for review July 15, 2024 21:24
Copy link
Member

@hackartisan hackartisan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great. I think we will need a few more things but I'm open to adding those in a later PR:

  • deletion markers, ideally relevant to the project in question
  • some stuff we know we never want to get, like Events
  • some stuff we will/may get in the future but aren't doing in this phase, like some scanned resources, non-public material, ephemera that's not an image

I had one question in line.

read USERNAME
lpass login $USERNAME
export CR_PAT=$(lpass show "Shared-ITIMS-Passwords/dpul_collections_fixture_container_github_token" --notes)
echo $CR_PAT | docker login ghcr.io -u pulbot --password-stdin
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does CR_PAT mean?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

container password token?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No clue, I copied this from a different script we have in figgy. 🤷

This adds three DeletionMarkers for folders that are a part of the box
that's a member of that project, 10 events, and 10 scanned resources.
@tpendragon
Copy link
Contributor Author

@hackartisan @eliotjordan This should be ready now.

@tpendragon
Copy link
Contributor Author

127.0.0.1 postgres@postgres=# select COUNT(*), internal_resource from orm_resources GROUP BY internal_resource;
 count | internal_resource
-------+--------------------
     1 | EphemeraBox
     5 | EphemeraField
   469 | FileSet
    10 | Event
    86 | EphemeraFolder
     1 | EphemeraProject
    43 | EphemeraVocabulary
    10 | ScannedResource
  1559 | EphemeraTerm
     3 | DeletionMarker
(10 rows)

Time: 6.880 ms

Copy link
Member

@hackartisan hackartisan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@hackartisan hackartisan merged commit 9c8af1a into main Jul 19, 2024
1 check passed
@hackartisan hackartisan deleted the 30-fixture-container branch July 19, 2024 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create a fixture database container that has Figgy resources in it we can test against
3 participants