Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenShift prep work #10314

Merged
merged 6 commits into from
Feb 23, 2024
Merged

OpenShift prep work #10314

merged 6 commits into from
Feb 23, 2024

Conversation

poikilotherm
Copy link
Contributor

@poikilotherm poikilotherm commented Feb 9, 2024

Please note:

We have narrowed the scope of this pull request. Its focus now is prep work for our images to run under OpenShift as any arbitrary user id. Plus some clean up.

We have reverted the commit (8547dbf) that added some Kubernetes config files (and added jkube as a dependency). We will pick up this work in the next issue:


What this PR does / why we need it:

For MOC we wish to support running a proof of concept on OpenShift using our shiny containers.

This is also a testbed / preparation to enable folks to run Dataverse in production on more or less any Kubernetes distribution.

Which issue(s) this PR closes:

Special notes for your reviewer:
This is in its early stages. More to be done and discuss. Basically here to ease pushing new images with a tag to GHCR.

Suggestions on how to test this:

  1. Install CRC
  2. Setup crc as explained in their docs.
  3. Start a OC cluster with more than default resources: crc start -c 8 -d 64 -m 16384 (8 CPU, 64Gi HDD, 16Gi RAM)
  4. Generate the manifests and deploy away:
cd modules/container-k8s
mvn -Pct k8s:resource k8s:deploy
  1. Watch the pods, watch the logs.
  2. You can access the application via oc port-forward pods/dataverse-X-X-X 8080 and then going to http://localhost:8080 with your browser.

Does this PR introduce a user interface change? If mockups are available, please link/include them here:
None

Is there a release notes update needed for this change?:
Maybe later

Additional documentation:
None (yet)

To enable the user with a random, arbitrary UID to write into the overlay filesystem, we need to set proper file permissions.
This should not affect users on Docker or other K8s distributions, as the security is more lenient there.

It is not ideal to write into overlayfs, as it impacts performance and may lead to unintended side effects. This is a workaround to at least get going.

 See https://docs.openshift.com/container-platform/4.14/openshift_images/create-images.html#use-uid_create-images for a detailed reference
By defining pre- and postboot file locations within the Dockerfile, it wasn't able to change the location by changing CONFIG_DIR env var.

This is fixed now, allowing simpler backing of the dir location with an (ephemeral) volume.
The location where to create the temporary file was wrong, fixed now.
@poikilotherm poikilotherm added the Component: Containers Anything related to cloudy Dataverse, shipped in containers. label Feb 9, 2024
@poikilotherm poikilotherm self-assigned this Feb 9, 2024
@poikilotherm poikilotherm changed the title Openshift Proof of Concept OpenShift Proof of Concept Feb 9, 2024
Copy link

github-actions bot commented Feb 9, 2024

📦 Pushed preview images as

ghcr.io/gdcc/dataverse:openshift-poc
ghcr.io/gdcc/configbaker:openshift-poc

🚢 See on GHCR. Use by referencing with full name as printed above, mind the registry name.

@pdurbin
Copy link
Member

pdurbin commented Feb 13, 2024

As I've mentioned in Zulip and Slack, I'm having a little trouble with this PR. I'm getting Directory /dv/docroot for docroot space (see dataverse.files.docroot) exists, but is not writeable. I'm not sure if this difference is because I'm on a Mac (I tried Intel and M2) whereas Oliver is on Linux. He's quite busy this week but I hope to pick his brain soon.

2024-02-15 update: @DanNiESh suggested the following change to modules/container-k8s/src/main/jkube/dataverse-deployment.yaml and it's helping me make progress:

   replicas: 1
   template:
     spec:
+      securityContext:
+        runAsUser: 1000
+        fsGroup: 1000
       containers:
         - name: dataverse
           image: ghcr.io/gdcc/dataverse:openshift-poc

The root dataverse was created:

$ kubectl exec --stdin --tty dataverse-5f6bb9f895-lrfvb -- bash
Defaulted container "dataverse" out of: dataverse, bootstrap
payara@dataverse-5f6bb9f895-lrfvb:~$ curl -s localhost:8080/api/dataverses/:root | jq .
{
  "status": "OK",
  "data": {
    "id": 1,
    "alias": "root",
    "name": "Root",
    "dataverseContacts": [
      {
        "displayOrder": 0,
        "contactEmail": "[email protected]"
      }
    ],
    "permissionRoot": true,
    "description": "The root dataverse.",
    "dataverseType": "UNCATEGORIZED",
    "creationDate": "2024-02-15T14:29:34Z"
  }
}

Once I forward the ports...

oc port-forward pods/dataverse-5f6bb9f895-lrfvb 8080

... I can get at the Dataverse web interface:

Screenshot 2024-02-15 at 2 45 45 PM

There's still work to do with Solr, obviously, but this is great progress!

Rather than spam this PR too much, I'm making noise in Zulip: https://dataverse.zulipchat.com/#narrow/stream/375812-containers/topic/OpenShift

Also, we had a great conversation about all this in the container meeting today: recording, notes.

@pdurbin pdurbin added Status: Needs Input Applied to issues in need of input from someone currently unavailable Size: 10 A percentage of a sprint. 7 hours. and removed Status: Needs Input Applied to issues in need of input from someone currently unavailable labels Feb 13, 2024
@pdurbin pdurbin changed the title OpenShift Proof of Concept OpenShift prep work Feb 23, 2024
@pdurbin pdurbin marked this pull request as ready for review February 23, 2024 20:48
Copy link
Member

@pdurbin pdurbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Will merge. Containers still run just fine with this prep work for OpenShift.

I just left this note at the top of the description to avoid confusion about what is being merged in this pull request:


We have narrowed the scope of this pull request. Its focus now is prep work for our images to run under OpenShift as any arbitrary user id. Plus some clean up.

We have reverted the commit (8547dbf) that added some Kubernetes config files (and added jkube as a dependency). We will pick up this work in the next issue:

@pdurbin pdurbin merged commit dc597ae into develop Feb 23, 2024
13 of 14 checks passed
@pdurbin pdurbin deleted the openshift-poc branch February 23, 2024 20:56
@pdurbin pdurbin added this to the 6.2 milestone Feb 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Containers Anything related to cloudy Dataverse, shipped in containers. Size: 10 A percentage of a sprint. 7 hours.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants