Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Xpublish topic pages #57

Merged
merged 3 commits into from
May 15, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions _topics/2024-05-01-06-xpublish.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
number: 6
title: XPublish Enhancements
pis:
- Jonathan Joyce (@jonmjoyce)
- Matt Iannucci (@mpiannucci)
contributors:
- TBD
github: ioos/ioos-code-sprint/issues/42
slack:
- python
breakout:
- General Python
year:
- 2024
---

This topic will focus on improving several aspects of the [XPublish](https://github.com/xpublish-community/xpublish) project. Pull requests will be submitted to the XPublish project and/or one of its relevant plugin repos.

Initially we will discuss the long-term vision for the project to help determine the milestones and architecture. From there we can split into subgroups to tackle specific problems. The group is not necessarily limited to addressing these goals, and other XPublish-related work is welcome.

**Goal 1: Platform Standardization Improvements**

Create a standardized method for deploying and hosting XPublish, including documentation.

- Produce a basic Dockerfile for running XPublish
- Write supporting documentation for running on different platforms
- Investigate and address additional considerations for running XPublish (logging, scaling, loading data, etc.)
- Improve configuration management

**Goal 2: Benchmark and Enhance XPublish Performance**

Address underlying XPublish architecture for large datasets, ensuring that data delivery can scale and maintain high performance.

- Create and document methods for measuring XPublish performance
- Identify bottlenecks and prototype potential solutions

**Expected Outcomes:**

The goal of this working group is to introduce new users to the XPublish ecosystem, improve the baseline deployment and documentation, and support new innovations to help with additional features and workloads.

**Skills required:**

- Python
- Basic understanding of Pangeo stack (xarray, dask, zarr)
- Familiarity with web service API concepts

**Difficulty:**

All experience levels will be able to contribute. We will have novice and advanced topics to work on.

**Relevant links:**

XPublish GitHub: https://github.com/xpublish-community/xpublish
XPublish Experiments: https://github.com/xpublish-experiments/
Existing XREDS Xpublish host: https://github.com/asascience-open/xreds
Live XPublish Prototype: https://nextgen-dev.ioos.us/xreds/

**Functioning Prototype**

**Workflow**