Migration website from gatsby to docusaurus with all pages and configs (

#121) * New pages migrated from gatsby. All configs Signed-off-by: tito1212 <[email protected]> * Move spec and apidocs from gatsby website. Update readme. Update deploy and deploy-test config with building docs. Move function to utils Signed-off-by: tito1212 <[email protected]> * Remove integration page. Update files from website commit 3d09dc94529de93f8bab6c519f74f00882608892 Signed-off-by: tito1212 <[email protected]> * Changes after review. Footer, background body, links style, ecosystem table style page Signed-off-by: tito1212 <[email protected]> * Blog page with custom style grid items Signed-off-by: tito1212 <[email protected]> * MERGE COMMIT update meetup room Signed-off-by: tito1212 <[email protected]> * MERGE COMMIT openlineage javadoc update Signed-off-by: tito1212 <[email protected]> * Footer for blog page Signed-off-by: tito1212 <[email protected]> * Fix links in build for integration and apidocs Signed-off-by: tito1212 <[email protected]> --------- Signed-off-by: tito1212 <[email protected]> Co-authored-by: tito1212 <[email protected]>
OpenLineage · Mar 24, 2023 · 7f3fc92 · 7f3fc92
1 parent f3c5eff
commit 7f3fc92
Show file tree

Hide file tree

Showing 372 changed files with 134,590 additions and 6,291 deletions.
diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml
@@ -17,6 +17,8 @@ jobs:
           cache: yarn
       - name: Install dependencies
         run: yarn install --frozen-lockfile
+      - name: Build openapi docs
+        run: yarn run build:docs
       - name: Build website
         run: yarn build
       - name: Deploy to GitHub Pages

diff --git a/.github/workflows/test-deploy.yml b/.github/workflows/test-deploy.yml
@@ -17,6 +17,8 @@ jobs:
           cache: yarn
       - name: Install dependencies
         run: yarn install --frozen-lockfile
+      - name: Test build openapi docs
+        run: yarn run build:docs
       - name: Test build website
         run: yarn build
       - uses: actions/upload-artifact@v3

diff --git a/README.md b/README.md
@@ -4,6 +4,45 @@ NOTE: These docs are a work in progress. For current documentation, consult the
 
 This is a Docusaurus site, and all content can be found in `docs/`. Contributions are welcome in the form of issues or pull requests. Pages that require attention have been marked with Docusaurus Admonitions.
 
+### New posts
+
+We love new blog posts, and welcome content about OpenLineage! Topics include:
+* experiences from users of all kinds
+* supporting products and technologies
+* proposals for discussion
+
+If you are familiar with the GitHub pull request process, it is easy to propose a new blog post:
+
+1. Fork this project.
+2. Make a new directory in `/blog`. The name of the directory will become part of the posts's URL, so choose something descriptive and unique.
+2. Create an `index.mdx` file in the new directory containing your blog content. Use one of the other posts as a template. The `title`, `date`, `author`, `image`, `banner`, and `description` front matter fields are all required.
+3. Create artwork! The `banner.svg` image will be shown at the top of the blog post, and should have dimensions of 670x160. The `image.svg` will be shown in the blog index, and should have dimensions of 670x480. These should both be SVG files.
+4. Build the site locally if you want to see it in a browser and build confidence in your formatting choices.
+5. Commit your changes and submit a pull request. 
+
+To match the artwork on the rest of the site, you can use sketchbeast.com to create your images:
+
+1. First, choose a source image to use as inspiration - perhaps something thematically related to your post, or an inside joke. It'll be our little secret.
+2. Then, upload the image to sketchbeast.com, change "mixed shapes" to "rectangles", and click the button. Feel free to play with the other settings as you like, but stick with rectangles.
+3. Download the resulting SVG using the button on the image in the film strip at the bottom of the screen, and crop to the proper size.
+4. Delight in being the only person who knows what the source image was; resolve to choose something more subversive/clever next time.
+
+(If you'd rather not go through all of this, feel free to submit your PR without artwork and we'll make something for you - we like doing it, it's fun!) 
+
+### Changes to basepages
+
+If you want to make a change to a basepage - e.g. to add a new member to the Ecosystem page - the best way is to submit a pull request.
+
+These basepages can be found in `src/pages`, and are formatted in markdown.
+
+### Building openapi docs
+
+To build the openapi docs using `redoc-cli`, run:
+
+```
+% yarn run build:docs
+```
+
 ## Local development
 
 First, clone the repo.
@@ -15,12 +54,6 @@ $ nvm install 16
 $ nvm use 16
 ```
 
-Install Yarn and the Gatsby CLI:
-
-```
-$ npm install -g yarn gatsby-cli
-```
-
 Run Yarn to install all of the Node dependencies for the project:
 
 ```
@@ -56,3 +89,10 @@ $ yarn start --port 3001
 ## Deployment
 
 Once the site has been launched, pull requests to `main` will cause a new doc site to be shipped via GitHub Pages.
+
+The site is deployed using the [Gatsby Publish GitHub action](https://github.com/OpenLineage/docs/blob/main/.github/workflows/deploy.yml) whenever a change is merged into `main`. 
+
+This GitHub Action will:
+* Execute `scripts/build-docs.sh`, which performs a build of the OpenAPI docs based on the latest version of the spec that has been published into `static/spec` by the [OpenLineage release script](https://github.com/OpenLineage/OpenLineage/blob/main/spec/release.sh). The resulting docs are placed into `static/apidocs/openapi`.
+* Execute `yarn run build`, which performs a build of the Gatsby landing pages and places them into `public/`. The `static/` directory, containing the OpenAPI and Java client documentation, is copied into `public/` during this step.
+* Replace the contents of the `gh-pages` branch of the [org domain repo](https://github.com/OpenLineage/OpenLineage.github.io) with the contents of `public/`. This will cause that repo's GitHub Action to deploy the new content.
diff --git a/blog/0.1-release/banner.svg b/blog/0.1-release/banner.svg
diff --git a/blog/0.1-release/image.svg b/blog/0.1-release/image.svg
diff --git a/blog/0.1-release/index.mdx b/blog/0.1-release/index.mdx
@@ -0,0 +1,37 @@
+---
+title: Introducing OpenLineage 0.1.0
+date: 2021-09-03
+author: Julien Le Dem
+image: ./image.svg
+banner: ./banner.svg
+description: We are pleased to announce the initial release of OpenLineage. This release includes the core specification, data model, clients, and integrations with common data tools.
+---
+We are pleased to announce the initial release of OpenLineage. This release includes the core specification, data model, clients, and integrations with common data tools.
+
+![](./banner.svg#full-width)
+
+<!--truncate-->
+
+We are pleased to announce the initial release of OpenLineage. This is the culmination of a broad community effort, and establishes a common framework for data lineage collection and analysis.
+
+We want to thank [all the contributors](https://github.com/OpenLineage/OpenLineage/graphs/contributors) as well all the projects and companies involved in the design (in alphabetical order): [Airflow](https://airflow.apache.org), [Astronomer](https://www.astronomer.io), [Datakin](https://datakin.com), [Data Mesh](https://datameshlearning.com), [dbt](https://www.getdbt.com), [Egeria](https://egeria.odpi.org), [GetInData](https://getindata.com), [Great Expectations](https://greatexpectations.io), [Iceberg](https://iceberg.apache.org) (and others that I am probably forgetting).
+
+This release includes:
+* The initial 1-0-0 release of the [OpenLineage specification](https://github.com/OpenLineage/OpenLineage/blob/main/spec/OpenLineage.md)
+* A core lineage model of Jobs, Runs and Datasets
+  * Core facets
+  * Data Quality Metrics and statistics
+  * Dataset schema
+  * Source code location
+  * SQL
+* Clients that send OpenLineage events to an HTTP backend
+  * Java
+  * Python
+* [Integrations](https://github.com/OpenLineage/OpenLineage/tree/main/integration) that collect lineage metadata as OpenLineage events
+  * Apache Airflow with support for BigQuery, Great Expectations, Postgres, Redshift, Snowflake
+  * Apache Spark
+  * dbt
+
+This is only the beginning. We invite everyone interested to [consult and contribute to the roadmap](https://github.com/OpenLineage/OpenLineage/projects). The roadmap currently contains, among other things: adding support for [Kafka](https://github.com/OpenLineage/OpenLineage/issues/152), [BI dashboards](https://github.com/OpenLineage/OpenLineage/issues/207), and [column level lineage](https://github.com/OpenLineage/OpenLineage/issues/148)...but you can influence it by participating!
+
+Follow the [repo](https://github.com/OpenLineage/OpenLineage) to stay updated. And, as always, you can [join the conversation](http://bit.ly/OpenLineageSlack) on Slack.
diff --git a/blog/backfilling-airflow-dags-using-marquez/backfill.png b/blog/backfilling-airflow-dags-using-marquez/backfill.png