docs: v3 self-host documentation (#1077)

---------

Co-authored-by: steffen911 <[email protected]>

components-mdx/architecture-description-v3.mdx

@@ -0,0 +1,15 @@
+Langfuse consists of two application containers, storage components, and an optional LLM API/Gateway.
+
+- [**Application Containers**](/self-hosting/infrastructure/containers)
+  - Langfuse Web: The main web application serving the Langfuse UI and APIs.
+  - Langfuse Worker: A worker that asynchronously processes events.
+- **Storage Components**:
+  - [Postgres](/self-hosting/infrastructure/postgres): The main database for transactional workloads.
+  - [Clickhouse](/self-hosting/infrastructure/clickhouse): High-performance OLAP database which stores traces, observations, and scores.
+  - [Redis/Valkey cache](/self-hosting/infrastructure/cache): A fast in-memory data structure store. Used for queue and cache operations.
+  - [S3/Blob Store](/self-hosting/infrastructure/blobstorage): Object storage to persist all incoming events, multi-modal inputs, and large exports.
+- [**LLM API / Gateway**](/self-hosting/infrastructure/llm-api): Some features depend on an external LLM API or gateway.
+
+Langfuse can be deployed within a VPC or on-premises in high-security environments.
+Internet access is optional.
+See [networking](/self-hosting/networking) documentation for more details.

components-mdx/architecture-diagram-v2.mdx

@@ -0,0 +1,13 @@
+```mermaid
+flowchart TB
+    User["User, API, SDKs"]
+    subgraph vpc["VPC"]
+        Web["Web Server (langfuse/langfuse)"]
+        Postgres[Postgres Database]
+    end
+    LLM["LLM API / Gateway (BYO, optional, same VPC or VPC peering)"]
+
+    User --> Web
+    Web --> Postgres
+    Web -.->|"optional for playground"| LLM
+```

components-mdx/architecture-diagram-v3.mdx

@@ -0,0 +1,26 @@
+```mermaid
+flowchart TB
+    User["User, API, SDKs"]
+    subgraph vpc["VPC"]
+        Web["Web Server (langfuse/langfuse)"]
+        Worker["Worker (langfuse/worker)"]
+        Postgres[Postgres Database]
+        Cache["Cache & Queue (Redis/Valkey)"]
+        Clickhouse["Clickhouse Database (observability data)"]
+        S3[S3/Blob Store]
+    end
+    LLM["LLM API / Gateway (BYO, optional, same VPC or VPC peering)"]
+
+    User --> Web
+    Web --> S3
+    Web --> Postgres
+    Web --> Cache
+    Web --> Clickhouse
+    Web -.->|"optional for playground"| LLM
+
+    Cache --> Worker
+    Worker --> Clickhouse
+    Worker --> Postgres
+    Worker --> S3
+    Worker -.->|"optional for evals"| LLM
+```

components-mdx/self-host-features.mdx

@@ -0,0 +1,42 @@
+import { Lock, Shield, Network, Users, Brush, Workflow } from "lucide-react";
+
+import { Cards } from "nextra/components";
+
+<Cards num={3}>
+  <Cards.Card
+    icon={<Lock size="24" />}
+    title="Authentication & SSO"
+    href="/self-hosting/authentication-and-sso"
+    arrow
+  />
+  <Cards.Card
+    icon={<Shield size="24" />}
+    title="Encryption"
+    href="/self-hosting/encryption"
+    arrow
+  />
+  <Cards.Card
+    icon={<Workflow size="24" />}
+    title="Headless Initialization"
+    href="/self-hosting/headless-initialization"
+    arrow
+  />
+  <Cards.Card
+    icon={<Network size="24" />}
+    title="Networking"
+    href="/self-hosting/networking"
+    arrow
+  />
+  <Cards.Card
+    icon={<Users size="24" />}
+    title="Organization Creators"
+    href="/self-hosting/organization-creators"
+    arrow
+  />
+  <Cards.Card
+    icon={<Brush size="24" />}
+    title="UI Customization"
+    href="/self-hosting/ui-customization"
+    arrow
+  />
+</Cards>

components-mdx/self-host-help-footer.mdx

@@ -0,0 +1,7 @@
+## Support
+
+If you experience any issues, please create an [issue on GitHub](/issues) or contact the maintainers ([support](/support)).
+
+For support with production deployments, the Langfuse team provides dedicated enterprise support. To learn more, reach out to [email protected] or [schedule a demo](/schedule-demo).
+
+Alternatively, you may consider using Langfuse Cloud, which is a fully managed version of Langfuse. You can find information about its security and privacy [here](/docs/data-security-privacy).

components/FooterMenu.tsx

@@ -125,7 +125,7 @@ const menuItems: {
       },
       {
         name: "Self-hosting",
-        href: "/docs/deployment/self-host",
+        href: "/self-hosting",
       },
       {
         name: "Open Source",

components/MainContentWrapper.tsx

@@ -1,5 +1,6 @@
 import { useRouter } from "next/router";
 import { useState } from "react";
+import { useConfig } from "nextra-theme-docs";
 import { Button } from "./ui/button";
 import {
   Calendar,
@@ -20,15 +21,25 @@ const pathsWithoutFooterWidgets = ["/imprint", "/blog"];
 
 export const MainContentWrapper = (props) => {
   const router = useRouter();
+  const { frontMatter } = useConfig();
   const cookbook = COOKBOOK_ROUTE_MAPPING.find(
     (cookbook) => cookbook.path === router.pathname
   );
 
+  const versionLabel = frontMatter.label;
+
   return (
     <>
       {cookbook ? (
         <NotebookBanner src={cookbook.ipynbPath} className="mb-4" />
       ) : null}
+      {versionLabel && (
+        <div>
+          <span className="inline-flex items-center rounded-md bg-secondary px-2 py-1 mt-5 text-xs font-medium text-secondary-foreground">
+            {versionLabel}
+          </span>
+        </div>
+      )}
       {props.children}
       {!pathsWithoutFooterWidgets.includes(router.pathname) ? (
         <div

components/MenuSwitcher.tsx

@@ -1,4 +1,4 @@
-import { CircleHelp } from "lucide-react";
+import { CircleHelp, Server } from "lucide-react";
 import { FileCode } from "lucide-react";
 import { LibraryBig } from "lucide-react";
 import Link from "next/link";
@@ -11,6 +11,7 @@ export const MenuSwitcher = () => {
     <div className="-mx-2 hidden md:block">
       {[
         { title: "Docs", path: "/docs", Icon: LibraryBig },
+        { title: "Self Hosting", path: "/self-hosting", Icon: Server },
         { title: "Guides", path: "/guides", Icon: FileCode },
         { title: "FAQ", path: "/faq", Icon: CircleHelp },
       ].map((item) =>

components/home/OpenSource.tsx

@@ -14,11 +14,11 @@ export default function OpenSource() {
         description={
           <span>
             Langfuse is committed to open source. You can also run it{" "}
-            <Link href="/docs/deployment/local" className="underline">
+            <Link href="/self-hosting/local" className="underline">
               locally
             </Link>{" "}
             or{" "}
-            <Link href="/docs/deployment/self-host" className="underline">
+            <Link href="/self-hosting" className="underline">
               self-hosted
             </Link>
             .

components/home/Pricing.tsx

@@ -98,7 +98,7 @@ const tiers = {
     {
       name: "Open Source",
       id: "tier-self-hosted-oss",
-      href: "/docs/deployment/self-host",
+      href: "/self-hosting",
       featured: true,
       description:
         "Self-host all core Langfuse features for free without any limitations.",
@@ -1167,7 +1167,7 @@ const faqs = [
   {
     question: "Can I self-host Langfuse?",
     answer:
-      "Yes, Langfuse is open source and you can run Langfuse <a class='underline' href='/docs/deployment/local'>locally using docker compose<a/> or for <a class='underline' href='/docs/deployment/self-host'>production use via docker<a/> and a standalone database.",
+      "Yes, Langfuse is open source and you can run Langfuse <a class='underline' href='/self-hosting/local'>locally using docker compose<a/> or for <a class='underline' href='/self-hosting'>production use via docker<a/> and a standalone database.",
   },
   {
     question: "Where is the data stored?",

cookbook/integration_amazon_bedrock.ipynb

@@ -331,7 +331,7 @@
         "## Additional Resources\n",
         "\n",
         "- Metadocs, [Monitoring your Langchain app's cost using Bedrock with Langfuse](https://www.metadocs.co/2024/07/03/monitor-your-langchain-app-cost-using-bedrock-with-langfuse/), featuring Langchain integration and custom model price definitions for Bedrock models.\n",
-        "- [Self-hosting guide](https://langfuse.com/docs/deployment/self-host) to deploy Langfuse on AWS."
+        "- [Self-hosting guide](https://langfuse.com/self-hosting) to deploy Langfuse on AWS."
       ]
     }
   ],

cookbook/integration_ollama.ipynb

@@ -31,7 +31,7 @@
     "\n",
     "### Local Deployment of Langfuse\n",
     "\n",
-    "Of course, you can also locally deploy Langfuse to run models and trace LLM outputs only on your own device. [Here](https://langfuse.com/docs/deployment/local) is a guide on how to run Langfuse on your local machine using Docker Compose. This method is ideal for testing Langfuse and troubleshooting integration issues.\n",
+    "Of course, you can also locally deploy Langfuse to run models and trace LLM outputs only on your own device. [Here](https://langfuse.com/self-hosting/local) is a guide on how to run Langfuse on your local machine using Docker Compose. This method is ideal for testing Langfuse and troubleshooting integration issues.\n",
     "\n",
     "For this example, we will use the Langfuse cloud version."
    ]

cookbook/python_decorators.ipynb

@@ -48,7 +48,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "If you haven't done so yet, [sign up to Langfuse](https://cloud.langfuse.com/auth/sign-up) and obtain your API keys from the project settings. You can also [self-host](https://langfuse.com/docs/deployment/self-host) Langfuse."
+        "If you haven't done so yet, [sign up to Langfuse](https://cloud.langfuse.com/auth/sign-up) and obtain your API keys from the project settings. You can also [self-host](https://langfuse.com/self-hosting) Langfuse."
       ]
     },
     {

next.config.mjs

@@ -225,6 +225,19 @@ const nonPermanentRedirects = [
   ["/changelog/2024-09-04-headless-initialization-or-self-hosted-deployments", "/changelog/2024-09-04-headless-initialization-of-self-hosted-deployments"],
   ["/docs/deployment/v3", "/docs/deployment/v3/overview"],
 
+  // new self-hosting section
+  ["/docs/deployment/feature-overview", "/self-hosting/license-key"],
+  ["/docs/deployment/local", "/self-hosting/local"],
+  ["/docs/deployment/self-host", "/self-hosting"],
+  ["/docs/deployment/v3/overview", "/self-hosting"],
+  ["/docs/deployment/v3/migrate-v2-to-v3", "/self-hosting/upgrade-guides/upgrade-v2-to-v3"],
+  ["/docs/deployment/v3/troubleshooting", "/self-hosting/troubleshooting"],
+  ["/docs/deployment/v3/guides/docker-compose", "/self-hosting/docker-compose"],
+  ["/docs/deployment/v3/guides/kubernetes-helm", "/self-hosting/kubernetes-helm"],
+  ["/docs/deployment/v3/components/clickhouse", "/self-hosting/infrastructure/clickhouse"],
+  ["/docs/deployment/v3/components/redis", "/self-hosting/infrastructure/cache"],
+  ["/docs/deployment/v3/components/blobstorage", "/self-hosting/infrastructure/blobstorage"],
+
   // Reorder Tracing section
   ["/docs/tracing/overview", "/docs/tracing"],
   ["/docs/tracing-features", "/docs/tracing"],

pages/_meta.tsx

@@ -24,6 +24,11 @@ export default {
     type: "page",
     title: "Docs",
   },
+  "self-hosting": {
+    type: "page",
+    title: "Self Hosting",
+    // hidden from main menu via overrides.css, nextra display:hidden otherwise breaks type:page
+  },
   guides: {
     type: "page",
     title: "Guides",

pages/blog/2024-11-17-launch-week-2.mdx

@@ -159,7 +159,7 @@ List of additional features that were released this week:
 - [`llms.txt`](/changelog/2024-11-17-llms-txt): Easily use the Langfuse documentation in Cursor and other LLM editors via the new `llms.txt` file.
 - [`/docs`](/docs): New documentation start page with a simplified overview of all Langfuse features.
 - [Self-hosted Pro Plan](/pricing-self-host): Get access to additional features without the need for a sales call or enterprise pricing. All core Langfuse features are OSS without limitations, see [comparison](/pricing-self-host) for more details.
-- [Developer Preview of v3 (self-hosted)](/docs/deployment/v3/overview): v3 is the biggest release in Langfuse history. After running large parts of it on Langfuse Cloud for a while, an initial developer preview for self-hosted users is now available.
+- [Developer Preview of v3 (self-hosted)](/self-hosting): v3 is the biggest release in Langfuse history. After running large parts of it on Langfuse Cloud for a while, an initial developer preview for self-hosted users is now available.
 
 ## Don't Miss Out
 

pages/blog/announcing-our-seed-round.mdx

@@ -108,7 +108,7 @@ After having built the data and observability layer we close the loop and return
 
 Based on the foundation of our existing observability stack, we are building out the leading open source LLM analytics platform. The job of our users is not easy as there is lots of short-term noise about frameworks, models, and emerging techniques for working with LLMs. These are exciting yet hard to navigate times. We aspire to guide developers with the right insights to make informed decisions grounded in real-world data, user feedback and evaluations.
 
-We are open, the project is super easy to [self-host](/docs/deployment/self-host), and you can try Langfuse Cloud without a credit card. Join us on [Discord](/discord), give Langfuse a spin ([sign-up](https://cloud.langfuse.com)), or try the [live demo](/demo).
+We are open, the project is super easy to [self-host](/self-hosting), and you can try Langfuse Cloud without a credit card. Join us on [Discord](/discord), give Langfuse a spin ([sign-up](https://cloud.langfuse.com)), or try the [live demo](/demo).
 
 Thank you all!<br/>Clemens, Max and Marc
 

pages/blog/update-2023-08.mdx

@@ -233,7 +233,7 @@ To reduce the friction of self-hosting, we now publish Docker images for the Lan
 docker pull ghcr.io/langfuse/langfuse
 ```
 
-For detailed instructions, see [self-hosting](/docs/deployment/self-host) and [local setup](/docs/deployment/local) documentation.
+For detailed instructions, see [self-hosting](/self-hosting) and [local setup](/self-hosting/local) documentation.
 
 ## 🚢 What's Next?
 

pages/blog/update-2023-10.mdx

@@ -114,7 +114,7 @@ We've integrated the application of migrations to the startup process of the Doc
 docker pull ghcr.io/langfuse/langfuse:latest
 ```
 
-Check [self-hosting docs](/docs/deployment/self-host) for full details on how to deploy Langfuse in a production environment.
+Check [self-hosting docs](/self-hosting) for full details on how to deploy Langfuse in a production environment.
 
 Optionally, you can also one-click deploy Langfuse to Railway:
 
@@ -175,7 +175,7 @@ _Use the `j` and `k` keys to navigate even faster._
 
 ### Sign in via Google and GitHub
 
-You can now authenticate using **Google** and **GitHub**. Also supported for self-hosted deployments. Check [docs](/docs/deployment/self-host) for details.
+You can now authenticate using **Google** and **GitHub**. Also supported for self-hosted deployments. Check [docs](/self-hosting) for details.
 
 ### Delete projects
 

pages/changelog/2023-08-16-prebuilt-docker-image.mdx

@@ -16,5 +16,5 @@ docker pull ghcr.io/langfuse/langfuse:latest
 
 See docs:
 
-- [Run Langfuse locally](/docs/deployment/local) using `docker compose`
-- [Self-host Langfuse](/docs/deployment/self-host)
+- [Run Langfuse locally](/self-hosting/local) using `docker compose`
+- [Self-host Langfuse](/self-hosting)

pages/changelog/2023-10-31-simplified-self-hosting.mdx

@@ -13,7 +13,7 @@ import { ChangelogHeader } from "@/components/changelog/ChangelogHeader";
 docker pull ghcr.io/langfuse/langfuse:latest
 ```
 
-See [self-hosting documentation](/docs/deployment/self-host) for details when deploying Langfuse for the first time.
+See [self-hosting documentation](/self-hosting) for details when deploying Langfuse for the first time.
 
 Alternatively, deploy to Railway using this template:
 

pages/changelog/2023-11-03-sso-enforcement.mdx

@@ -11,4 +11,4 @@ import { ChangelogHeader } from "@/components/changelog/ChangelogHeader";
 
 When self-hosting, you can also enforce SSO by adding your domain as an environment variable: `AUTH_DOMAINS_WITH_SSO_ENFORCEMENT`.
 
-You can find up-to-date information on configuring SSO and environment variables in our [documentation](/docs/deployment/self-host).
+You can find up-to-date information on configuring SSO and environment variables in our [documentation](/self-hosting/authentication-and-sso).

pages/changelog/2024-01-29-custom-model-prices.mdx

@@ -77,7 +77,7 @@ We updated the API and UI to include usage and cost on most endpoints. Let us kn
 
 [Langfuse Cloud](/docs/deployment/cloud): No action is required.
 
-If you [self-host](/docs/deployment/self-host) Langfuse, new model/price definitions are applied to all newly ingested traces. You need to run a [migration script](/docs/deployment/self-host#migrate-models) to apply them to your existing data in Langfuse.
+If you [self-host](/self-hosting) Langfuse, new model/price definitions are applied to all newly ingested traces. You need to run a [migration script](/self-hosting/upgrade-guides/upgrade-v1-to-v2) to apply them to your existing data in Langfuse.
 
 ### More details
 

pages/changelog/2024-04-13-images-on-docker-hub.mdx

@@ -9,7 +9,7 @@ import { ChangelogHeader } from "@/components/changelog/ChangelogHeader";
 
 <ChangelogHeader />
 
-This change made deploying Langfuse easier across various platforms that do not natively support the GitHub Container Registry (e.g., Google Cloud Run). For more details, see the updated [self-hosting instructions](/docs/deployment/self-host).
+This change made deploying Langfuse easier across various platforms that do not natively support the GitHub Container Registry (e.g., Google Cloud Run). For more details, see the updated [self-hosting instructions](/self-hosting).
 
 Links:
 

pages/changelog/2024-08-14-ui-customization.mdx

@@ -20,4 +20,4 @@ This feature is available in the [Enterprise Edition](/docs/open-source) (self-h
 | `LANGFUSE_UI_SUPPORT_HREF`       | Customize the support link reference in the menu and settings.                          |
 | `LANGFUSE_UI_FEEDBACK_HREF`      | Replace the default feedback widget with your own feedback link.                        |
 
-_Source: [self-hosting documentation](/docs/self-host)_
+_Source: [self-hosting documentation](/self-hosting/ui-customization)_

pages/changelog/2024-08-15-deployment-as-porter-add-on.mdx

@@ -10,6 +10,6 @@ import { ChangelogHeader } from "@/components/changelog/ChangelogHeader";
 
 <ChangelogHeader />
 
-The add-on automatically configures the necessary [environment variables](/docs/deployment/self-host#configuring-environment-variables), handles HTTPS and custom domains, autoscales the application, and allows you to connect your Porter-managed Postgres database to Langfuse by injecting the credentials.
+The add-on automatically configures the necessary [environment variables](/self-hosting/configuration), handles HTTPS and custom domains, autoscales the application, and allows you to connect your Porter-managed Postgres database to Langfuse by injecting the credentials.
 
 Learn more about [Porter](https://porter.run) and [Porter Add-ons](https://www.porter.run/addons).

pages/changelog/2024-09-04-headless-initialization-of-self-hosted-deployments.mdx

@@ -34,6 +34,6 @@ Organization
 
 ## Get Started
 
-Checkout the [self-hosting documentation](/docs/deployment/self-host#initialization) for a full list of available environment variables.
+Checkout the [self-hosting documentation](/self-hosting/headless-initialization) for a full list of available environment variables.
 
 We're excited to see how this new feature will simplify your Langfuse deployments. As always, we welcome your feedback and suggestions for further improvements!

pages/changelog/2024-09-13-custom-basepath-for-self-hosted-deployments.mdx

@@ -16,4 +16,4 @@ To use a custom base path:
 1. Set the `NEXT_PUBLIC_BASE_PATH` environment variable at build time.
 2. Build the Docker image. As this base path is inlined in static assets, you cannot use the prebuilt docker images.
 
-See the [self-hosting documentation](/docs/deployment/self-host#custom-base-path) for more details. Thanks to everyone who contributed to the [discussion](https://github.com/orgs/langfuse/discussions/2400) on this!
+See the [self-hosting documentation](/self-hosting/custom-base-path) for more details. Thanks to everyone who contributed to the [discussion](https://github.com/orgs/langfuse/discussions/2400) on this!

pages/changelog/2024-12-03-new-sso-providers-github-enterprise-and-keycloak.mdx

@@ -30,4 +30,4 @@ If you need a specific additional identity provider, please let us know by [crea
 
 On Langfuse Cloud, please reach out to [support](/support) to configure a custom SSO identity provider for your domain.
 
-When self-hosting Langfuse, see our [self-hosting documentation](/docs/deployment/self-host#sso) for setup instructions.
+When self-hosting Langfuse, see our [self-hosting documentation](/self-hosting/authentication-and-sso) for setup instructions.