Skip to content

Latest commit

 

History

History
565 lines (492 loc) · 14.3 KB

01-ingress-passthrough-tls-with-snihosts.md

File metadata and controls

565 lines (492 loc) · 14.3 KB
Raw

Istio数据面配置解析01：在Ingress Gateway中对基于sniHosts的TLS请求进行路由

[TOC]

概述

本文介绍了在Istio中直接转发TLS请求的场景：外部服务通过Igress Gateway访问Mesh内部的TLS服务。

TLS类型的路由属于TCP，所以无Envoy route相关配置，路由均在listener中进行。

本文也为了对TCP类型的路由在Envoy中的配置进行一些解析。

相关拓扑

01-ingress-passthrough-tls-with-snihosts-1

  • 外部客户端发送相关tls请求。
  • 请求被发送至lb，并由lb转发至ingressgateway pod。
  • ingressgateway中的istio-proxy container根据相关snihosts路由规则将请求转发至相关nginx pod。
  • 请求被nginx pod中的istio-proxy container截获，并转发至nginx container。

01-ingress-passthrough-tls-with-snihosts-2

  • 因为tls类型的路由属于tcp，所以无envoy route相关配置，路由均在listener中进行。
  • 2个nginx pod，label为别为version: v1和version: v2。
  • 使用istio destinationrule为不同的label定义不同的subset。
  • 使用istio gateway定义ingressgateway接收443端口的请求。
  • 将ingressgateway的443端口的tls策略设置为passthrough。
  • 在ingressgateway中使用istio virtualservice定义到两个节点的基于tls的envoy route。
  • 指定sni名称为*.v1.nginx.internal.istio的请求被路由到subset v1。
  • 指定sni名称为*.v2.nginx.internal.istio的请求被路由到subset v2。

相关配置

DestinationRule

openssl req \
-newkey rsa:4096 -nodes -sha256 -keyout 1st.nginx.key \
-out 1st.nginx.csr

echo subjectAltName = DNS:www.1st.nginx > 1st.nginx.cnf

openssl x509 \
-req -days 3655 -in 1st.nginx.csr -CA ca.crt -CAkey ca.key \
-CAcreateserial -extfile 1st.nginx.cnf -out 1st.nginx.crt

openssl req \
-newkey rsa:4096 -nodes -sha256 -keyout 2nd.nginx.key \
-out 2nd.nginx.csr

echo subjectAltName = DNS:www.2nd.nginx > 2nd.nginx.cnf

openssl x509 \
-req -days 3655 -in 2nd.nginx.csr -CA ca.crt -CAkey ca.key \
-CAcreateserial -extfile 2nd.nginx.cnf -out 2nd.nginx.crt

kubectl create secret tls sec-nginx-v1 --key ./1st.nginx.key --cert ./1st.nginx.crt
kubectl create secret tls sec-nginx-v2 --key ./2nd.nginx.key --cert ./2nd.nginx.crt
  • 为nginx-v1 pod配置证书和私钥，并将证书和私钥生成k8s secret sec-nginx-v1。
  • 为nginx-v2 pod配置证书和私钥，并将证书和私钥生成k8s secret sec-nginx-v2。
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-v1
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
        version: v1
    spec:
      containers:
      - name: nginx
        image: 192.168.0.61/istio-example/nginx
        ports:
        - containerPort: 443
        volumeMounts:
        - mountPath: /etc/nginx/conf.d/
          readOnly: true
          name: conf
        - mountPath: /etc/nginx/html/
          readOnly: true
          name: index
        - mountPath: /etc/nginx/certs
          readOnly: true
          name: certs
      volumes:
      - name: conf
        configMap:
          name: cm-nginx-v1
          items:
            - key: default.conf
              path: default.conf
      - name: index
        configMap:
          name: cm-nginx-v1
          items:
            - key: index.html
              path: index.html
      - name: certs
        secret:
          secretName: sec-nginx-v1
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: cm-nginx-v1
data:
  default.conf: |
    server {
      listen       443 ssl;
      server_name  loalhost;

      location / {
        root   /etc/nginx/html/;
        index  index.html index.htm;
      }

      ssl_certificate      /etc/nginx/certs/tls.crt;
      ssl_certificate_key  /etc/nginx/certs/tls.key;
      ssl_verify_depth     2;

      error_page   500 502 503 504  /50x.html;
      location = /50x.html {
        root   /usr/share/nginx/html;
      }
    }
  index.html: |
    <!DOCTYPE html>
    <html>
    <head>
    <title>Welcome to nginx!</title>
    <style>
      body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
      }
    </style>
    </head>
    <body>
    <h1>Welcome to nginx!</h1>
    <h1>nginx01</h1>
    <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p>
    <p>For online documentation and support please refer to
    <a href="http://nginx.org/">nginx.org</a>.<br/>
    Commercial support is available at
    <a href="http://nginx.com/">nginx.com</a>.</p>
    <p><em>Thank you for using nginx.</em></p>
    </body>
    </html>
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-v2
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
        version: v2
    spec:
      containers:
      - name: nginx
        image: 192.168.0.61/istio-example/nginx
        ports:
        - containerPort: 443
        volumeMounts:
        - mountPath: /etc/nginx/conf.d/
          readOnly: true
          name: conf
        - mountPath: /etc/nginx/html/
          readOnly: true
          name: index
        - mountPath: /etc/nginx/certs
          readOnly: true
          name: certs
      volumes:
      - name: conf
        configMap:
          name: cm-nginx-v2
          items:
            - key: default.conf
              path: default.conf
      - name: index
        configMap:
          name: cm-nginx-v2
          items:
            - key: index.html
              path: index.html
      - name: certs
        secret:
          secretName: sec-nginx-v2
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: cm-nginx-v2
data:
  default.conf: |
    server {
      listen       443 ssl;
      server_name  loalhost;

      location / {
        root   /etc/nginx/html/;
        index  index.html index.htm;
      }

      ssl_certificate      /etc/nginx/certs/tls.crt;
      ssl_certificate_key  /etc/nginx/certs/tls.key;
      ssl_verify_depth     2;

      error_page   500 502 503 504  /50x.html;
      location = /50x.html {
        root   /usr/share/nginx/html;
      }
    }
  index.html: |
    <!DOCTYPE html>
    <html>
    <head>
    <title>Welcome to nginx!</title>
    <style>
      body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
      }
    </style>
    </head>
    <body>
    <h1>Welcome to nginx!</h1>
    <h1>nginx02</h1>
    <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p>
    <p>For online documentation and support please refer to
    <a href="http://nginx.org/">nginx.org</a>.<br/>
    Commercial support is available at
    <a href="http://nginx.com/">nginx.com</a>.</p>
    <p><em>Thank you for using nginx.</em></p>
    </body>
    </html>
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  type: ClusterIP
  ports:
  - port: 443
    name: https
  selector:
    app: nginx
  • 使用configmap cm-nginx-v1定制nginx-v1的配置文件和测试页面。
  • 使用configmap cm-nginx-v2定制nginx-v2的配置文件和测试页面。
  • 配置nginx-v1 deployment，挂载configmap cm-nginx-v1和secret sec-nginx-v1。
  • 配置nginx-v2 deployment，挂载configmap cm-nginx-v1和secret sec-nginx-v1。
  • 配置nginx service，端口为443，名称为https。
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: dr-sni
spec:
  host: nginx.default.svc.cluster.local
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
  • destinationrule相关配置。
  • 为cluster nginx.default.svc.cluster.local定义subset。
  • 将label为version: v1的endpoint deployment定义为subset v1。
  • 将label为version: v2的endpoint deployment定义为subset v2。
{
        "name": "outbound|443|v1|nginx.default.svc.cluster.local",
        "type": "EDS",
        "edsClusterConfig": {
            "edsConfig": {
                "ads": {}
            },
            "serviceName": "outbound|443|v1|nginx.default.svc.cluster.local"
        },
        "connectTimeout": "1.000s",
        "circuitBreakers": {
            "thresholds": [
                {}
            ]
        }
    }

{
        "name": "outbound|443|v2|nginx.default.svc.cluster.local",
        "type": "EDS",
        "edsClusterConfig": {
            "edsConfig": {
                "ads": {}
            },
            "serviceName": "outbound|443|v2|nginx.default.svc.cluster.local"
        },
        "connectTimeout": "1.000s",
        "circuitBreakers": {
            "thresholds": [
                {}
            ]
        }
    }
  • envoy cluster相关配置。
  • 在destinationrule定义完成后，envoy会生成2个subset相关的cluster，分别为outbound|443|v1|nginx.default.svc.cluster.local和outbound|443|v2|nginx.default.svc.cluster.local。
{"svc": "nginx.default.svc.cluster.local:https", "ep": [
{
    "endpoint": {
      "Family": 0,
      "Address": "10.234.0.58",
      "Port": 443,
      "ServicePort": {
        "name": "https",
        "port": 443,
        "protocol": "HTTPS"
      },
…
    "labels": {
      "app": "nginx",
      "pod-template-hash": "3875363210",
      "version": "v2"
    },
…
    "endpoint": {
      "Family": 0,
      "Address": "10.234.0.59",
      "Port": 443,
      "ServicePort": {
        "name": "https",
        "port": 443,
        "protocol": "HTTPS"
      },
…
    "labels": {
      "app": "nginx",
      "pod-template-hash": "1126943858",
      "version": "v1"
    },
  • envoy endpoint相关配置。
  • 这2个cluster会与envoy endpoint相关联。

Gateway和VirtualService

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: gw-sni
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: PASSTHROUGH
    hosts:
    - "*"
  • gateway相关配置。
  • 为ingress gateway设置监听。
  • 打开ingress gateway的443端口进行监听，端口类型为https。
  • tls类型为passthrough。
  • 所有host均可以注册到这个监听。
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: vs-sni-gw
spec:
  hosts:
  - "*.v1.nginx.internal.istio"
  - "*.v2.nginx.internal.istio"
  gateways:
  - gw-sni
  tls:
  - match:
    - sniHosts:
      - "*.v1.nginx.internal.istio"
    route:
    - destination:
        host: nginx.default.svc.cluster.local
        port: 
          number: 443
        subset: v1
  - match:
    - sniHosts:
      - "*.v2.nginx.internal.istio"
    route:
    - destination:
        host: nginx.default.svc.cluster.local
        port: 
          number: 443
        subset: v2
  • ingress gateway层面virtualservice相关配置。
  • ingress gateway在接收到外部lb转发的请求后，按照sni的信息，对请求进行路由。
  • sni为*.v1.nginx.internall.istio的443端口的请求，被转发至nginx.default.svc.cluster.local的v1版本的443端口。
  • sni为*.v2.nginx.internal.istio的443端口的请求，被转发至nginx.default.svc.cluster.local的v2版本的443端口。
{
        "name": "0.0.0.0_443",
        "address": {
            "socketAddress": {
                "address": "0.0.0.0",
                "portValue": 443
            }
        },
        "filterChains": [
            {
                "filterChainMatch": {
                    "serverNames": [
                        "*.v1.nginx.internal.istio"
                    ]
                },
…
                    {
                        "name": "envoy.tcp_proxy",
                        "config": {
                            "cluster": "outbound|443|v1|nginx.default.svc.cluster.local",
                            "stat_prefix": "outbound|443|v1|nginx.default.svc.cluster.local"
                        }
                    }
…
            {
                "filterChainMatch": {
                    "serverNames": [
                        "*.v2.nginx.internal.istio"
                    ]
                },
…
                    {
                        "name": "envoy.tcp_proxy",
                        "config": {
                            "cluster": "outbound|443|v2|nginx.default.svc.cluster.local",
                            "stat_prefix": "outbound|443|v2|nginx.default.svc.cluster.local"
                        }
                    }
  • ingress gateway envoy listener相关配置。
  • 因为virtualservice类型为tls，所以路由会在listener中直接进行。
  • ingress gateway envoy listener的filterchain会将servername为*.v1.nginx.internal.istio的请求全部转发至outbound|443|v1|nginx.default.svc.cluster.local。
  • egress gateway envoy listener的filterchain会将servername为*.v2.nginx.internal.istio的请求全部转发至outbound|443|v2|nginx.default.svc.cluster.local。

测试结果

[~]$ curl https://api.v1.nginx.internal.istio -k
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
  body {
    width: 35em;
    margin: 0 auto;
    font-family: Tahoma, Verdana, Arial, sans-serif;
  }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<h1>nginx01</h1>
<p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[~]$

[~]$ curl https://api.v2.nginx.internal.istio -k
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
  body {
    width: 35em;
    margin: 0 auto;
    font-family: Tahoma, Verdana, Arial, sans-serif;
  }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<h1>nginx02</h1>
<p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[~]$
  • 测试结果。
  • 到api.v1.nginx.internal.istio的请求被正确转发至nginx-v1。
  • 到api.v2.nginx.internal.istio的请求被正确转发至nginx-v2。