My tech blog https://imesh.club

The rule of Istio Gateway port definition

Translate by google

The port and protocol in the Gateway resource define the listener port and protocol in ingressgateway (envoy).

However, the port in Gateway can be set to the port or targetPort of the ingressgateway svc, and finally the targetPort is used in envoy.

For example below, the port is defined as 443 in Gateway:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  namespace: istio-bookinfo
  name: bookinfo-gateway
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 8080  
      name: http
      protocol: HTTP 
    hosts:
    - "*"
  - port:
      number: 443 #####use for assembling the route name that under the listener
      name: https #####use for assembling the route name that under the listener
      protocol: HTTPS #####use for assembling the route name that under the listener
    tls:
      mode: SIMPLE
      serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
      privateKey: /etc/istio/ingressgateway-certs/tls.key
    hosts:
    - "*"

While, when you check the envoy configurations, the listener port actually is 8443

[root@k8s-master-v1-16 networking]# istioctl proxy-config listener istio-ingressgateway-7b869dcfb5-lfqz9.istio-system --port 8443 -o json
[
    {
        "name": "0.0.0.0_8443",
        "address": {
            "socketAddress": {
                "address": "0.0.0.0",
                "portValue": 8443 《《《《《8443!!
            }
        },
        "filterChains": [
            {
                "filters": [
                    {
                        "name": "envoy.http_connection_manager",
                        "typedConfig": {
                            "@type": "type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager",
                            "statPrefix": "outbound_0.0.0.0_8443",
                            "rds": {
                                "configSource": {
                                    "ads": {}
                                },
                                ###The route name is assembled by protocol + port number + portname, Note here: use 443,not 8443
                                "routeConfigName": "https.443.https.bookinfo-gateway.istio-bookinfo"
                            },

This is because the ingressgateway’s 443 svc port is corresponding to targetPort 8443:

 - name: https
    nodePort: 30975
    port: 443
    protocol: TCP
    targetPort: 8443

If the port is defined as 8443 in the gateway, the result is same, will generate 8443 listener in envoy.

When there is only one gateway definition, it is ok to use port or the corresponding targetport of the ingressgateway, both will get the same listener port in evnoy lastly. But please be noted, if there are multiple gateway definitions and set same protocol, then the port in the gateways must be same. Otherwise, the listener will only associate the routeConfigName generated by the gateway that created afterwards, and the associated route will not contain the virtualservice logic that related to the first created gateway.

For example, configure HTTP:8080 in the gateway in the istio-bookinfo namespace (the svc 80 of ingressgateway corresponds to the 8080 of the pod):

apiVersion: v1
items:
 
  spec:
    selector:
      istio: ingressgateway
    servers:
    - hosts:
      - '*'
      port:
        name: http
        number: 8080
        protocol: HTTP
    - hosts:
      - istiobookinfo.lab.f5se.io
      port:
        name: https
        number: 443
        protocol: HTTPS

The gateway in the httpbin namespace configures HTTP:80,

[root@k8s-master-v1-16 httpbin]# kubectl get gateways.networking.istio.io -n httpbin -o yaml
apiVersion: v1
items:
 
  spec:
    selector:
      istio: ingressgateway
    servers:
    - hosts:
      - httpbin.lab.f5se.io
      port:
        name: http-httpbin
        number: 80
        protocol: HTTP

This leads to the fact that the envoy listener is actually actually associated with the later created http.80 route:

[root@k8s-master-v1-16 ~]# istioctl proxy-config listener istio-ingressgateway-7b869dcfb5-gh2vr.istio-system --port 8080 -o json
[
    {
        "name": "0.0.0.0_8080",
        "address": {
            "socketAddress": {
                "address": "0.0.0.0",
                "portValue": 8080
            }
        },
        "filterChains": [
            {
                "filters": [
                    {
                        "name": "envoy.http_connection_manager",
                        "typedConfig": {
                            "@type": "type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager",
                            "statPrefix": "outbound_0.0.0.0_8080",
                            "rds": {
                                "configSource": {
                                    "ads": {}
                                },
                                "routeConfigName": "http.80"

Therefore, in these multi gateways configuration, which ultimately shares the listener port, the port number and protocol name need to be the same (BEST practice is using the port number that envoy actually listens to), but the port must be named differently (if the port name is the same, envoy will show the conflicting configuration logs):

2020-07-10T09:56:26.192152Z	warning	envoy config	[external/envoy/source/common/config/grpc_subscription_impl.cc:101] gRPC config for type.googleapis.com/envoy.api.v2.Listener rejected: 
Error adding/updating listener(s) 0.0.0.0_8080: duplicate listener 0.0.0.0_8080 found

Gateways with the same protocol and the same port, no matter whether they are in the same namespace, can only share one listener in envoy, so they will also share same route. If you accidentally configure the same virtualservice under different namespaces at this time, it will cause envoy to generate two identical match conditions in the same route (and the associated cluster is different namespace suffix, This resulting in abnormal access)

Check more istio practice detail at my tech blog https://imesh.club

Jing Lin (林静)
Senior Solution Architect@F5

Check my tech blog for more https://imesh.club

Related

comments powered by Disqus